[RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi,

By accepting the challenges raised in recent discussion of advanced
support for history rebasing and editing in Git, I hopefully figured out
a clean and elegant method of rebasing merges that I think is "The Right
Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
second meaning: a "Trivial Merge (TM)", see below.]

Let me begin by outlining the method in git terms, and special thanks
here must go to "Johannes Sixt" <j6t@kdbg.org> for his original bright
idea to use "cherry-pick -m1" to rebase merge commits.

End of preface -- here we go.

Given 2 original branches, b1 and b2, and a merge commit M that joins
them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
also that B1' and B2' happen to be the tip commits on b1' and b2',
respectively.

To produce merge commit M' that joins b1' and b2', the following
operations will suffice:

1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
3. Merge --no-ff new b2' to current new b1', to produce UM'.
4. Get rid of U1' and U2' by re-writing parent references of UM' from
   U1' and U2' to  B1' and B2', respectively, to produce M'.
5. Mission complete.

Let's now see why and how the method actually works.

Firs off, let me introduce you to my new friend, the Trivial Merge, or
(TM) for short. By definition, (TM) is a merge that introduces
absolutely no differences to the sides of the merge. (I also like to
sometimes call him "Angel Merge", both as the most beautiful of all
merges, and as direct antithesis to "evil merge".)

One very nice thing about (TM) is that to safely rebase it, it suffices
to merge its (rebased) parents. It is safe in this case, as (TM) itself
doesn't posses any content changes, and thus none could be missed by
replacing it with another merge commit.

I bet most of us have never seen (TM) in practice though, so let's see
how (TM) can help us handle general case of some random merge. What I'm
going to do is to create a virtual (TM) and see how it goes from there.

Let's start with this history:

  M
 / \
B1  B2

And let's transform it to the following one, contextually equivalent to
the original, by introducing 2 simple utility commits U1 and U2, and a
new utility merge commit UM:

  UM
 /  \
U1   U2
|    |
B1   B2

Here content of any of the created UM, U1, and U2 is the same, and is
exact copy of original content of M. I.e., provided [A] denotes
"content of commit A", we have:

[UM] = [U1] = [U2] = [M]

Stress again how these changes to the history preserve the exact content
of the original merge ([UM] = [M]), and how U1 an U2 represent content
changes due to merge on either side[*], and how neither preceding nor
subsequent commits content would be affected by the change of
representation.

Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
exactly our new friend -- the "Trivial Merge (TM)" his true self,
introducing zero changes to content.

Next we rebase our new representation of the history and we get:

  UM'
 /  \
U1'  U2'
|    |
B1'  B2'

Here UM' is bare merge of U1' and U2', in exact accordance with the
method of rebasing a (TM) we've already discussed above, and U1' and U2'
are rebased versions of U1 and U2, obtained by usual rebasing methods
for non-merge commits.

(Note, however, that at this point UM' is not necessarily a (TM)
anymore, so in real implementation it may make sense to check if UM' is
not a (TM) and stop for possible user amendment.)

Finally, to get to our required merge commit M', we get the content of
UM' and record two actual parents of the merge:

  M'
 / \
B1' B2'

Where [M'] = [UM'].

That's it. Mission complete.

I expect the method to have the following nice features:

- it carefully preserves user changes by rebasing the merge commit
itself, in a way that is semantically similar to rebasing simple
(non-merge) commits, yet it allows changes made to branches during
history editing to propagate over corresponding merge commit that joins
the branches, even automatically when the changes don't conflict, as
expected.

- it has provision for detection of even slightest chances of ending up
with surprising merge (just check if UM' is still (TM)), so that
implementation could stop for user inspection and amendment when
appropriate, yet it is capable of handling trivial cases smoothly and
automatically.

- it never falls back to simple invocation of merge operation on rebased
original branches themselves, thus avoiding the problem of lack of
knowledge of how the merge at hand has been performed in the first
place. It doesn't prevent implementation from letting user to manually
perform whatever merge she wishes when suspect result is automatically
detected though.

- it extends trivially to octopus merges.

- it appears shiny to the point that it will likely be able to handle
even darkest evil merges nicely, no special treatment required.

Footnote:

[*] We may as well consider the (UM,U1,U2) trio to be semantically split
representation of git merge commit, where U1 and U2 represent content
changes to the sides, and UM represents pure history joint. Or, the
other way around, we may consider git merge commit to be optimized
representation of this trio. I think this split representation could
help to simplify reasoning about git merges in general.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Fri, Feb 16, 2018 at 5:08 AM, Sergey Organov <sorganov@gmail.com> wrote:
> Hi,
>
> By accepting the challenges raised in recent discussion of advanced
> support for history rebasing and editing in Git, I hopefully figured out
> a clean and elegant method of rebasing merges that I think is "The Right
> Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
> second meaning: a "Trivial Merge (TM)", see below.]
>
> Let me begin by outlining the method in git terms, and special thanks
> here must go to "Johannes Sixt" <j6t@kdbg.org> for his original bright
> idea to use "cherry-pick -m1" to rebase merge commits.
>
> End of preface -- here we go.
>

I hope to take a more detailed look at this, also possibly with some
attempts at re-creating the process by hand to see it in practice.

> Given 2 original branches, b1 and b2, and a merge commit M that joins
> them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
> also that B1' and B2' happen to be the tip commits on b1' and b2',
> respectively.
>
> To produce merge commit M' that joins b1' and b2', the following
> operations will suffice:
>
> 1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
> 2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
> 3. Merge --no-ff new b2' to current new b1', to produce UM'.
> 4. Get rid of U1' and U2' by re-writing parent references of UM' from
>    U1' and U2' to  B1' and B2', respectively, to produce M'.
> 5. Mission complete.
>

Seems pretty straight forward, go to each branch and cherry-pick the
merge respective to its relative parent, and then finally re-merge
everything, and consume the intermittent commits.

> Let's now see why and how the method actually works.
>
> Firs off, let me introduce you to my new friend, the Trivial Merge, or
> (TM) for short. By definition, (TM) is a merge that introduces
> absolutely no differences to the sides of the merge. (I also like to
> sometimes call him "Angel Merge", both as the most beautiful of all
> merges, and as direct antithesis to "evil merge".)
>
> One very nice thing about (TM) is that to safely rebase it, it suffices
> to merge its (rebased) parents. It is safe in this case, as (TM) itself
> doesn't posses any content changes, and thus none could be missed by
> replacing it with another merge commit.
>
> I bet most of us have never seen (TM) in practice though, so let's see
> how (TM) can help us handle general case of some random merge. What I'm
> going to do is to create a virtual (TM) and see how it goes from there.
>
> Let's start with this history:
>
>   M
>  / \
> B1  B2
>
> And let's transform it to the following one, contextually equivalent to
> the original, by introducing 2 simple utility commits U1 and U2, and a
> new utility merge commit UM:
>
>   UM
>  /  \
> U1   U2
> |    |
> B1   B2
>
> Here content of any of the created UM, U1, and U2 is the same, and is
> exact copy of original content of M. I.e., provided [A] denotes
> "content of commit A", we have:
>
> [UM] = [U1] = [U2] = [M]
>
> Stress again how these changes to the history preserve the exact content
> of the original merge ([UM] = [M]), and how U1 an U2 represent content
> changes due to merge on either side[*], and how neither preceding nor
> subsequent commits content would be affected by the change of
> representation.
>
> Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
> exactly our new friend -- the "Trivial Merge (TM)" his true self,
> introducing zero changes to content.
>
> Next we rebase our new representation of the history and we get:
>
>   UM'
>  /  \
> U1'  U2'
> |    |
> B1'  B2'
>
> Here UM' is bare merge of U1' and U2', in exact accordance with the
> method of rebasing a (TM) we've already discussed above, and U1' and U2'
> are rebased versions of U1 and U2, obtained by usual rebasing methods
> for non-merge commits.
>
> (Note, however, that at this point UM' is not necessarily a (TM)
> anymore, so in real implementation it may make sense to check if UM' is
> not a (TM) and stop for possible user amendment.)
>

This might be a bit tricky for a user to understand what the process
is, especially if they don't understand how it's creating special U1'
and U2' commits. However, it *is* the cleanest method I've either seen
or thought of for presenting the conflict to the user.

> Finally, to get to our required merge commit M', we get the content of
> UM' and record two actual parents of the merge:
>
>   M'
>  / \
> B1' B2'
>
> Where [M'] = [UM'].
>
> That's it. Mission complete.
>
> I expect the method to have the following nice features:
>
> - it carefully preserves user changes by rebasing the merge commit
> itself, in a way that is semantically similar to rebasing simple
> (non-merge) commits, yet it allows changes made to branches during
> history editing to propagate over corresponding merge commit that joins
> the branches, even automatically when the changes don't conflict, as
> expected.
>

Right.

> - it has provision for detection of even slightest chances of ending up
> with surprising merge (just check if UM' is still (TM)), so that
> implementation could stop for user inspection and amendment when
> appropriate, yet it is capable of handling trivial cases smoothly and
> automatically.

Nice!

>
> - it never falls back to simple invocation of merge operation on rebased
> original branches themselves, thus avoiding the problem of lack of
> knowledge of how the merge at hand has been performed in the first
> place. It doesn't prevent implementation from letting user to manually
> perform whatever merge she wishes when suspect result is automatically
> detected though.
>

Right, since we're re-creating the intermittent commits U1' and U2'
first based on the original merge, that's how we manage to maintain
the result of the merge. I like it.

> - it extends trivially to octopus merges.
>
> - it appears shiny to the point that it will likely be able to handle
> even darkest evil merges nicely, no special treatment required.
>

Yep, and I like that it has a pretty reasonable way of presenting
conflicts for resolution. It may be a bit tricky to explain the use of
the intermittent commits U1' and U2' though.

> Footnote:
>
> [*] We may as well consider the (UM,U1,U2) trio to be semantically split
> representation of git merge commit, where U1 and U2 represent content
> changes to the sides, and UM represents pure history joint. Or, the
> other way around, we may consider git merge commit to be optimized
> representation of this trio. I think this split representation could
> help to simplify reasoning about git merges in general.

Yes, I think this concept is pretty useful. I think it could be useful
as a way of showing how the merge worked.

Thanks,
Jake

>
> -- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Jake,

Jacob Keller <jacob.keller@gmail.com> writes:
> On Fri, Feb 16, 2018 at 5:08 AM, Sergey Organov <sorganov@gmail.com> wrote:
>> Hi,
>>
>> By accepting the challenges raised in recent discussion of advanced
>> support for history rebasing and editing in Git, I hopefully figured out
>> a clean and elegant method of rebasing merges that I think is "The Right
>> Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
>> second meaning: a "Trivial Merge (TM)", see below.]
>>
>> Let me begin by outlining the method in git terms, and special thanks
>> here must go to "Johannes Sixt" <j6t@kdbg.org> for his original bright
>> idea to use "cherry-pick -m1" to rebase merge commits.
>>
>> End of preface -- here we go.
>>
>
> I hope to take a more detailed look at this, also possibly with some
> attempts at re-creating the process by hand to see it in practice.

Thank you for your interest and for the review, and yes, some testing is
what the idea desperately needs. Unfortunately I don't have much time
for it right now, nor am I fluent enough in git internals to actually
make even a raw prototype really soon, sorry. Anybody who is interested
is very welcome to volunteer!

[...]
>
> This might be a bit tricky for a user to understand what the process
> is, especially if they don't understand how it's creating special U1'
> and U2' commits. However, it *is* the cleanest method I've either seen
> or thought of for presenting the conflict to the user.
>
[...]
>> - it appears shiny to the point that it will likely be able to handle
>> even darkest evil merges nicely, no special treatment required.
>>
>
> Yep, and I like that it has a pretty reasonable way of presenting
> conflicts for resolution. It may be a bit tricky to explain the use of
> the intermittent commits U1' and U2' though.

Yeah, I see how all this sends somewhat unique challenges to the
implementation to get user interaction in case of conflicts right, even
though all the basic concepts are old buddies and should be familiar to
the user.

That said, the recursive merge strategy comes to mind, where creating
virtual merge base may itself cause conflicts, so something similar
enough is likely to already exist.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 16/02/2018 14:08, Sergey Organov wrote:
> 
> By accepting the challenges raised in recent discussion of advanced
> support for history rebasing and editing in Git, I hopefully figured out
> a clean and elegant method of rebasing merges that I think is "The Right
> Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
> second meaning: a "Trivial Merge (TM)", see below.]
> 
> Let me begin by outlining the method in git terms, and special thanks
> here must go to "Johannes Sixt" <j6t@kdbg.org> for his original bright
> idea to use "cherry-pick -m1" to rebase merge commits.
> 
> End of preface -- here we go.
> 
> Given 2 original branches, b1 and b2, and a merge commit M that joins
> them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
> also that B1' and B2' happen to be the tip commits on b1' and b2',
> respectively.
> 
> To produce merge commit M' that joins b1' and b2', the following
> operations will suffice:
> 
> 1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
> 2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
> 3. Merge --no-ff new b2' to current new b1', to produce UM'.
> 4. Get rid of U1' and U2' by re-writing parent references of UM' from
>    U1' and U2' to  B1' and B2', respectively, to produce M'.
> 5. Mission complete.
> 
> Let's now see why and how the method actually works.
> 
> Firs off, let me introduce you to my new friend, the Trivial Merge, or
> (TM) for short. By definition, (TM) is a merge that introduces
> absolutely no differences to the sides of the merge. (I also like to
> sometimes call him "Angel Merge", both as the most beautiful of all
> merges, and as direct antithesis to "evil merge".)
> 
> One very nice thing about (TM) is that to safely rebase it, it suffices
> to merge its (rebased) parents. It is safe in this case, as (TM) itself
> doesn't posses any content changes, and thus none could be missed by
> replacing it with another merge commit.
> 
> I bet most of us have never seen (TM) in practice though, so let's see
> how (TM) can help us handle general case of some random merge. What I'm
> going to do is to create a virtual (TM) and see how it goes from there.
> 
> Let's start with this history:
> 
>   M
>  / \
> B1  B2
> 
> And let's transform it to the following one, contextually equivalent to
> the original, by introducing 2 simple utility commits U1 and U2, and a
> new utility merge commit UM:
> 
>   UM
>  /  \
> U1   U2
> |    |
> B1   B2
> 
> Here content of any of the created UM, U1, and U2 is the same, and is
> exact copy of original content of M. I.e., provided [A] denotes
> "content of commit A", we have:
> 
> [UM] = [U1] = [U2] = [M]
> 
> Stress again how these changes to the history preserve the exact content
> of the original merge ([UM] = [M]), and how U1 an U2 represent content
> changes due to merge on either side[*], and how neither preceding nor
> subsequent commits content would be affected by the change of
> representation.
> 
> Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
> exactly our new friend -- the "Trivial Merge (TM)" his true self,
> introducing zero changes to content.
> 
> Next we rebase our new representation of the history and we get:
> 
>   UM'
>  /  \
> U1'  U2'
> |    |
> B1'  B2'
> 
> Here UM' is bare merge of U1' and U2', in exact accordance with the
> method of rebasing a (TM) we've already discussed above, and U1' and U2'
> are rebased versions of U1 and U2, obtained by usual rebasing methods
> for non-merge commits.
> 
> (Note, however, that at this point UM' is not necessarily a (TM)
> anymore, so in real implementation it may make sense to check if UM' is
> not a (TM) and stop for possible user amendment.)
> 
> Finally, to get to our required merge commit M', we get the content of
> UM' and record two actual parents of the merge:
> 
>   M'
>  / \
> B1' B2'
> 
> Where [M'] = [UM'].
> 
> That's it. Mission complete.
> 
> I expect the method to have the following nice features:
> 
> - it carefully preserves user changes by rebasing the merge commit
> itself, in a way that is semantically similar to rebasing simple
> (non-merge) commits, yet it allows changes made to branches during
> history editing to propagate over corresponding merge commit that joins
> the branches, even automatically when the changes don't conflict, as
> expected.
> 
> - it has provision for detection of even slightest chances of ending up
> with surprising merge (just check if UM' is still (TM)), so that
> implementation could stop for user inspection and amendment when
> appropriate, yet it is capable of handling trivial cases smoothly and
> automatically.
> 
> - it never falls back to simple invocation of merge operation on rebased
> original branches themselves, thus avoiding the problem of lack of
> knowledge of how the merge at hand has been performed in the first
> place. It doesn't prevent implementation from letting user to manually
> perform whatever merge she wishes when suspect result is automatically
> detected though.
> 
> - it extends trivially to octopus merges.
> 
> - it appears shiny to the point that it will likely be able to handle
> even darkest evil merges nicely, no special treatment required.
> 
> Footnote:
> 
> [*] We may as well consider the (UM,U1,U2) trio to be semantically split
> representation of git merge commit, where U1 and U2 represent content
> changes to the sides, and UM represents pure history joint. Or, the
> other way around, we may consider git merge commit to be optimized
> representation of this trio. I think this split representation could
> help to simplify reasoning about git merges in general.

I`m far from a Git expert, but merely a bit advanced user, so please 
take my opinion with a grain of salt.

That said, I`m really interested in this topic, which seems to (try 
to) address the only "bad feeling" I had with rebasing merges - being 
afraid of silently losing amendments by actually trying to "replay" 
the merge (where additional and possibly important context is 
missing), instead of really "rebasing" it (somehow).

Even though this behavior is known and documented, it still left some 
to be desired.

Might be I`m missing something, but so far I like how described 
approach just "feels right" (to me, for now), being really simple, 
yet robust :)

As my humble contribution to the cause and discussion itself, I`m 
providing possibly naive, yet simple demonstration script[1], and 
clearly showing where current `git rebase --preserve-merges` is 
lacking (in my opinion, even though being expected and documented), 
and how your proposal seems to improve the situation here.

Thanks for your thoughts, and hoping to see this going somewhere :)

Regards, Buga

[1] Demonstration script:
--- 8< ---
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

for i in {1..20}
do
	echo A$i >>test.txt
	git commit -am "A$i"
done

git checkout -b b1
sed -i '6iB1' test.txt
git commit -am "B1"

git checkout -b b2 HEAD^
sed -i '16iB2' test.txt
git commit -am "B2"

git checkout -b merge b1
git merge --no-commit b2
sed -i '12iX' test.txt # amend merge commit
git commit -am "M"
git tag original-merge

git checkout master
for i in {1..5}
do
	j=`expr "$i" + 20`
	sed -i "${i}iA${j}" test.txt
	git commit -am "A$j"
done

# (1) current merge rebasing logic,
# not preserving merge commit manual amendment, as documented
git rebase --preserve-merges master merge
git tag preserve-merges

# (2) simple/naive demonstration of proposed merge rebasing logic
# using described "Trivial Merge" (TM, or "Angel Merge"),
# preserving merge commit manual amendment :)
git checkout b1
git rebase master
git cherry-pick -m1 original-merge

git checkout b2
git rebase master
git cherry-pick -m2 original-merge

git branch -f merge b1
git checkout merge
git merge b2 --no-commit
git commit -a --reuse-message original-merge
git tag angel-merge

git reset --hard b1^
git read-tree --reset angel-merge
git update-ref refs/heads/merge "$(git show -s --format=%B original-merge | git commit-tree "$(git write-tree)" -p "$(git rev-parse b1^)" -p "$(git rev-parse b2^)")"
git tag -f angel-merge
git branch -f b1 b1^
git branch -f b2 b2^

# show resulting graph
echo
git log --all --decorate --oneline --graph

# comparison between original merge and rebased merge (1),
# showing merge commit amendment "X" being silently lost during rebase
echo
echo 'diff original-merge..preserve-merges:'
git diff original-merge..preserve-merges

# comparison between original merge and rebased merge (2),
# showing merge commit amendment "X" being preserved during rebase
# (not shown in diff)
echo
echo 'diff original-merge..angel-merge:'
git diff original-merge..angel-merge

# direct comparison between two merge rebasing approaches,
# merge commit amendment difference showing up
echo
echo 'diff preserve-merges angel-merge:'
git diff preserve-merges angel-merge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>

[...]

>
> Even though this behavior is known and documented, it still left some 
> to be desired.
>
> Might be I`m missing something, but so far I like how described 
> approach just "feels right" (to me, for now), being really simple, 
> yet robust :)
>
> As my humble contribution to the cause and discussion itself, I`m 
> providing possibly naive, yet simple demonstration script[1], and 
> clearly showing where current `git rebase --preserve-merges` is 
> lacking (in my opinion, even though being expected and documented), 
> and how your proposal seems to improve the situation here.

Thanks for the test-case, -- it's nice to see this thingy working!

>
> Thanks for your thoughts, and hoping to see this going somewhere :)

So do I. Even if nobody volunteers to adopt it, I hope I'll be able to
implement something myself, not very soon though.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Tue, 20 Feb 2018, Igor Djordjevic wrote:

> I`m really interested in this topic, which seems to (try to) address the
> only "bad feeling" I had with rebasing merges - being afraid of silently
> losing amendments by actually trying to "replay" the merge (where
> additional and possibly important context is missing), instead of really
> "rebasing" it (somehow).

If those amendments are what you are worried about, why not address them
specifically?

In other words, rather than performing the equivalent of

	git show <merge>^! | git apply

(which would of course completely ignore if the rewritten <merge>^2
dropped a patch, amended a patch, or even added a new patch), what you
really want is to figure out what changes the user made when merging, and
what merge strategy was used to begin with.

To see what I mean, look at the output of `git show 0fd90daba8`: it shows
how conflicts were resolved. By necessity, this is more complicated than a
simple diff: it is *not* as simple as taking a diff between two revisions
and applying that diff to a third revision. There were (at least) three
revisions involved in the original merge commit, and recreating that merge
commit faithfully means to represent the essence of the merge commit
faithfully enough to be able to replay it on a new set of at least three
revisions.  That can be simplified to two-way diffs only in very, very
special circumstances, and in all other cases this simplification will
simply fall on its nose.

If the proposed solution was to extend `git apply` to process combined
diffs, I would agree that we're on to something. That is not the proposed
solution, though.

In short: while I am sympathetic to the desire to keep things simple,
the idea to somehow side-step replaying the original merge seems to be
*prone* to be flawed. Any system that cannot accommodate
dropped/changed/added commits on either side of a merge is likely to be
too limited to be useful.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Buga,
>
> On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>
>> I`m really interested in this topic, which seems to (try to) address the
>> only "bad feeling" I had with rebasing merges - being afraid of silently
>> losing amendments by actually trying to "replay" the merge (where
>> additional and possibly important context is missing), instead of really
>> "rebasing" it (somehow).

[...]

> In short: while I am sympathetic to the desire to keep things simple,
> the idea to somehow side-step replaying the original merge seems to be
> *prone* to be flawed.

The proposed (TM) solution does replay the original merge.

> Any system that cannot accommodate dropped/changed/added commits on
> either side of a merge is likely to be too limited to be useful.

I believe the proposed (TM) solution handles all that nicely. It does
accommodate dropped/changed/added commits on either side of a merge,
symmetrically, and never silently drops user modifications.

If you think (TM) is flawed, please give us a test-case.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Mon, Feb 26, 2018 at 4:07 PM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Buga,
>
> On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>
>> I`m really interested in this topic, which seems to (try to) address the
>> only "bad feeling" I had with rebasing merges - being afraid of silently
>> losing amendments by actually trying to "replay" the merge (where
>> additional and possibly important context is missing), instead of really
>> "rebasing" it (somehow).
>
> If those amendments are what you are worried about, why not address them
> specifically?
>
> In other words, rather than performing the equivalent of
>
>         git show <merge>^! | git apply
>
> (which would of course completely ignore if the rewritten <merge>^2
> dropped a patch, amended a patch, or even added a new patch), what you
> really want is to figure out what changes the user made when merging, and
> what merge strategy was used to begin with.
>
> To see what I mean, look at the output of `git show 0fd90daba8`: it shows
> how conflicts were resolved. By necessity, this is more complicated than a
> simple diff: it is *not* as simple as taking a diff between two revisions
> and applying that diff to a third revision. There were (at least) three
> revisions involved in the original merge commit, and recreating that merge
> commit faithfully means to represent the essence of the merge commit
> faithfully enough to be able to replay it on a new set of at least three
> revisions.  That can be simplified to two-way diffs only in very, very
> special circumstances, and in all other cases this simplification will
> simply fall on its nose.
>
> If the proposed solution was to extend `git apply` to process combined
> diffs, I would agree that we're on to something. That is not the proposed
> solution, though.
>
> In short: while I am sympathetic to the desire to keep things simple,
> the idea to somehow side-step replaying the original merge seems to be
> *prone* to be flawed. Any system that cannot accommodate
> dropped/changed/added commits on either side of a merge is likely to be
> too limited to be useful.
>


The reason Sergey's solution works is because he cherry picks the
merge using each parent first, and then merges the result of those. So
each branch of the merge gets one, and then you merge the result of
those cherry-picks. This preservers amendments and changes properly,
and should result in a good solution.

I agree that making "git apply" work with combined diffs could also be
another solution, but it may be trickier.

If this *doesn't* work, a test case showing that it doesn't work would
be appreciated. I'm hoping to be able to put together something soon,
but I haven't had time due to $dayjob.

> Ciao,
> Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Buga,
>
> On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>
>> I`m really interested in this topic, which seems to (try to) address the
>> only "bad feeling" I had with rebasing merges - being afraid of silently
>> losing amendments by actually trying to "replay" the merge (where
>> additional and possibly important context is missing), instead of really
>> "rebasing" it (somehow).
>
> If those amendments are what you are worried about, why not address them
> specifically?
>
> In other words, rather than performing the equivalent of
>
> 	git show <merge>^! | git apply
>
> (which would of course completely ignore if the rewritten <merge>^2
> dropped a patch, amended a patch, or even added a new patch),

You've already bit this poor thingy to death. Please rather try your
teeth on the proposed Trivial Merge (TM) method. Sorry, you will need to
actually read the proposal should you decide to.

> what you really want is to figure out what changes the user made when
> merging,

Exactly! It's all and only about changes, I'm glad you are starting to
get it.

> and what merge strategy was used to begin with.

No. It's in Git core philosophy to store only result, independent on the
way the result has been achieved. Only result matters. I could have got
merge result out of thin air, and Git won't care, and will store it for
me safely.

The proposed (TM) method will ensure it's still stored safely after
rebasing.

> To see what I mean, look at the output of `git show 0fd90daba8`: it shows
> how conflicts were resolved. By necessity, this is more complicated than a
> simple diff: it is *not* as simple as taking a diff between two revisions
> and applying that diff to a third revision. There were (at least) three
> revisions involved in the original merge commit, and recreating that merge
> commit faithfully means to represent the essence of the merge commit
> faithfully enough to be able to replay it on a new set of at least three
> revisions.

Even better: we have at least 3 source revisions and 2 new revisions to
base 3-rd new revision upon. Proposed (TM) method will use all of them.

> That can be simplified to two-way diffs only in very, very special
> circumstances, and in all other cases this simplification will simply
> fall on its nose.

No, Done Right (TM) it won't fall as N-way-diff is nothing more than N-1
simple diffs, trivially combined.

> If the proposed solution was to extend `git apply` to process combined
> diffs, I would agree that we're on to something. That is not the proposed
> solution, though.

I'm glad you finally got the point. Proposed (TM) method effectively
'git apply' combined diffs, yes. I just defined it in terms of commits
rather than diffs, as it's simpler to understand and to reason about it
that way.

>
> In short: while I am sympathetic to the desire to keep things simple,

Proposed (TM) method is complex enough to solve the problem at hand. No
need to ask for more complexity.

> the idea to somehow side-step replaying the original merge seems to be
> *prone* to be flawed.

Exactly! That's why current approach to rebase merges is flawed, and
that's why proposed (TM) method does reply entire original merges.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Jake,

On Mon, 26 Feb 2018, Jacob Keller wrote:

> On Mon, Feb 26, 2018 at 4:07 PM, Johannes Schindelin
> <Johannes.Schindelin@gmx.de> wrote:
> >
> > On Tue, 20 Feb 2018, Igor Djordjevic wrote:
> >
> >> I`m really interested in this topic, which seems to (try to) address the
> >> only "bad feeling" I had with rebasing merges - being afraid of silently
> >> losing amendments by actually trying to "replay" the merge (where
> >> additional and possibly important context is missing), instead of really
> >> "rebasing" it (somehow).
> >
> > If those amendments are what you are worried about, why not address them
> > specifically?
> >
> > In other words, rather than performing the equivalent of
> >
> >         git show <merge>^! | git apply
> >
> > (which would of course completely ignore if the rewritten <merge>^2
> > dropped a patch, amended a patch, or even added a new patch), what you
> > really want is to figure out what changes the user made when merging, and
> > what merge strategy was used to begin with.
> >
> > To see what I mean, look at the output of `git show 0fd90daba8`: it shows
> > how conflicts were resolved. By necessity, this is more complicated than a
> > simple diff: it is *not* as simple as taking a diff between two revisions
> > and applying that diff to a third revision. There were (at least) three
> > revisions involved in the original merge commit, and recreating that merge
> > commit faithfully means to represent the essence of the merge commit
> > faithfully enough to be able to replay it on a new set of at least three
> > revisions.  That can be simplified to two-way diffs only in very, very
> > special circumstances, and in all other cases this simplification will
> > simply fall on its nose.
> >
> > If the proposed solution was to extend `git apply` to process combined
> > diffs, I would agree that we're on to something. That is not the proposed
> > solution, though.
> >
> > In short: while I am sympathetic to the desire to keep things simple,
> > the idea to somehow side-step replaying the original merge seems to be
> > *prone* to be flawed. Any system that cannot accommodate
> > dropped/changed/added commits on either side of a merge is likely to be
> > too limited to be useful.
> >
> 
> 
> The reason Sergey's solution works is because he cherry picks the
> merge using each parent first, and then merges the result of those. So
> each branch of the merge gets one, and then you merge the result of
> those cherry-picks. This preservers amendments and changes properly,
> and should result in a good solution.

I saw your patch trying to add a minimal example, and I really want to run
away screaming.

Do you have any way to describe the idea in a simple, 3-5 lines long
paragraph?

So far, I just know that it is some sort of confusing criss-cross
cherry-picking and merging and stuff, but nothing in those steps shouts
out to me what the *idea* is.

If it would be something like "recreate the old merge, with merge
conflicts and all, then generate the diff to the actual tree of the merge
commit, then apply that to the newly-generated merge", I would understand.

I would still suspect that -s ours would be a hard nut for that method,
but I would understand that idea.

Thanks,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Sergey Organov <sorganov@gmail.com> writes:

> You've already bit this poor thingy to death. Please rather try your
> teeth on the proposed Trivial Merge (TM) method.

Whatever you do, do *NOT* call any part of your proposal "trivial
merge", unless you are actually using the term to mean what Git
calls "trivial merge".  The phrase has an established meaning in Git
and your attempt to abuse it to mean something entirely different is
adding unnecessary hindrance for other people to understand what you
want to perform.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Dscho,

On 27/02/2018 17:21, Johannes Schindelin wrote:
> 
> Do you have any way to describe the idea in a simple, 3-5 lines long 
> paragraph?
> 
> So far, I just know that it is some sort of confusing criss-cross 
> cherry-picking and merging and stuff, but nothing in those steps
> shouts out to me what the *idea* is.
> 
> If it would be something like "recreate the old merge, with merge 
> conflicts and all, then generate the diff to the actual tree of the
> merge commit, then apply that to the newly-generated merge", I would
> understand.

It would be more along the lines of "(1) rebase old merge commit parents, 
(2) generate separate diff between old merge commit and each of its 
parents, (3) apply each diff to their corresponding newly rebased 
parent respectively (as a temporary commit, one per rebased parent), 
(4) merge these temporary commits to generate 'rebased' merge commit, 
(5) drop temporary commits, recording their parents as parents of 
'rebased' merge commit (instead of dropped temporary commits)".

Implementation wise, steps (2) and (3) could also be done by simply 
copying old merge commit _snapshot_ on top of each of its parents as 
a temporary, non-merge commit, then rebasing (cherry-picking) these 
temporary commits on top of their rebased parent commits to produce 
rebased temporary commits (to be merged for generating 'rebased' 
merge commit in step (4)).

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 27/02/2018 19:55, Igor Djordjevic wrote:
> 
> It would be more along the lines of "(1) rebase old merge commit parents, 
> (2) generate separate diff between old merge commit and each of its 
> parents, (3) apply each diff to their corresponding newly rebased 
> parent respectively (as a temporary commit, one per rebased parent), 
> (4) merge these temporary commits to generate 'rebased' merge commit, 
> (5) drop temporary commits, recording their parents as parents of 
> 'rebased' merge commit (instead of dropped temporary commits)".
> 
> Implementation wise, steps (2) and (3) could also be done by simply 
> copying old merge commit _snapshot_ on top of each of its parents as 
> a temporary, non-merge commit, then rebasing (cherry-picking) these 
> temporary commits on top of their rebased parent commits to produce 
> rebased temporary commits (to be merged for generating 'rebased' 
> merge commit in step (4)).

For those still tagging along (and still confused), here are some 
diagrams (following what Sergey originally described). Note that 
actual implementation might be even simpler, but I believe it`s a bit 
easier to understand like this, using some "temporary" commits approach.

Here`s our starting position:

(0) ---X1---o---o---o---o---o---X2 (master)
       |\
       | A1---A2---A3
       |             \
       |              M (topic)
       |             /
       \-B1---B2---B3


Now, we want to rebase merge commit M from X1 onto X2. First, rebase
merge commit parents as usual:

(1) ---X1---o---o---o---o---o---X2
       |\                       |\
       | A1---A2---A3           | A1'--A2'--A3'
       |             \          |
       |              M         |
       |             /          |
       \-B1---B2---B3           \-B1'--B2'--B3'


That was commonly understandable part. Now, for "rebasing" the merge 
commit (keeping possible amendments), we do some extra work. First, 
we make two temporary commits on top of old merge parents, by using 
exact tree (snapshot) of commit M:

(2) ---X1---o---o---o---o---o---X2
       |\                       |\
       | A1---A2---A3---U1      | A1'--A2'--A3'
       |             \          |
       |              M         |
       |             /          |
       \-B1---B2---B3---U2      \-B1'--B2'--B3'


So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.

Now, we rebase these temporary commits, too:

(3) ---X1---o---o---o---o---o---X2
       |\                       |\
       | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
       |             \          |
       |              M         |
       |             /          |
       \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'


As a next step, we merge these temporary commits to produce our 
"rebased" merged commit M:

(4) ---X1---o---o---o---o---o---X2
       |\                       |\
       | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
       |             \          |                  \
       |              M         |                   M'
       |             /          |                  /
       \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'


Finally, we drop temporary commits, and record rebased commits A3' 
and B3' as our "rebased" merge commit parents instead (merge commit 
M' keeps its same tree/snapshot state, just gets parents replaced):

(5) ---X1---o---o---o---o---o---X2
       |\                       |\
       | A1---A2---A3---U1      | A1'--A2'--A3'
       |             \          |             \
       |              M         |              M'
       |             /          |             /
       \-B1---B2---B3---U2      \-B1'--B2'--B3'


And that`s it, our merge commit M has been "rebased" to M' :)

(6) ---X1---o---o---o---o---o---X2 (master)
                                |\
                                | A1'--A2'--A3'
                                |             \
                                |              M' (topic)
                                |             /
                                \-B1'--B2'--B3'


Important thing to note here is that in our step (3) above, still in 
terms of trees/snapshots (not diffs), U1' could still be equal to 
U2', produced merge commit M' tree thus being equal to both of them 
as well (merge commit introducing no changes to either of its 
parents, originally described by Sergey as "angel merge").

But it doesn`t have to be so - if any of the rebased commits A1 to A3 
or B1 to B3 was dropped or modified (or extra commits added, even), 
that would influence the trees (snapshots) produced after rebasing U1 
and U2 to U1' and U2', final merge M' reflecting all these changes as 
well, besides keeping original merge commit M amendments (preserving 
"evil merge").


Well, that`s some theory, now to hopefully confirm/test/polish all 
this... or trash it, if flawed beyond correction :P

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

[-- Attachment #1: Type: text/plain, Size: 10495 bytes --]

Hi Buga,

thank you for making this a lot more understandable to this thick
developer.

On Tue, 27 Feb 2018, Igor Djordjevic wrote:

> On 27/02/2018 19:55, Igor Djordjevic wrote:
> > 
> > It would be more along the lines of "(1) rebase old merge commit parents, 
> > (2) generate separate diff between old merge commit and each of its 
> > parents, (3) apply each diff to their corresponding newly rebased 
> > parent respectively (as a temporary commit, one per rebased parent), 
> > (4) merge these temporary commits to generate 'rebased' merge commit, 
> > (5) drop temporary commits, recording their parents as parents of 
> > 'rebased' merge commit (instead of dropped temporary commits)".
> > 
> > Implementation wise, steps (2) and (3) could also be done by simply 
> > copying old merge commit _snapshot_ on top of each of its parents as 
> > a temporary, non-merge commit, then rebasing (cherry-picking) these 
> > temporary commits on top of their rebased parent commits to produce 
> > rebased temporary commits (to be merged for generating 'rebased' 
> > merge commit in step (4)).
> 
> For those still tagging along (and still confused), here are some 
> diagrams (following what Sergey originally described). Note that 
> actual implementation might be even simpler, but I believe it`s a bit 
> easier to understand like this, using some "temporary" commits approach.
> 
> Here`s our starting position:
> 
> (0) ---X1---o---o---o---o---o---X2 (master)
>        |\
>        | A1---A2---A3
>        |             \
>        |              M (topic)
>        |             /
>        \-B1---B2---B3
> 
> 
> Now, we want to rebase merge commit M from X1 onto X2. First, rebase
> merge commit parents as usual:
> 
> (1) ---X1---o---o---o---o---o---X2
>        |\                       |\
>        | A1---A2---A3           | A1'--A2'--A3'
>        |             \          |
>        |              M         |
>        |             /          |
>        \-B1---B2---B3           \-B1'--B2'--B3'
> 
> 
> That was commonly understandable part.

Good. Let's assume that I want to do this interactively (because let's
face it, rebase is boring unless we shake up things a little). And let's
assume that A1 is my only change to the README, and that I realized that
it was incorrect and I do not want the world to see it, so I drop A1'.

Let's see how things go from here:

> Now, for "rebasing" the merge commit (keeping possible amendments), we
> do some extra work. First, we make two temporary commits on top of old
> merge parents, by using exact tree (snapshot) of commit M:
> 
> (2) ---X1---o---o---o---o---o---X2
>        |\                       |\
>        | A1---A2---A3---U1      | A1'--A2'--A3'
>        |             \          |
>        |              M         |
>        |             /          |
>        \-B1---B2---B3---U2      \-B1'--B2'--B3'

Okay, everything would still be the same except that I still have dropped
A1'.

> So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.
> 
> Now, we rebase these temporary commits, too:
> 
> (3) ---X1---o---o---o---o---o---X2
>        |\                       |\
>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>        |             \          |
>        |              M         |
>        |             /          |
>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'

I still want to drop A1 in this rebase, so A1' is still missing.

And now it starts to get interesting.

The diff between A3 and U1 does not touch the README, of course, as I said
that only A1 changed the README.  But the diff between B3 and U2 does
change the README, thanks to M containing A1 change.

Therefore, the diff between B3' and U2' will also have this change to the
README. That change that I wanted to drop.

> As a next step, we merge these temporary commits to produce our
> "rebased" merged commit M:
> 
> (4) ---X1---o---o---o---o---o---X2
>        |\                       |\
>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>        |             \          |                  \
>        |              M         |                   M'
>        |             /          |                  /
>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'

And here, thanks to B3'..U2' changing the README, M' will also have that
change that I wanted to see dropped.

Note that A1' is still dropped in my example.

> Finally, we drop temporary commits, and record rebased commits A3' 
> and B3' as our "rebased" merge commit parents instead (merge commit 
> M' keeps its same tree/snapshot state, just gets parents replaced):
> 
> (5) ---X1---o---o---o---o---o---X2
>        |\                       |\
>        | A1---A2---A3---U1      | A1'--A2'--A3'
>        |             \          |             \
>        |              M         |              M'
>        |             /          |             /
>        \-B1---B2---B3---U2      \-B1'--B2'--B3'

Now, thanks to U2' being dropped (and A1' *still* being dropped), the
change in the README that is still in M' is really only in M'. No other
rebased commit has it. That makes it look as if M' introduced this change
in addition to the changes that were merged between the merge parents.

This is what an "evil merge" is: it does more than just combine the
previously diverging branches. It introduces another change that was not
in any non-merge commits before it.

Sometimes, such an "evil merge" is necessary. For example, when master
converted a couple more `hash` references to `oid` including, say, in a
function signature, and the merged branch contains a new caller using that
old function signature: in this case, the merge commit must convert those
`hash` references to `oid` references, or the code won't compile.

In my example, where I dropped A1' specifically so that that embarrasingly
incorrect change to the README would not be seen by the world, though, the
evil merge would be truly evil: it would show said change to the world.
The exact opposite of what I wanted.

> And that`s it, our merge commit M has been "rebased" to M' :)
> 
> (6) ---X1---o---o---o---o---o---X2 (master)
>                                 |\
>                                 | A1'--A2'--A3'
>                                 |             \
>                                 |              M' (topic)
>                                 |             /
>                                 \-B1'--B2'--B3'
> 
> 
> Important thing to note here is that in our step (3) above, still in 
> terms of trees/snapshots (not diffs), U1' could still be equal to 
> U2', produced merge commit M' tree thus being equal to both of them 
> as well (merge commit introducing no changes to either of its 
> parents, originally described by Sergey as "angel merge").
> 
> But it doesn`t have to be so - if any of the rebased commits A1 to A3 
> or B1 to B3 was dropped or modified (or extra commits added, even), 
> that would influence the trees (snapshots) produced after rebasing U1 
> and U2 to U1' and U2', final merge M' reflecting all these changes as 
> well, besides keeping original merge commit M amendments (preserving 
> "evil merge").

Sadly, it would also introduce evil merges in certain circumstances, as I
demonstrated above.

> Well, that`s some theory, now to hopefully confirm/test/polish all 
> this... or trash it, if flawed beyond correction :P

It would have been nice to have such a simple solution ;-)

So the most obvious way to try to fix this design would be to recreate the
original merge first, even with merge conflicts, and then trying to use the
diff between that and the actual original merge commit. In your example,
this would look like that:

---X1---o---o---o---o---o---X2
   |\                       |\
   | A1---A2---A3--         | A1'--A2'--A3'
   |             \ \        |
   |              M M²      |
   |             / /        |
   \-B1---B2---B3--         \-B1'--B2'--B3'

Note that M² would be generated somewhat like this: `git checkout
A3; git merge -s recursive B3; git add -u; git commit`. If there are merge
conflicts, then the result would include the conflict markers.

Now we would generate something similar to those U1/U2 commits: a
single-parent commit R that reflects the diff between M and M²:

---X1---o---o---o---o---o---X2
   |\                       |\
   | A1---A2---A3           | A1'--A2'--A3'
   |             \          |
   |              M---R     |
   |             /          |
   \-B1---B2---B3           \-B1'--B2'--B3'

Note that the tree of R is identical to the tree of M². We can now
proceed to generate the merge between A3' and B3' (possibly with merge
conflicts) and then reverting R on top:

---X1---o---o---o---o---o---X2
   |\                       |\
   | A1---A2---A3           | A1'--A2'--A3'
   |             \          |              \
   |              M---R     |               M'---M³
   |             /          |              /
   \-B1---B2---B3           \-B1'--B2'--B3'

Of course, as before, the idea would be to squash the reverted
changes into the final merge commit.

Now, would this work?

I doubt it, for at least two reasons:

- if there are merge conflicts between A3/B3 and between A3'/B3', those
  merge conflicts will very likely look very different, and the conflicts
  when reverting R will contain those nested conflicts: utterly confusing.
  And those conflicts will look even more confusing if a patch (such as
  A1') was dropped during an interactive rebase.

- One of the promises was that the new way would also handle merge
  strategies other than recursive. What would happen, for example, if M
  was generated using `-s ours` (read: dropping the B* patches' changes)
  and if B1 had been cherry-picked into the history between X1..X2?

  Reverting R would obviously revert those B1 changes, even if B1' would
  obviously not even be part of the rebased history!

Yes, I agree that this `-s ours` example is quite concocted, but the point
of this example is not how plausible it is, but how easy it is to come up
with a scenario where this design to "rebase merge commits" results in
very, very unwanted behavior.

But maybe I missed something obvious, and the design can still be fixed
somehow?

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 27/02/2018 20:59, Igor Djordjevic wrote:
> 
> (3) ---X1---o---o---o---o---o---X2
>        |\                       |\
>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>        |             \          |
>        |              M         |
>        |             /          |
>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> 

Meh, I hope I`m rushing it now, but for example, if we had decided to 
drop commit A2 during an interactive rebase (so losing A2' from 
diagram above), wouldn`t U2' still introduce those changes back, once 
U1' and U2' are merged, being incorrect/unwanted behavior...? :/

p.s. Looks like Johannes already elaborated on this in the meantime, 
let`s see... (goes reading that other e-mail[1])

[1] https://public-inbox.org/git/nycvar.QRO.7.76.6.1802272330290.56@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 27/02/2018 20:59, Igor Djordjevic wrote:
>> 
>> (3) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>        |             \          |
>>        |              M         |
>>        |             /          |
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>> 
>
> Meh, I hope I`m rushing it now, but for example, if we had decided to 
> drop commit A2 during an interactive rebase (so losing A2' from 
> diagram above), wouldn`t U2' still introduce those changes back, once 
> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>
> p.s. Looks like Johannes already elaborated on this in the meantime, 
> let`s see... (goes reading that other e-mail[1])

As long as we are talking about rebase that allows us users to
adjust and make changes ("rebase -i" being the prime and most
flexible example), it is easy to imagine that A1'--A3' and B1'--B3'
have almost no relation to their original counterparts.  After all,
mechanical merge won't be able to guess the intention of the change
humans make, so depending on what happend during X1 and X2 that
happend outside of these two topics, what's required to bring series
A and B to series A' and B' can be unlimited.  So from that alone,
it should be clear that replaying difference between M and A3 (and M
and B3) on top of U1' and U2' is hopeless as a general solution.

It is acceptable as long as a solution fails loudly when it does the
wrong thing, but I do not think the apporach can produce incorrect
result silently, as your example shows above.

What you _COULD_ learn from an old merge is to compute:

    - a trial and mechanical merge between A3 and B3; call that pre-M

    - diff to bring pre-M to M (call that patch evil-M); which is
      what the person who made M did to resolve the textual and
      semantic conflicts necessary to merge these two topics.

Then when merging A3' and B3', you could try to mechanically merge
them (call that pre-M'), and apply evil-M, which may apply cleanly
on top of pre-M', or it may not.  When there aren't so huge a
difference between series A and A' (and series B and B'), the result
would probably be a moral equivalent of Sergay's "replay" (so this
approach will also silently produce a wrong result without human
supervision).  One edge the evil-M approach has over Sergey's "dual
cherry pick" is that it separates and highlights non-mechanical
conflict resolution out of mechanical merges in a human readable
form (i.e. the patch evil-M).

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Tue, Feb 27, 2018 at 8:21 AM, Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
> Hi Jake,
>
> On Mon, 26 Feb 2018, Jacob Keller wrote:
>
>> On Mon, Feb 26, 2018 at 4:07 PM, Johannes Schindelin
>> <Johannes.Schindelin@gmx.de> wrote:
>> >
>> > On Tue, 20 Feb 2018, Igor Djordjevic wrote:
>> >
>> >> I`m really interested in this topic, which seems to (try to) address the
>> >> only "bad feeling" I had with rebasing merges - being afraid of silently
>> >> losing amendments by actually trying to "replay" the merge (where
>> >> additional and possibly important context is missing), instead of really
>> >> "rebasing" it (somehow).
>> >
>> > If those amendments are what you are worried about, why not address them
>> > specifically?
>> >
>> > In other words, rather than performing the equivalent of
>> >
>> >         git show <merge>^! | git apply
>> >
>> > (which would of course completely ignore if the rewritten <merge>^2
>> > dropped a patch, amended a patch, or even added a new patch), what you
>> > really want is to figure out what changes the user made when merging, and
>> > what merge strategy was used to begin with.
>> >
>> > To see what I mean, look at the output of `git show 0fd90daba8`: it shows
>> > how conflicts were resolved. By necessity, this is more complicated than a
>> > simple diff: it is *not* as simple as taking a diff between two revisions
>> > and applying that diff to a third revision. There were (at least) three
>> > revisions involved in the original merge commit, and recreating that merge
>> > commit faithfully means to represent the essence of the merge commit
>> > faithfully enough to be able to replay it on a new set of at least three
>> > revisions.  That can be simplified to two-way diffs only in very, very
>> > special circumstances, and in all other cases this simplification will
>> > simply fall on its nose.
>> >
>> > If the proposed solution was to extend `git apply` to process combined
>> > diffs, I would agree that we're on to something. That is not the proposed
>> > solution, though.
>> >
>> > In short: while I am sympathetic to the desire to keep things simple,
>> > the idea to somehow side-step replaying the original merge seems to be
>> > *prone* to be flawed. Any system that cannot accommodate
>> > dropped/changed/added commits on either side of a merge is likely to be
>> > too limited to be useful.
>> >
>>
>>
>> The reason Sergey's solution works is because he cherry picks the
>> merge using each parent first, and then merges the result of those. So
>> each branch of the merge gets one, and then you merge the result of
>> those cherry-picks. This preservers amendments and changes properly,
>> and should result in a good solution.
>
> I saw your patch trying to add a minimal example, and I really want to run
> away screaming.
>
> Do you have any way to describe the idea in a simple, 3-5 lines long
> paragraph?
>
> So far, I just know that it is some sort of confusing criss-cross
> cherry-picking and merging and stuff, but nothing in those steps shouts
> out to me what the *idea* is.
>

Sergey's posted explained it more in detail, at
https://public-inbox.org/git/87y3jtqdyg.fsf@javad.com/

I was mostly just attempting to re-create it in a test case to show
that it could work.

> If it would be something like "recreate the old merge, with merge
> conflicts and all, then generate the diff to the actual tree of the merge
> commit, then apply that to the newly-generated merge", I would understand.
>

It's more or less:

Rebase each parent, then cherry-pick -m<N> the original merge to that
parent, then you merge the result of each cherry-pick, then use the
resulting final merged tree to create the merge pointing at the real
parents instead of the cherry-pick merges.

> I would still suspect that -s ours would be a hard nut for that method,
> but I would understand that idea.
>

The goal of the process isn't to know or understand the "-s ours"
strategy, but simply re-create the contents of the original merge
faithfully, while still preserving the changes done when rebasing the
side branches. Thus it should re-create the contents generated by "-s
ours" the first time, but it doesn't need to do or know anything
special about how the content was created.

> Thanks,
> Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Tue, Feb 27, 2018 at 10:14 AM, Junio C Hamano <gitster@pobox.com> wrote:
> Sergey Organov <sorganov@gmail.com> writes:
>
>> You've already bit this poor thingy to death. Please rather try your
>> teeth on the proposed Trivial Merge (TM) method.
>
> Whatever you do, do *NOT* call any part of your proposal "trivial
> merge", unless you are actually using the term to mean what Git
> calls "trivial merge".  The phrase has an established meaning in Git
> and your attempt to abuse it to mean something entirely different is
> adding unnecessary hindrance for other people to understand what you
> want to perform.

Agreed, I think we need better terminology here, the current words for
(TM) are definitely *not* trivial merges. Same for "angel merge", I
don't think that term really works well either.

The goal of the process is to split the merge apart to its components
for each side branch and then bring them back together after applying
them to the newly rebased branches.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Jacob Keller]

On Tue, Feb 27, 2018 at 3:40 PM, Igor Djordjevic
<igor.d.djordjevic@gmail.com> wrote:
> On 27/02/2018 20:59, Igor Djordjevic wrote:
>>
>> (3) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>        |             \          |
>>        |              M         |
>>        |             /          |
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>>
>
> Meh, I hope I`m rushing it now, but for example, if we had decided to
> drop commit A2 during an interactive rebase (so losing A2' from
> diagram above), wouldn`t U2' still introduce those changes back, once
> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>
> p.s. Looks like Johannes already elaborated on this in the meantime,
> let`s see... (goes reading that other e-mail[1])
>
> [1] https://public-inbox.org/git/nycvar.QRO.7.76.6.1802272330290.56@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/


In that case, the method won't work well at all, so I think we need a
different approach.

Thanks,
Jake

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 01:36, Jacob Keller wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> > >
> >
> > Meh, I hope I`m rushing it now, but for example, if we had decided to
> > drop commit A2 during an interactive rebase (so losing A2' from
> > diagram above), wouldn`t U2' still introduce those changes back, once
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> 
> In that case, the method won't work well at all, so I think we need a
> different approach.
> 

Hmm, still rushing it, but what about adding an additional step, 
something like this: 
 
 (4) ---X1---o---o---o---o---o---X2
        |\                       |\
        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'--R1
        |             \          |
        |              M         |
        |             /          |
        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'--R2


... where:

  R1 = git diff U2 U2' | git apply --3way
  R2 = git diff U1 U1' | git apply --3way

(this is just to explain the point, might be there is a better way to 
produce Rx)

So, we still use Ux' to preserve merge commit M amendments, but also 
Rx to catch any changes happening between Ux and Ux' caused by 
interactive rebase commit manipulation (add/amend/drop).

Note that R*1* is produced by applying diff from U*2*' side, and vice 
versa (as it`s the other side that can erroneously introduce dropped 
commit changes back, like U2' in case of dropped A2').

From here we continue as before - merging R1 and R2, then rewriting 
merge commit parents to point to A3' and B3' (dropping Ux` and Rx).

This seems to be working inside my (too trivial?) test case, for 
interactive adding, dropping, and amending of rebased commits, 
resulting "rebased" merge containing all the added/modified/dropped 
changes, plus the original merge amendment, all as expected :P

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 02:33, Igor Djordjevic wrote:
> 
> This seems to be working inside my (too trivial?) test case, for 
> interactive adding, dropping, and amending of rebased commits, 
> resulting "rebased" merge containing all the added/modified/dropped 
> changes, plus the original merge amendment, all as expected :P

In case anyone has a wish to examine my (now pretty messy) test 
script, here it is - sorry for not having time to clean it up! :(

What I get when I diff original and "rebased" merge is this:

    diff --git a/test.txt b/test.txt
    index a82470b..d458032 100644
    --- a/test.txt
    +++ b/test.txt
    @@ -1,10 +1,14 @@
    +A21
    +A22
    +A23
    +A24
    +A25
     A1
     A2
    -B11
    +B1111
     A3
     A4
     A5
    -B12
     A6
     A7
     A8
    @@ -14,6 +18,7 @@ A10
     A11
     A12
     A13
    +BX
     A14
     B2
     A15


... where A21 to A25 are additions due to new base, B11 was 
interactively amended to B1111, B12 was interactively dropped, and BX 
interactively added :)

We don`t see line X here, being an "evil merge" amendment being 
correctly preserved from original merge commit (thus not a 
difference). If we do `git show` of the "rebased" merge, we get this, 
as expected:

    diff --cc test.txt
    index b173cef,fad39a8..d458032
    --- a/test.txt
    +++ b/test.txt
    @@@ -13,6 -13,6 +13,7 @@@ A
      A7
      A8
      A9
    ++X
      A10
      A11
      A12


Regards, Buga

-- 8< --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

for i in {1..20}
do
	echo A$i >>test.txt
	git commit -am "A$i"
done

git checkout -b b1
sed -i '3iB11' test.txt
git commit -am "B11"
sed -i '7iB12' test.txt
git commit -am "B12"

git checkout -b b2 HEAD^
sed -i '16iB2' test.txt
git commit -am "B2"

git checkout -b merge b1
git merge --no-commit b2
sed -i '12iX' test.txt # amend merge commit
git commit -am "M"
git tag original-merge

git checkout master
for i in {1..5}
do
	j=`expr "$i" + 20`
	sed -i "${i}iA${j}" test.txt
	git commit -am "A$j"
done

# simple/naive demonstration of proposed merge rebasing logic
# using described "Trivial Merge" (TM, or "Angel Merge"),
# preserving merge commit manual amendments, but still respecting
# interactively rebased added/modified/dropped commits :)

# read -p "Press enter to continue"
git checkout b1
git cherry-pick -m1 original-merge && git tag U1
git reset --hard HEAD^^ # drop U1 and last b1 commit
sed -i '/B11/c\B1111' test.txt
git commit -a --amend --no-edit
git rebase master
git cherry-pick U1 && git tag U1-prime

# read -p "Press enter to continue"
git checkout b2
git cherry-pick -m2 original-merge && git tag U2
git reset --hard HEAD^ # drop U2
git rebase master
sed -i '20iBX' test.txt
git commit -am "BX" # add new commit
git cherry-pick U2 && git tag U2-prime

git diff U1 U1-prime | git apply --3way && git commit -m "U2-second" && git tag U2-second
git checkout b1
git diff U2 U2-prime | git apply --3way && git commit -m "U1-second" && git tag U1-second

# read -p "Press enter to continue"
git branch -f merge b1
git checkout merge
git merge b2 --no-commit
git commit -a --reuse-message original-merge
git tag angel-merge

# read -p "Press enter to continue"
git reset --hard b1^
git read-tree --reset angel-merge
git update-ref refs/heads/merge "$(git show -s --format=%B original-merge | git commit-tree "$(git write-tree)" -p "$(git rev-parse b1^^)" -p "$(git rev-parse b2^^)")"
git tag -f angel-merge
git checkout angel-merge .
git branch -f b1 b1^^
git branch -f b2 b2^^

# show resulting graph
echo
git log --all --decorate --oneline --graph

# comparison between original merge and rebased merge,
# showing merge commit amendment "X" being preserved during rebase
# (not shown in diff)
echo
echo 'diff original-merge angel-merge:'
git diff original-merge angel-merge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Johannes,

On 28/02/2018 00:27, Johannes Schindelin wrote:
> 
> thank you for making this a lot more understandable to this thick
> developer.

Hehe, no problem, it primarily served fighting my own thickness ;)

> > Finally, we drop temporary commits, and record rebased commits A3' 
> > and B3' as our "rebased" merge commit parents instead (merge commit 
> > M' keeps its same tree/snapshot state, just gets parents replaced):
> >
> > (5) ---X1---o---o---o---o---o---X2
> >        |\                       |\
> >        | A1---A2---A3---U1      | A1'--A2'--A3'
> >        |             \          |             \
> >        |              M         |              M'
> >        |             /          |             /
> >        \-B1---B2---B3---U2      \-B1'--B2'--B3'
> 
> ...
> 
> In my example, where I dropped A1' specifically so that that embarrasingly
> incorrect change to the README would not be seen by the world, though, the
> evil merge would be truly evil: it would show said change to the world.
> The exact opposite of what I wanted.

Yeah, I`m afraid that`s what my testing produced as well :( Back to 
the drawing board...

> It would have been nice to have such a simple solution ;-)

Eh, the worst thing is the feeling I have, like it`s just around the 
corner, but we`re somehow missing it :P

> So the most obvious way to try to fix this design would be to recreate the
> original merge first, even with merge conflicts, and then trying to use the
> diff between that and the actual original merge commit.

For simplicity sake, this is something I would like to avoid (if 
possible), and also for the reasons you mentioned yourself:

> Now, would this work?
> 
> I doubt it, for at least two reasons:
> 
> - if there are merge conflicts between A3/B3 and between A3'/B3', those
>   merge conflicts will very likely look very different, and the conflicts
>   when reverting R will contain those nested conflicts: utterly confusing.
>   And those conflicts will look even more confusing if a patch (such as
>   A1') was dropped during an interactive rebase.
> 
> - One of the promises was that the new way would also handle merge
>   strategies other than recursive. What would happen, for example, if M
>   was generated using `-s ours` (read: dropping the B* patches' changes)
>   and if B1 had been cherry-picked into the history between X1..X2?
> 
>   Reverting R would obviously revert those B1 changes, even if B1' would
>   obviously not even be part of the rebased history!
> 
> ...
> 
> But maybe I missed something obvious, and the design can still be fixed
> somehow?

Would additional step as suggested in [1] (using R1 and R2 to "catch" 
interactive rebase additions/amendments/drops, on top of U1' and 
U2'), make more sense (or provide an additional clue, at least)?

It`s late here, and I`m really rushing it now, so please forgive me if 
it`s a stupid one... :$

Regards, Buga

[1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d0d1@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Junio,

On 28/02/2018 01:10, Junio C Hamano wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> > >
> >
> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
> > drop commit A2 during an interactive rebase (so losing A2' from 
> > diagram above), wouldn`t U2' still introduce those changes back, once 
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> 
> As long as we are talking about rebase that allows us users to
> adjust and make changes ("rebase -i" being the prime and most
> flexible example), it is easy to imagine that A1'--A3' and B1'--B3'
> have almost no relation to their original counterparts.  After all,
> mechanical merge won't be able to guess the intention of the change
> humans make, so depending on what happend during X1 and X2 that
> happend outside of these two topics, what's required to bring series
> A and B to series A' and B' can be unlimited.  So from that alone,
> it should be clear that replaying difference between M and A3 (and M
> and B3) on top of U1' and U2' is hopeless as a general solution.

Yeah, I`ve encountered it in my (trivial) test case :(

> It is acceptable as long as a solution fails loudly when it does the
> wrong thing, but I do not think the apporach can produce incorrect
> result silently, as your example shows above.

Hmm, I think my example doesn`t even try to prevent failing, but it 
should otherwise be perfectly capable of doing so (and doing it 
loudly) - for example, it`s enough to diff U1' and U2' - if not the 
same, user might want to confirm the "rebased" merge outcome, as 
either something went wrong, or interactive rebase happened... or 
both :) (it`s what Sergey originally explained, seeming to be a solid 
safety net, though more testing would be good)

> What you _COULD_ learn from an old merge is to compute:
> 
>     - a trial and mechanical merge between A3 and B3; call that pre-M
> 
>     - diff to bring pre-M to M (call that patch evil-M); which is
>       what the person who made M did to resolve the textual and
>       semantic conflicts necessary to merge these two topics.
> 
> Then when merging A3' and B3', you could try to mechanically merge
> them (call that pre-M'), and apply evil-M, which may apply cleanly
> on top of pre-M', or it may not.  When there aren't so huge a
> difference between series A and A' (and series B and B'), the result
> would probably be a moral equivalent of Sergay's "replay" (so this
> approach will also silently produce a wrong result without human
> supervision).  One edge the evil-M approach has over Sergey's "dual
> cherry pick" is that it separates and highlights non-mechanical
> conflict resolution out of mechanical merges in a human readable
> form (i.e. the patch evil-M).

This seems to be what Johannes wrote about[1], too, but also 
something I think would be good to avoid, if possible, not 
complicating it too much :P

Maybe something like that latest thought[2] could help, using 
additional R1 and R2 commits to handle interactive rebase 
additions/amendments/drops, on top of U1' and U2'? Yeah, and 
that`s not complicating it... ;) :D

Regards, Buga

[1] https://public-inbox.org/git/nycvar.QRO.7.76.6.1802272330290.56@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/
[2] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d0d1@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 03:12, Igor Djordjevic wrote:
> 
> Would additional step as suggested in [1] (using R1 and R2 to "catch" 
> interactive rebase additions/amendments/drops, on top of U1' and 
> U2'), make more sense (or provide an additional clue, at least)?
> 
> [1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d0d1@gmail.com/

Heh, no sleeping tonight :P

Anyway, from (yet another) ad hoc test, additional step mentioned in 
[1] above seems to handle this case, too (merge with `-s ours` 
dropping B* patches, plus B1 cherry-picked between X1..X2):

On 28/02/2018 00:27, Johannes Schindelin wrote:
> 
> - One of the promises was that the new way would also handle merge
>   strategies other than recursive. What would happen, for example, if M
>   was generated using `-s ours` (read: dropping the B* patches' changes)
>   and if B1 had been cherry-picked into the history between X1..X2?
> 
>   Reverting R would obviously revert those B1 changes, even if B1' would
>   obviously not even be part of the rebased history!
> 
> Yes, I agree that this `-s ours` example is quite concocted, but the point
> of this example is not how plausible it is, but how easy it is to come up
> with a scenario where this design to "rebase merge commits" results in
> very, very unwanted behavior.

And not just that - it worked with additional interactive rebase 
adding, amending and removing commits, on top of all this still 
preserving original `-s ours` merge commit evil-merge amendment, too, 
all as expected (or so it seems) :P

Again, for the brave ones, here`s another messy test script (not 
tightly related to the last discussed diagrams, but based on my 
original "demonstration" script[2])... Hopefully I get to make a 
proper (and sane) sample soon... if not missing something here in the 
first place ;)

Regards, Buga

[2] https://public-inbox.org/git/bbe64321-4d3a-d3fe-8bb9-58b600fabf35@gmail.com/

-- 8< --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

for i in {1..20}
do
	echo A$i >>test.txt
	git commit -am "A$i"
done

git checkout -b b1
sed -i '3iB11' test.txt
git commit -am "B11"
sed -i '7iB12' test.txt
git commit -am "B12"

git checkout -b b2 HEAD^
sed -i '16iB21' test.txt
git commit -am "B21"
sed -i '18iB22' test.txt
git commit -am "B22"

git checkout -b merge b1
git merge -s ours --no-commit b2
sed -i '12iX' test.txt # amend merge commit
git commit -am "M"
git tag original-merge

git checkout master
git cherry-pick b2
for i in {1..5}
do
	j=`expr "$i" + 20`
	sed -i "${i}iA${j}" test.txt
	git commit -am "A$j"
done

# simple/naive demonstration of proposed merge rebasing logic
# using described "Trivial Merge" (TM, or "Angel Merge"),
# preserving merge commit manual amendments, but still respecting
# interactively rebased added/modified/dropped commits :)

# read -p "Press enter to continue"
git checkout b1
git cherry-pick -m1 original-merge && git tag U1
git reset --hard HEAD^^ # drop U1 and last b1 commit
sed -i '/B11/c\B1111' test.txt
git commit -a --amend --no-edit
git rebase master
git cherry-pick U1 && git tag U1-prime

# read -p "Press enter to continue"
git checkout b2
git cherry-pick -m2 original-merge && git tag U2
git reset --hard HEAD^ # drop U2
git rebase master
sed -i '20iBX' test.txt
git commit -am "BX" # add new commit
git cherry-pick U2 && git tag U2-prime

git diff U1 U1-prime | git apply --3way && git commit -m "U2-second" && git tag U2-second
git checkout b1
git diff U2 U2-prime | git apply --3way && git commit -m "U1-second" && git tag U1-second

# read -p "Press enter to continue"
git branch -f merge b1
git checkout merge
git merge b2 --no-commit
git commit -a --reuse-message original-merge
git tag angel-merge

# read -p "Press enter to continue"
git reset --hard b1^
git read-tree --reset angel-merge
git update-ref refs/heads/merge "$(git show -s --format=%B original-merge | git commit-tree "$(git write-tree)" -p "$(git rev-parse b1^^)" -p "$(git rev-parse b2^^)")"
git tag -f angel-merge
git checkout angel-merge .
git branch -f b1 b1^^
git branch -f b2 b2^^

# show resulting graph
echo
git log --all --decorate --oneline --graph

# comparison between original merge and rebased merge,
# showing merge commit amendment "X" being preserved during rebase
# (not shown in diff)
echo
echo 'diff original-merge angel-merge:'
git diff original-merge angel-merge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Junio C Hamano <gitster@pobox.com> writes:

> Sergey Organov <sorganov@gmail.com> writes:
>
>> You've already bit this poor thingy to death. Please rather try your
>> teeth on the proposed Trivial Merge (TM) method.
>
> Whatever you do, do *NOT* call any part of your proposal "trivial
> merge", unless you are actually using the term to mean what Git
> calls "trivial merge".  The phrase has an established meaning in Git
> and your attempt to abuse it to mean something entirely different is
> adding unnecessary hindrance for other people to understand what you
> want to perform.

Yeah, got it. It's confusing indeed.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 27/02/2018 20:59, Igor Djordjevic wrote:
>> 
>> (3) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>        |             \          |
>>        |              M         |
>>        |             /          |
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>> 
>
> Meh, I hope I`m rushing it now, but for example, if we had decided to 
> drop commit A2 during an interactive rebase (so losing A2' from 
> diagram above), wouldn`t U2' still introduce those changes back, once 
> U1' and U2' are merged, being incorrect/unwanted behavior...? :/

I think the method will handle this nicely.

When you "drop" A2, will A3' change, or stay intact?

If it changes, say, to A3'', the U1' will change to U1'', and the method
will propagate the change automatically.

If it A3' doesn't change, then there are no changes to take.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 28/02/2018 01:36, Jacob Keller wrote:
>> 
>> > > (3) ---X1---o---o---o---o---o---X2
>> > >        |\                       |\
>> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>> > >        |             \          |
>> > >        |              M         |
>> > >        |             /          |
>> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>> > >
>> >
>> > Meh, I hope I`m rushing it now, but for example, if we had decided to
>> > drop commit A2 during an interactive rebase (so losing A2' from
>> > diagram above), wouldn`t U2' still introduce those changes back, once
>> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>> 
>> In that case, the method won't work well at all, so I think we need a
>> different approach.
>> 
>
> Hmm, still rushing it, but what about adding an additional step, 
> something like this:

I think it's unneeded, as it should work fine without it, see another
reply.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Junio C Hamano <gitster@pobox.com> writes:

> Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:
>
>> On 27/02/2018 20:59, Igor Djordjevic wrote:
>>> 
>>> (3) ---X1---o---o---o---o---o---X2
>>>        |\                       |\
>>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>>        |             \          |
>>>        |              M         |
>>>        |             /          |
>>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>>> 
>>
>> Meh, I hope I`m rushing it now, but for example, if we had decided to 
>> drop commit A2 during an interactive rebase (so losing A2' from 
>> diagram above), wouldn`t U2' still introduce those changes back, once 
>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>
>> p.s. Looks like Johannes already elaborated on this in the meantime, 
>> let`s see... (goes reading that other e-mail[1])
>
> As long as we are talking about rebase that allows us users to
> adjust and make changes ("rebase -i" being the prime and most
> flexible example), it is easy to imagine that A1'--A3' and B1'--B3'
> have almost no relation to their original counterparts.  After all,
> mechanical merge won't be able to guess the intention of the change
> humans make, so depending on what happend during X1 and X2 that
> happend outside of these two topics, what's required to bring series
> A and B to series A' and B' can be unlimited.

You discuss some different method here. In my original proposal U1' and
U2' are to be merged. Nothing should be replayed on top of them. I.e.,
U1' is _already_ the result of replaying difference between M and A3 on
top of A3'.

> So from that alone, it should be clear that replaying difference
> between M and A3 (and M and B3) on top of U1' and U2' is hopeless as a
> general solution.

Getting U1' from U1 is the same complexity as getting A3' from A3, with
the same caveats. So, as general solution, it's as good as rebase of
non-merge commit.

> It is acceptable as long as a solution fails loudly when it does the
> wrong thing, but I do not think the apporach can produce incorrect
> result silently, as your example shows above.

I still believe the method handles simple cases automatically and
correctly and allows to immediately stop for amendment should something
suspect appear.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Buga,
>
> thank you for making this a lot more understandable to this thick
> developer.
>
> On Tue, 27 Feb 2018, Igor Djordjevic wrote:
>
>> On 27/02/2018 19:55, Igor Djordjevic wrote:
>> > 
>> > It would be more along the lines of "(1) rebase old merge commit parents, 
>> > (2) generate separate diff between old merge commit and each of its 
>> > parents, (3) apply each diff to their corresponding newly rebased 
>> > parent respectively (as a temporary commit, one per rebased parent), 
>> > (4) merge these temporary commits to generate 'rebased' merge commit, 
>> > (5) drop temporary commits, recording their parents as parents of 
>> > 'rebased' merge commit (instead of dropped temporary commits)".
>> > 
>> > Implementation wise, steps (2) and (3) could also be done by simply 
>> > copying old merge commit _snapshot_ on top of each of its parents as 
>> > a temporary, non-merge commit, then rebasing (cherry-picking) these 
>> > temporary commits on top of their rebased parent commits to produce 
>> > rebased temporary commits (to be merged for generating 'rebased' 
>> > merge commit in step (4)).
>> 
>> For those still tagging along (and still confused), here are some 
>> diagrams (following what Sergey originally described). Note that 
>> actual implementation might be even simpler, but I believe it`s a bit 
>> easier to understand like this, using some "temporary" commits approach.
>> 
>> Here`s our starting position:
>> 
>> (0) ---X1---o---o---o---o---o---X2 (master)
>>        |\
>>        | A1---A2---A3
>>        |             \
>>        |              M (topic)
>>        |             /
>>        \-B1---B2---B3
>> 
>> 
>> Now, we want to rebase merge commit M from X1 onto X2. First, rebase
>> merge commit parents as usual:
>> 
>> (1) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3           | A1'--A2'--A3'
>>        |             \          |
>>        |              M         |
>>        |             /          |
>>        \-B1---B2---B3           \-B1'--B2'--B3'
>> 
>> 
>> That was commonly understandable part.
>
> Good. Let's assume that I want to do this interactively (because let's
> face it, rebase is boring unless we shake up things a little). And let's
> assume that A1 is my only change to the README, and that I realized that
> it was incorrect and I do not want the world to see it, so I drop A1'.
>
> Let's see how things go from here:
>
>> Now, for "rebasing" the merge commit (keeping possible amendments), we
>> do some extra work. First, we make two temporary commits on top of old
>> merge parents, by using exact tree (snapshot) of commit M:
>> 
>> (2) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'
>>        |             \          |
>>        |              M         |
>>        |             /          |
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'
>
> Okay, everything would still be the same except that I still have dropped
> A1'.
>
>> So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.
>> 
>> Now, we rebase these temporary commits, too:
>> 
>> (3) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>        |             \          |
>>        |              M         |
>>        |             /          |
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>
> I still want to drop A1 in this rebase, so A1' is still missing.
>
> And now it starts to get interesting.
>
> The diff between A3 and U1 does not touch the README, of course, as I said
> that only A1 changed the README.  But the diff between B3 and U2 does
> change the README, thanks to M containing A1 change.
>
> Therefore, the diff between B3' and U2' will also have this change to the
> README. That change that I wanted to drop.
>
>> As a next step, we merge these temporary commits to produce our
>> "rebased" merged commit M:
>> 
>> (4) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>        |             \          |                  \
>>        |              M         |                   M'
>>        |             /          |                  /
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>
> And here, thanks to B3'..U2' changing the README, M' will also have that
> change that I wanted to see dropped.
>
> Note that A1' is still dropped in my example.
>
>> Finally, we drop temporary commits, and record rebased commits A3' 
>> and B3' as our "rebased" merge commit parents instead (merge commit 
>> M' keeps its same tree/snapshot state, just gets parents replaced):
>> 
>> (5) ---X1---o---o---o---o---o---X2
>>        |\                       |\
>>        | A1---A2---A3---U1      | A1'--A2'--A3'
>>        |             \          |             \
>>        |              M         |              M'
>>        |             /          |             /
>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'
>
> Now, thanks to U2' being dropped (and A1' *still* being dropped), the
> change in the README that is still in M' is really only in M'. No other
> rebased commit has it. That makes it look as if M' introduced this change
> in addition to the changes that were merged between the merge parents.

Except that original proposal suggests to cosider this a conflict and
to stop for amendment, as U1' and U2' now differ.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Jacob Keller <jacob.keller@gmail.com> writes:

> On Tue, Feb 27, 2018 at 10:14 AM, Junio C Hamano <gitster@pobox.com> wrote:
>> Sergey Organov <sorganov@gmail.com> writes:
>>
>>> You've already bit this poor thingy to death. Please rather try your
>>> teeth on the proposed Trivial Merge (TM) method.
>>
>> Whatever you do, do *NOT* call any part of your proposal "trivial
>> merge", unless you are actually using the term to mean what Git
>> calls "trivial merge".  The phrase has an established meaning in Git
>> and your attempt to abuse it to mean something entirely different is
>> adding unnecessary hindrance for other people to understand what you
>> want to perform.
>
> Agreed, I think we need better terminology here, the current words for
> (TM) are definitely *not* trivial merges. Same for "angel merge", I
> don't think that term really works well either.

Agreed.

How do we call a merge that introduces no differences on either side of
the merge then? Is there some English for even more trivial than what
Git calls "trivial merge"?

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 28/02/2018 03:12, Igor Djordjevic wrote:
>> 
>> Would additional step as suggested in [1] (using R1 and R2 to "catch" 
>> interactive rebase additions/amendments/drops, on top of U1' and 
>> U2'), make more sense (or provide an additional clue, at least)?
>> 
>> [1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d0d1@gmail.com/
>
> Heh, no sleeping tonight :P
>
> Anyway, from (yet another) ad hoc test, additional step mentioned in 
> [1] above seems to handle this case, too (merge with `-s ours` 
> dropping B* patches, plus B1 cherry-picked between X1..X2):
>
> On 28/02/2018 00:27, Johannes Schindelin wrote:
>> 
>> - One of the promises was that the new way would also handle merge
>>   strategies other than recursive. What would happen, for example, if M
>>   was generated using `-s ours` (read: dropping the B* patches' changes)
>>   and if B1 had been cherry-picked into the history between X1..X2?
>> 
>>   Reverting R would obviously revert those B1 changes, even if B1' would
>>   obviously not even be part of the rebased history!
>> 
>> Yes, I agree that this `-s ours` example is quite concocted, but the point
>> of this example is not how plausible it is, but how easy it is to come up
>> with a scenario where this design to "rebase merge commits" results in
>> very, very unwanted behavior.
>
> And not just that - it worked with additional interactive rebase 
> adding, amending and removing commits, on top of all this still 
> preserving original `-s ours` merge commit evil-merge amendment, too, 
> all as expected (or so it seems) :P

Great! I do believe that once we start from some sensible approach, many
kinds of improvements are possible. I'll definitely need to take close
look at what you came up with, thanks!

I'd like to say though that nobody should expect miracles from automated
rebasing of merges when we get to complex history editing. It will need
to retreat to manual merge, sooner or later.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 06:21, Sergey Organov wrote:
> 
> > > > > (3) ---X1---o---o---o---o---o---X2
> > > > >        |\                       |\
> > > > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > > > >        |             \          |
> > > > >        |              M         |
> > > > >        |             /          |
> > > > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> > > > >
> > > >
> > > > Meh, I hope I`m rushing it now, but for example, if we had decided to
> > > > drop commit A2 during an interactive rebase (so losing A2' from
> > > > diagram above), wouldn`t U2' still introduce those changes back, once
> > > > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> > >
> > > In that case, the method won't work well at all, so I think we need a
> > > different approach.
> > >
> >
> > Hmm, still rushing it, but what about adding an additional step, 
> > something like this:
> 
> I think it's unneeded, as it should work fine without it, see another
> reply.

Unfortunately, I have a broken test case saying different - it could 
very well be a flawed test, too, but let`s elaborate in that 
other sub-thread[1], indeed.

Regards, Buga

[1] https://public-inbox.org/git/87606hoflx.fsf@javad.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 06:44, Sergey Organov wrote:
> 
> > > Here`s our starting position:
> > >
> > > (0) ---X1---o---o---o---o---o---X2 (master)
> > >        |\
> > >        | A1---A2---A3
> > >        |             \
> > >        |              M (topic)
> > >        |             /
> > >        \-B1---B2---B3
> > >
> > >
> > > Now, we want to rebase merge commit M from X1 onto X2. First, rebase
> > > merge commit parents as usual:
> > >
> > > (1) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3           | A1'--A2'--A3'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3           \-B1'--B2'--B3'
> > >
> > >
> > > That was commonly understandable part.
> >
> > Good. Let's assume that I want to do this interactively (because let's
> > face it, rebase is boring unless we shake up things a little). And let's
> > assume that A1 is my only change to the README, and that I realized that
> > it was incorrect and I do not want the world to see it, so I drop A1'.
> >
> > Let's see how things go from here:
> >
> > > Now, for "rebasing" the merge commit (keeping possible amendments), we
> > > do some extra work. First, we make two temporary commits on top of old
> > > merge parents, by using exact tree (snapshot) of commit M:
> > >
> > > (2) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'
> >
> > Okay, everything would still be the same except that I still have dropped
> > A1'.
> >
> > > So here, in terms of _snapshots_ (trees, not diffs), U1 = U2 = M.
> > >
> > > Now, we rebase these temporary commits, too:
> > >
> > > (3) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> >
> > I still want to drop A1 in this rebase, so A1' is still missing.
> >
> > And now it starts to get interesting.
> >
> > The diff between A3 and U1 does not touch the README, of course, as I said
> > that only A1 changed the README.  But the diff between B3 and U2 does
> > change the README, thanks to M containing A1 change.
> >
> > Therefore, the diff between B3' and U2' will also have this change to the
> > README. That change that I wanted to drop.
> >
> > > As a next step, we merge these temporary commits to produce our
> > > "rebased" merged commit M:
> > >
> > > (4) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > >        |             \          |                  \
> > >        |              M         |                   M'
> > >        |             /          |                  /
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> >
> > And here, thanks to B3'..U2' changing the README, M' will also have that
> > change that I wanted to see dropped.
> >
> > Note that A1' is still dropped in my example.
> >
> > > Finally, we drop temporary commits, and record rebased commits A3' 
> > > and B3' as our "rebased" merge commit parents instead (merge commit 
> > > M' keeps its same tree/snapshot state, just gets parents replaced):
> > >
> > > (5) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'
> > >        |             \          |             \
> > >        |              M         |              M'
> > >        |             /          |             /
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'
> >
> > Now, thanks to U2' being dropped (and A1' *still* being dropped), the
> > change in the README that is still in M' is really only in M'. No other
> > rebased commit has it. That makes it look as if M' introduced this change
> > in addition to the changes that were merged between the merge parents.
> 
> Except that original proposal suggests to cosider this a conflict and
> to stop for amendment, as U1' and U2' now differ.

Thanks for bringing this one up again - I focused on a mere example 
of how the approach could work, not stressing enough the simplicity 
of recognizing when it really doesn`t, which is an important aspect 
to know as well, indeed, supporting much needed trust that no bad 
things could happen behind our back.

Either work as expected, or fail loudly, yes.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 06:19, Sergey Organov wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> > >
> >
> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
> > drop commit A2 during an interactive rebase (so losing A2' from 
> > diagram above), wouldn`t U2' still introduce those changes back, once 
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> 
> I think the method will handle this nicely.

That`s what I thought as well. And then I made a test. And then the 
test broke... Now, might be the test was flawed in the first place, 
but thinking about it a bit more, it does seem to make sense not to 
handle this case nicely :/

> When you "drop" A2, will A3' change, or stay intact?

When you "drop" commit A2 from rebasing, patch (diff) A3' does stay 
the same - but resulting tree (snapshot) after applying it does not.

By removing commit A2 from your rebase, you`re saying that changes 
introduced in that commit shouldn`t ever happen in the rebased tree, 
so trees/snapshots of all rebased commits coming after dropped A2 
will have these changes missing, in comparison to trees of original 
commits they`re being rebased from.

> If it changes, say, to A3'', the U1' will change to U1'', and the method
> will propagate the change automatically.
> 
> If it A3' doesn't change, then there are no changes to take.

All true, but note what I wrote in the original message - the issue 
of dropping A2 is not U1, but U2, and that one doesn`t change.

In this case, U1' will correctly represent U1 rebased on top of A3' 
(where A2 is now missing), no problem there.

But on the other end, as U2 holds changes between original merge and 
B3 (being A1, A2, A3 and evil-merge M), it will also still hold 
changes introduced by original A2.

Rebasing it onto B3' brings all these changes along, and once merged 
with U1' to produce "rebased" merge it unexpectedly introduces 
supposedly dropped commit A2 changes in their full glory.

Yes, considering this situation a conflict, as originally proposed, 
by simply noticing that U1' and U2' differ, helps this to fail 
loudly without doing the wrong thing.

But U1' and U2' are really to be expected to stay the same in 
non-interactive rebase case only, where it doesn`t help interactive 
rebase case at all if it is to fail most of the time (where U1' and 
U2' don`t have to be, but should be expected to possibly be different).

So while your original proposal currently seems like it could be 
working nicely for non-interactive rebase (and might be some simpler 
interactive ones), now hitting/acknowledging its first real use 
limit, my additional quick attempt[1] just tries to aid pretty 
interesting case of complicated interactive rebase, too, where we 
might be able to do better as well, still using you original proposal 
as a base idea :)

Regards, Buga

[1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d0d1@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 28/02/2018 07:14, Sergey Organov wrote:
> 
> > > Would additional step as suggested in [1] (using R1 and R2 to "catch" 
> > > interactive rebase additions/amendments/drops, on top of U1' and 
> > > U2'), make more sense (or provide an additional clue, at least)?
> > > 
> > > [1] https://public-inbox.org/git/8829c395-fb84-2db0-9288-f7b28fa0d0d1@gmail.com/
> > 
> > Anyway, from (yet another) ad hoc test, additional step mentioned in 
> > [1] above seems to handle this case, too (merge with `-s ours` 
> > dropping B* patches, plus B1 cherry-picked between X1..X2)
> > 
> > ...
> > 
> > And not just that - it worked with additional interactive rebase 
> > adding, amending and removing commits, on top of all this still 
> > preserving original `-s ours` merge commit evil-merge amendment, too, 
> > all as expected (or so it seems) :P
> 
> Great! I do believe that once we start from some sensible approach, many
> kinds of improvements are possible. I'll definitely need to take close
> look at what you came up with, thanks!
> 
> I'd like to say though that nobody should expect miracles from automated
> rebasing of merges when we get to complex history editing. It will need
> to retreat to manual merge, sooner or later.

I agree, and as I really liked "the feeling" of the original approach 
you described, it felt bad to (presumably) see it failing in what 
doesn`t seem to be neither too complex nor rare situation of dropping 
a commit during an interactive rebase, thus motivation to try to 
improve it, if possible, wasn`t lacking :)
 
Eh, might be I`m just naively ignorant at the moment as well, but I`m 
trying to work my way through it... ;)

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 28/02/2018 21:25, Igor Djordjevic wrote:
> 
> But U1' and U2' are really to be expected to stay the same in 
> non-interactive rebase case only...

Just to rephrase to "could be expected" here, meaning still not 
necessarily in this case, either - I`ve just witnessed 
non-interactive rebase Johannes previously described[1], merge with 
`-s ours` (dropping B* patches), plus B1 cherry-picked between 
X1..X2, eventually coming up with different U1' and U2', too (which 
would produce a wrong end result, if continued).

But I guess this should go to the "complex history" pile, good thing 
still being that diff safety check between U1' and U2' works as 
expected, thus automatic merge rebase can be aborted and command 
given back to the user for closer inspection.

[1] https://public-inbox.org/git/nycvar.QRO.7.76.6.1802272330290.56@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 28/02/2018 21:25, Igor Djordjevic wrote:
>> 
>> But U1' and U2' are really to be expected to stay the same in 
>> non-interactive rebase case only...
>
> Just to rephrase to "could be expected" here, meaning still not 
> necessarily in this case, either - I`ve just witnessed 
> non-interactive rebase Johannes previously described[1], merge with 
> `-s ours` (dropping B* patches), plus B1 cherry-picked between 
> X1..X2, eventually coming up with different U1' and U2', too (which 
> would produce a wrong end result, if continued).

Even non-interactive rebase may bring arbitrary asymmetric changes on
both sides of the merge, especially likely when resolving conflicts
needs to take place. OTOH, interactive history editing session could
have merges that we don't intend to edit at all. Overall, neither
non-interactive case is in fact much simpler, nor interactive case is so
much different.

> But I guess this should go to the "complex history" pile, good thing 
> still being that diff safety check between U1' and U2' works as 
> expected, thus automatic merge rebase can be aborted and command 
> given back to the user for closer inspection.

Exactly. This fact hopefully won't stop us from looking for more
suitable automated handling of the general case though. It should still
be our goal to reduce the number of such aborts and to suggest better
merge result for amendment when we are still aborting.

Your proposal hopefully is such a valuable improvement.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

[...]
>> > Hmm, still rushing it, but what about adding an additional step, 
>> > something like this:
>> 
>> I think it's unneeded, as it should work fine without it, see another
>> reply.
>
> Unfortunately, I have a broken test case saying different - it could 
> very well be a flawed test, too, but let`s elaborate in that 
> other sub-thread[1], indeed.

Yeah, I was too fast to reply and I was wrong, sorry about it.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>
> On 28/02/2018 06:19, Sergey Organov wrote:
>> 
>> > > (3) ---X1---o---o---o---o---o---X2
>> > >        |\                       |\
>> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>> > >        |             \          |
>> > >        |              M         |
>> > >        |             /          |
>> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>> > >
>> >
>> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
>> > drop commit A2 during an interactive rebase (so losing A2' from 
>> > diagram above), wouldn`t U2' still introduce those changes back, once 
>> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>> 
>> I think the method will handle this nicely.
>
> That`s what I thought as well. And then I made a test. And then the 
> test broke... Now, might be the test was flawed in the first place, 
> but thinking about it a bit more, it does seem to make sense not to 
> handle this case nicely :/

Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
this more carefully in the first place.

[...]

> So while your original proposal currently seems like it could be 
> working nicely for non-interactive rebase (and might be some simpler 
> interactive ones), now hitting/acknowledging its first real use 
> limit, my additional quick attempt[1] just tries to aid pretty 
> interesting case of complicated interactive rebase, too, where we 
> might be able to do better as well, still using you original proposal 
> as a base idea :)

Yes, thank you for pushing me back to reality! :-) The work and thoughts
you are putting into solving the puzzle are greatly appreciated!

Thinking about it overnight, I now suspect that original proposal had a
mistake in the final merge step. I think that what you did is a way to
fix it, and I want to try to figure what exactly was wrong in the
original proposal and to find simpler way of doing it right.

The likely solution is to use original UM as a merge-base for final
3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
though, as that's exactly UM from which both U1' and U2' have diverged
due to rebasing and other history editing.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 01/03/2018 06:39, Sergey Organov wrote:
> 
> > > (3) ---X1---o---o---o---o---o---X2
> > >        |\                       |\
> > >        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> > >        |             \          |
> > >        |              M         |
> > >        |             /          |
> > >        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> > >
> > 
> > Meh, I hope I`m rushing it now, but for example, if we had decided to 
> > drop commit A2 during an interactive rebase (so losing A2' from 
> > diagram above), wouldn`t U2' still introduce those changes back, once 
> > U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> > 
> > [...]
> 
> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
> this more carefully in the first place.

No problem, that`s why we`re discussing it, and I`m glad we`re 
aligned now, so we can move forward :)

> > So while your original proposal currently seems like it could be 
> > working nicely for non-interactive rebase (and might be some simpler 
> > interactive ones), now hitting/acknowledging its first real use 
> > limit, my additional quick attempt[1] just tries to aid pretty 
> > interesting case of complicated interactive rebase, too, where we 
> > might be able to do better as well, still using you original proposal 
> > as a base idea :)
> 
> Yes, thank you for pushing me back to reality! :-) The work and thoughts
> you are putting into solving the puzzle are greatly appreciated!

You`re welcome, and I am enjoying it :)

> Thinking about it overnight, I now suspect that original proposal had a
> mistake in the final merge step. I think that what you did is a way to
> fix it, and I want to try to figure what exactly was wrong in the
> original proposal and to find simpler way of doing it right.
> 
> The likely solution is to use original UM as a merge-base for final
> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> though, as that's exactly UM from which both U1' and U2' have diverged
> due to rebasing and other history editing.

Yes, this might be it...! ;)

To prove myself it works, I`ve assembled a pretty crazy `-s ours`  
merge interactive rebase scenario, and it seems this passes the test, 
ticking all the check boxes (I could think of) :P

Let`s see our starting situation:

 (0) ---X8--B2'--X9 (master)
        |\
        | A1---A2---A3 (A)
        |             \
        |              M (topic)
        |             /
        \-B1---B2---B3 (B)


Here, merge commit M is done with `-s ours` (obsoleting branch "B"), 
plus amended to make it an "evil merge", where a commit B2 from 
obsoleted branch "B" is cherry picked to "master".

Now, we want to rebase "topic" (M) onto updated "master" (X9), but to 
make things more interesting, we`ll do it interactively, with some 
amendments, drops, additions and even more cherry-picks!

This is what the final result looks like:

 (1) ---X8--B2'--X9 (master)
                 |\
                 | A12--A2'---B3' (A)
                 |             \
                 |              M' (topic)
                 |             /
                 \-B1'--B3'---B4  (B)


During interactive rebase, on branch "A", we amended A1 into A12, 
dropped A3 and cherry-picked B3. On branch "B", B4 is added, B2' being 
omitted automatically as already present in "master".

So... In comparison to original merge commit M, rebased merge commit 
M' is expected to:

 - Add X9, from updated "master"
 - Have A1 changed to A12, due to A12 commit amendment
 - Keep A2, rebased as A2'
 - Remove A3, due to dropped A3 commit
 - Keep amendment from original (evil) merge commit M
 - Miss B1' like M does B, due to original `-s ours` merge strategy
 - Add B2, cherry-picked as B2' into "master"
 - Add B3, cherry-picked as B3' into "A"
 - Add B4, added to "B"
 - Most important, provide safety mechanism to "fail loud", being 
   aware of non-trivial things going on, allowing to stop for user 
   inspection/decision


There, I hope I didn`t miss any expectation. And, it _seems_ to work 
exactly as expected :D

Not to leave this to imagination only, and hopefully helping others 
to get to speed and possibly discuss this, pointing to still possible 
flaws, I`m adding a demo script[1], showing how this exact example 
works.

Note that script _is_ coined to avoid rebase conflicts, as they`re not 
currently important for the point to be made here.

In real life, except for usual possibility for conflicts during 
commit rebasing, we might experience _three_ possible conflict 
situations once "rebased" merge itself is to be created - two when 
rebasing each of temporary merge helper commits, and one on the 
"rebased" merge itself. This is something where we might think about 
user experience, not introducing (too much) confusion...

Regards, Buga

[1] Demonstration script:
-- >8 --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

# prepare repository
for i in {1..8}
do
	echo X$i >>test.txt
	git commit -am "X$i"
done

# prepare branch A
git checkout -b A
sed -i '2iA1' test.txt
git commit -am "A1"
sed -i '4iA2' test.txt
git commit -am "A2"
sed -i '6iA3' test.txt
git commit -am "A3"

# prepare branch B
git checkout -b B master
sed -i '5iB1' test.txt
git commit -am "B1"
sed -i '7iB2' test.txt
git commit -am "B2"
sed -i '9iB3' test.txt
git commit -am "B3"

git checkout -b topic A
git merge -s ours --no-commit B # merge A and B with `-s ours`
sed -i '8iM' test.txt           # amend merge commit ("evil merge")
git commit -am "M"
git tag original-merge

# master moves on...
git checkout master
git cherry-pick B^     # cherry-pick B2 into master
sed -i "1iX9" test.txt # add X9
git commit -am "X9"

# (0) ---X8--B2'--X9 (master)
#        |\
#        | A1---A2---A3 (A)
#        |             \
#        |              M (topic)
#        |             /
#        \-B1---B2---B3 (B)

# simple/naive demonstration of proposed merge rebasing logic
# using described new approach, preserving merge commit manual
# amendments, testing `-s ours` merge with cherry-picking from
# obsoleted part, but still respecting interactively rebased
# added/modified/dropped/cherry-picked commits :)

git checkout A
git cherry-pick -m1 original-merge # prepare temporary helper commit U1
git tag U1
git reset --hard HEAD^^            # drop U1 and A3 from A
sed -i '/A1/c\A12' test.txt        # amend A1 to A12
git commit -a --amend --no-edit
git rebase master                  # rebase A onto master
git cherry-pick B                  # cherry-pick B3 into A
git cherry-pick U1                 # "rebase" temporary helper commit U1
git tag U1-prime

git checkout B
git cherry-pick -m2 original-merge # prepare temporary helper commit U2
git tag U2
git reset --hard HEAD^             # drop U2 from B
git rebase master                  # rebase B onto master
sed -i '12iB4' test.txt            # add B4
git commit -am "B4"
git cherry-pick U2                 # "rebase" temporary helper commit U2
git tag U2-prime

git branch -f topic A
git checkout topic
# merge rebased temporary commits U1' and U2',
# using original merge commit as a merge base,
# producing "rebased" merge commit M'
git read-tree -m --aggressive original-merge A B
git merge-index -o git-merge-one-file -a

# recognize complex stuff going on during rebasing merge commit,
# allowing user to inspect result, edit, and continue or abort
git diff --quiet U1-prime U2-prime
if test $? -ne 0
then
	# PLACEHOLDER
	# chance to inspect result, like:
	git diff original-merge
	# edit if needed, continue or abort
fi

# drop rebased temporary commits U1' and U2'
git branch -f A A^
git branch -f B B^

# record branches A and B as parents of "rebased" merge commit M',
# updating topic branch
git update-ref refs/heads/topic "$(git show -s --format=%B original-merge | git commit-tree "$(git write-tree)" -p "$(git rev-parse A)" -p "$(git rev-parse B)")"
git tag angel-merge

# (1) ---X8--B2'--X9 (master)
#                 |\
#                 | A12--A2'---B3' (A)
#                 |             \
#                 |              M' (topic)
#                 |             /
#                 \-B1'--B3'---B4  (B)

# show resulting graph
# echo
# git log --all --decorate --oneline --graph

# in comparison to original merge commit M, rebased merge commit 
# M' is expected to:
#
# - Add X9, from updated "master"
# - Have A1 changed to A12, due to A12 commit amendment
# - Keep A2, rebased as A2'
# - Remove A3, due to dropped A3 commit
# - Keep amendment from original (evil) merge commit M
# - Miss B1' like M does B, due to original `-s ours` merge strategy
# - Add B2, cherry-picked as B2' into "master"
# - Add B3, cherry-picked as B3' into "A"
# - Add B4, added to "B"
#
# echo
# echo 'diff original-merge angel-merge:'
# git diff original-merge angel-merge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>
> On 01/03/2018 06:39, Sergey Organov wrote:

[...]

>> 
>> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
>> this more carefully in the first place.
>
> No problem, that`s why we`re discussing it, and I`m glad we`re 
> aligned now, so we can move forward :)
>
>> > So while your original proposal currently seems like it could be 
>> > working nicely for non-interactive rebase (and might be some simpler 
>> > interactive ones), now hitting/acknowledging its first real use 
>> > limit, my additional quick attempt[1] just tries to aid pretty 
>> > interesting case of complicated interactive rebase, too, where we 
>> > might be able to do better as well, still using you original proposal 
>> > as a base idea :)
>> 
>> Yes, thank you for pushing me back to reality! :-) The work and thoughts
>> you are putting into solving the puzzle are greatly appreciated!
>
> You`re welcome, and I am enjoying it :)
>
>> Thinking about it overnight, I now suspect that original proposal had a
>> mistake in the final merge step. I think that what you did is a way to
>> fix it, and I want to try to figure what exactly was wrong in the
>> original proposal and to find simpler way of doing it right.
>> 
>> The likely solution is to use original UM as a merge-base for final
>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>> though, as that's exactly UM from which both U1' and U2' have diverged
>> due to rebasing and other history editing.
>
> Yes, this might be it...! ;)
>
> To prove myself it works, I`ve assembled a pretty crazy `-s ours`  
> merge interactive rebase scenario, and it seems this passes the test, 
> ticking all the check boxes (I could think of) :P

I must admit it's quite a relief to hear this. What we now have is so
simple and obvious that it'd be a huge disappointment if didn't work!

> Here, merge commit M is done with `-s ours` (obsoleting branch "B"), 
> plus amended to make it an "evil merge", where a commit B2 from 
> obsoleted branch "B" is cherry picked to "master".

[...]

> There, I hope I didn`t miss any expectation. And, it _seems_ to work 
> exactly as expected :D

That's very nice, to the level of being even suspect! :-)

To avoid falling into euphoria though, we need to keep in mind that
"expectations" is rather vague concept, and we likely still need to stop
for user amendment unless we absolutely sure nothing surprising happens.
I.e., we better require U1'==U2' test to succeed to proceed non-stop
automatically. Besides, it will be somewhat inline with what 'rerere'
does.

> In real life, except for usual possibility for conflicts during 
> commit rebasing, we might experience _three_ possible conflict 
> situations once "rebased" merge itself is to be created - two when 
> rebasing each of temporary merge helper commits, and one on the 
> "rebased" merge itself. This is something where we might think about 
> user experience, not introducing (too much) confusion...

Yeah, this is terribly important issue to take care of! Relative
simplicity of the concept itself raises the chances of finding a
suitable solution, I hope.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Phillip Wood]

On 02/03/18 01:16, Igor Djordjevic wrote:
> 
> Hi Sergey,
> 
> On 01/03/2018 06:39, Sergey Organov wrote:
>>
>>>> (3) ---X1---o---o---o---o---o---X2
>>>>        |\                       |\
>>>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>>>        |             \          |
>>>>        |              M         |
>>>>        |             /          |
>>>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>>>>
>>>
>>> Meh, I hope I`m rushing it now, but for example, if we had decided to 
>>> drop commit A2 during an interactive rebase (so losing A2' from 
>>> diagram above), wouldn`t U2' still introduce those changes back, once 
>>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>>
>>> [...]
>>
>> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
>> this more carefully in the first place.
> 
> No problem, that`s why we`re discussing it, and I`m glad we`re 
> aligned now, so we can move forward :)
> 
>>> So while your original proposal currently seems like it could be 
>>> working nicely for non-interactive rebase (and might be some simpler 
>>> interactive ones), now hitting/acknowledging its first real use 
>>> limit, my additional quick attempt[1] just tries to aid pretty 
>>> interesting case of complicated interactive rebase, too, where we 
>>> might be able to do better as well, still using you original proposal 
>>> as a base idea :)
>>
>> Yes, thank you for pushing me back to reality! :-) The work and thoughts
>> you are putting into solving the puzzle are greatly appreciated!
> 
> You`re welcome, and I am enjoying it :)
> 
>> Thinking about it overnight, I now suspect that original proposal had a
>> mistake in the final merge step. I think that what you did is a way to
>> fix it, and I want to try to figure what exactly was wrong in the
>> original proposal and to find simpler way of doing it right.
>>
>> The likely solution is to use original UM as a merge-base for final
>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>> though, as that's exactly UM from which both U1' and U2' have diverged
>> due to rebasing and other history editing.
> 
> Yes, this might be it...! ;)
> 
> To prove myself it works, I`ve assembled a pretty crazy `-s ours`  
> merge interactive rebase scenario, and it seems this passes the test, 
> ticking all the check boxes (I could think of) :P

It is interesting to think what it means to faithfully rebase a '-s
ours' merge. In your example the rebase does not introduce any new
changes into branch B that it doesn't introduce to branch A. Had it
added a fixup to branch B1 for example or if the topology was more
complex so that B ended up with some other changes that the rebase did
not introduce into A, then M' would contain those extra changes whereas
'--recreate-merges' with '-s ours' (once it supports it) would not.

> 
> Let`s see our starting situation:
> 
>  (0) ---X8--B2'--X9 (master)
>         |\
>         | A1---A2---A3 (A)
>         |             \
>         |              M (topic)
>         |             /
>         \-B1---B2---B3 (B)
> 
> 
> Here, merge commit M is done with `-s ours` (obsoleting branch "B"), 
> plus amended to make it an "evil merge", where a commit B2 from 
> obsoleted branch "B" is cherry picked to "master".
> 
> Now, we want to rebase "topic" (M) onto updated "master" (X9), but to 
> make things more interesting, we`ll do it interactively, with some 
> amendments, drops, additions and even more cherry-picks!
> 
> This is what the final result looks like:
> 
>  (1) ---X8--B2'--X9 (master)
>                  |\
>                  | A12--A2'---B3' (A)
>                  |             \
>                  |              M' (topic)
>                  |             /
>                  \-B1'--B3'---B4  (B)
> 
> 
> During interactive rebase, on branch "A", we amended A1 into A12, 
> dropped A3 and cherry-picked B3. On branch "B", B4 is added, B2' being 
> omitted automatically as already present in "master".
> 
> So... In comparison to original merge commit M, rebased merge commit 
> M' is expected to:
> 
>  - Add X9, from updated "master"
>  - Have A1 changed to A12, due to A12 commit amendment
>  - Keep A2, rebased as A2'
>  - Remove A3, due to dropped A3 commit
>  - Keep amendment from original (evil) merge commit M
>  - Miss B1' like M does B, due to original `-s ours` merge strategy
>  - Add B2, cherry-picked as B2' into "master"
>  - Add B3, cherry-picked as B3' into "A"
>  - Add B4, added to "B"
>  - Most important, provide safety mechanism to "fail loud", being 
>    aware of non-trivial things going on, allowing to stop for user 
>    inspection/decision
> 
> 
> There, I hope I didn`t miss any expectation. And, it _seems_ to work 
> exactly as expected :D
> 
> Not to leave this to imagination only, and hopefully helping others 
> to get to speed and possibly discuss this, pointing to still possible 
> flaws, I`m adding a demo script[1], showing how this exact example 
> works.
> 
> Note that script _is_ coined to avoid rebase conflicts, as they`re not 
> currently important for the point to be made here.
> 
> In real life, except for usual possibility for conflicts during 
> commit rebasing, we might experience _three_ possible conflict 
> situations once "rebased" merge itself is to be created - two when 
> rebasing each of temporary merge helper commits, and one on the 
> "rebased" merge itself. This is something where we might think about 
> user experience, not introducing (too much) confusion...
> 
> Regards, Buga
> 
> [1] Demonstration script:
> -- >8 --
> #!/bin/sh
> 
> # rm -rf ./.git
> # rm -f ./test.txt
> 
> git init
> 
> touch ./test.txt
> git add -- test.txt
> 
> # prepare repository
> for i in {1..8}
> do
> 	echo X$i >>test.txt
> 	git commit -am "X$i"
> done
> 
> # prepare branch A
> git checkout -b A
> sed -i '2iA1' test.txt
> git commit -am "A1"
> sed -i '4iA2' test.txt
> git commit -am "A2"
> sed -i '6iA3' test.txt
> git commit -am "A3"
> 
> # prepare branch B
> git checkout -b B master
> sed -i '5iB1' test.txt
> git commit -am "B1"
> sed -i '7iB2' test.txt
> git commit -am "B2"
> sed -i '9iB3' test.txt
> git commit -am "B3"
> 
> git checkout -b topic A
> git merge -s ours --no-commit B # merge A and B with `-s ours`
> sed -i '8iM' test.txt           # amend merge commit ("evil merge")
> git commit -am "M"
> git tag original-merge
> 
> # master moves on...
> git checkout master
> git cherry-pick B^     # cherry-pick B2 into master
> sed -i "1iX9" test.txt # add X9
> git commit -am "X9"
> 
> # (0) ---X8--B2'--X9 (master)
> #        |\
> #        | A1---A2---A3 (A)
> #        |             \
> #        |              M (topic)
> #        |             /
> #        \-B1---B2---B3 (B)
> 
> # simple/naive demonstration of proposed merge rebasing logic
> # using described new approach, preserving merge commit manual
> # amendments, testing `-s ours` merge with cherry-picking from
> # obsoleted part, but still respecting interactively rebased
> # added/modified/dropped/cherry-picked commits :)
> 
> git checkout A
> git cherry-pick -m1 original-merge # prepare temporary helper commit U1
> git tag U1
> git reset --hard HEAD^^            # drop U1 and A3 from A
> sed -i '/A1/c\A12' test.txt        # amend A1 to A12
> git commit -a --amend --no-edit
> git rebase master                  # rebase A onto master
> git cherry-pick B                  # cherry-pick B3 into A
> git cherry-pick U1                 # "rebase" temporary helper commit U1
> git tag U1-prime
> 
> git checkout B
> git cherry-pick -m2 original-merge # prepare temporary helper commit U2
> git tag U2
> git reset --hard HEAD^             # drop U2 from B
> git rebase master                  # rebase B onto master
> sed -i '12iB4' test.txt            # add B4
> git commit -am "B4"
> git cherry-pick U2                 # "rebase" temporary helper commit U2
> git tag U2-prime
> 
> git branch -f topic A
> git checkout topic
> # merge rebased temporary commits U1' and U2',
> # using original merge commit as a merge base,
> # producing "rebased" merge commit M'
> git read-tree -m --aggressive original-merge A B
> git merge-index -o git-merge-one-file -a
> 
> # recognize complex stuff going on during rebasing merge commit,
> # allowing user to inspect result, edit, and continue or abort
> git diff --quiet U1-prime U2-prime
> if test $? -ne 0
> then
> 	# PLACEHOLDER
> 	# chance to inspect result, like:
> 	git diff original-merge
> 	# edit if needed, continue or abort
> fi
> 
> # drop rebased temporary commits U1' and U2'
> git branch -f A A^
> git branch -f B B^
> 
> # record branches A and B as parents of "rebased" merge commit M',
> # updating topic branch
> git update-ref refs/heads/topic "$(git show -s --format=%B original-merge | git commit-tree "$(git write-tree)" -p "$(git rev-parse A)" -p "$(git rev-parse B)")"
> git tag angel-merge
> 
> # (1) ---X8--B2'--X9 (master)
> #                 |\
> #                 | A12--A2'---B3' (A)
> #                 |             \
> #                 |              M' (topic)
> #                 |             /
> #                 \-B1'--B3'---B4  (B)
> 
> # show resulting graph
> # echo
> # git log --all --decorate --oneline --graph
> 
> # in comparison to original merge commit M, rebased merge commit 
> # M' is expected to:
> #
> # - Add X9, from updated "master"
> # - Have A1 changed to A12, due to A12 commit amendment
> # - Keep A2, rebased as A2'
> # - Remove A3, due to dropped A3 commit
> # - Keep amendment from original (evil) merge commit M
> # - Miss B1' like M does B, due to original `-s ours` merge strategy
> # - Add B2, cherry-picked as B2' into "master"
> # - Add B3, cherry-picked as B3' into "A"
> # - Add B4, added to "B"
> #
> # echo
> # echo 'diff original-merge angel-merge:'
> # git diff original-merge angel-merge
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 01/03/18 05:39, Sergey Organov wrote:
> 
> Hi Igor,
> 
> Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:
> 
>> Hi Sergey,
>>
>> On 28/02/2018 06:19, Sergey Organov wrote:
>>>
>>>>> (3) ---X1---o---o---o---o---o---X2
>>>>>        |\                       |\
>>>>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>>>>        |             \          |
>>>>>        |              M         |
>>>>>        |             /          |
>>>>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>>>>>
>>>>
>>>> Meh, I hope I`m rushing it now, but for example, if we had decided to 
>>>> drop commit A2 during an interactive rebase (so losing A2' from 
>>>> diagram above), wouldn`t U2' still introduce those changes back, once 
>>>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>>
>>> I think the method will handle this nicely.
>>
>> That`s what I thought as well. And then I made a test. And then the 
>> test broke... Now, might be the test was flawed in the first place, 
>> but thinking about it a bit more, it does seem to make sense not to 
>> handle this case nicely :/
> 
> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
> this more carefully in the first place.
> 
> [...]
> 
>> So while your original proposal currently seems like it could be 
>> working nicely for non-interactive rebase (and might be some simpler 
>> interactive ones), now hitting/acknowledging its first real use 
>> limit, my additional quick attempt[1] just tries to aid pretty 
>> interesting case of complicated interactive rebase, too, where we 
>> might be able to do better as well, still using you original proposal 
>> as a base idea :)
> 
> Yes, thank you for pushing me back to reality! :-) The work and thoughts
> you are putting into solving the puzzle are greatly appreciated!
> 
> Thinking about it overnight, I now suspect that original proposal had a
> mistake in the final merge step. I think that what you did is a way to
> fix it, and I want to try to figure what exactly was wrong in the
> original proposal and to find simpler way of doing it right.
> 
> The likely solution is to use original UM as a merge-base for final
> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> though, as that's exactly UM from which both U1' and U2' have diverged
> due to rebasing and other history editing.

Hi Sergey, I've been following this discussion from the sidelines,
though I haven't had time to study all the posts in this thread in
detail. I wonder if it would be helpful to think of rebasing a merge as
merging the changes in the parents due to the rebase back into the
original merge. So for a merge M with parents A B C that are rebased to
A' B' C' the rebased merge M' would be constructed by (ignoring shell
quoting issues)

git checkout --detach M
git merge-recursive A -- M A'
tree=$(git write-tree)
git merge-recursive B -- $tree B'
tree=$(git write-tree)
git merge-recursive C -- $tree C'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

This should pull in all the changes from the parents while preserving
any evil conflict resolution in the original merge. It superficially
reminds me of incremental merging [1] but it's so long since I looked at
that I'm not sure if there are any significant similarities.

Best Wishes

Phillip

[1] https://github.com/mhagger/git-imerge

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Phillip Wood]

On 02/03/18 11:17, Phillip Wood wrote:
> 
> On 02/03/18 01:16, Igor Djordjevic wrote:
>>
>> Hi Sergey,
>>
>> On 01/03/2018 06:39, Sergey Organov wrote:
>>>
>>>>> (3) ---X1---o---o---o---o---o---X2
>>>>>        |\                       |\
>>>>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>>>>        |             \          |
>>>>>        |              M         |
>>>>>        |             /          |
>>>>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>>>>>
>>>>
>>>> Meh, I hope I`m rushing it now, but for example, if we had decided to 
>>>> drop commit A2 during an interactive rebase (so losing A2' from 
>>>> diagram above), wouldn`t U2' still introduce those changes back, once 
>>>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>>>
>>>> [...]
>>>
>>> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
>>> this more carefully in the first place.
>>
>> No problem, that`s why we`re discussing it, and I`m glad we`re 
>> aligned now, so we can move forward :)
>>
>>>> So while your original proposal currently seems like it could be 
>>>> working nicely for non-interactive rebase (and might be some simpler 
>>>> interactive ones), now hitting/acknowledging its first real use 
>>>> limit, my additional quick attempt[1] just tries to aid pretty 
>>>> interesting case of complicated interactive rebase, too, where we 
>>>> might be able to do better as well, still using you original proposal 
>>>> as a base idea :)
>>>
>>> Yes, thank you for pushing me back to reality! :-) The work and thoughts
>>> you are putting into solving the puzzle are greatly appreciated!
>>
>> You`re welcome, and I am enjoying it :)
>>
>>> Thinking about it overnight, I now suspect that original proposal had a
>>> mistake in the final merge step. I think that what you did is a way to
>>> fix it, and I want to try to figure what exactly was wrong in the
>>> original proposal and to find simpler way of doing it right.
>>>
>>> The likely solution is to use original UM as a merge-base for final
>>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>>> though, as that's exactly UM from which both U1' and U2' have diverged
>>> due to rebasing and other history editing.
>>
>> Yes, this might be it...! ;)
>>
>> To prove myself it works, I`ve assembled a pretty crazy `-s ours`  
>> merge interactive rebase scenario, and it seems this passes the test, 
>> ticking all the check boxes (I could think of) :P

Hi Igor

> It is interesting to think what it means to faithfully rebase a '-s
> ours' merge.
I should have explained that I mean is a faithful rebase one that
adheres to the semantics of '-s ours' (i.e. ignores any changes in the
side branch) or one that merges new changes from the side branch into
the content of the original merge? In your example you add B4 to B. If
M' preserves the semantics of '-s ours' then it will not contain the
changes in B4. I think your result does (correct me if I'm wrong) so it
is preserving the content of the original merge rather than the
semantics of it.

Best Wishes

Phillip

(ignore the rest of what I wrote earlier I don't think it's correct)
 In your example the rebase does not introduce any new
> changes into branch B that it doesn't introduce to branch A. Had it
> added a fixup to branch B1 for example or if the topology was more
> complex so that B ended up with some other changes that the rebase did
> not introduce into A, then M' would contain those extra changes whereas
> '--recreate-merges' with '-s ours' (once it supports it) would not.
> 
>>
>> Let`s see our starting situation:
>>
>>  (0) ---X8--B2'--X9 (master)
>>         |\
>>         | A1---A2---A3 (A)
>>         |             \
>>         |              M (topic)
>>         |             /
>>         \-B1---B2---B3 (B)
>>
>>
>> Here, merge commit M is done with `-s ours` (obsoleting branch "B"), 
>> plus amended to make it an "evil merge", where a commit B2 from 
>> obsoleted branch "B" is cherry picked to "master".
>>
>> Now, we want to rebase "topic" (M) onto updated "master" (X9), but to 
>> make things more interesting, we`ll do it interactively, with some 
>> amendments, drops, additions and even more cherry-picks!
>>
>> This is what the final result looks like:
>>
>>  (1) ---X8--B2'--X9 (master)
>>                  |\
>>                  | A12--A2'---B3' (A)
>>                  |             \
>>                  |              M' (topic)
>>                  |             /
>>                  \-B1'--B3'---B4  (B)
>>
>>
>> During interactive rebase, on branch "A", we amended A1 into A12, 
>> dropped A3 and cherry-picked B3. On branch "B", B4 is added, B2' being 
>> omitted automatically as already present in "master".
>>
>> So... In comparison to original merge commit M, rebased merge commit 
>> M' is expected to:
>>
>>  - Add X9, from updated "master"
>>  - Have A1 changed to A12, due to A12 commit amendment
>>  - Keep A2, rebased as A2'
>>  - Remove A3, due to dropped A3 commit
>>  - Keep amendment from original (evil) merge commit M
>>  - Miss B1' like M does B, due to original `-s ours` merge strategy
>>  - Add B2, cherry-picked as B2' into "master"
>>  - Add B3, cherry-picked as B3' into "A"
>>  - Add B4, added to "B"
>>  - Most important, provide safety mechanism to "fail loud", being 
>>    aware of non-trivial things going on, allowing to stop for user 
>>    inspection/decision
>>
>>
>> There, I hope I didn`t miss any expectation. And, it _seems_ to work 
>> exactly as expected :D
>>
>> Not to leave this to imagination only, and hopefully helping others 
>> to get to speed and possibly discuss this, pointing to still possible 
>> flaws, I`m adding a demo script[1], showing how this exact example 
>> works.
>>
>> Note that script _is_ coined to avoid rebase conflicts, as they`re not 
>> currently important for the point to be made here.
>>
>> In real life, except for usual possibility for conflicts during 
>> commit rebasing, we might experience _three_ possible conflict 
>> situations once "rebased" merge itself is to be created - two when 
>> rebasing each of temporary merge helper commits, and one on the 
>> "rebased" merge itself. This is something where we might think about 
>> user experience, not introducing (too much) confusion...
>>
>> Regards, Buga
>>
>> [1] Demonstration script:
>> -- >8 --
>> #!/bin/sh
>>
>> # rm -rf ./.git
>> # rm -f ./test.txt
>>
>> git init
>>
>> touch ./test.txt
>> git add -- test.txt
>>
>> # prepare repository
>> for i in {1..8}
>> do
>> 	echo X$i >>test.txt
>> 	git commit -am "X$i"
>> done
>>
>> # prepare branch A
>> git checkout -b A
>> sed -i '2iA1' test.txt
>> git commit -am "A1"
>> sed -i '4iA2' test.txt
>> git commit -am "A2"
>> sed -i '6iA3' test.txt
>> git commit -am "A3"
>>
>> # prepare branch B
>> git checkout -b B master
>> sed -i '5iB1' test.txt
>> git commit -am "B1"
>> sed -i '7iB2' test.txt
>> git commit -am "B2"
>> sed -i '9iB3' test.txt
>> git commit -am "B3"
>>
>> git checkout -b topic A
>> git merge -s ours --no-commit B # merge A and B with `-s ours`
>> sed -i '8iM' test.txt           # amend merge commit ("evil merge")
>> git commit -am "M"
>> git tag original-merge
>>
>> # master moves on...
>> git checkout master
>> git cherry-pick B^     # cherry-pick B2 into master
>> sed -i "1iX9" test.txt # add X9
>> git commit -am "X9"
>>
>> # (0) ---X8--B2'--X9 (master)
>> #        |\
>> #        | A1---A2---A3 (A)
>> #        |             \
>> #        |              M (topic)
>> #        |             /
>> #        \-B1---B2---B3 (B)
>>
>> # simple/naive demonstration of proposed merge rebasing logic
>> # using described new approach, preserving merge commit manual
>> # amendments, testing `-s ours` merge with cherry-picking from
>> # obsoleted part, but still respecting interactively rebased
>> # added/modified/dropped/cherry-picked commits :)
>>
>> git checkout A
>> git cherry-pick -m1 original-merge # prepare temporary helper commit U1
>> git tag U1
>> git reset --hard HEAD^^            # drop U1 and A3 from A
>> sed -i '/A1/c\A12' test.txt        # amend A1 to A12
>> git commit -a --amend --no-edit
>> git rebase master                  # rebase A onto master
>> git cherry-pick B                  # cherry-pick B3 into A
>> git cherry-pick U1                 # "rebase" temporary helper commit U1
>> git tag U1-prime
>>
>> git checkout B
>> git cherry-pick -m2 original-merge # prepare temporary helper commit U2
>> git tag U2
>> git reset --hard HEAD^             # drop U2 from B
>> git rebase master                  # rebase B onto master
>> sed -i '12iB4' test.txt            # add B4
>> git commit -am "B4"
>> git cherry-pick U2                 # "rebase" temporary helper commit U2
>> git tag U2-prime
>>
>> git branch -f topic A
>> git checkout topic
>> # merge rebased temporary commits U1' and U2',
>> # using original merge commit as a merge base,
>> # producing "rebased" merge commit M'
>> git read-tree -m --aggressive original-merge A B
>> git merge-index -o git-merge-one-file -a
>>
>> # recognize complex stuff going on during rebasing merge commit,
>> # allowing user to inspect result, edit, and continue or abort
>> git diff --quiet U1-prime U2-prime
>> if test $? -ne 0
>> then
>> 	# PLACEHOLDER
>> 	# chance to inspect result, like:
>> 	git diff original-merge
>> 	# edit if needed, continue or abort
>> fi
>>
>> # drop rebased temporary commits U1' and U2'
>> git branch -f A A^
>> git branch -f B B^
>>
>> # record branches A and B as parents of "rebased" merge commit M',
>> # updating topic branch
>> git update-ref refs/heads/topic "$(git show -s --format=%B original-merge | git commit-tree "$(git write-tree)" -p "$(git rev-parse A)" -p "$(git rev-parse B)")"
>> git tag angel-merge
>>
>> # (1) ---X8--B2'--X9 (master)
>> #                 |\
>> #                 | A12--A2'---B3' (A)
>> #                 |             \
>> #                 |              M' (topic)
>> #                 |             /
>> #                 \-B1'--B3'---B4  (B)
>>
>> # show resulting graph
>> # echo
>> # git log --all --decorate --oneline --graph
>>
>> # in comparison to original merge commit M, rebased merge commit 
>> # M' is expected to:
>> #
>> # - Add X9, from updated "master"
>> # - Have A1 changed to A12, due to A12 commit amendment
>> # - Keep A2, rebased as A2'
>> # - Remove A3, due to dropped A3 commit
>> # - Keep amendment from original (evil) merge commit M
>> # - Miss B1' like M does B, due to original `-s ours` merge strategy
>> # - Add B2, cherry-picked as B2' into "master"
>> # - Add B3, cherry-picked as B3' into "A"
>> # - Add B4, added to "B"
>> #
>> # echo
>> # echo 'diff original-merge angel-merge:'
>> # git diff original-merge angel-merge
>>
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Jacob Keller]

On Fri, Mar 2, 2018 at 3:17 AM, Phillip Wood <phil@philandanna.no-ip.org> wrote:
>
> It is interesting to think what it means to faithfully rebase a '-s
> ours' merge. In your example the rebase does not introduce any new
> changes into branch B that it doesn't introduce to branch A. Had it
> added a fixup to branch B1 for example or if the topology was more
> complex so that B ended up with some other changes that the rebase did
> not introduce into A, then M' would contain those extra changes whereas
> '--recreate-merges' with '-s ours' (once it supports it) would not.
>

Unless the method of merging was stored, I don't think we *can*
correctly automate resolving of "-s ours" because all we store is the
resulting content, and we don't know how or why the user generated it
as such. I believe the "correct" solution in any case would be to take
the content we DO know and then ask the user to stop for amendments.

Thanks,
Jake

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Jacob Keller]

On Fri, Mar 2, 2018 at 4:36 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> On 02/03/18 11:17, Phillip Wood wrote:
>>
>> On 02/03/18 01:16, Igor Djordjevic wrote:
>>>
>>> Hi Sergey,
>>>
>>> On 01/03/2018 06:39, Sergey Organov wrote:
>>>>
>>>>>> (3) ---X1---o---o---o---o---o---X2
>>>>>>        |\                       |\
>>>>>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
>>>>>>        |             \          |
>>>>>>        |              M         |
>>>>>>        |             /          |
>>>>>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
>>>>>>
>>>>>
>>>>> Meh, I hope I`m rushing it now, but for example, if we had decided to
>>>>> drop commit A2 during an interactive rebase (so losing A2' from
>>>>> diagram above), wouldn`t U2' still introduce those changes back, once
>>>>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
>>>>>
>>>>> [...]
>>>>
>>>> Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
>>>> this more carefully in the first place.
>>>
>>> No problem, that`s why we`re discussing it, and I`m glad we`re
>>> aligned now, so we can move forward :)
>>>
>>>>> So while your original proposal currently seems like it could be
>>>>> working nicely for non-interactive rebase (and might be some simpler
>>>>> interactive ones), now hitting/acknowledging its first real use
>>>>> limit, my additional quick attempt[1] just tries to aid pretty
>>>>> interesting case of complicated interactive rebase, too, where we
>>>>> might be able to do better as well, still using you original proposal
>>>>> as a base idea :)
>>>>
>>>> Yes, thank you for pushing me back to reality! :-) The work and thoughts
>>>> you are putting into solving the puzzle are greatly appreciated!
>>>
>>> You`re welcome, and I am enjoying it :)
>>>
>>>> Thinking about it overnight, I now suspect that original proposal had a
>>>> mistake in the final merge step. I think that what you did is a way to
>>>> fix it, and I want to try to figure what exactly was wrong in the
>>>> original proposal and to find simpler way of doing it right.
>>>>
>>>> The likely solution is to use original UM as a merge-base for final
>>>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>>>> though, as that's exactly UM from which both U1' and U2' have diverged
>>>> due to rebasing and other history editing.
>>>
>>> Yes, this might be it...! ;)
>>>
>>> To prove myself it works, I`ve assembled a pretty crazy `-s ours`
>>> merge interactive rebase scenario, and it seems this passes the test,
>>> ticking all the check boxes (I could think of) :P
>
> Hi Igor
>
>> It is interesting to think what it means to faithfully rebase a '-s
>> ours' merge.
> I should have explained that I mean is a faithful rebase one that
> adheres to the semantics of '-s ours' (i.e. ignores any changes in the
> side branch) or one that merges new changes from the side branch into
> the content of the original merge? In your example you add B4 to B. If
> M' preserves the semantics of '-s ours' then it will not contain the
> changes in B4. I think your result does (correct me if I'm wrong) so it
> is preserving the content of the original merge rather than the
> semantics of it.
>
> Best Wishes
>
> Phillip
>

I believe this was always the outline of this type of approach, as per
Sergey's original email.

We only have the content, and we don't know the semantics (nor, I
think, should we attempt to understand or figure out the semantics).

Regards,
Jake

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 02/03/2018 06:40, Sergey Organov wrote:
> 
> > So... In comparison to original merge commit M, rebased merge commit 
> > M' is expected to:
> > 
> >  - Add X9, from updated "master"
> >  - Have A1 changed to A12, due to A12 commit amendment
> >  - Keep A2, rebased as A2'
> >  - Remove A3, due to dropped A3 commit
> >  - Keep amendment from original (evil) merge commit M
> >  - Miss B1' like M does B, due to original `-s ours` merge strategy
> >  - Add B2, cherry-picked as B2' into "master"
> >  - Add B3, cherry-picked as B3' into "A"
> >  - Add B4, added to "B"
> >  - Most important, provide safety mechanism to "fail loud", being 
> >    aware of non-trivial things going on, allowing to stop for user 
> >    inspection/decision
> > 
> > 
> > There, I hope I didn`t miss any expectation. And, it _seems_ to work 
> > exactly as expected :D
> 
> That's very nice, to the level of being even suspect! :-)

Heh, indeed :) I`m already thinking of some even more complex 
situations. The more use/test cases, the merrier.

Like if number of merge parents (branches) gets changed during 
interactive rebase...

> To avoid falling into euphoria though, we need to keep in mind that
> "expectations" is rather vague concept, and we likely still need to stop
> for user amendment unless we absolutely sure nothing surprising happens.
> I.e., we better require U1'==U2' test to succeed to proceed non-stop
> automatically. Besides, it will be somewhat inline with what 'rerere'
> does.

I totally agree, and I think whatever we come up with, we`ll always 
be missing some deeper context of the original merge, so U1'==U2' 
test is a must - if it fails, even if we didn`t get any conflicts and 
could otherwise proceed automatically, better stop for user sanity check.

I`m still thinking if there could be a situation where test passes, 
but result might still be suspicious (and worth inspecting), though.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Phillip,

On 02/03/2018 17:00, Jacob Keller wrote:
> 
> > It is interesting to think what it means to faithfully rebase a '-s
> > ours' merge. In your example the rebase does not introduce any new
> > changes into branch B that it doesn't introduce to branch A. Had it
> > added a fixup to branch B1 for example or if the topology was more
> > complex so that B ended up with some other changes that the rebase did
> > not introduce into A, then M' would contain those extra changes whereas
> > '--recreate-merges' with '-s ours' (once it supports it) would not.
> 
> Unless the method of merging was stored, I don't think we *can*
> correctly automate resolving of "-s ours" because all we store is the
> resulting content, and we don't know how or why the user generated it
> as such. I believe the "correct" solution in any case would be to take
> the content we DO know and then ask the user to stop for amendments.
 
I agree with Jake, and for the exact same reasons.

That said, I`d like to see what mentioned '--recreate-merges' with 
'-s ours' does (or would do, once it supports it), would you have a 
pointer for me where to look at?

But if that`s something yet to come, might be it`s still open for 
discussion. I mean, even this topic started inside original 
`--recreate-merges` one[1], and hopefully it can still bring 
improvements there (sooner or later).

Thanks, Buga

[1] https://public-inbox.org/git/cover.1516225925.git.johannes.schindelin@gmx.de/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Phillip,

> On Fri, Mar 2, 2018 at 4:36 AM, Phillip Wood wrote:
> > 
> > > It is interesting to think what it means to faithfully rebase a '-s
> > > ours' merge.
> > 
> > I should have explained that I mean is a faithful rebase one that
> > adheres to the semantics of '-s ours' (i.e. ignores any changes in the
> > side branch) or one that merges new changes from the side branch into
> > the content of the original merge? In your example you add B4 to B. If
> > M' preserves the semantics of '-s ours' then it will not contain the
> > changes in B4. I think your result does (correct me if I'm wrong) so it
> > is preserving the content of the original merge rather than the
> > semantics of it.

Yeah, I understood what you mean, and I see you noticed that B4 
commit, for which I did anticipate possibly bringing up a discussion 
like this ;)

I agree with Jake here, my thoughts exactly (what I wrote in that 
other subthread[1], too):

On 02/03/2018 17:02, Jacob Keller wrote:
> 
> We only have the content, and we don't know the semantics (nor, I
> think, should we attempt to understand or figure out the semantics).

Hmm, I wanted to elaborate a bit here, but that sentence seems to 
summarize the pure essence of it, and whatever I write looks like 
just repeating the same stuff again...

That`s just it. And stopping to give the user a chance to 
review/amend the result, where he might decide he actually did want 
something else - so all good.

Otherwise, I would be interested to learn how context/semantics 
guessing could provide a better default action (without introducing 
more complexity for might not be much benefit, if any).

But in the end, I guess we can just discuss the "most sane default" 
to present to the user (introduce or obsolete that new commit B4, in 
the discussed example[2]), as we should definitely stop for amending 
anyway, not proceeding automatically whenever U1' != U2'.

Oh, and what about situations where we introduce new or drop existing 
branches (which should be possible with new `--recreate-merges`)...? 
"Preserving old branch semantics" may have even less sense here - the 
(possibly heavily reorganized) content is the only thing we have, 
where context will (and should) be provided by the user.

And I guess being consistent is pretty important, too - if you add 
new content during merge rebase, it should always show up in the 
merge, period. It seems pretty confusing to find out one of the 
branches "declared special" (even more if it`s based on uncertain 
guess-work), so when you add something to it it`s just "swallowed", 
as the whole branch is always obsoleted, for now and ever.

I might even see a value in such behavior, but only as a conscious 
user action, not something done automatically... I guess? :)

Regards, Buga

[1] https://public-inbox.org/git/f26cdbe2-1bc3-02ff-7b99-12a6ebab5a70@gmail.com/
[2] https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e5516655b3@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Phillip,

On 02/03/2018 12:31, Phillip Wood wrote:
> 
> > Thinking about it overnight, I now suspect that original proposal had a
> > mistake in the final merge step. I think that what you did is a way to
> > fix it, and I want to try to figure what exactly was wrong in the
> > original proposal and to find simpler way of doing it right.
> >
> > The likely solution is to use original UM as a merge-base for final
> > 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> > though, as that's exactly UM from which both U1' and U2' have diverged
> > due to rebasing and other history editing.
> 
> Hi Sergey, I've been following this discussion from the sidelines,
> though I haven't had time to study all the posts in this thread in
> detail. I wonder if it would be helpful to think of rebasing a merge as
> merging the changes in the parents due to the rebase back into the
> original merge. So for a merge M with parents A B C that are rebased to
> A' B' C' the rebased merge M' would be constructed by (ignoring shell
> quoting issues)
> 
> git checkout --detach M
> git merge-recursive A -- M A'
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> git merge-recursive C -- $tree C'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> 
> This should pull in all the changes from the parents while preserving
> any evil conflict resolution in the original merge. It superficially
> reminds me of incremental merging [1] but it's so long since I looked at
> that I'm not sure if there are any significant similarities.
> 
> [1] https://github.com/mhagger/git-imerge

Interesting, from quick test[3], this seems to produce the same 
result as that other test I previously provided[2], where temporary 
commits U1' and U2' are finally merged with original M as a base :)

Just that this looks like even more straight-forward approach...?

The only thing I wonder of here is how would we check if the 
"rebased" merge M' was "clean", or should we stop for user amendment? 
With that other approach Sergey described, we have U1'==U2' to test with.

By the way, is there documentation for `git merge-recursive` 
anywhere, besides the code itself...? :$

Thanks, Buga

[2] https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e5516655b3@gmail.com/
[3] Quick test script:
-- >8 --
#!/bin/sh

# rm -rf ./.git
# rm -f ./test.txt

git init

touch ./test.txt
git add -- test.txt

# prepare repository
for i in {1..8}
do
	echo X$i >>test.txt
	git commit -am "X$i"
done

# prepare branch A
git checkout -b A
sed -i '2iA1' test.txt
git commit -am "A1"
sed -i '4iA2' test.txt
git commit -am "A2"
sed -i '6iA3' test.txt
git commit -am "A3"

# prepare branch B
git checkout -b B master
sed -i '5iB1' test.txt
git commit -am "B1"
sed -i '7iB2' test.txt
git commit -am "B2"
sed -i '9iB3' test.txt
git commit -am "B3"

git checkout -b topic A
git merge -s ours --no-commit B # merge A and B with `-s ours`
sed -i '8iM' test.txt           # amend merge commit ("evil merge")
git commit -am "M"
git tag M #original-merge

# master moves on...
git checkout master
git cherry-pick B^     # cherry-pick B2 into master
sed -i "1iX9" test.txt # add X9
git commit -am "X9"

# (0) ---X8--B2'--X9 (master)
#        |\
#        | A1---A2---A3 (A)
#        |             \
#        |              M (topic)
#        |             /
#        \-B1---B2---B3 (B)

# simple/naive demonstration of proposed merge rebasing logic
# using iterative merge-recursive, preserving merge commit manual
# amendments, testing `-s ours` merge with cherry-picking from
# obsoleted part, but still respecting interactively rebased
# added/modified/dropped/cherry-picked commits :)

git checkout -b A-prime A
git reset --hard HEAD^             # drop A3 from A
sed -i '/A1/c\A12' test.txt        # amend A1 to A12
git commit -a --amend --no-edit
git rebase master                  # rebase A onto master
git cherry-pick B                  # cherry-pick B3 into A

git checkout -b B-prime B
git rebase master                  # rebase B onto master
sed -i '12iB4' test.txt            # add B4
git commit -am "B4"

git checkout --detach M
git merge-recursive A -- M A-prime
tree="$(git write-tree)"
git merge-recursive B -- $tree B-prime
tree="$(git write-tree)"
git tag M-prime "$(git log --pretty=%B -1 M | git commit-tree $tree -p A-prime -p B-prime)"

git update-ref refs/heads/topic "$(git rev-parse M-prime)"
git checkout topic

# (1) ---X8--B2'--X9 (master)
#                 |\
#                 | A12--A2'---B3' (A)
#                 |             \
#                 |              M' (topic)
#                 |             /
#                 \-B1'--B3'---B4  (B)

# show resulting graph
# echo
# git log --all --decorate --oneline --graph

# in comparison to original merge commit M, rebased merge commit 
# M' is expected to:
#
# - Add X9, from updated "master"
# - Have A1 changed to A12, due to A12 commit amendment
# - Keep A2, rebased as A2'
# - Remove A3, due to dropped A3 commit
# - Keep amendment from original (evil) merge commit M
# - Miss B1' like M does B, due to original `-s ours` merge strategy
# - Add B2, cherry-picked as B2' into "master"
# - Add B3, cherry-picked as B3' into "A"
# - Add B4, added to "B"
#
# echo
# echo 'diff M M-prime:'
# git diff M M-prime

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 02/03/2018 19:14, Igor Djordjevic wrote:
> 
> > > It is interesting to think what it means to faithfully rebase a '-s
> > > ours' merge. In your example the rebase does not introduce any new
> > > changes into branch B that it doesn't introduce to branch A. Had it
> > > added a fixup to branch B1 for example or if the topology was more
> > > complex so that B ended up with some other changes that the rebase did
> > > not introduce into A, then M' would contain those extra changes whereas
> > > '--recreate-merges' with '-s ours' (once it supports it) would not.
> >
> > Unless the method of merging was stored, I don't think we *can*
> > correctly automate resolving of "-s ours" because all we store is the
> > resulting content, and we don't know how or why the user generated it
> > as such. I believe the "correct" solution in any case would be to take
> > the content we DO know and then ask the user to stop for amendments.
>  
> I agree with Jake, and for the exact same reasons.
> 
> That said, I`d like to see what mentioned '--recreate-merges' with 
> '-s ours' does (or would do, once it supports it), would you have a 
> pointer for me where to look at?
> 
> But if that`s something yet to come, might be it`s still open for 
> discussion. I mean, even this topic started inside original 
> `--recreate-merges` one[1], and hopefully it can still bring 
> improvements there (sooner or later).
> 
> [1] https://public-inbox.org/git/cover.1516225925.git.johannes.schindelin@gmx.de/

Ok, I went through mentioned `--recreate-merges` topic again, and I 
think I see one crucial merge commit handling difference.

In there, as it is at the moment, merge commits are really to be 
_recreated_... as the option name seems to imply ;)

In terms of interactive rebasing, it actually comes from "todo list" 
becoming much more powerful, gaining ability to create (new) merges, 
which is wonderful from the aspect of history rewriting (what rebase 
is all about).

But, I would argue it is a different concept from actually _rebasing_ 
existing merge commits, being what we`re discussing about here.

Yes, you can use merge commit (re)creation to "rebase" existing merge 
commit so the end result is the same, being what `--recreate-merges` 
now does, but that only goes for some (uninteresting?) merge commits 
where not knowing the deeper context of how the merge commit is 
originally created is not important as no content is to be lost (due 
to missing that deeper and utterly unknown context).

But as soon as you try to do that with more complex merge commits, 
like holding manual amendments and conflict resolutions, it doesn`t 
really work as expected - which I demonstrated in my original example 
script[1] in this topic, original merge commit amendment lost on 
rebase, and even worse - that happened silently.

Now, not to get misinterpreted to pick sides in "(re)create" vs 
"rebase" merge commit discussion, I just think these two (should) have 
a different purpose, and actually having both inside interactive rebase 
is what we should be aiming for.

And that`s what I think is important to understand before any further 
discussion - _(re)creating_ existing merge commits is not the same as 
_rebasing_ them, even though the former can sometimes be used to 
achieve the latter.

Regards, Buga

[1] https://public-inbox.org/git/bbe64321-4d3a-d3fe-8bb9-58b600fabf35@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Plillip and Igor,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:
> Hi Phillip,
>
> On 02/03/2018 12:31, Phillip Wood wrote:
>> 
>> > Thinking about it overnight, I now suspect that original proposal had a
>> > mistake in the final merge step. I think that what you did is a way to
>> > fix it, and I want to try to figure what exactly was wrong in the
>> > original proposal and to find simpler way of doing it right.
>> >
>> > The likely solution is to use original UM as a merge-base for final
>> > 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>> > though, as that's exactly UM from which both U1' and U2' have diverged
>> > due to rebasing and other history editing.
>> 
>> Hi Sergey, I've been following this discussion from the sidelines,
>> though I haven't had time to study all the posts in this thread in
>> detail. I wonder if it would be helpful to think of rebasing a merge as
>> merging the changes in the parents due to the rebase back into the
>> original merge. So for a merge M with parents A B C that are rebased to
>> A' B' C' the rebased merge M' would be constructed by (ignoring shell
>> quoting issues)
>> 
>> git checkout --detach M
>> git merge-recursive A -- M A'
>> tree=$(git write-tree)
>> git merge-recursive B -- $tree B'
>> tree=$(git write-tree)
>> git merge-recursive C -- $tree C'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> 
>> This should pull in all the changes from the parents while preserving
>> any evil conflict resolution in the original merge. It superficially
>> reminds me of incremental merging [1] but it's so long since I looked at
>> that I'm not sure if there are any significant similarities.
>> 
>> [1] https://github.com/mhagger/git-imerge
>
> Interesting, from quick test[3], this seems to produce the same 
> result as that other test I previously provided[2], where temporary 
> commits U1' and U2' are finally merged with original M as a base :)

Looks like sound approach and it's interesting if these 2 methods do in
fact always bring the same result. Because if we look at the (now fixed)
original approach closely, it also just gathers the changes in merge
parents into U1' and U2', then merges the changes back into the original
M (=U1=U2=UM).

Overall, this one looks like another implementation of essentially the
same method and confirms that we all have the right thought direction
here.

>
> Just that this looks like even more straight-forward approach...?
>
> The only thing I wonder of here is how would we check if the 
> "rebased" merge M' was "clean", or should we stop for user amendment? 
> With that other approach Sergey described, we have U1'==U2' to test
> with.

That's an advantage of the original, yes.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Igor,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

[...]

> Now, not to get misinterpreted to pick sides in "(re)create" vs 
> "rebase" merge commit discussion, I just think these two (should) have 
> a different purpose, and actually having both inside interactive rebase 
> is what we should be aiming for.

Yes, if the user has an existing merge that he intends to throw away by
re-merging from scratch, he should be given a way to do it during
history editing session, no argues.

What I argue against is that this mode of operation is the default one,
let alone the only one.

> And that`s what I think is important to understand before any further 
> discussion - _(re)creating_ existing merge commits is not the same as 
> _rebasing_ them, even though the former can sometimes be used to 
> achieve the latter.

Yes, indeed. Sometimes creating new merge instead of original does the
job of rebasing the original, only it does it by pure accident.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Sat, 3 Mar 2018, Igor Djordjevic wrote:

> By the way, is there documentation for `git merge-recursive` 
> anywhere, besides the code itself...? :$

I am not aware of any. The commit message adding the command is not very
illuminating (https://github.com/git-for-windows/git/commit/720d150c4):

    Add a new merge strategy by Fredrik Kuivinen.

    I really wanted to try this out, instead of asking for an adjustment
    to the 'git merge' driver and waiting.  For now the new strategy is
    called 'fredrik' and not in the list of default strategies to be tried.

    The script wants Python 2.4 so this commit also adjusts Debian and RPM
    build procecure files.

Digging through https://public-inbox.org/git/ during that time frame comes
up with this hit, though:

https://public-inbox.org/git/20050907164734.GA20198@c165.ib.student.liu.se/

which is still not a good documentation of the algorithm. You can probably
dig further yourself, but I think I can describe it very quickly here:

To merge two commits recursively, you first have to find their "merge
bases". If there was an obvious branch point, then that is the merge base.
But when you start a branch off of master, then work a bit, then merge
master, you already have two merge bases.

The trick about the recursive merge is to reduce the number of merge bases
iteratively to one. It does that by taking two merge bases, and performing
a recursive merge on them, which generates a "virtual" commit, the
condensed merge base. That one is then merged recursively with the next
merge base, until there is only one left.

A recursive merge of two commits with exactly one merge base is simply a
three-way merge.

I vaguely remember that there was something funny about the order in which
order you want to process the merge bases: if you did it in one
(chronological) direction, it worked beautifully, in the other direction
it would generate tons of merge conflicts or something like that.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Phillip,

On Fri, 2 Mar 2018, Phillip Wood wrote:

> On 01/03/18 05:39, Sergey Organov wrote:
> > 
> > Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:
> > 
> >> On 28/02/2018 06:19, Sergey Organov wrote:
> >>>
> >>>>> (3) ---X1---o---o---o---o---o---X2
> >>>>>        |\                       |\
> >>>>>        | A1---A2---A3---U1      | A1'--A2'--A3'--U1'
> >>>>>        |             \          |
> >>>>>        |              M         |
> >>>>>        |             /          |
> >>>>>        \-B1---B2---B3---U2      \-B1'--B2'--B3'--U2'
> >>>>>
> >>>>
> >>>> Meh, I hope I`m rushing it now, but for example, if we had decided to 
> >>>> drop commit A2 during an interactive rebase (so losing A2' from 
> >>>> diagram above), wouldn`t U2' still introduce those changes back, once 
> >>>> U1' and U2' are merged, being incorrect/unwanted behavior...? :/
> >>>
> >>> I think the method will handle this nicely.
> >>
> >> That`s what I thought as well. And then I made a test. And then the 
> >> test broke... Now, might be the test was flawed in the first place, 
> >> but thinking about it a bit more, it does seem to make sense not to 
> >> handle this case nicely :/
> > 
> > Yeah, I now see it myself. I'm sorry for being lazy and not inspecting
> > this more carefully in the first place.
> > 
> > [...]
> > 
> >> So while your original proposal currently seems like it could be 
> >> working nicely for non-interactive rebase (and might be some simpler 
> >> interactive ones), now hitting/acknowledging its first real use 
> >> limit, my additional quick attempt[1] just tries to aid pretty 
> >> interesting case of complicated interactive rebase, too, where we 
> >> might be able to do better as well, still using you original proposal 
> >> as a base idea :)
> > 
> > Yes, thank you for pushing me back to reality! :-) The work and thoughts
> > you are putting into solving the puzzle are greatly appreciated!
> > 
> > Thinking about it overnight, I now suspect that original proposal had a
> > mistake in the final merge step. I think that what you did is a way to
> > fix it, and I want to try to figure what exactly was wrong in the
> > original proposal and to find simpler way of doing it right.
> > 
> > The likely solution is to use original UM as a merge-base for final
> > 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
> > though, as that's exactly UM from which both U1' and U2' have diverged
> > due to rebasing and other history editing.
> 
> Hi Sergey, I've been following this discussion from the sidelines,
> though I haven't had time to study all the posts in this thread in
> detail. I wonder if it would be helpful to think of rebasing a merge as
> merging the changes in the parents due to the rebase back into the
> original merge. So for a merge M with parents A B C that are rebased to
> A' B' C' the rebased merge M' would be constructed by (ignoring shell
> quoting issues)
> 
> git checkout --detach M
> git merge-recursive A -- M A'
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> git merge-recursive C -- $tree C'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

(The $tree obviously wants to be passed as parameter to commit-tree, but
it's easy to get the idea.)

> This should pull in all the changes from the parents while preserving
> any evil conflict resolution in the original merge. It superficially
> reminds me of incremental merging [1] but it's so long since I looked at
> that I'm not sure if there are any significant similarities.

Interesting.

Basically, this is pretending that A'/B'/C' were not the result of
rebases, but of merges between A/B/C and upstream, and then performing an
octopus merge of M, A', B' and C'.

It is a beautiful application of the duality between merges and rebases:
ideally, they both result in the same tree [*1*].

That is a pretty clean and simple-to-describe paradigm to work off of, and
to reason about.

Ciao,
Dscho

Footnote *1*: I frequently use that duality in work to rebase "clean"
patches on top of upstream, and use that rebased branch to figure out how
to resolve merge conflicts when merging upstream into a long-running
branch (whose tree is identical to the pre-rebase version of the clean
patches). And yes, I *think* that this is essentially Michael Haggerty's
`git-imerge` idea.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Phillip Wood]

On 02/03/18 23:33, Igor Djordjevic wrote:
> Hi Phillip,
> 
>> On Fri, Mar 2, 2018 at 4:36 AM, Phillip Wood wrote:
>>>
>>>> It is interesting to think what it means to faithfully rebase a '-s
>>>> ours' merge.
>>>
>>> I should have explained that I mean is a faithful rebase one that
>>> adheres to the semantics of '-s ours' (i.e. ignores any changes in the
>>> side branch) or one that merges new changes from the side branch into
>>> the content of the original merge? In your example you add B4 to B. If
>>> M' preserves the semantics of '-s ours' then it will not contain the
>>> changes in B4. I think your result does (correct me if I'm wrong) so it
>>> is preserving the content of the original merge rather than the
>>> semantics of it.
> 
> Yeah, I understood what you mean, and I see you noticed that B4 
> commit, for which I did anticipate possibly bringing up a discussion 
> like this ;)
> 
> I agree with Jake here, my thoughts exactly (what I wrote in that 
> other subthread[1], too):
> 
> On 02/03/2018 17:02, Jacob Keller wrote:
>>
>> We only have the content, and we don't know the semantics (nor, I
>> think, should we attempt to understand or figure out the semantics).
> 
> Hmm, I wanted to elaborate a bit here, but that sentence seems to 
> summarize the pure essence of it, and whatever I write looks like 
> just repeating the same stuff again...
> 
> That`s just it. And stopping to give the user a chance to 
> review/amend the result, where he might decide he actually did want 
> something else - so all good.
> 
> Otherwise, I would be interested to learn how context/semantics 
> guessing could provide a better default action (without introducing 
> more complexity for might not be much benefit, if any).

I don't think its possible to guess the semantics of the original merge
as users can use custom merge strategies and amend the result. It would
be possible to detect and unamended '-s ours' merge but special casing
that may end up causing users more confusion rather than helping them.

> But in the end, I guess we can just discuss the "most sane default" 
> to present to the user (introduce or obsolete that new commit B4, in 
> the discussed example[2]), as we should definitely stop for amending 
> anyway, not proceeding automatically whenever U1' != U2'.

I can see the reason for that but I'm concerned that it might get
annoying with an interactive rebase as it would stop whenever one of the
commits on a topic branch that is a parent of a merge gets amended.
(squashing and reordering existing commits on a topic branch would be OK
though)

> Oh, and what about situations where we introduce new or drop existing 
> branches (which should be possible with new `--recreate-merges`)...? 
> "Preserving old branch semantics" may have even less sense here - the 
> (possibly heavily reorganized) content is the only thing we have, 
> where context will (and should) be provided by the user.

In this scheme there is now way to change the parents of a merge so
preserving the old branch sementics is well defined. If the user wants
to change the parents of the merge then this scheme wont help them.

> And I guess being consistent is pretty important, too - if you add 
> new content during merge rebase, it should always show up in the 
> merge, period. 

Yes, that should make it easy for the user to know what to expect from
rebase.

> It seems pretty confusing to find out one of the 
> branches "declared special" (even more if it`s based on uncertain 
> guess-work), so when you add something to it it`s just "swallowed", 
> as the whole branch is always obsoleted, for now and ever.
> 
> I might even see a value in such behavior, but only as a conscious 
> user action, not something done automatically... I guess? :)
> 
> Regards, Buga
> 
> [1] https://public-inbox.org/git/f26cdbe2-1bc3-02ff-7b99-12a6ebab5a70@gmail.com/
> [2] https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e5516655b3@gmail.com/
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 03/03/18 00:29, Igor Djordjevic wrote:
> Hi Phillip,
> 
> On 02/03/2018 12:31, Phillip Wood wrote:
>>
>>> Thinking about it overnight, I now suspect that original proposal had a
>>> mistake in the final merge step. I think that what you did is a way to
>>> fix it, and I want to try to figure what exactly was wrong in the
>>> original proposal and to find simpler way of doing it right.
>>>
>>> The likely solution is to use original UM as a merge-base for final
>>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>>> though, as that's exactly UM from which both U1' and U2' have diverged
>>> due to rebasing and other history editing.
>>
>> Hi Sergey, I've been following this discussion from the sidelines,
>> though I haven't had time to study all the posts in this thread in
>> detail. I wonder if it would be helpful to think of rebasing a merge as
>> merging the changes in the parents due to the rebase back into the
>> original merge. So for a merge M with parents A B C that are rebased to
>> A' B' C' the rebased merge M' would be constructed by (ignoring shell
>> quoting issues)
>>
>> git checkout --detach M
>> git merge-recursive A -- M A'
>> tree=$(git write-tree)
>> git merge-recursive B -- $tree B'
>> tree=$(git write-tree)
>> git merge-recursive C -- $tree C'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>>
>> This should pull in all the changes from the parents while preserving
>> any evil conflict resolution in the original merge. It superficially
>> reminds me of incremental merging [1] but it's so long since I looked at
>> that I'm not sure if there are any significant similarities.
>>
>> [1] https://github.com/mhagger/git-imerge
> 
> Interesting, from quick test[3], this seems to produce the same 
> result as that other test I previously provided[2], where temporary 
> commits U1' and U2' are finally merged with original M as a base :)
> 
> Just that this looks like even more straight-forward approach...?
> 
> The only thing I wonder of here is how would we check if the 
> "rebased" merge M' was "clean", or should we stop for user amendment? 
> With that other approach Sergey described, we have U1'==U2' to test with.

I think (though I haven't rigorously proved to myself) that in the
absence of conflicts this scheme has well defined semantics (the merges
can be commuted), so the result should be predicable from the users
point of view so maybe it could just offer an option to stop.

> 
> By the way, is there documentation for `git merge-recursive` 
> anywhere, besides the code itself...? :$

Not that I'm aware of unfortunately. It's pretty simple though

git merge-recursive [<options>] <merge-bases> -- <our-head> <their-head>

The options are listed on the 'git merge' man page but you specify them
as '--option' rather than '-Xoption'. I'm not sure what the exact
requirements are for the index, having it match <our-head> should always
be safe.

> 
> Thanks, Buga
> 
> [2] https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e5516655b3@gmail.com/
> [3] Quick test script:
> -- >8 --
> #!/bin/sh
> 
> # rm -rf ./.git
> # rm -f ./test.txt
> 
> git init
> 
> touch ./test.txt
> git add -- test.txt
> 
> # prepare repository
> for i in {1..8}
> do
> 	echo X$i >>test.txt
> 	git commit -am "X$i"
> done
> 
> # prepare branch A
> git checkout -b A
> sed -i '2iA1' test.txt
> git commit -am "A1"
> sed -i '4iA2' test.txt
> git commit -am "A2"
> sed -i '6iA3' test.txt
> git commit -am "A3"
> 
> # prepare branch B
> git checkout -b B master
> sed -i '5iB1' test.txt
> git commit -am "B1"
> sed -i '7iB2' test.txt
> git commit -am "B2"
> sed -i '9iB3' test.txt
> git commit -am "B3"
> 
> git checkout -b topic A
> git merge -s ours --no-commit B # merge A and B with `-s ours`
> sed -i '8iM' test.txt           # amend merge commit ("evil merge")
> git commit -am "M"
> git tag M #original-merge
> 
> # master moves on...
> git checkout master
> git cherry-pick B^     # cherry-pick B2 into master
> sed -i "1iX9" test.txt # add X9
> git commit -am "X9"
> 
> # (0) ---X8--B2'--X9 (master)
> #        |\
> #        | A1---A2---A3 (A)
> #        |             \
> #        |              M (topic)
> #        |             /
> #        \-B1---B2---B3 (B)
> 
> # simple/naive demonstration of proposed merge rebasing logic
> # using iterative merge-recursive, preserving merge commit manual
> # amendments, testing `-s ours` merge with cherry-picking from
> # obsoleted part, but still respecting interactively rebased
> # added/modified/dropped/cherry-picked commits :)
> 
> git checkout -b A-prime A
> git reset --hard HEAD^             # drop A3 from A
> sed -i '/A1/c\A12' test.txt        # amend A1 to A12
> git commit -a --amend --no-edit
> git rebase master                  # rebase A onto master
> git cherry-pick B                  # cherry-pick B3 into A
> 
> git checkout -b B-prime B
> git rebase master                  # rebase B onto master
> sed -i '12iB4' test.txt            # add B4
> git commit -am "B4"
> 
> git checkout --detach M
> git merge-recursive A -- M A-prime
> tree="$(git write-tree)"
> git merge-recursive B -- $tree B-prime
> tree="$(git write-tree)"
> git tag M-prime "$(git log --pretty=%B -1 M | git commit-tree $tree -p A-prime -p B-prime)"
> 
> git update-ref refs/heads/topic "$(git rev-parse M-prime)"
> git checkout topic
> 
> # (1) ---X8--B2'--X9 (master)
> #                 |\
> #                 | A12--A2'---B3' (A)
> #                 |             \
> #                 |              M' (topic)
> #                 |             /
> #                 \-B1'--B3'---B4  (B)
> 
> # show resulting graph
> # echo
> # git log --all --decorate --oneline --graph
> 
> # in comparison to original merge commit M, rebased merge commit 
> # M' is expected to:
> #
> # - Add X9, from updated "master"
> # - Have A1 changed to A12, due to A12 commit amendment
> # - Keep A2, rebased as A2'
> # - Remove A3, due to dropped A3 commit
> # - Keep amendment from original (evil) merge commit M
> # - Miss B1' like M does B, due to original `-s ours` merge strategy
> # - Add B2, cherry-picked as B2' into "master"
> # - Add B3, cherry-picked as B3' into "A"
> # - Add B4, added to "B"
> #
> # echo
> # echo 'diff M M-prime:'
> # git diff M M-prime
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 05/03/18 05:00, Sergey Organov wrote:
> Hi Plillip and Igor,
> 
> Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:
>> Hi Phillip,
>>
>> On 02/03/2018 12:31, Phillip Wood wrote:
>>>
>>>> Thinking about it overnight, I now suspect that original proposal had a
>>>> mistake in the final merge step. I think that what you did is a way to
>>>> fix it, and I want to try to figure what exactly was wrong in the
>>>> original proposal and to find simpler way of doing it right.
>>>>
>>>> The likely solution is to use original UM as a merge-base for final
>>>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty natural
>>>> though, as that's exactly UM from which both U1' and U2' have diverged
>>>> due to rebasing and other history editing.
>>>
>>> Hi Sergey, I've been following this discussion from the sidelines,
>>> though I haven't had time to study all the posts in this thread in
>>> detail. I wonder if it would be helpful to think of rebasing a merge as
>>> merging the changes in the parents due to the rebase back into the
>>> original merge. So for a merge M with parents A B C that are rebased to
>>> A' B' C' the rebased merge M' would be constructed by (ignoring shell
>>> quoting issues)
>>>
>>> git checkout --detach M
>>> git merge-recursive A -- M A'
>>> tree=$(git write-tree)
>>> git merge-recursive B -- $tree B'
>>> tree=$(git write-tree)
>>> git merge-recursive C -- $tree C'
>>> tree=$(git write-tree)
>>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>>>
>>> This should pull in all the changes from the parents while preserving
>>> any evil conflict resolution in the original merge. It superficially
>>> reminds me of incremental merging [1] but it's so long since I looked at
>>> that I'm not sure if there are any significant similarities.
>>>
>>> [1] https://github.com/mhagger/git-imerge
>>
>> Interesting, from quick test[3], this seems to produce the same 
>> result as that other test I previously provided[2], where temporary 
>> commits U1' and U2' are finally merged with original M as a base :)
> 
> Looks like sound approach and it's interesting if these 2 methods do in
> fact always bring the same result. Because if we look at the (now fixed)
> original approach closely, it also just gathers the changes in merge
> parents into U1' and U2', then merges the changes back into the original
> M (=U1=U2=UM).
> 
> Overall, this one looks like another implementation of essentially the
> same method and confirms that we all have the right thought direction
> here.
> 

Yes, I think they are doing the same thing. If there are no conflicts
then U1' is the should as "git merge-recursive A -- M A'". My patch
algebra isn't very good, but I think one ought to be able to show that
in the absence of conflicts the two approaches are equivalent.

>>
>> Just that this looks like even more straight-forward approach...?
>>
>> The only thing I wonder of here is how would we check if the 
>> "rebased" merge M' was "clean", or should we stop for user amendment? 
>> With that other approach Sergey described, we have U1'==U2' to test
>> with.
> 
> That's an advantage of the original, yes.

I wonder if just having a predicable result rather than forcing the
rebase to stop if the user just squashes a fixup commit into a topic
branch that is the parent of a merge might be more convenient in practice.

Best Wishes

Phillip

> -- Sergey
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Phillip,

Phillip Wood <phillip.wood@talktalk.net> writes:

> On 03/03/18 00:29, Igor Djordjevic wrote:
>> Hi Phillip,

[...]

>> 
>> The only thing I wonder of here is how would we check if the 
>> "rebased" merge M' was "clean", or should we stop for user amendment? 
>> With that other approach Sergey described, we have U1'==U2' to test with.
>
> I think (though I haven't rigorously proved to myself) that in the
> absence of conflicts this scheme has well defined semantics (the merges
> can be commuted), so the result should be predicable from the users
> point of view so maybe it could just offer an option to stop.

Yes, hopefully it's predictable, but is it the intended one? We don't
know, so there is still some level of uncertainty.

When in doubt, I try to find similar cases. There are two I'm aware of:

1. "git merge" just commits the result when there are no conflicts.
However, it supposedly has been run by the user just now, and thus user
can amend what he gets. That's effectively a stop for amendment from our
POV.

2. When rebasing, "rerere", when fires, stages the changes, and rebasing
stops for amendment. For me "rerere" behavior is rather annoying (I've
never in fact amended what it prepared), but I always assumed there are
good reasons it behaves this way.

Overall, to be consistent, it seems we do need to stop at U1' != U2', at
least by default. Additional options could be supported then to specify
user intentions, both on the command level and in the todo list,
provided it proves to be useful.

-- Sergey

[RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi,

This is v2 of my "Rebasing merges" proposal.

Significant changes are:

1. Fixed mistake in the final merge step in the original proposal: wrong
   merge base was used. Thanks everybody who provided test-cases, and
   special thanks to Igor Djordjevic <igor.d.djordjevic@gmail.com> for
   implementing and testing a few variants of the method.

2. Added discussion of the exact place where handling of special
   frequent cases such as "git merge -ours", if any, should go.

3. I now use "True Merge" name instead of former "Trivial Merge", to
   avoid confusion with what Git documentation calls "trivial merge",
   thanks to Junio C Hamano <gitster@pobox.com> for pointing this out.

During discussion of the original proposal, yet another way of
implementing a true rebase of a merge commit has been suggested by
Phillip Wood <phillip.wood@dunelm.org.uk>[1]. His method also gathers
the changes on both sides of the merge and then merges them back to the
original merge, so both methods have similar concept and differ in
implementation. It looks like both implementations bring the same
result, at least it was so in the limited testing that Igor performed.

[1] https://public-inbox.org/git/6c8749ca-ec5d-b4b7-f1a0-50d9ad2949a5@talktalk.net/


-------8<-------------8<------

By accepting the challenges raised in recent discussion of advanced
support for history rebasing and editing in Git, I hopefully figured out
a clean and elegant method of rebasing merges that I think is "The Right
Way (TM)" to perform this so far troublesome operation. ["(TM)" here has
second meaning: "True Merge" (TM), see below.]

Let me begin with quick outline of the method, to illustrate the
simplicity of the approach, and special thanks here must go to "Johannes
Sixt" <j6t@kdbg.org> for his original bright idea to use "cherry-pick
-m1" to rebase merge commits.

Given 2 original branches, b1 and b2, and a merge commit M that joins
them, suppose we've already rebased b1 to b1', and b2 to b2'. Suppose
also that B1' and B2' happen to be the tip commits on b1' and b2',
respectively.

To produce merge commit M' that joins b1' and b2', the following
operations will suffice:

1. Checkout b2' and cherry-pick -m2 M, to produce U2' (and new b2').
2. Checkout b1' and cherry-pick -m1 M, to produce U1' (and new b1').
3. Perform 3-way merge of U1' and U2' using original M as merge base,
   to get UM'.
4. Get rid of U1' and U2' by re-writing parent references of UM' from
   U1' and U2' to  B1' and B2', respectively, to produce M'.
5. Mission complete.

Let's now turn to the method itself and see why and how it actually
works.

First off, let me introduce you to my new friend, the True Merge, or
(TM) for short. By definition, (TM) is a merge that brings absolutely
no differences to the sides of the merge. (I also like to call him
"Angel Merge" (AM), both as being the most beautiful of all merges, and
as direct antithesis to "[d]evil merge"; or even "Perfect Merge" (PM),
but the latter goes after lunch time.)

Being trivial history joint and nothing else, (TM)/(AM)/(PM) is safe and
easy to be rebased (see below). However, since most of us have never met
(TM) in practice, you probably wonder how (TM) can actually help us
handle general case of rebasing of some random merge.

Let's start with this history:

    M
   / \
  B1  B2

where B1 and B2 are tip commits of 2 branches, and M is the merge commit
that joins them. Let's transform this history to the following one,
contextually equivalent to the original, by introducing 2 non-merge
utility commits U1 and U2, and a new utility merge commit UM:

    UM
   /  \
  U1   U2
  |    |
  B1   B2

were contents of all the created commits match, and are exact copies of
the original content of M. I.e., provided [A] denotes "content of commit
A", we have:

  [UM] = [U1] = [U2] = [M]

Stress again how these changes to the history preserve the exact content
of the original merge ([UM] = [M]), how U1 an U2 represent content
changes due to the merge on either side[*], and how content of neither
preceding nor subsequent commits is affected by the change of
representation.

Now observe that as [U1] = [UM], and [U2] = [UM], the UM happens to be
exactly our new friend -- the "True Merge (TM)" his true self,
introducing exactly zero changes to content. Pure history joint.

Next, we separately rebase both branches of our new representation of
the history to whatever new base we need, and we get:

  U1'  U2'
  |    |
  B1'  B2'

where U1' and U2' are rebased versions of U1 and U2, obtained by usual
rebasing methods for non-merge commits.

Finally, let's merge back our branches.

To perform the right kind of merge, notice that U1' and U2' have
diverged from U1 and U2, respectively. Further, provided [U1] = [U2] =
[UM] = [M], they have both diverged from the original merge commit
M. Therefore, to merge U1' and U2' into UM', it suffices to use 3-way
merge using original M as the merge base:

    UM'
   /  \
  U1'  U2'
   \  /
    M

Note that unlike merging of B1' and B2' that current "git rebase
--preserve-merges/--recreate-merges" performs, merging of U1' and U2' is
safe, as original UM, being (TM), doesn't itself carry any content
changes, and thus none could be missed by ignoring the UM. Essentially,
reproducing a (TM) is just joining corresponding histories back, and
that's what merge operation is all about.

If the resulting U1' and U2' happen to match ([U1'] = [U2']), we have
got our lovely (TM) back and we are clear to proceed automatically.

OTOH, if the resulting U1' and U2' differ, we should better stop for
user inspection and possible amendment of the resulting UM'. At this
point special treatment of any rather frequent cases (such as "git merge
-ours") could be applied before amendment, if need to be. Amendment by
replacing UM' with re-merge of B2' and B1' could be suggested to the
user as well. Options are plenty.

Finally, to get to our required merge commit M', we get the content of
UM' (possibly amended by the user), and record two actual parents of the
merge:

    M'
   / \
  B1' B2'

where [M'] = [UM'].

That's it. Mission complete.

The method is expected to have the following nice features:

- it carefully preserves user changes by rebasing the merge commit
itself, in a way that is semantically similar to rebasing simple
(non-merge) commits, yet it allows changes made to branches during
history editing to propagate over corresponding merge commit that joins
the branches, even automatically when the changes don't conflict, as
expected.

- it has provision for detection of even slightest chances of ending up
with surprising merge (just check if UM' is still (TM)), so that
implementation could stop for user inspection and amendment when
appropriate, yet it is capable of handling trivial cases smoothly and
automatically.

- it never falls back to simple invocation of merge operation on rebased
original branches themselves, thus avoiding the problem of lack of
knowledge of how the merge at hand has been performed in the first
place.

- it allows implementation to let user manually perform whatever merge
she wishes when suspect result is automatically detected.

- it extends trivially to octopus merges.

- it appears to be shiny to the point that it will likely be able to
handle even darkest [d]evil merges nicely, no special treatment
required.

Footnote:

[*] We may as well consider the (UM,U1,U2) trio to be semantically split
representation of git merge commit, where U1 and U2 represent content
changes to the sides, and UM represents pure history joint. Or, the
other way around, we may consider git merge commit to be optimized
representation of this trio. I think this split representation could
help to simplify reasoning about git merges in general.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Phillip Wood <phillip.wood@talktalk.net> writes:

> I wonder if just having a predicable result rather than forcing the
> rebase to stop if the user just squashes a fixup commit into a topic
> branch that is the parent of a merge might be more convenient in practice.

Unless I am misunderstanding what you are saying, that is pretty
much what I have automated for my daily rebuild of the 'pu' branch

Non-textual semantic conflicts are made (in the best case just once)
as a separate commit on top of mechanical auto-merge whose focus is
predictability (rather than cleverness) done by Git, and then that
separate commit is kept outside the history.  When replaying these
merges to rebuild the 'pu' branch, after resetting the tip to
'master', each topic is merged mechanically, and if such a fix-up
commit is present, "cherry-pick --no-commit" applies it and then
"commit --amend --no-edit" to adjust the merge.  I find it quite
valuable to have a separate record of what "evil" non-mechanical
adjustment was done, which I know won't be lost in the noise when
these merges need to be redone daily or more often.

The Appendix in Documentation/howto/maintain-git.txt talks about
this process.  You can see what topics have such merge-fix defined
by peeking https://github.com/gitster/git/ repository.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Phillip,

On Tue, 6 Mar 2018, Phillip Wood wrote:

> On 03/03/18 00:29, Igor Djordjevic wrote:
> > 
> > On 02/03/2018 12:31, Phillip Wood wrote:
> >>
> >>> Thinking about it overnight, I now suspect that original proposal
> >>> had a mistake in the final merge step. I think that what you did is
> >>> a way to fix it, and I want to try to figure what exactly was wrong
> >>> in the original proposal and to find simpler way of doing it right.
> >>>
> >>> The likely solution is to use original UM as a merge-base for final
> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
> >>> natural though, as that's exactly UM from which both U1' and U2'
> >>> have diverged due to rebasing and other history editing.
> >>
> >> Hi Sergey, I've been following this discussion from the sidelines,
> >> though I haven't had time to study all the posts in this thread in
> >> detail. I wonder if it would be helpful to think of rebasing a merge
> >> as merging the changes in the parents due to the rebase back into the
> >> original merge. So for a merge M with parents A B C that are rebased
> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
> >> shell quoting issues)
> >>
> >> git checkout --detach M
> >> git merge-recursive A -- M A'
> >> tree=$(git write-tree)
> >> git merge-recursive B -- $tree B'
> >> tree=$(git write-tree)
> >> git merge-recursive C -- $tree C'
> >> tree=$(git write-tree)
> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> >>
> >> This should pull in all the changes from the parents while preserving
> >> any evil conflict resolution in the original merge. It superficially
> >> reminds me of incremental merging [1] but it's so long since I looked at
> >> that I'm not sure if there are any significant similarities.
> >>
> >> [1] https://github.com/mhagger/git-imerge
> > 
> > Interesting, from quick test[3], this seems to produce the same 
> > result as that other test I previously provided[2], where temporary 
> > commits U1' and U2' are finally merged with original M as a base :)
> > 
> > Just that this looks like even more straight-forward approach...?
> > 
> > The only thing I wonder of here is how would we check if the 
> > "rebased" merge M' was "clean", or should we stop for user amendment? 
> > With that other approach Sergey described, we have U1'==U2' to test with.
> 
> I think (though I haven't rigorously proved to myself) that in the
> absence of conflicts this scheme has well defined semantics (the merges
> can be commuted), so the result should be predicable from the users
> point of view so maybe it could just offer an option to stop.

I am not so sure that the result is independent of the order of the
merges. In other words, I am not necessarily certain that it is impossible
to concoct A,A',B,B' commits where merging B'/B before A'/A has a
different result than merging A'/A before B'/B.

Remember, when constructing counter-examples to hypotheses, those
counter-examples do not really *have* to make sense on their own. For
example, A' could introduce *completely different* changes from A, and the
same is true for B' and B.

I could imagine, for example, that using a ton of consecutive empty lines,
and using patches that insert something into these empty lines (and are
thusly inherently ambiguous when said set of empty lines has changed),
could even introduce a merge conflict in one order, but no conflict in the
other.

Even so, I think that merging in the order of the parents makes the most
sense, and that using that strategy makes sense, too, because you really
have to try hard to make it fail.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Phillip & Buga,

On Tue, 6 Mar 2018, Phillip Wood wrote:

> On 02/03/18 23:33, Igor Djordjevic wrote:
> > 
> > [...]
> > Otherwise, I would be interested to learn how context/semantics 
> > guessing could provide a better default action (without introducing 
> > more complexity for might not be much benefit, if any).
> 
> I don't think its possible to guess the semantics of the original merge
> as users can use custom merge strategies and amend the result. It would
> be possible to detect and unamended '-s ours' merge but special casing
> that may end up causing users more confusion rather than helping them.

FWIW I agree.

My original plan was to always merge recursively and suggest to use `exec`
commands if anything else is needed.

But now with that excellent new idea to perform successive three-way
merges of the original merge commit with the new tips, using the old tips
as merge base, I am considering to change that.

There is a big problem here, though: consistency. See below for more
musings about that.

> > And I guess being consistent is pretty important, too - if you add new
> > content during merge rebase, it should always show up in the merge,
> > period. 
> 
> Yes, that should make it easy for the user to know what to expect from
> rebase.

Indeed. We have seen time and time again that consistent behavior is the
only thing that lets us adhere to the Law of Least Surprise.

And here lies the rub: do we really want to let `merge -C <commit>` behave
completely differently than `merge`? Granted, in one case we provide a
template merge commit, in the other case, we do not. And the idea is
already to behave differently, although that difference only extends to
the commit message so far.

But given the benefit (i.e. that the strategy to transform the original
merge commit into the new merge commit), I am willing to run that risk,
especially since I foresee only few users wanting to create new merge
commits from scratch using the `merge` todo command.

Of course, even then we need to be careful: the user might have
*changed* or *moved* the original `merge` command. For example, if the
merge command read:

	merge -C deadbee cafecafe bedbedbed

and the user switched the order of the merged branches into

	merge -C deadbee bedbedbed cafecafe

we would have to detect the changed order of the arguments so that we
could still find the original branch tips.

But the user might also have changed the branch(es) to merge completely,
in which case we might not even be able to find original branch tips.

My preferred solution would be to let the `merge` command figure out
whether the passed arguments correspond to the rewritten versions of the
original merge parents. And only in that case would we use the fancy
strategy, in all other cases we would fall back to performing a regular
recursive (or octopus) merge.

How does that sound?

It will be slightly inconsistent. But in a defendable way, I think.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 06/03/2018 19:12, Johannes Schindelin wrote:
> 
> > > And I guess being consistent is pretty important, too - if you add new
> > > content during merge rebase, it should always show up in the merge,
> > > period. 
> >
> > Yes, that should make it easy for the user to know what to expect from
> > rebase.
> 
> Indeed. We have seen time and time again that consistent behavior is the
> only thing that lets us adhere to the Law of Least Surprise.
> 
> And here lies the rub: do we really want to let `merge -C <commit>` behave
> completely differently than `merge`? Granted, in one case we provide a
> template merge commit, in the other case, we do not. And the idea is
> already to behave differently, although that difference only extends to
> the commit message so far.
> 
> But given the benefit (i.e. that the strategy to transform the original
> merge commit into the new merge commit), I am willing to run that risk,
> especially since I foresee only few users wanting to create new merge
> commits from scratch using the `merge` todo command.
> 
> Of course, even then we need to be careful: the user might have
> *changed* or *moved* the original `merge` command. For example, if the
> merge command read:
> 
> 	merge -C deadbee cafecafe bedbedbed
> 
> and the user switched the order of the merged branches into
> 
> 	merge -C deadbee bedbedbed cafecafe
> 
> we would have to detect the changed order of the arguments so that we
> could still find the original branch tips.
> 
> But the user might also have changed the branch(es) to merge completely,
> in which case we might not even be able to find original branch tips.
> 
> My preferred solution would be to let the `merge` command figure out
> whether the passed arguments correspond to the rewritten versions of the
> original merge parents. And only in that case would we use the fancy
> strategy, in all other cases we would fall back to performing a regular
> recursive (or octopus) merge.
> 
> How does that sound?
> 
> It will be slightly inconsistent. But in a defendable way, I think.

I like where this discussion is heading, and here`s what I thought 
about it :)

First, starting from non-interactive rebase, I guess we may now agree 
that _rebasing_ merges is an actually expected behavior, not recreating 
them (thus keeping manual conflict resolutions and amendments, not 
losing them).

Now, interactive rebase is a totally different story, we already said 
user can change pretty much about everything, making merge 
_recreation_ to be a more sane choice, but let`s leave this other 
extreme for a brief moment.

In the least interesting situation, though, user could just review 
and close todo list, without changing anything - and in that case it 
would be important, consistency wise, to behave exactly like in case 
of non-interactive rebase, meaning still rebasing merges, not 
recreating them.

Ok, so that still aligns with what`s written so far - we need to be 
able to rebase merges interactively, too (not just recreate them), to 
stay consistent in less complex interactive rebases.

But, what if user really wants to _recreate_ merges, for whatever 
reason? Come on, this is interactive rebase we`re talking about, why 
being restrictive? :)

Here`s a twist - not letting `merge` trying to be too smart by 
figuring out whether passed arguments correspond to rewritten 
versions of the original merge parents (which would be too 
restrictive, too, I`m afraid), but just be explicit about it, instead!

So, it could be something like:

	merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed


The format is still something to think about, but the point is rather 
simple - explicitly map old and new merge parents, showing this 
inside todo list by default.

This makes it much easier for later processing (and correct, no need 
to guess which one goes where), but also gives more power to the 
user, being able to decide which merge parents get "rebased", and 
which ones should go into the merge just like "new".

So if a user gets an interactive todo list like that and just closes 
it, we still have exact situation like non-interactive rebase (and no 
guessing on implementation side).

But, user might still decide to introduce new merge parents into the 
mix, even, where we could then just be merging those (as there is no 
old merge parent to actually rebase from):

	merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed new-branch
	
Here, "new-branch" is something new, introduced inside interactive 
rebase, and it will be just merged into the other two (which are 
still being rebased).

Also, another example - if original merge parent "123abc" was merged 
from the other side using `-s ours` strategy, that means all the 
content this branch originally had will still be missing from the 
rebased merge (expect for what`s been cherry-picked elsewhere).

But, I would argue it`s quite legit to want to revise that decision, 
and let that content in this time. To make that happen, one would 
just remove "123abc:" from the todo list:

	merge -C deadbee cafecafe 234bcd:bedbedbed new-branch

..., meaning that only "bedbedbed" should be rebased according 
original merge parent "234bcd", where both "cafecafe" and "new-branch" 
should be just merged in, no previous context existing (no rebase).

In the end, user might even decide to swap old/new parent mapping, 
and that should be possible, too (might be pretty strange, though, 
causing conflicts, but we shouldn`t judge).

Or, one could map old merge parent to some totally new merge parent, 
like "new-branch" in that example above, all this being a fair game.

As one might suspect, to recreate merge from scratch instead, just 
drop all the old merge parents mappings:

	merge -C deadbee cafecafe bedbedbed


There, in my opinion, something like this provides the most 
consistent user experience (we always behave the same), while adding 
additional possibilities on top (getting to actually decide which 
merge parents get rebased, and which just merged), plus avoids all 
guess work (old and new merge parents, to be used for merge rebasing, 
are explicitly mapped).

What do you think? (unrelated to the parent mapping format itself, 
which could/should probably be made better, if possible)

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Johannes,

On 05/03/2018 18:29, Johannes Schindelin wrote:
> 
> > By the way, is there documentation for `git merge-recursive` 
> > anywhere, besides the code itself...? :$
> 
> I am not aware of any. The commit message adding the command is not very
> illuminating (https://github.com/git-for-windows/git/commit/720d150c4):
> 
>     Add a new merge strategy by Fredrik Kuivinen.
> 
>     I really wanted to try this out, instead of asking for an adjustment
>     to the 'git merge' driver and waiting.  For now the new strategy is
>     called 'fredrik' and not in the list of default strategies to be tried.
> 
>     The script wants Python 2.4 so this commit also adjusts Debian and RPM
>     build procecure files.
> 
> Digging through https://public-inbox.org/git/ during that time frame comes
> up with this hit, though:
> 
> https://public-inbox.org/git/20050907164734.GA20198@c165.ib.student.liu.se/
> 
> which is still not a good documentation of the algorithm. You can probably
> dig further yourself, but I think I can describe it very quickly here:
> 
> To merge two commits recursively, you first have to find their "merge
> bases". If there was an obvious branch point, then that is the merge base.
> But when you start a branch off of master, then work a bit, then merge
> master, you already have two merge bases.
> 
> The trick about the recursive merge is to reduce the number of merge bases
> iteratively to one. It does that by taking two merge bases, and performing
> a recursive merge on them, which generates a "virtual" commit, the
> condensed merge base. That one is then merged recursively with the next
> merge base, until there is only one left.
> 
> A recursive merge of two commits with exactly one merge base is simply a
> three-way merge.
> 
> I vaguely remember that there was something funny about the order in which
> order you want to process the merge bases: if you did it in one
> (chronological) direction, it worked beautifully, in the other direction
> it would generate tons of merge conflicts or something like that.

Thanks, this is very informative (together with the linked discussion).

Not remembering seeing this before, I wasn`t really sure if this was 
some undocumented core Git (plumbing?) utility (used withing Git 
itself, too), or just a leftover (yet still useful) sample tool.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> I don't think its possible to guess the semantics of the original merge
>> as users can use custom merge strategies and amend the result. It would
>> be possible to detect and unamended '-s ours' merge but special casing
>> that may end up causing users more confusion rather than helping them.
>
> FWIW I agree.

I think it is a mistake to sacrifice predictability only to add
cleverness that sometimes work.  Elsewhere in the thread, I think I
saw an argument to treat interactive and non-interactive something
very different, but there is no fundamental difference between them
(it is far easier with interactive to force the command to "port"
each change to a vastly different context) so having consistent
behaviour between the two cases is important, too.

>
> My original plan was to always merge recursively and suggest to use `exec`
> commands if anything else is needed.
>
> But now with that excellent new idea to perform successive three-way
> merges of the original merge commit with the new tips, using the old tips
> as merge base, I am considering to change that.

OK, does this mean we want to wait before merging the "recreate
merge" topic down to 'next'?  For more than a few weeks, it has been
slated for 'next'.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Phillip,
>
> On Tue, 6 Mar 2018, Phillip Wood wrote:
>
>> On 03/03/18 00:29, Igor Djordjevic wrote:
>> > 
>> > On 02/03/2018 12:31, Phillip Wood wrote:
>> >>
>> >>> Thinking about it overnight, I now suspect that original proposal
>> >>> had a mistake in the final merge step. I think that what you did is
>> >>> a way to fix it, and I want to try to figure what exactly was wrong
>> >>> in the original proposal and to find simpler way of doing it right.
>> >>>
>> >>> The likely solution is to use original UM as a merge-base for final
>> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
>> >>> natural though, as that's exactly UM from which both U1' and U2'
>> >>> have diverged due to rebasing and other history editing.
>> >>
>> >> Hi Sergey, I've been following this discussion from the sidelines,
>> >> though I haven't had time to study all the posts in this thread in
>> >> detail. I wonder if it would be helpful to think of rebasing a merge
>> >> as merging the changes in the parents due to the rebase back into the
>> >> original merge. So for a merge M with parents A B C that are rebased
>> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
>> >> shell quoting issues)
>> >>
>> >> git checkout --detach M
>> >> git merge-recursive A -- M A'
>> >> tree=$(git write-tree)
>> >> git merge-recursive B -- $tree B'
>> >> tree=$(git write-tree)
>> >> git merge-recursive C -- $tree C'
>> >> tree=$(git write-tree)
>> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> >>
>> >> This should pull in all the changes from the parents while preserving
>> >> any evil conflict resolution in the original merge. It superficially
>> >> reminds me of incremental merging [1] but it's so long since I looked at
>> >> that I'm not sure if there are any significant similarities.
>> >>
>> >> [1] https://github.com/mhagger/git-imerge
>> > 
>> > Interesting, from quick test[3], this seems to produce the same 
>> > result as that other test I previously provided[2], where temporary 
>> > commits U1' and U2' are finally merged with original M as a base :)
>> > 
>> > Just that this looks like even more straight-forward approach...?
>> > 
>> > The only thing I wonder of here is how would we check if the 
>> > "rebased" merge M' was "clean", or should we stop for user amendment? 
>> > With that other approach Sergey described, we have U1'==U2' to test with.
>> 
>> I think (though I haven't rigorously proved to myself) that in the
>> absence of conflicts this scheme has well defined semantics (the merges
>> can be commuted), so the result should be predicable from the users
>> point of view so maybe it could just offer an option to stop.
>
> I am not so sure that the result is independent of the order of the
> merges. In other words, I am not necessarily certain that it is impossible
> to concoct A,A',B,B' commits where merging B'/B before A'/A has a
> different result than merging A'/A before B'/B.
>
> Remember, when constructing counter-examples to hypotheses, those
> counter-examples do not really *have* to make sense on their own. For
> example, A' could introduce *completely different* changes from A, and the
> same is true for B' and B.
>
> I could imagine, for example, that using a ton of consecutive empty lines,
> and using patches that insert something into these empty lines (and are
> thusly inherently ambiguous when said set of empty lines has changed),
> could even introduce a merge conflict in one order, but no conflict in the
> other.
>
> Even so, I think that merging in the order of the parents makes the most
> sense, and that using that strategy makes sense, too, because you really
> have to try hard to make it fail.

Alternatively, consider to adopt the original approach that has none of
these issues as it uses exactly the same method for rebasing merge
commits that you are already using for rebasing simple commits, not to
mention the advantage of the built-in consistency check.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Tue, 6 Mar 2018, Sergey Organov wrote:

> This is v2 of my "Rebasing merges" proposal.

Didn't we settle on Phillip's "perform successive three-way merges between
the original merge commit and the new tips with the old tips as base"
strategy? It has the following advantages:

- it is *very simple* to describe

- it is *very easy* to reason about, once it is pointed out that rebases
  and merges result in the same trees.

... and BTW...

> 3. I now use "True Merge" name instead of former "Trivial Merge", to
>    avoid confusion with what Git documentation calls "trivial merge",
>    thanks to Junio C Hamano <gitster@pobox.com> for pointing this out.

"True Merge" is probably also a candidate for improvement. If what you
refer to is a "true" merge, that means all others are "untrue" merges???

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 7 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Tue, 6 Mar 2018, Phillip Wood wrote:
> >
> >> On 03/03/18 00:29, Igor Djordjevic wrote:
> >> > 
> >> > On 02/03/2018 12:31, Phillip Wood wrote:
> >> >>
> >> >>> Thinking about it overnight, I now suspect that original proposal
> >> >>> had a mistake in the final merge step. I think that what you did is
> >> >>> a way to fix it, and I want to try to figure what exactly was wrong
> >> >>> in the original proposal and to find simpler way of doing it right.
> >> >>>
> >> >>> The likely solution is to use original UM as a merge-base for final
> >> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
> >> >>> natural though, as that's exactly UM from which both U1' and U2'
> >> >>> have diverged due to rebasing and other history editing.
> >> >>
> >> >> Hi Sergey, I've been following this discussion from the sidelines,
> >> >> though I haven't had time to study all the posts in this thread in
> >> >> detail. I wonder if it would be helpful to think of rebasing a merge
> >> >> as merging the changes in the parents due to the rebase back into the
> >> >> original merge. So for a merge M with parents A B C that are rebased
> >> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
> >> >> shell quoting issues)
> >> >>
> >> >> git checkout --detach M
> >> >> git merge-recursive A -- M A'
> >> >> tree=$(git write-tree)
> >> >> git merge-recursive B -- $tree B'
> >> >> tree=$(git write-tree)
> >> >> git merge-recursive C -- $tree C'
> >> >> tree=$(git write-tree)
> >> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> >> >>
> >> >> This should pull in all the changes from the parents while preserving
> >> >> any evil conflict resolution in the original merge. It superficially
> >> >> reminds me of incremental merging [1] but it's so long since I looked at
> >> >> that I'm not sure if there are any significant similarities.
> >> >>
> >> >> [1] https://github.com/mhagger/git-imerge
> >> > 
> >> > Interesting, from quick test[3], this seems to produce the same 
> >> > result as that other test I previously provided[2], where temporary 
> >> > commits U1' and U2' are finally merged with original M as a base :)
> >> > 
> >> > Just that this looks like even more straight-forward approach...?
> >> > 
> >> > The only thing I wonder of here is how would we check if the 
> >> > "rebased" merge M' was "clean", or should we stop for user amendment? 
> >> > With that other approach Sergey described, we have U1'==U2' to test with.
> >> 
> >> I think (though I haven't rigorously proved to myself) that in the
> >> absence of conflicts this scheme has well defined semantics (the merges
> >> can be commuted), so the result should be predicable from the users
> >> point of view so maybe it could just offer an option to stop.
> >
> > I am not so sure that the result is independent of the order of the
> > merges. In other words, I am not necessarily certain that it is impossible
> > to concoct A,A',B,B' commits where merging B'/B before A'/A has a
> > different result than merging A'/A before B'/B.
> >
> > Remember, when constructing counter-examples to hypotheses, those
> > counter-examples do not really *have* to make sense on their own. For
> > example, A' could introduce *completely different* changes from A, and the
> > same is true for B' and B.
> >
> > I could imagine, for example, that using a ton of consecutive empty lines,
> > and using patches that insert something into these empty lines (and are
> > thusly inherently ambiguous when said set of empty lines has changed),
> > could even introduce a merge conflict in one order, but no conflict in the
> > other.
> >
> > Even so, I think that merging in the order of the parents makes the most
> > sense, and that using that strategy makes sense, too, because you really
> > have to try hard to make it fail.
> 
> Alternatively, consider to adopt the original approach that has none of
> these issues as it uses exactly the same method for rebasing merge
> commits that you are already using for rebasing simple commits, not to
> mention the advantage of the built-in consistency check.

Surely I misunderstand?

How can your approach -- which relies *very much* on having the original
parent commits -- not *require* that consistency check?

What would your approach (that still has no satisfyingly trivial
explanation, in my mind) do if somebody edited a `merge` command and let
it merge a completely unrelated commit?

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Wed, 7 Mar 2018, Igor Djordjevic wrote:

> On 05/03/2018 18:29, Johannes Schindelin wrote:
> > 
> > > By the way, is there documentation for `git merge-recursive`
> > > anywhere, besides the code itself...? :$
> > 
> > I am not aware of any. The commit message adding the command is not very
> > illuminating (https://github.com/git-for-windows/git/commit/720d150c4):
> > 
> >     Add a new merge strategy by Fredrik Kuivinen.
> > 
> >     I really wanted to try this out, instead of asking for an adjustment
> >     to the 'git merge' driver and waiting.  For now the new strategy is
> >     called 'fredrik' and not in the list of default strategies to be tried.
> > 
> >     The script wants Python 2.4 so this commit also adjusts Debian and RPM
> >     build procecure files.
> > 
> > Digging through https://public-inbox.org/git/ during that time frame comes
> > up with this hit, though:
> > 
> > https://public-inbox.org/git/20050907164734.GA20198@c165.ib.student.liu.se/
> > 
> > which is still not a good documentation of the algorithm. You can probably
> > dig further yourself, but I think I can describe it very quickly here:
> > 
> > To merge two commits recursively, you first have to find their "merge
> > bases". If there was an obvious branch point, then that is the merge base.
> > But when you start a branch off of master, then work a bit, then merge
> > master, you already have two merge bases.
> > 
> > The trick about the recursive merge is to reduce the number of merge bases
> > iteratively to one. It does that by taking two merge bases, and performing
> > a recursive merge on them, which generates a "virtual" commit, the
> > condensed merge base. That one is then merged recursively with the next
> > merge base, until there is only one left.
> > 
> > A recursive merge of two commits with exactly one merge base is simply a
> > three-way merge.
> > 
> > I vaguely remember that there was something funny about the order in which
> > order you want to process the merge bases: if you did it in one
> > (chronological) direction, it worked beautifully, in the other direction
> > it would generate tons of merge conflicts or something like that.
> 
> Thanks, this is very informative (together with the linked discussion).
> 
> Not remembering seeing this before, I wasn`t really sure if this was 
> some undocumented core Git (plumbing?) utility (used withing Git 
> itself, too), or just a leftover (yet still useful) sample tool.

You raise a good point. The "recursive merge" is not really documented
properly. Maybe you can enhance my vague description and provide a patch
for Documentation/technical/?

Thanks,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Junio,

On Tue, 6 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> I don't think its possible to guess the semantics of the original merge
> >> as users can use custom merge strategies and amend the result. It would
> >> be possible to detect and unamended '-s ours' merge but special casing
> >> that may end up causing users more confusion rather than helping them.
> >
> > FWIW I agree.
> 
> I think it is a mistake to sacrifice predictability only to add
> cleverness that sometimes work.  Elsewhere in the thread, I think I
> saw an argument to treat interactive and non-interactive something
> very different, but there is no fundamental difference between them
> (it is far easier with interactive to force the command to "port"
> each change to a vastly different context) so having consistent
> behaviour between the two cases is important, too.

I could be swayed both ways, but Buga already pointed out that we do not
have to compromise any consistency, simply by adding some syntactic sugar
to the `merge` command so that the different behavior is *explicit*.

> > My original plan was to always merge recursively and suggest to use `exec`
> > commands if anything else is needed.
> >
> > But now with that excellent new idea to perform successive three-way
> > merges of the original merge commit with the new tips, using the old tips
> > as merge base, I am considering to change that.
> 
> OK, does this mean we want to wait before merging the "recreate
> merge" topic down to 'next'?  For more than a few weeks, it has been
> slated for 'next'.

Maybe a few more days.

My current thinking is to rework the handling of -c vs -C and *not* have
two different todo_command enum values, but rather introduce an unsigned
integer that has flags such as TODO_MERGE_EDIT.

And for this new behavior, we could introduce a new flag
(TODO_MERGE_REBASE_MERGE_COMMIT or something less unwieldy) and set that
flag via

	merge -R -C <commit> <merge>...

(i.e. via a new flag `-R`).

I want to discuss this in the other subthread, though.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Tue, 6 Mar 2018, Igor Djordjevic wrote:

> On 06/03/2018 19:12, Johannes Schindelin wrote:
> > 
> > > > And I guess being consistent is pretty important, too - if you add new
> > > > content during merge rebase, it should always show up in the merge,
> > > > period. 
> > >
> > > Yes, that should make it easy for the user to know what to expect from
> > > rebase.
> > 
> > [...]
> > 
> > It will be slightly inconsistent. But in a defendable way, I think.
> 
> I like where this discussion is heading, and here`s what I thought 
> about it :)
> 
> [...]
> 
> Here`s a twist - not letting `merge` trying to be too smart by 
> figuring out whether passed arguments correspond to rewritten 
> versions of the original merge parents (which would be too 
> restrictive, too, I`m afraid), but just be explicit about it, instead!

That's the missing piece, I think.

> So, it could be something like:
> 
> 	merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed

I like where this is heading, too, but I do not think that we can do this
on a per-MERGE_HEAD basis. The vast majority of merge commits, in
practice, have two parents. So the `merge` command would actually only
have one revision to merge (because HEAD is the implicit first parent). So
that is easy.

But as soon as you go octopus, you can either perform an octopus merge, or
rebase the original merge commit. You cannot really mix and match here.

Unless we reimplement the octopus merge (which works quite a bit
differently from the "rebase merge commit" strategy, even if it is
incremental, too), which has its own challenges: if there are merge
conflicts before merging the last MERGE_HEAD, the octopus merge will exit
with status 2, telling you "Should not be doing an octopus.". While we
will want to keep merge conflict markers and continue with the "rebase the
original merge commit" strategy.

And it would slam the door shut for adding support for *other* merge
strategies to perform a more-than-two-parents merge.

Also, I do not think that it makes a whole lot of sense in practice to let
users edit what will be used for "original parent". If the user wants to
do complicated stuff, they can already do that, via `exec`. The `merge`
command really should be about facilitating common workflows, guiding the
user to what is sane.

Currently my favorite idea is to introduce a new flag: -R (for "rebase the
original merge commit"). It would look like this:

	merge -R -C <original-merge> <merge-head> # <oneline>

This flag would of course trigger the consistency check (does the number
of parents of the original merge commit agree with the parameter list? Was
an original merge commit specified to begin with?), and it would not fall
back to the recursive merge, but error out if that check failed.

Side note: I wonder whether we really need to perform the additional check
that ensures that the <merge-head> refers to the rewritten version of the
original merge commit's parent.

Second side note: if we can fast-forward, currently we prefer that, and I
think we should keep that behavior with -R, too.

If the user wants to force a new merge, they simply remove that -R flag.

What do you think?

Ciao,
Dscho

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Tue, 6 Mar 2018, Sergey Organov wrote:
>
>> This is v2 of my "Rebasing merges" proposal.
>
> Didn't we settle on Phillip's "perform successive three-way merges between
> the original merge commit and the new tips with the old tips as base"
> strategy?

It seems you did, dunno exactly why.

The main problem with this decision is that we still don't see how and
when to stop for user amendment using this method. OTOH, the original
has this issue carefully discussed.

> It has the following advantages:
>
> - it is *very simple* to describe

The original is as simple if not simpler:

"rebase sides of the merge commit and then three-way merge them back
using original merge commit as base"

No problems with octopuses, and no virtual merge bases of recursive
merges to reason about.

> - it is *very easy* to reason about, once it is pointed out that rebases
>   and merges result in the same trees.

The original is as easy to reason about, if not easier, especially
as recursive merge strategy is not being used there in new ways.

I honestly don't see any advantages of Phillip's method over the
original, except personal preferences. At the same time, I have no
objection of using it either, provided consistency check problem is
solved there as well.

>
> ... and BTW...
>
>> 3. I now use "True Merge" name instead of former "Trivial Merge", to
>>    avoid confusion with what Git documentation calls "trivial merge",
>>    thanks to Junio C Hamano <gitster@pobox.com> for pointing this out.
>
> "True Merge" is probably also a candidate for improvement. If what you
> refer to is a "true" merge, that means all others are "untrue"
> merges???

[d]evil merges, obviously.

Seriously, it's pure history joint. Just "joint' will do.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 7 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Tue, 6 Mar 2018, Sergey Organov wrote:
> >
> >> This is v2 of my "Rebasing merges" proposal.
> >
> > Didn't we settle on Phillip's "perform successive three-way merges
> > between the original merge commit and the new tips with the old tips
> > as base" strategy?
> 
> It seems you did, dunno exactly why.

That is not true. You make it sound like I was the only one who liked
this, and not Phillip and Buga, too.

Are you interested in the best solution, or in your solution :-)

> The main problem with this decision is that we still don't see how and
> when to stop for user amendment using this method. OTOH, the original
> has this issue carefully discussed.

Why would we want to stop, unless there are merge conflicts?

> > It has the following advantages:
> >
> > - it is *very simple* to describe
> 
> The original is as simple if not simpler:
> 
> "rebase sides of the merge commit and then three-way merge them back
> using original merge commit as base"

And that is also wrong, as I had proved already! Only Buga's addition made
it robust against dropping/modifying commits, and that addition also makes
it more complicated.

And it still has no satisfactory simple explanation why it works.

> No problems with octopuses, and no virtual merge bases of recursive
> merges to reason about.

But those are no problems for Phillip's strategy, either!

So your point is...?

> > - it is *very easy* to reason about, once it is pointed out that
> > rebases and merges result in the same trees.
> 
> The original is as easy to reason about, if not easier, especially as
> recursive merge strategy is not being used there in new ways.

So do it. I still have to hear a single-sentence, clear and obvious
explanation why it works.

And please do not describe why your original version works, because it
does not work. Describe why the one amended with Buga's hack works.

> I honestly don't see any advantages of Phillip's method over the
> original, except personal preferences. At the same time, I have no
> objection of using it either, provided consistency check problem is
> solved there as well.

Okay, let me reiterate then, because I do not want this point to be
missed:

Phillip's method is essentially merging the new tips into the original
merge, pretending that the new tips were not rebased but merged into
upstream.

So it exploits the duality of the rebase and merge operation, which both
result in identical trees (potentially after resolving merge conflicts).

I cannot think of any such interpretation for your proposal augmented by
Buga's fix-ups. And I haven't heard any such interpretation from your
side, either.

> >> 3. I now use "True Merge" name instead of former "Trivial Merge", to
> >>    avoid confusion with what Git documentation calls "trivial merge",
> >>    thanks to Junio C Hamano <gitster@pobox.com> for pointing this out.
> >
> > "True Merge" is probably also a candidate for improvement. If what you
> > refer to is a "true" merge, that means all others are "untrue"
> > merges???
> 
> [d]evil merges, obviously.
> 
> Seriously, it's pure history joint. Just "joint' will do.

You might want to try harder to stick with the existing nomenclature. That
would make it a lot easier to discuss.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Wed, 7 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > On Tue, 6 Mar 2018, Phillip Wood wrote:
>> >
>> >> On 03/03/18 00:29, Igor Djordjevic wrote:
>> >> > 
>> >> > On 02/03/2018 12:31, Phillip Wood wrote:
>> >> >>
>> >> >>> Thinking about it overnight, I now suspect that original proposal
>> >> >>> had a mistake in the final merge step. I think that what you did is
>> >> >>> a way to fix it, and I want to try to figure what exactly was wrong
>> >> >>> in the original proposal and to find simpler way of doing it right.
>> >> >>>
>> >> >>> The likely solution is to use original UM as a merge-base for final
>> >> >>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
>> >> >>> natural though, as that's exactly UM from which both U1' and U2'
>> >> >>> have diverged due to rebasing and other history editing.
>> >> >>
>> >> >> Hi Sergey, I've been following this discussion from the sidelines,
>> >> >> though I haven't had time to study all the posts in this thread in
>> >> >> detail. I wonder if it would be helpful to think of rebasing a merge
>> >> >> as merging the changes in the parents due to the rebase back into the
>> >> >> original merge. So for a merge M with parents A B C that are rebased
>> >> >> to A' B' C' the rebased merge M' would be constructed by (ignoring
>> >> >> shell quoting issues)
>> >> >>
>> >> >> git checkout --detach M
>> >> >> git merge-recursive A -- M A'
>> >> >> tree=$(git write-tree)
>> >> >> git merge-recursive B -- $tree B'
>> >> >> tree=$(git write-tree)
>> >> >> git merge-recursive C -- $tree C'
>> >> >> tree=$(git write-tree)
>> >> >> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> >> >>
>> >> >> This should pull in all the changes from the parents while preserving
>> >> >> any evil conflict resolution in the original merge. It superficially
>> >> >> reminds me of incremental merging [1] but it's so long since I looked at
>> >> >> that I'm not sure if there are any significant similarities.
>> >> >>
>> >> >> [1] https://github.com/mhagger/git-imerge
>> >> > 
>> >> > Interesting, from quick test[3], this seems to produce the same 
>> >> > result as that other test I previously provided[2], where temporary 
>> >> > commits U1' and U2' are finally merged with original M as a base :)
>> >> > 
>> >> > Just that this looks like even more straight-forward approach...?
>> >> > 
>> >> > The only thing I wonder of here is how would we check if the 
>> >> > "rebased" merge M' was "clean", or should we stop for user amendment? 
>> >> > With that other approach Sergey described, we have U1'==U2' to test with.
>> >> 
>> >> I think (though I haven't rigorously proved to myself) that in the
>> >> absence of conflicts this scheme has well defined semantics (the merges
>> >> can be commuted), so the result should be predicable from the users
>> >> point of view so maybe it could just offer an option to stop.
>> >
>> > I am not so sure that the result is independent of the order of the
>> > merges. In other words, I am not necessarily certain that it is impossible
>> > to concoct A,A',B,B' commits where merging B'/B before A'/A has a
>> > different result than merging A'/A before B'/B.
>> >
>> > Remember, when constructing counter-examples to hypotheses, those
>> > counter-examples do not really *have* to make sense on their own. For
>> > example, A' could introduce *completely different* changes from A, and the
>> > same is true for B' and B.
>> >
>> > I could imagine, for example, that using a ton of consecutive empty lines,
>> > and using patches that insert something into these empty lines (and are
>> > thusly inherently ambiguous when said set of empty lines has changed),
>> > could even introduce a merge conflict in one order, but no conflict in the
>> > other.
>> >
>> > Even so, I think that merging in the order of the parents makes the most
>> > sense, and that using that strategy makes sense, too, because you really
>> > have to try hard to make it fail.
>> 
>> Alternatively, consider to adopt the original approach that has none of
>> these issues as it uses exactly the same method for rebasing merge
>> commits that you are already using for rebasing simple commits, not to
>> mention the advantage of the built-in consistency check.
>
> Surely I misunderstand?
>
> How can your approach -- which relies *very much* on having the original
> parent commits -- not *require* that consistency check?

I don't understand what you mean, sorry. Could you please point me
to the *require* you talk about in the original proposal?

> What would your approach (that still has no satisfyingly trivial
> explanation, in my mind)

Here is one-liner: rebase sides of the merge commit and then 3-way
merge them, using original merge commit as merge base.

> do if somebody edited a `merge` command and let it merge a completely
> unrelated commit?

Don't see a problem, sorry. The method should still work, provided you have
original merge commit and two new parents for the new merge.

You rebase sides of original merge to the new parents, then 3-way merge
the results using original merge as base.

Once again, if you can rebase simple commit, the method allows to rebase
the merge either.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Wed, 7 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > On Tue, 6 Mar 2018, Sergey Organov wrote:
>> >
>> >> This is v2 of my "Rebasing merges" proposal.
>> >
>> > Didn't we settle on Phillip's "perform successive three-way merges
>> > between the original merge commit and the new tips with the old tips
>> > as base" strategy?
>> 
>> It seems you did, dunno exactly why.
>
> That is not true. You make it sound like I was the only one who liked
> this, and not Phillip and Buga, too.
>
> Are you interested in the best solution, or in your solution :-)

I'm interested in any that works, and only you say that those suggested
by Phillip is somehow superior. I still believe it's mine that superior,
even if slightly.

>
>> The main problem with this decision is that we still don't see how and
>> when to stop for user amendment using this method. OTOH, the original
>> has this issue carefully discussed.
>
> Why would we want to stop, unless there are merge conflicts?

There is somewhat lengthy discussion about it that you probably missed.
Not to repeat it, just see how 'rerere' works when it fires during
rebase, even with no conflicts.

>
>> > It has the following advantages:
>> >
>> > - it is *very simple* to describe
>> 
>> The original is as simple if not simpler:
>> 
>> "rebase sides of the merge commit and then three-way merge them back
>> using original merge commit as base"
>
> And that is also wrong, as I had proved already! Only Buga's addition made
> it robust against dropping/modifying commits, and that addition also makes
> it more complicated.

No. Get your facts straight. The [RFC v2] already fixed that original
mistake. Could you please finally read it?

> And it still has no satisfactory simple explanation why it works.

It has. It's there in the [RFC v2]. You seem to be the only one who
doesn't get it. I suppose you just didn't bother to read.

>> No problems with octopuses, and no virtual merge bases of recursive
>> merges to reason about.
>
> But those are no problems for Phillip's strategy, either!

I thought it was you who started to discuss virtual merge bases and
related problems, as well as how it's difficult to support octopus
merges, but it's fine with me if there are none of these problems.

>
> So your point is...?

Still the same -- use what's better, the [RFC v2].

>
>> > - it is *very easy* to reason about, once it is pointed out that
>> > rebases and merges result in the same trees.
>> 
>> The original is as easy to reason about, if not easier, especially as
>> recursive merge strategy is not being used there in new ways.
>
> So do it. I still have to hear a single-sentence, clear and obvious
> explanation why it works.
>
> And please do not describe why your original version works, because it
> does not work.

Original [RFC] didn't work because of rather simple mistake that I've
already admitted and fixed. [RFC v2] has got the fix. Read [RFC v2] and
get your facts straight.

> Describe why the one amended with Buga's hack works.

It doesn't matter as these hacks are not needed anymore.

>
>> I honestly don't see any advantages of Phillip's method over the
>> original, except personal preferences. At the same time, I have no
>> objection of using it either, provided consistency check problem is
>> solved there as well.
>
> Okay, let me reiterate then, because I do not want this point to be
> missed:
>
> Phillip's method is essentially merging the new tips into the original
> merge, pretending that the new tips were not rebased but merged into
> upstream.
>
> So it exploits the duality of the rebase and merge operation, which both
> result in identical trees (potentially after resolving merge
> conflicts).
>
> I cannot think of any such interpretation for your proposal augmented by
> Buga's fix-ups. And I haven't heard any such interpretation from your
> side, either.

No fix-ups or augmentations are needed. It was a mistake that has been
fixed in [RFC v2]. You've missed essential part of the discussion.

Read the [RFC v2], please:

Significant changes are:

1. Fixed mistake in the final merge step in the original proposal: wrong
   merge base was used.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> OK, does this mean we want to wait before merging the "recreate
>> merge" topic down to 'next'?  For more than a few weeks, it has been
>> slated for 'next'.
>
> Maybe a few more days.
> ...
> I want to discuss this in the other subthread, though.

If we are talking about a drastic change, a few more days may not be
sufficient, but we are not in a hurry, as this already sounds like a
2.18 material anyway.  As you made it clear that it is OK not to
merge the current one for now, my objective of asking the question
is already satisfied ;-)

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 7 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > How can your approach -- which relies *very much* on having the
> > original parent commits -- not *require* that consistency check?
> 
> I don't understand what you mean, sorry. Could you please point me to
> the *require* you talk about in the original proposal?

Imagine a todo list that contains this line

	merge -C abcdef 123456

and now the user edits it (this is an interactive rebase, after all),
adding another merge head:

	merge -C abcdef 987654 123456

Now your strategy would have a serious problem: to find the original
version of 987654. If there was one.

> > What would your approach (that still has no satisfyingly trivial
> > explanation, in my mind)
> 
> Here is one-liner: rebase sides of the merge commit and then 3-way
> merge them, using original merge commit as merge base.

But I already pointed out how that would undo a commit having been
dropped.

> > do if somebody edited a `merge` command and let it merge a completely
> > unrelated commit?
> 
> Don't see a problem, sorry. The method should still work, provided you have
> original merge commit and two new parents for the new merge.

That is assuming a lot. That is exactly what this consistency check is
for, that I mentioned earlier, and which you listed as a downside of
Phillip's strategy (forgetting that your strategy has the same downside,
so...).

But I guess that you are still talking about the non-interactive version
of the rebase, and missed that our conversation proceeded to the point
where we want that same strategy to work *also* in the interactive version
(and not have a completely different functionality depending whether you
use --interactive or not)?

Ciao,
Johannes

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 7 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > On Wed, 7 Mar 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> 
> >> > On Tue, 6 Mar 2018, Sergey Organov wrote:
> >> >
> >> >> This is v2 of my "Rebasing merges" proposal.
> >> >
> >> > Didn't we settle on Phillip's "perform successive three-way merges
> >> > between the original merge commit and the new tips with the old
> >> > tips as base" strategy?
> >> 
> >> It seems you did, dunno exactly why.
> >
> > That is not true. You make it sound like I was the only one who liked
> > this, and not Phillip and Buga, too.
> >
> > Are you interested in the best solution, or in your solution :-)
> 
> I'm interested in any that works, and only you say that those suggested
> by Phillip is somehow superior. I still believe it's mine that superior,
> even if slightly.

That is misrepresenting what happened.

First, you came up with a strategy. I pointed out shortcomings that
implied that we cannot use it unchanged. Then, Buga fixed your strategy by
using additional steps (making the process more complicated than before,
still without a simple-enough explanation for my liking, fixing the
shortcomings). Then, Phillip presented a super-simple strategy and Buga
confirmed that it also fixes the shortcomings I pointed out.

I am very excited that we finally found something that works *and* is easy
to reason about.

Let's focus on that strategy rather than going back to the strategy which
has known flaws and only an unsatisfyingly complex explanation.

> >> The main problem with this decision is that we still don't see how
> >> and when to stop for user amendment using this method. OTOH, the
> >> original has this issue carefully discussed.
> >
> > Why would we want to stop, unless there are merge conflicts?
> 
> There is somewhat lengthy discussion about it that you probably missed.
> Not to repeat it, just see how 'rerere' works when it fires during
> rebase, even with no conflicts.

I did not miss that discussion. My question was a follow-up: Why would we
want to stop, unless there are merge conflicts?

> >> > It has the following advantages:
> >> >
> >> > - it is *very simple* to describe
> >> 
> >> The original is as simple if not simpler:
> >> 
> >> "rebase sides of the merge commit and then three-way merge them back
> >> using original merge commit as base"
> >
> > And that is also wrong, as I had proved already! Only Buga's addition made
> > it robust against dropping/modifying commits, and that addition also makes
> > it more complicated.
> 
> No. Get your facts straight. The [RFC v2] already fixed that original
> mistake. Could you please finally read it?

I do not see Buga's additions, and it is still a lengthy document that is
hard to understand.

Phillip's alternative, in contrast, fit in at most two 80x25 pages and was
intuitive (at least after seeing that the tree resulting from a merge is
identical to the tree resulting from a rebase, once all merge conflicts
are handled appropriately).

> > And it still has no satisfactory simple explanation why it works.
> 
> It has. It's there in the [RFC v2]. You seem to be the only one who
> doesn't get it. I suppose you just didn't bother to read.

I tried to read it, and got lost in all those figures that really do not
do anything to make this strategy obvious to me.

Granted, I now know *how* it works.

I gave up understanding *why* it is supposed to work.

With Phillip's mail, it only took me 5 minutes to get a rudimentary
understanding why it works.

> >> No problems with octopuses, and no virtual merge bases of recursive
> >> merges to reason about.
> >
> > But those are no problems for Phillip's strategy, either!
> 
> I thought it was you who started to discuss virtual merge bases and
> related problems, as well as how it's difficult to support octopus
> merges, but it's fine with me if there are none of these problems.

Yes, I started explaining virtual merge bases, and how they are used in
the recursive merge. Because I was asked how the recursive merge works,
and I happen to know how it works very intimately.

> > So your point is...?
> 
> Still the same -- use what's better, the [RFC v2].

I strongly disagree that your approach is superior. It is more complex,
and still has no simple answer to the question "why is this supposed to do
what I want it to do?"

It does a lot of criss-crossing, and when I see that, I immediately
suspect that it would lose information e.g. when commits were amended
during the rebase. There are just way too many paths for obsolete changes
to creep in again.

> >> > - it is *very easy* to reason about, once it is pointed out that
> >> > rebases and merges result in the same trees.
> >> 
> >> The original is as easy to reason about, if not easier, especially as
> >> recursive merge strategy is not being used there in new ways.
> >
> > So do it. I still have to hear a single-sentence, clear and obvious
> > explanation why it works.

Please.

> > And please do not describe why your original version works, because it
> > does not work.
> 
> Original [RFC] didn't work because of rather simple mistake that I've
> already admitted and fixed. [RFC v2] has got the fix. Read [RFC v2] and
> get your facts straight.

I don't care how many mistakes you made, and where the original ideas had
to be enhanced. I am only interested in the outcome.

We already have a nice, simple outcome, and to be quite honest: I find
this discussion here a bit pointless. Why would I abandon a simple
strategy that obviously works for a complex strategy where it still is not
obvious under what circumstances it works, and why?

> > Describe why the one amended with Buga's hack works.
> 
> It doesn't matter as these hacks are not needed anymore.

So maybe you want to also describe the "interdiff", and not expect anybody
to read two lengthy documents and figure out where the changes are.

> >> I honestly don't see any advantages of Phillip's method over the
> >> original, except personal preferences. At the same time, I have no
> >> objection of using it either, provided consistency check problem is
> >> solved there as well.
> >
> > Okay, let me reiterate then, because I do not want this point to be
> > missed:
> >
> > Phillip's method is essentially merging the new tips into the original
> > merge, pretending that the new tips were not rebased but merged into
> > upstream.
> >
> > So it exploits the duality of the rebase and merge operation, which both
> > result in identical trees (potentially after resolving merge
> > conflicts).
> >
> > I cannot think of any such interpretation for your proposal augmented by
> > Buga's fix-ups. And I haven't heard any such interpretation from your
> > side, either.
> 
> No fix-ups or augmentations are needed. It was a mistake that has been
> fixed in [RFC v2]. You've missed essential part of the discussion.
> 
> Read the [RFC v2], please:

Okay, I am done here. If all my questions and all my concerns are answered
with the suggestion to spend an hour pouring over a lengthy description of
your strategy that is so much more complex than my preferred alternative,
asking me to figure out from that long document what the answers to my
questions are, then I respectfully decline. I do have other things to care
about, too, and I will *not* spend an hour trying to figure out answers
only because you refuse to give them directly.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Junio,

On Wed, 7 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> OK, does this mean we want to wait before merging the "recreate
> >> merge" topic down to 'next'?  For more than a few weeks, it has been
> >> slated for 'next'.
> >
> > Maybe a few more days.
> > ...
> > I want to discuss this in the other subthread, though.
> 
> If we are talking about a drastic change, a few more days may not be
> sufficient, but we are not in a hurry, as this already sounds like a
> 2.18 material anyway.

It is not at all a drastic change. It will actually make the current patch
series better (simplifying the "can we fast-forward?" check).

I just want to make sure that I already have Phillip's strategy working,
but it will be yet another topic branch on top of the topic branch that
will add support for octopus merges *after* the current --recreate-merges
topic branch ;-)

> As you made it clear that it is OK not to merge the current one for now,
> my objective of asking the question is already satisfied ;-)

Depending how much GitMerge will occupy my time, I hope to have something
polished by tomorrow.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Junio,

On Tue, 6 Mar 2018, Junio C Hamano wrote:

> Phillip Wood <phillip.wood@talktalk.net> writes:
> 
> > I wonder if just having a predicable result rather than forcing the
> > rebase to stop if the user just squashes a fixup commit into a topic
> > branch that is the parent of a merge might be more convenient in
> > practice.
> 
> Unless I am misunderstanding what you are saying, that is pretty much
> what I have automated for my daily rebuild of the 'pu' branch
> 
> Non-textual semantic conflicts are made (in the best case just once)
> as a separate commit on top of mechanical auto-merge whose focus is
> predictability (rather than cleverness) done by Git, and then that
> separate commit is kept outside the history.  When replaying these
> merges to rebuild the 'pu' branch, after resetting the tip to
> 'master', each topic is merged mechanically, and if such a fix-up
> commit is present, "cherry-pick --no-commit" applies it and then
> "commit --amend --no-edit" to adjust the merge.  I find it quite
> valuable to have a separate record of what "evil" non-mechanical
> adjustment was done, which I know won't be lost in the noise when
> these merges need to be redone daily or more often.

So essentially, you have something that `git rerere` would have learned,
but as a commit?

Might make sense to make that procedure more accessible to others, too ;-)
I know that *I* would use it.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> If we are talking about a drastic change, a few more days may not be
>> sufficient, but we are not in a hurry, as this already sounds like a
>> 2.18 material anyway.
>
> It is not at all a drastic change. It will actually make the current patch
> series better (simplifying the "can we fast-forward?" check).
>
> I just want to make sure that I already have Phillip's strategy working,
> but it will be yet another topic branch on top of the topic branch that
> will add support for octopus merges *after* the current --recreate-merges
> topic branch ;-)

Oh, if the "not redoing the merge afresh, but attempt to reuse the
previous merge" that was discussed is going to be done as an
update/addition to the "redo the merge afresh" you already had in
production forever (and I had in 'pu' for quite a while in various
polished-ness during iteration), then I do prefer merging down what
has already proven to be 'next' worthy without waiting for the
discussion and your local verification of Phillip's new thing,
especially given that you'll be giving an explicit control to the
users which variant of "merge" insn will be used and the addition
of the Phillip's thing won't be a backward-compatibility issue when
it comes later.

>> As you made it clear that it is OK not to merge the current one for now,
>> my objective of asking the question is already satisfied ;-)
>
> Depending how much GitMerge will occupy my time, I hope to have something
> polished by tomorrow.

My "for now" above was just for the coming few days.  Don't rush
things, and use valuable in-person time wisely, and have fun.

Thanks.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

>> Non-textual semantic conflicts are made (in the best case just once)
>> as a separate commit on top of mechanical auto-merge whose focus is
>> predictability (rather than cleverness) done by Git, and then that
>> separate commit is kept outside the history.  When replaying these
>> merges to rebuild the 'pu' branch, after resetting the tip to
>> 'master', each topic is merged mechanically, and if such a fix-up
>> commit is present, "cherry-pick --no-commit" applies it and then
>> "commit --amend --no-edit" to adjust the merge.  I find it quite
>> valuable to have a separate record of what "evil" non-mechanical
>> adjustment was done, which I know won't be lost in the noise when
>> these merges need to be redone daily or more often.
>
> So essentially, you have something that `git rerere` would have learned,
> but as a commit?

You probably wouldn't be asking that if you read what you cut out
when you quoted above ;-) 

There are a collection of cherry-pickable commits in hierarchy under
refs/merge-fix.  They are indexed by the branch that will cause
semantic conflicts that do not involve textual conflicts at all (the
important implication of which is that 'rerere' fundamentally will
not trigger to help resolving them) [*1*], and are used to create
evil merge when a corresponding branch is merged to 'pu' (and down).

[Footnote]

*1* One topic adds an extra parameter to read_index_from() that has
been and still is defined in a file and merged to 'pu' first, while
another topic adds a new callsite for the same function in a file
that the former topic does not touch, hence a merge of the latter
topic has no textual conflict to the file with a new callsite, but
still needs adjusting.  That sort of think.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Phillip Wood]

On 06/03/18 18:12, Johannes Schindelin wrote:
> Hi Phillip,
> 
> On Tue, 6 Mar 2018, Phillip Wood wrote:
> 
>> On 03/03/18 00:29, Igor Djordjevic wrote:
>>>
>>> On 02/03/2018 12:31, Phillip Wood wrote:
>>>>
>>>>> Thinking about it overnight, I now suspect that original proposal
>>>>> had a mistake in the final merge step. I think that what you did is
>>>>> a way to fix it, and I want to try to figure what exactly was wrong
>>>>> in the original proposal and to find simpler way of doing it right.
>>>>>
>>>>> The likely solution is to use original UM as a merge-base for final
>>>>> 3-way merge of U1' and U2', but I'm not sure yet. Sounds pretty
>>>>> natural though, as that's exactly UM from which both U1' and U2'
>>>>> have diverged due to rebasing and other history editing.
>>>>
>>>> Hi Sergey, I've been following this discussion from the sidelines,
>>>> though I haven't had time to study all the posts in this thread in
>>>> detail. I wonder if it would be helpful to think of rebasing a merge
>>>> as merging the changes in the parents due to the rebase back into the
>>>> original merge. So for a merge M with parents A B C that are rebased
>>>> to A' B' C' the rebased merge M' would be constructed by (ignoring
>>>> shell quoting issues)
>>>>
>>>> git checkout --detach M
>>>> git merge-recursive A -- M A'
>>>> tree=$(git write-tree)
>>>> git merge-recursive B -- $tree B'
>>>> tree=$(git write-tree)
>>>> git merge-recursive C -- $tree C'
>>>> tree=$(git write-tree)
>>>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>>>>
>>>> This should pull in all the changes from the parents while preserving
>>>> any evil conflict resolution in the original merge. It superficially
>>>> reminds me of incremental merging [1] but it's so long since I looked at
>>>> that I'm not sure if there are any significant similarities.
>>>>
>>>> [1] https://github.com/mhagger/git-imerge
>>>
>>> Interesting, from quick test[3], this seems to produce the same 
>>> result as that other test I previously provided[2], where temporary 
>>> commits U1' and U2' are finally merged with original M as a base :)
>>>
>>> Just that this looks like even more straight-forward approach...?
>>>
>>> The only thing I wonder of here is how would we check if the 
>>> "rebased" merge M' was "clean", or should we stop for user amendment? 
>>> With that other approach Sergey described, we have U1'==U2' to test with.
>>
>> I think (though I haven't rigorously proved to myself) that in the
>> absence of conflicts this scheme has well defined semantics (the merges
>> can be commuted), so the result should be predicable from the users
>> point of view so maybe it could just offer an option to stop.
> 
> I am not so sure that the result is independent of the order of the
> merges. In other words, I am not necessarily certain that it is impossible
> to concoct A,A',B,B' commits where merging B'/B before A'/A has a
> different result than merging A'/A before B'/B.
> 
> Remember, when constructing counter-examples to hypotheses, those
> counter-examples do not really *have* to make sense on their own. For
> example, A' could introduce *completely different* changes from A, and the
> same is true for B' and B.
> 
> I could imagine, for example, that using a ton of consecutive empty lines,
> and using patches that insert something into these empty lines (and are
> thusly inherently ambiguous when said set of empty lines has changed),
> could even introduce a merge conflict in one order, but no conflict in the
> other.

Yes I should have thought of that given I've just been working on making
'add -p' more robust when there are lots of identical lines.

> Even so, I think that merging in the order of the parents makes the most
> sense, and that using that strategy makes sense, too, because you really
> have to try hard to make it fail.
> 
> Ciao,
> Dscho
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Phillip Wood]

On 07/03/18 07:26, Johannes Schindelin wrote:
> Hi Buga,
> 
> On Tue, 6 Mar 2018, Igor Djordjevic wrote:
> 
>> On 06/03/2018 19:12, Johannes Schindelin wrote:
>>>
>>>>> And I guess being consistent is pretty important, too - if you add new
>>>>> content during merge rebase, it should always show up in the merge,
>>>>> period. 
>>>>
>>>> Yes, that should make it easy for the user to know what to expect from
>>>> rebase.
>>>
>>> [...]
>>>
>>> It will be slightly inconsistent. But in a defendable way, I think.
>>
>> I like where this discussion is heading, and here`s what I thought 
>> about it :)
>>
>> [...]
>>
>> Here`s a twist - not letting `merge` trying to be too smart by 
>> figuring out whether passed arguments correspond to rewritten 
>> versions of the original merge parents (which would be too 
>> restrictive, too, I`m afraid), but just be explicit about it, instead!
> 
> That's the missing piece, I think.
> 
>> So, it could be something like:
>>
>> 	merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed
> 
> I like where this is heading, too, but I do not think that we can do this
> on a per-MERGE_HEAD basis. The vast majority of merge commits, in
> practice, have two parents. So the `merge` command would actually only
> have one revision to merge (because HEAD is the implicit first parent). So
> that is easy.
> 
> But as soon as you go octopus, you can either perform an octopus merge, or
> rebase the original merge commit. You cannot really mix and match here.
> 
> Unless we reimplement the octopus merge (which works quite a bit
> differently from the "rebase merge commit" strategy, even if it is
> incremental, too), which has its own challenges: if there are merge
> conflicts before merging the last MERGE_HEAD, the octopus merge will exit
> with status 2, telling you "Should not be doing an octopus.". While we
> will want to keep merge conflict markers and continue with the "rebase the
> original merge commit" strategy.
> 
> And it would slam the door shut for adding support for *other* merge
> strategies to perform a more-than-two-parents merge.
> 
> Also, I do not think that it makes a whole lot of sense in practice to let
> users edit what will be used for "original parent". If the user wants to
> do complicated stuff, they can already do that, via `exec`. The `merge`
> command really should be about facilitating common workflows, guiding the
> user to what is sane.
> 
> Currently my favorite idea is to introduce a new flag: -R (for "rebase the
> original merge commit"). It would look like this:
> 
> 	merge -R -C <original-merge> <merge-head> # <oneline>
>
> This flag would of course trigger the consistency check (does the number
> of parents of the original merge commit agree with the parameter list? Was
> an original merge commit specified to begin with?), and it would not fall
> back to the recursive merge, but error out if that check failed.
> 
> Side note: I wonder whether we really need to perform the additional check
> that ensures that the <merge-head> refers to the rewritten version of the
> original merge commit's parent.
> 
> Second side note: if we can fast-forward, currently we prefer that, and I
> think we should keep that behavior with -R, too.

I think that would be a good idea to avoid unpleasant surprises.

> If the user wants to force a new merge, they simply remove that -R flag.
> 
> What do you think?

I did wonder about using 'pick <original-merge>' for rebasing merges and
keeping 'merge ...' for recreating them but I'm not sure if that is a
good idea. It has the advantage that the user cannot specify the wrong
parents for the merge to be rebased as 'git rebase' would work out if
the parents have been rebased, but maybe it's a bit magical to use pick
for merge commits. Also there isn't such a simple way for the user to go
from 'rabase this merge' to 'recreate this merge' as they'd have to
write the whole merge line themselves (though I guess something like
emacs' git-rebase.el would be able to help with that)

Best Wishes

Phillip


> Ciao,
> Dscho
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Phillip Wood]

On 08/03/18 11:20, Phillip Wood wrote:
> On 07/03/18 07:26, Johannes Schindelin wrote:
>> Hi Buga,
>>
>> On Tue, 6 Mar 2018, Igor Djordjevic wrote:
>>
>>> On 06/03/2018 19:12, Johannes Schindelin wrote:
>>>>
>>>>>> And I guess being consistent is pretty important, too - if you add new
>>>>>> content during merge rebase, it should always show up in the merge,
>>>>>> period. 
>>>>>
>>>>> Yes, that should make it easy for the user to know what to expect from
>>>>> rebase.
>>>>
>>>> [...]
>>>>
>>>> It will be slightly inconsistent. But in a defendable way, I think.
>>>
>>> I like where this discussion is heading, and here`s what I thought 
>>> about it :)
>>>
>>> [...]
>>>
>>> Here`s a twist - not letting `merge` trying to be too smart by 
>>> figuring out whether passed arguments correspond to rewritten 
>>> versions of the original merge parents (which would be too 
>>> restrictive, too, I`m afraid), but just be explicit about it, instead!
>>
>> That's the missing piece, I think.
>>
>>> So, it could be something like:
>>>
>>> 	merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed
>>
>> I like where this is heading, too, but I do not think that we can do this
>> on a per-MERGE_HEAD basis. The vast majority of merge commits, in
>> practice, have two parents. So the `merge` command would actually only
>> have one revision to merge (because HEAD is the implicit first parent). So
>> that is easy.
>>
>> But as soon as you go octopus, you can either perform an octopus merge, or
>> rebase the original merge commit. You cannot really mix and match here.
>>
>> Unless we reimplement the octopus merge (which works quite a bit
>> differently from the "rebase merge commit" strategy, even if it is
>> incremental, too), which has its own challenges: if there are merge
>> conflicts before merging the last MERGE_HEAD, the octopus merge will exit
>> with status 2, telling you "Should not be doing an octopus.". While we
>> will want to keep merge conflict markers and continue with the "rebase the
>> original merge commit" strategy.
>>
>> And it would slam the door shut for adding support for *other* merge
>> strategies to perform a more-than-two-parents merge.
>>
>> Also, I do not think that it makes a whole lot of sense in practice to let
>> users edit what will be used for "original parent". If the user wants to
>> do complicated stuff, they can already do that, via `exec`. The `merge`
>> command really should be about facilitating common workflows, guiding the
>> user to what is sane.
>>
>> Currently my favorite idea is to introduce a new flag: -R (for "rebase the
>> original merge commit"). It would look like this:
>>
>> 	merge -R -C <original-merge> <merge-head> # <oneline>
>>
>> This flag would of course trigger the consistency check (does the number
>> of parents of the original merge commit agree with the parameter list? Was
>> an original merge commit specified to begin with?), and it would not fall
>> back to the recursive merge, but error out if that check failed.
>>
>> Side note: I wonder whether we really need to perform the additional check
>> that ensures that the <merge-head> refers to the rewritten version of the
>> original merge commit's parent.
>>
>> Second side note: if we can fast-forward, currently we prefer that, and I
>> think we should keep that behavior with -R, too.
> 
> I think that would be a good idea to avoid unpleasant surprises.

Oops that was referring to the first side note. I think fast forwarding
is a good idea. I'm not so sure about checking that <merge-head> refers
to the rewritten version of the original merge commit's parent any more
though. Having thought some more, I think we would want to allow the
user to rearrange a topic branch that is the parent of a merge and that
would require allowing a different parent as the old parent could be
dropped or swapped with another commit in the branch. I can't think of a
way to mechanically check that the new parent is 'somehow derived from'
the old one.

>> If the user wants to force a new merge, they simply remove that -R flag.
>>
>> What do you think?
> 
> I did wonder about using 'pick <original-merge>' for rebasing merges and
> keeping 'merge ...' for recreating them but I'm not sure if that is a
> good idea. It has the advantage that the user cannot specify the wrong
> parents for the merge to be rebased as 'git rebase' would work out if
> the parents have been rebased, but maybe it's a bit magical to use pick
> for merge commits. Also there isn't such a simple way for the user to go
> from 'rabase this merge' to 'recreate this merge' as they'd have to
> write the whole merge line themselves (though I guess something like
> emacs' git-rebase.el would be able to help with that)

Scrub that, it is too magical and I don't think it would work with
rearranged commits - it's making the --preserve-merges mistake all over
again. It's a shame to have 'merge' mean 'recreate the merge' and
'rebase the merge' but I don't think there is an easy way round that.

> Best Wishes
> 
> Phillip
> 
> 
>> Ciao,
>> Dscho
>>
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Dscho,

On 07/03/2018 08:26, Johannes Schindelin wrote:
> 
> > So, it could be something like:
> >
> > 	merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed
> 
> I like where this is heading, too, but I do not think that we can do this
> on a per-MERGE_HEAD basis. The vast majority of merge commits, in
> practice, have two parents. So the `merge` command would actually only
> have one revision to merge (because HEAD is the implicit first parent). So
> that is easy.
> 
> But as soon as you go octopus, you can either perform an octopus merge, or
> rebase the original merge commit. You cannot really mix and match here.
> 
> Unless we reimplement the octopus merge (which works quite a bit
> differently from the "rebase merge commit" strategy, even if it is
> incremental, too), which has its own challenges: if there are merge
> conflicts before merging the last MERGE_HEAD, the octopus merge will exit
> with status 2, telling you "Should not be doing an octopus.". While we
> will want to keep merge conflict markers and continue with the "rebase the
> original merge commit" strategy.
> 
> And it would slam the door shut for adding support for *other* merge
> strategies to perform a more-than-two-parents merge.

The thing is, in my opinion, as long as we are _rebasing_, you can`t 
pick any merge strategy, as it doesn`t really make much sense. If you 
do want a specific strategy, than that`s _recreating_ a merge, and it 
goes fine with what you already have for `--recreate-merges`.

On merge rebasing, the underline strategy we decide to use is just an 
implementation detail, picking the one that works best (or the only 
one that works, even), user should have nothing to do with it.

> Also, I do not think that it makes a whole lot of sense in practice to let
> users edit what will be used for "original parent". If the user wants to
> do complicated stuff, they can already do that, via `exec`. The `merge`
> command really should be about facilitating common workflows, guiding the
> user to what is sane.

I thought of a situation like this:

(1) ---o---o---o---M--- (master)
        \         /
         X1--X2--X3 (topic)


Merge M was done with `-s ours`, obsoleting "topic" branch. But, I 
later realized that I actually do want that X2 commit in master.

Now, I guess the most obvious approach is just cherry-picking it, but 
what if I would like to do something like this instead, with an 
interactive rebase (and rebasing the merge, not recreating it):

(2) ---o---o---o---M'--- (master)
       |\         /|
       | X1'-X3'-/ | (topic)
       |           |
       \--X2'------/ (new)


This way, and having "topic" inherit original merge behavior due to 
merge rebasing, X1' and X3' would still be missing from M' as they 
did originally from M, but X2' would now be included, as it`s coming 
from a new branch, forged during interactive rebase.

(note - implementation wise, this still wouldn`t be an octopus merge ;)

> The vast majority of merge commits, in practice, have two parents. So
> the `merge` command would actually only have one revision to merge
> (because HEAD is the implicit first parent).

Now, this is something I actually overlooked :( 

I guess order of parent commits could then be used to map to old 
commit parents, being a limitation in comparison to direct 
old-parent:new-parent mapping, but might be a more straightforward 
user experience...

Though in case of octopus merge, where one would like to drop a 
branch from the middle, being merged with `-s ours`, that would be 
impossible, as then the next branch would be taking over dropped 
branch merge parent, yielding an incorrect result.

So in this case:

(3) ---o---o---o---M--- (master)
       |\         /|
       | X1--X3--/ | (topic)
       |           |
       \--X2-------/ (new)

... where "topic" was merged using `-s ours`, we wouldn`t be able to 
just remove whole "topic" branch from the rebased merge without 
influencing it incorrectly.

With (any kind of) explicit old-parent:new-parent mapping, this is 
possible (and shouldn`t be any harder, implementation wise).

Now, it`s a different story if we`re interested in such exotic 
scenarios in the first place, but if possible, I would be all for 
it... :)

> Currently my favorite idea is to introduce a new flag: -R (for "rebase the
> original merge commit"). It would look like this:
> 
> 	merge -R -C <original-merge> <merge-head> # <oneline>
> 
> This flag would of course trigger the consistency check (does the number
> of parents of the original merge commit agree with the parameter list? Was
> an original merge commit specified to begin with?), and it would not fall
> back to the recursive merge, but error out if that check failed.
> 
> Side note: I wonder whether we really need to perform the additional check
> that ensures that the <merge-head> refers to the rewritten version of the
> original merge commit's parent.

No, and even worse - I think we must not do that, as that merge 
parent might be moved elsewhere, which should be allowed.

When I say "merge parent", I really mean "merge parent _branch_" (or 
so to say, merge parent context), I was deliberately avoiding term 
"merge parent commit" as it`s more than that.

> Second side note: if we can fast-forward, currently we prefer that, and I
> think we should keep that behavior with -R, too.

I agree.

> If the user wants to force a new merge, they simply remove that -R flag.
> 
> What do you think?

Having more replies in the thread, I`ll comment the format there, 
might be more appropriate, as discussion already evolved since the 
last time I`ve checked, and I might have a new idea... ;) :P

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Phillip and Johannes, 

On 08/03/2018 12:20, Phillip Wood wrote:
> 
> I did wonder about using 'pick <original-merge>' for rebasing merges and
> keeping 'merge ...' for recreating them but I'm not sure if that is a
> good idea. It has the advantage that the user cannot specify the wrong
> parents for the merge to be rebased as 'git rebase' would work out if
> the parents have been rebased, but maybe it's a bit magical to use pick
> for merge commits. Also there isn't such a simple way for the user to go
> from 'rabase this merge' to 'recreate this merge' as they'd have to
> write the whole merge line themselves (though I guess something like
> emacs' git-rebase.el would be able to help with that)

Hmm, funny enough, `pick <original merge>` was something I though 
about originally, too, feeling that it might make more sense in terms 
on what`s really going on, but I guess I wanted it incorporated into 
`--recreate-merges` too much that I tried really hard to fit it in, 
without changing it much :/

And now that I said this in a previous reply:

> The thing is, in my opinion, as long as we are _rebasing_, you can`t 
> pick any merge strategy, as it doesn`t really make much sense. If you 
> do want a specific strategy, than that`s _recreating_ a merge, and it 
> goes fine with what you already have for `--recreate-merges`.
> 
> On merge rebasing, the underline strategy we decide to use is just an 
> implementation detail, picking the one that works best (or the only 
> one that works, even), user should have nothing to do with it.

The difference between "rebase merge commit" and "recreate merge 
commit" might starting to be more evident.

So... I might actually go for this one now. And (trying to stick with 
explicit mappings, still :P), now that we`re not married to `merge` 
expectations a user may already have, maybe a format like this:

  pick <original-merge> <original-parent1>:HEAD <original-parent2>:<new-parent2>


Here, original-merge is a _commit_, where original-parent and 
new-parent are _labels_ (in terms of `--recreate-merges`).

Everything else I previously said still holds - one is allowed to 
change or drop mappings, and add or drop new merge parents. Yes, in 
case user does something "stupid", he`ll get a lot of conflicts, but 
hey, we shouldn`t judge.

p.s. Are we moving towards `--rebase-merges` I mentioned in that 
other topic[1], as an add-on series after `--recreate-merges` hits 
the mainstream (as-is)...? :P

Regards, Buga

[1] https://public-inbox.org/git/bc9f82fb-fd18-ee45-36a4-921a1381b32e@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 08/03/2018 13:16, Phillip Wood wrote:
> 
> > Side note: I wonder whether we really need to perform the additional check
> > that ensures that the <merge-head> refers to the rewritten version of the
> > original merge commit's parent.
> > 
> > [...]
> 
> Oops that was referring to the first side note. I think fast forwarding
> is a good idea. I'm not so sure about checking that <merge-head> refers
> to the rewritten version of the original merge commit's parent any more
> though. Having thought some more, I think we would want to allow the
> user to rearrange a topic branch that is the parent of a merge and that
> would require allowing a different parent as the old parent could be
> dropped or swapped with another commit in the branch. I can't think of a
> way to mechanically check that the new parent is 'somehow derived from'
> the old one.

Exactly, we must not depend on exact parent commits, but on parent 
"branches" (so to say).

And that is why I think explicit mapping would be pretty helpful (if 
not the only approach).

> > I did wonder about using 'pick <original-merge>' for rebasing merges and
> > keeping 'merge ...' for recreating them but I'm not sure if that is a
> > good idea. It has the advantage that the user cannot specify the wrong
> > parents for the merge to be rebased as 'git rebase' would work out if
> > the parents have been rebased, but maybe it's a bit magical to use pick
> > for merge commits. Also there isn't such a simple way for the user to go
> > from 'rabase this merge' to 'recreate this merge' as they'd have to
> > write the whole merge line themselves (though I guess something like
> > emacs' git-rebase.el would be able to help with that)
> 
> Scrub that, it is too magical and I don't think it would work with
> rearranged commits - it's making the --preserve-merges mistake all over
> again. It's a shame to have 'merge' mean 'recreate the merge' and
> 'rebase the merge' but I don't think there is an easy way round that.

I actually like `pick` for _rebasing_ merge commits, as `pick` is 
already used for rebasing non-merge commits, too, so it feels natural.

Then `merge` is left to do what it is meant for - merging (or 
"recreate the merge", in the given context).

I tried to outline a possible user interface in that other reply[1], 
elaborating it a bit, too,

Regards, Buga

[1] https://public-inbox.org/git/f3872fb9-01bc-b2f1-aee9-cfc0e4db77d6@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Jacob Keller]

On Thu, Mar 8, 2018 at 3:20 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> On 07/03/18 07:26, Johannes Schindelin wrote:
>> Hi Buga,
>>
>> On Tue, 6 Mar 2018, Igor Djordjevic wrote:
>>
>>> On 06/03/2018 19:12, Johannes Schindelin wrote:
>>>>
>>>>>> And I guess being consistent is pretty important, too - if you add new
>>>>>> content during merge rebase, it should always show up in the merge,
>>>>>> period.
>>>>>
>>>>> Yes, that should make it easy for the user to know what to expect from
>>>>> rebase.
>>>>
>>>> [...]
>>>>
>>>> It will be slightly inconsistent. But in a defendable way, I think.
>>>
>>> I like where this discussion is heading, and here`s what I thought
>>> about it :)
>>>
>>> [...]
>>>
>>> Here`s a twist - not letting `merge` trying to be too smart by
>>> figuring out whether passed arguments correspond to rewritten
>>> versions of the original merge parents (which would be too
>>> restrictive, too, I`m afraid), but just be explicit about it, instead!
>>
>> That's the missing piece, I think.
>>
>>> So, it could be something like:
>>>
>>>      merge -C deadbee 123abc:cafecafe 234bcd:bedbedbed
>>
>> I like where this is heading, too, but I do not think that we can do this
>> on a per-MERGE_HEAD basis. The vast majority of merge commits, in
>> practice, have two parents. So the `merge` command would actually only
>> have one revision to merge (because HEAD is the implicit first parent). So
>> that is easy.
>>
>> But as soon as you go octopus, you can either perform an octopus merge, or
>> rebase the original merge commit. You cannot really mix and match here.
>>
>> Unless we reimplement the octopus merge (which works quite a bit
>> differently from the "rebase merge commit" strategy, even if it is
>> incremental, too), which has its own challenges: if there are merge
>> conflicts before merging the last MERGE_HEAD, the octopus merge will exit
>> with status 2, telling you "Should not be doing an octopus.". While we
>> will want to keep merge conflict markers and continue with the "rebase the
>> original merge commit" strategy.
>>
>> And it would slam the door shut for adding support for *other* merge
>> strategies to perform a more-than-two-parents merge.
>>
>> Also, I do not think that it makes a whole lot of sense in practice to let
>> users edit what will be used for "original parent". If the user wants to
>> do complicated stuff, they can already do that, via `exec`. The `merge`
>> command really should be about facilitating common workflows, guiding the
>> user to what is sane.
>>
>> Currently my favorite idea is to introduce a new flag: -R (for "rebase the
>> original merge commit"). It would look like this:
>>
>>       merge -R -C <original-merge> <merge-head> # <oneline>
>>
>> This flag would of course trigger the consistency check (does the number
>> of parents of the original merge commit agree with the parameter list? Was
>> an original merge commit specified to begin with?), and it would not fall
>> back to the recursive merge, but error out if that check failed.
>>
>> Side note: I wonder whether we really need to perform the additional check
>> that ensures that the <merge-head> refers to the rewritten version of the
>> original merge commit's parent.
>>
>> Second side note: if we can fast-forward, currently we prefer that, and I
>> think we should keep that behavior with -R, too.
>
> I think that would be a good idea to avoid unpleasant surprises.
>
>> If the user wants to force a new merge, they simply remove that -R flag.
>>
>> What do you think?
>
> I did wonder about using 'pick <original-merge>' for rebasing merges and
> keeping 'merge ...' for recreating them but I'm not sure if that is a
> good idea. It has the advantage that the user cannot specify the wrong
> parents for the merge to be rebased as 'git rebase' would work out if
> the parents have been rebased, but maybe it's a bit magical to use pick
> for merge commits. Also there isn't such a simple way for the user to go
> from 'rabase this merge' to 'recreate this merge' as they'd have to
> write the whole merge line themselves (though I guess something like
> emacs' git-rebase.el would be able to help with that)
>
> Best Wishes
>
> Phillip
>

Since the ultimate commit hashes of newly rebased commits would be
unknown at the time of writing the todo file, I'm not sure how this
would work to specify the parents?

>
>> Ciao,
>> Dscho
>>
>

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 08/03/2018 16:16, Igor Djordjevic wrote:
> 
> > Unless we reimplement the octopus merge (which works quite a bit
> > differently from the "rebase merge commit" strategy, even if it is
> > incremental, too), which has its own challenges: if there are merge
> > conflicts before merging the last MERGE_HEAD, the octopus merge will exit
> > with status 2, telling you "Should not be doing an octopus.". While we
> > will want to keep merge conflict markers and continue with the "rebase the
> > original merge commit" strategy.
> >
> > [...]
> 
> The thing is, in my opinion, as long as we are _rebasing_, you can`t 
> pick any merge strategy, as it doesn`t really make much sense. If you 
> do want a specific strategy, than that`s _recreating_ a merge, and it 
> goes fine with what you already have for `--recreate-merges`.
> 
> On merge rebasing, the underlying strategy we decide to use is just an 
> implementation detail, picking the one that works best (or the only 
> one that works, even), user should have nothing to do with it.

Just to add, if not already assumable, that I think we should stop 
and let user react on conflicts on each of the "rebase the original 
commit" strategy steps (rebase first parent, rebase second parent... 
merge parents).

I guess this stresses not using real "octopus merge" strategy even in 
case where we`re rebasing octopus merge commit even more (and aligns 
nicely with what you seem to expect already).

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 06/03/2018 12:45, Sergey Organov wrote:
> 
> > > The only thing I wonder of here is how would we check if the 
> > > "rebased" merge M' was "clean", or should we stop for user amendment? 
> > > With that other approach Sergey described, we have U1'==U2' to test with.
> >
> > I think (though I haven't rigorously proved to myself) that in the
> > absence of conflicts this scheme has well defined semantics (the merges
> > can be commuted), so the result should be predicable from the users
> > point of view so maybe it could just offer an option to stop.
> 
> Yes, hopefully it's predictable, but is it the intended one? We don't
> know, so there is still some level of uncertainty.
> 
> When in doubt, I try to find similar cases. There are two I'm aware of:
> 
> 1. "git merge" just commits the result when there are no conflicts.
> However, it supposedly has been run by the user just now, and thus user
> can amend what he gets. That's effectively a stop for amendment from our
> POV.
> 
> 2. When rebasing, "rerere", when fires, stages the changes, and rebasing
> stops for amendment. For me "rerere" behavior is rather annoying (I've
> never in fact amended what it prepared), but I always assumed there are
> good reasons it behaves this way.
> 
> Overall, to be consistent, it seems we do need to stop at U1' != U2', at
> least by default. Additional options could be supported then to specify
> user intentions, both on the command level and in the todo list,
> provided it proves to be useful.

Just to say I agree with this, `if U1' == U2' then proceed else stop` 
seems as a good sanity check.

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Johannes,

On 07/03/2018 15:08, Johannes Schindelin wrote:
> 
> > > Didn't we settle on Phillip's "perform successive three-way merges
> > > between the original merge commit and the new tips with the old tips
> > > as base" strategy?
> >
> > It seems you did, dunno exactly why.
> 
> That is not true. You make it sound like I was the only one who liked
> this, and not Phillip and Buga, too.

For myself, I do actually favor Sergey`s approach in general, but 
_implemented_ through what Phillip described (or a mixture of both, to 
be precise). But, let me explain... :)

> > The main problem with this decision is that we still don't see how and
> > when to stop for user amendment using this method. OTOH, the original
> > has this issue carefully discussed.
> 
> Why would we want to stop, unless there are merge conflicts?

Because we can reliably know that something "unusual" happened - and 
by that I don`t necessarily mean "wrong", but just might be worth 
user inspection.

For example, situation like this (M is made on A3 with `-s ours`, 
obsoleting Bx commits):

(1) ---X8--X9 (master)
       |\
       | A1---A2---A3
       |             \
       |              M (topic)
       |             /
       \-B1---B2---B3

... where we want to rebase M onto X9 is what I would call "usual 
stuff", but this situation (M is still made on A3 with `-s ours`, 
obsoleting Bx commits, but note cherry-picked B2'):

(2) ---X8--B2'--X9 (master)
       |\
       | A1---A2---A3
       |             \
       |              M (topic)
       |             /
       \-B1---B2---B3

... where we still want to rebase M onto X9 is what we might consider 
"unusual", because we noticed that something that shouldn`t be part 
of the rebased merge commit (due to previous `-s ours`) actually got 
in there (due to later cherry-pick), and just wanting the user to 
check and confirm.

This is the major reason why I would prefer Sergey`s approach in 
general... and might be I also have a good explanation on *why* it 
works, but let`s get there ;)

(p.s. This is only one, but certainly not the only case)

> > "rebase sides of the merge commit and then three-way merge them back
> > using original merge commit as base"
> 
> And that is also wrong, as I had proved already! Only Buga's addition made
> it robust against dropping/modifying commits, and that addition also makes
> it more complicated.

No, this is actually right, that sentence nicely describing _how_ it 
works. That addition of mine was just including the correct merge 
base (being the original merge commit that we are "rebasing"), and 
that`s what Sergey is talking about.

> And it still has no satisfactory simple explanation why it works.

Getting there... :)

> > > - it is *very easy* to reason about, once it is pointed out that
> > > rebases and merges result in the same trees.
> >
> > The original is as easy to reason about, if not easier, especially as
> > recursive merge strategy is not being used there in new ways.
> 
> So do it. I still have to hear a single-sentence, clear and obvious
> explanation why it works.
> 
> And please do not describe why your original version works, because it
> does not work. Describe why the one amended with Buga's hack works.
> 
> > I honestly don't see any advantages of Phillip's method over the
> > original, except personal preferences. At the same time, I have no
> > objection of using it either, provided consistency check problem is
> > solved there as well.
> 
> Okay, let me reiterate then, because I do not want this point to be
> missed:
> 
> Phillip's method is essentially merging the new tips into the original
> merge, pretending that the new tips were not rebased but merged into
> upstream.
> 
> So it exploits the duality of the rebase and merge operation, which both
> result in identical trees (potentially after resolving merge conflicts).
> 
> I cannot think of any such interpretation for your proposal augmented by
> Buga's fix-ups. And I haven't heard any such interpretation from your
> side, either.

Ok here it goes (I might be still wrong, but please bare with me).

What Sergey originally described also uses the "duality of the rebase 
and merge operation", too ;) Just in a bit different way (and might 
be a more straightforward one?).

Here`s a starting point, two commits A and B, merged into M:

(3) ---A
        \
         M
        /
    ---B


According the "patch theory"[1] (which might not be too popular 
around here, but should serve the purpose for what I`m trying to 
explain), each merge commit can be "transformed" to two non-merge 
commits, one on top of each of the merge parents, where new commit 
brings its original merge parent commit tree to the state of the 
merge commit tree:

(4) ---A---U1



    ---B---U2


Now, we have two new commits, U1 and U2, each having the same tree as 
previous merge commit M, but representing changes in regards to 
specific parents - and this is essentially what Sergey`s original 
approach was using (whether he knew it, or not).

When it comes to rebasing, it`s pretty simple, too. As this:

(5) ---X1---o---o---o---o---o---X2 (master)
       |\
       | A1---A2---A3
       |             \
       |              M
       |             /
       \-B1---B2---B3

... actually equals this:

(6) ---X1---o---o---o---o---o---X2 (master)
       |\
       | A1---A2---A3---U1
       |
       |
       |
       \-B1---B2---B3---U2

... where trees of M, U1 and U2 are same, and we can use the regular 
rebase semantics and rebase it to this:

(7) ---X1---o---o---o---o---o---X2 (master)
                                |\
                                | A1'--A2'--A3'--U1'
                                |
                                |
                                |
                                \-B1'--B2'--B3'--U2'

... which is essentially this again:

(8) ---X1---o---o---o---o---o---X2 (master)
                                |\
                                | A1'--A2'--A3'
                                |            \
                                |             M'
                                |            /
                                \-B1'--B2'--B3'

... where it is still true that trees of U1', U2' and M' are still 
the same. So we managed to rebase a merge commit without ever doing a 
merge :) (note that, practically, we _can_ finally even merge U1' and 
U2' to produce M', it shouldn`t really matter as all the trees are 
the same, so merge will be a no-op)

But, as we saw (what you`ve explained), this doesn`t really work in 
case one of the sides of the merge gets "disbalanced" (so to say), 
like dropping a commit (which could also happen non-interactively, 
where a commit has been cherry-picked to a different branch, but
previously obsoleted by `-s ours` merge).

As observed, dropped commit could still wrongly get into final merge 
commit tree (or cherry-picked commit wrongly not get there), due to 
the nature of those rebased U1 and U2 temporary commits to hold all 
the differences in regards to their specific merge commit parent.

A natural improvement to original idea which Sergey eventually came 
up with after my findings (which you now ended up calling a hack, even, 
but I would respectfully disagree), is to use original merge commit M 
as a merge base for final merge of U1' and U2' - and here is why it 
works, too, and why I wouldn`t consider it a hack, but a proper (and 
a solid) solution.

Merge commit M is what we are rebasing, so we should have a way to 
somehow learn from it, alongside U1 and U2 temporary commits - and 
that is what using original merge commit M as a base does for us. It 
helps us spot any "disbalances" that happened in comparison to original 
merge parent trees, which is exactly what we`re interested in.

In ideal case (as per "patch theory"[1]), final merge of rebased U1' 
and U2' with original merge commit M as a base would be rather simple, 
too (not to say pointless again), as both U1' and U2' would have same 
changes in comparison to M, namely all the changes from commit we are 
rebasing from (X1 above) to commit we are rebasing to (X2 above).

But in a more complex case like this:

(9) ---X1---B2'---o---o---o---o---X2 (master)
                                  |\
                                  | A12--A2'---B3'
                                  |             \
                                  |              M'
                                  |             /
                                  \-B1'--B3'---B4

..., being what we are ultimately interested in, it helps us notice 
all kind of changes on our rebased merge parent branches, and act 
accordingly (as shown by my previously posted test script[2]).

All this said (and hoping anyone is still reading this), to shed some 
light on what I meant by "favoring Sergey`s approach in general, but 
_implemented_ through what Phillip described".

I think we should use temporary commits U1 and U2, being a sound 
approach backed up by the "patch theory"[1], as it helps us notice 
any "disbalances" of the rebased merge commit parents trees (so we 
can stop for user inspection), finally merging them with original 
merge commit as a base, in order to spot additional changes on trees 
of the merge commit parents we are rebasing.

But, the merging steps themselves, backed up by a need to stop for 
conflict resolution as soon as one happens, should be performed 
_iteratively_, like Phillip showed us... I would think :)

And, for example, that would do wonders when we introduce completely 
new branch during an interactive rebase, too, for example:

(10) ---X1---B2'---o---o---o---o---X2 (master)
        |\                        /|\
        | A1---A2---A3---U1       || A12--A2'---B3'---U1'
        |             \           ||             \
        |              M          ||              M'
        |             /           ||             /|
        \-B1---B2---B3---U2       |\---B3'---B4-/-|---U2'
                                  |               |
                                  \-----B1'-------/


In this situation, we would merge all temporary Ux commits using 
original merge M as a merge base, and then merge _that_ resulting 
tree with B1' (being a newly introduced merge commit parent), using 
whatever merge base is most appropriate between the two (in this 
case, X2).

So, for diagram (10) above, I guess something like this:

	git merge-recursive U1' -- M U2'
	tree="$(git write-tree)"
	# in case of original merge being octopus, we would continue like:
	# git merge-recursive $tree -- M U3'
	# tree="$(git write-tree)"
	# git merge-recursive $tree -- M U4'
	# ... and so on, then finally:
	git merge-recursive $tree -- "$(git merge-base U1' U2' B1')" B1'
	# in more general case, it would be:
	# git merge-recursive $tree -- "$(git merge-base <all-parents-of-new-merge-commit>)" B1'
	tree="$(git write-tree)"
	git tag M' "$(git log --pretty=%B -1 M | git commit-tree $tree -p B3' -p B4 -p B1')"

... where we would stop for conflict resolution after each failing 
merge-recursive attempt (which is also true for producing rebased U1' 
and U2' in the first place).

There, I would still need to make a test script doing this, but I 
hope the concept is clear, and I`m not missing something crucial here.

Comments welcome :)

Regards, Buga

[1] https://en.wikibooks.org/wiki/Understanding_Darcs/Patch_theory
[2] https://public-inbox.org/git/f1a960dc-cc5c-e7b0-10b6-39e5516655b3@gmail.com/

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 08/03/2018 20:58, Igor Djordjevic wrote:
> 
> 	git merge-recursive U1' -- M U2'
> 	tree="$(git write-tree)"
> 	# in case of original merge being octopus, we would continue like:
> 	# git merge-recursive $tree -- M U3'
> 	# tree="$(git write-tree)"
> 	# git merge-recursive $tree -- M U4'
> 	# ... and so on, then finally:
> 	git merge-recursive $tree -- "$(git merge-base U1' U2' B1')" B1'
> 	# in more general case, it would be:
> 	# git merge-recursive $tree -- "$(git merge-base <all-parents-of-new-merge-commit>)" B1'
> 	tree="$(git write-tree)"
> 	git tag M' "$(git log --pretty=%B -1 M | git commit-tree $tree -p B3' -p B4 -p B1')"

That last line should obviously read just:

	git log --pretty=%B -1 M | git commit-tree $tree -p B3' -p B4 -p B1'

..., above mentioned `git tag M'` part being a leftover from my other test script.

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 08/03/2018 21:27, Igor Djordjevic wrote:
> 
> > 	git merge-recursive U1' -- M U2'
> > 	tree="$(git write-tree)"
> > 	# in case of original merge being octopus, we would continue like:
> > 	# git merge-recursive $tree -- M U3'
> > 	# tree="$(git write-tree)"
> > 	# git merge-recursive $tree -- M U4'
> > 	# ... and so on, then finally:
> > 	git merge-recursive $tree -- "$(git merge-base U1' U2' B1')" B1'
> > 	# in more general case, it would be:
> > 	# git merge-recursive $tree -- "$(git merge-base <all-parents-of-new-merge-commit>)" B1'
> > 	tree="$(git write-tree)"
> > 	git tag M' "$(git log --pretty=%B -1 M | git commit-tree $tree -p B3' -p B4 -p B1')"
> 
> That last line should obviously read just:
> 
> 	git log --pretty=%B -1 M | git commit-tree $tree -p B3' -p B4 -p B1'
> 
> ..., above mentioned `git tag M'` part being a leftover from my other test script.

Eh, pardon me, I managed to mess up all the merge-recursive lines, 
too, in regards to where the merge-base commit goes... Here`s a 
complete (and corrected) sample:

	git merge-recursive M -- U1' U2'
	tree="$(git write-tree)"
	# in case of original merge being octopus, we would continue like:
	# git merge-recursive M -- $tree U3'
	# tree="$(git write-tree)"
	# git merge-recursive M -- $tree U4'
	# ... and so on, then finally:
	git merge-recursive "$(git merge-base U1' U2' B1')" -- $tree B1'
	# ... or even:
	# git merge-recursive "$(git merge-base B3' B4 B1')" -- $tree B1'
	# as in more general case, it would be:
	# git merge-recursive "$(git merge-base <all-parents-of-new-merge-commit>)" -- $tree B1'
	tree="$(git write-tree)"
	git log --pretty=%B -1 M | git commit-tree $tree -p B3' -p B4 -p B1'

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

On 08/03/2018 20:58, Igor Djordjevic wrote:
> 
> > Phillip's method is essentially merging the new tips into the original
> > merge, pretending that the new tips were not rebased but merged into
> > upstream.
> 
> [...]
> 
> Here`s a starting point, two commits A and B, merged into M:
> 
> (3) ---A
>         \
>          M
>         /
>     ---B
> 
> 
> According the "patch theory"[1] (which might not be too popular 
> around here, but should serve the purpose for what I`m trying to 
> explain), each merge commit can be "transformed" to two non-merge 
> commits, one on top of each of the merge parents, where new commit 
> brings its original merge parent commit tree to the state of the 
> merge commit tree:
> 
> (4) ---A---U1
> 
> 
> 
>     ---B---U2
> 
> 
> Now, we have two new commits, U1 and U2, each having the same tree as 
> previous merge commit M, but representing changes in regards to 
> specific parents - and this is essentially what Sergey`s original 
> approach was using (whether he knew it, or not).
> 
> When it comes to rebasing, it`s pretty simple, too. As this:
> 
> (5) ---X1---o---o---o---o---o---X2 (master)
>        |\
>        | A1---A2---A3
>        |             \
>        |              M
>        |             /
>        \-B1---B2---B3
> 
> ... actually equals this:
> 
> (6) ---X1---o---o---o---o---o---X2 (master)
>        |\
>        | A1---A2---A3---U1
>        |
>        |
>        |
>        \-B1---B2---B3---U2
> 
> ... where trees of M, U1 and U2 are same, and we can use the regular 
> rebase semantics and rebase it to this:
> 
> (7) ---X1---o---o---o---o---o---X2 (master)
>                                 |\
>                                 | A1'--A2'--A3'--U1'
>                                 |
>                                 |
>                                 |
>                                 \-B1'--B2'--B3'--U2'
> 
> ... which is essentially this again:
> 
> (8) ---X1---o---o---o---o---o---X2 (master)
>                                 |\
>                                 | A1'--A2'--A3'
>                                 |            \
>                                 |             M'
>                                 |            /
>                                 \-B1'--B2'--B3'
> 

Having explained all this, I realized this is the same "essentially 
merging the new tips into the original pretending that the new tips 
were not rebased but merged into upstream" as Phillip`s one, just 
that we have additional temporary commits U1 and U2 (as per mentioned 
"patch theory") :)

Merging U1' and U2' with M as a base can initially be presented like 
this as well (what Phillip`s approach would yield):

	git merge-recursive U1 -- M U1'
	tree="$(git write-tree)"
	git merge-recursive U2 -- $tree U2'
	tree="$(git write-tree)"
	
..., where we know U1 = U2 = M (in regards to trees), so this is the 
same as:

	git merge-recursive M -- M U1'
	tree="$(git write-tree)"
	git merge-recursive M -- $tree U2'
	tree="$(git write-tree)"

..., which can be further simplified, being the same as:

	git merge-recursive M -- U1' U2'
	tree="$(git write-tree)"

... which is exactly what Sergey`s (updated) approach suggests, 
merging U1' and U2' with M as merge-base (and shown inside that 
sample implementation script I provided[1]) :)

With all this said, I think it`s safe to say these two approaches are 
exactly the same, just Sergey`s being simplified (thus harder to 
initially understand), and the only actual question is whether we 
value U1' and U2' enough, as possible "something suspicious happened" 
indicators, to use them, or not.

I would think yes, but I would be open for more samples of where do 
they become useful for reporting "suspicious activity", too.

Regards, Buga

[1] https://public-inbox.org/git/b11785bd-5c96-43c1-95d8-b28eccfd13c8@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Junio,

On Thu, 8 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> If we are talking about a drastic change, a few more days may not be
> >> sufficient, but we are not in a hurry, as this already sounds like a
> >> 2.18 material anyway.
> >
> > It is not at all a drastic change. It will actually make the current patch
> > series better (simplifying the "can we fast-forward?" check).
> >
> > I just want to make sure that I already have Phillip's strategy working,
> > but it will be yet another topic branch on top of the topic branch that
> > will add support for octopus merges *after* the current --recreate-merges
> > topic branch ;-)
> 
> Oh, if the "not redoing the merge afresh, but attempt to reuse the
> previous merge" that was discussed is going to be done as an
> update/addition to the "redo the merge afresh" you already had in
> production forever (and I had in 'pu' for quite a while in various
> polished-ness during iteration), then I do prefer merging down what
> has already proven to be 'next' worthy without waiting for the
> discussion and your local verification of Phillip's new thing,
> especially given that you'll be giving an explicit control to the
> users which variant of "merge" insn will be used and the addition
> of the Phillip's thing won't be a backward-compatibility issue when
> it comes later.

I would like to stress that the `--recreate-merges` functionality has
*not* beed in production forever. It is a reimplementation in pure C of
the Unix shell script that has been in production for five years
(unchanged only for the last half year, the last critical fix happened in
January last year).

So I do see the value here to use `next` as test bed, and once the patch
series will hit `next`, I will also merge it into Git for Windows (as an
EXPERIMENTAL feature).

As such, I am quite comfortable with refactoring a bit here and there, for
example how to handle the case where a `merge` can be fast-forwarded.

I think the changes I made in the last few days were an actual improvement
to readability, even if the only reason for those changes was that I
wanted to accommodate the "rebase merge commits" thing.

FWIW I am fine with bumping this down to 2.18. I got a little bit of
valuable feedback at GitMerge, e.g. that --recreate-merges does not (yet)
have a mode where it can update refs corresponding to the rebased commits.
(This could actually turn out to be independent of --recreate-merges,
even).

I already pushed out the updated branch, and it can be inspected at

	https://github.com/git/git/pull/447

The reason I did not send this out yet is that I want to give it a final
look-over myself, so that I do not waste people's time again (as I did
with that monster interdiff that was pretty bogus, too).

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Junio,

On Thu, 8 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> >> Non-textual semantic conflicts are made (in the best case just once)
> >> as a separate commit on top of mechanical auto-merge whose focus is
> >> predictability (rather than cleverness) done by Git, and then that
> >> separate commit is kept outside the history.  When replaying these
> >> merges to rebuild the 'pu' branch, after resetting the tip to
> >> 'master', each topic is merged mechanically, and if such a fix-up
> >> commit is present, "cherry-pick --no-commit" applies it and then
> >> "commit --amend --no-edit" to adjust the merge.  I find it quite
> >> valuable to have a separate record of what "evil" non-mechanical
> >> adjustment was done, which I know won't be lost in the noise when
> >> these merges need to be redone daily or more often.
> >
> > So essentially, you have something that `git rerere` would have learned,
> > but as a commit?
> 
> You probably wouldn't be asking that if you read what you cut out
> when you quoted above ;-) 

Sorry to be so stupid, and no, the part I cut did not clarify it for me,
hence my question.

> There are a collection of cherry-pickable commits in hierarchy under
> refs/merge-fix.  They are indexed by the branch that will cause
> semantic conflicts that do not involve textual conflicts at all (the
> important implication of which is that 'rerere' fundamentally will
> not trigger to help resolving them) [*1*], and are used to create
> evil merge when a corresponding branch is merged to 'pu' (and down).

I see. My question was unclear, I agree. Please let me re-try:

So essentially, what your cherry-pick'able commits are is a way to store
what rerere would have stored (i.e. the set of merge conflicts together
with their resolution)?

If so, what tooling do you have to identify quickly what to cherry-pick,
given merge conflicts?

(I know I could spend some half hour to scour your `todo` branch and the
documentation you wrote about how you maintain Git, but you already know
the answer to this question, and it would probably be interesting to
others who are as eager to have better tools for handling merge conflicts
as I am, too, so it would save time overall to discuss this single
question here.)

> *1* One topic adds an extra parameter to read_index_from() that has
> been and still is defined in a file and merged to 'pu' first, while
> another topic adds a new callsite for the same function in a file
> that the former topic does not touch, hence a merge of the latter
> topic has no textual conflict to the file with a new callsite, but
> still needs adjusting.  That sort of think.

Okay, so it is even more intricate than `rerere`: it does not only store
the merge conflicts (by grace of using the "patch ID" of the merge
conflicts) together with their resolution, but instead it has some sort of
idea of what context needs to be met to require the resolution?

Color me intrigued.

If you really found a way to automate describing, say, that a function
signature changed in one branch, and a caller was introduced in another,
and that merging those requires adjusting the caller in a specific way, I
now *really* think that this should be made available more broadly.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Thu, 8 Mar 2018, Igor Djordjevic wrote:

> On 08/03/2018 13:16, Phillip Wood wrote:
> > 
> > > I did wonder about using 'pick <original-merge>' for rebasing merges
> > > and keeping 'merge ...' for recreating them but I'm not sure if that
> > > is a good idea. It has the advantage that the user cannot specify
> > > the wrong parents for the merge to be rebased as 'git rebase' would
> > > work out if the parents have been rebased, but maybe it's a bit
> > > magical to use pick for merge commits. Also there isn't such a
> > > simple way for the user to go from 'rabase this merge' to 'recreate
> > > this merge' as they'd have to write the whole merge line themselves
> > > (though I guess something like emacs' git-rebase.el would be able to
> > > help with that)
> > 
> > Scrub that, it is too magical and I don't think it would work with
> > rearranged commits - it's making the --preserve-merges mistake all
> > over again. It's a shame to have 'merge' mean 'recreate the merge' and
> > 'rebase the merge' but I don't think there is an easy way round that.
> 
> I actually like `pick` for _rebasing_ merge commits, as `pick` is 
> already used for rebasing non-merge commits, too, so it feels natural.

Phillip is right, though: this would repeat the design mistake of
--preserve-merges.

We must not forget that the interactive mode is the target here, and that
the syntax (as well as the generated todo list) must allow for easy
modification. The `pick <merge>` approach does not allow that, so we
cannot use it.

The `merge -R -C <original-commit> <merge-head>` approach is a lot better:
it offers the flexibility, without sacrificing the ease when not modifying
the todo list.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Thu, 8 Mar 2018, Igor Djordjevic wrote:

> On 08/03/2018 12:20, Phillip Wood wrote:
> > 
> > I did wonder about using 'pick <original-merge>' for rebasing merges
> > and keeping 'merge ...' for recreating them but I'm not sure if that
> > is a good idea. It has the advantage that the user cannot specify the
> > wrong parents for the merge to be rebased as 'git rebase' would work
> > out if the parents have been rebased, but maybe it's a bit magical to
> > use pick for merge commits. Also there isn't such a simple way for the
> > user to go from 'rabase this merge' to 'recreate this merge' as they'd
> > have to write the whole merge line themselves (though I guess
> > something like emacs' git-rebase.el would be able to help with that)
> 
> Hmm, funny enough, `pick <original merge>` was something I though about
> originally, too, feeling that it might make more sense in terms on
> what`s really going on, but I guess I wanted it incorporated into
> `--recreate-merges` too much that I tried really hard to fit it in,
> without changing it much :/

The `pick <original-merge>` syntax is too limited to allow reordering, let
alone changing the parents.

> And now that I said this in a previous reply:
> 
> > The thing is, in my opinion, as long as we are _rebasing_, you can`t 
> > pick any merge strategy, as it doesn`t really make much sense. If you 
> > do want a specific strategy, than that`s _recreating_ a merge, and it 
> > goes fine with what you already have for `--recreate-merges`.
> > 
> > On merge rebasing, the underline strategy we decide to use is just an 
> > implementation detail, picking the one that works best (or the only 
> > one that works, even), user should have nothing to do with it.
> 
> The difference between "rebase merge commit" and "recreate merge 
> commit" might starting to be more evident.

True.

> So... I might actually go for this one now. And (trying to stick with 
> explicit mappings, still :P), now that we`re not married to `merge` 
> expectations a user may already have, maybe a format like this:
> 
>   pick <original-merge> <original-parent1>:HEAD <original-parent2>:<new-parent2>

I do not really like it, as it makes things a ton less intuitive. If you
did not know about this here discussion, and you did not read the manual
(and let's face it: a UI that does not require users to read the manual is
vastly superior to a UI that does), and you encountered this command:

	merge deadbeef cafecafe:download-button

what would you think those parameters would mean?

Granted, encountering

	merge -R -C deadbeef download-button # Merge branch 'download-button'

is still not *quite* as intuitive as I would wish. Although, to be honest,
if I encountered this, I would think that I should probably leave the -R
and the -C deadbeef alone, and that I could change what is getting merged
by changing the `download-button` parameter.

> p.s. Are we moving towards `--rebase-merges` I mentioned in that 
> other topic[1], as an add-on series after `--recreate-merges` hits 
> the mainstream (as-is)...? :P

That's an interesting question. One that I do not want to answer alone,
but I would be in favor of `--rebase-merges` as it is IMHO a much better
name for what this option is all about.

Other opinions?

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Jake,

On Thu, 8 Mar 2018, Jacob Keller wrote:

> On Thu, Mar 8, 2018 at 3:20 AM, Phillip Wood <phillip.wood@talktalk.net> wrote:
> > I did wonder about using 'pick <original-merge>' for rebasing merges
> > and keeping 'merge ...' for recreating them but I'm not sure if that
> > is a good idea. It has the advantage that the user cannot specify the
> > wrong parents for the merge to be rebased as 'git rebase' would work
> > out if the parents have been rebased, but maybe it's a bit magical to
> > use pick for merge commits. Also there isn't such a simple way for the
> > user to go from 'rabase this merge' to 'recreate this merge' as they'd
> > have to write the whole merge line themselves (though I guess
> > something like emacs' git-rebase.el would be able to help with that)
> 
> Since the ultimate commit hashes of newly rebased commits would be
> unknown at the time of writing the todo file, I'm not sure how this
> would work to specify the parents?

I agree with Phillip's follow-up that the `pick <original-merge>` syntax
would pose a problem, but for different reasons: We already tried it, with
--preserve-merges, and it is just a really stupid syntax that does not
allow the user even to reorder commits. Or drop commits (except at the
very end of the todo list).

As to your concern: The ultimate commit hashes do not have to be known
until the merge commit is rebased. The current approach is to use labels
for them (`label <name>`), which Simply Works.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Thu, 8 Mar 2018, Igor Djordjevic wrote:

> On 08/03/2018 16:16, Igor Djordjevic wrote:
> > 
> > > Unless we reimplement the octopus merge (which works quite a bit
> > > differently from the "rebase merge commit" strategy, even if it is
> > > incremental, too), which has its own challenges: if there are merge
> > > conflicts before merging the last MERGE_HEAD, the octopus merge will exit
> > > with status 2, telling you "Should not be doing an octopus.". While we
> > > will want to keep merge conflict markers and continue with the "rebase the
> > > original merge commit" strategy.
> > >
> > > [...]
> > 
> > The thing is, in my opinion, as long as we are _rebasing_, you can`t 
> > pick any merge strategy, as it doesn`t really make much sense. If you 
> > do want a specific strategy, than that`s _recreating_ a merge, and it 
> > goes fine with what you already have for `--recreate-merges`.
> > 
> > On merge rebasing, the underlying strategy we decide to use is just an 
> > implementation detail, picking the one that works best (or the only 
> > one that works, even), user should have nothing to do with it.
> 
> Just to add, if not already assumable, that I think we should stop 
> and let user react on conflicts on each of the "rebase the original 
> commit" strategy steps (rebase first parent, rebase second parent... 
> merge parents).

I am not sure about that. Actually, I am pretty certain that we should
imitate the recursive merge, which "accumulates" merge conflicts and only
presents the final result, possibly with nested merge conflicts.

In practice, we see that rarely, and more often than not, in those cases
the user wants to re-do the merge differently, anyway.

In the scenario we discuss here, I would wager a bet that the user
encountering nested merge conflicts would most likely see that the rebase
tried to rebase a merge, then `git reset --hard` and re-do the merge
manually (i.e. *recreate* rather than *rebase*, most likely with
(non-nested) merge conflicts).

> I guess this stresses not using real "octopus merge" strategy even in 
> case where we`re rebasing octopus merge commit even more (and aligns 
> nicely with what you seem to expect already).

My mistake, I should have pointed you my implementation:

https://github.com/git/git/commit/d41a29ceb61ff445efd0f97a836f381de57c5a41

The full thicket of branches can be found here:

https://github.com/git/git/compare/master...dscho:sequencer-shears

I chose to make this an add-on patch after adding support for octopus
merges because it actually makes it easier to think about "rebase merge
commits" independently of the number of merge parents.

Ciao,
Dscho

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Thu, 8 Mar 2018, Igor Djordjevic wrote:

> On 07/03/2018 15:08, Johannes Schindelin wrote:
> > 
> > > > Didn't we settle on Phillip's "perform successive three-way merges
> > > > between the original merge commit and the new tips with the old tips
> > > > as base" strategy?
> > >
> > > It seems you did, dunno exactly why.
> > 
> > That is not true. You make it sound like I was the only one who liked
> > this, and not Phillip and Buga, too.
> 
> For myself, I do actually favor Sergey`s approach in general, but 
> _implemented_ through what Phillip described (or a mixture of both, to 
> be precise). But, let me explain... :)

So as you explained later in this sub-thread, Sergey's approach is
essentially the same as Phillip's.

I still do not understand Sergey's approach on a fundamental level. I
mean, I can follow his instructions how to implement his algorithm, but it
is as if I had a blindfold on and somebody guided me through a maze: I
understand *what* I am supposed to do, but I have no clue *why*.

And admittedly, I got very frustrated when a document was thrown my way
that is too long to read in one sitting, and all of my attempts at getting
clear and comprehensible answers to specific questions were met with "go
and read that document, I am sure you will understand then".

For something as fundamental to my daily workflow as an interactive rebase
(*especially* when trying to maintain the branch topology), this is no
good at all.

Since you already confirmed that there is essentially no difference
between the two approaches, I will simply go with the one I understand, in
particular I understand *why* it works.

But let's read on, maybe I will change my mind based on your explanations
(which do answer my questions, thank you so much for that)...

> > > The main problem with this decision is that we still don't see how
> > > and when to stop for user amendment using this method. OTOH, the
> > > original has this issue carefully discussed.
> > 
> > Why would we want to stop, unless there are merge conflicts?
> 
> Because we can reliably know that something "unusual" happened - and by
> that I don`t necessarily mean "wrong", but just might be worth user
> inspection.

We have a similar conundrum in recursive merges. Remember how multiple
merge bases are merged recursively? There can be merge conflicts, too, in
*any* of the individual merges involved, and indeed, there are (under
relatively rare circumstances).

Since we already faced that problem, and we already answered it by
presenting possibly nested merge conflicts, I am in strong favor of
keeping our new scenario consistent: present possibly-nested merge
conflicts.

As far as I understand, one of the arguments in favor of the current
approach was: there is no good way to tell the user where they are, and
how to continue from there. So better just to continue and present the
user with the entire set of conflicts, and have an obvious way out.

> For example, situation like this (M is made on A3 with `-s ours`, 
> obsoleting Bx commits):
> 
> (1) ---X8--X9 (master)
>        |\
>        | A1---A2---A3
>        |             \
>        |              M (topic)
>        |             /
>        \-B1---B2---B3
> 
> ... where we want to rebase M onto X9 is what I would call "usual 
> stuff", but this situation (M is still made on A3 with `-s ours`, 
> obsoleting Bx commits, but note cherry-picked B2'):
> 
> (2) ---X8--B2'--X9 (master)
>        |\
>        | A1---A2---A3
>        |             \
>        |              M (topic)
>        |             /
>        \-B1---B2---B3
> 
> ... where we still want to rebase M onto X9 is what we might consider 
> "unusual", because we noticed that something that shouldn`t be part 
> of the rebased merge commit (due to previous `-s ours`) actually got 
> in there (due to later cherry-pick), and just wanting the user to 
> check and confirm.

We already have those scenarios when performing a regular interactive
rebase, where a patch was already applied upstream. In the normal case,
the user is not even shown B2, thanks to the --cherry-pick option used in
generating the todo list.

Granted, in some cases --cherry-pick does not detect that, and then we
generate a todo list including B2, and when that patch is applied, the
interactive rebase stops, saying that there are no changes to be
committed.

And this behavior is exactly the same with --recreate-merges!

So I do not think that it would make sense to bother the user *again* when
rebasing the merge commit.

If there are merge conflicts, yes, we will have to. If there are none
(even if your U1' != U2'), it would be outright annoying to stop.

> > > "rebase sides of the merge commit and then three-way merge them back
> > > using original merge commit as base"
> > 
> > And that is also wrong, as I had proved already! Only Buga's addition
> > made it robust against dropping/modifying commits, and that addition
> > also makes it more complicated.
> 
> No, this is actually right, that sentence nicely describing _how_ it 
> works.

Does it? Because that's exactly backwards from how Phillip's approach
works: it certainly does not use the original merge commit as base. It
uses the non-rebased merge parent as base, one at a time.

Now I am curious...

> That addition of mine was just including the correct merge base (being
> the original merge commit that we are "rebasing"), and that`s what
> Sergey is talking about.

Okay, render me still unconvinced that Sergey's approach is actually
correct.

> > > > - it is *very easy* to reason about, once it is pointed out that
> > > > rebases and merges result in the same trees.
> > >
> > > The original is as easy to reason about, if not easier, especially as
> > > recursive merge strategy is not being used there in new ways.
> > 
> > So do it. I still have to hear a single-sentence, clear and obvious
> > explanation why it works.

So the closest I got was the description with the *original merge commit*
as merge base, which I disagree with. It does not make sense to me.

Consecutive three-way merges between the original merge commit and the
merge parents (with the *original merge parents* as merge base,
respectively), still makes the most sense to me: it is a merge between the
amendmends to the merge commit and the changes introduced by rebasing the
merge parents.

I am now getting the sense as if Sergey's approach (with your, let's say,
"fix") is trying to apply too much, and by using the original merge commit
as merge base then tries to undo part of that.

> > > I honestly don't see any advantages of Phillip's method over the
> > > original, except personal preferences. At the same time, I have no
> > > objection of using it either, provided consistency check problem is
> > > solved there as well.
> > 
> > Okay, let me reiterate then, because I do not want this point to be
> > missed:
> > 
> > Phillip's method is essentially merging the new tips into the original
> > merge, pretending that the new tips were not rebased but merged into
> > upstream.
> > 
> > So it exploits the duality of the rebase and merge operation, which both
> > result in identical trees (potentially after resolving merge conflicts).
> > 
> > I cannot think of any such interpretation for your proposal augmented by
> > Buga's fix-ups. And I haven't heard any such interpretation from your
> > side, either.
> 
> Ok here it goes (I might be still wrong, but please bare with me).
> 
> What Sergey originally described also uses the "duality of the rebase 
> and merge operation", too ;) Just in a bit different way (and might 
> be a more straightforward one?).

I will be the judge whether it looks more straight-forward to me. :-)

> Here`s a starting point, two commits A and B, merged into M:
> 
> (3) ---A
>         \
>          M
>         /
>     ---B
> 
> 
> According the "patch theory"[1] (which might not be too popular 
> around here, but should serve the purpose for what I`m trying to 
> explain),

I think that the patch theory is not usually quoted here because

1) originally, Darcs had this snake oil way of (over-)selling its theory
   as having "roots in quantum mechanics" [*1*] at the time Git took off,
   and

2) it does not really have a good concept of rebasing, even if it should
   be theoretically possible to integrate that into the "theory of
   patches".

> each merge commit can be "transformed" to two non-merge commits, one on
> top of each of the merge parents, where new commit brings its original
> merge parent commit tree to the state of the merge commit tree:
> 
> (4) ---A---U1
> 
> 
> 
>     ---B---U2

You get the same result if you stick to regular graph theory, of course.
All you need to do is to interpret the directed vertices between nodes as
a patch & parent relationship. Simple.

> Now, we have two new commits, U1 and U2, each having the same tree as 
> previous merge commit M, but representing changes in regards to 
> specific parents - and this is essentially what Sergey`s original 
> approach was using (whether he knew it, or not).
> 
> When it comes to rebasing, it`s pretty simple, too. As this:
> 
> (5) ---X1---o---o---o---o---o---X2 (master)
>        |\
>        | A1---A2---A3
>        |             \
>        |              M
>        |             /
>        \-B1---B2---B3
> 
> ... actually equals this:
> 
> (6) ---X1---o---o---o---o---o---X2 (master)
>        |\
>        | A1---A2---A3---U1
>        |
>        |
>        |
>        \-B1---B2---B3---U2
> 
> ... where trees of M, U1 and U2 are same,

Okay, so you basically duplicated the merge commit and dropped its
semantics as a merge commit. That is a very big difference to Phillip's
approach already.

It opens the door to ambiguities, as we will see later.

> and we can use the regular rebase semantics and rebase it to this:
> 
> (7) ---X1---o---o---o---o---o---X2 (master)
>                                 |\
>                                 | A1'--A2'--A3'--U1'
>                                 |
>                                 |
>                                 |
>                                 \-B1'--B2'--B3'--U2'
> 
> ... which is essentially this again:
> 
> (8) ---X1---o---o---o---o---o---X2 (master)
>                                 |\
>                                 | A1'--A2'--A3'
>                                 |            \
>                                 |             M'
>                                 |            /
>                                 \-B1'--B2'--B3'
> 
> ... where it is still true that trees of U1', U2' and M' are still 
> the same. So we managed to rebase a merge commit without ever doing a 
> merge :) (note that, practically, we _can_ finally even merge U1' and 
> U2' to produce M', it shouldn`t really matter as all the trees are 
> the same, so merge will be a no-op)

U1' and U2' do not have the same tree, though *especially* when the user
did not edit the todo list to insert/modify/drop/reorder commits.

This violation of expectations is of course caused by "duplicating the
merge commit" into U1 and U2.

But let's continue.

> But, as we saw (what you`ve explained), this doesn`t really work in 
> case one of the sides of the merge gets "disbalanced" (so to say), 
> like dropping a commit (which could also happen non-interactively, 
> where a commit has been cherry-picked to a different branch, but
> previously obsoleted by `-s ours` merge).

Precisely. A *very* important counter-argument to this approach so far.

> As observed, dropped commit could still wrongly get into final merge 
> commit tree (or cherry-picked commit wrongly not get there), due to 
> the nature of those rebased U1 and U2 temporary commits to hold all 
> the differences in regards to their specific merge commit parent.

The reason for this, of course, is that either U1's or U2's diff will show
those differences, *and we still try to rebase them even if the user
already dropped them*.

But let's continue.

> A natural improvement to original idea which Sergey eventually came 
> up with after my findings (which you now ended up calling a hack, even, 
> but I would respectfully disagree), is to use original merge commit M 
> as a merge base for final merge of U1' and U2' - and here is why it 
> works, too, and why I wouldn`t consider it a hack, but a proper (and 
> a solid) solution.
> 
> Merge commit M is what we are rebasing, so we should have a way to 
> somehow learn from it, alongside U1 and U2 temporary commits - and 
> that is what using original merge commit M as a base does for us. It 
> helps us spot any "disbalances" that happened in comparison to original 
> merge parent trees, which is exactly what we`re interested in.

... except that it gets the direction wrong. Rather than trying to *avoid*
rebasing possibly dropped changes, it tries to kind of "undo" them by
using a merge base that does not really make sense (unless you think of it
as a "revert").

It would make sense if M could interpreted as a branch point. But it
cannot, as we specifically did *not* continue to develop the merge parents
from that merge commit.

Instead, what we did was to branch off of the original branch point X1.

Reframing the rebase of a sub-branch (X1..A3) as merge with upstream,
however, we can interpret M and A3' as revisions we want to merge, with A3
as the merge base.

(You can think of it in terms of the "theory of patches" thusly: if X1..A3
is represented by patch K, X1..X2 by patch L, and X2..A3' by K', then what
we want to merge into M is K^(-1).L.K', which is precisely what A3..A3'
translates to.)

You can also think of it as diverging changes going from A3: one direction
was to merge (resulting in the commit M), the other direction was to
rebase onto X2 (resulting in A3'), and a 3-way merge M <- A3 -> A3' will
reconcile those changes (and you will want to repeat with B3/B3', too).

Let's put that into the context of your example: instead of introducing U1
and U2, we introduce V1 and V2 right away, as temporary *merge* commits,
where the tree of V1 is identical to the one of A3', and the tree of V2
to that of B3'.

 ---X1---o---o---o---o---o---X2 (master)
    |                        |\
    |                        | A1'--A2'--A3'--V1
    |\                       |               /
    | -A1---A2---A3----------+---------------
    |              \         |
    |               M        \-B1'--B2'--B3'--V2
    |              /                         /
     \-B1---B2---B3--------------------------

Note: the *really* important difference is that these temporary commits
are based on the *rebased* history rather than the *unrebased* history.

Second note: this is very related to Git for Windows' original strategy of
coping with the non-fast-forwarding nature of rebases: after rebasing the
Windows-specific patches on top of core Git, we merged (with `-s ours`)
the unrebased history. This 1) makes it fast-forwardable, and 2) still
rebases the patches [*2*].

Third note: my favorite mental model is still the duality of rebasing and
merging, in which case V1 and V2 would not have A3' and B3' as first
parents, but X2.

Phillip's strategy is to merge M with V1 and V2.

This translates to "merge the amendments of the merge commit M with the
changes introduced by rebasing its merge parents".

The really cute part about this is that (in contrast to using U1' and
U2'), we do not merge the amendments of the merge commit multiple times,
but exactly once. And therefore, we do not need to "undo" them by using
the original merge commit as merge base, either (which would introduce
many more opportunities for merge conflicts to creep in, oftentimes
unnecessary conflicts to begin with).

> In ideal case (as per "patch theory"[1]), final merge of rebased U1' 
> and U2' with original merge commit M as a base would be rather simple, 
> too (not to say pointless again), as both U1' and U2' would have same 
> changes in comparison to M, namely all the changes from commit we are 
> rebasing from (X1 above) to commit we are rebasing to (X2 above).
> 
> But in a more complex case like this:
> 
> (9) ---X1---B2'---o---o---o---o---X2 (master)
>                                   |\
>                                   | A12--A2'---B3'
>                                   |             \
>                                   |              M'
>                                   |             /
>                                   \-B1'--B3'---B4
> 
> ..., being what we are ultimately interested in, it helps us notice 
> all kind of changes on our rebased merge parent branches, and act 
> accordingly (as shown by my previously posted test script[2]).

To use the "theory of patches" to explain why Phillip's approach is so
much more appealing: in Sergey's approach, we will rebase U1 (which is
"sort of" B1.B2.B3 in the "theory of patches"). If the cherry-pick of B2'
caused merge conflicts that had to be resolved, then these merge conflicts
will have to be resolved *again* when rebasing U1 (because B2 is *part of*
U1). And of course they will have to be resolved *again* when merging U1'
with U2'.

In short: Phillip's approach is a short-cut that avoids unnecessary merge
conflicts because it avoids rebasing changes that would need to be undone
right away anyway.

> All this said (and hoping anyone is still reading this), to shed some 
> light on what I meant by "favoring Sergey`s approach in general, but 
> _implemented_ through what Phillip described".
> 
> I think we should use temporary commits U1 and U2, being a sound 
> approach backed up by the "patch theory"[1], as it helps us notice 
> any "disbalances" of the rebased merge commit parents trees (so we 
> can stop for user inspection), finally merging them with original 
> merge commit as a base, in order to spot additional changes on trees 
> of the merge commit parents we are rebasing.

In Phillip's approach, we do not need to rebase the amendments of the
merge commit M twice (or even more times, for octopus merges). Therefore,
there is no opportunity for these imbalances.

> But, the merging steps themselves, backed up by a need to stop for 
> conflict resolution as soon as one happens, should be performed 
> _iteratively_, like Phillip showed us... I would think :)

From a user interface perspective, this is a bad idea: you would now have
to communicate to the user *where* in the process they are.

And for that you would have to explain that process to them. Including
"theory of patches" and all.

Taking me as a prime example, you now know how tedious and impractical
that would be.

> And, for example, that would do wonders when we introduce completely 
> new branch during an interactive rebase, too, for example:
> 
> (10) ---X1---B2'---o---o---o---o---X2 (master)
>         |\                        /|\
>         | A1---A2---A3---U1       || A12--A2'---B3'---U1'
>         |             \           ||             \
>         |              M          ||              M'
>         |             /           ||             /|
>         \-B1---B2---B3---U2       |\---B3'---B4-/-|---U2'
>                                   |               |
>                                   \-----B1'-------/
> 
> 
> In this situation, we would merge all temporary Ux commits using 
> original merge M as a merge base, and then merge _that_ resulting 
> tree with B1' (being a newly introduced merge commit parent), using 
> whatever merge base is most appropriate between the two (in this 
> case, X2).

The semantics of octopus merges are very different from regular recursive
merges. I am not sure that we want to go there in this discussion...

If, on the other hand, you do not try to turn M from a regular 2-parent
merge commit into an octopus merge during an interactive rebase, but
instead split the B branch into two branches *before* merging the result
into the rebased merge commit, we are in the same square as
reordering/dropping/inserting/modifying patches: to neither of the
presented strategies would it matter what kind of branch topology the
merge parents have.

For the record: I am still not convinced that Phillip's and Sergey's
approach are equivalent, even in terms of the "theory of patches". But if
they are, then Phillip's version is a shorter version that avoids applying
changes just to revert them right away. And in a setting where each patch
can cause merge conflicts (as the interactive rebase is), the less changes
you have to apply, the better.

Ciao,
Dscho

> [1] https://en.wikibooks.org/wiki/Understanding_Darcs/Patch_theory

Footnote *1*: https://web.archive.org/web/20050511033324/http://darcs.net/
And it must be said: the "theory of patches" are not rooted in quantum
mechanics. They might be inspired by Darc's inventor being more familiar
with it than with fundamental algebra, for sure. The "theory of patches"
is nothing else than an algebra on graph vertices, though.

Footnote *2*: We called this strategy the rebasing merge. We replaced it
by the opposite strategy ("merging rebase", i.e. starting with the `git
merge -s ours <unrebased>` and *then* applying the rebased patches)
because the latter makes it really obvious where the Windows-specific
patches *start*.

Of course, you could draw the diagram very easily also using the merging
rebase, the conclusion would still be the same.

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Fri, 9 Mar 2018, Igor Djordjevic wrote:

> On 08/03/2018 20:58, Igor Djordjevic wrote:
> > 
> > > Phillip's method is essentially merging the new tips into the original
> > > merge, pretending that the new tips were not rebased but merged into
> > > upstream.
> > 
> > [...]
> > 
> > Here`s a starting point, two commits A and B, merged into M:
> > 
> > (3) ---A
> >         \
> >          M
> >         /
> >     ---B
> > 
> > 
> > According the "patch theory"[1] (which might not be too popular 
> > around here, but should serve the purpose for what I`m trying to 
> > explain), each merge commit can be "transformed" to two non-merge 
> > commits, one on top of each of the merge parents, where new commit 
> > brings its original merge parent commit tree to the state of the 
> > merge commit tree:
> > 
> > (4) ---A---U1
> > 
> > 
> > 
> >     ---B---U2
> > 
> > 
> > Now, we have two new commits, U1 and U2, each having the same tree as 
> > previous merge commit M, but representing changes in regards to 
> > specific parents - and this is essentially what Sergey`s original 
> > approach was using (whether he knew it, or not).
> > 
> > When it comes to rebasing, it`s pretty simple, too. As this:
> > 
> > (5) ---X1---o---o---o---o---o---X2 (master)
> >        |\
> >        | A1---A2---A3
> >        |             \
> >        |              M
> >        |             /
> >        \-B1---B2---B3
> > 
> > ... actually equals this:
> > 
> > (6) ---X1---o---o---o---o---o---X2 (master)
> >        |\
> >        | A1---A2---A3---U1
> >        |
> >        |
> >        |
> >        \-B1---B2---B3---U2
> > 
> > ... where trees of M, U1 and U2 are same, and we can use the regular 
> > rebase semantics and rebase it to this:
> > 
> > (7) ---X1---o---o---o---o---o---X2 (master)
> >                                 |\
> >                                 | A1'--A2'--A3'--U1'
> >                                 |
> >                                 |
> >                                 |
> >                                 \-B1'--B2'--B3'--U2'
> > 
> > ... which is essentially this again:
> > 
> > (8) ---X1---o---o---o---o---o---X2 (master)
> >                                 |\
> >                                 | A1'--A2'--A3'
> >                                 |            \
> >                                 |             M'
> >                                 |            /
> >                                 \-B1'--B2'--B3'
> > 
> 
> Having explained all this, I realized this is the same "essentially 
> merging the new tips into the original pretending that the new tips 
> were not rebased but merged into upstream" as Phillip`s one, just 
> that we have additional temporary commits U1 and U2 (as per mentioned 
> "patch theory") :)

But if the old tips had been merged into upstream (resulting in the new
tips), then the merge bases would be *the old tips*.

I am still not sure for what scenarios Phillip's strategy is the same as
Sergey's (updated) one, as the former strategy can do completely without
temporary commits, and cannot introduce ambiguities when rebasing the
changes introduced by M (i.e. the "amendmendts" we talked about).

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Dscho,

On 11/03/2018 13:00, Johannes Schindelin wrote:
> 
> > I actually like `pick` for _rebasing_ merge commits, as `pick` is 
> > already used for rebasing non-merge commits, too, so it feels natural.
> 
> Phillip is right, though: this would repeat the design mistake of
> --preserve-merges.
> 
> We must not forget that the interactive mode is the target here, and that
> the syntax (as well as the generated todo list) must allow for easy
> modification. The `pick <merge>` approach does not allow that, so we
> cannot use it.
> 
> The `merge -R -C <original-commit> <merge-head>` approach is a lot better:
> it offers the flexibility, without sacrificing the ease when not modifying
> the todo list.

Eh, I`m afraid the quote you took is missing the rest of its 
(important) context, where I mentioned already proposed format for 
`pick` in that other subthread[1], including other parameters beside 
merge commit to pick, as that parent mapping.

I agree with both of you that `pick <merge-commit>` is inflexible 
(not to say just plain wrong), but I never thought about it like that.

If we are to extract further mentioned explicit old:new merge 
parameter mapping to a separate discussion point, what we`re 
eventually left with is just replacing this:

	merge -R -C <original--merge-commit> <merge-head>

... with this:

	pick <original--merge-commit> <merge-head>


That is what I had in mind, seeming possibly more straightforward and 
beautifully aligned with previously existing (and well known) 
`rebase` terminology.

Not to say this would make it possible to use other `rebase -i` todo 
list commands, too, like if you want to amend/edit merge commit after 
it was rebased, you would write:

	edit <original--merge-commit> <merge-head>

..., where in case you would simply like to reword its commit 
message, it would be just:

	reword <original--merge-commit> <merge-head>


Even `squash` and `fixup` could have their place in combination with 
a (to be rebased) merge commit, albeit in a pretty exotic rebases, 
thus these could probably be just disallowed - for the time being, at 
least.

The real power would be buried in implementation, learning to rebase 
merge commits, so user is left with a very familiar interface, slightly 
adapted do accommodate a bit different nature of merge commit in 
comparison to an ordinary one, also to allow a bit more of interactive 
rebase functionality, but it would pretty much stay the same, without 
even a need to learn about new `merge`, `-R`, `-C`, and so on.

Yes, those would have its purpose, but for real merging then 
(creating new merges, or recreating old ones), not necessarily for 
merge rebasing.

With state of `merge -R -C ...` (that `-R` being the culprit), it 
kind of feels like we`re now trying to bolt "rebase merges" 
functionality onto a totally different one (recreate merges, serving 
a different purpose), making them both needlessly dependent on each 
other, further complicating user interface, making it more confusing 
and less tunable as per each separate functionality needs (rebase vs. 
recreate).

I guess I`m the one to pretty much blame here, too, as I really 
wanted `--recreate-merges` to handle "rebase merges" better, only to 
later realize it might not be the best tool for the job, and that a 
more separate approach would be better (at least not through the same 
`merge` todo list command)...

Regards, Buga

[1] https://public-inbox.org/git/f3872fb9-01bc-b2f1-aee9-cfc0e4db77d6@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Dscho,

On 11/03/2018 13:08, Johannes Schindelin wrote:
> 
> > Hmm, funny enough, `pick <original merge>` was something I though about
> > originally, too, feeling that it might make more sense in terms on
> > what`s really going on, but I guess I wanted it incorporated into
> > `--recreate-merges` too much that I tried really hard to fit it in,
> > without changing it much :/
> 
> The `pick <original-merge>` syntax is too limited to allow reordering, let
> alone changing the parents.

I agree, `pick <original-merge>` syntax alone is never what I had in 
mind, so it`s missing further context here, touched in that other 
subthread[1]. My fault, sorry for confusion.

> >   pick <original-merge> <original-parent1>:HEAD <original-parent2>:<new-parent2>
> 
> I do not really like it, as it makes things a ton less intuitive. If you
> did not know about this here discussion, and you did not read the manual
> (and let's face it: a UI that does not require users to read the manual is
> vastly superior to a UI that does), and you encountered this command:
> 
> 	merge deadbeef cafecafe:download-button
> 
> what would you think those parameters would mean?
> 
> Granted, encountering
> 
> 	merge -R -C deadbeef download-button # Merge branch 'download-button'
> 
> is still not *quite* as intuitive as I would wish. Although, to be honest,
> if I encountered this, I would think that I should probably leave the -R
> and the -C deadbeef alone, and that I could change what is getting merged
> by changing the `download-button` parameter.

Agreed, encountering mapping is slightly more complicated, but I 
would argue it`s much more powerful at the same time, too, thus 
pretty much worth it.

Without it, actually, it seems like we`re repeating the mistake of 
`--preserve-merges`, where we`re assuming too much (order of new and 
old parents being the same, and I guess number of them, too).

Oh, and as we`re still discussing in terms of `merge` command, using 
(elsewhere mentioned[1]) `pick` instead, it might be even less 
non-intuitive, as we`re not married to `merge` semantics any more:

	pick deadbeef cafecafe:download-button


And might be calling it "non-intuitive" is unfair, I guess it would 
rather be "not familiar yet", being case with any new functionality, 
let alone a very powerful one, where getting a clue on what it does 
at the beginning could do wonders later.

Sacrificing that power for a bit of perceived simplicity, where it 
actually assumes stuff on its own (trying to stay simple for the 
user), doesn`t seem as a good way to go in the long run.

Sometimes one just needs to read the manual, and I don`t really think 
this is a ton complicated, but just something we didn`t really have 
before (real merge rebasing), so it requires a moment to grasp the 
concept.

But I`m still not sure if there isn`t a better way to present 
explicit mapping, just that <old>:<new> seemed as the most straightforward 
one to pass on the point for the purpose of discussing it.

And I`m not reluctant to simplifying user interface, or for dropping 
the explicit mapping altogether, even, just for carefully measuring 
what we could lose without explicit mapping - think flexibility, but 
ease and correctness of implementation, too, as we need to guess the 
old merge parents and which new one they should correspond to.

> > p.s. Are we moving towards `--rebase-merges` I mentioned in that 
> > other topic[1], as an add-on series after `--recreate-merges` hits 
> > the mainstream (as-is)...? :P
> 
> That's an interesting question. One that I do not want to answer alone,
> but I would be in favor of `--rebase-merges` as it is IMHO a much better
> name for what this option is all about.

Saying in favor of `--rebase-merges`, you mean as a separate option, 
alongside `--recreate-merges` (once that series lands)?

That does seem as the most clean, intuitive and straightforward 
solution. Depending on the option you provide (recreate vs rebase), 
todo list would be populated accordingly by default - but important 
thing is "todo list parser" would know to parse both, so one can 
still adapt todo list to both recreate (some) and rebase (some other) 
merges at the same time.

Of course, this only once `--rebase-merges` gets implemented, too, 
but as we had waited for it for so long, it won`t hurt to wait a bit 
more and possibly do it more properly, than rush it now and make a 
confusing user interface, needlessly welding two functionalities 
together (rebase vs. recreate).

But I guess you already knew my thoughts, so let`s see what other`s 
think, too ;)

Regards, Buga

[1] https://public-inbox.org/git/f4e6237a-84dc-1aa8-150d-041806e2416e@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Dscho,

On 11/03/2018 13:11, Johannes Schindelin wrote:
> 
> > > I did wonder about using 'pick <original-merge>' for rebasing merges
> > > and keeping 'merge ...' for recreating them but I'm not sure if that
> > > is a good idea. It has the advantage that the user cannot specify the
> > > wrong parents for the merge to be rebased as 'git rebase' would work
> > > out if the parents have been rebased, but maybe it's a bit magical to
> > > use pick for merge commits. Also there isn't such a simple way for the
> > > user to go from 'rabase this merge' to 'recreate this merge' as they'd
> > > have to write the whole merge line themselves (though I guess
> > > something like emacs' git-rebase.el would be able to help with that)
> >
> > Since the ultimate commit hashes of newly rebased commits would be
> > unknown at the time of writing the todo file, I'm not sure how this
> > would work to specify the parents?
> 
> I agree with Phillip's follow-up that the `pick <original-merge>` syntax
> would pose a problem, but for different reasons: We already tried it, with
> --preserve-merges, and it is just a really stupid syntax that does not
> allow the user even to reorder commits. Or drop commits (except at the
> very end of the todo list).

Hehe, please excuse me, but in the light of that other explicit (or 
not) parent mapping discussion[1], I would take a chance to be really 
sneaky here and say that being non-explicit "is just a really stupid 
syntax that does not allow the user even to reorder rebased merge 
parents. Or drop parents (except at the very end of the parent list)." ;)

[1] https://public-inbox.org/git/b329bb98-f9d6-3d51-2513-465aad2fa37a@gmail.com/

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Dscho,

On 11/03/2018 16:47, Johannes Schindelin wrote:
> 
> > > > Phillip's method is essentially merging the new tips into the original
> > > > merge, pretending that the new tips were not rebased but merged into
> > > > upstream.
> > >
> > > [...]
> > >
> > > Here`s a starting point, two commits A and B, merged into M:
> > >
> > > (3) ---A
> > >         \
> > >          M
> > >         /
> > >     ---B
> > >
> > >
> > > According the "patch theory"[1] (which might not be too popular 
> > > around here, but should serve the purpose for what I`m trying to 
> > > explain), each merge commit can be "transformed" to two non-merge 
> > > commits, one on top of each of the merge parents, where new commit 
> > > brings its original merge parent commit tree to the state of the 
> > > merge commit tree:
> > >
> > > (4) ---A---U1
> > >
> > >
> > >
> > >     ---B---U2
> > >
> > >
> > > Now, we have two new commits, U1 and U2, each having the same tree as 
> > > previous merge commit M, but representing changes in regards to 
> > > specific parents - and this is essentially what Sergey`s original 
> > > approach was using (whether he knew it, or not).
> > >
> > > When it comes to rebasing, it`s pretty simple, too. As this:
> > >
> > > (5) ---X1---o---o---o---o---o---X2 (master)
> > >        |\
> > >        | A1---A2---A3
> > >        |             \
> > >        |              M
> > >        |             /
> > >        \-B1---B2---B3
> > >
> > > ... actually equals this:
> > >
> > > (6) ---X1---o---o---o---o---o---X2 (master)
> > >        |\
> > >        | A1---A2---A3---U1
> > >        |
> > >        |
> > >        |
> > >        \-B1---B2---B3---U2
> > >
> > > ... where trees of M, U1 and U2 are same, and we can use the regular 
> > > rebase semantics and rebase it to this:
> > >
> > > (7) ---X1---o---o---o---o---o---X2 (master)
> > >                                 |\
> > >                                 | A1'--A2'--A3'--U1'
> > >                                 |
> > >                                 |
> > >                                 |
> > >                                 \-B1'--B2'--B3'--U2'
> > >
> > > ... which is essentially this again:
> > >
> > > (8) ---X1---o---o---o---o---o---X2 (master)
> > >                                 |\
> > >                                 | A1'--A2'--A3'
> > >                                 |            \
> > >                                 |             M'
> > >                                 |            /
> > >                                 \-B1'--B2'--B3'
> > >
> >
> > Having explained all this, I realized this is the same "essentially 
> > merging the new tips into the original pretending that the new tips 
> > were not rebased but merged into upstream" as Phillip`s one, just 
> > that we have additional temporary commits U1 and U2 (as per mentioned 
> > "patch theory") :)
> 
> But if the old tips had been merged into upstream (resulting in the new
> tips), then the merge bases would be *the old tips*.

Exactly, and that is what part you`ve cut out of the quote was 
showing :) By Phillip`s implementation, we would start with *old tips* 
as merge bases, indeed (old tips being U1 and U2 in this case), where 
it further gets transformed as previously written:

> 	git merge-recursive U1 -- M U1'
> 	tree="$(git write-tree)"
> 	git merge-recursive U2 -- $tree U2'
> 	tree="$(git write-tree)"
> 
> ..., where we know U1 = U2 = M (in regards to trees), so this is the 
> same as:
> 
> 	git merge-recursive M -- M U1'
> 	tree="$(git write-tree)"
> 	git merge-recursive M -- $tree U2'
> 	tree="$(git write-tree)"

Here, `git merge-recursive M -- M U1'` simply equals to U1' tree 
(being a fast-forward merge), so we can write the two merges above as
a single merge, too:

> 	git merge-recursive M -- U1' U2'
> 	tree="$(git write-tree)"
> 
> ... which is exactly what Sergey`s (updated) approach suggests, 
> merging U1' and U2' with M as merge-base (and shown inside that 
> sample implementation script I provided[1]) :)

So from *old tips* being the rebased merge base (Phillip), we got to 
*old merge commit* being the rebased merge base (Sergey), or vice 
versa. Does this shed a bit more light on it now? Or you wanted to 
point out something else in the first place...?

> I am still not sure for what scenarios Phillip's strategy is the same as
> Sergey's (updated) one, as the former strategy can do completely without
> temporary commits [...]

I think the root of misunderstanding might be coming from the fact 
that Sergey was mainly describing a general concept (without a 
strictly defined implementation strategy, not being restricted to a 
specific one), where Phillip came up with a solution that eventually 
seems to use the same concept (as those transformations above should 
show), but simplifying it further inside a concrete implementation.

By saying that Phillip "simplified it", even though transformations 
shown above might show different, I mean he managed to further 
decompose what Sergey was aiming for, abstracting temporary commits U1 
and U2 out of the equation, thus making them optional, but not 
required.

So, if easier to implement and reason about, I think what Phillip 
described is a way to go to produce the rebased merged commit - but 
in case we want to have that "clean rebased merge" check that U1' == 
U2' comparison does provide, as they should really be the same (tree 
wise) in simple (and the most used?) merge rebasing, we can do that 
after the fact, even.

Might be this check could be the most useful in non-interactive 
rebases, where rebased merge parents` trees could be expected to stay 
"balanced" more often (U1' == U2'), without interactive fiddling to 
"disbalance" them. I don`t know, just thinking out loud.

> [...] and cannot introduce ambiguities when rebasing the
> changes introduced by M (i.e. the "amendmendts" we talked about).

Hmm, not following here, which ambiguities are we talking about?

Thanks, Buga

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Dscho,

I`m yet to read (and reason about) your whole (very informative) 
reply, but I just wanted to address this part first, as it might be a 
clear end-game situation already, due to a mutual agreement, all the 
rest being purely academic, interesting, but not any more (that) 
important to discuss.

On 11/03/2018 16:40, Johannes Schindelin wrote:
> 
> > For myself, I do actually favor Sergey`s approach in general, but 
> > _implemented_ through what Phillip described (or a mixture of both, to 
> > be precise). But, let me explain... :)
> 
> So as you explained later in this sub-thread, Sergey's approach is
> essentially the same as Phillip's.
> 
> I still do not understand Sergey's approach on a fundamental level. I
> mean, I can follow his instructions how to implement his algorithm, but it
> is as if I had a blindfold on and somebody guided me through a maze: I
> understand *what* I am supposed to do, but I have no clue *why*.
> 
> And admittedly, I got very frustrated when a document was thrown my way
> that is too long to read in one sitting, and all of my attempts at getting
> clear and comprehensible answers to specific questions were met with "go
> and read that document, I am sure you will understand then".
> 
> For something as fundamental to my daily workflow as an interactive rebase
> (*especially* when trying to maintain the branch topology), this is no
> good at all.
> 
> Since you already confirmed that there is essentially no difference
> between the two approaches, I will simply go with the one I understand, in
> particular I understand *why* it works.
> 
> But let's read on, maybe I will change my mind based on your explanations
> (which do answer my questions, thank you so much for that)...

No problem, I learned much myself trying to write those explanations 
in the first place, and I still need to read on yet myself, seeing 
how well my explanations actually fared :) Thank you for still 
holding on, though.

But I just wanted to point out that you can really just go with what 
Phillip described if you find that easier to reason about (and/or 
implement), there`s even no need for mind changing, as essentially, 
and in my opinion, it seems to be just a bit different implementation 
of the same concept (but not requiring temporary commits).

That said, *if* we decide we like temporary commit U1' == U2' consistency 
check (especially for non-interactive rebase, maybe), we can produce 
these after the fact for the sake of the check only.

I will come with a follow-up, but all the rest might be less important.

Regards, Buga

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Sun, 11 Mar 2018, Igor Djordjevic wrote:

> On 11/03/2018 16:47, Johannes Schindelin wrote:
> > 
> > > Having explained all this, I realized this is the same "essentially
> > > merging the new tips into the original pretending that the new tips
> > > were not rebased but merged into upstream" as Phillip`s one, just
> > > that we have additional temporary commits U1 and U2 (as per
> > > mentioned "patch theory") :)
> > 
> > But if the old tips had been merged into upstream (resulting in the
> > new tips), then the merge bases would be *the old tips*.
> 
> Exactly, and that is what part you`ve cut out of the quote was 
> showing :) By Phillip`s implementation, we would start with *old tips* 
> as merge bases, indeed (old tips being U1 and U2 in this case),

I really do not see how it would make sense to take the original merge
commit as merge base in this scenario. It makes no *logical* sense: in
which interpretation did you develop changes in two divergent directions
from that merge commit?

Whereas if you use the old tips as merge bases, you can say very easily
what those two directions were: one merged with other merge parents, the
other direction rebased on top of upstream. There. Two divergent sets of
changes that we want to reconcile ("merge"). Easy as apple pie.

> where it further gets transformed as previously written:
> 
> > 	git merge-recursive U1 -- M U1'
> > 	tree="$(git write-tree)"
> > 	git merge-recursive U2 -- $tree U2'
> > 	tree="$(git write-tree)"
> > 
> > ..., where we know U1 = U2 = M (in regards to trees), so this is the 
> > same as:
> > 
> > 	git merge-recursive M -- M U1'
> > 	tree="$(git write-tree)"
> > 	git merge-recursive M -- $tree U2'
> > 	tree="$(git write-tree)"
> 
> Here, `git merge-recursive M -- M U1'` simply equals to U1' tree 
> (being a fast-forward merge), so we can write the two merges above as
> a single merge, too:
> 
> > 	git merge-recursive M -- U1' U2'
> > 	tree="$(git write-tree)"
> > 
> > ... which is exactly what Sergey`s (updated) approach suggests, 
> > merging U1' and U2' with M as merge-base (and shown inside that 
> > sample implementation script I provided[1]) :)
> 
> So from *old tips* being the rebased merge base (Phillip), we got to 
> *old merge commit* being the rebased merge base (Sergey), or vice 
> versa. Does this shed a bit more light on it now? Or you wanted to 
> point out something else in the first place...?

Okay, I'll trust you that these stunts show that the two strategies are
equivalent as to what their results are.

The biggest difference is that it is easy for me to see the motivation
behind Phillip's strategy, whereas I am still puzzled why one would come
up with a complicated strategy that splits merge commits and re-merges
them later, and why it should work in general (I still suspect that this
is not the case).

Where "easy" meant that I had to spend 1h still to figure out why using
the unrebased merge parents as merge bases. The same amount of time did
not allow me to wrap my head around Sergey's verbose explanations.

But I'll take your word for it that the strategies are equivalent, and go
with the one that has both a simpler explanation (in my mind, at least),
and an more robust implementation.

> > I am still not sure for what scenarios Phillip's strategy is the same as
> > Sergey's (updated) one, as the former strategy can do completely without
> > temporary commits [...]
> 
> I think the root of misunderstanding might be coming from the fact 
> that Sergey was mainly describing a general concept (without a 
> strictly defined implementation strategy, not being restricted to a 
> specific one), where Phillip came up with a solution that eventually 
> seems to use the same concept (as those transformations above should 
> show), but simplifying it further inside a concrete implementation.

Well, Sergey started off by suggesting the "rebase the patch relatively to
the first parent always" strategy, then came up with a long-ish email
describing a different approach (which I slowly realize is related to the
first strategy, and it would have been *much* appreciated if it was not
left to the reader to figure that one out), then incorporated what I
called your hack (again, no clear and concise description what changed,
just throwing a bunch of big bones to the dogs with the next long-ish
document).

So I will not apologize for stopping to pay so much attention to that
sub-thread at some point.

> By saying that Phillip "simplified it", even though transformations
> shown above might show different, I mean he managed to further decompose
> what Sergey was aiming for, abstracting temporary commits U1 and U2 out
> of the equation, thus making them optional, but not required.

That is not how I read Phillip's mail. It was more like "how about this
instead". And it was simple enough, with clear example how to implement
it, that I thought about that single mail for an hour, until I was
satisfied that the motivation behind this strategy is sound.

Then you confirmed that it worked on your examples, and that's what
settled the score here.

> So, if easier to implement and reason about, I think what Phillip 
> described is a way to go to produce the rebased merged commit - but 
> in case we want to have that "clean rebased merge" check that U1' == 
> U2' comparison does provide, as they should really be the same (tree 
> wise) in simple (and the most used?) merge rebasing, we can do that 
> after the fact, even.

I do not know what that U1' == U2' check would buy us, as Phillip's
strategy does not require rebasing the amendments *twice*.

> Might be this check could be the most useful in non-interactive 
> rebases, where rebased merge parents` trees could be expected to stay 
> "balanced" more often (U1' == U2'), without interactive fiddling to 
> "disbalance" them. I don`t know, just thinking out loud.

Again, Phillip's strategy does not leave things to be "disbalanced".

> > [...] and cannot introduce ambiguities when rebasing the
> > changes introduced by M (i.e. the "amendmendts" we talked about).
> 
> Hmm, not following here, which ambiguities are we talking about?

U1' vs U2' of course. Those are two things that can be different, even if
they ideally would have identical trees.

Phillip's strategy does not leave that room for ambiguity.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Sun, 11 Mar 2018, Igor Djordjevic wrote:

> I agree with both of you that `pick <merge-commit>` is inflexible 
> (not to say just plain wrong), but I never thought about it like that.
> 
> If we are to extract further mentioned explicit old:new merge 
> parameter mapping to a separate discussion point, what we`re 
> eventually left with is just replacing this:
> 
> 	merge -R -C <original--merge-commit> <merge-head>
> 
> ... with this:
> 
> 	pick <original--merge-commit> <merge-head>

I see where you are coming from.

I also see where users will be coming from. Reading a todo list in the
editor is as much documentation as it is a "program to execute". And I am
afraid that reading a command without even mentioning the term "merge"
once is pretty misleading in this setting.

And even from the theoretical point of view: cherry-picking non-merge
commits is *so much different* from "rebasing merge commits" as discussed
here, so much so that using the same command would be even more
misleading.

> That is what I had in mind, seeming possibly more straightforward and 
> beautifully aligned with previously existing (and well known) 
> `rebase` terminology.
> 
> Not to say this would make it possible to use other `rebase -i` todo 
> list commands, too, like if you want to amend/edit merge commit after 
> it was rebased, you would write:
> 
> 	edit <original--merge-commit> <merge-head>
> 
> ..., where in case you would simply like to reword its commit 
> message, it would be just:
> 
> 	reword <original--merge-commit> <merge-head>
> 
> 
> Even `squash` and `fixup` could have their place in combination with 
> a (to be rebased) merge commit, albeit in a pretty exotic rebases, 
> thus these could probably be just disallowed - for the time being, at 
> least.

Sure, for someone who read the manual, that would be easy to use. Of
course, that's the minority.

Also: the `edit` command is poorly named to begin with. A much cleaner
design would be to introduce the `break` command as suggested by Stephan.

> The real power would be buried in implementation, learning to rebase 
> merge commits, so user is left with a very familiar interface, slightly 
> adapted do accommodate a bit different nature of merge commit in 
> comparison to an ordinary one, also to allow a bit more of interactive 
> rebase functionality, but it would pretty much stay the same, without 
> even a need to learn about new `merge`, `-R`, `-C`, and so on.
> 
> Yes, those would have its purpose, but for real merging then 
> (creating new merges, or recreating old ones), not necessarily for 
> merge rebasing.
> 
> With state of `merge -R -C ...` (that `-R` being the culprit), it 
> kind of feels like we`re now trying to bolt "rebase merges" 
> functionality onto a totally different one (recreate merges, serving 
> a different purpose), making them both needlessly dependent on each 
> other, further complicating user interface, making it more confusing 
> and less tunable as per each separate functionality needs (rebase vs. 
> recreate).
> 
> I guess I`m the one to pretty much blame here, too, as I really 
> wanted `--recreate-merges` to handle "rebase merges" better, only to 
> later realize it might not be the best tool for the job, and that a 
> more separate approach would be better (at least not through the same 
> `merge` todo list command)...
> 
> [1] https://public-inbox.org/git/f3872fb9-01bc-b2f1-aee9-cfc0e4db77d6@gmail.com/

Well, the `-R` option is no worse than `git merge`'s `-s <strategy>`
option (which *also* changes the strategies rather drastically).

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

I just have two thoughts to contribute as answer, so please excuse the
heavily cut quoted text:

On Sun, 11 Mar 2018, Igor Djordjevic wrote:

> Sometimes one just needs to read the manual, and I don`t really think
> this is a ton complicated, but just something we didn`t really have
> before (real merge rebasing), so it requires a moment to grasp the
> concept.

If that were the case, we would not keep getting bug reports about
--preserve-merges failing to reorder patches.

> Saying in favor of `--rebase-merges`, you mean as a separate option,
> alongside `--recreate-merges` (once that series lands)?

No. I am against yet another option. The only reason I pollute the option
name space further with --recreate-merges is that it would be confusing to
users if the new mode was called --preserve-merges=v2 (but work *totally
differently*).

Ciao,
Dscho

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

[...]

> That said, *if* we decide we like temporary commit U1' == U2' consistency 
> check (especially for non-interactive rebase, maybe), we can produce 
> these after the fact for the sake of the check only.

I don't believe interactive vs. non-interactive split is actually
helpful. I'd consider non-interactive just a special case of interactive
when user didn't edit the todo list, nothing more. No special treatment
should be required.

For one, consistency checks in both modes has similar importance, even
if only because there could be parts of history being interactively
rebased which the user didn't intend to edit, nor actually edited during
given session.

Now let me get back to pros and cons of the two approaches to rebasing
merges we have. Below I still advocate my approach by further discussing
the differences, but simultaneously I'd like to emphasize that whatever
particular way of rebasing merges will finally be used, it will be a
huge step forward and I'm glad I've raised the issue in the first place.

First, please consider the fact that my "rebase sides" method has yet
another nice property: it reduces back to original "rebase the commit"
operation when you apply it to a non-merge commit. In other words, it's
true generalization on top of rebasing of simple commit.

OTOH, Phillip's approach, when reduced to non-merge commit, still does a
version of rebase, but very specific one, and in inverse manner. I.e.,
rather than merging changes of the commit to be rebased into the new
base, it merges changes introduced by the new base into the commit being
rebased.

One consequence is that when conflict occurs, Phillip's approach will
give surprising order of ours vs theirs changes, inverted with respect
to those of the usual rebase of non-merge commit, while my approach will
give exact non-merge commit semantics. It could likely be fixed by
slightly modifying Phillip's approach, but it will make its
implementation more complex.

Another consequence is that, provided my version is used, all options
that tune "simple commit rebase" behavior will automagically work for
rebasing merge commits, in exactly the same manner. OTOH, Phillip's
approach, without special attention in implementation, will do the same
thing no matter what -m -s, or -X options say.

Yet another consequence is that my approach will likely result in better
code reuse. Even though mine seems to be harder to implement stand-alone
than Phillip's one, it should be actually easier to implement inside the
"git rebase", as it will use exactly the same machinery that "git
rebase" already uses to rebase simple commits, adding only final "git
merge-recursive" (or "git merge-resolve", or "git merge-octopus", -- any
of them  will do the job), which current implementation already performs
as well, for re-creating merges from scratch.

Second thought, unrelated to the above. To me it seems that my "rebasing
sides" approach, being entirely symmetric, is cleaner than incremental
merging suggested by Phillip, as with my approach one will still deal
with branches independently, in the same way as for simple commits,
until the single final merge operation. This comes with drawback of 1
additional step in my approach when compared to the Phillip's one
though, but then mine has definitely no issues with the exact order of
merges.

Overall, to me it seems that unmodified Phillip's approach will bring
unnecessarily wide set of new user experiences, and fixing it will
require some tricks in implementation, for no apparent reason.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Wed, 7 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > How can your approach -- which relies *very much* on having the
>> > original parent commits -- not *require* that consistency check?
>> 
>> I don't understand what you mean, sorry. Could you please point me to
>> the *require* you talk about in the original proposal?
>
> Imagine a todo list that contains this line
>
> 	merge -C abcdef 123456
>
> and now the user edits it (this is an interactive rebase, after all),
> adding another merge head:
>
> 	merge -C abcdef 987654 123456
>
> Now your strategy would have a serious problem: to find the original
> version of 987654. If there was one.

We are talking about different checks then. My method has a built-in
check that Pillip's one doesn't. All the external checks, if any, will
have to be the same.

>
>> > What would your approach (that still has no satisfyingly trivial
>> > explanation, in my mind)
>> 
>> Here is one-liner: rebase sides of the merge commit and then 3-way
>> merge them, using original merge commit as merge base.
>
> But I already pointed out how that would undo a commit having been
> dropped.

No. Not for this version. You did it for originally flawed version of
the method, that has been already fixed by addition of "using original
merge commit as merge base" in the above sentence, and that was the
exact reason for [RFC v2], that in turn is explicitly stated at the
beginning of [RFC v2].

>
>> > do if somebody edited a `merge` command and let it merge a completely
>> > unrelated commit?
>> 
>> Don't see a problem, sorry. The method should still work, provided you have
>> original merge commit and two new parents for the new merge.
>
> That is assuming a lot. That is exactly what this consistency check is
> for, that I mentioned earlier, and which you listed as a downside of
> Phillip's strategy (forgetting that your strategy has the same downside,
> so...).

Again, we are talking about different checks. My method has a built-in
check that Pillip's doesn't. All the external checks, if any, will have
to be the same.

> But I guess that you are still talking about the non-interactive version
> of the rebase, and missed that our conversation proceeded to the point
> where we want that same strategy to work *also* in the interactive version
> (and not have a completely different functionality depending whether you
> use --interactive or not)?

For me, non-interactive is an interactive one with unmodified todo list.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,

[...]

> That is misrepresenting what happened.

No, it's you who are spreading misinformation, probably unintentional,
but still.

> First, you came up with a strategy. I pointed out shortcomings that
> implied that we cannot use it unchanged. Then, Buga fixed your strategy by
> using additional steps (making the process more complicated than before,
> still without a simple-enough explanation for my liking, fixing the
> shortcomings). Then, Phillip presented a super-simple strategy and Buga
> confirmed that it also fixes the shortcomings I pointed out.

Except that you've missed very essential thing: even before Phillip
presented his method, the original has been fixed and simultaneously
became even simpler. It's now entirely described in [RFC v2] that you
apparently still refuse to read.

> I am very excited that we finally found something that works *and* is easy
> to reason about.

You have chances to be even more exited as we in fact have 2 of them
that both work and are both easy to reason about.

> Let's focus on that strategy rather than going back to the strategy which
> has known flaws and only an unsatisfyingly complex explanation.

Not that fast, as it now has no known flaws and still has surprisingly
simple explanation. It also has its own niceties that are currently
being discussed elsewhere in the thread.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Buga,
>
> On Sun, 11 Mar 2018, Igor Djordjevic wrote:
>
>> I agree with both of you that `pick <merge-commit>` is inflexible 
>> (not to say just plain wrong), but I never thought about it like that.
>> 
>> If we are to extract further mentioned explicit old:new merge 
>> parameter mapping to a separate discussion point, what we`re 
>> eventually left with is just replacing this:
>> 
>> 	merge -R -C <original--merge-commit> <merge-head>
>> 
>> ... with this:
>> 
>> 	pick <original--merge-commit> <merge-head>
>
> I see where you are coming from.
>
> I also see where users will be coming from. Reading a todo list in the
> editor is as much documentation as it is a "program to execute". And I am
> afraid that reading a command without even mentioning the term "merge"
> once is pretty misleading in this setting.
>
> And even from the theoretical point of view: cherry-picking non-merge
> commits is *so much different* from "rebasing merge commits" as discussed
> here, so much so that using the same command would be even more
> misleading.

This last statement is plain wrong when applied to the method in the
[RFC] you are replying to. Using the method in [RFC], "cherry-pick
non-merge" is nothing more or less than reduced version of generic
"cherry-pick merge", exactly as it should be.

Or, in other words, "cherry-pick merge" is generalization of
"cherry-pick non-merge" to multiple parents.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Dscho,

[...]

> I think the root of misunderstanding might be coming from the fact
> that Sergey was mainly describing a general concept (without a
> strictly defined implementation strategy, not being restricted to a
> specific one), where Phillip came up with a solution that eventually
> seems to use the same concept (as those transformations above should
> show), but simplifying it further inside a concrete implementation.

As a side-note, starting from sound general concept leaves a hope to
end-up with something like Git, while starting from an implementation,
however nice it is, gives a danger of ending-up with something like Bzr.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

[...]

> The biggest difference is that it is easy for me to see the motivation
> behind Phillip's strategy, whereas I am still puzzled why one would come
> up with a complicated strategy that splits merge commits and re-merges
> them later, and why it should work in general (I still suspect that this
> is not the case).

Because I believe that rebasing simple commit (1 parent) should be
nothing else but reduced version of rebasing any commit (N parents) at
N=1. The [RFC v2] being discussed provides exactly such a method.

OTOH, check what Phillip's version does at N=1. Is it the same as
"rebase simple commit" strategy you already happily use? If not, please
explain why it must be different.

> Where "easy" meant that I had to spend 1h still to figure out why using
> the unrebased merge parents as merge bases.

That's because you try to figure out something that is not there in the
[RFC v2]. I suggest to forget everything you've already imagined and
just read the [RFC v2] proposal afresh. It should take about 10 minutes
or less to get it. Really.

> The same amount of time did not allow me to wrap my head around
> Sergey's verbose explanations.

Honestly, I don't believe it, sorry, but I'm willing to explain anything
you wish to be explained in _[RFC v2]_.

> But I'll take your word for it that the strategies are equivalent, and go
> with the one that has both a simpler explanation (in my mind, at least),
> and an more robust implementation.

It's up to you, and it'd still be much better than what we have now, but
you will need to face the (I think unfortunate) consequences I just
summarized elsewhere in the thread.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Dscho,

On 11/03/2018 23:04, Igor Djordjevic wrote:
> 
> I`m yet to read (and reason about) your whole (very informative) 
> reply, but I just wanted to address this part first, as it might be a 
> clear end-game situation already, due to a mutual agreement, all the 
> rest being purely academic, interesting, but not any more (that) 
> important to discuss.

Ok, here`s the follow-up.

It`s "for discussion sake only", nothing really groundbreaking in 
here, I would think.

On 11/03/2018 16:40, Johannes Schindelin wrote:
> 
> > > > The main problem with this decision is that we still don't see how
> > > > and when to stop for user amendment using this method. OTOH, the
> > > > original has this issue carefully discussed.
> > >
> > > Why would we want to stop, unless there are merge conflicts?
> >
> > Because we can reliably know that something "unusual" happened - and by
> > that I don`t necessarily mean "wrong", but just might be worth user
> > inspection.
> 
> We have a similar conundrum in recursive merges. Remember how multiple
> merge bases are merged recursively? There can be merge conflicts, too, in
> *any* of the individual merges involved, and indeed, there are (under
> relatively rare circumstances).
> 
> Since we already faced that problem, and we already answered it by
> presenting possibly nested merge conflicts, I am in strong favor of
> keeping our new scenario consistent: present possibly-nested merge
> conflicts.

This is something I didn`t really know (possibly-nested merge 
conflicts already being a regular part of Git user experience), 
thanks for explaining it.

In the light of this, I can only agree, let`s keep it consistent.

If anyone ever decides / finds out there`s a better approach in 
regards to user experience, this might get revised, but it`s a 
different beast altogether, yes.

> As far as I understand, one of the arguments in favor of the current
> approach was: there is no good way to tell the user where they are, and
> how to continue from there. So better just to continue and present the
> user with the entire set of conflicts, and have an obvious way out.

Yes, I see this as the main concern, too. I would have expected that 
being in a kind of a "limbo" for a while shouldn`t be too bad, but I 
guess that`s too academic (and inexperienced) thought, and from a 
practical point of view one may not really know how to approach the 
(iterative) conflicts in the first place, not knowing his exact 
position (nor what`s to come)...?

Or, might be we _can_ provide enough clues on where we currently are 
(even if still inside some intermediate state)...? But, this still 
might be a topic for the future, indeed, and unrelated to rebasing 
merges alone (as you pointed out already).

> > For example, situation like this (M is made on A3 with `-s ours`, 
> > obsoleting Bx commits):
> >
> > (1) ---X8--X9 (master)
> >        |\
> >        | A1---A2---A3
> >        |             \
> >        |              M (topic)
> >        |             /
> >        \-B1---B2---B3
> >
> > ... where we want to rebase M onto X9 is what I would call "usual 
> > stuff", but this situation (M is still made on A3 with `-s ours`, 
> > obsoleting Bx commits, but note cherry-picked B2'):
> >
> > (2) ---X8--B2'--X9 (master)
> >        |\
> >        | A1---A2---A3
> >        |             \
> >        |              M (topic)
> >        |             /
> >        \-B1---B2---B3
> >
> > ... where we still want to rebase M onto X9 is what we might consider 
> > "unusual", because we noticed that something that shouldn`t be part 
> > of the rebased merge commit (due to previous `-s ours`) actually got 
> > in there (due to later cherry-pick), and just wanting the user to 
> > check and confirm.
> 
> We already have those scenarios when performing a regular interactive
> rebase, where a patch was already applied upstream. In the normal case,
> the user is not even shown B2, thanks to the --cherry-pick option used in
> generating the todo list.
> 
> Granted, in some cases --cherry-pick does not detect that, and then we
> generate a todo list including B2, and when that patch is applied, the
> interactive rebase stops, saying that there are no changes to be
> committed.
> 
> And this behavior is exactly the same with --recreate-merges!
> 
> So I do not think that it would make sense to bother the user *again* when
> rebasing the merge commit.

This seems fair enough. Phillip also pointed out it might be more 
annoyance then help, but as no one was really sure of the possibilities 
we`re discussing here, I thought being better to play it a bit on the 
safe side, for the first time, at least.

I would still like to see more examples of where this U1' == U2' 
check actually helps, and counter ones, where it only serves to annoy. 
Might be we only discover them in the future, though, once the new 
functionality is in use.

> If there are merge conflicts, yes, we will have to. If there are none
> (even if your U1' != U2'), it would be outright annoying to stop.

I hope you`re right :)

> > > > "rebase sides of the merge commit and then three-way merge them back
> > > > using original merge commit as base"
> > >
> > > And that is also wrong, as I had proved already! Only Buga's addition
> > > made it robust against dropping/modifying commits, and that addition
> > > also makes it more complicated.
> >
> > No, this is actually right, that sentence nicely describing _how_ it 
> > works.
> 
> Does it? Because that's exactly backwards from how Phillip's approach
> works: it certainly does not use the original merge commit as base. It
> uses the non-rebased merge parent as base, one at a time.

Hmm, might we are all misunderstanding each other here - what I meant 
is that what Sergey wrote about his approach is exactly how _his_ 
approach works (quoted above): "rebase sides of the merge commit and 
then three-way merge them back using original merge commit as base".

> So the closest I got was the description with the *original merge commit*
> as merge base, which I disagree with. It does not make sense to me.
> 
> Consecutive three-way merges between the original merge commit and the
> merge parents (with the *original merge parents* as merge base,
> respectively), still makes the most sense to me: it is a merge between the
> amendmends to the merge commit and the changes introduced by rebasing the
> merge parents.

This is what I hopefully managed to explain as essentially being the 
same concept, using those "merge-recursive transformations", where we 
can get from using *original merge parents* as a merge base, to using 
*original merge commit* as a merge base[1] (when using U1 and U2).

> I am now getting the sense as if Sergey's approach (with your, let's say,
> "fix") is trying to apply too much, and by using the original merge commit
> as merge base then tries to undo part of that.

This is an interesting way to look at it. If you observe rebased 
temporary commits U1' and U2' alone, then I guess the answer would be 
*yes*, even.

But I think they should only be observed as an intermediate step to 
reach a final (rebased) merge commit, and the missing (and inseparable) 
link is that original merge commit used as a base for merging them.

They _did_ become from that original merge commit (tree) in the first 
place, merge commit alone transformed to more separate (but dependent 
commits), thus they are still somewhat related/dependent to it after 
their rebasing and prior the final merge (more on it in a graph later).

> > > > I honestly don't see any advantages of Phillip's method over the
> > > > original, except personal preferences. At the same time, I have no
> > > > objection of using it either, provided consistency check problem is
> > > > solved there as well.
> > >
> > > Okay, let me reiterate then, because I do not want this point to be
> > > missed:
> > >
> > > Phillip's method is essentially merging the new tips into the original
> > > merge, pretending that the new tips were not rebased but merged into
> > > upstream.
> > >
> > > So it exploits the duality of the rebase and merge operation, which both
> > > result in identical trees (potentially after resolving merge conflicts).
> > >
> > > I cannot think of any such interpretation for your proposal augmented by
> > > Buga's fix-ups. And I haven't heard any such interpretation from your
> > > side, either.
> >
> > Ok here it goes (I might be still wrong, but please bare with me).
> >
> > What Sergey originally described also uses the "duality of the rebase 
> > and merge operation", too ;) Just in a bit different way (and might 
> > be a more straightforward one?).
> 
> I will be the judge whether it looks more straight-forward to me. :-)

Hehe, fair enough ;) I was speaking mainly for myself (at the time) :)

> > Here`s a starting point, two commits A and B, merged into M:
> >
> > (3) ---A
> >         \
> >          M
> >         /
> >     ---B
> >
> >
> > According the "patch theory"[1] (which might not be too popular 
> > around here, but should serve the purpose for what I`m trying to 
> > explain),
> 
> I think that the patch theory is not usually quoted here because
> 
> 1) originally, Darcs had this snake oil way of (over-)selling its theory
>    as having "roots in quantum mechanics" [*1*] at the time Git took off,
>    and
> 
> 2) it does not really have a good concept of rebasing, even if it should
>    be theoretically possible to integrate that into the "theory of
>    patches".

Yeah, I actually remembered some Linus` comment about "theory of 
patches" being nice... well, in theory (like for scientist), but not 
that much useful in practice - I`m paraphrasing now, here`s a proper 
reference[2].

Basically, it did provide me with a hint to take "patch theory" with 
a grain of salt, just that I did seem to find a logical explanation 
in there for what Sergey was originally describing, and what otherwise 
did "feel" as a good thing, but I wanted some more support on _why_ 
it feels so (on why it works, as you asked yourself, while it did seem 
to work, nonetheless).

> > each merge commit can be "transformed" to two non-merge commits, one on
> > top of each of the merge parents, where new commit brings its original
> > merge parent commit tree to the state of the merge commit tree:
> >
> > (4) ---A---U1
> >
> >
> >
> >     ---B---U2
> 
> You get the same result if you stick to regular graph theory, of course.
> All you need to do is to interpret the directed vertices between nodes as
> a patch & parent relationship. Simple.

Not knowing much about graph theory myself (other than what I grasped 
through Git so far), I`ll take your word here - and might be educate 
myself more about it, too :)

> > Now, we have two new commits, U1 and U2, each having the same tree as 
> > previous merge commit M, but representing changes in regards to 
> > specific parents - and this is essentially what Sergey`s original 
> > approach was using (whether he knew it, or not).
> >
> > When it comes to rebasing, it`s pretty simple, too. As this:
> >
> > (5) ---X1---o---o---o---o---o---X2 (master)
> >        |\
> >        | A1---A2---A3
> >        |             \
> >        |              M
> >        |             /
> >        \-B1---B2---B3
> >
> > ... actually equals this:
> >
> > (6) ---X1---o---o---o---o---o---X2 (master)
> >        |\
> >        | A1---A2---A3---U1
> >        |
> >        |
> >        |
> >        \-B1---B2---B3---U2
> >
> > ... where trees of M, U1 and U2 are same,
> 
> Okay, so you basically duplicated the merge commit and dropped its
> semantics as a merge commit. That is a very big difference to Phillip's
> approach already.

Yes, the purpose here was to explain why Sergey`s approach (still) 
works, not to compare it with Phillip`s (which also works).

From here on, we can use simple (and existing) rebase functionality / 
semantics to really rebase that original merge. There are no new 
concepts needed, it just works (TM) :)

> It opens the door to ambiguities, as we will see later.
> 
> > and we can use the regular rebase semantics and rebase it to this:
> >
> > (7) ---X1---o---o---o---o---o---X2 (master)
> >                                 |\
> >                                 | A1'--A2'--A3'--U1'
> >                                 |
> >                                 |
> >                                 |
> >                                 \-B1'--B2'--B3'--U2'
> >
> > ... which is essentially this again:
> >
> > (8) ---X1---o---o---o---o---o---X2 (master)
> >                                 |\
> >                                 | A1'--A2'--A3'
> >                                 |            \
> >                                 |             M'
> >                                 |            /
> >                                 \-B1'--B2'--B3'
> >
> > ... where it is still true that trees of U1', U2' and M' are still 
> > the same. So we managed to rebase a merge commit without ever doing a 
> > merge :) (note that, practically, we _can_ finally even merge U1' and 
> > U2' to produce M', it shouldn`t really matter as all the trees are 
> > the same, so merge will be a no-op)
> 
> U1' and U2' do not have the same tree, though *especially* when the user
> did not edit the todo list to insert/modify/drop/reorder commits.

I lost you here - how come they don`t? The only difference between U1 
and U1' will be what`s between X1 and X2, and the same is true for U2 
and U2'. And as we know U1 = U2, it means U1' = U2' still holds.

I`m following further...

> This violation of expectations is of course caused by "duplicating the
> merge commit" into U1 and U2.

It`s just a change of semantics, a natural transformation (I wouldn`t 
know a more "proper, scientific" term, sorry), all still being 
correct. But I guess we can call it "duplicating" as well.

> But let's continue.
> 
> > But, as we saw (what you`ve explained), this doesn`t really work in 
> > case one of the sides of the merge gets "disbalanced" (so to say), 
> > like dropping a commit (which could also happen non-interactively, 
> > where a commit has been cherry-picked to a different branch, but
> > previously obsoleted by `-s ours` merge).
> 
> Precisely. A *very* important counter-argument to this approach so far.
> 
> > As observed, dropped commit could still wrongly get into final merge 
> > commit tree (or cherry-picked commit wrongly not get there), due to 
> > the nature of those rebased U1 and U2 temporary commits to hold all 
> > the differences in regards to their specific merge commit parent.
> 
> The reason for this, of course, is that either U1's or U2's diff will show
> those differences, *and we still try to rebase them even if the user
> already dropped them*.
> 
> But let's continue.
> 
> > A natural improvement to original idea which Sergey eventually came 
> > up with after my findings (which you now ended up calling a hack, even, 
> > but I would respectfully disagree), is to use original merge commit M 
> > as a merge base for final merge of U1' and U2' - and here is why it 
> > works, too, and why I wouldn`t consider it a hack, but a proper (and 
> > a solid) solution.
> >
> > Merge commit M is what we are rebasing, so we should have a way to 
> > somehow learn from it, alongside U1 and U2 temporary commits - and 
> > that is what using original merge commit M as a base does for us. It 
> > helps us spot any "disbalances" that happened in comparison to original 
> > merge parent trees, which is exactly what we`re interested in.
> 
> ... except that it gets the direction wrong. Rather than trying to *avoid*
> rebasing possibly dropped changes, it tries to kind of "undo" them by
> using a merge base that does not really make sense (unless you think of it
> as a "revert").

I disagree, but I`ll explain more later. The point is that we are 
really rebasing original merge commit, where rebased merge commit 
might have missed some stuff (due to possibly dropped changes) - so 
once we successfully rebased the original merge commit, we are to 
acknowledge additional changes we didn`t catch so far (as we would 
catch additions), and act accordingly.

Using original merge commit for the purpose feels natural to me - 
it`s where we are starting from, using both parent branches evolution 
to get where we`re heading to, being a rebased merge commit.

> It would make sense if M could interpreted as a branch point. But it
> cannot, as we specifically did *not* continue to develop the merge parents
> from that merge commit.

But we did (for the sake of rebasing), transforming the very merge 
commit into those commits developed on top of its merge parents :)

> Instead, what we did was to branch off of the original branch point X1.
> 
> Reframing the rebase of a sub-branch (X1..A3) as merge with upstream,
> however, we can interpret M and A3' as revisions we want to merge, with A3
> as the merge base.
> 
> (You can think of it in terms of the "theory of patches" thusly: if X1..A3
> is represented by patch K, X1..X2 by patch L, and X2..A3' by K', then what
> we want to merge into M is K^(-1).L.K', which is precisely what A3..A3'
> translates to.)
> 
> You can also think of it as diverging changes going from A3: one direction
> was to merge (resulting in the commit M), the other direction was to
> rebase onto X2 (resulting in A3'), and a 3-way merge M <- A3 -> A3' will
> reconcile those changes (and you will want to repeat with B3/B3', too).
> 
> Let's put that into the context of your example: instead of introducing U1
> and U2, we introduce V1 and V2 right away, as temporary *merge* commits,
> where the tree of V1 is identical to the one of A3', and the tree of V2
> to that of B3'.
> 
>  ---X1---o---o---o---o---o---X2 (master)
>     |                        |\
>     |                        | A1'--A2'--A3'--V1
>     |\                       |               /
>     | -A1---A2---A3----------+---------------
>     |              \         |
>     |               M        \-B1'--B2'--B3'--V2
>     |              /                         /
>      \-B1---B2---B3--------------------------
> 
> Note: the *really* important difference is that these temporary commits
> are based on the *rebased* history rather than the *unrebased* history.

This might be an important advantage of Phillip`s "simplification", 
indeed.

> Third note: my favorite mental model is still the duality of rebasing and
> merging, in which case V1 and V2 would not have A3' and B3' as first
> parents, but X2.
> 
> Phillip's strategy is to merge M with V1 and V2.
> 
> This translates to "merge the amendments of the merge commit M with the
> changes introduced by rebasing its merge parents".

All this I agree with. And all this can be used to support what 
Sergey described, too (being a bit different, though) :) Let`s see...

 ---X1---o---o---o---o---o---X2 (master)
    |                        |\
    |                        | A1'--A2'--A3'--U1'
    |\                       |               /
    | -A1---A2---A3---U1-----+---------------
    |              \ /       |
    |               M        \-B1'--B2'--B3'--U2'
    |              / \                       /
     \-B1---B2---B3---U2---------------------


To get the original merge commit M, you can just merge U1 and U2 
again back to single commit - original commit M is a natural merge-base. 
It`s the same with rebased commits U1' and U2' - to get rebased merge 
commit, you merge them. Again, original commit M is a natural 
merge-base.

This is why I disagreed above with statement that merging U1' and U2' 
with base M doesn`t make sense. I think this shows it does.

> The really cute part about this is that (in contrast to using U1' and
> U2'), we do not merge the amendments of the merge commit multiple times,
> but exactly once. And therefore, we do not need to "undo" them by using
> the original merge commit as merge base, either (which would introduce
> many more opportunities for merge conflicts to creep in, oftentimes
> unnecessary conflicts to begin with).

This might be true, which is why I might find Phillip`s approach to 
be a further optimization of what Sergey came up with. But I would 
still like to do some tests concerning manually amended conflicts, 
will report once I do.

Though, we do not merge the amendments of the merge commit multiple 
times here either, but we do rebase them multiple times, true (being 
held in each rebased Ux' commit again).

> > In ideal case (as per "patch theory"[1]), final merge of rebased U1' 
> > and U2' with original merge commit M as a base would be rather simple, 
> > too (not to say pointless again), as both U1' and U2' would have same 
> > changes in comparison to M, namely all the changes from commit we are 
> > rebasing from (X1 above) to commit we are rebasing to (X2 above).
> >
> > But in a more complex case like this:
> >
> > (9) ---X1---B2'---o---o---o---o---X2 (master)
> >                                   |\
> >                                   | A12--A2'---B3'
> >                                   |             \
> >                                   |              M'
> >                                   |             /
> >                                   \-B1'--B3'---B4
> >
> > ..., being what we are ultimately interested in, it helps us notice 
> > all kind of changes on our rebased merge parent branches, and act 
> > accordingly (as shown by my previously posted test script).
> 
> To use the "theory of patches" to explain why Phillip's approach is so
> much more appealing: in Sergey's approach, we will rebase U1 (which is
> "sort of" B1.B2.B3 in the "theory of patches"). If the cherry-pick of B2'
> caused merge conflicts that had to be resolved, then these merge conflicts
> will have to be resolved *again* when rebasing U1 (because B2 is *part of*
> U1). And of course they will have to be resolved *again* when merging U1'
> with U2'.

I think I disagree here - U1 doesn`t purely hold B1.B2.B3, but their 
_application onto A3 tree_ - with all conflicts resolutions that might 
have happened. Thus rebasing U1 on top of A3' should be as safe as it 
gets.

Am I missing something?

> In short: Phillip's approach is a short-cut that avoids unnecessary merge
> conflicts because it avoids rebasing changes that would need to be undone
> right away anyway.

I don`t agree it avoids conflicts (as resolutions of those are already 
part of U1, I`ll need to test), but I do agree on Phillip`s approach 
seeming to be a shortcut, though.

> > All this said (and hoping anyone is still reading this), to shed some 
> > light on what I meant by "favoring Sergey`s approach in general, but 
> > _implemented_ through what Phillip described".
> >
> > I think we should use temporary commits U1 and U2, being a sound 
> > approach backed up by the "patch theory"[1], as it helps us notice 
> > any "disbalances" of the rebased merge commit parents trees (so we 
> > can stop for user inspection), finally merging them with original 
> > merge commit as a base, in order to spot additional changes on trees 
> > of the merge commit parents we are rebasing.
> 
> In Phillip's approach, we do not need to rebase the amendments of the
> merge commit M twice (or even more times, for octopus merges). Therefore,
> there is no opportunity for these imbalances.

First sentence is true, conclusion not (necessarily) - it`s not 
rebasing amendments that introduces "disbalances", but changes in 
rebased merge parents` trees other than what found inside X1.X2.

> > But, the merging steps themselves, backed up by a need to stop for 
> > conflict resolution as soon as one happens, should be performed 
> > _iteratively_, like Phillip showed us... I would think :)
> 
> From a user interface perspective, this is a bad idea: you would now have
> to communicate to the user *where* in the process they are.
> 
> And for that you would have to explain that process to them. Including
> "theory of patches" and all.
> 
> Taking me as a prime example, you now know how tedious and impractical
> that would be.

I don`t (or didn`t, at least) find this as terrible as you`re 
describing it - yes, we`d need a way to explain people that they`re 
in an intermediate state, on the way to rebase a merge commit, but 
that`s all they would really need to know.

Might be it would be enough to say "currently rebasing merge parent 
X" (or "commit X"), then later "currently rebasing merge parent Y", 
or something along the lines of what rebase already does, just to 
provide a hint so one can act accordingly (resolve conflicts), and 
--continue.

But, as you previously mentioned this already being a known issue 
(possibly nested merge conflicts), which I wasn`t really aware of, I 
would refrain from further discussing it now and just agree to do 
what we do in the other case, too, as you suggested.

> > And, for example, that would do wonders when we introduce completely 
> > new branch during an interactive rebase, too, for example:
> >
> > (10) ---X1---B2'---o---o---o---o---X2 (master)
> >         |\                        /|\
> >         | A1---A2---A3---U1       || A12--A2'---B3'---U1'
> >         |             \           ||             \
> >         |              M          ||              M'
> >         |             /           ||             /|
> >         \-B1---B2---B3---U2       |\---B3'---B4-/-|---U2'
> >                                   |               |
> >                                   \-----B1'-------/
> >
> >
> > In this situation, we would merge all temporary Ux commits using 
> > original merge M as a merge base, and then merge _that_ resulting 
> > tree with B1' (being a newly introduced merge commit parent), using 
> > whatever merge base is most appropriate between the two (in this 
> > case, X2).
> 
> The semantics of octopus merges are very different from regular recursive
> merges. I am not sure that we want to go there in this discussion...
> 
> If, on the other hand, you do not try to turn M from a regular 2-parent
> merge commit into an octopus merge during an interactive rebase, but
> instead split the B branch into two branches *before* merging the result
> into the rebased merge commit, we are in the same square as
> reordering/dropping/inserting/modifying patches: to neither of the
> presented strategies would it matter what kind of branch topology the
> merge parents have.
> 
> For the record: I am still not convinced that Phillip's and Sergey's
> approach are equivalent, even in terms of the "theory of patches". [...]

And you shouldn`t be, nor it was my wish to conclude so, but only to 
(try to) explain _why_ Sergey`s approach works _as well_, without anything 
magical about it (nor wrong), thus no need to be afraid of it, either.

> [...] But if
> they are, then Phillip's version is a shorter version that avoids applying
> changes just to revert them right away. And in a setting where each patch
> can cause merge conflicts (as the interactive rebase is), the less changes
> you have to apply, the better.

I would still like to see some comparisons (for myself), but this is 
fair enough for now, and understandable.

Thank you for a very interesting (and educative!) read :)

Regards, Buga

[1] https://public-inbox.org/git/6362804d-e204-a9e0-9ff0-51d8497ce921@gmail.com/
[2] https://public-inbox.org/git/Pine.LNX.4.64.0604030727250.3781@g5.osdl.org/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Dscho,

On 12/03/2018 11:37, Johannes Schindelin wrote:
> 
> > If we are to extract further mentioned explicit old:new merge 
> > parameter mapping to a separate discussion point, what we`re 
> > eventually left with is just replacing this:
> >
> > 	merge -R -C <original--merge-commit> <merge-head>
> >
> > ... with this:
> >
> > 	pick <original--merge-commit> <merge-head>
> 
> I see where you are coming from.
> 
> I also see where users will be coming from. Reading a todo list in the
> editor is as much documentation as it is a "program to execute". And I am
> afraid that reading a command without even mentioning the term "merge"
> once is pretty misleading in this setting.
> 
> And even from the theoretical point of view: cherry-picking non-merge
> commits is *so much different* from "rebasing merge commits" as discussed
> here, so much so that using the same command would be even more
> misleading.

I would disagree here, as it seems you`re going too much into 
implementation and theory here, where it shouldn`t really matter from 
the user`s point of view - the point is to rebase a commit, `pick` it 
from one place and plant it elsewhere.

Yes, some commits might have a bit different semantics then others 
(merge vs non-merge), but it should just be an implementation detail, 
in my opinion, no need to leak it in user`s face (more than necessary).

I feel that "merge" is a command that works really well in the 
mindset of (re)creating merges. But if we are "only" rebasing an 
existing merge, `pick` seems much more appropriate (to me, at least), 
and it aligns with what I`m already expecting `pick` to be doing.

Down below, if we are (re)creating the merge, or doing magic to 
somehow just port it over, should be irrelevant. So "rebase" equals 
"pick and plant" (port), not "merge".

> > That is what I had in mind, seeming possibly more straightforward and 
> > beautifully aligned with previously existing (and well known) 
> > `rebase` terminology.
> >
> > Not to say this would make it possible to use other `rebase -i` todo 
> > list commands, too, like if you want to amend/edit merge commit after 
> > it was rebased, you would write:
> >
> > 	edit <original--merge-commit> <merge-head>
> >
> > ..., where in case you would simply like to reword its commit 
> > message, it would be just:
> >
> > 	reword <original--merge-commit> <merge-head>
> >
> >
> > Even `squash` and `fixup` could have their place in combination with 
> > a (to be rebased) merge commit, albeit in a pretty exotic rebases, 
> > thus these could probably be just disallowed - for the time being, at 
> > least.
> 
> Sure, for someone who read the manual, that would be easy to use. Of
> course, that's the minority.

I`m not following you here - the point is these are already existing 
commands, which would still fit in just nicely, so nothing new to 
learn nor read.

Now, if we are to discuss use cases where people don`t even know what 
they`re doing, I would think that misses the point. Besides, it`s 
always easier to make more mistakes when you introduce yet more 
commands/semantics to think about/learn, and I think it can be avoided 
here, for the better.

> Also: the `edit` command is poorly named to begin with. A much cleaner
> design would be to introduce the `break` command as suggested by Stephan.

This is orthogonal to what we`re discussing. Existing commands might 
not be perfect, but that`s what we have now, so let`s be consistent, 
not putting additional burden on the user there, at least.

But for the record - I tend to agree, I often find myself wondering 
if `edit`-ed commit means `rebase` stops after applying the changes 
and _before_ making the commit itself (so we just edit and 
--continue), or _after_ it (so we edit, `commit --amend` and 
--continue).

> > The real power would be buried in implementation, learning to rebase 
> > merge commits, so user is left with a very familiar interface, slightly 
> > adapted do accommodate a bit different nature of merge commit in 
> > comparison to an ordinary one, also to allow a bit more of interactive 
> > rebase functionality, but it would pretty much stay the same, without 
> > even a need to learn about new `merge`, `-R`, `-C`, and so on.
> >
> > Yes, those would have its purpose, but for real merging then 
> > (creating new merges, or recreating old ones), not necessarily for 
> > merge rebasing.
> >
> > With state of `merge -R -C ...` (that `-R` being the culprit), it 
> > kind of feels like we`re now trying to bolt "rebase merges" 
> > functionality onto a totally different one (recreate merges, serving 
> > a different purpose), making them both needlessly dependent on each 
> > other, further complicating user interface, making it more confusing 
> > and less tunable as per each separate functionality needs (rebase vs. 
> > recreate).
> >
> > I guess I`m the one to pretty much blame here, too, as I really 
> > wanted `--recreate-merges` to handle "rebase merges" better, only to 
> > later realize it might not be the best tool for the job, and that a 
> > more separate approach would be better (at least not through the same 
> > `merge` todo list command)...
> >
> > [1] https://public-inbox.org/git/f3872fb9-01bc-b2f1-aee9-cfc0e4db77d6@gmail.com/
> 
> Well, the `-R` option is no worse than `git merge`'s `-s <strategy>`
> option (which *also* changes the strategies rather drastically).

What I wrote above, -R seems unfitting for `merge`, as we are not 
really merging, but rebasing (conceptually, not implementation wise).

And it makes even less sense in the light of that previous comment 
where "rebasing a merge" shouldn`t even allow selecting a specific 
strategy, as that would imply (re)creating the merge instead.

So using `merge` for rebasing existing (merge) commit seems as rather 
misusing the concept of `merging`.

I don`t know, I`m thinking if we are looking at todo list from 
different perspectives - to you, it seems to be a sequence of 
commands to create something new (yes, from something that already 
exists, but that`s implementation detail). In that context, using 
`merge` might have sense (even if still being a very special merge).

But to me, as we already have `pick` and not `commit` to rebase 
commits, it means we are not creating but rather reusing what we have 
(being an important concept to reason about), thus `pick` would still 
fit in for picking a merge commit just nicely, and more naturally, too.

*Maybe if* `-R` would be turned into a full fledged "rebase" (or 
"rebasing") merge strategy, written as:

	merge -s rebase -C <original--merge-commit> <merge-head>

..., might be it would make more sense to me (aligned with what 
`merge` is expected to do). But even then, I might prefer having 
`pick` as a syntactic sugar over `merge -s rebase`, at least.

In the light of this, I _could_ even see `-R` as short for `-s 
rebase`, but it still seems a bit awkward and forced.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 12/03/2018 13:56, Sergey Organov wrote:
> 
> > > I agree with both of you that `pick <merge-commit>` is inflexible 
> > > (not to say just plain wrong), but I never thought about it like that.
> > >
> > > If we are to extract further mentioned explicit old:new merge 
> > > parameter mapping to a separate discussion point, what we`re 
> > > eventually left with is just replacing this:
> > >
> > > 	merge -R -C <original--merge-commit> <merge-head>
> > >
> > > ... with this:
> > >
> > > 	pick <original--merge-commit> <merge-head>
> >
> > I see where you are coming from.
> >
> > I also see where users will be coming from. Reading a todo list in the
> > editor is as much documentation as it is a "program to execute". And I am
> > afraid that reading a command without even mentioning the term "merge"
> > once is pretty misleading in this setting.
> >
> > And even from the theoretical point of view: cherry-picking non-merge
> > commits is *so much different* from "rebasing merge commits" as discussed
> > here, so much so that using the same command would be even more
> > misleading.
> 
> This last statement is plain wrong when applied to the method in the
> [RFC] you are replying to. Using the method in [RFC], "cherry-pick
> non-merge" is nothing more or less than reduced version of generic
> "cherry-pick merge", exactly as it should be.
> 
> Or, in other words, "cherry-pick merge" is generalization of
> "cherry-pick non-merge" to multiple parents.

I think Sergey does have a point here, his approach showing it.

Phillip`s simplification might be further from it, though, but we`re 
talking implementation again - important mental model should just be 
"rebasing a commit" (merge or non-merge), how we`re doing it is 
irrelevant for the user, the point (goal) is the same.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Dscho,

On 12/03/2018 11:46, Johannes Schindelin wrote:
> 
> > Sometimes one just needs to read the manual, and I don`t really think
> > this is a ton complicated, but just something we didn`t really have
> > before (real merge rebasing), so it requires a moment to grasp the
> > concept.
> 
> If that were the case, we would not keep getting bug reports about
> --preserve-merges failing to reorder patches.

Not sure where that is heading to, but what I`m arguing about is that 
introducing new commands and concepts (`merge`, and with `-R`) just 
makes the situation even worse (more stuff to grasp).

Reusing existing concepts where possible doesn`t have this problem.

> > Saying in favor of `--rebase-merges`, you mean as a separate option,
> > alongside `--recreate-merges` (once that series lands)?
> 
> No. I am against yet another option. The only reason I pollute the option
> name space further with --recreate-merges is that it would be confusing to
> users if the new mode was called --preserve-merges=v2 (but work *totally
> differently*).

I see. So I take you`re thinking about renaming `--recreate-merges` 
to `--rebase-merges` instead?

That would seem sensible, too, I think, being the default usage mode 
in the first place. Being able to actually (re)create merges, too, 
once user goes interactive, would be "just" an additional (nice and 
powerful) feature on top of it.

Regards, Buga

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Dscho,

On 12/03/2018 11:20, Johannes Schindelin wrote:
> 
> > > [...] and cannot introduce ambiguities when rebasing the
> > > changes introduced by M (i.e. the "amendmendts" we talked about).
> >
> > Hmm, not following here, which ambiguities are we talking about?
> 
> U1' vs U2' of course. Those are two things that can be different, even if
> they ideally would have identical trees.
> 
> Phillip's strategy does not leave that room for ambiguity.

Ehm, in Sergey`s approach, this is not an issue, but a feature :)

If U1' != U2', it just means a more complex rebase happened, but it 
doesn`t compromise the result (rebased merge) in any way.

On the other hand, if U1' == U2', we can be pretty sure that merge 
rebasing went as clean as possible.

That`s the idea, at least.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

[...]

> I don`t know, I`m thinking if we are looking at todo list from
> different perspectives - to you, it seems to be a sequence of
> commands to create something new (yes, from something that already
> exists, but that`s implementation detail). In that context, using
> `merge` might have sense (even if still being a very special merge).
>
> But to me, as we already have `pick` and not `commit` to rebase
> commits, it means we are not creating but rather reusing what we have
> (being an important concept to reason about), thus `pick` would still
> fit in for picking a merge commit just nicely, and more naturally,
> too.

Exactly.

Fundamentally, a good editing tool should first be able to just safely
reproduce back what it got, only then everything else goes.

From this follows that git history-editing tool should start from sound
history representation, i.e., some text representation of the DAG that
allows to re-create the DAG back.

The simplest way is to just list all the nodes in some topological
order, along with references to the parents. Then, to simplify the list,
first parent, unless explicitly specified, could be assumed to be the
preceding item in the list.

Next, we actually need _to do_ something with this, so we convert this
to a _todo_ list by prepending action to each element of the list (isn't
it Lisp once again?). Let the action be called 'pick'. Then, e.g., the
piece of history:

           B1-------B2
          /           \
 S--M0---M1---- M2 ----M3 -- M4

from M0 to M4 will be represented like this:

skip S          # Just to have a handy reference
pick M0         # Implicit first parent (S)
pick M1         # Implicit first parent (M0)
pick M2         # Implicit first parent (M1)
pick B1 M1      # Explicit first parent
pick B2         # Implicit first parent (B1)
pick M3 M2 B2   # Explicit first and second parents
pick M4

Which basically gets us back to what you are advocating.

Here is another variant, using command options to specify parents (I
also exchanged order of branch and mainline):

skip S          # To have the base reference handy
pick M0         # Implicit first parent (S)
pick B1         # Implicit first parent (M0)
pick B2         # Implicit first parent (B1)
pick M1 -1 M0   # Explicit first parent M0
pick M2         # Implicit first parent (M1)
pick M3 -2 B2   # Implicit first parent (M2) and
                # explicit second parent B2
pick M4         # Implicit first parent (M3)

I like this one even better.

IMHO, this is indeed a good starting point. No special treatment for
merges is needed so far.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Phillip,

Phillip Wood <phillip.wood@talktalk.net> writes:

[...]

> Hi Sergey, I've been following this discussion from the sidelines,
> though I haven't had time to study all the posts in this thread in
> detail. I wonder if it would be helpful to think of rebasing a merge as
> merging the changes in the parents due to the rebase back into the
> original merge. So for a merge M with parents A B C that are rebased to
> A' B' C' the rebased merge M' would be constructed by (ignoring shell
> quoting issues)
>
> git checkout --detach M
> git merge-recursive A -- M A'
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> git merge-recursive C -- $tree C'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')

I wonder if it's OK to exchange the order of heads in the first merge
(also dropped C for brevity):

git checkout --detach A'
git merge-recursive A -- A' M
tree=$(git write-tree)
git merge-recursive B -- $tree B'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')

If so, don't the first 2 lines now read: "rebase (first parent of) M on
top of A'"?

If so, then it could be implemented so that it reduces back to regular
rebase of non-merges when applied to a single-parent commit, similar to
the method in the RFC, striking out one of advantages of the RFC.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Junio C Hamano]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> So essentially, what your cherry-pick'able commits are is a way to store
> what rerere would have stored (i.e. the set of merge conflicts together
> with their resolution)?

If rerere would have stored, I wouldn't have separate band-aid
system on top.  These fix-up commits are usually on parts that do
not get involved in textual conflicts; "rerere" which relies on
having textual conflicts (the "shape" of the text in the conflicted
region is what lets "rerere" index into its database to find the
recorded resolution) wouldn't have stored them and that is
fundamental.

> If so, what tooling do you have to identify quickly what to cherry-pick,
> given merge conflicts?

It exactly is the issue I've been trying to find ideal solution for
quite a while and not successfully.  Here is a sample thread

  https://public-inbox.org/git/xmqqeft3u0u5.fsf@gitster.mtv.corp.google.com/#t

and every message I mention "merge-fix" is relevant.

The current band-aid system punts and indexes the merge-fix changes
by merely a branch name.  When refs/merge-fix/X exists, what it
means is "When branch X is merged to an integration branch, it is
likely that the integration branch _already_ has merged an unnamed
topic that causes semantic conflicts and requires this fix-up".
This needs occasional manual adjustment---e.g. when the topic X
turns out to be a lot more stable than the other topic Y that was
causing us trouble with semantic conflicts, I may at some point
reorder the topics and have topic X advance to 'next' before topic Y
does.  And when that happens, when I merge X to 'next', because Y is
not yet in 'next', I shouldn't apply refs/merge-fix/X (often, an
attempt to cherry-pick it on top of a merge of X into 'next' would
fail, which would be a bit of safety, but not always).  What I
should do instead is to rename refs/merge-fix/X to refs/merge-fix/Y
immediately before merging X to 'next', so that the cherry-pick is
not applied.  When rebuilding 'master'->'jch'->'pu' chain, X (now in
'next') will be merged before Y (not in 'next') gets merged, and
when it is Y's turn to be merged, the merge-fix I used to apply when
merging topic X will be applied.

In the ideal world (I think I'm repeating the ideas raised in the
thread quoted), the merge-fix database should be indexed with a pair
of commit object names (e.g. a step in branch X that adds a new
callsite for function frotz() and a step in branch Y that changes
the function signature of frotz()), and teach the system to
cherry-pick refs/merge-fix/A-B to resolve semantic conflicts, when
both commits A and B appears in the integration branch for the first
time.  And make sure these are kept up-to-date across rebasing of
commits A and B.  After rebasing the topics X and Y that contained
the commits A and B, if they became C and D, the system somehow
needs to be able to locate the previous merge-fix that was valid for
A-B pair when C-D pair gets merged.

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 13/03/2018 17:10, Sergey Organov wrote:
> 
> > Hi Sergey, I've been following this discussion from the sidelines,
> > though I haven't had time to study all the posts in this thread in
> > detail. I wonder if it would be helpful to think of rebasing a merge as
> > merging the changes in the parents due to the rebase back into the
> > original merge. So for a merge M with parents A B C that are rebased to
> > A' B' C' the rebased merge M' would be constructed by (ignoring shell
> > quoting issues)
> >
> > git checkout --detach M
> > git merge-recursive A -- M A'
> > tree=$(git write-tree)
> > git merge-recursive B -- $tree B'
> > tree=$(git write-tree)
> > git merge-recursive C -- $tree C'
> > tree=$(git write-tree)
> > M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
> 
> I wonder if it's OK to exchange the order of heads in the first merge
> (also dropped C for brevity):

It should be, being "left" or "right" hand side ("theirs" or "ours") 
of the three-way merge shouldn`t matter, they`re still both equally 
compared to the merge-base.

> git checkout --detach A'
> git merge-recursive A -- A' M
> tree=$(git write-tree)
> git merge-recursive B -- $tree B'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> 
> If so, don't the first 2 lines now read: "rebase (first parent of) M on
> top of A'"?

Hmm, lol, yes...? :) So basically, this:

(1)	git checkout --detach M
	git merge-recursive A -- M A'
	tree=$(git write-tree)
	...

... is equivalent to this:

(2)	git checkout --detach A'
	git merge-recursive A -- A' M
	tree=$(git write-tree)
	...

..., being equivalent to this:

(3)	git checkout --detach A'
	git cherry-pick -m 1 M
	tree=$(git write-tree)
	...

..., where in all three cases that `$tree` is equivalent to U1' we 
discussed about so much already :)

I tested it like this as well, slightly modifying previously sent out 
script (like this one[1]), and it still seems to be working ;) Nice!

> If so, then it could be implemented so that it reduces back to regular
> rebase of non-merges when applied to a single-parent commit, similar to
> the method in the RFC, striking out one of advantages of the RFC.

I guess so, but I think it now boils down only to what one finds 
easier to reason about even more.

I`m just glad we got to U1' from this perspective as well, hopefully 
adding even more faith in the overall concept, being beaten from both 
ends and dropping out to be the same (minus minor implementation details).

Regards, Buga

[1] https://public-inbox.org/git/872944c4-ca97-9f55-a424-86d1e3299a22@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>
> On 13/03/2018 17:10, Sergey Organov wrote:
>> 
>> > Hi Sergey, I've been following this discussion from the sidelines,
>> > though I haven't had time to study all the posts in this thread in
>> > detail. I wonder if it would be helpful to think of rebasing a merge as
>> > merging the changes in the parents due to the rebase back into the
>> > original merge. So for a merge M with parents A B C that are rebased to
>> > A' B' C' the rebased merge M' would be constructed by (ignoring shell
>> > quoting issues)
>> >
>> > git checkout --detach M
>> > git merge-recursive A -- M A'
>> > tree=$(git write-tree)
>> > git merge-recursive B -- $tree B'
>> > tree=$(git write-tree)
>> > git merge-recursive C -- $tree C'
>> > tree=$(git write-tree)
>> > M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB' -pC')
>> 
>> I wonder if it's OK to exchange the order of heads in the first merge
>> (also dropped C for brevity):
>
> It should be, being "left" or "right" hand side ("theirs" or "ours") 
> of the three-way merge shouldn`t matter, they`re still both equally 
> compared to the merge-base.
>
>> git checkout --detach A'
>> git merge-recursive A -- A' M
>> tree=$(git write-tree)
>> git merge-recursive B -- $tree B'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
>> 
>> If so, don't the first 2 lines now read: "rebase (first parent of) M on
>> top of A'"?
>
> Hmm, lol, yes...? :) So basically, this:
>
> (1)	git checkout --detach M
> 	git merge-recursive A -- M A'
> 	tree=$(git write-tree)
> 	...
>
> ... is equivalent to this:
>
> (2)	git checkout --detach A'
> 	git merge-recursive A -- A' M
> 	tree=$(git write-tree)
> 	...
>
> ..., being equivalent to this:
>
> (3)	git checkout --detach A'
> 	git cherry-pick -m 1 M
> 	tree=$(git write-tree)
> 	...
>
> ..., where in all three cases that `$tree` is equivalent to U1' we 
> discussed about so much already :)

Exactly, and thanks for noticing that it's actually U1', that happens to
soon become rather handy, see below.

> I tested it like this as well, slightly modifying previously sent out 
> script (like this one[1]), and it still seems to be working ;) Nice!

Very nice of you, thanks!

Yet another outcome of this transformation is that the fist step is now
free to (and probably should) utilize all the options (-s, -X, etc.)
that usual rebase has:

git-rebase-first-parent --onto A' M
tree=$(git write-tree)

where 'git-rebase-first-parent' is whatever machinery is currently being
used to rebase simple non-merge commit.

[

Moreover, Phillip's method could further be transformed to what is in
RFC, not that I think it should, see below. Just for the sake of
completeness though, here is the essential missing transformation that
makes Phillip's method symmetric, after which it becomes true special
case of the RFC with particular rebase-first-parent implementation:

git checkout --detach A'
git merge-recursive A -- A' M
tree_U1'=$(git write-tree)
git checkout --detach B'
git merge-recursive B -- B' M
tree_U2'=$(git write-tree)
git merge-recursive M -- $tree_U1' $tree_U2'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')

]

>
>> If so, then it could be implemented so that it reduces back to regular
>> rebase of non-merges when applied to a single-parent commit, similar to
>> the method in the RFC, striking out one of advantages of the RFC.
>
> I guess so, but I think it now boils down only to what one finds 
> easier to reason about even more.

I actually think there is more to it. It's incremental asymmetric nature
of the Phillip's approach that I now find rather appealing and worth to
be used in practice.

While the RFC approach, being entirely symmetric, is nice from the POV
of theory and reasoning, yet simple to implement, the actual user
interface of Git is inherently asymmetric with respect to merges (one
merges side-branch(es) to mainline), so asymmetric approach of the
Phillip's method should give smoother user experience, even if only
because of 1 less merge.

There are still 2 issues about the implementation that need to be
discussed though:

1. Still inverted order of the second merge compared to RFC.

It'd be simple to "fix" again, except I'm not sure it'd be better, and
as there is no existing experiences with this step to follow, it
probably should be left as in the original, where it means "merge the
changes made in B' (w.r.t B) into our intermediate version of the
resulting merge".

The original Phillip's version seems to better fit the asymmetry between
mainline and side-branch handling.

The actual difference will be only in the order of ours vs theirs in
conflicts though, and thus it's not that critical.

2. The U1' == U2' consistency check in RFC that I still think is worth
to be implemented.

In application to the method being discussed, we only need the check if
the final merge went without conflicts, so the user was not already
involved, and the check itself is then pretty simple:

 "proceed without stop only if $tree = $tree_U1'"

Its equivalence to the U1' == U2' test in the RFC follows from the fact
that if M' is non-conflicting merge of U1' and U2', then M' == U1' if
and only if U2' == U1'.

Finally, here is a sketch of the implementation that I'd suggest to
use:

git-rebase-first-parent --onto A' M
tree_U1'=$(git write-tree)
git merge-recursive B -- $tree_U1' B'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
[ $conflicted_last_merge = "yes" ] ||
  trees-match $tree_U1' $tree || 
  stop-for-user-amendment

where 'git-rebase-first-parent' denotes whatever machinery is currently
being used to rebase simple non-merge commit. Handy approximation of
which for stand-alone scripting is:

git checkout --detach A' && git cherry-pick -m 1 M

[As an interesting note, observe how, after all, that original Johannes
Sixt's idea of rebasing of merge commit by cherry-picking its first
parent is back there.]

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Dscho,
>
> On 07/03/2018 08:26, Johannes Schindelin wrote:

[...]

>> Second side note: if we can fast-forward, currently we prefer that, and I
>> think we should keep that behavior with -R, too.
>
> I agree.

I'm admittedly somewhat lost in the discussion, but are you talking
fast-forward on _rebasing_ existing merge? Where would it go in any of
the suggested algorithms of rebasing and why?

I readily see how it can break merges. E.g., any "git merge --ff-only
--no-ff" merge will magically disappear. So, even if somehow supported,
fast-forward should not be performed by default during _rebasing_ of a
merge.

>> If the user wants to force a new merge, they simply remove that -R
>> flag.

Alternatively, they'd replace 'pick' with 'merge', as they already do
for other actions. "A plurality is not to be posited without necessity".

Please, _please_, don't use 'merge' command to 'pick' merge commits!
It's utterly confusing!

Thinking about it I've got an idea that what we actually need is
--no-flatten flag that, when used alone, will just tell "git rebase" to
stop flattening history, and which will be implicitly imposed by
--recreate-merges (and --preserve-merges).

Then the only thing the --recreate-merges will tune is to put 'merge'
directives into the todo list for merge commits, exactly according to
what its name suggests, while the default behavior will be to put 'pick'
with suitable syntax into the todo. And arguments to the
--recreate-merge will specify additional options for the 'merge'
directive, obviously.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 14/03/2018 15:24, Sergey Organov wrote:
> 
> > > Second side note: if we can fast-forward, currently we prefer that, and I
> > > think we should keep that behavior with -R, too.
> >
> > I agree.
> 
> I'm admittedly somewhat lost in the discussion, but are you talking
> fast-forward on _rebasing_ existing merge? Where would it go in any of
> the suggested algorithms of rebasing and why?
> 
> I readily see how it can break merges. E.g., any "git merge --ff-only
> --no-ff" merge will magically disappear. So, even if somehow supported,
> fast-forward should not be performed by default during _rebasing_ of a
> merge.

Hmm, now that you brought this up, I can only agree, of course.

What I had in my mind was more similar to "no-rebase-cousins", like 
if we can get away without actually rebasing the merge but still 
using the original one, do it. But I guess that`s not what Johannes 
originally asked about.

This is another definitive difference between rebasing (`pick`?) and 
recreating (`merge`) a merge commit - in the case where we`re rebasing, 
of course it doesn`t make sense to drop commit this time (due to 
fast-forward). This does make sense in recreating the merge (only).

> > > If the user wants to force a new merge, they simply remove that -R
> > > flag.

And this sounds wrong now, too, because we actually have _three_
possible behaviors here - (1) rebase merge commit, which should 
always do what its told (so no fast-forwarding, otherwise the whole 
concept of rebasing a merge commit doesn`t make sense), and recreate 
merge commit, which should (2) by default use fast-forward where 
possible (or whatever the settings say), but (3) also be possible to 
force a new merge as well (through standard `--no-ff`, I guess, or 
something).

> Alternatively, they'd replace 'pick' with 'merge', as they already do
> for other actions. "A plurality is not to be posited without necessity".
> 
> Please, _please_, don't use 'merge' command to 'pick' merge commits!
> It's utterly confusing!

I agree here, as previously discussed[1], but let`s hear Johannes.

> Thinking about it I've got an idea that what we actually need is
> --no-flatten flag that, when used alone, will just tell "git rebase" to
> stop flattening history, and which will be implicitly imposed by
> --recreate-merges (and --preserve-merges).
> 
> Then the only thing the --recreate-merges will tune is to put 'merge'
> directives into the todo list for merge commits, exactly according to
> what its name suggests, while the default behavior will be to put 'pick'
> with suitable syntax into the todo. And arguments to the
> --recreate-merge will specify additional options for the 'merge'
> directive, obviously.

This seem to basically boil down to what I mentioned previously[2] 
through use of new `--rebase-merges` alongside `--recreate-merges`, just 
that you named it `--no-flatten` here, but the point is the same - and 
not something Johannes liked, "polluting" rebase option space further.

I would agree with him, and settling onto `--rebase-merges` _instead_ of 
`--recreate-merges` seems as a more appropriate name, indeed, now that 
default behavior is actually merge commit rebasing and not recreating 
(recreating still being possible through user editing the todo list).

Now, the only thing left seems to be agreeing on actual command to 
use to rebase the merge commit, to `pick` it, so to say... ;)

Regards, Buga

[1] https://public-inbox.org/git/77b695d0-7564-80d7-d9e6-70a531e66eda@gmail.com/
[2] https://public-inbox.org/git/b329bb98-f9d6-3d51-2513-465aad2fa37a@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 14/03/2018 08:21, Sergey Organov wrote:
> 
> There are still 2 issues about the implementation that need to be
> discussed though:
> 
> 1. Still inverted order of the second merge compared to RFC.
> 
> It'd be simple to "fix" again, except I'm not sure it'd be better, and
> as there is no existing experiences with this step to follow, it
> probably should be left as in the original, where it means "merge the
> changes made in B' (w.r.t B) into our intermediate version of the
> resulting merge".
> 
> The original Phillip's version seems to better fit the asymmetry between
> mainline and side-branch handling.
> 
> The actual difference will be only in the order of ours vs theirs in
> conflicts though, and thus it's not that critical.

Shouldn`t this be easy to solve just by changing the order of <head> 
and <remote>, on passing to `git merge-recursive`, if needed? (or 
that`s what you meant by "simple to fix"?)

> 2. The U1' == U2' consistency check in RFC that I still think is worth
> to be implemented.

At the moment, I think we`d appreciate test cases where it actually 
proves useful, as the general consensus seems to be leaning towards 
it possibly being annoying (over-paranoid).

> In application to the method being discussed, we only need the check if
> the final merge went without conflicts, so the user was not already
> involved, and the check itself is then pretty simple:
> 
>  "proceed without stop only if $tree = $tree_U1'"
> 
> Its equivalence to the U1' == U2' test in the RFC follows from the fact
> that if M' is non-conflicting merge of U1' and U2', then M' == U1' if
> and only if U2' == U1'.

Nicely spot! I`m glad there`s still (kind of) former U1' == U2' check 
in this approach, too, in case it proves useful :)

> Finally, here is a sketch of the implementation that I'd suggest to
> use:
> 
> git-rebase-first-parent --onto A' M
> tree_U1'=$(git write-tree)
> git merge-recursive B -- $tree_U1' B'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> [ $conflicted_last_merge = "yes" ] ||
>   trees-match $tree_U1' $tree || 
>   stop-for-user-amendment

Yes, in case where we would want the "no-op merge" check (equivalent 
to U1' == U2' with original approach), this aligns with something I 
would expect.

Note that all the "rebase merge commit" steps leading to the check 
will/should probably be observed as a single one from user`s perspective 
(in worst case ending with nested conflicts we discussed), thus 
`$conflicted_last_merge` is not related to `merge-recursive` step(s) 
only, but `rebase-first-parent`, too (just in case this isn`t implied).

Might be easier to reason about simply as `[ $conflicts = "yes" ] || `

> where 'git-rebase-first-parent' denotes whatever machinery is currently
> being used to rebase simple non-merge commit. Handy approximation of
> which for stand-alone scripting is:
> 
> git checkout --detach A' && git cherry-pick -m 1 M
> 
> [As an interesting note, observe how, after all, that original Johannes
> Sixt's idea of rebasing of merge commit by cherry-picking its first
> parent is back there.]

Heh ;) It`s always a bit enlightening when people start from different 
positions, opposing ones, even (or so it may seem, at least), but 
eventually end up in the same place, through means of open (minded) 
discussion.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 14/03/2018 15:24, Sergey Organov wrote:
[...]
>> Thinking about it I've got an idea that what we actually need is
>> --no-flatten flag that, when used alone, will just tell "git rebase" to
>> stop flattening history, and which will be implicitly imposed by
>> --recreate-merges (and --preserve-merges).
>> 
>> Then the only thing the --recreate-merges will tune is to put 'merge'
>> directives into the todo list for merge commits, exactly according to
>> what its name suggests, while the default behavior will be to put 'pick'
>> with suitable syntax into the todo. And arguments to the
>> --recreate-merge will specify additional options for the 'merge'
>> directive, obviously.
>
> This seem to basically boil down to what I mentioned previously[2] 
> through use of new `--rebase-merges` alongside `--recreate-merges`, just 
> that you named it `--no-flatten` here, but the point is the same - and 
> not something Johannes liked, "polluting" rebase option space further.

Not quite so. The problem with --XXX-merges flags is that they do two
things at once: they say _what_ to do and _how_ to do it. Clean UI
designs usually have these things separate, and that's what I propose.

The --[no-]flatten says _what_ (not) to do, and --recreate-merges says
_how_ exactly it will be performed. In this model --no-flatten could
have been called, say --preserve-shape, but not --rebase-merges.

To minimize pollution, the _how_ part could rather be made option value:

--no-flatten[=<strategy>]

where <strategy> is 'rebase', 'remerge', etc.

In this case we will need separate option to specify strategy options,
if required, that will lead us to something similar to the set of merge
strategies options.

> I would agree with him, and settling onto `--rebase-merges` _instead_ of 
> `--recreate-merges` seems as a more appropriate name, indeed, now that 
> default behavior is actually merge commit rebasing and not recreating 
> (recreating still being possible through user editing the todo list).

I hope he'd be pleased to be able to say --no-flatten=remerge and get
back his current mode of operation, that he obviously has a good use
for.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>
> On 14/03/2018 08:21, Sergey Organov wrote:
>> 
>> There are still 2 issues about the implementation that need to be
>> discussed though:
>> 
>> 1. Still inverted order of the second merge compared to RFC.
>> 
>> It'd be simple to "fix" again, except I'm not sure it'd be better, and
>> as there is no existing experiences with this step to follow, it
>> probably should be left as in the original, where it means "merge the
>> changes made in B' (w.r.t B) into our intermediate version of the
>> resulting merge".
>> 
>> The original Phillip's version seems to better fit the asymmetry between
>> mainline and side-branch handling.
>> 
>> The actual difference will be only in the order of ours vs theirs in
>> conflicts though, and thus it's not that critical.
>
> Shouldn`t this be easy to solve just by changing the order of <head> 
> and <remote>, on passing to `git merge-recursive`, if needed? (or 
> that`s what you meant by "simple to fix"?)

Yes, that's exactly what I meant, except it looks cleaner as is, so I
don't think the exchange is called for.

>> 2. The U1' == U2' consistency check in RFC that I still think is worth
>> to be implemented.
>
> At the moment, I think we`d appreciate test cases where it actually 
> proves useful, as the general consensus seems to be leaning towards 
> it possibly being annoying (over-paranoid).

As we now have a simple way to actually check it even in this algorithm,
I'd suggest command-line option to either relax or enforce the check,
whatever the default is. For the default, I'd still opt for safety, as
without it we will gather little experience with this new matter.

Honestly, without this check available, I'd likely vote for at least an
option for stopping on every rebased merge, on the ground that if
rebasing a non-merge could be a trouble, rebasing a merge is at least
double-trouble, and it's not that frequent anyway. So the check we
discuss is actually a way to make all the process much less paranoid,
not more.

By the way, nobody yet commented about "rerere" behavior that basically
stops rebasing every time it fires. Do you consider it over-paranoid?

As for test cases, I have none myself, but "-s ours" merge may be an
example of an actual trouble.

If we don't treat it specially, then changes to side branch will be
silently propagated over the merge, that's obviously not what is needed,
provided user keeps his intention to leave the merge "-s ours".

If we do treat it specially, it could be the case that the merge in
question only looks like "-s ours" by pure accident, and thus changes to
the side branch should be propagated.

I don't see how we can safely proceed without stop for user assistance.
Had we already achieved some consensus on this issue?

>
>> In application to the method being discussed, we only need the check if
>> the final merge went without conflicts, so the user was not already
>> involved, and the check itself is then pretty simple:
>> 
>>  "proceed without stop only if $tree = $tree_U1'"
>> 
>> Its equivalence to the U1' == U2' test in the RFC follows from the fact
>> that if M' is non-conflicting merge of U1' and U2', then M' == U1' if
>> and only if U2' == U1'.
>
> Nicely spot! I`m glad there`s still (kind of) former U1' == U2' check 
> in this approach, too, in case it proves useful :)
>
>> Finally, here is a sketch of the implementation that I'd suggest to
>> use:
>> 
>> git-rebase-first-parent --onto A' M
>> tree_U1'=$(git write-tree)
>> git merge-recursive B -- $tree_U1' B'
>> tree=$(git write-tree)
>> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
>> [ $conflicted_last_merge = "yes" ] ||
>>   trees-match $tree_U1' $tree || 
>>   stop-for-user-amendment
>
> Yes, in case where we would want the "no-op merge" check (equivalent 
> to U1' == U2' with original approach), this aligns with something I 
> would expect.
>
> Note that all the "rebase merge commit" steps leading to the check 
> will/should probably be observed as a single one from user`s perspective 
> (in worst case ending with nested conflicts we discussed), thus 
> `$conflicted_last_merge` is not related to `merge-recursive` step(s) 
> only, but `rebase-first-parent`, too (just in case this isn`t implied).
>
> Might be easier to reason about simply as `[ $conflicts = "yes" ] || `

No. For this check it's essential to ensure that no tweaking of the
content has been performed under the hood after the user has resolved
conflicts, i.e., after he has been involved last time.

If all this is done in one "huge merge" step from user point of view,
then the check belongs to this merge, as this is the last (and the only)
one. If it's done in steps (and I vote for it), only the last merge
status is essential for the check, preceding merges don't matter.

As I said, putting myself on the user side, I'd prefer entirely separate
first step of the algorithm, exactly as written, with its own conflict
resolution, all running entirely the same way as it does with non-merge
commits. I'm used to it and don't want to learn something new without
necessity. I.e., I'd prefer to actually see it in two separate stages,
like this:

Rebasing mainline of the merge...
[.. possible conflicts resolution ..]
Merging in changes to side branch(es)...
[.. possible conflicts resolution ..]

And if the second stage gives non-trivial conflicts, I'd like to have a
simple way to just do "merge -s ours <heads>" on top of already rebased
mainline of the merge and go with it. Note that the latter is
significantly different than re-merging everything from scratch, that
would be the only choice with "all-in-one" approach, and it essentially
gives me back those simple "rebase first parent and just record other
parents" semantics when needed.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Sergey,

On 15/03/2018 07:00, Sergey Organov wrote:
> 
> > > Thinking about it I've got an idea that what we actually need is
> > > --no-flatten flag that, when used alone, will just tell "git rebase" to
> > > stop flattening history, and which will be implicitly imposed by
> > > --recreate-merges (and --preserve-merges).
> > >
> > > Then the only thing the --recreate-merges will tune is to put 'merge'
> > > directives into the todo list for merge commits, exactly according to
> > > what its name suggests, while the default behavior will be to put 'pick'
> > > with suitable syntax into the todo. And arguments to the
> > > --recreate-merge will specify additional options for the 'merge'
> > > directive, obviously.
> >
> > This seem to basically boil down to what I mentioned previously[2] 
> > through use of new `--rebase-merges` alongside `--recreate-merges`, just 
> > that you named it `--no-flatten` here, but the point is the same - and 
> > not something Johannes liked, "polluting" rebase option space further.
> 
> Not quite so. The problem with --XXX-merges flags is that they do two
> things at once: they say _what_ to do and _how_ to do it. Clean UI
> designs usually have these things separate, and that's what I propose.
> 
> The --[no-]flatten says _what_ (not) to do, and --recreate-merges says
> _how_ exactly it will be performed. In this model --no-flatten could
> have been called, say --preserve-shape, but not --rebase-merges.
> 
> To minimize pollution, the _how_ part could rather be made option value:
> 
> --no-flatten[=<strategy>]
> 
> where <strategy> is 'rebase', 'remerge', etc.
> 
> In this case we will need separate option to specify strategy options,
> if required, that will lead us to something similar to the set of merge
> strategies options.
> 
> > I would agree with him, and settling onto `--rebase-merges` _instead_ of 
> > `--recreate-merges` seems as a more appropriate name, indeed, now that 
> > default behavior is actually merge commit rebasing and not recreating 
> > (recreating still being possible through user editing the todo list).
> 
> I hope he'd be pleased to be able to say --no-flatten=remerge and get
> back his current mode of operation, that he obviously has a good use
> for.

Makes sense, I like it, thanks for elaborating. [ Especially that you 
used "(no) flatten" phrasing, where original `--preserve-merges` 
documentation says it`s used "not to flatten the history", nice touch ;) ]

Not sure if I would prefer original `recreate` instead of `remerge` 
as that particular strategy name, though, but might be I`m just more 
used to the former one at the moment.

But if I think about it more, "recreate" seems straightforward enough - 
create something (again), kind of making it more obvious that it is a 
new thing (that we are now creating, again, based on some old thing).

On the other hand, "remerge" communicates "merge something again", 
which doesn`t necessarily mean creating a new thing (based on, but 
not attached to old thing), and could also be interpreted as "merge 
existing thing again" (and leaves me wondering if it would better 
suite some other strategy possible in the future).

Not sure if that explanation suffices, but it comes to "recreate a 
merge" having more clear meaning than "remerge a merge", being 
somewhat ambiguous, thus confusing (to me, at least).

I don`t know, thinking too much?

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 15/03/2018 08:52, Sergey Organov wrote:
> 
> > > 2. The U1' == U2' consistency check in RFC that I still think is worth
> > > to be implemented.
> >
> > At the moment, I think we`d appreciate test cases where it actually 
> > proves useful, as the general consensus seems to be leaning towards 
> > it possibly being annoying (over-paranoid).
> 
> As we now have a simple way to actually check it even in this algorithm,
> I'd suggest command-line option to either relax or enforce the check,
> whatever the default is. For the default, I'd still opt for safety, as
> without it we will gather little experience with this new matter.
> 
> Honestly, without this check available, I'd likely vote for at least an
> option for stopping on every rebased merge, on the ground that if
> rebasing a non-merge could be a trouble, rebasing a merge is at least
> double-trouble, and it's not that frequent anyway. So the check we
> discuss is actually a way to make all the process much less paranoid,
> not more.
> 
> By the way, nobody yet commented about "rerere" behavior that basically
> stops rebasing every time it fires. Do you consider it over-paranoid?

I wouldn`t really know, my workflows are usually/still rather simple, I 
don`t think I`ve ever used it on purpose, and I don`t really remember 
I`ve triggered it by accident, not having it stop for amendment, at least.

You did say you find it annoying yourself, though ;) But also accepting 
it as something that probably has a good reason, though (thus not 
considering it over-paranoid, even if annoying).

> As for test cases, I have none myself, but "-s ours" merge may be an
> example of an actual trouble.
> 
> If we don't treat it specially, then changes to side branch will be
> silently propagated over the merge, that's obviously not what is needed,
> provided user keeps his intention to leave the merge "-s ours".
> 
> If we do treat it specially, it could be the case that the merge in
> question only looks like "-s ours" by pure accident, and thus changes to
> the side branch should be propagated.
> 
> I don't see how we can safely proceed without stop for user assistance.
> Had we already achieved some consensus on this issue?

I don`t know, from what Johannes said in the past, I got an 
impression that this is to be expected ("by design"), and not worth 
bothering to stop for. And he is one of the heaviest users of (merge) 
rebasing I know.

Personally, I still feel it would make sense to stop in case like 
this, indeed, but it`s just my humble (and not necessarily much 
educated) opinion.

> > > Finally, here is a sketch of the implementation that I'd suggest to
> > > use:
> > >
> > > git-rebase-first-parent --onto A' M
> > > tree_U1'=$(git write-tree)
> > > git merge-recursive B -- $tree_U1' B'
> > > tree=$(git write-tree)
> > > M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> > > [ $conflicted_last_merge = "yes" ] ||
> > >   trees-match $tree_U1' $tree || 
> > >   stop-for-user-amendment
> >
> > Yes, in case where we would want the "no-op merge" check (equivalent 
> > to U1' == U2' with original approach), this aligns with something I 
> > would expect.
> >
> > Note that all the "rebase merge commit" steps leading to the check 
> > will/should probably be observed as a single one from user`s perspective 
> > (in worst case ending with nested conflicts we discussed), thus 
> > `$conflicted_last_merge` is not related to `merge-recursive` step(s) 
> > only, but `rebase-first-parent`, too (just in case this isn`t implied).
> >
> > Might be easier to reason about simply as `[ $conflicts = "yes" ] || `
> 
> No. For this check it's essential to ensure that no tweaking of the
> content has been performed under the hood after the user has resolved
> conflicts, i.e., after he has been involved last time.
> 
> If all this is done in one "huge merge" step from user point of view,
> then the check belongs to this merge, as this is the last (and the only)
> one. If it's done in steps (and I vote for it), only the last merge
> status is essential for the check, preceding merges don't matter.

"Huge merge" step (from user point of view) is exactly how I perceived 
Johannes` opinion on it, describing it`s already part of Git user 
experience (with possible nested conflicts), while otherwise possibly 
hard to explain where we are precisely at in the moment of stopping for 
(intermediate) conflict resolution.

Thus only `$conflicts`, meaning anything in the "huge merge", as no user 
action/tweaking/involvement can happen until the "huge merge" is done.

> As I said, putting myself on the user side, I'd prefer entirely separate
> first step of the algorithm, exactly as written, with its own conflict
> resolution, all running entirely the same way as it does with non-merge
> commits. I'm used to it and don't want to learn something new without
> necessity. I.e., I'd prefer to actually see it in two separate stages,
> like this:
> 
> Rebasing mainline of the merge...
> [.. possible conflicts resolution ..]
> Merging in changes to side branch(es)...
> [.. possible conflicts resolution ..]
> 
> And if the second stage gives non-trivial conflicts, I'd like to have a
> simple way to just do "merge -s ours <heads>" on top of already rebased
> mainline of the merge and go with it. Note that the latter is
> significantly different than re-merging everything from scratch, that
> would be the only choice with "all-in-one" approach, and it essentially
> gives me back those simple "rebase first parent and just record other
> parents" semantics when needed.

I`m undecided here, and while I do see a point in what you`re saying, 
this being new to general public I dont`t think you being accustomed 
to it is a very strong argument :)

Yes, having more steps would mean more power/options to the user, but 
more complexity to explain to and guide him through as well, not really 
sure where the line should be drawn - for the first time, at least.

Also note that, for example, in case side branch(es) dropped some 
commits (interactively or otherwise), first step alone would still 
reintroduce those dropped changes, thus later possible `merge -s ours 
<heads>` would be a pretty bad "evil merge" case and a wrong thing to 
do in general.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,

[...]

>> As I said, putting myself on the user side, I'd prefer entirely separate
>> first step of the algorithm, exactly as written, with its own conflict
>> resolution, all running entirely the same way as it does with non-merge
>> commits. I'm used to it and don't want to learn something new without
>> necessity. I.e., I'd prefer to actually see it in two separate stages,
>> like this:
>> 
>> Rebasing mainline of the merge...
>> [.. possible conflicts resolution ..]
>> Merging in changes to side branch(es)...
>> [.. possible conflicts resolution ..]
>> 
>> And if the second stage gives non-trivial conflicts, I'd like to have a
>> simple way to just do "merge -s ours <heads>" on top of already rebased
>> mainline of the merge and go with it. Note that the latter is
>> significantly different than re-merging everything from scratch, that
>> would be the only choice with "all-in-one" approach, and it essentially
>> gives me back those simple "rebase first parent and just record other
>> parents" semantics when needed.
>
> I`m undecided here, and while I do see a point in what you`re saying, 
> this being new to general public I dont`t think you being accustomed 
> to it is a very strong argument :)

Sure. It's mostly that having already familiar step separate seems to be
a good idea, as well as resulting isolation of the new stuff, where I
readily agree not to granulate it further. As if the latter actually
makes any difference... Octopus merges? I mean, really?

> Yes, having more steps would mean more power/options to the user, but 
> more complexity to explain to and guide him through as well, not really 
> sure where the line should be drawn - for the first time, at least.

A good thing is that while it runs smoothly it still runs smoothly both
ways.

> Also note that, for example, in case side branch(es) dropped some 
> commits (interactively or otherwise), first step alone would still 
> reintroduce those dropped changes, thus later possible `merge -s ours 
> <heads>` would be a pretty bad "evil merge" case and a wrong thing to 
> do in general.

Except that my presumption is that the second step has been run already
and has stopped due to conflicts, so I see the conflicting result of
dropping those commits on side branch(es), check the previous state of
the right side of the conflicting merge, and decide those state, being
the result of the fist step after possibly demanding conflicts
resolution, is fine after all. Thus I just re-merge -x ours the
branch(es), instead of re-merging everythig from scratch only to finally
get back to the same result, be it evil or not, the hard way.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

On 15/03/2018 00:11, Igor Djordjevic wrote:
> 
> > > > Second side note: if we can fast-forward, currently we prefer
> > > > that, and I think we should keep that behavior with -R, too.
> > >
> > > I agree.
> > 
> > I'm admittedly somewhat lost in the discussion, but are you
> > talking fast-forward on _rebasing_ existing merge? Where would it
> > go in any of the suggested algorithms of rebasing and why?
> > 
> > I readily see how it can break merges. E.g., any "git merge
> > --ff-only --no-ff" merge will magically disappear. So, even if
> > somehow supported, fast-forward should not be performed by default
> > during _rebasing_ of a merge.
> 
> Hmm, now that you brought this up, I can only agree, of course.
> 
> What I had in my mind was more similar to "no-rebase-cousins", like 
> if we can get away without actually rebasing the merge but still 
> using the original one, do it. But I guess that`s not what Johannes 
> originally asked about.
> 
> This is another definitive difference between rebasing (`pick`?) and 
> recreating (`merge`) a merge commit - in the case where we`re rebasing, 
> of course it doesn`t make sense to drop commit this time (due to 
> fast-forward). This does make sense in recreating the merge (only).

Eh, I might take this back. I think my original interpretation (and 
agreement) to fast-forwarding is correct.

But the confusion here comes from `--no-ff` as used for merging, as 
opposed to `--no-ff` as used for rebasing. I _think_ Johannes meant 
the latter one.

In rebasing, `--no-ff` means that even if a commit inside todo list 
isn`t to be changed, do not reuse it but create a new one. Here`s 
excerpt from the docs[1]:

  --no-ff
    With --interactive, cherry-pick all rebased commits instead of 
    fast-forwarding over the unchanged ones. This ensures that the 
    entire history of the rebased branch is composed of new commits.

    Without --interactive, this is a synonym for --force-rebase.


So fast-forwarding in case of rebasing (merge commits as well) is 
something you would want by default, as it wouldn`t drop/lose 
anything, but merely reuse existing commit (if unchanged), instead of 
cherry-picking (rebasing) it into a new (merge) commit anyway.

The same goes for this part:

> > > > If the user wants to force a new merge, they simply remove that
> > > > -R flag.

This means that using `-R` flag is sensitive to `--no-ff` rebase 
option, original merge commit _reused_ (fast-forwarded) when possible 
(unchanged, and `--no-ff` not provided), or original merge commit 
_rebased_ (when changed, or `--no-ff` provided).

If `-R` flag is removed, then merge commit is always _recreated_, no 
matter if `--no-ff` option is used or not.

p.s. I`m still a bit opposed to `-R` flag in the first place, as 
discussed elsewhere[2][3], but that`s unrelated to this fast-forward 
discussion.

Related to it, though, if `pick` command would be used instead of 
`merge -R` to signal merge commit _rebasing_, it would fit into 
existing logic nicely, where `pick` is already sensitive to `--no-ff` 
option (for rebasing regular commits). Then `merge` alone could be 
naturally (and only) used for _recreating_ merge commits, as 
originally intended (and intuitively expected).

Regards, Buga

[1] https://git-scm.com/docs/git-rebase#git-rebase---no-ff
[2] https://public-inbox.org/git/77b695d0-7564-80d7-d9e6-70a531e66eda@gmail.com/
[3] https://public-inbox.org/git/a3d40dca-f508-5853-89bc-1f9ab393416b@gmail.com/

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Igor Djordjevic]

Hi Sergey,

On 16/03/2018 08:31, Sergey Organov wrote:
> 
> > > As I said, putting myself on the user side, I'd prefer entirely
> > > separate first step of the algorithm, exactly as written, with
> > > its own conflict resolution, all running entirely the same way as
> > > it does with non-merge commits. I'm used to it and don't want to
> > > learn something new without necessity. I.e., I'd prefer to
> > > actually see it in two separate stages, like this:
> > >
> > > Rebasing mainline of the merge...
> > > [.. possible conflicts resolution ..]
> > > Merging in changes to side branch(es)...
> > > [.. possible conflicts resolution ..]
> > >
> > > And if the second stage gives non-trivial conflicts, I'd like to
> > > have a simple way to just do "merge -s ours <heads>" on top of
> > > already rebased mainline of the merge and go with it. Note that
> > > the latter is significantly different than re-merging everything
> > > from scratch, that would be the only choice with "all-in-one"
> > > approach, and it essentially gives me back those simple "rebase
> > > first parent and just record other parents" semantics when
> > > needed.
> > 
> > [...]
> > 
> > Also note that, for example, in case side branch(es) dropped some 
> > commits (interactively or otherwise), first step alone would still 
> > reintroduce those dropped changes, thus later possible `merge -s ours 
> > <heads>` would be a pretty bad "evil merge" case and a wrong thing to 
> > do in general.
> 
> Except that my presumption is that the second step has been run already
> and has stopped due to conflicts, so I see the conflicting result of
> dropping those commits on side branch(es), check the previous state of
> the right side of the conflicting merge, and decide those state, being
> the result of the fist step after possibly demanding conflicts
> resolution, is fine after all. Thus I just re-merge -x ours the
> branch(es), instead of re-merging everythig from scratch only to finally
> get back to the same result, be it evil or not, the hard way.

Might be my comment missed the point here, it should have been more 
about what you said regarding "first step having its own conflict 
resolution" - in case of dropped commits on side branch(es), you would 
be trying to resolve conflicts using one tree that doesn`t/shouldn`t 
even exist anymore (rebased merge commit first parent changes), which 
might be pretty confusing, only to find the "second stage" later 
removing changes that you might have actually picked as "first stage" 
conflict resolution, making it all even worse.

Only once "huge merge" is done completely (meaning all steps involved 
in merge commit rebasing), user can have a more realistic overview of 
(possibly nested, even) conflicts to resolve (and knowing his resolution 
will actually stick).

Regarding `merge -s ours <heads>` you mention, as you say it would 
happen only after "huge merge" is complete (with possible conflicts), 
I guess it`s unrelated to having "merge commit rebasing" happen in 
one go ("huge merge"), or iteratively, in stages (from user`s 
perspective, unrelated to underlying implementation)...?

Thus I`m questioning use-case for step-by-step merge commit rebasing 
where each stage has its own conflict resolution, in the face of it 
possibly being more confusing than helpful.

Otherwise, I see the point in what you would like to accomplish with 
that `merge -s ours <heads>` (not from scratch), but I`m not sure 
what would be the most sane way to allow it, and if it would be worth 
it in the first place, seeming to be a pretty exotic use case.

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> On 15/03/2018 00:11, Igor Djordjevic wrote:
>> 
>> > > > Second side note: if we can fast-forward, currently we prefer
>> > > > that, and I think we should keep that behavior with -R, too.
>> > >
>> > > I agree.
>> > 
>> > I'm admittedly somewhat lost in the discussion, but are you
>> > talking fast-forward on _rebasing_ existing merge? Where would it
>> > go in any of the suggested algorithms of rebasing and why?
>> > 
>> > I readily see how it can break merges. E.g., any "git merge
>> > --ff-only --no-ff" merge will magically disappear. So, even if
>> > somehow supported, fast-forward should not be performed by default
>> > during _rebasing_ of a merge.
>> 
>> Hmm, now that you brought this up, I can only agree, of course.
>> 
>> What I had in my mind was more similar to "no-rebase-cousins", like 
>> if we can get away without actually rebasing the merge but still 
>> using the original one, do it. But I guess that`s not what Johannes 
>> originally asked about.
>> 
>> This is another definitive difference between rebasing (`pick`?) and 
>> recreating (`merge`) a merge commit - in the case where we`re rebasing, 
>> of course it doesn`t make sense to drop commit this time (due to 
>> fast-forward). This does make sense in recreating the merge (only).
>
> Eh, I might take this back. I think my original interpretation (and 
> agreement) to fast-forwarding is correct.
>
> But the confusion here comes from `--no-ff` as used for merging, as 
> opposed to `--no-ff` as used for rebasing. I _think_ Johannes meant 
> the latter one.
>
> In rebasing, `--no-ff` means that even if a commit inside todo list 
> isn`t to be changed, do not reuse it but create a new one. Here`s 
> excerpt from the docs[1]:
>
>   --no-ff
>     With --interactive, cherry-pick all rebased commits instead of 
>     fast-forwarding over the unchanged ones. This ensures that the 
>     entire history of the rebased branch is composed of new commits.
>
>     Without --interactive, this is a synonym for --force-rebase.
>
>
> So fast-forwarding in case of rebasing (merge commits as well) is 
> something you would want by default, as it wouldn`t drop/lose 
> anything, but merely reuse existing commit (if unchanged), instead of 
> cherry-picking (rebasing) it into a new (merge) commit anyway.

This sounds like breakage. E.g., it seems to be breaking every "-x ours"
merge out there.

Fast-forwarding existing merge, one way or another, still seems to be
wrong idea to me, as merge commit is not only about content change, but
also about joint point at particular place in the DAG.

As for fast-forwarding re-merge, explicitly requested, I'm not sure. On
one hand, it's inline with the default "git merge" behavior, on the
other hand, it still feels wrong, somehow.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Buga,

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>
> On 16/03/2018 08:31, Sergey Organov wrote:
>> 
>> > > As I said, putting myself on the user side, I'd prefer entirely
>> > > separate first step of the algorithm, exactly as written, with
>> > > its own conflict resolution, all running entirely the same way as
>> > > it does with non-merge commits. I'm used to it and don't want to
>> > > learn something new without necessity. I.e., I'd prefer to
>> > > actually see it in two separate stages, like this:
>> > >
>> > > Rebasing mainline of the merge...
>> > > [.. possible conflicts resolution ..]
>> > > Merging in changes to side branch(es)...
>> > > [.. possible conflicts resolution ..]
>> > >
>> > > And if the second stage gives non-trivial conflicts, I'd like to
>> > > have a simple way to just do "merge -s ours <heads>" on top of
>> > > already rebased mainline of the merge and go with it. Note that
>> > > the latter is significantly different than re-merging everything
>> > > from scratch, that would be the only choice with "all-in-one"
>> > > approach, and it essentially gives me back those simple "rebase
>> > > first parent and just record other parents" semantics when
>> > > needed.
>> > 
>> > [...]
>> > 
>> > Also note that, for example, in case side branch(es) dropped some 
>> > commits (interactively or otherwise), first step alone would still 
>> > reintroduce those dropped changes, thus later possible `merge -s ours 
>> > <heads>` would be a pretty bad "evil merge" case and a wrong thing to 
>> > do in general.
>> 
>> Except that my presumption is that the second step has been run already
>> and has stopped due to conflicts, so I see the conflicting result of
>> dropping those commits on side branch(es), check the previous state of
>> the right side of the conflicting merge, and decide those state, being
>> the result of the fist step after possibly demanding conflicts
>> resolution, is fine after all. Thus I just re-merge -x ours the
>> branch(es), instead of re-merging everythig from scratch only to finally
>> get back to the same result, be it evil or not, the hard way.
>
> Might be my comment missed the point here, it should have been more 
> about what you said regarding "first step having its own conflict 
> resolution" - in case of dropped commits on side branch(es), you would 
> be trying to resolve conflicts using one tree that doesn`t/shouldn`t 
> even exist anymore (rebased merge commit first parent changes), which 
> might be pretty confusing, only to find the "second stage" later 
> removing changes that you might have actually picked as "first stage" 
> conflict resolution, making it all even worse.
>
> Only once "huge merge" is done completely (meaning all steps involved 
> in merge commit rebasing), user can have a more realistic overview of 
> (possibly nested, even) conflicts to resolve (and knowing his resolution 
> will actually stick).
>
> Regarding `merge -s ours <heads>` you mention, as you say it would 
> happen only after "huge merge" is complete (with possible conflicts), 
> I guess it`s unrelated to having "merge commit rebasing" happen in 
> one go ("huge merge"), or iteratively, in stages (from user`s 
> perspective, unrelated to underlying implementation)...?
>
> Thus I`m questioning use-case for step-by-step merge commit rebasing 
> where each stage has its own conflict resolution, in the face of it 
> possibly being more confusing than helpful.
>
> Otherwise, I see the point in what you would like to accomplish with 
> that `merge -s ours <heads>` (not from scratch), but I`m not sure 
> what would be the most sane way to allow it, and if it would be worth 
> it in the first place, seeming to be a pretty exotic use case.

I find your arguments sound, except that I somehow feel that "exotic use
case" will solve 90% of conflicting merge rebases, as merges are about
the mainline in the first place. It's likely nothing more than personal
feeling though, so I'd simply agree to disagree at this point, at least
until some actual experience is gained.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Igor Djordjevic]

Hi Sergey,

On 19/03/2018 06:44, Sergey Organov wrote:
> 
> > > > > > Second side note: if we can fast-forward, currently we prefer
> > > > > > that, and I think we should keep that behavior with -R, too.
> > > > >
> > > > > I agree.
> > > >
> > > > I'm admittedly somewhat lost in the discussion, but are you
> > > > talking fast-forward on _rebasing_ existing merge? Where would it
> > > > go in any of the suggested algorithms of rebasing and why?
> > > >
> > > > I readily see how it can break merges. E.g., any "git merge
> > > > --ff-only --no-ff" merge will magically disappear. So, even if
> > > > somehow supported, fast-forward should not be performed by default
> > > > during _rebasing_ of a merge.
> > >
> > > Hmm, now that you brought this up, I can only agree, of course.
> > >
> > > What I had in my mind was more similar to "no-rebase-cousins", like 
> > > if we can get away without actually rebasing the merge but still 
> > > using the original one, do it. But I guess that`s not what Johannes 
> > > originally asked about.
> > >
> > > This is another definitive difference between rebasing (`pick`?) and 
> > > recreating (`merge`) a merge commit - in the case where we`re rebasing, 
> > > of course it doesn`t make sense to drop commit this time (due to 
> > > fast-forward). This does make sense in recreating the merge (only).
> >
> > Eh, I might take this back. I think my original interpretation (and 
> > agreement) to fast-forwarding is correct.
> >
> > But the confusion here comes from `--no-ff` as used for merging, as 
> > opposed to `--no-ff` as used for rebasing. I _think_ Johannes meant 
> > the latter one.
> >
> > In rebasing, `--no-ff` means that even if a commit inside todo list 
> > isn`t to be changed, do not reuse it but create a new one. Here`s 
> > excerpt from the docs[1]:
> >
> >   --no-ff
> >     With --interactive, cherry-pick all rebased commits instead of 
> >     fast-forwarding over the unchanged ones. This ensures that the 
> >     entire history of the rebased branch is composed of new commits.
> >
> >     Without --interactive, this is a synonym for --force-rebase.
> >
> >
> > So fast-forwarding in case of rebasing (merge commits as well) is 
> > something you would want by default, as it wouldn`t drop/lose 
> > anything, but merely reuse existing commit (if unchanged), instead of 
> > cherry-picking (rebasing) it into a new (merge) commit anyway.
> 
> This sounds like breakage. E.g., it seems to be breaking every "-x ours"
> merge out there.

Either you are not understanding how rebase fast-forward works, or 
I`m missing what you are pointing to... Mind explaining how can 
something that`s left unchanged suddenly become a breakage?

> Fast-forwarding existing merge, one way or another, still seems to be
> wrong idea to me, as merge commit is not only about content change, but
> also about joint point at particular place in the DAG.

Not sure what this has to do with rebase fast-forwarding, either - 
nothing changes for fast-forwarded (merge or non-merge) commit in 
question, both content, joint point and everything else stays exactly 
the same. If anything changed, then it can`t/won`t be fast-forwarded, 
being unchanged is a prerequisite.

Let me elaborate a bit. Here`s a starting diagram:

(1) ---X1---X2---X3 (master)
                 |\
                 | A1---A2---A3
                 |            \
                 |             M---C1---C2 (topic)
                 |            /
                 \-B1---B2---B3


With "topic" being active branch, we start interactive rebase with 
`git rebase -i master`. Generated todo list will hold commits A1 to 
A3, B1 to B3, M and C1 to C2.

Now, if we decide to `edit` commit C1, leaving everything else the 
same, fast-forward logic will make the new situation look like this:

(2) ---X1---X2---X3 (master)
                 |\
                 | A1---A2---A3
                 |            \
                 |             M---C1'--C2' (topic)
                 |            /
                 \-B1---B2---B3


Notice how only C1 and C2 changed to C1' and C2'? That`s rebase 
fast-forwarding, noticing earlier commits left unchanged, thus 
reusing original ones.

No matter what, no breakage can happen to M in this case, as it`s 
left (reused) exactly as it was - it`s fast-forward rebased.

If we `edit`-ed commit A2, we would have ended in a situation like this:

(3) ---X1---X2---X3 (master)
                 |\
                 | A1---A2'--A3'
                 |            \
                 |             M'--C1'--C2' (topic)
                 |            /
                 \-B1---B2---B3


This time we have new commits A2', A3', M', C1' and C2' - so 
everything influenced by the change that happened will be changed 
(merge commit as well), where all the rest can still be reused 
(fast-forwarded).

If we had started rebasing with `git rebase -i --no-ff master`, no 
matter which commits we `edit` (or none, even), we would end up with 
this instead:

(4) ---X1---X2---X3 (master)
                 |\
                 | A1'--A2'--A3'
                 |            \
                 |             M'--C1'--C2' (topic)
                 |            /
                 \-B1'--B2'--B3'


So even in case where nothing is changed, no rebase fast-forwarding 
is performed (as requested), and each and every commit is changed (old 
commit replaced with its rebased version, commit hash changed).

Now, if our starting position looked like this instead:

(5) ---X1---X2---X3---X4---X5 (master)
                 |\
                 | A1---A2---A3
                 |            \
                 |             M---C1---C2 (topic)
                 |            /
                 \-B1---B2---B3

... and we simply do `git rebase -i master` again, causing all side 
commits to be rebased onto X5, then no fast forwarding can be done, as 
all commits _are_ changed (by having their parent changed, no matter if 
we additionally edit them or not), so even without `--no-ff` option we 
end up with this:

(6) ---X1---X2---X3---X4---X5 (master)
                           |\
                           | A1'--A2'--A3'
                           |            \
                           |             M'--C1'--C2' (topic)
                           |            /
                           \-B1'--B2'--B3'


Does this settle your concerns, or I`m missing something?

> As for fast-forwarding re-merge, explicitly requested, I'm not sure. On
> one hand, it's inline with the default "git merge" behavior, on the
> other hand, it still feels wrong, somehow.

Regarding fast-forwarding in context of merging, in case where we are 
recreating merges (not rebasing them), following existing `git merge` 
logic might make sense, where I would expect rebasing todo list `merge` 
command to pick-up tricks from `git merge` as needed, like learning 
to accept `--no-ff` option, for example, thus not fast-forwarding 
merges (on request) even when possible.

Though, I do agree that in case you want to recreate an existing merge 
(instead of just rebasing it), `merge` command fast-forwarding might 
probably not be what you want for the most of the time, but I`m afraid 
having rebase todo list `merge` command default behavior different than 
`git merge` default one (in regards to fast-forwarding) would be 
confusing... or not?

From what I could grasp so far, usually Git commands` default 
behavior is (explained to be) chosen per "most common use case", so 
might be non fast-forwarding would be fine as default for rebase todo 
list `merge` command, even though different than `git merge` itself...?

Regards, Buga

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:

> Hi Sergey,
>
> On 19/03/2018 06:44, Sergey Organov wrote:
>> 
>> > > > > > Second side note: if we can fast-forward, currently we prefer
>> > > > > > that, and I think we should keep that behavior with -R, too.
>> > > > >
>> > > > > I agree.
>> > > >
>> > > > I'm admittedly somewhat lost in the discussion, but are you
>> > > > talking fast-forward on _rebasing_ existing merge? Where would it
>> > > > go in any of the suggested algorithms of rebasing and why?
>> > > >
>> > > > I readily see how it can break merges. E.g., any "git merge
>> > > > --ff-only --no-ff" merge will magically disappear. So, even if
>> > > > somehow supported, fast-forward should not be performed by default
>> > > > during _rebasing_ of a merge.
>> > >
>> > > Hmm, now that you brought this up, I can only agree, of course.
>> > >
>> > > What I had in my mind was more similar to "no-rebase-cousins", like 
>> > > if we can get away without actually rebasing the merge but still 
>> > > using the original one, do it. But I guess that`s not what Johannes 
>> > > originally asked about.
>> > >
>> > > This is another definitive difference between rebasing (`pick`?) and 
>> > > recreating (`merge`) a merge commit - in the case where we`re rebasing, 
>> > > of course it doesn`t make sense to drop commit this time (due to 
>> > > fast-forward). This does make sense in recreating the merge (only).
>> >
>> > Eh, I might take this back. I think my original interpretation (and 
>> > agreement) to fast-forwarding is correct.
>> >
>> > But the confusion here comes from `--no-ff` as used for merging, as 
>> > opposed to `--no-ff` as used for rebasing. I _think_ Johannes meant 
>> > the latter one.
>> >
>> > In rebasing, `--no-ff` means that even if a commit inside todo list 
>> > isn`t to be changed, do not reuse it but create a new one. Here`s 
>> > excerpt from the docs[1]:
>> >
>> >   --no-ff
>> >     With --interactive, cherry-pick all rebased commits instead of 
>> >     fast-forwarding over the unchanged ones. This ensures that the 
>> >     entire history of the rebased branch is composed of new commits.
>> >
>> >     Without --interactive, this is a synonym for --force-rebase.
>> >
>> >
>> > So fast-forwarding in case of rebasing (merge commits as well) is 
>> > something you would want by default, as it wouldn`t drop/lose 
>> > anything, but merely reuse existing commit (if unchanged), instead of 
>> > cherry-picking (rebasing) it into a new (merge) commit anyway.
>> 
>> This sounds like breakage. E.g., it seems to be breaking every "-x ours"
>> merge out there.
>
> Either you are not understanding how rebase fast-forward works, or 
> I`m missing what you are pointing to... Mind explaining how can 
> something that`s left unchanged suddenly become a breakage?

It was misunderstanding on my side indeed, sorry.

>
>> Fast-forwarding existing merge, one way or another, still seems to be
>> wrong idea to me, as merge commit is not only about content change, but
>> also about joint point at particular place in the DAG.
>
> Not sure what this has to do with rebase fast-forwarding, either - 
> nothing changes for fast-forwarded (merge or non-merge) commit in 
> question, both content, joint point and everything else stays exactly 
> the same. If anything changed, then it can`t/won`t be fast-forwarded, 
> being unchanged is a prerequisite.
>
> Let me elaborate a bit. Here`s a starting diagram:

[... detailed explanation skipped for brevity ...]

> Does this settle your concerns, or I`m missing something?

Yes, it does, thank you! Leaving as many leading commits as possible
unchanged during rebase is what fast-forward mean in this case then, and
it's pretty OK with me.

>> As for fast-forwarding re-merge, explicitly requested, I'm not sure. On
>> one hand, it's inline with the default "git merge" behavior, on the
>> other hand, it still feels wrong, somehow.
>
> Regarding fast-forwarding in context of merging, in case where we are 
> recreating merges (not rebasing them), following existing `git merge` 
> logic might make sense, where I would expect rebasing todo list `merge` 
> command to pick-up tricks from `git merge` as needed, like learning 
> to accept `--no-ff` option, for example, thus not fast-forwarding 
> merges (on request) even when possible.
>
> Though, I do agree that in case you want to recreate an existing merge 
> (instead of just rebasing it), `merge` command fast-forwarding might 
> probably not be what you want for the most of the time, but I`m afraid 
> having rebase todo list `merge` command default behavior different than 
> `git merge` default one (in regards to fast-forwarding) would be 
> confusing... or not?
>
> From what I could grasp so far, usually Git commands` default 
> behavior is (explained to be) chosen per "most common use case", so 
> might be non fast-forwarding would be fine as default for rebase todo 
> list `merge` command, even though different than `git merge`
> itself...?

As far as I can tell, fast-forward as default for "git merge" has been
chosen to avoid excessive unintended merges in typical workflows, and
therefore this decision is not actually applicable to rebasing, I think.
I'm inclined to disable merges to fast-forward during history rebasing,
at least by default.

As far as I can tell, current "--preserve-merges" does fast-forward and
breaks "git merge --ff-only --no-ff" merges, among other things it
breaks, and that was my primary concern here. Didn't check
what "--recreate-merges" does though.

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Mon, 12 Mar 2018, Sergey Organov wrote:

> [...]
> 
> Yet another consequence is that my approach will likely result in better
> code reuse.

This is a purely academic speculation. At least until somebody implements
Phillip's method. Oh wait, I already started to implement it, and it was
not exactly hard to implement:

https://github.com/dscho/git/commit/26d2858800a4e0d3cc6313ddb54dd4d2ce516f31

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Mon, 12 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > On Wed, 7 Mar 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> 
> >> > How can your approach -- which relies *very much* on having the
> >> > original parent commits -- not *require* that consistency check?
> >> 
> >> I don't understand what you mean, sorry. Could you please point me to
> >> the *require* you talk about in the original proposal?
> >
> > Imagine a todo list that contains this line
> >
> > 	merge -C abcdef 123456
> >
> > and now the user edits it (this is an interactive rebase, after all),
> > adding another merge head:
> >
> > 	merge -C abcdef 987654 123456
> >
> > Now your strategy would have a serious problem: to find the original
> > version of 987654. If there was one.
> 
> We are talking about different checks then. My method has a built-in
> check that Pillip's one doesn't.

Since you did not bother to elaborate, I have to assume that your
"built-in check" is that thing where intermediate merges can give you
conflicts?

If so, there is a possibility in Phillip's method for such conflicts, too:
we have to perform as many 3-way merges as there are parent commits.

It does make me uncomfortable to have to speculate what you meant, though.

> >> > What would your approach (that still has no satisfyingly trivial
> >> > explanation, in my mind)
> >> 
> >> Here is one-liner: rebase sides of the merge commit and then 3-way
> >> merge them, using original merge commit as merge base.
> >
> > But I already pointed out how that would undo a commit having been
> > dropped.
> 
> No. Not for this version. You did it for originally flawed version of
> the method, that has been already fixed by addition of "using original
> merge commit as merge base" in the above sentence, and that was the
> exact reason for [RFC v2], that in turn is explicitly stated at the
> beginning of [RFC v2].

Since there is no "interdiff" between your versions, it is unreasonably
hard to deduce this, and since you seem to be unwilling to explain... I'll
just leave it at that.

Ciao,
Johannes

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Mon, 12 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> > Hi Sergey,
> 
> [...]
> 
> > That is misrepresenting what happened.
> 
> No, it's you who are spreading misinformation, probably unintentional,
> but still.

Way to go, Sergey. Way to go.

> [... more of the same...]
>
> > Let's focus on that strategy rather than going back to the strategy
> > which has known flaws and only an unsatisfyingly complex explanation.
> 
> Not that fast, as it now has no known flaws and still has surprisingly
> simple explanation. It also has its own niceties that are currently
> being discussed elsewhere in the thread.

I find it surprisingly complicated. Of course, that may be just me, but I
am not exactly a noob when it comes to interactive rebases.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Tue, 13 Mar 2018, Igor Djordjevic wrote:

> On 12/03/2018 13:56, Sergey Organov wrote:
> > 
> > > > I agree with both of you that `pick <merge-commit>` is inflexible
> > > > (not to say just plain wrong), but I never thought about it like
> > > > that.
> > > >
> > > > If we are to extract further mentioned explicit old:new merge
> > > > parameter mapping to a separate discussion point, what we`re
> > > > eventually left with is just replacing this:
> > > >
> > > > 	merge -R -C <original--merge-commit> <merge-head>
> > > >
> > > > ... with this:
> > > >
> > > > 	pick <original--merge-commit> <merge-head>
> > >
> > > I see where you are coming from.
> > >
> > > I also see where users will be coming from. Reading a todo list in
> > > the editor is as much documentation as it is a "program to execute".
> > > And I am afraid that reading a command without even mentioning the
> > > term "merge" once is pretty misleading in this setting.
> > >
> > > And even from the theoretical point of view: cherry-picking
> > > non-merge commits is *so much different* from "rebasing merge
> > > commits" as discussed here, so much so that using the same command
> > > would be even more misleading.
> > 
> > This last statement is plain wrong when applied to the method in the
> > [RFC] you are replying to.

That is only because the RFC seems to go out of its way to break down a
single merge commit into as many commits as there are merge commit
parents.

This is a pretty convoluted way to think about it: if you have three
parent commits, for example, that way of thinking would introduce three
intermediate commits, one with the changes of parent 2 & 3 combined, one
with the changes of parent 1 & 3 combined, and one with the changes of
parent 1 & 2 combined.

To rebase those commits, you essentially have to rebase *every parent's
changes twice*.

It gets worse with merge commits that have 4 parents. In that case, you
have to rebase every parent's changes *three times*.

And so on.

> > Using the method in [RFC], "cherry-pick non-merge" is nothing more or
> > less than reduced version of generic "cherry-pick merge", exactly as
> > it should be.

I really get the impression that you reject Phillip's proposal on the
ground of not being yours. In other words, the purpose of this here
argument is to praise one proposal because of its heritage, rather than
trying to come up with the best solution.

On that basis, I will go with the proposal that is clearly the simplest
and does the job and gets away with avoiding unnecessary work.

> > Or, in other words, "cherry-pick merge" is generalization of
> > "cherry-pick non-merge" to multiple parents.
> 
> I think Sergey does have a point here, his approach showing it.

His approach is showing that he wants to shoehorn the "rebase a merge
commit" idea into a form where you can cherry-pick *something*.

It does not have to make sense. And to me, it really does not.

> Phillip`s simplification might be further from it, though, but we`re 
> talking implementation again - important mental model should just be 
> "rebasing a commit" (merge or non-merge), how we`re doing it is 
> irrelevant for the user, the point (goal) is the same.

Except that Phillip's simplification is not a simplification. It comes
from a different point of view: trying to reconcile the diverging changes.

Phillip's is a true generalization of the "rebase vs merge" story: it is
no longer about merging, or about rebasing, but about reconciling
divergent commit histories, with whatever tool is appropriate.

Ciao,
Dscho

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Mon, 12 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > [...]
> >
> > Where "easy" meant that I had to spend 1h still to figure out why
> > using the unrebased merge parents as merge bases.
> 
> That's because you try to figure out something that is not there in the
> [RFC v2]. I suggest to forget everything you've already imagined and
> just read the [RFC v2] proposal afresh. It should take about 10 minutes
> or less to get it. Really.
> 
> > The same amount of time did not allow me to wrap my head around
> > Sergey's verbose explanations.
> 
> Honestly, I don't believe it, sorry, but I'm willing to explain anything
> you wish to be explained in _[RFC v2]_.

No, really. If you cannot bring yourself to believe my words, then I hate
to break it to you: I am not lying.

As to "I'm willing to explain anything you wish to be explained in RFC
v2": I was asking, and asking, and asking again, for a simple summary of
the idea behind your proposal. Nothing. That was the answer.

I had to figure it out myself: the idea is to *create* fake commits,
non-merge ones, for every single merge commit parent. Those fake commits
combine the changes of *all* merge commit parents *but one*. And then
those commits are rebased, individually, with tons of opportunities for
merge conflicts. Repeated ones. And then that result is merged.

Except that there is something more convoluted going on because of that
dumb, annoying requirement that this also has to work interactively. Where
somebody like myself might have done something really annoying such as
dropping commits, or even amending them with changes that had not been in
the previous version of the merge commit parents.

So then, after doing a ton of work to rebase the original merge commit's
changes, we perform three-way merges with the already-rebased parents (or
actually, the new tips, because as I pointed out, the parent commit may
have been dropped or reordered) to *undo* those painfully rebased changes.

No matter how much you are married to RFC v2: it *does* do unnecessary
work, it *does* result in a *lot* more opportunity for merge conflicts,
and as a bonus: it even introduces the opportunity to come up with
two versions of the rebased merge commit that disagree with one another.

> > But I'll take your word for it that the strategies are equivalent, and
> > go with the one that has both a simpler explanation (in my mind, at
> > least), and an more robust implementation.
> 
> It's up to you, and it'd still be much better than what we have now, but
> you will need to face the (I think unfortunate) consequences I just
> summarized elsewhere in the thread.

No, it was not up to me. It was up to you to convince me (or for that
matter, anybody else on the Git mailing list), that your approaches are
essentially the same.

But they are not, as your RFC v2 includes a detour of unnecessary work.
Even if the end result is theoretically the same, *practically* your
approach forces a lot of work on the user, work that is often just thrown
away!

Let's take a concrete example. Like, an example that really came up. On
the Git mailing list. A tangible example that shapes my experience, and
more importantly: other Git users' experience as well.

I introduced a change to the funny construct `void *data = data;` that was
meant to fool GCC versions that could not figure out that data would not
be used uninitialized. While that shut up GCC, it upset other compilers
that now said that this new construct does not make sense. My approach was
to introduce the macro `FAKE_INIT(type, name, value)` which would be
expanded to `type name = name;` for GCC, and to `type name = (type)value`
for all other compilers.

This was a change I wanted to cook in Git for Windows for a couple of
iterations until I am sure it works as expected, also on non-Windows
platforms, and with other compilers than MSVC, GCC and Clang.

Recently Ramsay Jones spent the time to research this issue a lot deeper
than it was done before, and found out *which* GCC versions are affected,
and introduced a patch series that fixes this problem for real, *undoing*
the `void *data = data;` mess.

Obviously, this fix conflicts with my work-around.

Okay, good, so what would happen, hypothetically, if the Git garden shears
I use (and which you probably still haven't studied, even if I pointed you
to it several times, but expecting me to read RFC v2 at the same time
instead of answering my questions about it) were adjusted to use RFC v2 to
rebase merges?

To answer that, I first have to tell you that Git for Windows' branch
thicket consists of roughly 70 topic branches that are partially
criss-cross-merged. There are roughly 40 merge commits (44 if I counted
correctly) on the commit graph between HEAD and that work-around that
conflicts with Ramsay's fix.

So the first thing that would happen when rebasing the branch thicket is
this: I would encounter the merge conflict when my workaround is
cherry-picked, realize that my work-around is no longer necessary, and
call `git rebase --skip` and that's that.

The Git garden shears currently do not even try to rebase merge commits,
so that really would be that. I might encounter unrelated conflicts, but
nothing about the `void *data = data;` issue again.

With Phillip's approach, the same is true, as the 40+ merge commits would
be rebased with my work-around being undone by the first three-way merge,
the second three-way merge being unaffected (and no further 3-way merge
necessary because I do not do octopus merges in Git for Windows branch
thicket) and that's that.

(There is a chance, of course, that I misunderstood, or that I missed
something. The proof lies in the pudding. Somebody will have to try this,
and this somebody is probably me.)

With your RFC v2 approach, however, the original tips of all of those 40+
merges would be "cherry-picked" (via those intermediate fake commits).
Conflicting every single time. And of course the subsequent three-way
merge to undo the changes (because I dropped the change via `git rebase
--skip`) would conflict *again*.

That's an awful lot of merge conflicts. One might be tempted to suggest
complexifying the entire procedure by requiring `rerere`, and my initial
answer would be that this still conflicts when the context lines of the
merge conflict changed, but the truth is: this level of complexity, this
amount of merge conflicts, is not even necessary, as seen by looking at
Phillip's strategy.

This is what I referred to as "unecessary work". And to me, as a power
(read: frequent) user of the closest thing we have to --recreate-merges in
the wild, it is not funny. Not funny at all.

BTW this is the level of detail I would have wished your answers to my
repeated questions for clarification of your RFC v2 to be. And that is
what I expect in response to my valid questions in the future.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Buga,

On Tue, 13 Mar 2018, Igor Djordjevic wrote:

> On 12/03/2018 11:46, Johannes Schindelin wrote:
> > 
> > > Sometimes one just needs to read the manual, and I don`t really
> > > think this is a ton complicated, but just something we didn`t really
> > > have before (real merge rebasing), so it requires a moment to grasp
> > > the concept.
> > 
> > If that were the case, we would not keep getting bug reports about
> > --preserve-merges failing to reorder patches.
> 
> Not sure where that is heading to, but what I`m arguing about is that 
> introducing new commands and concepts (`merge`, and with `-R`) just 
> makes the situation even worse (more stuff to grasp).

The problem with re-using `pick` is that its concept does not apply to
merges. The cherry-pick of a non-merge commit is well-defined: the current
HEAD is implicitly chosen as the cherry-picked commit's (single) parent
commit. There is no ambiguity here.

But for merge commits, we need to specify the parent commits (apart from
the first one) *explicitly*. There was no need for that in the `pick`
command, nor in the concept of a cherry-pick.

> Reusing existing concepts where possible doesn`t have this problem.

Existing concepts are great. As long as they fit the requirements of the
new scenarios. In this case, `pick` does *not* fit the requirement of
"rebase a merge commit".

If you really want to force the `pick` concept onto the use case where you
need to "reapply" merges, then the closest you get really is Sergey's
idea, which I came to reject when considering its practical implications.

Even so, you would have to make the `pick` command more complicated to
support merge commits. And whatever you would do to extend the `pick`
command would *not make any sense* to the current use case of the `pick`
command.

The real problem, of course, is that a non-merge commit, when viewed from
the perspective of the changes it introduced, is a very different beast
than a merge commit: it does not need to reconcile changes, ever, because
there is really only one "patch" to one revision. That is very different
from a merge commit, whose changes can even disagree with one another (and
in fact be resolved with changes disagreeing *yet again*)!

> > > Saying in favor of `--rebase-merges`, you mean as a separate option,
> > > alongside `--recreate-merges` (once that series lands)?
> > 
> > No. I am against yet another option. The only reason I pollute the
> > option name space further with --recreate-merges is that it would be
> > confusing to users if the new mode was called --preserve-merges=v2
> > (but work *totally differently*).
> 
> I see. So I take you`re thinking about renaming `--recreate-merges` to
> `--rebase-merges` instead?

Thinking about it. Nothing will happen before v2.17.0 on that front,
though, because -- unlike you gentle people -- I have to focus on
stabilizing Git's code base now.

> That would seem sensible, too, I think, being the default usage mode in
> the first place. Being able to actually (re)create merges, too, once
> user goes interactive, would be "just" an additional (nice and powerful)
> feature on top of it.

The implementation detail is, of course, that I will introduce this with
the technically-simpler strategy: always recreating merge commits with the
recursive strategy. A follow-up patch series will add support for rebasing
merge commits, and then use it by default.

This latter part will need a lot of experimentation, though. That's why I
want the --recreate-merges patch series cooking in `next` first.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Junio,

On Tue, 13 Mar 2018, Junio C Hamano wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > If so, what tooling do you have to identify quickly what to
> > cherry-pick, given merge conflicts?
> 
> It exactly is the issue I've been trying to find ideal solution for
> quite a while and not successfully.  Here is a sample thread
> 
>   https://public-inbox.org/git/xmqqeft3u0u5.fsf@gitster.mtv.corp.google.com/#t
> 
> and every message I mention "merge-fix" is relevant.
>
> [... detailed explanation of the current "band-aid" system...]

Thank you for the very thorough account. I have been struggling with this,
too, even chatting to Michael Haggerty about this at the Contributors'
Summit (where I missed you a lot). He pointed me to a blog post of his
about the very interesting concept of "obsolete markers" in Hg:

http://softwareswirl.blogspot.de/2013/05/obsolete-markers-in-mercurial.html

Granted, that concept really makes most sense for rebase, but I wonder
whether the concept could be extended to help your (and my) use case of
frequently-changing targets (for me, it is more rebase targets, for you it
is merge targets). It would probably be implemented using commit notes, to
allow for bidirectional mappings.

Ciao,
Dscho
> 
> The current band-aid system punts and indexes the merge-fix changes
> by merely a branch name.  When refs/merge-fix/X exists, what it
> means is "When branch X is merged to an integration branch, it is
> likely that the integration branch _already_ has merged an unnamed
> topic that causes semantic conflicts and requires this fix-up".
> This needs occasional manual adjustment---e.g. when the topic X
> turns out to be a lot more stable than the other topic Y that was
> causing us trouble with semantic conflicts, I may at some point
> reorder the topics and have topic X advance to 'next' before topic Y
> does.  And when that happens, when I merge X to 'next', because Y is
> not yet in 'next', I shouldn't apply refs/merge-fix/X (often, an
> attempt to cherry-pick it on top of a merge of X into 'next' would
> fail, which would be a bit of safety, but not always).  What I
> should do instead is to rename refs/merge-fix/X to refs/merge-fix/Y
> immediately before merging X to 'next', so that the cherry-pick is
> not applied.  When rebuilding 'master'->'jch'->'pu' chain, X (now in
> 'next') will be merged before Y (not in 'next') gets merged, and
> when it is Y's turn to be merged, the merge-fix I used to apply when
> merging topic X will be applied.
> 
> In the ideal world (I think I'm repeating the ideas raised in the
> thread quoted), the merge-fix database should be indexed with a pair
> of commit object names (e.g. a step in branch X that adds a new
> callsite for function frotz() and a step in branch Y that changes
> the function signature of frotz()), and teach the system to
> cherry-pick refs/merge-fix/A-B to resolve semantic conflicts, when
> both commits A and B appears in the integration branch for the first
> time.  And make sure these are kept up-to-date across rebasing of
> commits A and B.  After rebasing the topics X and Y that contained
> the commits A and B, if they became C and D, the system somehow
> needs to be able to locate the previous merge-fix that was valid for
> A-B pair when C-D pair gets merged.
> 
> 

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 14 Mar 2018, Sergey Organov wrote:

> Igor Djordjevic <igor.d.djordjevic@gmail.com> writes:
> 
> > On 07/03/2018 08:26, Johannes Schindelin wrote:
> 
> [...]
> 
> >> Second side note: if we can fast-forward, currently we prefer that,
> >> and I think we should keep that behavior with -R, too.
> >
> > I agree.
> 
> I'm admittedly somewhat lost in the discussion,

... and cutting so much context does not help...

The "fast-forward" here refers to the thing `git rebase` does unless you
call it with `--force-rebase`: if we are about to `pick` a commit, and
HEAD already points to its parent commit, we do not evey cherry-pick: we
just fast-forward to the original commit.

Likewise, `merge -C <commit>` will simply fast-forward to the specified
commit if HEAD is pointing to its first parent and the specified merge
heads are also identical to the respective original parent commits.

What I said with my comment was that `merge -R -C <commit>` will behave in
exactly the same way.

> but are you talking fast-forward on _rebasing_ existing merge? Where
> would it go in any of the suggested algorithms of rebasing and why?

And this is something I did not talk about yet, but it does come up in
practice: the main use case of rebasing branches is to follow an
"upstream", of course. And guess what? From time to time, some of the
branches get merged upstream (or, in Git's own source code, applied). In
this case, the Git garden shears *do* skip the merge (as the new merge
head is already an ancestor of HEAD). In --recreate-merges, it *will*
create a new merge commit, though, which is a bit annoying.

The best way to handle this would *probably* look similar to how "empty"
commits (i.e. commits whose patch is empty) are handled by rebase. But it
is not yet clear to me how that would look in practice, as `pick <commit>`
is a single operation that can be easily commented out in the todo list
depending on `--allow-empty`, while the `merge -C <commit>` command is not
the entire operation, as there may be `pick` commands in the merge head
that could potentially be skipped due to merge conflicts with
already-applied versions of the same patches.

If this sounds unclear to you, please do ask for clarification. Although
this is currently not my highest priority in the `sequencer-shears` branch
thicket (where `--recreate-merges` is the first part).

> I readily see how it can break merges. E.g., any "git merge --ff-only
> --no-ff" merge will magically disappear.

Did you mean `git merge --no-ff`? Combining `--ff-only` with `--no-ff`
does not make sense.

> So, even if somehow supported, fast-forward should not be performed by
> default during _rebasing_ of a merge.

That statement is too general to be correct.

*If* we detect that the original merge could have been fast-forwarded
instead (and it is very easy to detect that in --make-list), we would have
to handle that similar to afore-mentioned empty commits in conjunction
with `--allow-empty`.

If the original merge could not have been fast-forwarded, but during the
rebase it *can*, we should skip it, as the reason for the merge commit
is clearly no longer there.

This, by the way, is an insight you can really only win by using
--recreate-merges (or the Git garden shears). And using it a *lot*.
Because otherwise, you will not even guess correctly what will, and what
won't, come up in practice.

> >> If the user wants to force a new merge, they simply remove that -R
> >> flag.
> 
> Alternatively, they'd replace 'pick' with 'merge', as they already do
> for other actions. "A plurality is not to be posited without necessity".

No, no, no and no again!

We learned how *wrong* the `pick` approach was with --preserve-merges!
Have you *ever* used --preserve-merges in any real way? If so, you *have*
encountered the problems, and probably directed several lengthy curses my
way for that lousy design.

A pick is a pick is a pick. Of a single patch with metadata such as the
author, commit message and the date.

A merge is not a patch. While a pick introduces a specific change on top
of a single revision, the changes introduced by a merge are ideally
already there. It is conceptually *very* different.

Sure, a merge commit sometimes needs to introduce extra changes on top of
the merged changes, sure, that happens (and is called "evil merge" in Git
parlance). Those *additional* changes should be kept as minimal as
possible, in particular there should only be changes *necessitated* by the
reconciled changes.

So no, a `pick` is not a `merge`. Not at all.

> Please, _please_, don't use 'merge' command to 'pick' merge commits!
> It's utterly confusing!

Please, please, please *work* with branch thickets for a while. And you
will see that it makes a *huge* difference!

For example, within a block of `pick` lines, it is relatively safe to
reorder. You see more or less what changes interact with one another.

Be *very* careful when reordering `merge` lines, in particular when you
have a real-world branch thicket, not just a toy example.

Do *not* use `pick` for merges. Not. Ever.

By the way, let this here discussion serve as Yet Another Example why it
is important to not only consider new features theoretically. Only in
practice do you find out where your theory sounded too good to be actually
true.

> Thinking about it I've got an idea that what we actually need is
> --no-flatten flag that, when used alone, will just tell "git rebase" to
> stop flattening history, and which will be implicitly imposed by
> --recreate-merges (and --preserve-merges).

... and this flag would only make sense if every `git rebase` involved a
todo list.

Which it does not.

Ciao,
Johannes

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Tue, 13 Mar 2018, Igor Djordjevic wrote:

> On 12/03/2018 11:20, Johannes Schindelin wrote:
> > 
> > > > [...] and cannot introduce ambiguities when rebasing the
> > > > changes introduced by M (i.e. the "amendmendts" we talked about).
> > >
> > > Hmm, not following here, which ambiguities are we talking about?
> > 
> > U1' vs U2' of course. Those are two things that can be different, even if
> > they ideally would have identical trees.
> > 
> > Phillip's strategy does not leave that room for ambiguity.
> 
> Ehm, in Sergey`s approach, this is not an issue, but a feature :)

Well, in my use cases, this would not be a good feature. It would be
highly annoying, confusing, and cost me tons of time.

> If U1' != U2', it just means a more complex rebase happened, but it 
> doesn`t compromise the result (rebased merge) in any way.

No, it just means that your strategy failed to give a consistent answer to
the question "what would the rebased merge commit's tree look like".

> On the other hand, if U1' == U2', we can be pretty sure that merge
> rebasing went as clean as possible.

With the backsplanation I gave for Phillip's strategy, I can be as sure
that rebasing went as clean as possible if it does not produce merge
conflicts: it reconciles the changes introduced by 1) rebasing the merge
tips with the changes introduced by 2) the original merge commit relative
to its parents.

And even if it produces merge conflicts, I know at least that those are
conflicts between those two sets of changes.

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Buga,

On Fri, 16 Mar 2018, Igor Djordjevic wrote:

> [...]
>
> Yes, having more steps would mean more power/options to the user, but
> more complexity to explain to and guide him through as well, not really
> sure where the line should be drawn - for the first time, at least.

If you want to avoid having a huge discussion with me about bias, male
privilege and how unaware most men are of it, and how it excludes half the
potential usership/talented developers, and how representation matters --
and believe me, you do want to avoid this discussion -- you will want to
avoid referring to the user as a "he".

It might be true in your case. But it is also true in your case that you
are Russian. Yet you write English here, probably to avoid excluding
people from the discussion. And you should demonstrate the same courtesy
to people who do not happen to identify with the same gender as you do.

In short: every time you think of a user or a developer as "he", take a
step back and reflect how excluded *you* would feel if someone forced you
to change that to "she". That is exactly how much you exclude non-males if
you think of them as "he". Just don't.

The remedy is easy: use the gender-neutral "they". Which is, by the way,
not a modern invention as of late (in contrast to the "he" you use):
https://en.wikipedia.org/wiki/Singular_they#Older_usage

Ciao,
Dscho

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Buga,
>
> On Tue, 13 Mar 2018, Igor Djordjevic wrote:
>
>> On 12/03/2018 13:56, Sergey Organov wrote:
>> > 
>> > > > I agree with both of you that `pick <merge-commit>` is inflexible
>> > > > (not to say just plain wrong), but I never thought about it like
>> > > > that.
>> > > >
>> > > > If we are to extract further mentioned explicit old:new merge
>> > > > parameter mapping to a separate discussion point, what we`re
>> > > > eventually left with is just replacing this:
>> > > >
>> > > > 	merge -R -C <original--merge-commit> <merge-head>
>> > > >
>> > > > ... with this:
>> > > >
>> > > > 	pick <original--merge-commit> <merge-head>
>> > >
>> > > I see where you are coming from.
>> > >
>> > > I also see where users will be coming from. Reading a todo list in
>> > > the editor is as much documentation as it is a "program to execute".
>> > > And I am afraid that reading a command without even mentioning the
>> > > term "merge" once is pretty misleading in this setting.
>> > >
>> > > And even from the theoretical point of view: cherry-picking
>> > > non-merge commits is *so much different* from "rebasing merge
>> > > commits" as discussed here, so much so that using the same command
>> > > would be even more misleading.
>> > 
>> > This last statement is plain wrong when applied to the method in the
>> > [RFC] you are replying to.
>
> That is only because the RFC seems to go out of its way to break down a
> single merge commit into as many commits as there are merge commit
> parents.

Complex entity is being split for ease of reasoning. People tend to use
this often.

> This is a pretty convoluted way to think about it: if you have three
> parent commits, for example, that way of thinking would introduce three
> intermediate commits, one with the changes of parent 2 & 3 combined, one
> with the changes of parent 1 & 3 combined, and one with the changes of
> parent 1 & 2 combined.

No.

> To rebase those commits, you essentially have to rebase *every parent's
> changes twice*.

No.

> It gets worse with merge commits that have 4 parents. In that case, you
> have to rebase every parent's changes *three times*.

Sorry, the [RFC] has nothing of the above. Once again, it's still just
as simple is: rebase every side of the merge then merge the results
using the original merge commit as a merge base.

And if you can't or don't want to grok the explanation in the RFC, just
forget the explanation, no problem.

> And so on.
>
>> > Using the method in [RFC], "cherry-pick non-merge" is nothing more or
>> > less than reduced version of generic "cherry-pick merge", exactly as
>> > it should be.
>
> I really get the impression that you reject Phillip's proposal on the
> ground of not being yours. In other words, the purpose of this here
> argument is to praise one proposal because of its heritage, rather than
> trying to come up with the best solution.

No. As the discussion evolved, I inclined to conclusion that modified
Phillip's algorithm is actually better suited for the implementation
[1].

> On that basis, I will go with the proposal that is clearly the simplest
> and does the job and gets away with avoiding unnecessary work.

These algorithms are actually the same one, as has already been shown
elsewhere in the discussion. Asymmetric incremental nature of the
Phillip's one is apparently better suited for naturally asymmetrical way
Git already handles merging. FYI, here is the latest proposal that came
out of discussion [1]:

git-rebase-first-parent --onto A' M
tree_U1'=$(git write-tree)
git merge-recursive B -- $tree_U1' B'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
[ $conflicted_last_merge = "yes" ] ||
  trees-match $tree_U1' $tree || 
  stop-for-user-amendment

where 'git-rebase-first-parent' denotes whatever machinery is currently
being used to rebase simple non-merge commit.

>
>> > Or, in other words, "cherry-pick merge" is generalization of
>> > "cherry-pick non-merge" to multiple parents.
>> 
>> I think Sergey does have a point here, his approach showing it.
>
> His approach is showing that he wants to shoehorn the "rebase a merge
> commit" idea into a form where you can cherry-pick *something*.
>
> It does not have to make sense. And to me, it really does not.

Except that Phillip's one does exactly this as well, only in incremental
manner, as shown in [1].

>
>> Phillip`s simplification might be further from it, though, but we`re 
>> talking implementation again - important mental model should just be 
>> "rebasing a commit" (merge or non-merge), how we`re doing it is 
>> irrelevant for the user, the point (goal) is the same.
>
> Except that Phillip's simplification is not a simplification. It comes
> from a different point of view: trying to reconcile the diverging
> changes.

They are essentially the same as one easily converts to another and back
[1]. They will only bring different user experience in case of
conflicts.

> Phillip's is a true generalization of the "rebase vs merge" story: it is
> no longer about merging, or about rebasing, but about reconciling
> divergent commit histories, with whatever tool is appropriate.

Whatever. They are essentially the same thing. The only difference is
incremental vs parallel [1].

References:

[1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com/

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Mon, 12 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >
>> > On Wed, 7 Mar 2018, Sergey Organov wrote:
>> >
>> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >> 
>> >> > How can your approach -- which relies *very much* on having the
>> >> > original parent commits -- not *require* that consistency check?
>> >> 
>> >> I don't understand what you mean, sorry. Could you please point me to
>> >> the *require* you talk about in the original proposal?
>> >
>> > Imagine a todo list that contains this line
>> >
>> > 	merge -C abcdef 123456
>> >
>> > and now the user edits it (this is an interactive rebase, after all),
>> > adding another merge head:
>> >
>> > 	merge -C abcdef 987654 123456
>> >
>> > Now your strategy would have a serious problem: to find the original
>> > version of 987654. If there was one.
>> 
>> We are talking about different checks then. My method has a built-in
>> check that Pillip's one doesn't.
>
> Since you did not bother to elaborate, I have to assume that your
> "built-in check" is that thing where intermediate merges can give you
> conflicts?
>
> If so, there is a possibility in Phillip's method for such conflicts, too:
> we have to perform as many 3-way merges as there are parent commits.
>
> It does make me uncomfortable to have to speculate what you meant,
> though.

It doesn't matter anymore as this check could easily be added to
Phillip's algorithm as well, see [1].

[1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Dear Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Mon, 12 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > [...]
>> >
>> > Where "easy" meant that I had to spend 1h still to figure out why
>> > using the unrebased merge parents as merge bases.
>> 
>> That's because you try to figure out something that is not there in the
>> [RFC v2]. I suggest to forget everything you've already imagined and
>> just read the [RFC v2] proposal afresh. It should take about 10 minutes
>> or less to get it. Really.
>> 
>> > The same amount of time did not allow me to wrap my head around
>> > Sergey's verbose explanations.
>> 
>> Honestly, I don't believe it, sorry, but I'm willing to explain anything
>> you wish to be explained in _[RFC v2]_.
>
> No, really. If you cannot bring yourself to believe my words, then I hate
> to break it to you: I am not lying.
>
> As to "I'm willing to explain anything you wish to be explained in RFC
> v2": I was asking, and asking, and asking again, for a simple summary of
> the idea behind your proposal. Nothing. That was the answer.

No. The answer rather was this simple explanation that I gave you
multiple times already "rebase each side of the merge, then merge the
results back using original merge commit as the merge base". Yet you say
there was none. I'm confused.

Well, as it seems you grok Phillip's notation just fine, here is RFC
algorithm in this notation [1]:

git checkout --detach A'
git merge-recursive A -- A' M
tree_U1'=$(git write-tree)
git checkout --detach B'
git merge-recursive B -- B' M
tree_U2'=$(git write-tree)
git merge-recursive M -- $tree_U1' $tree_U2'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')

> I had to figure it out myself: the idea is to *create* fake commits,
> non-merge ones, for every single merge commit parent. Those fake commits
> combine the changes of *all* merge commit parents *but one*. And then
> those commits are rebased, individually, with tons of opportunities for
> merge conflicts. Repeated ones. And then that result is merged.

Wrong. See above.

Anyway, it doesn't matter anymore, see [1].

References:

[1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com

-- Sergey

Re: [RFC v2] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Mon, 12 Mar 2018, Sergey Organov wrote:
>
>> [...]
>> 
>> Yet another consequence is that my approach will likely result in better
>> code reuse.
>
> This is a purely academic speculation. At least until somebody implements
> Phillip's method. Oh wait, I already started to implement it, and it was
> not exactly hard to implement:
>
> https://github.com/dscho/git/commit/26d2858800a4e0d3cc6313ddb54dd4d2ce516f31

Nice! Please see [1] for some recent relevant discussion.

[1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com/

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Buga,
>
> On Tue, 13 Mar 2018, Igor Djordjevic wrote:
>
>> On 12/03/2018 11:46, Johannes Schindelin wrote:
>> > 
>> > > Sometimes one just needs to read the manual, and I don`t really
>> > > think this is a ton complicated, but just something we didn`t really
>> > > have before (real merge rebasing), so it requires a moment to grasp
>> > > the concept.
>> > 
>> > If that were the case, we would not keep getting bug reports about
>> > --preserve-merges failing to reorder patches.
>> 
>> Not sure where that is heading to, but what I`m arguing about is that 
>> introducing new commands and concepts (`merge`, and with `-R`) just 
>> makes the situation even worse (more stuff to grasp).
>
> The problem with re-using `pick` is that its concept does not apply to
> merges. The cherry-pick of a non-merge commit is well-defined: the current
> HEAD is implicitly chosen as the cherry-picked commit's (single) parent
> commit. There is no ambiguity here.
>
> But for merge commits, we need to specify the parent commits (apart from
> the first one) *explicitly*. There was no need for that in the `pick`
> command, nor in the concept of a cherry-pick.
>
>> Reusing existing concepts where possible doesn`t have this problem.
>
> Existing concepts are great. As long as they fit the requirements of the
> new scenarios. In this case, `pick` does *not* fit the requirement of
> "rebase a merge commit".

It does, provided you use suitable syntax.

> If you really want to force the `pick` concept onto the use case where
> you need to "reapply" merges, then the closest you get really is
> Sergey's idea, which I came to reject when considering its practical
> implications.

Which one, and what are the implications that are bad, I wonder?

> Even so, you would have to make the `pick` command more complicated to
> support merge commits. And whatever you would do to extend the `pick`
> command would *not make any sense* to the current use case of the `pick`
> command.

It would rather make a lot of sense. Please don't use 'merge' to pick
commits, merge ones or not!

> The real problem, of course, is that a non-merge commit, when viewed from
> the perspective of the changes it introduced, is a very different beast
> than a merge commit: it does not need to reconcile changes, ever, because
> there is really only one "patch" to one revision. That is very different
> from a merge commit, whose changes can even disagree with one another (and
> in fact be resolved with changes disagreeing *yet again*)!

You'd still 'pick' it though, not 'merge'. You don't merge "merge
commit", it makes no sense. It only makes perfect sense when you get rid
of original "merge commit" and re-merge from scratch, as you were doing
till now.

>> > > Saying in favor of `--rebase-merges`, you mean as a separate option,
>> > > alongside `--recreate-merges` (once that series lands)?
>> > 
>> > No. I am against yet another option. The only reason I pollute the
>> > option name space further with --recreate-merges is that it would be
>> > confusing to users if the new mode was called --preserve-merges=v2
>> > (but work *totally differently*).
>> 
>> I see. So I take you`re thinking about renaming `--recreate-merges` to
>> `--rebase-merges` instead?
>
> Thinking about it. Nothing will happen before v2.17.0 on that front,
> though, because -- unlike you gentle people -- I have to focus on
> stabilizing Git's code base now.
>
>> That would seem sensible, too, I think, being the default usage mode in
>> the first place. Being able to actually (re)create merges, too, once
>> user goes interactive, would be "just" an additional (nice and powerful)
>> feature on top of it.
>
> The implementation detail is, of course, that I will introduce this with
> the technically-simpler strategy: always recreating merge commits with the
> recursive strategy. A follow-up patch series will add support for rebasing
> merge commits, and then use it by default.

Switching to use it by default would be backward incompatible again? Yet
another option to obsolete? Sigh. 

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Johannes Schindelin]

Hi Sergey,

On Tue, 27 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Mon, 12 Mar 2018, Sergey Organov wrote:
> >
> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> >
> >> > On Wed, 7 Mar 2018, Sergey Organov wrote:
> >> >
> >> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >> >> 
> >> >> > How can your approach -- which relies *very much* on having the
> >> >> > original parent commits -- not *require* that consistency check?
> >> >> 
> >> >> I don't understand what you mean, sorry. Could you please point me
> >> >> to the *require* you talk about in the original proposal?
> >> >
> >> > Imagine a todo list that contains this line
> >> >
> >> > 	merge -C abcdef 123456
> >> >
> >> > and now the user edits it (this is an interactive rebase, after
> >> > all), adding another merge head:
> >> >
> >> > 	merge -C abcdef 987654 123456
> >> >
> >> > Now your strategy would have a serious problem: to find the
> >> > original version of 987654. If there was one.
> >> 
> >> We are talking about different checks then. My method has a built-in
> >> check that Pillip's one doesn't.
> >
> > Since you did not bother to elaborate, I have to assume that your
> > "built-in check" is that thing where intermediate merges can give you
> > conflicts?
> >
> > If so, there is a possibility in Phillip's method for such conflicts,
> > too: we have to perform as many 3-way merges as there are parent
> > commits.
> >
> > It does make me uncomfortable to have to speculate what you meant,
> > though.
> 
> It doesn't matter anymore as this check could easily be added to
> Phillip's algorithm as well, see [1].
> 
> [1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com

Ah, and there I was, thinking that finally you would answer my questions
directly, instead you keep directing me elsewhere ("read that! Somewhere
in there you will find the answer you are looking for").

My time is a bit too valuable, and I will not continue a discussion where
my questions are constantly deflected that way.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Sergey,

On Tue, 27 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> >
> > On Tue, 13 Mar 2018, Igor Djordjevic wrote:
> >
> >> On 12/03/2018 13:56, Sergey Organov wrote:
> >> > 
> >> > > > I agree with both of you that `pick <merge-commit>` is inflexible
> >> > > > (not to say just plain wrong), but I never thought about it like
> >> > > > that.
> >> > > >
> >> > > > If we are to extract further mentioned explicit old:new merge
> >> > > > parameter mapping to a separate discussion point, what we`re
> >> > > > eventually left with is just replacing this:
> >> > > >
> >> > > > 	merge -R -C <original--merge-commit> <merge-head>
> >> > > >
> >> > > > ... with this:
> >> > > >
> >> > > > 	pick <original--merge-commit> <merge-head>
> >> > >
> >> > > I see where you are coming from.
> >> > >
> >> > > I also see where users will be coming from. Reading a todo list in
> >> > > the editor is as much documentation as it is a "program to execute".
> >> > > And I am afraid that reading a command without even mentioning the
> >> > > term "merge" once is pretty misleading in this setting.
> >> > >
> >> > > And even from the theoretical point of view: cherry-picking
> >> > > non-merge commits is *so much different* from "rebasing merge
> >> > > commits" as discussed here, so much so that using the same command
> >> > > would be even more misleading.
> >> > 
> >> > This last statement is plain wrong when applied to the method in the
> >> > [RFC] you are replying to.
> >
> > That is only because the RFC seems to go out of its way to break down a
> > single merge commit into as many commits as there are merge commit
> > parents.
> 
> Complex entity is being split for ease of reasoning. People tend to use
> this often.

Sure. Divide and conquer. Not duplicate and complicate, though.

> > This is a pretty convoluted way to think about it: if you have three
> > parent commits, for example, that way of thinking would introduce three
> > intermediate commits, one with the changes of parent 2 & 3 combined, one
> > with the changes of parent 1 & 3 combined, and one with the changes of
> > parent 1 & 2 combined.
> 
> No.

Sorry. This is unacceptable. If you disagree, sure, you are free to do
that. If you want to contribute to a fruitful discussion, just saying "No"
without explaining why you *think* that my statement is wrong is just...
unconstructive.

> > To rebase those commits, you essentially have to rebase *every
> > parent's changes twice*.
> 
> No.

Same here.

> > It gets worse with merge commits that have 4 parents. In that case, you
> > have to rebase every parent's changes *three times*.
> 
> Sorry, the [RFC] has nothing of the above. Once again, it's still just
> as simple is: rebase every side of the merge then merge the results
> using the original merge commit as a merge base.
> 
> And if you can't or don't want to grok the explanation in the RFC, just
> forget the explanation, no problem.

Your RFC talks about U1 and U2, for the two merge parents.

Obviously this strategy can be generalized to n parents. I thought you had
thought of that and simply did not bother to talk about it.

Sorry, my mistake. I should not assume so much.

> > And so on.
> >
> >> > Using the method in [RFC], "cherry-pick non-merge" is nothing more or
> >> > less than reduced version of generic "cherry-pick merge", exactly as
> >> > it should be.
> >
> > I really get the impression that you reject Phillip's proposal on the
> > ground of not being yours. In other words, the purpose of this here
> > argument is to praise one proposal because of its heritage, rather than
> > trying to come up with the best solution.
> 
> No. As the discussion evolved, I inclined to conclusion that modified
> Phillip's algorithm is actually better suited for the implementation
> [1].

Again a link.

If that's what you are looking for, I will throw a hundred links your way
and see how constructive a discussion you find that.

> > On that basis, I will go with the proposal that is clearly the simplest
> > and does the job and gets away with avoiding unnecessary work.
> 
> These algorithms are actually the same one, as has already been shown
> elsewhere in the discussion.

I disproved that already. My example showed that instead of reconciling
the diverging changes starting from the original merge parents, RFC v2
tries to rebase those parents first, and then use the original merge
commit as base of "diverging changes" that never started from that
original merge commit.

Essentially, where Phillip's strategy imitates a cherry-pick's 3-way
merge, your strategy tries to rebase the merge tips independently from the
user (who already rebased them, thank you very much), and then runs a
*revert*: while a cherry-pick uses the picked commit's parent as merge
base, a revert uses the to-be-reverted commit itself as merge base.

In short: Phillip's strategy is only equivalent to yours if you ignore the
fact that you perform unnecessary work only to undo it in the end.

> Asymmetric incremental nature of the Phillip's one is apparently better
> suited for naturally asymmetrical way Git already handles merging. FYI,
> here is the latest proposal that came out of discussion [1]:

And another link.

> git-rebase-first-parent --onto A' M
> tree_U1'=$(git write-tree)
> git merge-recursive B -- $tree_U1' B'
> tree=$(git write-tree)
> M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
> [ $conflicted_last_merge = "yes" ] ||
>   trees-match $tree_U1' $tree || 
>   stop-for-user-amendment

And a bunch of commands from which the reader is expected to deduce the
idea.

> >> > Or, in other words, "cherry-pick merge" is generalization of
> >> > "cherry-pick non-merge" to multiple parents.
> >> 
> >> I think Sergey does have a point here, his approach showing it.
> >
> > His approach is showing that he wants to shoehorn the "rebase a merge
> > commit" idea into a form where you can cherry-pick *something*.
> >
> > It does not have to make sense. And to me, it really does not.
> 
> Except that Phillip's one does exactly this as well, only in incremental
> manner, as shown in [1].

And yet another link.

> >> Phillip`s simplification might be further from it, though, but we`re 
> >> talking implementation again - important mental model should just be 
> >> "rebasing a commit" (merge or non-merge), how we`re doing it is 
> >> irrelevant for the user, the point (goal) is the same.
> >
> > Except that Phillip's simplification is not a simplification. It comes
> > from a different point of view: trying to reconcile the diverging
> > changes.
> 
> They are essentially the same as one easily converts to another and back
> [1].

Repeating this does not make it more true.

With your method, this branch structure:

- A - B
    \   \
      C - D

would be rebased by first cherry-picking B, then C, then doing it *again*
because you need to construct U1 (which is kind of C) and U2 (which is
kind of B) and rebase those, and if the user resolved merge conflicts
while rebasing B, those merge conflicts will have to be resolved *again*
when rebasing U2, and if the user dropped part of B, U2 will still have to
rebase them, and the final merge with the original D as merge base will
have to undo those changes.

Your strategy involves *a lot* more work, and *a lot* more opportunities
for merge conflicts, and it also allows for giving incongruent answers
to the question "what should the tree of the rebased merge commit look
like?".

> They will only bring different user experience in case of conflicts.

Oh yes, they do. The amount, to begin with.

> > Phillip's is a true generalization of the "rebase vs merge" story: it is
> > no longer about merging, or about rebasing, but about reconciling
> > divergent commit histories, with whatever tool is appropriate.
> 
> Whatever. They are essentially the same thing. The only difference is
> incremental vs parallel [1].

You know, if you promise to answer whatever questions I have, just simply
stop throwing around links. Answer my questions. To the point. Not
deflecting. This is getting ridiculous.

> [1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com/

I will allow myself the joke and answer the concerns you had in this mail
thusly:

https://public-inbox.org/git/nycvar.QRO.7.76.6.1803261405170.77@ZVAVAG-6OXH6DA.rhebcr.pbec.zvpebfbsg.pbz/

So that you cannot miss it: there is a really good, real-world example
showing why Phillip's strategy is so much more practical than yours,
proving that they are not equivalent (except in a very narrow, purely
theoretical sense that ignores the possibility of merge conflicts). Look
for "concrete example" in that mail.

Another most important thing from that mail is really, really important.
So I will stress it instead of expecting you to pick up on it, by
repeating it here:

	BTW this is the level of detail I would have wished your answers
	to my repeated questions for clarification of your RFC v2 to be.
	And that is what I expect in response to my valid questions in the
	future.

Instead, you chose to fling a link in my direction again.

And yes, I read your mail, and no, it does not clarify anything, or even
addresses my objections.

If you contest my understanding of your strategy (where I say that U1 is
essentially the changes of the *other* merged branch), you will have to do
a much better job at explaining yourself. "No" is definitely not adequate.

And don't promise to answer my questions if you plan only on throwing a
link at me. That is no good.

So to repeat my point (that you contested without any argument, by a rude
and totally unmeritedly terse "No"):

> > To rebase those commits, you essentially have to rebase *every
> > parent's changes twice*.
> 
> No.

In the parlance of your RFC v2, where you start with this history (which I
translated into the left-to-right notation that is used in pretty much all
of Git's own documentation about interactive rebases, which you apparently
either did not read, or chose *not* to imitate, creating yet another
unnecessary diversion):

- B1
     \
- B2 - M

You now insert U1 and U2 with trees identical to M:

- B1 - U1
          \
- B2 - U2 - M

So U1 is essentially B2 cherry-picked on top of B1, and U2 is essentially
B1 cherry-picked on top of B2.

These U1/U2 commits are now to be cherry-picked on top of the rebased B1'
and B2'. I spare you more diagrams, you get the idea.

Now, the changes in U1/U2 *are* the changes of the merge parents, that's
how they were constructed.

Since they repeat what B1 and B2 are about, and since B1'/B2' means they
are rebased, and since U1'/U2' are *also* rebased, but independently...

	...  you essentially have to rebase *every parent's changes twice*.

The answer "No" to this is... astonishing.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Sergey,

On Tue, 27 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > On Tue, 13 Mar 2018, Igor Djordjevic wrote:
> >
> >> On 12/03/2018 11:46, Johannes Schindelin wrote:
> >> > 
> >> > > Sometimes one just needs to read the manual, and I don`t really
> >> > > think this is a ton complicated, but just something we didn`t
> >> > > really have before (real merge rebasing), so it requires a moment
> >> > > to grasp the concept.
> >> > 
> >> > If that were the case, we would not keep getting bug reports about
> >> > --preserve-merges failing to reorder patches.
> >> 
> >> Not sure where that is heading to, but what I`m arguing about is that
> >> introducing new commands and concepts (`merge`, and with `-R`) just
> >> makes the situation even worse (more stuff to grasp).
> >
> > The problem with re-using `pick` is that its concept does not apply to
> > merges. The cherry-pick of a non-merge commit is well-defined: the
> > current HEAD is implicitly chosen as the cherry-picked commit's
> > (single) parent commit. There is no ambiguity here.
> >
> > But for merge commits, we need to specify the parent commits (apart
> > from the first one) *explicitly*. There was no need for that in the
> > `pick` command, nor in the concept of a cherry-pick.
> >
> >> Reusing existing concepts where possible doesn`t have this problem.
> >
> > Existing concepts are great. As long as they fit the requirements of
> > the new scenarios. In this case, `pick` does *not* fit the requirement
> > of "rebase a merge commit".
> 
> It does, provided you use suitable syntax.

You know what `pick` would also do, provided you use suitable syntax? Pick
your nose.

Don't blame me for this ridiculous turn the discussion took.

Of course, using the suitable syntax you can do anything. Unless there is
*already* a syntax and you cannot break it for backwards-compatibility
reasons, as is the case here.

But I'll stop here. Even my account how there are conceptual differences
between the changes in merge vs non-merge commits (the non-merge commit
*introduces* changes, the merge commit *reconciles existing* changes)
seems to fly by without convincing you.

I use rebase every day. I use the Git garden shears every week. If you do
not trust my experience with these things, nothing will convince you. You
are just stuck with your pre-existing opinion.

> > If you really want to force the `pick` concept onto the use case where
> > you need to "reapply" merges, then the closest you get really is
> > Sergey's idea, which I came to reject when considering its practical
> > implications.
> 
> Which one, and what are the implications that are bad, I wonder?

The strategy described in RFC v2, which does too much work, forces the
user to potentially address the same merge conflicts multiple times, and
worst of all: risks merge conflicts with changes the user *already*
dropped.

> > Even so, you would have to make the `pick` command more complicated to
> > support merge commits. And whatever you would do to extend the `pick`
> > command would *not make any sense* to the current use case of the `pick`
> > command.
> 
> It would rather make a lot of sense. Please don't use 'merge' to pick
> commits, merge ones or not!

It would rather make a lot of sense. If you completely ignored everything
I said about preserve-merges. If you ignored what I said about problems
moving regular `pick` lines across merge commits. If you ignored all the
experience I have with Git garden shears and that I tried really patiently
for an impatient man to impart on you.

> > The real problem, of course, is that a non-merge commit, when viewed
> > from the perspective of the changes it introduced, is a very different
> > beast than a merge commit: it does not need to reconcile changes,
> > ever, because there is really only one "patch" to one revision. That
> > is very different from a merge commit, whose changes can even disagree
> > with one another (and in fact be resolved with changes disagreeing
> > *yet again*)!
> 
> You'd still 'pick' it though, not 'merge'. You don't merge "merge
> commit", it makes no sense. It only makes perfect sense when you get rid
> of original "merge commit" and re-merge from scratch, as you were doing
> till now.

No, you merge "merge head". And you use "merge commit"'s commit message.
*That* makes sense.

Picking a merge commit? Not so. What do you merge? The original merge
commit's second parent? Or a rebased version thereof? What if that commit
has been `pick`ed *twice*?

No, you can repeat it all you want, it still does not make sense. Now that
I think of the possiblity of picking the original parents multiple times,
it does not even make theoretical sense.

> > The implementation detail is, of course, that I will introduce this with
> > the technically-simpler strategy: always recreating merge commits with the
> > recursive strategy. A follow-up patch series will add support for rebasing
> > merge commits, and then use it by default.
> 
> Switching to use it by default would be backward incompatible again? Yet
> another option to obsolete? Sigh. 

Oh wow.

Backwards compatibility of a feature that existed only as a topic branch
in `next` before being worked on more? Any other splendid ideas?

And what's that about another option to obsolete? Who said that I would
obsolete any newly-introduced option?

I would introduce either --recreate-merges or --rebase-merges, and then
just stick with it.

I guess it is my turn to sigh.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution (Road Clear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>
> On Tue, 27 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > On Mon, 12 Mar 2018, Sergey Organov wrote:
>> >
>> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >> >
>> >> > On Wed, 7 Mar 2018, Sergey Organov wrote:
>> >> >
>> >> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >> >> 
>> >> >> > How can your approach -- which relies *very much* on having the
>> >> >> > original parent commits -- not *require* that consistency check?
>> >> >> 
>> >> >> I don't understand what you mean, sorry. Could you please point me
>> >> >> to the *require* you talk about in the original proposal?
>> >> >
>> >> > Imagine a todo list that contains this line
>> >> >
>> >> > 	merge -C abcdef 123456
>> >> >
>> >> > and now the user edits it (this is an interactive rebase, after
>> >> > all), adding another merge head:
>> >> >
>> >> > 	merge -C abcdef 987654 123456
>> >> >
>> >> > Now your strategy would have a serious problem: to find the
>> >> > original version of 987654. If there was one.
>> >> 
>> >> We are talking about different checks then. My method has a built-in
>> >> check that Pillip's one doesn't.
>> >
>> > Since you did not bother to elaborate, I have to assume that your
>> > "built-in check" is that thing where intermediate merges can give you
>> > conflicts?
>> >
>> > If so, there is a possibility in Phillip's method for such conflicts,
>> > too: we have to perform as many 3-way merges as there are parent
>> > commits.
>> >
>> > It does make me uncomfortable to have to speculate what you meant,
>> > though.
>> 
>> It doesn't matter anymore as this check could easily be added to
>> Phillip's algorithm as well, see [1].
>> 
>> [1] https://public-inbox.org/git/87efkn6s1h.fsf@javad.com
>
> Ah, and there I was, thinking that finally you would answer my questions
> directly, instead you keep directing me elsewhere ("read that! Somewhere
> in there you will find the answer you are looking for").

Except I've copy-pasted it for /you/ from that reference in another
answer to /you/, and /you/ denied it there as being unexplained. As it
actually happens to be discussed and explained in the referenced
material, should I rather copy-paste the entire reference to fulfill
your requirements?

Here I repeat, directly again, that essential quote from that reference,
in case you forgot it:

<QUOTE>
git-rebase-first-parent --onto A' M
tree_U1'=$(git write-tree)
git merge-recursive B -- $tree_U1' B'
tree=$(git write-tree)
M'=$(git log --pretty=%B -1 M | git commit-tree -pA' -pB')
[ $conflicted_last_merge = "yes" ] ||
  trees-match $tree_U1' $tree || 
  stop-for-user-amendment
  
where 'git-rebase-first-parent' denotes whatever machinery is currently
being used to rebase simple non-merge commit.
</QUOTE>

> My time is a bit too valuable, and I will not continue a discussion where
> my questions are constantly deflected that way.

No deflection on my side was ever intended. The referenced discussion
actually has explanations. Maybe one whole page of reading, and it is to
be read in context, and then a few follow-ups in that discussion could
also be of interest, provided you are interested. I'm sorry should you
have no time for that.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,

[...]

> But I'll stop here. Even my account how there are conceptual differences
> between the changes in merge vs non-merge commits (the non-merge commit
> *introduces* changes, the merge commit *reconciles existing* changes)
> seems to fly by without convincing you.

Good for you, but Git should keep caring about content, it should care
not about meaning. Please leave it to the user to assign meaning to
their content.

If you rather want a SCM that focuses on meaning, I'd suggest to look at
Bzr and see how it goes.

> I use rebase every day. I use the Git garden shears every week. If you do
> not trust my experience with these things, nothing will convince you. 

Unfortunately you have exactly zero experience with rebasing merges as
you've never actually rebased them till now, and it's rebasing merges
that matters in this particular discussion.

> You are just stuck with your pre-existing opinion.

I'm afraid that it's rather your huge experience with re-creating merges
that makes you stuck to your pre-existing opinion and carefully shields
you from experiencing actual paradigm shift.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi Sergey,
>

[...]

>> >> Reusing existing concepts where possible doesn`t have this problem.
>> >
>> > Existing concepts are great. As long as they fit the requirements of
>> > the new scenarios. In this case, `pick` does *not* fit the requirement
>> > of "rebase a merge commit".
>> 
>> It does, provided you use suitable syntax.
>
> You know what `pick` would also do, provided you use suitable syntax? Pick
> your nose.
>
> Don't blame me for this ridiculous turn the discussion took.
>
> Of course, using the suitable syntax you can do anything. Unless there is
> *already* a syntax and you cannot break it for backwards-compatibility
> reasons, as is the case here.

Backward compatibility to what? To a broken '--preserve-merges'? I had a
feel you've invented '--recreate-merges' exactly to break that
compatibility. No?

Or is it "Backwards compatibility of a feature that existed only as a
topic branch in `next` before being worked on more?", as you say
yourself below?

[...]

>> > The implementation detail is, of course, that I will introduce this with
>> > the technically-simpler strategy: always recreating merge commits with the
>> > recursive strategy. A follow-up patch series will add support for rebasing
>> > merge commits, and then use it by default.
>> 
>> Switching to use it by default would be backward incompatible again? Yet
>> another option to obsolete? Sigh. 
>
> Oh wow.
>
> Backwards compatibility of a feature that existed only as a topic branch
> in `next` before being worked on more? Any other splendid ideas?

Either you care about compatibility or not. You can't have it both ways,
sorry.

And "technically-simpler strategy: always recreating merge commits with
the recursive strategy" vs. "rebasing merge commits" is not just a minor
strategy change, it's entire paradigm shift in handling merge commits
while rebasing. I'm afraid you will still come up with a wrong design
unless you finally accept this fact.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Jacob Keller <jacob.keller@gmail.com> writes:

> On Tue, Mar 27, 2018 at 10:57 PM, Sergey Organov <sorganov@gmail.com> wrote:
>>
>> Hi Johannes,
>>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> > Hi Sergey,
>> >
>>
>> [...]
>>
>> >> >> Reusing existing concepts where possible doesn`t have this problem.
>> >> >
>> >> > Existing concepts are great. As long as they fit the requirements of
>> >> > the new scenarios. In this case, `pick` does *not* fit the
> requirement
>> >> > of "rebase a merge commit".
>> >>
>> >> It does, provided you use suitable syntax.
>> >
>> > You know what `pick` would also do, provided you use suitable syntax?
> Pick
>> > your nose.
>> >
>> > Don't blame me for this ridiculous turn the discussion took.
>> >
>> > Of course, using the suitable syntax you can do anything. Unless there
> is
>> > *already* a syntax and you cannot break it for backwards-compatibility
>> > reasons, as is the case here.
>>
>> Backward compatibility to what? To a broken '--preserve-merges'? I had a
>> feel you've invented '--recreate-merges' exactly to break that
>> compatibility. No?
>>
>> Or is it "Backwards compatibility of a feature that existed only as a
>> topic branch in `next` before being worked on more?", as you say
>> yourself below?
>>
>
> I'm pretty sure he meant that changing the meaning and behavior of "pick"
> is incompatible, as people use scripts which check the edit lists, and
> these scripts would expect pick to behave in a certain way.

Are we still speaking about that new --recreate-merges feature? You
already care for compatibility for it? You expect there are already
scripts that use it?

Once again, it seems like you care and don't care about backward
compatibility at the same time, here is your phrase below:

"He absolutely cares about compatibility, but in this case, the feature
has not yet been merged into an official release."

Are we still speaking about that new --recreate-merges feature?

Do you guys care for compatibility for this particular --recreate-merges
feature or not? I'm lost. "Yes" or "No" answer, if you please!

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Jacob Keller <jacob.keller@gmail.com> writes:

> On Tue, Mar 27, 2018 at 10:57 PM, Sergey Organov <sorganov@gmail.com> wrote:
>>
>> Hi Johannes,
>>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

[...]

> I'm pretty sure the fact has already been accepted, as he did indeed
> implement and develop a strategy for rebasing the merges (Phillip's
> strategy). He hasn't chosen to re-write all the code such that it was
> "always" this method, but rather kept it as an incremental patch on top as
> it makes it easier to review the changes since we've already spent time
> looking at and reviewing the --recreate-merges patches.

That's perfectly OK with me, except that he apparently still can't
accept the fact that rebasing a non-merge is not fundamentally different
from rebasing a merge.

"Rebase non-merge" is just a special case of generic "rebase commit",
provided we do have generic method that is capable to rebase any commit,
and we do have it, Phillip's or not.

> Having watched from the sidelines, I've been unable to completely
> understand and parse the strategies completely, but I've also found
> Phillip's method to be easier to understand.

It doesn't matter at all for this particular discussion. Let's call the
method "rebase a commit", a black-box, that is capable to rebase any
commit. I don't care what implementation is inside. Rebasing a commit is
still rebasing a commit, and it should not be called "merge" in the todo
list.

> As someone who's read the discussion on the sidelines, it certainly
> does feel like there is some misunderstanding on both sides. Neither
> of you have been able to get the other to see what you clearly both
> believe strongly.

Calling "rebase" operation "merge" is wrong no matter what method is
used to rebase a commit. Isn't it obvious? It's currently called "pick"
in the todo and it seems natural to continue to use that name for
picking a commit, whatever number of parents it happens to have.

> Unfortunately I do not have any suggestion as to how to resolve the
> misunderstanding.

This sub-thread is not about method at all, so no resolution on that
matter is required here. This sub-thread is about todo format only.

> Sergey's method appears to me to be more complex, and I agree that the
> extra steps could cause more merge conflicts, at least in how it was
> originally conceptualized and implemented. It is possible that we are
> mis-understanding the terminology for U1 and U2? It sure seems like it
> introduces more changes for merge conflicts than the strategy proposed by
> Phillip. However, the latest editions also sound a lot closer to Phillip's
> strategy in general, so maybe I have mis-understood how it works and what
> is fundamentally different about the two strategies.

There is nothing fundamentally different between them and thus I don't
care in this discussion what exact method is being used.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Jacob Keller <jacob.keller@gmail.com> writes:

> On Wed, Mar 28, 2018 at 4:29 AM, Sergey Organov <sorganov@gmail.com> wrote:
>
>> Jacob Keller <jacob.keller@gmail.com> writes:
>>
>> > On Tue, Mar 27, 2018 at 10:57 PM, Sergey Organov <sorganov@gmail.com>
>> wrote:
>> >>
>> >> Hi Johannes,
>> >>
>> >> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> >> > Hi Sergey,
>> >> >
>> >>
>> >> [...]
>> >>
>> >> >> >> Reusing existing concepts where possible doesn`t have this
>> problem.
>> >> >> >
>> >> >> > Existing concepts are great. As long as they fit the requirements
>> of
>> >> >> > the new scenarios. In this case, `pick` does *not* fit the
>> > requirement
>> >> >> > of "rebase a merge commit".
>> >> >>
>> >> >> It does, provided you use suitable syntax.
>> >> >
>> >> > You know what `pick` would also do, provided you use suitable syntax?
>> > Pick
>> >> > your nose.
>> >> >
>> >> > Don't blame me for this ridiculous turn the discussion took.
>> >> >
>> >> > Of course, using the suitable syntax you can do anything. Unless there
>> > is
>> >> > *already* a syntax and you cannot break it for backwards-compatibility
>> >> > reasons, as is the case here.
>> >>
>> >> Backward compatibility to what? To a broken '--preserve-merges'? I had a
>> >> feel you've invented '--recreate-merges' exactly to break that
>> >> compatibility. No?
>> >>
>> >> Or is it "Backwards compatibility of a feature that existed only as a
>> >> topic branch in `next` before being worked on more?", as you say
>> >> yourself below?
>> >>
>> >
>> > I'm pretty sure he meant that changing the meaning and behavior of "pick"
>> > is incompatible, as people use scripts which check the edit lists, and
>> > these scripts would expect pick to behave in a certain way.
>>
>> Are we still speaking about that new --recreate-merges feature? You
>> already care for compatibility for it? You expect there are already
>> scripts that use it?
>>
>> Once again, it seems like you care and don't care about backward
>> compatibility at the same time, here is your phrase below:
>>
>> "He absolutely cares about compatibility, but in this case, the feature
>> has not yet been merged into an official release."
>>
>> Are we still speaking about that new --recreate-merges feature?
>>
>> Do you guys care for compatibility for this particular --recreate-merges
>> feature or not? I'm lost. "Yes" or "No" answer, if you please!
>>
>> -- Sergey
>>
>
> I care about the general compatibility of the rebase todo list regardless
> of which options you enabled on the command line to generate it.

It's a good thing in general, yes. However, I recall I was told by the
author that --recreate-merges was introduced exactly to break backward
compatibility of the todo list. If so, could we please agree to stop
using backward compatibility as an objection in the discussion of this
particular feature?

> Yes this has a bit of problem because *any* new todo command will
> break the todo list, but it's better to only add new commands rather
> than change semantics of existing ones.

I'm not against new commands in general. I'm against inventing new
entities without necessity, so, provided we did agree not to care about
compatibility, there should be some other necessity to invent yet
another command. I don't see such a necessity. Do you?

The main principle I stand for in this discussion though is that all the
commits should be treated equally as much as possible, new command or no
new command. Doing otherwise will lead to all kinds of troubles and
confusion, both in implementation and in user experience.

Overall, what I think is needed is extending the syntax of existing todo
commands to handle merge commits. This will give a provision to get it
right this time. Otherwise it will likely end up being yet another
subject of deprecation in the future.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi,

On Thu, 29 Mar 2018, Sergey Organov wrote:

> Jacob Keller <jacob.keller@gmail.com> writes:
> 
> > I care about the general compatibility of the rebase todo list
> > regardless of which options you enabled on the command line to
> > generate it.
> 
> It's a good thing in general, yes. However, I recall I was told by the
> author that --recreate-merges was introduced exactly to break backward
> compatibility of the todo list. If so, could we please agree to stop
> using backward compatibility as an objection in the discussion of this
> particular feature?

That is a serious misrepresentation of what I said.

If I had changed --preserve-merges to the new format, *that* would have
broken backwards-compatibility.

So the entire reason of introducing --recreate-merges was to *not have to
break backwards-compatibility*.

I definitely did not say the *exact opposite*.

Hopefully this clarifies your confusion,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> Hi,
>
> On Thu, 29 Mar 2018, Sergey Organov wrote:
>
>> Jacob Keller <jacob.keller@gmail.com> writes:
>> 
>> > I care about the general compatibility of the rebase todo list
>> > regardless of which options you enabled on the command line to
>> > generate it.
>> 
>> It's a good thing in general, yes. However, I recall I was told by the
>> author that --recreate-merges was introduced exactly to break backward
>> compatibility of the todo list. If so, could we please agree to stop
>> using backward compatibility as an objection in the discussion of this
>> particular feature?
>
> That is a serious misrepresentation of what I said.
>
> If I had changed --preserve-merges to the new format, *that* would have
> broken backwards-compatibility.
>
> So the entire reason of introducing --recreate-merges was to *not have to
> break backwards-compatibility*.
>
> I definitely did not say the *exact opposite*.

I'm sorry I committed ambiguity in my wording that allowed it to be
misinterpreted. I actually intended to say roughly the same thing you
are saying, as what matters for the discussion is that new todo list
format does not need to be (backward-)compatible to that of
--preserve-merges. 

> Hopefully this clarifies your confusion,

There was actually no confusion on my side, and I like your wording
better.

Except that you've managed to clarify your intentions without actually
addressing the primary concern:

Could we please agree to stop using backward compatibility as an
objection in the discussion of the  --recreate-merges feature?

Could we?

I understand you are still resistant to change 'pick' syntax, but it's
not because of backward-compatibility, right?

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Sergey,

On Fri, 30 Mar 2018, Sergey Organov wrote:

> Could we please agree to stop using backward compatibility as an
> objection in the discussion of the  --recreate-merges feature?

No.

The expectation of users as to what a `pick` is has not changed just
because you wish it would.

That is a matter of backwards-compatibility.

You see, if you are driving a car for a hundred years already, and then
switch to a different car, and it has a lever in the same place as your
previous car's windshield wiper, but in the new car it has a button that
activates the emergency driver seat ejection OMG *it has a seat ejection
like in the James Bond movies! Where can I get that car?* Sorry for
disgressing.

I am really concerned about that willingness to put an innocuous button,
so to speak, onto something users got really used to, over the course of a
decade or so, when that button should really be made red and blinking and
OMG where can I get that car?

So to reiterate, I am really interested in a practical solution that won't
cause nasty surprises. Meaning: `pick` != merge. That was a mistake in
preserve-merges, as I have only mentioned like a hundred times, and we
won't repeat it.

Now back to that important question: where can I get such a James Bond
car? Ideally also with Turbo Boost. Oh wait, that was somebody else's car.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Johannes Schindelin]

Hi Sergey,

On Wed, 28 Mar 2018, Sergey Organov wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
> 
> > I use rebase every day. I use the Git garden shears every week. If you
> > do not trust my experience with these things, nothing will convince
> > you. 
> 
> Unfortunately you have exactly zero experience with rebasing merges as
> you've never actually rebased them till now, and it's rebasing merges
> that matters in this particular discussion.

Who says that I have 0 experience with that? Oh yes, you do. Like, as if
you know.

Guess what I do with those Git garden shears' merges? Can you guess? Of
course you can. But you'll never know until I tell you. It is a little
silly to try to tell me that I do not have any experience with rebasing
merges when you have no idea what strategies I tried in the past.

Now, Phillip's strategy is clearly the best strategy I ever heard about,
and I am in the process of doing Actual Work to Put It To The Test.

> > You are just stuck with your pre-existing opinion.
> 
> I'm afraid that it's rather your huge experience with re-creating merges
> that makes you stuck to your pre-existing opinion and carefully shields
> you from experiencing actual paradigm shift.

You know what? Whatevs.

Ciao,
Johannes

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Fri, 30 Mar 2018, Sergey Organov wrote:
>
>> Could we please agree to stop using backward compatibility as an
>> objection in the discussion of the  --recreate-merges feature?
>
> No.
>
> The expectation of users as to what a `pick` is has not changed just
> because you wish it would.

As if I ever suggested to change user expectations. Could you please
stop putting words into my mouth?

I _am_ a user, and I expect 'pick' to pick commits, no matter how many
parents they might have.

And no, --preserve-merges did not ever pick commits with number of
parents more than one, it rather threw them away and re-merged the
heads. Calling it 'pick' was a huge mistake indeed! Fixing that mistake
is what I expect, as a user.

Just teach the 'pick' to correctly pick any commit, please!

>
> That is a matter of backwards-compatibility.

OK, fine, at least its only about user expectations and not about some
scripting incompatibility.

> You see, if you are driving a car for a hundred years already, and then
> switch to a different car, and it has a lever in the same place as your
> previous car's windshield wiper, but in the new car it has a button that
> activates the emergency driver seat ejection OMG *it has a seat ejection
> like in the James Bond movies! Where can I get that car?* Sorry for
> disgressing.

Except it's irrelevant as the 'pick' will still pick commits.

> I am really concerned about that willingness to put an innocuous button,
> so to speak, onto something users got really used to, over the course of a
> decade or so, when that button should really be made red and blinking and
> OMG where can I get that car?

It's irrelevant as the 'pick' will still pick commits.

> So to reiterate, I am really interested in a practical solution that won't
> cause nasty surprises.

I rather don't see how it possibly could cause any surprises, especially
compared to using 'merge' to pick commits.

> Meaning: `pick` != merge.

Exactly! Use 'merge' when you merge, as you are already doing. Use 'pick'
when you are picking. You don't merge "merge commit" when you are
picking it!

> That was a mistake in preserve-merges, as I have only mentioned like a
> hundred times, and we won't repeat it.

The mistake was that it used 'pick' to denote re-merge. You already
fixed that mistake by introducing 'merge' to re-merge, thanks God.

Please don't commit yet another mistake by now using 'merge' to pick!

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>
> On Wed, 28 Mar 2018, Sergey Organov wrote:
>
>> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>> 
>> > I use rebase every day. I use the Git garden shears every week. If you
>> > do not trust my experience with these things, nothing will convince
>> > you. 
>> 
>> Unfortunately you have exactly zero experience with rebasing merges as
>> you've never actually rebased them till now, and it's rebasing merges
>> that matters in this particular discussion.
>
> Who says that I have 0 experience with that? Oh yes, you do. Like, as if
> you know.

I just didn't see even single symptom of it in the discussion, still I
said nothing about it until you started to use your presumed experience
in place of true arguments.

> Guess what I do with those Git garden shears' merges? Can you guess? Of
> course you can. But you'll never know until I tell you. It is a little
> silly to try to tell me that I do not have any experience with rebasing
> merges when you have no idea what strategies I tried in the past.

Please notice that I never even started to discuss your 'merge'
directive, exactly because I believe you have huge experience both
implementing and using it that could be relied upon. Just don't mix-in
rebasing merge commits into it, as that is fundamentally different
operation.

And the other unspoken strategies you tried are irrelevant here, as you
declined the whole idea of replaying merge-the-commit instead of
replaying merge-the-operation until recently, and it seems you still do,
at least to some level, by attempting to use 'merge' operation to replay
"the (merge) _commit_". I'm afraid that whatever you've tried in the
past likely suffered from the same conceptual confusion, and thus did
not work indeed.

That said, negative experience is still an experience, often helpful,
but what is relevant here is that it likely was not about rebasing merge
commits at all, so trying to use this irrelevant experience in the
discussion to support your arguments, or rather lack of them, seems even
more unfair to me then using relevant experience for that purpose.

> Now, Phillip's strategy is clearly the best strategy I ever heard
> about,

For 'pick' vs 'merge', it's not about strategy, it's about concept. It
looks like you still believe that you somehow "merge" a merge commit
when you actually rebase it (with Phillip's strategy if you wish). You
don't merge it anywhere, period.

> and I am in the process of doing Actual Work to Put It To The Test.

That's the best outcome of the discussion I ever hoped for, seriously,
and I'll be all ears to listen to the outcomes of the experience you
will gain with it.

BTW, when you have it working, use the strategy you've implemented for
non-merge commits as well, as a good one should still work fine.
Moreover, it should better bring exactly the same results as the default
existing strategy being used for non-merge commits, even in conflicting
situations.

Hopefully /that/ experience will finally push you strong enough to get
the concept right, and you will finally understand that what you've 
implemented is nothing else but a _cherry-pick_ of a _merge commit_,
that reads simply _pick_, for brevity.

-- Sergey

Re: [RFC] Rebasing merges: a jorney to the ultimate solution(RoadClear)

[Sergey Organov]

Hi Johannes,

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Hi Sergey,
>

[...]

> In the parlance of your RFC v2, where you start with this history (which I
> translated into the left-to-right notation that is used in pretty much all
> of Git's own documentation about interactive rebases, which you apparently
> either did not read, or chose *not* to imitate, creating yet another
> unnecessary diversion):
>
>
> - B1
>      \
> - B2 - M

First, it should rather be:

- B1
    \
     M
    / 
- B2

as RFC presents essentially symmetric approach and I'd like it to be
explicit. Representation in RFC simply saves some vertical
space: 

  M
 / \
B1  B2

Another reason to use it is that I liked to somehow indicate that this
is about abstract DAG representation of Git history, not to be confused
with some actual practical Git history. So that, for example, the reader
can't be tempted to even try to assume that M has been necessarily
created by "git merge" operation in the first place.

That said, I'm sorry if it upsets you. I'll stick to your preferred
notation below.

>
> You now insert U1 and U2 with trees identical to M:
>
> - B1 - U1
>           \
> - B2 - U2 - M

 - B1 - U1 
          \
           UM
          / 
 - B2 - U2

_YES_. You've slightly screwed RFC as UM is not M anymore, having
different parents, but otherwise it's still right.

> So U1 is essentially B2 cherry-picked on top of B1, and U2 is essentially
> B1 cherry-picked on top of B2.

_NO_. No any cherry-picking has been involved, and I see absolutely no
reason to pretend there has, except to intentionally make otherwise
simple thing look tricky.

U1 tree is still M tree, and U2 tree is still M tree, and UM tree is
still M tree. That's what actually matters from RFC POV.

> These U1/U2 commits are now to be cherry-picked on top of the rebased B1'
> and B2'. I spare you more diagrams, you get the idea.

_YES_. Exactly 2 cherry-picks.

> Now, the changes in U1/U2 *are* the changes of the merge parents, that's
> how they were constructed.

Either _YES_, or _NO_, depending on the exact meaning of the term "the
changes of the merge parents" you've used, but I suspect it's _NO_,
taking into account your further inferences.

The U1/U2 are constructed by simply duplicating the tree of the original
merge commit M and thus they represent the changes _to_ the merge
parents B1/B2 introduced by M, and not the changes "_of_ the merge
parents" B1/B2, provided the latter meant to have some relation to the
changes introduced by the merge parents B1/B2 themselves.

>
> Since they repeat what B1 and B2 are about,

_NO_, they do not repeat what B1 and B2 are about at all. They rather
represent what M is about. In other words, whatever B1 and B2 are about,
the RFC method doesn't care.

And as this is fundamental misinterpretation of the RFC on your side, it
starts to be big _NO_ from now on...

> and since B1'/B2' means they are rebased, and since U1'/U2' are *also*
> rebased, but independently...
>
> 	...  you essentially have to rebase *every parent's changes
> 	twice*.

_NO_. U1' is rebase of U1 (on top of B1'), and U2' is rebase of U2 (on
top of B2'). Each of U1/U2 is rebased only once.

> The answer "No" to this is... astonishing.

It's still _NO_, sorry.

In fact, I could have said _NO_ the first time you started to assign
some arbitrary "meaning" to the commits, as RFC is about somewhat formal
proof of the method, using already well-known operations on the DAG, and
to criticize the RFC, you need to either find and show a _formal_
mistake somewhere in the proof logic, or to show a use-case where it
fails, as you did for RFC v1. Assigning arbitrary "meaning" to the DAG
nodes and operations on them won't do the trick, sorry.

I'd like to reach some agreement on formal correctness of the RFC first,
and then discuss the meanings, the implementations, and other
consequences based on well-established formal base.

-- Sergey