git rerere is confused with identical conflicts in multiple files

git rerere is confused with identical conflicts in multiple files

[Markos Chandras]

Hi,

(Please keep me on CC as I am not subscribed to the list)

I am having a weird problem when merging a branch which causes some
conflicts

This is the initial status for the rerere cache

$ tree .git/rr-cache
.git/rr-cache [error opening dir]

0 directories, 0 files


I then go ahead and merge the said branch and I end up with the
following conflicts

Auto-merging arch/mips/mm/sc-mips.c
Auto-merging arch/mips/mm/c-r4k.c
CONFLICT (content): Merge conflict in arch/mips/mm/c-r4k.c
Auto-merging arch/mips/kernel/traps.c
CONFLICT (content): Merge conflict in arch/mips/kernel/traps.c
Auto-merging arch/mips/kernel/spram.c
CONFLICT (content): Merge conflict in arch/mips/kernel/spram.c
Auto-merging arch/mips/kernel/idle.c
CONFLICT (content): Merge conflict in arch/mips/kernel/idle.c
Auto-merging arch/mips/kernel/cpu-probe.c
Auto-merging arch/mips/include/asm/mipsregs.h
Auto-merging arch/mips/include/asm/cpu.h
CONFLICT (content): Merge conflict in arch/mips/include/asm/cpu.h
Auto-merging arch/mips/include/asm/cpu-type.h
CONFLICT (content): Merge conflict in arch/mips/include/asm/cpu-type.h
Recorded preimage for 'arch/mips/include/asm/cpu-type.h'
Recorded preimage for 'arch/mips/include/asm/cpu.h'
Automatic merge failed; fix conflicts and then commit the result.

As you see, git only records two preimage files instead of 6.

the rr-cache is like this now

$ tree .git/rr-cache
.git/rr-cache
├── 5563edc0fb427275a0ca5677c93c40def8b53258
│   └── preimage
└── f175ff6228f624296b661664bce4ab4e84d712cc
    └── preimage

2 directories, 2 files

and .git/MERGE_RR

$ cat .git/MERGE_RR
5563edc0fb427275a0ca5677c93c40def8b53258
arch/mips/include/asm/cpu-type.hf175ff6228f624296b661664bce4ab4e84d712cc

arch/mips/include/asm/cpu.h5563edc0fb427275a0ca5677c93c40def8b53258
   arch/mips/kernel/idle.c5563edc0fb427275a0ca5677c93c40def8b53258
arch/mips/kernel/spram.c5563edc0fb427275a0ca5677c93c40def8b53258
arch/mips/kernel/traps.c5563edc0fb427275a0ca5677c93c40def8b53258
arch/mips/mm/c-r4k.c

so as you see, multiple files share the same hash. That's probably
because the "conflicting context ( the part between >>> <<<<)" in every
file but cpu.h is identical and git seems to calculate the hash purely
on the conflicting context. That makes git rerere thinks that it only
has to resolve 2 conflicts instead of 6.

Does anyone have an idea how to resolve that? If my assumption is
correct (I only looked at the git code briefly) I believe it would make
sense to throw the filepath into the sha1 calculation as well in order
to ensure it will not conflict with similar changes across different files.

-- 
markos

Re: git rerere is confused with identical conflicts in multiple files

[Junio C Hamano]

Markos Chandras <Markos.Chandras@imgtec.com> writes:

> $ cat .git/MERGE_RR
> 5563edc0fb427275a0ca5677c93c40def8b53258
> arch/mips/include/asm/cpu-type.hf175ff6228f624296b661664bce4ab4e84d712cc
>
> arch/mips/include/asm/cpu.h5563edc0fb427275a0ca5677c93c40def8b53258
>    arch/mips/kernel/idle.c5563edc0fb427275a0ca5677c93c40def8b53258
> arch/mips/kernel/spram.c5563edc0fb427275a0ca5677c93c40def8b53258
> arch/mips/kernel/traps.c5563edc0fb427275a0ca5677c93c40def8b53258
> arch/mips/mm/c-r4k.c
>
> so as you see, multiple files share the same hash. That's probably
> because the "conflicting context ( the part between >>> <<<<)" in every
> file but cpu.h is identical and git seems to calculate the hash purely
> on the conflicting context. That makes git rerere thinks that it only
> has to resolve 2 conflicts instead of 6.

Yes, that is by design, and should not change.  The thing is, you do
want to share the same resolution across files, regardless of the
path, when the recorded resolution replays cleanly [*1*].

> Does anyone have an idea how to resolve that? If my assumption is
> correct (I only looked at the git code briefly) I believe it would make
> sense to throw the filepath into the sha1 calculation as well in order
> to ensure it will not conflict with similar changes across different files.

Interesting coincidence, but I have been looking at this for the
past few weeks myself but when you are doing the maintainer of a
reasonably active project, time for your own development is hard
to come by X-<.

My current plan is to allow hashing exactly the same way, but
recording different preimage/postimage pairs as necessary.

Right now we do something like this:

    - See conflict; compute conflict ID.
    - Does rr-cache/$ID/ exist?
      - If not, record rr-cache/$ID/preimage
    - Add $ID to MERGE_RR.

And then for each $ID (and path) in MERGE_RR:

    - Does rr-cache/$ID/postimage exist?
      - If so, attempt three-way merge using preimage, postimage and
        thisimage.
        - Did three-way merge replay cleanly?  If so, be happy.
        - Did three-way merge conflict?  If so, punt.
    - Does path still have conflicts?
      - If not, record rr-cache/$ID/postimage.

The thing to fix is "did it conflict, if so punt" step.  Within the
same conflict ID, we would introduce the concept of "variant", and
allow you to keep rr_cache/$ID/{preimage,postimage}.$variant.  The
first part of the per MERGE_RR entry process would instead go like
so:

    - Does rr-cache/$ID/ has one or more postimages?
      - If so, for each variant, attempt three-way merge using
        preimage, postimage and thisimage.
      - Did one of the three-way merges replay cleanly?
        - If so, be happy.
        - If not, assign an unused variant to this path and change
          its MERGE_RR entry from $ID to $ID.$variant

    - Does path still have conflicts?
      - If not, record rr-cache/$ID/postimage for "variant".

The current "preimage", "postimage" will be kept as the first
variant in rr-cache/$ID/ directory.  The second variant will likely
be named (I don't have a code yet but have been slowly laying out
the fundation to allow us to do this) "preimage.0" and "postimage.0",
and the third one will have ".1" suffix.

This approach has the added benefit that existing rr-cache entries
will stay valid (in addition to being able to replay the same
resolution even after you renamed the path that conflict, unlike the
case when you hashed the pathname together to break the conflict ID
computtion).

A WIP has been published on jc/rerere topic in my repository for the
past few weeks, but I haven't reached the interesting "multi variant"
part yet, as I said.


[Footnote]

*1* My experience urges me to add "And most of the time, the same
resolution does apply cleanly even to multiple paths conflicted in
the same merge" to that sentence.

Re: git rerere is confused with identical conflicts in multiple files

[Markos Chandras]

Hi,

On 07/10/2015 03:13 PM, Junio C Hamano wrote:
> Markos Chandras <Markos.Chandras@imgtec.com> writes:
> 
>> $ cat .git/MERGE_RR
>> 5563edc0fb427275a0ca5677c93c40def8b53258
>> arch/mips/include/asm/cpu-type.hf175ff6228f624296b661664bce4ab4e84d712cc
>>
>> arch/mips/include/asm/cpu.h5563edc0fb427275a0ca5677c93c40def8b53258
>>    arch/mips/kernel/idle.c5563edc0fb427275a0ca5677c93c40def8b53258
>> arch/mips/kernel/spram.c5563edc0fb427275a0ca5677c93c40def8b53258
>> arch/mips/kernel/traps.c5563edc0fb427275a0ca5677c93c40def8b53258
>> arch/mips/mm/c-r4k.c
>>
>> so as you see, multiple files share the same hash. That's probably
>> because the "conflicting context ( the part between >>> <<<<)" in every
>> file but cpu.h is identical and git seems to calculate the hash purely
>> on the conflicting context. That makes git rerere thinks that it only
>> has to resolve 2 conflicts instead of 6.
> 
> Yes, that is by design, and should not change.  The thing is, you do
> want to share the same resolution across files, regardless of the
> path, when the recorded resolution replays cleanly [*1*].

I see.

> [...]
> 
> The thing to fix is "did it conflict, if so punt" step.  Within the
> same conflict ID, we would introduce the concept of "variant", and
> allow you to keep rr_cache/$ID/{preimage,postimage}.$variant.  The
> first part of the per MERGE_RR entry process would instead go like
> so:
> 
>     - Does rr-cache/$ID/ has one or more postimages?
>       - If so, for each variant, attempt three-way merge using
>         preimage, postimage and thisimage.
>       - Did one of the three-way merges replay cleanly?
>         - If so, be happy.
>         - If not, assign an unused variant to this path and change
>           its MERGE_RR entry from $ID to $ID.$variant
> 
>     - Does path still have conflicts?
>       - If not, record rr-cache/$ID/postimage for "variant".
> 
> The current "preimage", "postimage" will be kept as the first
> variant in rr-cache/$ID/ directory.  The second variant will likely
> be named (I don't have a code yet but have been slowly laying out
> the fundation to allow us to do this) "preimage.0" and "postimage.0",
> and the third one will have ".1" suffix.
> 
> This approach has the added benefit that existing rr-cache entries
> will stay valid (in addition to being able to replay the same
> resolution even after you renamed the path that conflict, unlike the
> case when you hashed the pathname together to break the conflict ID
> computtion).

I understand. Thanks for the explanation.

> 
> A WIP has been published on jc/rerere topic in my repository for the
> past few weeks, but I haven't reached the interesting "multi variant"
> part yet, as I said.

I am happy to test it when you have something more complete. If you can
reply to this e-mail when the 'variant' patch finds its way into the
master branch that would be great as well

-- 
markos