Collaborative conflict resolution feature request

Collaborative conflict resolution feature request

[Curtin, Eric]

Hi Guys,

Sometimes in our private git instance in the company I work for we merge branches that have been forked for months and there can be several or more people involved in the conflict resolution.

At the moment we have two options:

- One person, a branch manager, solves them by ringing people, holding meetings, using best judgement, etc.
- Somebody solves the conflicts they are involved with, marks everything as resolved and pushes (leaving <<< ==== >>>> delimiters in for unsolved conflicts) for the next person to continue. This sort of works although you falsely mark everything as resolved, leaving merge tools useless and many broken, unbuildable commits around in the branch.

Note: rebase and squashing commits is banned in our org, basically anything that would rewrite history on a remote branch.

Is there any existing or upcoming feature in git that could help make conflict resolution a more distributed, collaborative kind of task?

I also opened this as an issue in github as I feel it could be solved by either tool potentiall:

https://github.com/isaacs/github/issues/1816

Regards,

Eric Curtin

Software Engineer
Ovens Campus,
Cork,
Ireland

Dell EMC

Re: Collaborative conflict resolution feature request

[Johannes Sixt]

Am 12.06.20 um 16:08 schrieb Curtin, Eric:
> Sometimes in our private git instance in the company I work for we
> merge branches that have been forked for months and there can be
> several or more people involved in the conflict resolution.
> 
> At the moment we have two options:
> 
> - One person, a branch manager, solves them by ringing people,
> holding meetings, using best judgement, etc.
> - Somebody solves the conflicts they are involved with, marks
> everything as resolved and pushes (leaving <<< ==== >>>> delimiters
> in for unsolved conflicts) for the next person to continue. This sort
> of works although you falsely mark everything as resolved, leaving
> merge tools useless and many broken, unbuildable commits around in
> the branch.

Third option: Do not merge the whole branch in one big do-it-all-at-once
merge. Instead, pick strategic commits in the history of the branch such
that, when you merge them one after the other, each has only conflicts
in one particular area or topic, and so can be solved in a reasonable
amount of time with reasonable resources.

-- Hannes

Re: Collaborative conflict resolution feature request

[Christian Couder]

Hi,

On Fri, Jun 12, 2020 at 4:11 PM Curtin, Eric <Eric.Curtin@dell.com> wrote:

> Is there any existing or upcoming feature in git that could help make conflict resolution a more distributed, collaborative kind of task?

You might want to take a look at Michael Haggerty's 'git imerge':

https://github.com/mhagger/git-imerge

> I also opened this as an issue in github as I feel it could be solved by either tool potentiall:
>
> https://github.com/isaacs/github/issues/1816

I also made the same suggestion on the issue.

Best,
Christian.

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

Both great ideas! And have the same theory right? Merge until you come across the first conflicting commit in a branch to make the conflicts smaller right?

Thanks so much for your help! Any alternative ideas? I'm definitely going to try both techniques, although imerge seems like an automation of the first idea.

Anybody ever think of rewriting the imerge tool in C or whatever to get in merged into mainline git? Potentially I could do it as part of my masters thesis if Michael H and the git open source community agreed?

Regards,

Eric Curtin

Software Engineer
Ovens Campus,
Cork,
Ireland

Dell EMC

From: Christian Couder <christian.couder@gmail.com>
Sent: Saturday 13 June 2020 13:08
To: Curtin, Eric <Eric.Curtin@dell.com>
Cc: git@vger.kernel.org <git@vger.kernel.org>; Geary, Niall <Niall.Geary@dell.com>; rowlands, scott <Scott.Rowlands@dell.com>; Michael Haggerty <mhagger@alum.mit.edu>
Subject: Re: Collaborative conflict resolution feature request 
 

[EXTERNAL EMAIL] 

Hi,

On Fri, Jun 12, 2020 at 4:11 PM Curtin, Eric <Eric.Curtin@dell.com> wrote:

> Is there any existing or upcoming feature in git that could help make conflict resolution a more distributed, collaborative kind of task?

You might want to take a look at Michael Haggerty's 'git imerge':

https://github.com/mhagger/git-imerge

> I also opened this as an issue in github as I feel it could be solved by either tool potentiall:
>
> https://github.com/isaacs/github/issues/1816

I also made the same suggestion on the issue.

Best,
Christian.

Re: Collaborative conflict resolution feature request

[Philip Oakley]

On 13/06/2020 13:38, Curtin, Eric wrote:
> Both great ideas! And have the same theory right? Merge until you come across the first conflicting commit in a branch to make the conflicts smaller right?
I've also responded to the issue on GitHub:

"Do you have an implicit XY-problem where your processes are reinforcing
historical bad habits "we merge branches that have been forked for
months"? It sounds like the process is saying "We enjoy technical debt"
(of delaying the merge until it's really bad...).

Maybe have a parallel 'merge' branch that is used (say weekly) to do
trial merges and will essentially record the conflict resolutions while
they are fresh in folks memories. That branch is distinct from, either
of the two main branches, but will act as a filter and a hand rail to
highlight future difficulties."

Implicit in Git is the use of small patches, easy branching and frequent
merges, available to the individual coder. Most older "change control
systems" focus on *stopping* change. Git promotes change, because
reproduction & testing is cheap (almost zero). Git provides
*authentication* of code versions. The changes in the underlying
materials (from hardware to software), i.e. to bits and bytes, ripped up
the old rule book.

Also look at 'rerere'.
>
> Thanks so much for your help! Any alternative ideas? I'm definitely going to try both techniques, although imerge seems like an automation of the first idea.
>
> Anybody ever think of rewriting the imerge tool in C or whatever to get in merged into mainline git? Potentially I could do it as part of my masters thesis if Michael H and the git open source community agreed?
>
> Regards,
>
> Eric Curtin
>
> Software Engineer
> Ovens Campus,
> Cork,
> Ireland
>
> Dell EMC
>
> From: Christian Couder <christian.couder@gmail.com>
> Sent: Saturday 13 June 2020 13:08
> To: Curtin, Eric <Eric.Curtin@dell.com>
> Cc: git@vger.kernel.org <git@vger.kernel.org>; Geary, Niall <Niall.Geary@dell.com>; rowlands, scott <Scott.Rowlands@dell.com>; Michael Haggerty <mhagger@alum.mit.edu>
> Subject: Re: Collaborative conflict resolution feature request 
>  
>
> [EXTERNAL EMAIL] 
>
> Hi,
>
> On Fri, Jun 12, 2020 at 4:11 PM Curtin, Eric <Eric.Curtin@dell.com> wrote:
>
>> Is there any existing or upcoming feature in git that could help make conflict resolution a more distributed, collaborative kind of task?
> You might want to take a look at Michael Haggerty's 'git imerge':
>
> https://github.com/mhagger/git-imerge
>
>> I also opened this as an issue in github as I feel it could be solved by either tool potentiall:
>>
>> https://github.com/isaacs/github/issues/1816
> I also made the same suggestion on the issue.
>
> Best,
> Christian.

Re: Collaborative conflict resolution feature request

[Junio C Hamano]

Philip Oakley <philipoakley@iee.email> writes:

> Maybe have a parallel 'merge' branch that is used (say weekly) to do
> trial merges and will essentially record the conflict resolutions while
> they are fresh in folks memories. That branch is distinct from, either
> of the two main branches, but will act as a filter and a hand rail to
> highlight future difficulties."

An aside that probably would not directly help Eric, but I know the
above workflow helps reasonably well.  The 'pu' branch is rebuilt
not on top of 'next', but is rebuilt with all topics (including
those already in 'next') in flight directly on top of 'master',
which serves as a way to anticipate conflicts that will require
resolution in the future before the topics can enter 'next' branch.
And these resolutions are ...

> Also look at 'rerere'.

... remembered in the rerere database (and even trickier ones that
rerere cannot handle are recorded in the merge-fix commits, but that
is a separate story).  When topics are ready to be merged to 'next'
and to 'master', the correct resolutions are likely to be known and
the result tested in 'pu' and 'next', respectively, for some time
already.

Re: Collaborative conflict resolution feature request

[Christian Couder]

On Sat, Jun 13, 2020 at 2:38 PM Curtin, Eric <Eric.Curtin@dell.com> wrote:

> Anybody ever think of rewriting the imerge tool in C or whatever to get in merged into mainline git? Potentially I could do it as part of my masters thesis if Michael H and the git open source community agreed?

(We usually reply inline instead of top-posting here.)

My opinion is that it would be nice to have something like git-imerge
integrated into Git.

Re: Collaborative conflict resolution feature request

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> My opinion is that it would be nice to have something like git-imerge
> integrated into Git.

I am not sure what you mean by "integrated into", but if you are
talking about somehow reinventing it, I do not think it is a good
idea at all.  "imerge" works quite well already.

Or do you mean it would gain more exposure for being bundled
together?  I am not sure if that is a good idea, either.  

Surely, it would be convenient for end users if a single download
(or "apt-get install") gives everything useful, but that does not
have to be done at the ultimate upstream level between me and
Michael, and doing so at that level would mean the release schedule
needs to be coordinated (one may need to wait unnecessarily for the
other's pre-release freeze period, for example), among other loss of
flexibility.  Luckily, most end users would get their Git from
packagers and they are good at doing the bundling (i.e. the
"git-core" package may "suggest" the "git-imerge" package).

So...

Re: Collaborative conflict resolution feature request

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ...  Luckily, most end users would get their Git from
> packagers and they are good at doing the bundling (i.e. the
> "git-core" package may "suggest" the "git-imerge" package).
>
> So...

So my answer to your idea/opinion is that we shouldn't waste
engineering effort to "have something like imerge integrated into
git itself", but we should help distro packages to do the bundling
of "git" itself and all the good things around it.  One way of doing
it may be by keeping an official curated list of "third-party things
we find good" somewhere (it can be in-tree in my release tarballs,
but it does not have to be---some page on git-scm.com could just be
fine; as long as the quality of the list is maintained to our
standards, where the packagers and end users see it does not really
matter).

And such a list would also help those who prefer to build and
install things by hand.

Re: Collaborative conflict resolution feature request

[Philip Oakley]

On 13/06/2020 20:34, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> ...  Luckily, most end users would get their Git from
>> packagers and they are good at doing the bundling (i.e. the
>> "git-core" package may "suggest" the "git-imerge" package).
>>
>> So...
> So my answer to your idea/opinion is that we shouldn't waste
> engineering effort to "have something like imerge integrated into
> git itself", but we should help distro packages to do the bundling
> of "git" itself and all the good things around it.  One way of doing
> it may be by keeping an official curated list of "third-party things
> we find good" somewhere (it can be in-tree in my release tarballs,
> but it does not have to be---some page on git-scm.com could just be
> fine; as long as the quality of the list is maintained to our
> standards, where the packagers and end users see it does not really
> matter).
>
> And such a list would also help those who prefer to build and
> install things by hand.

For the imerge tool, it may be worth having an extra sub-heading (Merge
Tools?) within the "HOW TO RESOLVE CONFLICTS" section of the git-merge
man page.

The merge.guitool configuration does list a lot of pre-configured tools
(the list could be moved to the mergetool man page?)

Curating the list of tools maybe could be done in the same way the
config entries are now being done, i.e. by area, so they can be included
in the relevant man pages, with just a single source of 'nice tools'.

Philip

Re: Collaborative conflict resolution feature request

[Konstantin Tokarev]

13.06.2020, 22:22, "Junio C Hamano" <gitster@pobox.com>:
> Christian Couder <christian.couder@gmail.com> writes:
>
>>  My opinion is that it would be nice to have something like git-imerge
>>  integrated into Git.
>
> I am not sure what you mean by "integrated into", but if you are
> talking about somehow reinventing it, I do not think it is a good
> idea at all. "imerge" works quite well already.

No it doesn't. It has performance issues which makes it practically useless
on the large scale repositories because of slow progress even on fastest
machines. See for example 

https://github.com/mhagger/git-imerge/issues/66
https://github.com/mhagger/git-imerge/issues/149

Also, maybe a bit irrelevant, but it was recently removed from Gentoo
because it didn;t work with Python >= 3.7

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

> An aside that probably would not directly help Eric, but I know the
> above workflow helps reasonably well.  The 'pu' branch is rebuilt
> not on top of 'next', but is rebuilt with all topics (including
> those already in 'next') in flight directly on top of 'master',
> which serves as a way to anticipate conflicts that will require
> resolution in the future before the topics can enter 'next' branch.

Out of all the currently available options this solution helps. Thanks
Junio! I played with incremental merge techniques this weekend.
One problem with incrementally merging is that you start fixing
conflicts that are later invalidated by subsequent commits. So it
seems you end up doing more conflict resolution than necessary.
Unless I'm misusing the technique.

I think in order to create collaborative distributed conflict resolution,
you'd probably need a new type of commit a "partial-merge" commit,
that is like a pseudo-commit that you can push and doesn't break
builds. It would be a neat feature, at least for my team!

Regards,

Eric Curtin

Software Engineer

Ovens Campus,
Cork,
Ireland

Dell EMC

Re: Collaborative conflict resolution feature request

[Sergey Organov]

Philip Oakley <philipoakley@iee.email> writes:


[...]

> Also look at 'rerere'.

'rerere' is a superb feature, but isn't it local? If so, how could it
help for collaboration? What's the idea? Is there a way to share
'rerere'?

Thanks,
-- Sergey

Re: Collaborative conflict resolution feature request

[Philip Oakley]

On 15/06/2020 10:51, Sergey Organov wrote:
> Philip Oakley <philipoakley@iee.email> writes:
>
>
> [...]
>
>> Also look at 'rerere'.
> 'rerere' is a superb feature, but isn't it local? If so, how could it
> help for collaboration? What's the idea? Is there a way to share
> 'rerere'?
I saw this (rerere) is two parts. First was to ensure Eric was aware of
it as a possible capability for use by the 'merge manager', and others,
so that they didn't loose sight of their conflict resolutions for the
time when the 'big merge window' came around. E.g. So a dev could do a
local fix based on their rerere database and then send a patch to the
merge manager indication their approach to the resolution.

Meanwhile, second, at the moment the rerere database is 'local', mainly,
as I understand it because of the number of context lines a local user
has chosen (hence not immediately portable).

I personally believe that it should be possible to some how exchange
resolutions without that pre-optimisation of the context line choice. I
had a look back at the old rerere script and that had small fingerprints
of each resolution stored in the database (at least as I read it).
However when I look at the modern rerere database it looks like it has
full pre & post images, rather than just the conflicts, so I'm not yet
sure what's really happening (i.e. I haven't dived into the c code).

A possibly more sensible approach (to exchanging resolutions) is simply
to do the merge, without commit, then save that merge as if it's a
single side commit (with the other merge parent listed in the commit
message), and that commit can then be pushed/pulled etc and a variant of
`rerere train` can be used to recreate the local database. In a sense
the fake merge (why not a proper merge) is a variant of a stash where
you aren't wanting to pollute the branch trees with this extra 'flotsam'.

Philip
>

Re: Collaborative conflict resolution feature request

[Philip Oakley]

On 15/06/2020 10:28, Curtin, Eric wrote:
> I think in order to create collaborative distributed conflict resolution,
> you'd probably need a new type of commit a "partial-merge" commit,
> that is like a pseudo-commit that you can push and doesn't break
> builds. It would be a neat feature, at least for my team!
One thing I did note, from your prompting, is that merge doesn't take
any `-- <files>` options which could allow a quick way of selecting just
the few files that you, or the specific dev, is able to merge, so as to
create a partial merge on your side branch, and leave behind the
remaining difficult conflicts to be resolved later by the appropriate
dev in a sequence of rolling partial merges.

It could be effectively a special strategy. IIUC the '--' separator is
already supported by the underlying parser code, so may not be that
hard? (perhaps a local contribution to the codebase;-). Just a thought.

Philip

Re: Collaborative conflict resolution feature request

[Sergey Organov]

"Curtin, Eric" <Eric.Curtin@dell.com> writes:
> Hi Guys,
>
> Sometimes in our private git instance in the company I work for we
> merge branches that have been forked for months and there can be
> several or more people involved in the conflict resolution.
>
> At the moment we have two options:
>
> - One person, a branch manager, solves them by ringing people, holding
> meetings, using best judgement, etc.
> - Somebody solves the conflicts they are involved with, marks
> everything as resolved and pushes (leaving <<< ==== >>>> delimiters in
> for unsolved conflicts) for the next person to continue. This sort of
> works although you falsely mark everything as resolved, leaving merge
> tools useless and many broken, unbuildable commits around in the
> branch.
>
> Note: rebase and squashing commits is banned in our org, basically
> anything that would rewrite history on a remote branch.
>
> Is there any existing or upcoming feature in git that could help make
> conflict resolution a more distributed, collaborative kind of task?

That'd be great.

What we sometimes do when such a case appears (rather rarely due to
frequent merges, I admit) is to copy /entire/ git directory holding the
merge in progress to the next person in charge. This is the only way
that I'm aware of that keeps ability to do things like:

  $ f=foo.cc; git diff :1:./$f  :3:./$f

and

  $ f=foo.cc; git diff :1:./$f  :2:./$f

that helps a lot with complicated conflicts.

So, if such a solution ever appears, it'd apparently need a method of
sharing or re-creating (parts of) the index? Doesn't sound as an easy
problem.

-- Sergey

Re: Collaborative conflict resolution feature request

[Junio C Hamano]

Philip Oakley <philipoakley@iee.email> writes:

> It could be effectively a special strategy. IIUC the '--' separator is
> already supported by the underlying parser code, so may not be that
> hard? (perhaps a local contribution to the codebase;-). Just a thought.

Assuming that there are paths A and B that would leave conflict in
an attempted merge between commits X and Y, you somehow resolve the
conflict in A and leave B unresolved, what would you pass to the
other person and ask to resolve the conflicts in B?  It cannot be a
merge commit that records X and Y as its parents and a single tree
object as the result, because the whole point of this is that you do
not even know what to record for B.

I think the most important and useful part of this is to design the
data format used for that task of passing from you to the other
person.  The way to specify which paths are yours etc. are much less
interesting and trivial part of the story, I would think.

Re: Collaborative conflict resolution feature request

[Chris Torek]

On Mon, Jun 15, 2020 at 10:11 AM Junio C Hamano <gitster@pobox.com> wrote:
> Assuming that there are paths A and B that would leave conflict in
> an attempted merge between commits X and Y, you somehow resolve the
> conflict in A and leave B unresolved, what would you pass to the
> other person and ask to resolve the conflicts in B?  It cannot be a
> merge commit that records X and Y as its parents and a single tree
> object as the result, because the whole point of this is that you do
> not even know what to record for B.
>
> I think the most important and useful part of this is to design the
> data format used for that task of passing from you to the other
> person.  The way to specify which paths are yours etc. are much less
> interesting and trivial part of the story, I would think.

I've thought about this (some) myself in the past.  It seems to me that what
is needed is the ability to pass the complete unmerged state on.  That is,
we need something like five commits, made in a way similar to the way a
stash is made:

 * all resolved (stage 0) index entries (1 tree)
 * all index stage 1, 2, and 3 entries (3 trees)
 * and the current work-tree (1 tree)

This isn't quite complete as we also need the merge information; perhaps
that goes into a sixth commit, or can be wedged into the commit message
text of one (or more?) of the five trees.

The "resume unresolved merge" command then must unpack all of this
into the appropriate places.

Chris

Re: Collaborative conflict resolution feature request

[Philip Oakley]

Hi Junio,
On 15/06/2020 17:57, Junio C Hamano wrote:
> Philip Oakley <philipoakley@iee.email> writes:
>
>> It could be effectively a special strategy. IIUC the '--' separator is
>> already supported by the underlying parser code, so may not be that
>> hard? (perhaps a local contribution to the codebase;-). Just a thought.
> Assuming that there are paths A and B that would leave conflict in
> an attempted merge between commits X and Y,
Are we confusing the file merge X.A and Y.A with X.B and Y.B?

The scenario envisaged is that dev.a has responsibility over the .A file
merge, while dev.b will handle the merge for .B merge (e.g. different
parts of the driver code).

>  you somehow resolve the
> conflict in A and leave B unresolved,
So dev.a has resolved the .A conflicts, and the .B file(s) is still in
the .ours state - no conflict (i.e. we have an 'ours' strategy for all
files not in the A path(s)).

>  what would you pass to the
> other person and ask to resolve the conflicts in B?
Yes the merge that dev.a has performed is passed onto dev.b to complete
a second merge, apparently of the same commit, which is essentially
similar to cherry picking the .B files from the .theirs commit [1].

>   It cannot be a
> merge commit that records X and Y as its parents and a single tree
> object as the result, because the whole point of this is that you do
> not even know what to record for B.
if no merge is attempted for paths not in the A set then the other B set
are never 'in conflict'. As I understand Eric's conundrum, it's that a
plain merge throws too many conflicts at the one developer. This
suggestion would be the way to reduce the number of conflicts just to
the paths the one dev can handle. A lot will depend on how Eric's
problem is partitioned and the depth of the overlaps.
>
> I think the most important and useful part of this is to design the
> data format used for that task of passing from you to the other
> person.  The way to specify which paths are yours etc. are much less
> interesting and trivial part of the story, I would think.
A list of un-merged paths (i.e. not attempted) is one such format, surely?

Philip

[1] actually, it's more like a soft checkout or restore of those files
from the merging branch. (exchange format becomes list of merged
paths...).  Which  then leads easily to the available `git checkout`
options (assuming the approach has validity for Eric's scenario).

Re: Collaborative conflict resolution feature request

[Christian Couder]

On Sat, Jun 13, 2020 at 2:08 PM Christian Couder
<christian.couder@gmail.com> wrote:
>
> On Fri, Jun 12, 2020 at 4:11 PM Curtin, Eric <Eric.Curtin@dell.com> wrote:
>
> > Is there any existing or upcoming feature in git that could help make conflict resolution a more distributed, collaborative kind of task?
>
> You might want to take a look at Michael Haggerty's 'git imerge':
>
> https://github.com/mhagger/git-imerge

Not sure if it could be useful in your case but some people also like
a tool called git-mediate

See:
  - https://github.com/Peaker/git-mediate
  - https://medium.com/@yairchu/how-git-mediate-made-me-stop-fearing-merge-conflicts-and-start-treating-them-like-an-easy-game-of-a2c71b919984

Re: Collaborative conflict resolution feature request

[Chris Torek]

On Mon, Jun 15, 2020 at 10:32 AM Chris Torek <chris.torek@gmail.com> wrote:
> I've thought about this (some) myself in the past.  It seems to me that what
> is needed is the ability to pass the complete unmerged state on.

A few further thoughts:

 * Given one or more saved merges and either a clean state or an
   ongoing merge, we need a tool to combine these.  There are a lot of
   corner cases here but in general, if merge X has file F in conflict and
   merge Y has file F resolved, we can take the resolution from Y.

 * Partial merges (in the work-tree copy of a file) that are not yet added
   may be the trickiest.  A simple heuristic would be to look for the
   conflict markers and see if one work-tree copy has a resolution
   where another work-tree copy has a conflict.  Or, though this is
   harder, use the ours/theirs copies in the saved index trees to find
   actual conflicted regions and compare this to the work-tree copy
   to find resolved regions.

 * There is also an obvious question about what to do when combining
   two different proposed resolutions where the stage-zero and/or
   work-tree copies of the files don't match.

None of these preclude the basic ability to save and restore—and of
course transport, through fetch/push—the unmerged state, which I think
is the required enabling technology.  The ideas above are more for
combining parallel merge efforts.  If it's acceptable for dev A to merge
his/her part and pass the result to dev B, who merges theirs, and so
on, the above is not required.

Chris

Re: Collaborative conflict resolution feature request

[Stefan Moch]

Philip Oakley wrote:
> On 15/06/2020 10:51, Sergey Organov wrote:
>> Philip Oakley <philipoakley@iee.email> writes:
>>
>>
>> [...]
>>
>>> Also look at 'rerere'.
>> 'rerere' is a superb feature, but isn't it local? If so, how could it
>> help for collaboration? What's the idea? Is there a way to share
>> 'rerere'?
> I saw this (rerere) is two parts. First was to ensure Eric was aware of
> it as a possible capability for use by the 'merge manager', and others,
> so that they didn't loose sight of their conflict resolutions for the
> time when the 'big merge window' came around. E.g. So a dev could do a
> local fix based on their rerere database and then send a patch to the
> merge manager indication their approach to the resolution.
>
> Meanwhile, second, at the moment the rerere database is 'local', mainly,
> as I understand it because of the number of context lines a local user
> has chosen (hence not immediately portable).
>
> I personally believe that it should be possible to some how exchange
> resolutions without that pre-optimisation of the context line choice. I
> had a look back at the old rerere script and that had small fingerprints
> of each resolution stored in the database (at least as I read it).
> However when I look at the modern rerere database it looks like it has
> full pre & post images, rather than just the conflicts, so I'm not yet
> sure what's really happening (i.e. I haven't dived into the c code).
>
> A possibly more sensible approach (to exchanging resolutions) is simply
> to do the merge, without commit, then save that merge as if it's a
> single side commit (with the other merge parent listed in the commit
> message), and that commit can then be pushed/pulled etc and a variant of
> `rerere train` can be used to recreate the local database. In a sense
> the fake merge (why not a proper merge) is a variant of a stash where
> you aren't wanting to pollute the branch trees with this extra 'flotsam'.

There is a `contrib/rerere-train.sh` script in git's repository,
that can recreate rerere resolutions from existing merge commits.

Extending the options Eric outlined, a collaborative conflict
resolution workflow might thus be:

  * developers do test merges on temporary branches between their
    feature branch and the main development – or other feature
    branches if necessary (maybe create test merges on a regular
    basis to minimize the new conflicts)
  * these temporary branches get pushed, but not merged to other
    branches
  * the branch manager fetches these branches and uses
    `rerere-train.sh` to fill the local rerere database with
    conflict resolutions from the test merges
  * the temporary branches get deleted
  * the recorded resolutions get reused when needed (keep in mind
    rerere's gc config, see gc.rerereResolved and gc.rerereUnresolved)

The last discussion on rerere-train.sh on this list was here:

https://lore.kernel.org/git/BZAQIE4YND2I.Z7BFCW7BLH3K@penguin/

Re: Collaborative conflict resolution feature request

[Junio C Hamano]

Philip Oakley <philipoakley@iee.email> writes:

> Hi Junio,
> On 15/06/2020 17:57, Junio C Hamano wrote:
>> Philip Oakley <philipoakley@iee.email> writes:
>>
>>> It could be effectively a special strategy. IIUC the '--' separator is
>>> already supported by the underlying parser code, so may not be that
>>> hard? (perhaps a local contribution to the codebase;-). Just a thought.
>> Assuming that there are paths A and B that would leave conflict in
>> an attempted merge between commits X and Y,
> Are we confusing the file merge X.A and Y.A with X.B and Y.B?

No.

> The scenario envisaged is that dev.a has responsibility over the .A file
> merge, while dev.b will handle the merge for .B merge (e.g. different
> parts of the driver code).

Yes.  The question is, that division of labor is agreed between
humans dev.a and dev.b, but somehow need to be encoded in the data
passed from dev.a to dev.b that says "I've done with As and am
comfortable with the result; I didn't even look at Bs, but it's your
turn to deal with them".

After seeing "git pull" leave conflicts, "git checkout HEAD -- Bs"
(or checkout may be done out of MERGE_HEAD?  I dunno) would be a way
to get dev.a concentrate on As only, but the result cannot be tested
sensibly anyway, and it cannot be committed as the result as taking
"ours" or "theirs" for Bs was not dev.a's intention.  How to express
that "I didn't do anything to Bs" in such a way that can be
distinguished from "I did look at Bs, and I believe taking all from
my HEAD is the right resolution" was what I asked.

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

Hi Guys,

Yes I think you all understand the conundrum well. Conflict resolution
by definition is a collaborative effort, but git doesn't support it as a,
collaborative effort, only one user can resolve it in git. It will be hard to
change my whole orgs thinking around avoiding conflicts or making
them easier conflicts to solve. There will always be some conflicts.

>  * developers do test merges on temporary branches between their
>    feature branch and the main development – or other feature
>    branches if necessary (maybe create test merges on a regular
>    basis to minimize the new conflicts)
>  * these temporary branches get pushed, but not merged to other
>    branches
>  * the branch manager fetches these branches and uses
>    `rerere-train.sh` to fill the local rerere database with
>    conflict resolutions from the test merges
>  * the temporary branches get deleted
>  * the recorded resolutions get reused when needed (keep in mind
>    rerere's gc config, see gc.rerereResolved and gc.rerereUnresolved)

I certainly want to play around with rere and see if it helps things. I
suspect if I shared a technique like this with the 100 or so developers
on the project this will be deemed too complex though.

A per file solution isn't great either as some files can be large.
Per-conflict (between <<<< >>>> in a plain old text editor) is
reasonable.

What would be most ideal is a:

git merge
fix some conflicts, not others
git push

so someone else can work on it kind of solution....

We don't do plaintext email patches, we do typically merge
things via git cli/protocol or via GitHub Pull Request.

Regards,

Eric Curtin

Software Engineer
Ovens Campus,
Cork,
Ireland

Dell EMC

Re: Collaborative conflict resolution feature request

[Sergey Organov]

"Curtin, Eric" <Eric.Curtin@dell.com> writes:

> Hi Guys,
>
> Yes I think you all understand the conundrum well. Conflict resolution
> by definition is a collaborative effort, but git doesn't support it as a,
> collaborative effort, only one user can resolve it in git.

[As a side-note, I don't agree with "by definition" part of the
statement, nor with "only one user" part, so what is left?]

What I'd like to stress though is that there is a pitfall here: is it
feasible to try to support concurrent conflict resolution, or is it to
be sequential (even if in multiple turns)? I incline to the latter.

Concurrent conflict resolution would lead to conflicts in conflict
resolutions, that already sounds too complex to be useful for my taste,
and we already are in recursion that must be stopped somewhere, so it's
tempting to stop it one level up.

Admittedly, one can try to avoid conflicts in resolutions by splitting
content to independent parts (where Git is not easily applicable by
design, being the tool that manages entire content), but such a split
doesn't sound realistic, as if it were possible, it'd be easier to apply
it to avoid original conflicts in the first place. Then, a non-trivial
conflict resolution often needs changes elsewhere anyway, making strict
content split again problematic.

OTOH, anything sequential is intrinsically difficult for a distributed
system like Git, so it probably should better be implemented elsewhere.
This approach is simplified by the fact that fortunately Git doesn't
care how exactly you come-up with the resulting merge commit[*], so one
is free to use whatever external tool to achieve his goals.

-- Sergey

[*] Well, it actually does care in some corner cases, e.g., when it
drops or tries to re-create merges on rebase, and that will only be
eliminated once "evil merge" concept is finally buried, for better.

Re: Collaborative conflict resolution feature request

[demerphq]

On Mon, 15 Jun 2020 at 11:30, Curtin, Eric <Eric.Curtin@dell.com> wrote:
>
> > An aside that probably would not directly help Eric, but I know the
> > above workflow helps reasonably well.  The 'pu' branch is rebuilt
> > not on top of 'next', but is rebuilt with all topics (including
> > those already in 'next') in flight directly on top of 'master',
> > which serves as a way to anticipate conflicts that will require
> > resolution in the future before the topics can enter 'next' branch.
>
> Out of all the currently available options this solution helps. Thanks
> Junio! I played with incremental merge techniques this weekend.
> One problem with incrementally merging is that you start fixing
> conflicts that are later invalidated by subsequent commits. So it
> seems you end up doing more conflict resolution than necessary.
> Unless I'm misusing the technique.

I find that the solution in these cases is to first use interactive
rebase to squash and reorganize the commits in the branches so you
have a nice clean patch sequence. Once you have the branches cleaned
up and squashed into a sequence of reasonable topic based chunks you
then merge, sometimes it even means you dont get conflicts at all, git
merge is pretty smart.

And after that you change your workflows so the rule is that whomever
pushes first to the "trunk branch" wins, and the other guy has to do
the conflict resolution. People will start merging earlier and more
often so they can keep the conflicts to a minimum. :-) In other words
I second what Philip Oakley said about bad workflows. Merge early,
merge often, rollout early, rollout often, vote early, vote often. :-)

cheers,
Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

> And after that you change your workflows so the rule is that whomever
> pushes first to the "trunk branch" wins, and the other guy has to do
> the conflict resolution. People will start merging earlier and more
> often so they can keep the conflicts to a minimum. :-) In other words
> I second what Philip Oakley said about bad workflows. Merge early,
> merge often, rollout early, rollout often, vote early, vote often. :-)

I understand what you guys are saying, I agree merge early, merge often
does work best and it's what I've most often used. Our branching strategy
is split by subsystem (and sometimes specific features). So contributors are
not merging to the same long-term branches. So merge early, merge often,
doesn't help with conflicts.

Why? We don't have so much automated tests, etc. And that's not an
easy problem to solve although we are trying to improve in this area. For us
to make worthwhile tests, we would have to emulate a lot of hardware
from external vendors. Out test hardware is often shared among many.
So branching ensures that if we hit problems, it most likely came from
within our group, in the area of the code we are most knowledgeable
about.

And when your branching strategy is like this, it ends up being the
branch managers merging and having to fix conflicts, not the individual
contributors. In our git repo you will see broken commits with these
delimiters in (<<<< ==== >>>>) in order to "fake" this kind of
collaborative conflict resolution feature as described in the initial email.

Regards,

Eric Curtin

Software Engineer
Ovens Campus,
Cork,
Ireland

Dell EMC

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

> What I'd like to stress though is that there is a pitfall here: is it
> feasible to try to support concurrent conflict resolution, or is it to
> be sequential (even if in multiple turns)? I incline to the latter.

> Concurrent conflict resolution would lead to conflicts in conflict
> resolutions, that already sounds too complex to be useful for my taste,
> and we already are in recursion that must be stopped somewhere, so it's
> tempting to stop it one level up.

I think concurrent doesn't make sense, only sequential.

> I find that the solution in these cases is to first use interactive
> rebase to squash and reorganize the commits in the branches so you
> have a nice clean patch sequence. Once you have the branches cleaned
> up and squashed into a sequence of reasonable topic based chunks you
> then merge, sometimes it even means you dont get conflicts at all, git
> merge is pretty smart.

Again, as said in the initial email, anything that rewrites history,
recreates SHA's (such as rebase, squash, etc.) on a remote
branch is not allowed in our repo. Of course with unpushed
commits you can do some of these things as the remote end
knows no different.

Regards,

Eric Curtin

Software Engineer
Ovens Campus,
Cork,
Ireland

Dell EMC

Re: Collaborative conflict resolution feature request

[demerphq]

On Thu, 18 Jun 2020 at 11:28, Curtin, Eric <Eric.Curtin@dell.com> wrote:
>
> > What I'd like to stress though is that there is a pitfall here: is it
> > feasible to try to support concurrent conflict resolution, or is it to
> > be sequential (even if in multiple turns)? I incline to the latter.
>
> > Concurrent conflict resolution would lead to conflicts in conflict
> > resolutions, that already sounds too complex to be useful for my taste,
> > and we already are in recursion that must be stopped somewhere, so it's
> > tempting to stop it one level up.
>
> I think concurrent doesn't make sense, only sequential.
>
> > I find that the solution in these cases is to first use interactive
> > rebase to squash and reorganize the commits in the branches so you
> > have a nice clean patch sequence. Once you have the branches cleaned
> > up and squashed into a sequence of reasonable topic based chunks you
> > then merge, sometimes it even means you dont get conflicts at all, git
> > merge is pretty smart.
>
> Again, as said in the initial email, anything that rewrites history,
> recreates SHA's (such as rebase, squash, etc.) on a remote
> branch is not allowed in our repo. Of course with unpushed
> commits you can do some of these things as the remote end
> knows no different.

Ah I see, I missed that detail. We have a similar rule at work but
only for the "trunk" branch (what most people call "master"), topic
branches are allowed to change before the merge to trunk.

I guess there is no way to convince your policy makers that if commit
A and B are different but have the same tree hash they refer to the
same state on the disk? I have had audit conversations like that.

Anyway, sorry my reply wasn't helpful. Good luck.

cheers,
Yves

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

> Anyway, sorry my reply wasn't helpful. Good luck.

Not at all, I do appreciate all the suggestions, I learned a lot from this
thread in general. I think everybody in this thread has been very helpful.

This thread has gone a little cold. I did have a think about the sequential
vs concurrent resolutions and even had a conversation with one of my colleagues
about it.

Would it be reasonable if anyone could push a partial resolution but the book
stops there (once a user hits a conflict of a conflict is must be solved
locally)? I agree it doesn't make sense in most cases to support pushing
recursive conflict resolutions (even though the other part of me says if the
users wants to go down that path why stop them? You could even have a config
setting to allow N levels of conflicts to be pushed, the default setting being
exactly the way things are, none or 0!).

I know in my project we already "fake" this functionality like pointed out in
the first email, it's just unclean the way we do it, leaves broken commits
in the repo, you can no longer use difftools, etc.

Should I even consider this as a research idea for my thesis? Or another way
of wording this is, if someone sent the code to the git maintainers Junio, etc.
would it be merged into git?

Regards,

Eric Curtin

Software Engineer

Ovens Campus,
Cork,
Ireland

Dell EMC

Re: Collaborative conflict resolution feature request

[Christian Couder]

On Fri, Jun 19, 2020 at 11:17 AM Curtin, Eric <Eric.Curtin@dell.com> wrote:

> Would it be reasonable if anyone could push a partial resolution but the book
> stops there (once a user hits a conflict of a conflict is must be solved
> locally)? I agree it doesn't make sense in most cases to support pushing
> recursive conflict resolutions (even though the other part of me says if the
> users wants to go down that path why stop them? You could even have a config
> setting to allow N levels of conflicts to be pushed, the default setting being
> exactly the way things are, none or 0!).
>
> I know in my project we already "fake" this functionality like pointed out in
> the first email, it's just unclean the way we do it, leaves broken commits
> in the repo, you can no longer use difftools, etc.
>
> Should I even consider this as a research idea for my thesis? Or another way
> of wording this is, if someone sent the code to the git maintainers Junio, etc.
> would it be merged into git?

I think it could be an interesting feature to add to Git, as I agree
that some people often have this kind of issues with conflicts.

What could perhaps work is to develop a new command, maybe called `git
conflict` with subcommands for example to load, save and maybe push,
fetch and resolve partial resolutions of conflicts. The conflicts
could perhaps be stored as commits in the "refs/conflicts/" ref
namespace.

Re: Collaborative conflict resolution feature request

[Curtin, Eric]

> I think it could be an interesting feature to add to Git, as I agree
> that some people often have this kind of issues with conflicts.
>
> What could perhaps work is to develop a new command, maybe called `git
> conflict` with subcommands for example to load, save and maybe push,
> fetch and resolve partial resolutions of conflicts. The conflicts
> could perhaps be stored as commits in the "refs/conflicts/" ref
> namespace.

Yes and actually the more I read about things like `git rerere`, most of the
components are already there in git. What this concept kind of is, is a
pushable git rerere cache. A workflow could be something like:

git merge/rebase # fails with conflicts, maybe a prompt that to tell the user that some conflict resolutions have already been pushed
git rerere pop/apply # apply the already pushed conflicts
# fix your conflict with whatever technique or tool you prefer
git rerere create # commit your rerere (stash uses the term create) writing a message explaining why you resolved this conflict in a certain way
git push # or maybe `git rerere push` so the next user can "git fetch/git merge/etc." and work on the conflict resolution

And maybe add some other git stash like commands like show/list.
So other users can see who solved what conflict and why that decision was
made. I'm using terms like pop/apply/create just to be consistent with stash
also. The reason I make that a manual step and not an automatic step like
git rerere tends to do, is that existing users of git won't be aware of these
automated conflict resolutions and I'm sure many people won't like these
being applied automatically by default.

Admittedly I don't use git stash for temporary local changes, I just use
commits and branches, amend them, delete them etc. as I see fit. But I
think the concept is very similar, these are not complete buildable commits.

But I think you're right Chris if we start from scratch and call it `git conflict`
(although I was thinking of maybe `git resolve` or `git re`, since rerere already
abbreviates resolution to just re) we don't have to worry about backwards
compatibility with existing commands designed for a different purpose
like rerere. I'm not too worried about the naming as long as the workflow
is easy to use.

I might fly this by my lecturers as a research project. What do you think Chris
and Junio? I'd be happy to send you patches of the work in progress as it's
going along if you guys would be happy to see them?

Regards,

Eric Curtin

Software Engineer
Ovens Campus,
Cork,
Ireland

Dell EMC