git merge and cherry-pick and duplicated commits?

git merge and cherry-pick and duplicated commits?

[skillzero]

I created a branch from master, did a commit (8e9fdd), then did 2 more
commits (11c59c and 7024d), then did another commit (2daf23). From
master, I did a commit (47bd1b) then cherry-pick'd 2 commits from the
branch (11c59c and 7024d). When merged the branch into master, I see
the 2 cherry-picked commits twice in the log (once from the original
cherry-pick's and again from the merge).

I thought git would realize that master already had those 2 commits
and not add them again when merging?

Re: git merge and cherry-pick and duplicated commits?

[Brian Gernhardt]

On Jan 13, 2009, at 9:40 PM, skillzero@gmail.com wrote:

> I created a branch from master, did a commit (8e9fdd), then did 2 more
> commits (11c59c and 7024d), then did another commit (2daf23). From
> master, I did a commit (47bd1b) then cherry-pick'd 2 commits from the
> branch (11c59c and 7024d). When merged the branch into master, I see
> the 2 cherry-picked commits twice in the log (once from the original
> cherry-pick's and again from the merge).

Before the cherry-picks, your repository looks like this

o-o (master: 47bd1b)
  \
   o-A-B-o (branch:2daf23)

A and B are the two commits you cherry-picked (11c59c and 7024d)

After the cherry-picks, the repo looks like this:

o-o-A'-B' (master)
  \
   o-A-B-o (branch:2daf23)

A and A' are different commits.  Same with B and B'.  If you check the  
SHA1 of master at this point, it will NOT be 702fd... (B).  Cherry  
pick creates a new commit that (as far as git is concerned) is totally  
unrelated.

After the merge, you get:

o-o-A'-B'-o (master)
  \       /
   o-A-B-o

Since git has no knowledge that the cherry-picked (A' B') commits are  
related to their originals (A B), it displays both to you.  If you  
want, you can use the -x flag when you use "git cherry-pick" to add a  
line that describes the original source of the patch in the new commit  
which eases confusion when you look at the history, but will not stop  
them from being displayed.

(The reason git will still display them is that the cherry-picked  
commits may be different if there were conflicting changes on the  
branches.  Also, hiding those commits would give a false view of  
history since the changes were actually added to the repository  
twice.  Using gitk or "git log --graph" will show the commits on two  
different lines of development.)

> I thought git would realize that master already had those 2 commits
> and not add them again when merging?

The simplest method is to rebase branch after doing the cherry-picks.   
This should only be done if your branch has not been published.  From  
after the cherry-picks:

o-o-A'-B' (master)
  \
   o-A-B-o (branch:2daf23)

"git rebase master branch" would give you

o-o-A'-B' (master)
         \
          o'-o' (branch)

Git should detect that the changes from A and B were already present  
in master during the rebase and skip the commits.

~~ Brian Gernhardt

Re: git merge and cherry-pick and duplicated commits?

[skillzero]

On Tue, Jan 13, 2009 at 9:31 PM, Brian Gernhardt
<benji@silverinsanity.com> wrote:

> After the cherry-picks, the repo looks like this:
>
> o-o-A'-B' (master)
>  \
>  o-A-B-o (branch:2daf23)
>
> A and A' are different commits.  Same with B and B'.  If you check the SHA1
> of master at this point, it will NOT be 702fd... (B).  Cherry pick creates a
> new commit that (as far as git is concerned) is totally unrelated.

That's what I was somewhat disappointed by. Even though the result of
the commit had a different hash, I assumed git would keep some kind of
internal per-commit hash so it could tell later that two commits were
the same and not re-apply them.

> The simplest method is to rebase branch after doing the cherry-picks.  This
> should only be done if your branch has not been published.

The problem is, by the time I wanted to do the cherry-pick, I had
already committed other stuff to the branch. I tried doing 'git rebase
master branch' when on master and it just applied all the stuff from
master to branch.

Is there any way to apply a commit to 2 different branches (which have
diverged) in a way that git will remember so that when the 2 branches
merge later, it won't result in duplicate commits? I find that I often
make changes that days or weeks later find out that some other branch
needs that change and by then, there have been lots of commits to both
branches after the commit I want.

Re: git merge and cherry-pick and duplicated commits?

[skillzero]

I guess maybe a better question is how do people normally handle
situations like mine where I did some work on branch X and I later
realize I need only a portion of that work on branch Y? I'm not sure
how I can change my workflow to completely eliminate these situations.
For example, I often start a branch to add a new feature and I end up
fixing bug A on that branch. Then other people on my team decide they
need the fix for bug A immediately and can't wait for me to finish my
feature branch and do a full merge.

Is there some way I can change my workflow such that I can fix bug A
(maybe on a separate branch?) and somehow apply it to both both
branches in a way that won't result in duplicate commits?

Does this kind of thing ever happen with the Linux kernel or git
itself: somebody does a fix as part of their topic branch and the
Linux kernel or git master wants that particular fix now, but is not
ready for the full topic branch? Would they just suggest the fix be
separated into its own topic branch and that merged? If so, how would
that new topic branch merge into the original topic branch without
resulting in a duplicate commit when it's later merged into master?

Re: git merge and cherry-pick and duplicated commits?

[Johannes Sixt]

skillzero@gmail.com schrieb:
> Is there any way to apply a commit to 2 different branches (which have
> diverged) in a way that git will remember so that when the 2 branches
> merge later, it won't result in duplicate commits? I find that I often
> make changes that days or weeks later find out that some other branch
> needs that change and by then, there have been lots of commits to both
> branches after the commit I want.

Well, the way to do it is "careful planning".

If you have a *slight* suspicion that some change *might* be needed on a
different branch, then:

1. you commit the change on a branch of its own that forks off of the
merge-base of *all* the branches that *might* need it;

2. next, you merge this fix-up branch into the branch where you need it
first, which is very likely your current topic-under-development.

3. Later you can merge the branch into the other branches if you find that
it is really needed.

If you don't have the slight suspicion, then you have to take the
second-best route, namely to cherry-pick the commit onto a branch just
like in 1. above, and continue with 2. and 3. In this case you have the
commit twice, but not more than that.

-- Hannes

Re: git merge and cherry-pick and duplicated commits?

[skillzero]

On Tue, Jan 13, 2009 at 11:34 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:

> Well, the way to do it is "careful planning".
>
> If you have a *slight* suspicion that some change *might* be needed on a
> different branch, then:
>
> 1. you commit the change on a branch of its own that forks off of the
> merge-base of *all* the branches that *might* need it;
>
> 2. next, you merge this fix-up branch into the branch where you need it
> first, which is very likely your current topic-under-development.
>
> 3. Later you can merge the branch into the other branches if you find that
> it is really needed.

If I create a separate bug-fix-only branch X that forks from the
latest common commit of all the branches that might need it and some
of those branches already have commits after that merge base (e.g.
branch Z is 5 commits after the common merge base by the time I fix
the bug), will git be able to merge the new branch X into Z in a way
that will allow me to also merge branch X into my original feature
branch A and then later merge A into Z without duplicating the commit
that is now in both branch X and Z?

It seems like I'd run into my original duplicate commit problem
because even though branch X was originally based off the same parent
commit, it will have a different parent when it is merged into Z
because Z is no longer at that common merge commit (it's 5 commits
beyond it).

Re: git merge and cherry-pick and duplicated commits?

[Nanako Shiraishi]

Quoting skillzero@gmail.com:

> The problem is, by the time I wanted to do the cherry-pick, I had
> already committed other stuff to the branch. I tried doing 'git rebase
> master branch' when on master and it just applied all the stuff from
> master to branch.
>
> Is there any way to apply a commit to 2 different branches (which have
> diverged) in a way that git will remember so that when the 2 branches
> merge later, it won't result in duplicate commits? I find that I often
> make changes that days or weeks later find out that some other branch
> needs that change and by then, there have been lots of commits to both
> branches after the commit I want.

Johennes Sixt gave a good answer. You need to think before making your commits while you are developing to separate what change belongs to common code base and what change belongs to a specific feature.

I was reading gitster's blog (he has several "git tutorial" articles recently) and he talks about how to commit a change to a branch different from what you originally developed it on in today's entry. You may find it instructive, but the procedure is after you think about it and decide what change goes where.

-- 
Nanako Shiraishi
http://ivory.ap.teacup.com/nanako3/

Re: git merge and cherry-pick and duplicated commits?

[Johannes Sixt]

[Please reply-to-all on this list, to keep Cc: list]

skillzero@gmail.com schrieb:
> On Tue, Jan 13, 2009 at 11:34 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> 
>> Well, the way to do it is "careful planning".
>>
>> If you have a *slight* suspicion that some change *might* be needed on a
>> different branch, then:
>>
>> 1. you commit the change on a branch of its own that forks off of the
>> merge-base of *all* the branches that *might* need it;
>>
>> 2. next, you merge this fix-up branch into the branch where you need it
>> first, which is very likely your current topic-under-development.
>>
>> 3. Later you can merge the branch into the other branches if you find that
>> it is really needed.
> 
> If I create a separate bug-fix-only branch X that forks from the
> latest common commit of all the branches that might need it and some
> of those branches already have commits after that merge base (e.g.
> branch Z is 5 commits after the common merge base by the time I fix
> the bug), will git be able to merge the new branch X into Z in a way
> that will allow me to also merge branch X into my original feature
> branch A and then later merge A into Z without duplicating the commit
> that is now in both branch X and Z?
> 
> It seems like I'd run into my original duplicate commit problem
> because even though branch X was originally based off the same parent
> commit, it will have a different parent when it is merged into Z
> because Z is no longer at that common merge commit (it's 5 commits
> beyond it).

After you created the fixup, you have this situation:

    o--o--o   <- A (feature branch)
   /
--o--x        <- X (the fix-up branch)
   \
    o--o--o   <- Z (probably your master)

You merge the fix-up into the feature branch and continue developing the
feature:

    o--o--o--M--o--o   <- A
   /        /
--o--x-----'           <- X
   \
    o--o--o            <- Z

Other people need the fix in Z right now, so you merge it into Z as well:

    o--o--o--M--o--o   <- A
   /        /
--o--x-----<           <- X
   \        \
    o--o--o--N         <- Z

You complete your feature and merge it into Z:

    o--o--o--M--o--o     <- A
   /        /       \
--o--x-----<         \   <- X
   \        \         \
    o--o--o--N---------O <- Z

The fix-up commit is only once in your history.

-- Hannes

Re: git merge and cherry-pick and duplicated commits?

[Boaz Harrosh]

skillzero@gmail.com wrote:
> On Tue, Jan 13, 2009 at 11:34 PM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> 
>> Well, the way to do it is "careful planning".
>>
>> If you have a *slight* suspicion that some change *might* be needed on a
>> different branch, then:
>>
>> 1. you commit the change on a branch of its own that forks off of the
>> merge-base of *all* the branches that *might* need it;
>>
>> 2. next, you merge this fix-up branch into the branch where you need it
>> first, which is very likely your current topic-under-development.
>>
>> 3. Later you can merge the branch into the other branches if you find that
>> it is really needed.
> 
> If I create a separate bug-fix-only branch X that forks from the
> latest common commit of all the branches that might need it and some
> of those branches already have commits after that merge base (e.g.
> branch Z is 5 commits after the common merge base by the time I fix
> the bug), will git be able to merge the new branch X into Z in a way
> that will allow me to also merge branch X into my original feature
> branch A and then later merge A into Z without duplicating the commit
> that is now in both branch X and Z?
> 
> It seems like I'd run into my original duplicate commit problem
> because even though branch X was originally based off the same parent
> commit, it will have a different parent when it is merged into Z
> because Z is no longer at that common merge commit (it's 5 commits
> beyond it).
> --

No, if you use merges it will not duplicate. It will know exactly what
to do because it is the same commit in all branches.
Only git-cherry-pick will duplicate the same patch, but as a different
new commit. Then when merging the merge sees a merge conflict but since
it is exactly the same change it will accept it. The same happens if
two different patches have exact same hunk, the merge is smart to accept
the same change from two sources. What happen with cherry-pick is that all
the hunks match.

Boaz

Re: git merge and cherry-pick and duplicated commits?

[Thomas Rast]

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]

skillzero@gmail.com wrote:
> I thought git would realize that master already had those 2 commits
> and not add them again when merging?

[later]
> That's what I was somewhat disappointed by. Even though the result of
> the commit had a different hash, I assumed git would keep some kind of
> internal per-commit hash so it could tell later that two commits were
> the same and not re-apply them.

I think there's an important misunderstanding here: merging A into B
does *not* have anything to do with commits, or history for that
matter, beyond the differences from $(git merge-base A B) to A and
B.[*]

Along the same lines, nothing is ever re-applied during merging.
git-merge just figures out that you made the same change on both
sides, so it must have been a good change, so it must go into the end
result.  *How* you arrived at the same change---say, by
cherry-picking, or by getting the same result in that region from
otherwise different commits, or even from several commits---does *not*
matter in any way.

You can use 'git cherry', 'git log --left-right --cherry-pick', and
similar tools to find commits that are cherry-picked "duplicates", but
unless you rewrite history, they are there to stay.


[*] This is a simplification since as soon as the merge-base is not
unique, merge-recursive will actually start looking into history
further back.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch



[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: git merge and cherry-pick and duplicated commits?

[Alex Riesen]

2009/1/14 Thomas Rast <trast@student.ethz.ch>:
> skillzero@gmail.com wrote:
>> That's what I was somewhat disappointed by. Even though the result of
>> the commit had a different hash, I assumed git would keep some kind of
>> internal per-commit hash so it could tell later that two commits were
>> the same and not re-apply them.
>
> I think there's an important misunderstanding here: merging A into B
> does *not* have anything to do with commits, or history for that
> matter, beyond the differences from $(git merge-base A B) to A and
> B.[*]
>
> Along the same lines, nothing is ever re-applied during merging.
> git-merge just figures out that you made the same change on both
> sides, so it must have been a good change, so it must go into the end
> result.  *How* you arrived at the same change---say, by
> cherry-picking, or by getting the same result in that region from
> otherwise different commits, or even from several commits---does *not*
> matter in any way.

Yes, merge only considers what bytes (aka contents of
trees-directories and blobs-files) do the branches to be merged
have, compares them (by comparing their hashes) and if there
are differences tries to mix them together according to the merge
rules described somewhere in Documentation.

So this all is really just about what the branches contain, not
how they got it. It is the conflict resolution algorithm which uses
the history to find the best possible source blob or tree which was
changed by conflicting branches so the "mix" can be prepared as
close as possible to what would we do if we went looking for the
pieces manually.

Re: git merge and cherry-pick and duplicated commits?

[Sitaram Chamarty]

On 2009-01-14, skillzero@gmail.com <skillzero@gmail.com> wrote:
> I guess maybe a better question is how do people normally handle
> situations like mine where I did some work on branch X and I later
> realize I need only a portion of that work on branch Y? I'm not sure
> how I can change my workflow to completely eliminate these situations.
> For example, I often start a branch to add a new feature and I end up
> fixing bug A on that branch. Then other people on my team decide they
> need the fix for bug A immediately and can't wait for me to finish my
> feature branch and do a full merge.
>
> Is there some way I can change my workflow such that I can fix bug A
> (maybe on a separate branch?) and somehow apply it to both both
> branches in a way that won't result in duplicate commits?
>
> Does this kind of thing ever happen with the Linux kernel or git
> itself: somebody does a fix as part of their topic branch and the
> Linux kernel or git master wants that particular fix now, but is not
> ready for the full topic branch? Would they just suggest the fix be
> separated into its own topic branch and that merged? If so, how would
> that new topic branch merge into the original topic branch without
> resulting in a duplicate commit when it's later merged into master?

If I understand you right, you're talking about a situation
where a particular change is in two different branches, but
the SHA is different.

And yet, in your scenario, both of these branches somehow
get merged into the final branch.

In other words, a DAG of branches merging itself has a
merge, if that makes sense :-)

You may need to think about how commits flow from branch to
branch, and what branches are private, and therefore can be
rebased or change history, and what are public, and how
often and when the private ones rebase off of the public
ones.

Re: git merge and cherry-pick and duplicated commits?

[skillzero]

On Wed, Jan 14, 2009 at 12:34 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:

> After you created the fixup, you have this situation:
>
>    o--o--o   <- A (feature branch)
>   /
> --o--x        <- X (the fix-up branch)
>   \
>    o--o--o   <- Z (probably your master)
>
> You merge the fix-up into the feature branch and continue developing the
> feature:
>
>    o--o--o--M--o--o   <- A
>   /        /
> --o--x-----'           <- X
>   \
>    o--o--o            <- Z
>
> Other people need the fix in Z right now, so you merge it into Z as well:
>
>    o--o--o--M--o--o   <- A
>   /        /
> --o--x-----<           <- X
>   \        \
>    o--o--o--N         <- Z
>
> You complete your feature and merge it into Z:
>
>    o--o--o--M--o--o     <- A
>   /        /       \
> --o--x-----<         \   <- X
>   \        \         \
>    o--o--o--N---------O <- Z
>
> The fix-up commit is only once in your history.

Thanks for the info. That's what I was hoping, but I was thinking that
I'd get duplicate commits if I did that. I'll have to try it out when
I run into this situation again.

Related to this, is there a way to easily find the common merge base
given a bunch of a branches? When I want to fix a bug, I want to say
"Given branches A, B, C, D, and E, where should I fork my bug fix
branch from so that I can merge this branch into all those branches
without getting duplicate commits?".

Re: git merge and cherry-pick and duplicated commits?

[Peter Baumann]

On Wed, Jan 14, 2009 at 10:33:02AM -0800, skillzero@gmail.com wrote:
> On Wed, Jan 14, 2009 at 12:34 AM, Johannes Sixt <j.sixt@viscovery.net>
> wrote:
> 
[ ... ]
  
> Related to this, is there a way to easily find the common merge base
> given a bunch of a branches? When I want to fix a bug, I want to say
> "Given branches A, B, C, D, and E, where should I fork my bug fix
> branch from so that I can merge this branch into all those branches
> without getting duplicate commits?".

You should have a look at 'git help merge-base'

-Peter

Re: git merge and cherry-pick and duplicated commits?

[Junio C Hamano]

skillzero@gmail.com writes:

> Related to this, is there a way to easily find the common merge base
> given a bunch of a branches? When I want to fix a bug, I want to say
> "Given branches A, B, C, D, and E, where should I fork my bug fix
> branch from so that I can merge this branch into all those branches
> without getting duplicate commits?".

You do not necessarily have to fork from, nor merge into, any of them.

If you fixed a bug, you would hopefully know where the bug was injected at
into your history.  You may have bisected it down to one commit $BAD.  You
can fork your fix on top of that $BAD commit:

	$ git checkout -b fix-bug-foo $BAD

All of the branches that share the commit have the bug, so your fix could
be merged to all of them if you really wanted to, and you should do so if
these A...E branches are meant to be consumed on their own.

But if the branches A...E you are about are for developing independent
topics, and if their theme won't get affected by the bug, it is much
better not to merge the fix in.  You will have the merge for the fix in
your integration branch anyway.  It is preferable not to contaminate an
independent topic branch whose purpose is to cook its own theme with an
unrelated bugfix, even if it is brought in as a merge.

Re: git merge and cherry-pick and duplicated commits?

[Markus Heidelberg]

skillzero@gmail.com, 14.01.2009:
> On Wed, Jan 14, 2009 at 12:34 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> 
> > After you created the fixup, you have this situation:
> >
> >    o--o--o   <- A (feature branch)
> >   /
> > --o--x        <- X (the fix-up branch)
> >   \
> >    o--o--o   <- Z (probably your master)
> >
> > You merge the fix-up into the feature branch and continue developing the
> > feature:
> >
> >    o--o--o--M--o--o   <- A
> >   /        /
> > --o--x-----'           <- X
> >   \
> >    o--o--o            <- Z
> >
> > Other people need the fix in Z right now, so you merge it into Z as well:
> >
> >    o--o--o--M--o--o   <- A
> >   /        /
> > --o--x-----<           <- X
> >   \        \
> >    o--o--o--N         <- Z
> >
> > You complete your feature and merge it into Z:
> >
> >    o--o--o--M--o--o     <- A
> >   /        /       \
> > --o--x-----<         \   <- X
> >   \        \         \
> >    o--o--o--N---------O <- Z
> >
> > The fix-up commit is only once in your history.
> 
> Thanks for the info. That's what I was hoping, but I was thinking that
> I'd get duplicate commits if I did that. I'll have to try it out when
> I run into this situation again.

Note, that you'd get 2 merge commits for this fix-up commit into branch Z.
The first from merging X into Z, the second is created from merging X
into A and occures in Z when merging A into it.

Markus