git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
* Fast-forward able commit, otherwise fail
@ 2016-06-21  3:58 David Lightle
  2016-06-21  4:51 ` Junio C Hamano
  0 siblings, 1 reply; 9+ messages in thread
From: David Lightle @ 2016-06-21  3:58 UTC (permalink / raw)
  To: git

Hello,

I am trying to build a git workflow in our enterprise and am almost
evenly torn between a "rebase workflow" and git-flow (which uses merge
instead of rebase, obviously).  We are using Bitbucket for pull
requests as code reviews (right or wrong).

I apologize if my inexperience with git shows through, but I'm wanting
to achieve the following --

Maintain two long-running branches (develop as a stable pre-release,
master as a deployed version), with short-term feature branches off of
develop and short-term release branches that are based on develop and
merge into master eventually (ie. a lightweight git-flow).

However, as part of this, I would prefer to require/ensure that each
feature branch is up-to-date and otherwise able to be fast-forwarded;
we currently have this with the bitbucket server setting requiring
--ff-only.

The other half of what I would prefer is to still perform the merge
commit, though, to ensure separate tracking of the feature branches
(ie. --no-ff).

I know that I have read that --ff-only and --no-ff are contradictory,
but I believe that is somewhat ambiguously correct -- I believe the
two flags contradict in the sense of "what to do with the merge
commit", but not necessarily on the "when it can be done".

I read an ancient post about on this list (which predates the
tri-state of fast-forward) that I believe could have allowed this to
work.

I have also read a few articles and posts that achieve this more as a
matter of practice than a workflow enforcement via git flags; for this
to be something to potentially get absorbed into a Bitbucket workflow,
I suspect it would need to a git flag (they now support merging via
--ff, --no-ff, --ff-only, --squash, and --squash-ff-only for their
hosted solution, for example).

Here are a few articles that say they prefer this approach (rebase and
then merge --no-ff):
https://walkingthestack.blogspot.com/2012/05/why-you-should-use-git-merge-no-ff-when.html
http://blog.differential.com/best-way-to-merge-a-github-pull-request/
http://victorlin.me/tags/rebase

Can anyone share their thoughts on workflows that might achieve what
I'm looking at here or opinions on adding the functionality I'm
describing?

Thanks!

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-06-21  3:58 Fast-forward able commit, otherwise fail David Lightle
@ 2016-06-21  4:51 ` Junio C Hamano
  2016-06-22 23:09   ` Junio C Hamano
  0 siblings, 1 reply; 9+ messages in thread
From: Junio C Hamano @ 2016-06-21  4:51 UTC (permalink / raw)
  To: David Lightle; +Cc: git

David Lightle <dlightle@gmail.com> writes:

> I know that I have read that --ff-only and --no-ff are contradictory,
> but I believe that is somewhat ambiguously correct -- I believe the
> two flags contradict in the sense of "what to do with the merge
> commit", but not necessarily on the "when it can be done".

Traditionally we have mildly suggested against making otherwise
needless merge commit with "--no-ff".  We also have suggested
against keeping the history too linear, not because linear history
is bad, but because it will require rebasing soon before pushing out
your changes [*1*].

I personally would feel that what you seem to be aiming for is the
worst of both worlds in the sense that contributors are still forced
to rebase immediately before pushing out, which leads to
insufficiently tested version landing on truck, and at the same time
forcing the resulting history to have otherwise needless merges (the
latter is probably a much lessor of the two sins).

However, Git as a tool is not opinionated strongly enough to make it
hard to do these two things (independently).  I do not think it is
unreasonable to add a new mode of "merge" that rejects a resulting
history that is not shaped the way you like.  So far the command
rejected --ff-only and --no-ff given together, so if an updated Git
starts taking them together and creating a needless real merge and
failing only when the first parent does not fast-forward to the
second parent, nobody's existing workflow would be broken.

Having said that, you need to think things through.  Sample
questions you would want to be asking yourself are (not exhaustive):

 - What is your plan to _enforce_ your project participants to use
   this new mode of operation?

 - Do you _require_ your project participants to always pass a new
   option to "git merge" or "git pull"?

 - Do you force them to set some new configuration variables?  

 - Do you trust them once you tell them what to do?  

 - How will your project's trunk get their changes?

 - How you prevent some participants who misunderstood your
   instructions from pushing an incorrectly shaped history to your
   project?

If they are eventually pushing the result to a central repository
and that is how the project's overall history advances, then the
most reliable mechanism that is least limiting to your users is
pre-receive hook at that central repository that ensures the shape
of the history being pushed to update the branches.  You walk the
first-parent chain of the commits and ensure all the commits are
merges, and for each of these merges, its second parent is a
descendant of its first parent.  Otherwise you reject their push.

That way, your users can make normal merges while trying their
changes locally all they want without straightjacket.  Only when
they prepare the final history to be placed at the tip of the
project's official history, they need to make sure that the history
is shaped as you specified.


[Footnote]

*1* Your changes before rebasing may have seen enough
testing in one context (i.e. built on older base), but "rebase
immediately before push to ensure fast-forward" will force you to
publish a version that by definition will have little or no testing,
which may or may not work well with a different context.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-06-21  4:51 ` Junio C Hamano
@ 2016-06-22 23:09   ` Junio C Hamano
  2016-07-08 19:28     ` David Lightle
  2016-07-12 13:24     ` Stefan Haller
  0 siblings, 2 replies; 9+ messages in thread
From: Junio C Hamano @ 2016-06-22 23:09 UTC (permalink / raw)
  To: David Lightle; +Cc: git

Junio C Hamano <gitster@pobox.com> writes:

> However, Git as a tool is not opinionated strongly enough to make it
> hard to do these two things (independently).  I do not think it is
> unreasonable to add a new mode of "merge" that rejects a resulting
> history that is not shaped the way you like.  So far the command
> rejected --ff-only and --no-ff given together, so if an updated Git
> starts taking them together and creating a needless real merge and
> failing only when the first parent does not fast-forward to the
> second parent, nobody's existing workflow would be broken.
>
> Having said that, you need to think things through.  Sample
> questions you would want to be asking yourself are (not exhaustive):
>
>  - What is your plan to _enforce_ your project participants to use
>    this new mode of operation?
>
>  - Do you _require_ your project participants to always pass a new
>    option to "git merge" or "git pull"?
>
>  - Do you force them to set some new configuration variables?  
>
>  - Do you trust them once you tell them what to do?  
>
>  - How will your project's trunk get their changes?
>
>  - How you prevent some participants who misunderstood your
>    instructions from pushing an incorrectly shaped history to your
>    project?

Another thing to consider is that the proposed workflow would not
scale if your team becomes larger.  Requiring each and every commit
on the trunk to be a merge commit, whose second parent (i.e. the tip
of the feature branch) fast-forwards to the first parent of the
merge (i.e. you require the feature to be up-to-date), would mean
that Alice and Bob collaborating on this project would end up
working like this:

 A:    git pull --ff-only origin ;# starts working
 A:    git checkout -b topic-a
 A:    git commit; git commit; git commit
 B:    git pull --ff-only origin ;# starts working
 B:    git checkout -b topic-b
 B:    git commit; git commit

 A:    git checkout master && git merge --ff-only --no-ff topic-a
 A:    git push origin ;# happy

 B:    git checkout master && git merge --ff-only --no-ff topic-b
 B:    git push origin ;# fails!

 B:    git fetch origin ;# starts recovering
 B:    git reset --hard origin/master
 B:    git merge --ff-only --no-ff topic-b ;# fails!
 B:    git rebase origin/master topic-b
 B:    git checkout master && git merge --ff-only --no-ff topic-b
 B:    git push origin ;# hopefully nobody pushed in the meantime

The first push by Bob fails because his 'master', even though it is
a merge between the once-at-the-tip-of-public-master and topic-b
which was forked from that once-at-the-tip, it no longer fast-forwards
because Alice pushed her changes to the upstream.

And it is not sufficient to redo the previous merge after fetching
the updated upstream, because your additional "feature branch must
be up-to-date" requirement is now broken for topic-b.  Bob needs to
rebuild it on top of the latest, which includes what Alice pushed,
using rebase, do that merge again, and hope that nobody else pushed
to update the upstream in the meantime.  As you have more people
working simultanously on more features, Bob will have to spend more
time doing steps between "starts recovering" and "hopefully nobody
pushed in the meantime", because the probability is higher that
somebody other than Alice has pushed while Bob is trying to recover.

The time spend on recovery is not particularly productive, and this
workflow gives him a (wrong) incentive to do that recovery procedure
quickly to beat the other participants to become the first to push.

The workflow should instead be designed to incentivise participant
to spend more time to carefully inspect the result of his rebasing
to make sure that his changes still make sense in the context of the
updated codebase that contains changes from others since he forked
topic-b from the upstream, in order to ensure quality.

This "push quickly immediately after rebasing without carefully
checking the result of rebase" pressure is shared by a workflow that
requires a completely linear history, and not unique to your
proposed workflow, but because you also require a --no-ff merge to
the updated upstream, robbing even more time from the project
participants, it aggravates the problem.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-06-22 23:09   ` Junio C Hamano
@ 2016-07-08 19:28     ` David Lightle
  2016-07-08 22:19       ` Junio C Hamano
  2016-07-12 13:24     ` Stefan Haller
  1 sibling, 1 reply; 9+ messages in thread
From: David Lightle @ 2016-07-08 19:28 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: git

I've been trying to think of just what to say in response, so it has
taken me some time as well as I've been experimenting with our
workflow to try some alternate approaches while we still can.

In your above scenario with Alice & Bob, wouldn't that same issue come
up in _any_ rebase workflow (--ff-only)?  From what I've read this is
a typical consequence of requiring fast-forward merges; to be
effective, the rebase step must  be more likely to succeed in a time
window than for someone to have pushed additional changes (otherwise
you end up in a loop like above).  But I don't believe that's
inherently any worse with the change I'm asking about which  would
only take that existing flag (--ff-only) and additionally create an
actual merge commit when those conditions are present.

We were previously using the fast-forward-only approach and it worked
moderately well, but there is a learning curve to it more (rebase vs
merge).  However, because we want to follow a gitflow workflow, where
topic branches are created from develop and merged back into develop,
we want to ensure that is an actual merge commit to identify the
changes that occurred on that topic branch (with --ff-only it becomes
hard to distinguish changes from one topic branch to the next).

We will still be using develop as an integration branch which will
receive testing as well, so the rebased version will still be tested
before it gets released in our scenario.  But again that is a
rebase-workflow issue more than a rebase+merge commit issue, right?

Also GitLab already has introduced a "rebase before merging" automated
tool in their product, and BitBucket has a similar request oustanding,
and both (plus GitHub) allow squashing commits as part of the merge.
In fact, I just noticed that GitLab has built in the functionality I'm
looking for even, which is what they call "Merge commit with
semi-linear history" but I'm asking whether direct support for this
approach would be reasonable.  These approaches can all produce the
"untested merged product", but they support the way the users want to
use the system as well.  I'm not saying any approach is right or wrong
as I'm not qualified enough to say.

On Wed, Jun 22, 2016 at 6:09 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> However, Git as a tool is not opinionated strongly enough to make it
>> hard to do these two things (independently).  I do not think it is
>> unreasonable to add a new mode of "merge" that rejects a resulting
>> history that is not shaped the way you like.  So far the command
>> rejected --ff-only and --no-ff given together, so if an updated Git
>> starts taking them together and creating a needless real merge and
>> failing only when the first parent does not fast-forward to the
>> second parent, nobody's existing workflow would be broken.
>>
>> Having said that, you need to think things through.  Sample
>> questions you would want to be asking yourself are (not exhaustive):
>>
>>  - What is your plan to _enforce_ your project participants to use
>>    this new mode of operation?
>>
>>  - Do you _require_ your project participants to always pass a new
>>    option to "git merge" or "git pull"?
>>
>>  - Do you force them to set some new configuration variables?
>>
>>  - Do you trust them once you tell them what to do?
>>
>>  - How will your project's trunk get their changes?
>>
>>  - How you prevent some participants who misunderstood your
>>    instructions from pushing an incorrectly shaped history to your
>>    project?
>
> Another thing to consider is that the proposed workflow would not
> scale if your team becomes larger.  Requiring each and every commit
> on the trunk to be a merge commit, whose second parent (i.e. the tip
> of the feature branch) fast-forwards to the first parent of the
> merge (i.e. you require the feature to be up-to-date), would mean
> that Alice and Bob collaborating on this project would end up
> working like this:
>
>  A:    git pull --ff-only origin ;# starts working
>  A:    git checkout -b topic-a
>  A:    git commit; git commit; git commit
>  B:    git pull --ff-only origin ;# starts working
>  B:    git checkout -b topic-b
>  B:    git commit; git commit
>
>  A:    git checkout master && git merge --ff-only --no-ff topic-a
>  A:    git push origin ;# happy
>
>  B:    git checkout master && git merge --ff-only --no-ff topic-b
>  B:    git push origin ;# fails!
>
>  B:    git fetch origin ;# starts recovering
>  B:    git reset --hard origin/master
>  B:    git merge --ff-only --no-ff topic-b ;# fails!
>  B:    git rebase origin/master topic-b
>  B:    git checkout master && git merge --ff-only --no-ff topic-b
>  B:    git push origin ;# hopefully nobody pushed in the meantime
>
> The first push by Bob fails because his 'master', even though it is
> a merge between the once-at-the-tip-of-public-master and topic-b
> which was forked from that once-at-the-tip, it no longer fast-forwards
> because Alice pushed her changes to the upstream.
>
> And it is not sufficient to redo the previous merge after fetching
> the updated upstream, because your additional "feature branch must
> be up-to-date" requirement is now broken for topic-b.  Bob needs to
> rebuild it on top of the latest, which includes what Alice pushed,
> using rebase, do that merge again, and hope that nobody else pushed
> to update the upstream in the meantime.  As you have more people
> working simultanously on more features, Bob will have to spend more
> time doing steps between "starts recovering" and "hopefully nobody
> pushed in the meantime", because the probability is higher that
> somebody other than Alice has pushed while Bob is trying to recover.
>
> The time spend on recovery is not particularly productive, and this
> workflow gives him a (wrong) incentive to do that recovery procedure
> quickly to beat the other participants to become the first to push.
>
> The workflow should instead be designed to incentivise participant
> to spend more time to carefully inspect the result of his rebasing
> to make sure that his changes still make sense in the context of the
> updated codebase that contains changes from others since he forked
> topic-b from the upstream, in order to ensure quality.
>
> This "push quickly immediately after rebasing without carefully
> checking the result of rebase" pressure is shared by a workflow that
> requires a completely linear history, and not unique to your
> proposed workflow, but because you also require a --no-ff merge to
> the updated upstream, robbing even more time from the project
> participants, it aggravates the problem.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-07-08 19:28     ` David Lightle
@ 2016-07-08 22:19       ` Junio C Hamano
  2016-07-09  2:10         ` Jacob Keller
  0 siblings, 1 reply; 9+ messages in thread
From: Junio C Hamano @ 2016-07-08 22:19 UTC (permalink / raw)
  To: David Lightle; +Cc: git

David Lightle <dlightle@gmail.com> writes:

> In your above scenario with Alice & Bob, wouldn't that same issue come
> up in _any_ rebase workflow (--ff-only)?  From what I've read this is
> a typical consequence of requiring fast-forward merges; to be
> effective, the rebase step must  be more likely to succeed in a time
> window than for someone to have pushed additional changes (otherwise
> you end up in a loop like above).  But I don't believe that's
> inherently any worse with the change I'm asking about which  would
> only take that existing flag (--ff-only) and additionally create an
> actual merge commit when those conditions are present.

True, but that does not change the fact that you would be adding
_more_ code to support a workflow that we know to be bad.

> In fact, I just noticed that GitLab has built in the functionality I'm
> looking for even, which is what they call "Merge commit with
> semi-linear history" but I'm asking whether direct support for this
> approach would be reasonable.  These approaches can all produce the
> "untested merged product", but they support the way the users want to
> use the system as well.  I'm not saying any approach is right or wrong
> as I'm not qualified enough to say.

We, not being a for-profit entity, do not have a strong incentive to
add features that encourages bad workflow to the users.  Of course,
we also do not necessarily stop them if users really want to shoot
themselves in the foot ;-), but such a change would inevitably be of
lower priority to us.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-07-08 22:19       ` Junio C Hamano
@ 2016-07-09  2:10         ` Jacob Keller
  0 siblings, 0 replies; 9+ messages in thread
From: Jacob Keller @ 2016-07-09  2:10 UTC (permalink / raw)
  To: Junio C Hamano; +Cc: David Lightle, Git mailing list

On Fri, Jul 8, 2016 at 3:19 PM, Junio C Hamano <gitster@pobox.com> wrote:
> David Lightle <dlightle@gmail.com> writes:
>> In fact, I just noticed that GitLab has built in the functionality I'm
>> looking for even, which is what they call "Merge commit with
>> semi-linear history" but I'm asking whether direct support for this
>> approach would be reasonable.  These approaches can all produce the
>> "untested merged product", but they support the way the users want to
>> use the system as well.  I'm not saying any approach is right or wrong
>> as I'm not qualified enough to say.
>
> We, not being a for-profit entity, do not have a strong incentive to
> add features that encourages bad workflow to the users.  Of course,
> we also do not necessarily stop them if users really want to shoot
> themselves in the foot ;-), but such a change would inevitably be of
> lower priority to us.
>

I would add that after using a linear history at $dayjob a lot, i've
found the biggest reasoning for a linear history is that non-linear
history confuses the users. I'd rather advocate for learning how to
handle the more complex history than trying to force users into
quickly rebasing code which can lead to the issues Junio mentioned
above.

Thanks,
Jake

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-06-22 23:09   ` Junio C Hamano
  2016-07-08 19:28     ` David Lightle
@ 2016-07-12 13:24     ` Stefan Haller
  2016-07-12 15:49       ` Junio C Hamano
  2016-07-20 16:47       ` Philip Oakley
  1 sibling, 2 replies; 9+ messages in thread
From: Stefan Haller @ 2016-07-12 13:24 UTC (permalink / raw)
  To: Junio C Hamano, David Lightle; +Cc: git

Junio C Hamano <gitster@pobox.com> wrote:

> Another thing to consider is that the proposed workflow would not
> scale if your team becomes larger.  Requiring each and every commit
> on the trunk to be a merge commit, whose second parent (i.e. the tip
> of the feature branch) fast-forwards to the first parent of the
> merge (i.e. you require the feature to be up-to-date), would mean
> that Alice and Bob collaborating on this project would end up
> working like this:
> 
>  A:    git pull --ff-only origin ;# starts working
>  A:    git checkout -b topic-a
>  A:    git commit; git commit; git commit
>  B:    git pull --ff-only origin ;# starts working
>  B:    git checkout -b topic-b
>  B:    git commit; git commit
> 
>  A:    git checkout master && git merge --ff-only --no-ff topic-a
>  A:    git push origin ;# happy
> 
>  B:    git checkout master && git merge --ff-only --no-ff topic-b
>  B:    git push origin ;# fails!
> 
>  B:    git fetch origin ;# starts recovering
>  B:    git reset --hard origin/master
>  B:    git merge --ff-only --no-ff topic-b ;# fails!
>  B:    git rebase origin/master topic-b
>  B:    git checkout master && git merge --ff-only --no-ff topic-b
>  B:    git push origin ;# hopefully nobody pushed in the meantime
> 
> The first push by Bob fails because his 'master', even though it is
> a merge between the once-at-the-tip-of-public-master and topic-b
> which was forked from that once-at-the-tip, it no longer fast-forwards
> because Alice pushed her changes to the upstream.
> 
> And it is not sufficient to redo the previous merge after fetching
> the updated upstream, because your additional "feature branch must
> be up-to-date" requirement is now broken for topic-b.  Bob needs to
> rebuild it on top of the latest, which includes what Alice pushed,
> using rebase, do that merge again, and hope that nobody else pushed
> to update the upstream in the meantime.  As you have more people
> working simultanously on more features, Bob will have to spend more
> time doing steps between "starts recovering" and "hopefully nobody
> pushed in the meantime", because the probability is higher that
> somebody other than Alice has pushed while Bob is trying to recover.
> 
> The time spend on recovery is not particularly productive, and this
> workflow gives him a (wrong) incentive to do that recovery procedure
> quickly to beat the other participants to become the first to push.

I have read and re-read this thread a hundred times now, but I still
don't understand why it makes much of a difference whether or not Bob
rebases his branch onto master before pushing his merge. In both cases,
Alice and Bob have this race as to whose push succeeds, and in both
cases you end up with merge commits on master that are not well tested.

First of all, let me say that at my company we do use the workflow that
David suggests; we rebase topic branches onto master before merging them
(with --no-ff), and we like the resulting shape of the history. Even the
more experienced git users like it for its simplicity; it simply saves
us time and mental energy when digging through the history.

Second, we did indeed run into the scalability problems that you
describe [*1*]. However, we ran into this way before starting to require
the rebase-before-merge; in my experience, rebasing or not makes no
difference here. After all, the resulting tree state of the merge commit
is identical in both cases; it's just the individual commits on the
merged topic branch that have not been tested in the rebased case. But
if the merge commit is green, it is pretty unlikely in my experience
that one of the individual commits is not. It's theoretically possible
of course, just very, very unlikely.

So what am I missing?


[Footnote]
*1* These problems were so annoying for us that we invented technical
measures to solve them. We now have a web interface where developers can
grab a lock on the repo, or put themselves into a queue for the lock
when it's taken. There's a push hook that only allows pushing when you
hold the lock. This solves it nicely, because once you have the lock,
you can take all the time you need to make sure your merge compiles, and
run the test suite locally.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-07-12 13:24     ` Stefan Haller
@ 2016-07-12 15:49       ` Junio C Hamano
  2016-07-20 16:47       ` Philip Oakley
  1 sibling, 0 replies; 9+ messages in thread
From: Junio C Hamano @ 2016-07-12 15:49 UTC (permalink / raw)
  To: Stefan Haller; +Cc: David Lightle, git

lists@haller-berlin.de (Stefan Haller) writes:

> I have read and re-read this thread a hundred times now, but I still
> don't understand why it makes much of a difference whether or not Bob
> rebases his branch onto master before pushing his merge. In both cases,
> Alice and Bob have this race as to whose push succeeds, and in both
> cases you end up with merge commits on master that are not well tested.

One crucial difference is that at least you _know_ that $tip^2 has
been well tested, when you do not rebase, and you can check out
$tip^2 to study and understand how it was supposed to work, when the
merge of the topic into an updated mainline is found to be faulty.
That knowledge helps you to go forward--you'd need to adjust some
untold assumption $tip^2 made, which was broken somewhere between
$tip^2..$tip^1 on the mainline, to an updated reality.

Once you rebase, you end up with "This series of changes once was
working well, and during the rebase somewhere it stopped working",
unless you say something like "these 7 changes were originally
developed and tested on commit X" in the rebased commit.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Fast-forward able commit, otherwise fail
  2016-07-12 13:24     ` Stefan Haller
  2016-07-12 15:49       ` Junio C Hamano
@ 2016-07-20 16:47       ` Philip Oakley
  1 sibling, 0 replies; 9+ messages in thread
From: Philip Oakley @ 2016-07-20 16:47 UTC (permalink / raw)
  To: Junio C Hamano, David Lightle, Stefan Haller; +Cc: git

From: "Stefan Haller" <lists@haller-berlin.de>
> Junio C Hamano <gitster@pobox.com> wrote:
>
>> Another thing to consider is that the proposed workflow would not
>> scale if your team becomes larger.  Requiring each and every commit
>> on the trunk to be a merge commit, whose second parent (i.e. the tip
>> of the feature branch) fast-forwards to the first parent of the
>> merge (i.e. you require the feature to be up-to-date), would mean
>> that Alice and Bob collaborating on this project would end up
>> working like this:
>>
>>  A:    git pull --ff-only origin ;# starts working
>>  A:    git checkout -b topic-a
>>  A:    git commit; git commit; git commit
>>  B:    git pull --ff-only origin ;# starts working
>>  B:    git checkout -b topic-b
>>  B:    git commit; git commit
>>
>>  A:    git checkout master && git merge --ff-only --no-ff topic-a
>>  A:    git push origin ;# happy
>>
>>  B:    git checkout master && git merge --ff-only --no-ff topic-b
>>  B:    git push origin ;# fails!
>>
>>  B:    git fetch origin ;# starts recovering
>>  B:    git reset --hard origin/master
>>  B:    git merge --ff-only --no-ff topic-b ;# fails!
>>  B:    git rebase origin/master topic-b
>>  B:    git checkout master && git merge --ff-only --no-ff topic-b
>>  B:    git push origin ;# hopefully nobody pushed in the meantime
>>
>> The first push by Bob fails because his 'master', even though it is
>> a merge between the once-at-the-tip-of-public-master and topic-b
>> which was forked from that once-at-the-tip, it no longer fast-forwards
>> because Alice pushed her changes to the upstream.
>>
>> And it is not sufficient to redo the previous merge after fetching
>> the updated upstream, because your additional "feature branch must
>> be up-to-date" requirement is now broken for topic-b.  Bob needs to
>> rebuild it on top of the latest, which includes what Alice pushed,
>> using rebase, do that merge again, and hope that nobody else pushed
>> to update the upstream in the meantime.  As you have more people
>> working simultanously on more features, Bob will have to spend more
>> time doing steps between "starts recovering" and "hopefully nobody
>> pushed in the meantime", because the probability is higher that
>> somebody other than Alice has pushed while Bob is trying to recover.
>>
>> The time spend on recovery is not particularly productive, and this
>> workflow gives him a (wrong) incentive to do that recovery procedure
>> quickly to beat the other participants to become the first to push.
>
> I have read and re-read this thread a hundred times now, but I still
> don't understand why it makes much of a difference whether or not Bob
> rebases his branch onto master before pushing his merge. In both cases,
> Alice and Bob have this race as to whose push succeeds, and in both
> cases you end up with merge commits on master that are not well tested.

I'd like to put in comment from a different perspective.

You didn't say how often you expected these merge commits to be made 
(daily/hourly/2-minutes), and also didn't mention how long it's taking to do 
the tests - 5-mins/one hour/3 hours/all day, and in particular the values 
for the difficult cases.

For example it maybe: team size means typical small feature merge commits 
are ~twice/day, needing only 30 mins test time, however there are difficult 
(big feature) commits are every 1-3 weeks, and take all day (or more) to 
test, leaving an open window for small feature merges to sneak in and break 
the 'clean-test' process.

In the above case you have a management issue, where someone needs to 
(actively) manage the process so that there are no commits to master during 
that window, and rather they are put to 'next' (using Git' branch 
convention) and once the big feature has landed the 'next-features' can be 
rebased & tested and added on top of master.

Should the case be more that there is just some occasional overlaps from the 
30mins test, during the two per day window, then that should be more 
manageable by social and procedural convention.

The thing to notice is that the upstream is always good and well formed 
(because it rejects, or should reject, commits that 
aren't --ff-only --no-ff). So before any merge (either way) [i.e. just after 
rebase & test] master should be pulled to be locally up to date.

If there were no extra commits on top, then a quick --ff-only --no-ff merge 
and push, taking only a few seconds, will complete the feature. Otherwise, 
if there were extra commits on top, then it's a moderately quick rebase & 
re-test (an annoyance but handleable because its infrequent) should mean the 
updated commit will now pass the 'pull master' check cleanly - remember the 
commit/push rate means it's unlikely that a second clash happens, otherwise 
its back to the 'management' approach.


>
> First of all, let me say that at my company we do use the workflow that
> David suggests; we rebase topic branches onto master before merging them
> (with --no-ff), and we like the resulting shape of the history. Even the
> more experienced git users like it for its simplicity; it simply saves
> us time and mental energy when digging through the history.
>
> Second, we did indeed run into the scalability problems that you
> describe [*1*]. However, we ran into this way before starting to require
> the rebase-before-merge; in my experience, rebasing or not makes no
> difference here. After all, the resulting tree state of the merge commit
> is identical in both cases; it's just the individual commits on the
> merged topic branch that have not been tested in the rebased case. But
> if the merge commit is green, it is pretty unlikely in my experience
> that one of the individual commits is not. It's theoretically possible
> of course, just very, very unlikely.
>
> So what am I missing?
>
>
> [Footnote]
> *1* These problems were so annoying for us that we invented technical
> measures to solve them. We now have a web interface where developers can
> grab a lock on the repo, or put themselves into a queue for the lock
> when it's taken. There's a push hook that only allows pushing when you
> hold the lock. This solves it nicely, because once you have the lock,
> you can take all the time you need to make sure your merge compiles, and
> run the test suite locally.
> --

Philip 


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2016-07-20 21:01 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-21  3:58 Fast-forward able commit, otherwise fail David Lightle
2016-06-21  4:51 ` Junio C Hamano
2016-06-22 23:09   ` Junio C Hamano
2016-07-08 19:28     ` David Lightle
2016-07-08 22:19       ` Junio C Hamano
2016-07-09  2:10         ` Jacob Keller
2016-07-12 13:24     ` Stefan Haller
2016-07-12 15:49       ` Junio C Hamano
2016-07-20 16:47       ` Philip Oakley

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).