[RFC] rerere to recreate topic branches

[RFC] rerere to recreate topic branches

[Uwe Kleine-König]

Hello,

Ingo sent a similar mail some time ago[1], that's why I Cc: him.

For TopGit I want something similar as Ingo for -tip (I think): recreate
a branch with a cleaned up history but without throwing away merge
resolutions.

I think most points from Ingo's mail are already addressed.  Let's see
if my mail is equally successful :-)

One step that wasn't in Ingo's list, but I consider nice is the
following workflow:

	export GIT_RR_CACHE=$(mktemp -d)
	git rerere add-to-cache-from origin..$branch_to_rebuild
	script doing the actual rebuild

together with rerere.autocommit=true.  Even though rerere is not save in
general, this workflow seems fine.  I think with this approach sharing
the rr-cache isn't an issue anymore and merge conflicts that need manual
work should be quite rare, even if you do it for the first time.

I just found contrib/rerere-train.sh, which already addresses a part of
my plan.

One of my further goals is to do as much transformation without touching
the working copy (to gain speed and comfort).  For that I would need git
merge to operate on an index, only and only fall back to using the
working copy if manual intervention is needed.

I have to admit that up to now I didn't look into the corresponding
code, so consider this mail (only) as starter to collect ideas and
comments.

Best regards
Uwe

[1] http://thread.gmane.org/gmane.comp.version-control.git/85176/

-- 
Pengutronix e.K.                              | Uwe Kleine-König            |
Industrial Linux Solutions                    | http://www.pengutronix.de/  |

Re: [RFC] rerere to recreate topic branches

[Ingo Molnar]

* Uwe Kleine-König <u.kleine-koenig@pengutronix.de> wrote:

> Hello,
> 
> Ingo sent a similar mail some time ago[1], that's why I Cc: 
> him.

> [1] http://thread.gmane.org/gmane.comp.version-control.git/85176/

Just a few generic comments - i have written that mail 8 months 
ago.

Since then our -tip workflow has shifted in the direction of 
what i'd call an "append-mostly delta integration" workflow.

When i wrote that mail we used to do daily full reintegrations 
of 100-200 topic branches using git-rerere, but that was not 
really feasible in the long run for two reasons:

 - back then a full integration took up to 30 minutes with over 
   a 100 branches and on a fast box - if it was all 
   problem-free.

   [ It's much faster now both due to the speedups Git has done,
     due to smarter scripting we do and due to me using an SSD 
     to host the -tip repo. ]

 - the full integration itself 'locked' the whole repo for a 
   long time. The old integration branch was destroyed and a new 
   one constructued - with 1000-2000 commits. This was fragile 
   in a number of ways.

 - if there was any problem during the integration (a bad 
   conflict resolution) it often had to be repeated which easily 
   pushed the time to do the integration to over an hour.

So full integration of 1000-2000 commits spread out into 100+ 
topic branches required a lot of concentration and even then it 
was too fragile.

So now we do full reintegrations only about every two weeks, 
plus we do finegrained per commit and per pull "delta 
integrations" that is based on:

 DELTA_BRANCHES=$(git branch --no-merged | cut -c3-)
 git merge $DELTA_BRANCHES

Plus we have the same finegrained topical setup that allows the 
specific isolation of gone-bad topics without the collateral 
damage of them holding the full tree and other contributors 
hostage.

So when a topic goes bad i dont revert nor do i destroy those 
commits - i just rewind/reset the integration branch to before 
the bad topic merge point, exclude the topic from 
delta-integration (add it to a blacklist) and re-run the delta 
integration and notify the people who caused the breakage. It's 
about 30 seconds to do so there's no noticeable workflow hickup.

When the fix comes in 1-2 days later, the 'bad' topic branch is 
reactivated and gets picked up by delta integration.

So all in one, delta integration is a lot less stressful in 
practice, and undo is possible and easy, as most of the 
breakages are in fresh commits.

It's also very fast:

 earth4:~/tip> time todo-merge-all 
 searching for updated branches ...
 excluded branches: core/kill-the-BKL tracing/dump-tracer
 Updated branches: tracing/ftrace
 merging tracing/ftrace ... Merge made by recursive.           
  kernel/trace/trace_workqueue.c |   14 +++++++-------
  1 files changed, 7 insertions(+), 7 deletions(-)
 merging linus ... ... merge done.                             

 real	0m5.402s
 user	0m5.002s
 sys	0m0.445s

Even though there are more than 400 branches it has to check, 
100 of which are active (==carry new commits relative to Linus's 
latest kernel tree).

	Ingo