Rebasing multiple branches at once

Rebasing multiple branches at once

[Mike Hommey]

Hi,

I've had this kind of things to do more than once, and had to do it a
lot today, so I figured it would be worth discussing whether git-rebase
should be enhanced to support this, or if this should go in a separate
tool or whatever.

So here is what I'm trying to do in a not-too painful way:

I'm starting with something like this:
A---B---C---D---E
            \---F

where A is master, and E and F are two local topics with a common set of
things on top of master.

The next thing that happens is that master is updated, and I want to
rebase both topics on top of the new master.

So I now have:

A---G
\---B---C---D---E
            \---F

If I do the dumb thing, which is to do `git rebase master E` and `git
rebase master F`, I end up with:

A---G---B'---C'---D'---E'
    \---B"---C"---D"---F'

That is, I just lost the fast that E and F had common history.

I could go with `git rebase master E` followed by `git rebase --onto D'
D F` but that's cumbersome, especially when you have more than 2 topics,
not necessarily diverging at the same point (e.g. I might have another
topic that diverges at C)

So, what I end up doing is something like:
- git co -b merge E
- git merge --strategy ours F (and any other topic I might want to
  rebase)
- git rebase master --preserve-merges

If everything goes fine, then I can `git update-ref` the topics to each
parent of the merge branch.

But, usually, since rebase --preserve-merges redoes merges with the
default strategy, I end up with conflicts, and instead of trying to
figure stuff out, I just pick the rewritten sha1s from
.git/rebase-merge/rewritten/* to update the refs.

It is my understanding that the --strategy option to git-rebase is used
for the rebase itself, so I'm not sure there's a way to tell rebase to
use a specific strategy for the preserved merges only.

Anyways, it /seems/ like just allowing multiple branches on the git
rebase command line and make this work would improve things
significantly. The question then, is how would that interact with other
options (I'm thinking e.g. -i, but -i already has a problem with
--preserve-merges). But it does seem like it would be a worthwhile
improvement.

What do you think?

Mike

Re: Rebasing multiple branches at once

[Junio C Hamano]

Mike Hommey <mh@glandium.org> writes:

> So I now have:
>
> A---G
> \---B---C---D---E
>             \---F
>
> If I do the dumb thing, which is to do `git rebase master E` and `git
> rebase master F`, I end up with:
>
> A---G---B'---C'---D'---E'
>     \---B"---C"---D"---F'
>

What people seem to do is to teach the branch that ends with F that
its upstream is the local branch that ends with E, so that they can
be lazy when rebasing a branch that knows its upstream.  I suspect
that you would end up with

A---G---B'--C'--D'--E'--F'

instead if it is done naively, but if you really care that the
branch that ends with F does not have E, you presumably want to have
the branch that ends at D its own identity, so

 (1) 'master' or whatever that used to end at A and now its tip is
     at G;

 (2) the branch that ends at D whose upstream is 'master';

 (3) the branch that ends at E whose upstream is (2); and

 (4) the branch that ends at F whose upstream is (2).

I personally do not do that, though, because you'd need to remember
the order in which these three branches must be rebased (i.e. (2)
must be done first before rebasing (3) and (4) in any order).

Re: Rebasing multiple branches at once

[Jeff King]

On Sat, Dec 31, 2016 at 06:40:33PM -0800, Junio C Hamano wrote:

> What people seem to do is to teach the branch that ends with F that
> its upstream is the local branch that ends with E, so that they can
> be lazy when rebasing a branch that knows its upstream.  I suspect
> that you would end up with
> 
> A---G---B'--C'--D'--E'--F'
> 
> instead if it is done naively, but if you really care that the
> branch that ends with F does not have E, you presumably want to have
> the branch that ends at D its own identity, so
> 
>  (1) 'master' or whatever that used to end at A and now its tip is
>      at G;
> 
>  (2) the branch that ends at D whose upstream is 'master';
> 
>  (3) the branch that ends at E whose upstream is (2); and
> 
>  (4) the branch that ends at F whose upstream is (2).
> 
> I personally do not do that, though, because you'd need to remember
> the order in which these three branches must be rebased (i.e. (2)
> must be done first before rebasing (3) and (4) in any order).

I do occasionally have dependent topics, and use a topological sort to
order my rebases, which solves the problem. The code I use is in:

  https://github.com/peff/git/blob/meta/rebase

-Peff

Re: Rebasing multiple branches at once

[Johannes Sixt]

Am 31.12.2016 um 09:14 schrieb Mike Hommey:
> Hi,
>
> I've had this kind of things to do more than once, and had to do it a
> lot today, so I figured it would be worth discussing whether git-rebase
> should be enhanced to support this, or if this should go in a separate
> tool or whatever.
>
> So here is what I'm trying to do in a not-too painful way:
>
> I'm starting with something like this:
> A---B---C---D---E
>             \---F
>
> where A is master, and E and F are two local topics with a common set of
> things on top of master.
>
> The next thing that happens is that master is updated, and I want to
> rebase both topics on top of the new master.
>
> So I now have:
>
> A---G
> \---B---C---D---E
>             \---F
>
> If I do the dumb thing, which is to do `git rebase master E` and `git
> rebase master F`, I end up with:
>
> A---G---B'---C'---D'---E'
>     \---B"---C"---D"---F'
>
> That is, I just lost the fast that E and F had common history.

Git garden shears, perhaps?

https://public-inbox.org/git/alpine.DEB.2.20.1609111027330.129229@virtualbox/

-- Hannes

Rebasing Multiple branches at once...

[Rick Moynihan]

Hi,

I have a master branch, a dev branch and a number of feature branches 
from dev.  And I was wondering if there was an easy way to rebase dev 
and all of it's sub-branches onto master.

I know I can run this as a series of commands, and use --onto to do 
this, but was wondering if there was an easier way.  As running:

git rebase master

when on the dev branch only rebases dev and not it's dependent branches.

Thanks!

R.

Re: Rebasing Multiple branches at once...

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

On Thu, Oct 16, 2008 at 01:17:20PM +0100, Rick Moynihan <rick@calicojack.co.uk> wrote:
> I have a master branch, a dev branch and a number of feature branches from 
> dev.  And I was wondering if there was an easy way to rebase dev and all of 
> it's sub-branches onto master.

In general this is considered harmful. Why do you rebase your branch
from time to time? For example in git.git, topic branches are based on
master, then merged to master when they are ready, but they are never
rebased.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Rebasing Multiple branches at once...

[Rick Moynihan]

Miklos Vajna wrote:
> On Thu, Oct 16, 2008 at 01:17:20PM +0100, Rick Moynihan <rick@calicojack.co.uk> wrote:
>> I have a master branch, a dev branch and a number of feature branches from 
>> dev.  And I was wondering if there was an easy way to rebase dev and all of 
>> it's sub-branches onto master.
> 
> In general this is considered harmful. Why do you rebase your branch
> from time to time? For example in git.git, topic branches are based on
> master, then merged to master when they are ready, but they are never
> rebased.

Yes, but my understanding is that it's only harmful if you publish the 
branch (or dependent branches) which are being rebased.

So rebasing is very bad in these circumstances, but I fail to see why 
it's bad if these branches are kept private.


R.

Re: Rebasing Multiple branches at once...

[Miklos Vajna]

[-- Attachment #1: Type: text/plain, Size: 602 bytes --]

On Thu, Oct 16, 2008 at 03:48:11PM +0100, Rick Moynihan <rick@calicojack.co.uk> wrote:
> Yes, but my understanding is that it's only harmful if you publish the 
> branch (or dependent branches) which are being rebased.
> 
> So rebasing is very bad in these circumstances, but I fail to see why it's 
> bad if these branches are kept private.

Ah, I thought you publish your branches.

One reason may be that if you use merge, no history is lost, if you use
rebase, history is in the reflogs, so it'll be lost after some time. But
if you know what you are doing, then this is not a problem.

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Rebasing Multiple branches at once...

[Junio C Hamano]

Miklos Vajna <vmiklos@frugalware.org> writes:

> On Thu, Oct 16, 2008 at 01:17:20PM +0100, Rick Moynihan <rick@calicojack.co.uk> wrote:
>> I have a master branch, a dev branch and a number of feature branches from 
>> dev.  And I was wondering if there was an easy way to rebase dev and all of 
>> it's sub-branches onto master.
>
> In general this is considered harmful. Why do you rebase your branch
> from time to time? For example in git.git, topic branches are based on
> master, then merged to master when they are ready, but they are never
> rebased.

I do not think it is harmful per-se as long as they are not published.

But rebasing topic branches regularly (i.e. without reasons better than
"the master has more commits than before") is _pointless_.  The whole
point of a topic branch is to house the development related to one single
topic.  If you keep rebasing it, its progress (when you look at the
differences between topic@{1} and topic, topic@{2} and topic@{1}, ...)
would not be about the single topic, but about the single topic _and
all the other random things that happened on the master branch_.

When you merge a topic branch that forked from an older version of
'master', you may have conflicts, and you may want to resolve it earlier
and that might be why you would want to rebase.  But if you _know_ you are
going to have conflicts, you wouldn't wish to rebase all of them at once
to begin with, as you would need to resolve such conflicts yourself
anyway.  On the other hand, if you expect there won't be any conflict,
then there absolutely is no point in rebasing them.  If you want to make
sure that your topics would all work with the updated 'master', you are
much better off creating trial tree, merging your topics on top of the
updated 'master' one by one, than rebasing all of your topic branches.

Re: Rebasing Multiple branches at once...

[David Kastrup]

Rick Moynihan <rick@calicojack.co.uk> writes:

> Hi,
>
> I have a master branch, a dev branch and a number of feature branches
> from dev.  And I was wondering if there was an easy way to rebase dev
> and all of it's sub-branches onto master.
>
> I know I can run this as a series of commands, and use --onto to do
> this, but was wondering if there was an easier way.  As running:
>
> git rebase master
>
> when on the dev branch only rebases dev and not it's dependent branches.

Rebasing has no relation to dependent branches.  It creates a new branch
from the branch point.  After it finishes, it just reseats HEAD of the
branch to the new one.  There is no operation that would work implicitly
on originally dependent branches.

-- 
David Kastrup

Re: Rebasing Multiple branches at once...

[Rick Moynihan]

David Kastrup wrote:
> Rick Moynihan <rick@calicojack.co.uk> writes:
> 
>> Hi,
>>
>> I have a master branch, a dev branch and a number of feature branches
>> from dev.  And I was wondering if there was an easy way to rebase dev
>> and all of it's sub-branches onto master.
>>
>> I know I can run this as a series of commands, and use --onto to do
>> this, but was wondering if there was an easier way.  As running:
>>
>> git rebase master
>>
>> when on the dev branch only rebases dev and not it's dependent branches.
> 
> Rebasing has no relation to dependent branches.  It creates a new branch
> from the branch point.  After it finishes, it just reseats HEAD of the
> branch to the new one.  There is no operation that would work implicitly
> on originally dependent branches.

This appears to be true of the current implementation, but shouldn't it 
be possible to do this as a single operation?

e.g. when the situation is this with dev being the current branch.

o---o---o---o---o  master
      \
       o---o---o---o---o  dev (*)
                        \
                         o---o---o  topic

Running the hypothetical command:

git rebase master --all

Would produce this:

o---o---o---o---o  master
                  \
                   o---o---o---o---o  dev (*)
                                    \
                                     o---o---o  topic

I think this can be performed right now with a rebase followed by a 
rebase --onto

I can see how if there were conflicts in the rebase from dev, then you 
would need to resolve them all the way up your topic branches also.  Is 
there anything else that makes this a bad idea?

R.

Re: Rebasing Multiple branches at once...

[Robin Burchell]

On Thu, Oct 16, 2008 at 3:57 PM, Rick Moynihan <rick@calicojack.co.uk> wrote:
>
> This appears to be true of the current implementation, but shouldn't it be
> possible to do this as a single operation?
>
> e.g. when the situation is this with dev being the current branch.
>
> o---o---o---o---o  master
>     \
>      o---o---o---o---o  dev (*)
>                       \
>                        o---o---o  topic
>
> Running the hypothetical command:
>
> git rebase master --all
>
> Would produce this:
>
> o---o---o---o---o  master
>                 \
>                  o---o---o---o---o  dev (*)
>                                   \
>                                    o---o---o  topic
>
> I think this can be performed right now with a rebase followed by a rebase
> --onto
>
> I can see how if there were conflicts in the rebase from dev, then you would
> need to resolve them all the way up your topic branches also.  Is there
> anything else that makes this a bad idea?
>
> R.


Rebase is indeed useful IMHO in situations like this with multiple
related topic branches when needing to pull a single fix or two from
somewhere without messy merges (especially when that will end up with
rather a lot of merge commits in history - one or so for each branch,
which is not exactly desirable).

(resent after I learned how to use 'reply to all', sorry Rick)

Re: Rebasing Multiple branches at once...

[Toby Allsopp]

[-- Attachment #1: Type: text/plain, Size: 1017 bytes --]

On Fri, Oct 17 2008, Rick Moynihan wrote:

> Hi,
>
> I have a master branch, a dev branch and a number of feature branches
> from dev.  And I was wondering if there was an easy way to rebase dev
> and all of it's sub-branches onto master.
>
> I know I can run this as a series of commands, and use --onto to do
> this, but was wondering if there was an easier way.  As running:
>
> git rebase master
>
> when on the dev branch only rebases dev and not it's dependent
> branches.

I have a Perl script I use to rebase a number of topic branches as the
remote tracking branches they're based on move.  It handles the case of
topic based on other topics.  It is designed specifically for my
workflow, which is tracking a central Subversion repository using
git-svn, but I don't think it relies on using git-svn.  Anyway, you
might find it useful for inspiration.

The script outputs a sequence of commands and leaves the running of them
up to you because you may need to resolve conflicts at any point.

Regards,
Toby.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: git-rebase-branches --]
[-- Type: text/x-perl, Size: 2748 bytes --]

#!/usr/bin/perl

# Rebases master and everything based on master to the new trunk.  Use
# after a git-svn-fetch.

use strict;
use warnings;

use Getopt::Long;
use List::Util qw(first);

use Git;

my $dry_run;

GetOptions("dry-run|n" => \$dry_run)
  or die "usage error";

sub ref2branch {
    my $ref = shift;
    $ref =~ s,^refs/heads/,,
      or die "Not a branch: '$ref'";
    return $ref;
}

my $repo = Git->repository();

my %remotes_by_name;
my %remotes_by_hash;
my %remote_revs;

for ($repo->command('for-each-ref', 'refs/remotes')) {
    my ($hash, undef, $ref) = split;
    $remotes_by_name{$ref} = $hash;
    $remotes_by_hash{$hash} = $ref;
    $remote_revs{$ref} = [$repo->command('rev-list', $ref)];
}

my %heads_by_name;
my %heads_by_hash;

for ($repo->command('for-each-ref', 'refs/heads')) {
    my ($hash, undef, $ref) = split;
    $heads_by_name{$ref} = $hash;
    $heads_by_hash{$hash} = $ref;
}

my %roots;
my %heads_by_parent;

for my $head (sort keys %heads_by_name) {
    #print STDERR "Considering $head\n";
    my $parent;
    my $last_rev;
    for my $rev ($repo->command('rev-list', $head, '--not', keys %remotes_by_name)) {
        my $maybe_parent = $heads_by_hash{$rev};
        if ($maybe_parent && $maybe_parent ne $head) {
            #print STDERR "  found parent $maybe_parent\n";
            $parent = $maybe_parent;
            last;
        }
        $last_rev = $rev;
    }
    if ($parent) {
        push @{$heads_by_parent{$parent}}, $head;
    } elsif ($last_rev) {
        my $remote_base = $repo->command_oneline('rev-parse', "$last_rev^");
        my @remotes;
        #print STDERR "  last rev $last_rev $remote_base\n";
        for my $remote_name (sort keys %remotes_by_name) {
            my $remote = first { $_ eq $remote_base } @{$remote_revs{$remote_name}};
            if (defined($remote) && $remote eq $remote_base) {
                #print STDERR "  found remote $remote_name\n";
                push @remotes, $remote_name;
            }
        }
        if (@remotes == 1) {
            $roots{$head} = $remotes[0];
        } else {
            print STDERR "WARNING: Not exactly one candidate remote for $head: ",
                join(' ', @remotes), "\n";
        }
    }
}

for my $root (sort keys %roots) {
    my $remote = $roots{$root};
    my $short_root = ref2branch($root);
    $remote =~ s,^refs/,,;
    print "git rebase $remote $short_root\n";
    rebase_tree($root);
}

sub rebase_tree {
    my ($parent) = @_;
    for my $head (@{$heads_by_parent{$parent}}) {
        my $short_parent = ref2branch($parent);
        my $short_head = ref2branch($head);
        print "git rebase --onto $short_parent $heads_by_name{$parent} $short_head\n";
        rebase_tree($head);
    }
}