[PATCH] contrib/rebase-catchup: helper for updating old branches

[PATCH] contrib/rebase-catchup: helper for updating old branches

[Mike Frysinger]

For people who want to rebase their work onto the latest branch
(instead of merging), but there's many conflicting changes.  This
allows you to address those conflicts one-by-one and work through
each issue instead of trying to take them all on at once.

Signed-off-by: Mike Frysinger <vapier@gentoo.org>
---
If there's no interest in merging this into contrib, then this is more spam,
and anyone interested can use https://github.com/vapier/git-rebase-catchup

 contrib/rebase-catchup/README.md             |  61 ++++++
 contrib/rebase-catchup/git-rebase-catchup.py | 187 +++++++++++++++++++
 2 files changed, 248 insertions(+)
 create mode 100644 contrib/rebase-catchup/README.md
 create mode 100755 contrib/rebase-catchup/git-rebase-catchup.py

diff --git a/contrib/rebase-catchup/README.md b/contrib/rebase-catchup/README.md
new file mode 100644
index 000000000000..083d2012f833
--- /dev/null
+++ b/contrib/rebase-catchup/README.md
@@ -0,0 +1,61 @@
+# Rebase Catchup
+
+Helpful when you have a branch tracking an old commit, and a lot of conflicting
+changes have landed in the latest branch, but you still want to update.
+
+A single rebase to the latest commit will require addressing all the different
+changes at once which can be difficult, overwhelming, and error-prone.  Instead,
+if you rebased onto each intermediate conflicting point, you'd break up the work
+into smaller pieces, and be able to run tests to make sure things were still OK.
+
+## Example
+
+Let's say you have a branch that is currently 357 commits behind.  When you try
+rebasing onto the latest, it hits a lot of conflicts.  The tool will bisect down
+to find the most recent commit it can cleanly rebase onto.
+
+```sh
+$ git rebase-catchup
+Local branch resolved to "s-logs"
+Tracking branch resolved to "origin/master"
+Branch is 2 commits ahead and 357 commits behind
+Trying to rebase onto latest origin/master ... failed; falling back to bisect
+Rebasing onto origin/master~178 ... failed
+Rebasing onto origin/master~267 ... failed
+Rebasing onto origin/master~312 ... failed
+Rebasing onto origin/master~334 ... failed
+Rebasing onto origin/master~345 ... OK
+Rebasing onto origin/master~339 ... OK
+Rebasing onto origin/master~336 ... failed
+Rebasing onto origin/master~337 ... OK
+Rebasing onto origin/master~336 ... failed
+Found first failure origin/master~336
+```
+
+Now you know the first conflicting change is `origin/master~336`.  Rebase onto
+that directly and address all the problems (and run tests/etc...).  Then restart
+the process.
+
+```sh
+$ git rebase origin/master~336
+... address all the conflicts ...
+$ git rebase --continue
+$ git rebase-catchup
+Local branch resolved to "s-logs"
+Tracking branch resolved to "origin/master"
+Branch is 2 commits ahead and 335 commits behind
+Trying to rebase onto latest origin/master ... failed; falling back to bisect
+Rebasing onto origin/master~167 ... OK
+Rebasing onto origin/master~83 ... OK
+Rebasing onto origin/master~41 ... failed
+Rebasing onto origin/master~62 ... OK
+Rebasing onto origin/master~51 ... OK
+Rebasing onto origin/master~46 ... OK
+Rebasing onto origin/master~43 ... failed
+Rebasing onto origin/master~44 ... failed
+Rebasing onto origin/master~45 ... OK
+Rebasing onto origin/master~44 ... failed
+Found first failure origin/master~44
+```
+
+Now you're only 44 commits behind.  Keep doing this until you catchup!
diff --git a/contrib/rebase-catchup/git-rebase-catchup.py b/contrib/rebase-catchup/git-rebase-catchup.py
new file mode 100755
index 000000000000..2cc5c5381616
--- /dev/null
+++ b/contrib/rebase-catchup/git-rebase-catchup.py
@@ -0,0 +1,187 @@
+#!/usr/bin/env python3
+# Distributed under the terms of the GNU General Public License v2 or later.
+
+"""Helper to automatically rebase onto latest commit possible.
+
+Helpful when you have a branch tracking an old commit, and a lot of conflicting
+changes have landed in the latest branch, but you still want to update.
+
+A single rebase to the latest commit will require addressing all the different
+changes at once which can be difficult, overwhelming, and error-prone.  Instead,
+if you rebased onto each intermediate conflicting point, you'd break up the work
+into smaller pieces, and be able to run tests to make sure things were still OK.
+"""
+
+import argparse
+import subprocess
+import sys
+from typing import List, Tuple, Union
+
+
+assert sys.version_info >= (3, 7), f'Need Python 3.7+, not {sys.version_info}'
+
+
+def git(args: List[str], **kwargs) -> subprocess.CompletedProcess:
+    """Run git."""
+    kwargs.setdefault('check', True)
+    kwargs.setdefault('capture_output', True)
+    kwargs.setdefault('encoding', 'utf-8')
+    # pylint: disable=subprocess-run-check
+    return subprocess.run(['git'] + args, **kwargs)
+
+
+def rebase(target: str) -> bool:
+    """Try to rebase onto |target|."""
+    try:
+        git(['rebase', target])
+        return True
+    except KeyboardInterrupt:
+        git(['rebase', '--abort'])
+        print('aborted')
+        sys.exit(1)
+    except:
+        git(['rebase', '--abort'])
+        return False
+
+
+def rebase_bisect(lbranch: str,
+                  rbranch: str,
+                  behind: int,
+                  leave_rebase: bool = False,
+                  force_checkout: bool = False):
+    """Try to rebase branch as close to |rbranch| as possible."""
+    def attempt(pos: int) -> bool:
+        target = f'{rbranch}~{pos}'
+        print(f'Rebasing onto {target} ', end='')
+        print('.', end='', flush=True)
+        # Checking out these branches directly helps clobber orphaned files,
+        # but is usually unnessary, and can slow down the overall process.
+        if force_checkout:
+            git(['checkout', '-f', target])
+        print('.', end='', flush=True)
+        if force_checkout:
+            git(['checkout', '-f', lbranch])
+        print('. ', end='', flush=True)
+        ret = rebase(target)
+        print('OK' if ret else 'failed')
+        return ret
+
+    # "pmin" is the latest branch position while "pmax" is where we're now.
+    pmin = 0
+    pmax = behind
+    old_mid = None
+    first_fail = 0
+    while True:
+        mid = pmin + (pmax - pmin) // 2
+        if mid == old_mid or mid < pmin or mid >= pmax:
+            break
+        if attempt(mid):
+            pmax = mid
+        else:
+            first_fail = max(first_fail, mid)
+            pmin = mid
+        old_mid = mid
+
+    if pmin or pmax:
+        last_target = f'{rbranch}~{first_fail}'
+        if leave_rebase:
+            print('Restarting', last_target)
+            result = git(['rebase', last_target], check=False)
+            print(result.stdout.strip())
+        else:
+            print('Found first failure', last_target)
+    else:
+        print('All caught up!')
+
+
+def get_ahead_behind(lbranch: str, rbranch: str) -> Tuple[int, int]:
+    """Return number of commits |lbranch| is ahead & behind relative to |rbranch|."""
+    output = git(
+        ['rev-list', '--left-right', '--count', f'{lbranch}...{rbranch}']).stdout
+    return [int(x) for x in output.split()]
+
+
+def get_tracking_branch(branch: str) -> Union[str, None]:
+    """Return branch that |branch| is tracking."""
+    merge = git(['config', '--local', f'branch.{branch}.merge']).stdout.strip()
+    if not merge:
+        return None
+
+    remote = git(['config', '--local', f'branch.{branch}.remote']).stdout.strip()
+    if remote:
+        if merge.startswith('refs/heads/'):
+            merge = merge[11:]
+        return f'{remote}/{merge}'
+    else:
+        return merge
+
+
+def get_local_branch() -> str:
+    """Return the name of the local checked out branch."""
+    return git(['branch', '--show-current']).stdout.strip()
+
+
+def get_parser() -> argparse.ArgumentParser:
+    """Get CLI parser."""
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument(
+        '--skip-initial-rebase-latest', dest='initial_rebase',
+        action='store_false', default=True,
+        help='skip initial rebase attempt onto the latest branch')
+    parser.add_argument(
+        '--leave-at-last-failed-rebase', dest='leave_rebase',
+        action='store_true', default=False,
+        help='leave tree state at last failing rebase')
+    parser.add_argument(
+        '--checkout-before-rebase', dest='force_checkout',
+        action='store_true', default=False,
+        help='force checkout before rebasing to target (to cleanup orphans)')
+    parser.add_argument(
+        'branch', nargs='?',
+        help='branch to rebase onto')
+    return parser
+
+
+def main(argv: List[str]) -> int:
+    """The main entry point for scripts."""
+    parser = get_parser()
+    opts = parser.parse_args(argv)
+
+    lbranch = get_local_branch()
+    print(f'Local branch resolved to "{lbranch}"')
+    if not lbranch:
+        print('Unable to resolve local branch', file=sys.stderr)
+        return 1
+
+    if opts.branch:
+        rbranch = opts.branch
+    else:
+        rbranch = get_tracking_branch(lbranch)
+    print(f'Tracking branch resolved to "{rbranch}"')
+
+    ahead, behind = get_ahead_behind(lbranch, rbranch)
+    print(f'Branch is {ahead} commits ahead and {behind} commits behind')
+
+    if not behind:
+        print('Up-to-date!')
+    elif not ahead:
+        print('Fast forwarding ...')
+        git(['merge'])
+    else:
+        if opts.initial_rebase:
+            print(f'Trying to rebase onto latest {rbranch} ... ',
+                  end='', flush=True)
+            if rebase(rbranch):
+                print('OK!')
+                return 0
+            print('failed; falling back to bisect')
+        rebase_bisect(lbranch, rbranch, behind, leave_rebase=opts.leave_rebase,
+                      force_checkout=opts.force_checkout)
+
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))
-- 
2.30.0

Re: [PATCH] contrib/rebase-catchup: helper for updating old branches

[Junio C Hamano]

Mike Frysinger <vapier@gentoo.org> writes:

> For people who want to rebase their work onto the latest branch
> (instead of merging), but there's many conflicting changes.  This
> allows you to address those conflicts one-by-one and work through
> each issue instead of trying to take them all on at once.

I wonder how well this compares or complements with Michael
Haggerty's "git imerge".

> If there's no interest in merging this into contrib, then this is more spam,
> and anyone interested can use https://github.com/vapier/git-rebase-catchup

The thinking during the past several years is that the Git ecosystem
and userbase have grown large enough, and unlike our earlier years,
individual add-on's like this (and "imerge" I mentioned earlier) can
thrive without being in-tree to gain an undue exposure boost over
its competitors, so I doubt that adding more stuff to contrib/ would
be a good direction to go in the longer term.

>  create mode 100755 contrib/rebase-catchup/git-rebase-catchup.py

And it does not help for this to be written in Python (which we've
been moving away from).

Having said all that, the end-user community may benefit from having
a well curated set of third-party tools advertised at a single
autoritative-sounding place, but it is dubious that contrib/
subdirectory of my tree should be such a place (I won't be its
curator anyway).

Perhaps there is an opening for non-code contribution for those
without much programming experience but with great organizational
skills.

Thanks.

Re: [PATCH] contrib/rebase-catchup: helper for updating old branches

[Mike Frysinger]

On 08 Mar 2021 15:38, Junio C Hamano wrote:
> Mike Frysinger <vapier@gentoo.org> writes:
> > For people who want to rebase their work onto the latest branch
> > (instead of merging), but there's many conflicting changes.  This
> > allows you to address those conflicts one-by-one and work through
> > each issue instead of trying to take them all on at once.
> 
> I wonder how well this compares or complements with Michael
> Haggerty's "git imerge".

thanks, hadn't heard of that before

> > If there's no interest in merging this into contrib, then this is more spam,
> > and anyone interested can use https://github.com/vapier/git-rebase-catchup
> 
> The thinking during the past several years is that the Git ecosystem
> and userbase have grown large enough, and unlike our earlier years,
> individual add-on's like this (and "imerge" I mentioned earlier) can
> thrive without being in-tree to gain an undue exposure boost over
> its competitors, so I doubt that adding more stuff to contrib/ would
> be a good direction to go in the longer term.

i'm totally fine with "no".  thanks for the info.
-mike