Re: A workflow for local patch maintenance

[Jeff King <peff@peff.net>]

On Tue, Oct 08, 2013 at 07:12:22PM +0100, Tony Finch wrote:

> We often need to patch the software that we run in order to fix bugs
> quickly rather than wait for an official release, or to add functionality
> that we need. In many cases we have to maintain a locally-developed patch
> for a significant length of time, across multiple upstream releases,
> either because it is not yet ready for incorporation into a stable
> upstream version, or because it is too specific to our setup so will not
> be suitable for passing upstream without significant extra work.

Do you need to keep the modifications you make on top of upstream as a
nice, clean series of rebased patches? If not, then you can avoid the
repeated rebasing, and just use a more traditional topic-branch
workflow. Treat modifications from upstream as just another topic.

For example, start with some version (let's say 1.0) of the upstream
software as your "master" branch. If it's kept in git, build on
upstream's git history.  If all you get are tarballs, create an
"upstream" branch with v1.0, and fork "master" from it.

Build on master as you would if it were your own. Fork topic branches,
develop the topics, test them, and then merge them back to "master" when
they're ready (or do development straight on master, or whatever
workflow you're accustomed to).

When v1.1 of the upstream software comes out, create a "merge-upstream"
topic branch from the tip of your "master". If upstream is in git, just
"git merge v1.1" from upstream. If not, then checkout your pristine
"upstream" branch (which should still be sitting at the v1.0 commit),
and build a v1.1 commit on top of it. And then "git merge upstream" to
pick up the new changes.

Test your merge-upstream topic in isolation, and when you think it's
ready merge it into master and deploy.

The most difficult part is the merge of upstream into the topic branch.
But git's 3-way merge tends to do a pretty good job (e.g., if you
contributed your patches upstream, then there should be no conflict).
You can also break up the work by keeping the "merge" topic running for
a long time, and merging as often as possible from upstream. That breaks
the conflict resolution into smaller chunks, and lets you do it closer
to when the conflicting patches were actually made, when they are
hopefully closer in your mind. And you don't have to worry about having
a broken intermediate result, because you're not deploying it; you're
just keeping the topic up to date until you're ready to test it.

You can also try git-imerge, which can make big merges a little more
manageable (though it can also make them harder sometimes...):

  https://github.com/mhagger/git-imerge

The history for such a repository might look like:

       o--o--B--o--o--C  <-- upstream branch
      /       \        \
     o--o---o--o--o--o--D  <-- upstream-merge branch
    /      /        /    \
   A--o---E--o--o--F--o---G <-- master branch
    \    / \      /
     o--o   o----o  <-- topic branches

where:

  - A is the v1.0 commit you start at

  - B and C are milestones where you merged upstream into your
    upstream-merge topic branch. These could be releases (like v1.1), or
    they could just be random spots where you felt like merging to keep
    things up to date. It depends how you want to break up the conflict
    resolution

  - D is a state of the upstream-merge branch that you test to make sure
    the merge happened OK

  - E and F are merges of regular topic branches (i.e., the patches you
    are working on locally). Note that we also merge those up to the
    upstream-merge branch, so that we can resolve early any conflicts
    between what's happening on master and what's happening upstream.

  - G is the merge of D into the master branch, after we have decided
    it's good to deploy

This all assumes that "master" is your known-good state that you deploy
or ship. If you prefer to have a "deploy" or "maint" branch for
hotfixes, you can do that too.

Hope that helps,
-Peff