[filter-repo] How to reorder commits

[filter-repo] How to reorder commits

[Konstantin Ryabitsev]

Elijah:

It seems filter-repo can be used to easily reorder commits, but I'm not able
to quite figure out how to do it with callbacks. My goal is:

- keep the cover letter at the start of the series in order to cleanly
  delineate where their work starts (and thus avoid potential problems with
  rebases and finding fork-points with base branches, which tends to give
  newbies trouble)
- when the author is ready to submit their work, move the cover letter to the
  tip of the branch and tag it as "foo-bar-v1"
- when the author is ready to work on their changes, they run --reroll and we
  move the cover letter back to the start of the series, adding any necessary
  template "changes in vX" entries to it

(or maybe it will go completely differently -- it's all very fluid right now)

Basically, is git-filter-repo the right tool to reorder commits, or is it best
to go directly with fast-export/fast-import?

Thanks in advance,
Konstantin

Re: [filter-repo] How to reorder commits

[Elijah Newren]

On Thu, Jul 21, 2022 at 8:19 AM Konstantin Ryabitsev
<konstantin@linuxfoundation.org> wrote:
>
> Elijah:
>
> It seems filter-repo can be used to easily reorder commits, but I'm not able
> to quite figure out how to do it with callbacks. My goal is:
>
> - keep the cover letter at the start of the series in order to cleanly
>   delineate where their work starts (and thus avoid potential problems with
>   rebases and finding fork-points with base branches, which tends to give
>   newbies trouble)
> - when the author is ready to submit their work, move the cover letter to the
>   tip of the branch and tag it as "foo-bar-v1"
> - when the author is ready to work on their changes, they run --reroll and we
>   move the cover letter back to the start of the series, adding any necessary
>   template "changes in vX" entries to it
>
> (or maybe it will go completely differently -- it's all very fluid right now)
>
> Basically, is git-filter-repo the right tool to reorder commits, or is it best
> to go directly with fast-export/fast-import?
>
> Thanks in advance,
> Konstantin

Generally, it makes no sense to reorder commits in filter-repo, nor
does it make sense to do so using fast-export/fast-import directly.
fast-export/fast-import don't "apply a diff" to get a new tree, they
say "Make these files have these exact contents".  Perhaps an example
would help to explain what I mean...

Let's say you have a repository where some commit has a file
consisting of the numbers 2-9 on separate lines.  Then you had two
more commits after this one: Commit A has numbers 1-9 (i.e. inserting
a new first line with a "1"), and commit B has numbers 1-10 (i.e.
adding "10" to the last line of the file).  If you reverse those two
commits in the fast-export/fast-import stream, then B will still have
the numbers 1-10 and A will still have the numbers 1-9 in the file.
That makes it look like B added both lines, and then A removed the
final line, and the end result is a file with one less line than you
had before you reversed the commits.

So, generally, I think you should use a rebase-like tool for
reordering commits rather than anything based on
fast-export/fast-import.


Now, with an empty commit you wouldn't run into these issues and so
you could get away with reordering.  But there is another issue: in
the fast-import stream, you have to list all ancestors of a commit
before you list the commit.  And git-filter-repo parses commits as it
goes.  So you'd have to cache a commit or range of commits without
outputting them, and then later dump them once the new parents have
been output first.  I think a rebase-like tool would probably be
better for this job, but if you really want to, you could probably get
this working in filter-repo with a script similar to the ones in
contrib/filter-repo-demos, but making sure to use steps something like
the following:
  * Avoid commits getting dumped by setting "commit.dumped = 1" (a
"lie" which prevents git-filter-repo from immediately outputting the
commit after your callback)
  * Save the commits in a list somewhere
  * As you are processing the commit(s) that need to be moved earlier
modify the parents of these commit(s) (i.e. the commit.parents field)
appropriately
  * After you've output the commit(s) that needed to be moved earlier,
iterate through the saved commit(s) and modify their parents and call
commit.dump() on them.

Re: [filter-repo] How to reorder commits

[Konstantin Ryabitsev]

On Fri, Jul 22, 2022 at 04:18:48PM -0700, Elijah Newren wrote:
> > - keep the cover letter at the start of the series in order to cleanly
> >   delineate where their work starts (and thus avoid potential problems with
> >   rebases and finding fork-points with base branches, which tends to give
> >   newbies trouble)
> > - when the author is ready to submit their work, move the cover letter to the
> >   tip of the branch and tag it as "foo-bar-v1"
> > - when the author is ready to work on their changes, they run --reroll and we
> >   move the cover letter back to the start of the series, adding any necessary
> >   template "changes in vX" entries to it
> >
> > (or maybe it will go completely differently -- it's all very fluid right now)
> >
> > Basically, is git-filter-repo the right tool to reorder commits, or is it best
> > to go directly with fast-export/fast-import?
> >
> > Thanks in advance,
> > Konstantin
> 
> Generally, it makes no sense to reorder commits in filter-repo, nor
> does it make sense to do so using fast-export/fast-import directly.
> fast-export/fast-import don't "apply a diff" to get a new tree, they
> say "Make these files have these exact contents".  Perhaps an example
> would help to explain what I mean...

Thank you for the detailed reply, Elijah. You did convince that filter-repo is
not the right tool for this case, so I went with cherry-pick instead. Since my
case is very straightforward with simple ranges and a guarantee of no
conflicts, it's the most straightforward solution.

Cheers,
Konstantin