Re: Comparing rebase --am with --interactive via p3400

[Elijah Newren <newren@gmail.com>]

Hi Dscho,

On Thu, Jan 31, 2019 at 10:04 PM Johannes Schindelin
<Johannes.Schindelin@gmx.de> wrote:
>
> Hi Elijah,
>
> as discussed at the Contributors' Summit, I ran p3400 as-is (i.e. with the
> --am backend) and then with --keep-empty to force the interactive backend
> to be used. Here are the best of 10, on my relatively powerful Windows 10
> laptop, with current `master`.
>
> With regular rebase --am:
>
> 3400.2: rebase on top of a lot of unrelated changes             5.32(0.06+0.15)
> 3400.4: rebase a lot of unrelated changes without split-index   33.08(0.04+0.18)
> 3400.6: rebase a lot of unrelated changes with split-index      30.29(0.03+0.18)
>
> with --keep-empty to force the interactive backend:
>
> 3400.2: rebase on top of a lot of unrelated changes             3.92(0.03+0.18)
> 3400.4: rebase a lot of unrelated changes without split-index   33.92(0.03+0.22)
> 3400.6: rebase a lot of unrelated changes with split-index      38.82(0.03+0.16)

Awesome, thanks for checking that out.  I ran on both linux and mac
and saw similar relative performances.  Comparing am-based rebase to
an implied-interactive rebase on both linux and mac (with a version of
git including en/rebase-merge-on-sequencer so that -m gives the same
performance that you'd see with --keep-empty), I saw:

On Linux:

am-based rebase (without -m):

3400.2: rebase on top of a lot of unrelated changes             1.87(1.64+0.21)
3400.4: rebase a lot of unrelated changes without split-index   7.87(6.24+1.00)
3400.6: rebase a lot of unrelated changes with split-index      5.99(5.05+0.67)

interactive-machinery rebase (with -m):

3400.2: rebase on top of a lot of unrelated changes             1.80(1.60+0.19)
3400.4: rebase a lot of unrelated changes without split-index   6.78(5.70+0.91)
3400.6: rebase a lot of unrelated changes with split-index      6.92(5.70+0.89)


On Mac:

am-based rebase (without -m):

Test                                                            this tree
-------------------------------------------------------------------------------
3400.2: rebase on top of a lot of unrelated changes             2.68(1.68+0.68)
3400.4: rebase a lot of unrelated changes without split-index   8.89(5.86+2.94)
3400.6: rebase a lot of unrelated changes with split-index      7.87(5.35+2.51)


interactive-machinery rebase (with -m):

Test                                                            this tree
-------------------------------------------------------------------------------
3400.2: rebase on top of a lot of unrelated changes             1.99(1.61+0.77)
3400.4: rebase a lot of unrelated changes without split-index   8.63(5.38+3.38)
3400.6: rebase a lot of unrelated changes with split-index      9.36(5.53+3.95)

> I then changed it to -m to test the current scripted version, trying to
> let it run overnight, but my laptop eventually went to sleep and the tests
> were not even done. I'll let them continue and report back.
>
> My conclusion after seeing these numbers is: the interactive rebase is
> really close to the performance of the --am backend. So to me, it makes a
> total lot of sense to switch --merge over to it, and to make --merge the
> default. We still should investigate why the split-index performance is so
> significantly worse, though.

Cool, I'll update my patches to make --merge the default (building on
top of en/rebase-merge-on-sequencer) and post it as an RFC.  But yeah,
we should also check into why the split-index performance becomes a
bit worse with such a change.

Thanks,
Elijah