Re: [PATCH v4 00/10] The final building block for a faster rebase -i

[Johannes Schindelin <Johannes.Schindelin@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 1396 bytes --]

Hi René,

On Sat, 27 May 2017, René Scharfe wrote:

> Am 26.05.2017 um 05:15 schrieb Liam Beguin:
> > I tried to time the execution on an interactive rebase (on Linux) but
> > I did not notice a significant change in speed.
> > Do we have a way to measure performance / speed changes between version?
> 
> Well, there's performance test script p3404-rebase-interactive.sh.  You
> could run it e.g. like this:
> 
> 	$ (cd t/perf && ./run origin/master HEAD ./p3404*.sh)
> 
> This would compare the performance of master with the current branch
> you're on.  The results of p3404 are quite noisy for me on master,
> though (saw 15% difference between runs without any code changes), so
> take them with a bag of salt.

Indeed. Our performance tests are simply not very meaningful.

Part of it is the use of shell scripting (which defeats performance
testing pretty well), another part is that we have no performance testing
experts among us, and failed to attract any, so not only is the repeat
count ridiculously small, we also have no graphing worth speaking of (and
therefore it is impossible to even see trends, which is a rather important
visual way to verify sound performance testing).

Frankly, I have no illusion about this getting fixed, ever.

So yes, in the meantime we need to use those numbers with a considerable
amount of skepticism.

Ciao,
Dscho