* [PATCH v2 0/2] perf/aggregate: sort results by regression
@ 2018-03-26 7:24 Christian Couder
2018-03-26 7:24 ` [PATCH v2 1/2] perf/aggregate: add display_dir() Christian Couder
2018-03-26 7:24 ` [PATCH v2 2/2] perf/aggregate: add --sort-by=regression option Christian Couder
0 siblings, 2 replies; 3+ messages in thread
From: Christian Couder @ 2018-03-26 7:24 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Thomas Rast,
Ævar Arnfjörð Bjarmason, Christian Couder,
Eric Sunshine, Philip Oakley
This small patch series makes it easy to spot big performance
regressions, so that they can later be investigated.
For example:
$ ./aggregate.perl --sort-by=regression --subsection "without libpcre" v2.14.3 v2.15.1 v2.16.2 p4220-log-grep-engines.sh
+5.0% p4220-log-grep-engines.2 0.60(0.58+0.02) 0.63(0.59+0.04) v2.14.3 v2.15.1
+4.5% p4220-log-grep-engines.10 0.67(0.64+0.03) 0.70(0.67+0.02) v2.14.3 v2.15.1
+1.7% p4220-log-grep-engines.5 0.58(0.57+0.01) 0.59(0.59+0.00) v2.14.3 v2.15.1
+1.7% p4220-log-grep-engines.6 0.58(0.58+0.00) 0.59(0.56+0.02) v2.14.3 v2.15.1
+1.7% p4220-log-grep-engines.17 0.58(0.57+0.01) 0.59(0.56+0.02) v2.14.3 v2.15.1
+1.7% p4220-log-grep-engines.1 0.60(0.58+0.01) 0.61(0.60+0.01) v2.14.3 v2.15.1
+1.6% p4220-log-grep-engines.13 0.64(0.63+0.02) 0.65(0.63+0.01) v2.14.3 v2.15.1
+1.5% p4220-log-grep-engines.9 0.67(0.66+0.01) 0.68(0.67+0.01) v2.14.3 v2.15.1
+0.0% p4220-log-grep-engines.14 0.65(0.62+0.02) 0.65(0.63+0.02) v2.14.3 v2.15.1
+0.0% p4220-log-grep-engines.18 0.58(0.57+0.00) 0.58(0.56+0.02) v2.14.3 v2.15.1
-1.5% p4220-log-grep-engines.13 0.65(0.63+0.01) 0.64(0.62+0.01) v2.15.1 v2.16.2
-1.5% p4220-log-grep-engines.14 0.65(0.63+0.02) 0.64(0.60+0.03) v2.15.1 v2.16.2
-1.6% p4220-log-grep-engines.1 0.61(0.60+0.01) 0.60(0.58+0.02) v2.15.1 v2.16.2
-1.7% p4220-log-grep-engines.5 0.59(0.59+0.00) 0.58(0.55+0.02) v2.15.1 v2.16.2
-1.7% p4220-log-grep-engines.6 0.59(0.56+0.02) 0.58(0.55+0.02) v2.15.1 v2.16.2
-1.7% p4220-log-grep-engines.18 0.58(0.56+0.02) 0.57(0.55+0.02) v2.15.1 v2.16.2
-2.9% p4220-log-grep-engines.9 0.68(0.67+0.01) 0.66(0.64+0.02) v2.15.1 v2.16.2
-3.4% p4220-log-grep-engines.17 0.59(0.56+0.02) 0.57(0.55+0.01) v2.15.1 v2.16.2
-4.3% p4220-log-grep-engines.10 0.70(0.67+0.02) 0.67(0.66+0.01) v2.15.1 v2.16.2
-4.8% p4220-log-grep-engines.2 0.63(0.59+0.04) 0.60(0.57+0.03) v2.15.1 v2.16.2
Since V1, the indent issues have been fixed and the name of the option
has been changed to --sort-by=regression (from --sortbyregression) as
suggested by Junio.
Christian Couder (2):
perf/aggregate: add display_dir()
perf/aggregate: add --sort-by=regression option
t/perf/aggregate.perl | 70 +++++++++++++++++++++++++++++++++++++++----
1 file changed, 65 insertions(+), 5 deletions(-)
--
2.17.0.rc1.36.g098d832c9.dirty
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH v2 1/2] perf/aggregate: add display_dir()
2018-03-26 7:24 [PATCH v2 0/2] perf/aggregate: sort results by regression Christian Couder
@ 2018-03-26 7:24 ` Christian Couder
2018-03-26 7:24 ` [PATCH v2 2/2] perf/aggregate: add --sort-by=regression option Christian Couder
1 sibling, 0 replies; 3+ messages in thread
From: Christian Couder @ 2018-03-26 7:24 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Thomas Rast,
Ævar Arnfjörð Bjarmason, Christian Couder,
Eric Sunshine, Philip Oakley
This new helper function will be reused in a subsequent
commit.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/perf/aggregate.perl | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
index 821cf1498b..890d85fa7b 100755
--- a/t/perf/aggregate.perl
+++ b/t/perf/aggregate.perl
@@ -147,6 +147,11 @@ sub have_slash {
return 0;
}
+sub display_dir {
+ my ($d) = @_;
+ return exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d};
+}
+
sub print_default_results {
my %descrs;
my $descrlen = 4; # "Test"
@@ -168,8 +173,7 @@ sub print_default_results {
my %times;
my @colwidth = ((0)x@dirs);
for my $i (0..$#dirs) {
- my $d = $dirs[$i];
- my $w = length (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});
+ my $w = length display_dir($dirs[$i]);
$colwidth[$i] = $w if $w > $colwidth[$i];
}
for my $t (@subtests) {
@@ -188,8 +192,7 @@ sub print_default_results {
printf "%-${descrlen}s", "Test";
for my $i (0..$#dirs) {
- my $d = $dirs[$i];
- printf " %-$colwidth[$i]s", (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});
+ printf " %-$colwidth[$i]s", display_dir($dirs[$i]);
}
print "\n";
print "-"x$totalwidth, "\n";
--
2.17.0.rc1.36.g098d832c9.dirty
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH v2 2/2] perf/aggregate: add --sort-by=regression option
2018-03-26 7:24 [PATCH v2 0/2] perf/aggregate: sort results by regression Christian Couder
2018-03-26 7:24 ` [PATCH v2 1/2] perf/aggregate: add display_dir() Christian Couder
@ 2018-03-26 7:24 ` Christian Couder
1 sibling, 0 replies; 3+ messages in thread
From: Christian Couder @ 2018-03-26 7:24 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jeff King, Thomas Rast,
Ævar Arnfjörð Bjarmason, Christian Couder,
Eric Sunshine, Philip Oakley
One of the most interesting thing one can be interested in when
looking at performance test results is possible performance
regressions.
This new option makes it easy to spot such possible regressions.
This new option is named '--sort-by=regression' to make it
possible and easy to add other ways to sort the results, like for
example '--sort-by=utime'.
If we would like to sort according to how much the stime regressed
we could also add a new option called '--sort-by=regression:stime'.
Then '--sort-by=regression' could become a synonym for
'--sort-by=regression:rtime'.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
t/perf/aggregate.perl | 59 ++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 58 insertions(+), 1 deletion(-)
diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
index 890d85fa7b..48637ef64b 100755
--- a/t/perf/aggregate.perl
+++ b/t/perf/aggregate.perl
@@ -37,7 +37,7 @@ sub format_times {
}
my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests,
- $codespeed, $subsection, $reponame);
+ $codespeed, $sortby, $subsection, $reponame);
while (scalar @ARGV) {
my $arg = $ARGV[0];
my $dir;
@@ -46,6 +46,18 @@ while (scalar @ARGV) {
shift @ARGV;
next;
}
+ if ($arg =~ /--sort-by(?:=(.*))?/) {
+ shift @ARGV;
+ if (defined $1) {
+ $sortby = $1;
+ } else {
+ $sortby = shift @ARGV;
+ if (! defined $sortby) {
+ die "'--sort-by' requires an argument";
+ }
+ }
+ next;
+ }
if ($arg eq "--subsection") {
shift @ARGV;
$subsection = $ARGV[0];
@@ -209,6 +221,49 @@ sub print_default_results {
}
}
+sub print_sorted_results {
+ my ($sortby) = @_;
+
+ if ($sortby ne "regression") {
+ die "only 'regression' is supported as '--sort-by' argument";
+ }
+
+ my @evolutions;
+ for my $t (@subtests) {
+ my ($prevr, $prevu, $prevs, $prevrev);
+ for my $i (0..$#dirs) {
+ my $d = $dirs[$i];
+ my ($r, $u, $s) = get_times("$resultsdir/$prefixes{$d}$t.times");
+ if ($i > 0 and defined $r and defined $prevr and $prevr > 0) {
+ my $percent = 100.0 * ($r - $prevr) / $prevr;
+ push @evolutions, { "percent" => $percent,
+ "test" => $t,
+ "prevrev" => $prevrev,
+ "rev" => $d,
+ "prevr" => $prevr,
+ "r" => $r,
+ "prevu" => $prevu,
+ "u" => $u,
+ "prevs" => $prevs,
+ "s" => $s};
+ }
+ ($prevr, $prevu, $prevs, $prevrev) = ($r, $u, $s, $d);
+ }
+ }
+
+ my @sorted_evolutions = sort { $b->{percent} <=> $a->{percent} } @evolutions;
+
+ for my $e (@sorted_evolutions) {
+ printf "%+.1f%%", $e->{percent};
+ print " " . $e->{test};
+ print " " . format_times($e->{prevr}, $e->{prevu}, $e->{prevs});
+ print " " . format_times($e->{r}, $e->{u}, $e->{s});
+ print " " . display_dir($e->{prevrev});
+ print " " . display_dir($e->{rev});
+ print "\n";
+ }
+}
+
sub print_codespeed_results {
my ($subsection) = @_;
@@ -263,6 +318,8 @@ binmode STDOUT, ":utf8" or die "PANIC on binmode: $!";
if ($codespeed) {
print_codespeed_results($subsection);
+} elsif (defined $sortby) {
+ print_sorted_results($sortby);
} else {
print_default_results();
}
--
2.17.0.rc1.36.g098d832c9.dirty
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2018-03-26 7:25 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-26 7:24 [PATCH v2 0/2] perf/aggregate: sort results by regression Christian Couder
2018-03-26 7:24 ` [PATCH v2 1/2] perf/aggregate: add display_dir() Christian Couder
2018-03-26 7:24 ` [PATCH v2 2/2] perf/aggregate: add --sort-by=regression option Christian Couder
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).