git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
* [PATCH 0/8] Codespeed perf results
@ 2017-12-13 15:13 Christian Couder
  2017-12-13 15:13 ` [PATCH 1/8] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION} Christian Couder
                   ` (8 more replies)
  0 siblings, 9 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

This patch series is built on top of cc/perf-run-config which recently
graduated to master.

It makes it possible to send perf results to a Codespeed server. See
https://github.com/tobami/codespeed/ and web sites like
http://speed.pypy.org/ which are using Codespeed.

The end goal would be to have such a server always available to track
how the different git commands perform over time on different kind of
repos (small, medium, large, ...) with different optimizations on and
off (split-index, libpcre2, BLK_SHA1, ...)

With this series and a config file like:

$ cat perf.conf
[perf]
        dirsOrRevs = v2.12.0 v2.13.0
        repeatCount = 10
	sendToCodespeed = http://localhost:8000
	repoName = Git repo
[perf "with libpcre"]
        makeOpts = "DEVELOPER=1 USE_LIBPCRE=YesPlease"
[perf "without libpcre"]
        makeOpts = "DEVELOPER=1"

One should be able to just launch:

$ ./run --config perf.conf p7810-grep.sh

and then get nice graphs in a Codespeed instance running on
http://localhost:8000.

Caveat
~~~~~~

For now one has to create the "Git repo" environment in the Codespeed
admin interface. (We send the perf.repoName config variable in the
"environment" Codespeed field.) This is because Codespeed requires the
environment fields to be created and does not provide a simple way to
create these fields programmatically.

I might try to work around this problem in the future.

Links
~~~~~

This patch series is available here:

https://github.com/chriscool/git/commits/codespeed

The cc/perf-run-config patch series was discussed here:

v1: https://public-inbox.org/git/20170713065050.19215-1-chriscool@tuxfamily.org/
v2: https://public-inbox.org/git/CAP8UFD2j-UFh+9awz91gtZ-jusq7EUOExMgURO59vpf29jXS4A@mail.gmail.com/

Christian Couder (8):
  perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}
  perf/aggregate: refactor printing results
  perf/aggregate: implement codespeed JSON output
  perf/run: use $default_value instead of $4
  perf/run: add conf_opts argument to get_var_from_env_or_config()
  perf/run: learn about perf.codespeedOutput
  perf/run: learn to send output to codespeed server
  perf/run: read GIT_TEST_REPO_NAME from perf.repoName

 t/perf/aggregate.perl | 164 +++++++++++++++++++++++++++++++++++---------------
 t/perf/run            |  29 +++++++--
 2 files changed, 140 insertions(+), 53 deletions(-)

-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH 1/8] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 15:13 ` [PATCH 2/8] perf/aggregate: refactor printing results Christian Couder
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/aggregate.perl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
index e401208488..769d418708 100755
--- a/t/perf/aggregate.perl
+++ b/t/perf/aggregate.perl
@@ -70,7 +70,7 @@ if (not @tests) {
 }
 
 my $resultsdir = "test-results";
-if ($ENV{GIT_PERF_SUBSECTION} ne "") {
+if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne "") {
 	$resultsdir .= "/" . $ENV{GIT_PERF_SUBSECTION};
 }
 
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 2/8] perf/aggregate: refactor printing results
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
  2017-12-13 15:13 ` [PATCH 1/8] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION} Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 15:13 ` [PATCH 3/8] perf/aggregate: implement codespeed JSON output Christian Couder
                   ` (6 subsequent siblings)
  8 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/aggregate.perl | 96 +++++++++++++++++++++++++++------------------------
 1 file changed, 50 insertions(+), 46 deletions(-)

diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
index 769d418708..3609cb5dc3 100755
--- a/t/perf/aggregate.perl
+++ b/t/perf/aggregate.perl
@@ -100,13 +100,6 @@ sub read_descr {
 	return $line;
 }
 
-my %descrs;
-my $descrlen = 4; # "Test"
-for my $t (@subtests) {
-	$descrs{$t} = $shorttests{$t}.": ".read_descr("$resultsdir/$t.descr");
-	$descrlen = length $descrs{$t} if length $descrs{$t}>$descrlen;
-}
-
 sub have_duplicate {
 	my %seen;
 	for (@_) {
@@ -122,54 +115,65 @@ sub have_slash {
 	return 0;
 }
 
-my %newdirabbrevs = %dirabbrevs;
-while (!have_duplicate(values %newdirabbrevs)) {
-	%dirabbrevs = %newdirabbrevs;
-	last if !have_slash(values %dirabbrevs);
-	%newdirabbrevs = %dirabbrevs;
-	for (values %newdirabbrevs) {
-		s{^[^/]*/}{};
+sub print_default_results {
+	my %descrs;
+	my $descrlen = 4; # "Test"
+	for my $t (@subtests) {
+		$descrs{$t} = $shorttests{$t}.": ".read_descr("$resultsdir/$t.descr");
+		$descrlen = length $descrs{$t} if length $descrs{$t}>$descrlen;
 	}
-}
 
-my %times;
-my @colwidth = ((0)x@dirs);
-for my $i (0..$#dirs) {
-	my $d = $dirs[$i];
-	my $w = length (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});
-	$colwidth[$i] = $w if $w > $colwidth[$i];
-}
-for my $t (@subtests) {
-	my $firstr;
+	my %newdirabbrevs = %dirabbrevs;
+	while (!have_duplicate(values %newdirabbrevs)) {
+		%dirabbrevs = %newdirabbrevs;
+		last if !have_slash(values %dirabbrevs);
+		%newdirabbrevs = %dirabbrevs;
+		for (values %newdirabbrevs) {
+			s{^[^/]*/}{};
+		}
+	}
+
+	my %times;
+	my @colwidth = ((0)x@dirs);
 	for my $i (0..$#dirs) {
 		my $d = $dirs[$i];
-		$times{$prefixes{$d}.$t} = [get_times("$resultsdir/$prefixes{$d}$t.times")];
-		my ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};
-		my $w = length format_times($r,$u,$s,$firstr);
+		my $w = length (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});
 		$colwidth[$i] = $w if $w > $colwidth[$i];
-		$firstr = $r unless defined $firstr;
 	}
-}
-my $totalwidth = 3*@dirs+$descrlen;
-$totalwidth += $_ for (@colwidth);
-
-binmode STDOUT, ":utf8" or die "PANIC on binmode: $!";
+	for my $t (@subtests) {
+		my $firstr;
+		for my $i (0..$#dirs) {
+			my $d = $dirs[$i];
+			$times{$prefixes{$d}.$t} = [get_times("$resultsdir/$prefixes{$d}$t.times")];
+			my ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};
+			my $w = length format_times($r,$u,$s,$firstr);
+			$colwidth[$i] = $w if $w > $colwidth[$i];
+			$firstr = $r unless defined $firstr;
+		}
+	}
+	my $totalwidth = 3*@dirs+$descrlen;
+	$totalwidth += $_ for (@colwidth);
 
-printf "%-${descrlen}s", "Test";
-for my $i (0..$#dirs) {
-	my $d = $dirs[$i];
-	printf "   %-$colwidth[$i]s", (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});
-}
-print "\n";
-print "-"x$totalwidth, "\n";
-for my $t (@subtests) {
-	printf "%-${descrlen}s", $descrs{$t};
-	my $firstr;
+	printf "%-${descrlen}s", "Test";
 	for my $i (0..$#dirs) {
 		my $d = $dirs[$i];
-		my ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};
-		printf "   %-$colwidth[$i]s", format_times($r,$u,$s,$firstr);
-		$firstr = $r unless defined $firstr;
+		printf "   %-$colwidth[$i]s", (exists $dirabbrevs{$d} ? $dirabbrevs{$d} : $dirnames{$d});
 	}
 	print "\n";
+	print "-"x$totalwidth, "\n";
+	for my $t (@subtests) {
+		printf "%-${descrlen}s", $descrs{$t};
+		my $firstr;
+		for my $i (0..$#dirs) {
+			my $d = $dirs[$i];
+			my ($r,$u,$s) = @{$times{$prefixes{$d}.$t}};
+			printf "   %-$colwidth[$i]s", format_times($r,$u,$s,$firstr);
+			$firstr = $r unless defined $firstr;
+		}
+		print "\n";
+	}
 }
+
+binmode STDOUT, ":utf8" or die "PANIC on binmode: $!";
+
+print_default_results();
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 3/8] perf/aggregate: implement codespeed JSON output
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
  2017-12-13 15:13 ` [PATCH 1/8] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION} Christian Couder
  2017-12-13 15:13 ` [PATCH 2/8] perf/aggregate: refactor printing results Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 18:25   ` Eric Sunshine
  2017-12-13 20:33   ` Junio C Hamano
  2017-12-13 15:13 ` [PATCH 4/8] perf/run: use $default_value instead of $4 Christian Couder
                   ` (5 subsequent siblings)
  8 siblings, 2 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Codespeed (https://github.com/tobami/codespeed/) is an open source
project that can be used to track how some software performs over
time. It stores performance test results in a database and can show
nice graphs and charts on a web interface.

As it can be interesting to Codespeed to see how Git performance
evolves over time and releases, let's implement a Codespeed output
in "perf/aggregate.perl".

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/aggregate.perl | 68 +++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 66 insertions(+), 2 deletions(-)

diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
index 3609cb5dc3..6e15f62a9e 100755
--- a/t/perf/aggregate.perl
+++ b/t/perf/aggregate.perl
@@ -35,10 +35,15 @@ sub format_times {
 	return $out;
 }
 
-my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests);
+my (@dirs, %dirnames, %dirabbrevs, %prefixes, @tests, $codespeed);
 while (scalar @ARGV) {
 	my $arg = $ARGV[0];
 	my $dir;
+	if ($arg eq "--codespeed") {
+		$codespeed = 1;
+		shift @ARGV;
+		next;
+	}
 	last if -f $arg or $arg eq "--";
 	if (! -d $arg) {
 		my $rev = Git::command_oneline(qw(rev-parse --verify), $arg);
@@ -70,8 +75,10 @@ if (not @tests) {
 }
 
 my $resultsdir = "test-results";
+my $results_section = "";
 if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne "") {
 	$resultsdir .= "/" . $ENV{GIT_PERF_SUBSECTION};
+	$results_section = $ENV{GIT_PERF_SUBSECTION};
 }
 
 my @subtests;
@@ -174,6 +181,63 @@ sub print_default_results {
 	}
 }
 
+sub print_codespeed_results {
+	my ($results_section) = @_;
+
+	my $project = "Git";
+
+	my $executable;
+	if ($results_section eq "") {
+		$executable = `uname -o -p`;
+	} else {
+		$executable = $results_section;
+		chomp $executable;
+	}
+
+	my $environment;
+	if (exists $ENV{GIT_TEST_REPO_NAME} and $ENV{GIT_TEST_REPO_NAME} ne "") {
+		$environment = $ENV{GIT_TEST_REPO_NAME};
+	} elsif (exists $ENV{GIT_TEST_INSTALLED} and $ENV{GIT_TEST_INSTALLED} ne "") {
+		$environment = $ENV{GIT_TEST_INSTALLED};
+		$environment =~ s|/bin-wrappers$||;
+	} else {
+		$environment = `uname -r`;
+		chomp $environment;
+	}
+
+	my @data;
+
+	for my $t (@subtests) {
+		for my $d (@dirs) {
+			my $commitid = $prefixes{$d};
+			$commitid =~ s/^build_//;
+			$commitid =~ s/\.$//;
+			my ($result_value, $u, $s) = get_times("$resultsdir/$prefixes{$d}$t.times");
+
+			my %vals = (
+				"commitid" => $commitid,
+				"project" => $project,
+				"branch" => $dirnames{$d},
+				"executable" => $executable,
+				"benchmark" => $shorttests{$t} . " " . read_descr("$resultsdir/$t.descr"),
+				"environment" => $environment,
+				"result_value" => $result_value,
+			    );
+			push @data, \%vals;
+		}
+	}
+
+	#use Data::Dumper qw/ Dumper /;
+	#print Dumper(\@data);
+
+	use JSON;
+	print to_json(\@data, {utf8 => 1, pretty => 1}), "\n";
+}
+
 binmode STDOUT, ":utf8" or die "PANIC on binmode: $!";
 
-print_default_results();
+if ($codespeed) {
+	print_codespeed_results($results_section);
+} else {
+	print_default_results();
+}
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 4/8] perf/run: use $default_value instead of $4
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
                   ` (2 preceding siblings ...)
  2017-12-13 15:13 ` [PATCH 3/8] perf/aggregate: implement codespeed JSON output Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 20:40   ` Junio C Hamano
  2017-12-13 15:13 ` [PATCH 5/8] perf/run: add conf_opts argument to get_var_from_env_or_config() Christian Couder
                   ` (4 subsequent siblings)
  8 siblings, 1 reply; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/run | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/t/perf/run b/t/perf/run
index 43e4de49ef..bbd703dc4f 100755
--- a/t/perf/run
+++ b/t/perf/run
@@ -105,7 +105,7 @@ get_var_from_env_or_config () {
 	env_var="$1"
 	conf_sec="$2"
 	conf_var="$3"
-	# $4 can be set to a default value
+	default_value="$4" # optional
 
 	# Do nothing if the env variable is already set
 	eval "test -z \"\${$env_var+x}\"" || return
@@ -123,7 +123,7 @@ get_var_from_env_or_config () {
 	conf_value=$(git config -f "$GIT_PERF_CONFIG_FILE" "$var") &&
 	eval "$env_var=\"$conf_value\"" && return
 
-	test -n "${4+x}" && eval "$env_var=\"$4\""
+	test -n "${default_value+x}" && eval "$env_var=\"$default_value\""
 }
 
 run_subsection () {
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 5/8] perf/run: add conf_opts argument to get_var_from_env_or_config()
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
                   ` (3 preceding siblings ...)
  2017-12-13 15:13 ` [PATCH 4/8] perf/run: use $default_value instead of $4 Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 15:13 ` [PATCH 6/8] perf/run: learn about perf.codespeedOutput Christian Couder
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/run | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/t/perf/run b/t/perf/run
index bbd703dc4f..04ea5090f9 100755
--- a/t/perf/run
+++ b/t/perf/run
@@ -106,6 +106,7 @@ get_var_from_env_or_config () {
 	conf_sec="$2"
 	conf_var="$3"
 	default_value="$4" # optional
+	conf_opts="$5" # optional
 
 	# Do nothing if the env variable is already set
 	eval "test -z \"\${$env_var+x}\"" || return
@@ -116,11 +117,11 @@ get_var_from_env_or_config () {
 	if test -n "$GIT_PERF_SUBSECTION"
 	then
 		var="$conf_sec.$GIT_PERF_SUBSECTION.$conf_var"
-		conf_value=$(git config -f "$GIT_PERF_CONFIG_FILE" "$var") &&
+		conf_value=$(git config $conf_opts -f "$GIT_PERF_CONFIG_FILE" "$var") &&
 		eval "$env_var=\"$conf_value\"" && return
 	fi
 	var="$conf_sec.$conf_var"
-	conf_value=$(git config -f "$GIT_PERF_CONFIG_FILE" "$var") &&
+	conf_value=$(git config $conf_opts -f "$GIT_PERF_CONFIG_FILE" "$var") &&
 	eval "$env_var=\"$conf_value\"" && return
 
 	test -n "${default_value+x}" && eval "$env_var=\"$default_value\""
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 6/8] perf/run: learn about perf.codespeedOutput
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
                   ` (4 preceding siblings ...)
  2017-12-13 15:13 ` [PATCH 5/8] perf/run: add conf_opts argument to get_var_from_env_or_config() Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 15:13 ` [PATCH 7/8] perf/run: learn to send output to codespeed server Christian Couder
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/run | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/t/perf/run b/t/perf/run
index 04ea5090f9..4454a2713d 100755
--- a/t/perf/run
+++ b/t/perf/run
@@ -144,10 +144,15 @@ run_subsection () {
 		set -- . "$@"
 	fi
 
+	codespeed_opt=
+	test "$GIT_PERF_CODESPEED_OUTPUT" = "true" && codespeed_opt="--codespeed"
+
 	run_dirs "$@"
-	./aggregate.perl "$@"
+	./aggregate.perl $codespeed_opt "$@"
 }
 
+get_var_from_env_or_config "GIT_PERF_CODESPEED_OUTPUT" "perf" "codespeedOutput" "" --bool
+
 cd "$(dirname $0)"
 . ../../GIT-BUILD-OPTIONS
 
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 7/8] perf/run: learn to send output to codespeed server
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
                   ` (5 preceding siblings ...)
  2017-12-13 15:13 ` [PATCH 6/8] perf/run: learn about perf.codespeedOutput Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 15:13 ` [PATCH 8/8] perf/run: read GIT_TEST_REPO_NAME from perf.repoName Christian Couder
  2017-12-13 16:02 ` [PATCH 0/8] Codespeed perf results Philip Oakley
  8 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/run | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/t/perf/run b/t/perf/run
index 4454a2713d..7b7011f19b 100755
--- a/t/perf/run
+++ b/t/perf/run
@@ -148,10 +148,20 @@ run_subsection () {
 	test "$GIT_PERF_CODESPEED_OUTPUT" = "true" && codespeed_opt="--codespeed"
 
 	run_dirs "$@"
-	./aggregate.perl $codespeed_opt "$@"
+
+	if test -z "$GIT_PERF_SEND_TO_CODESPEED"
+	then
+		./aggregate.perl $codespeed_opt "$@"
+	else
+		json_res_file="test-results/$GIT_PERF_SUBSECTION/aggregate.json"
+		./aggregate.perl --codespeed "$@" | tee "$json_res_file"
+		send_data_url="$GIT_PERF_SEND_TO_CODESPEED/result/add/json/"
+		curl -v --request POST --data-urlencode "json=$(cat "$json_res_file")" "$send_data_url"
+	fi
 }
 
 get_var_from_env_or_config "GIT_PERF_CODESPEED_OUTPUT" "perf" "codespeedOutput" "" --bool
+get_var_from_env_or_config "GIT_PERF_SEND_TO_CODESPEED" "perf" "sendToCodespeed"
 
 cd "$(dirname $0)"
 . ../../GIT-BUILD-OPTIONS
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 8/8] perf/run: read GIT_TEST_REPO_NAME from perf.repoName
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
                   ` (6 preceding siblings ...)
  2017-12-13 15:13 ` [PATCH 7/8] perf/run: learn to send output to codespeed server Christian Couder
@ 2017-12-13 15:13 ` Christian Couder
  2017-12-13 16:02 ` [PATCH 0/8] Codespeed perf results Philip Oakley
  8 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 15:13 UTC (permalink / raw)
  To: git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
---
 t/perf/run | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/t/perf/run b/t/perf/run
index 7b7011f19b..279c2d41f6 100755
--- a/t/perf/run
+++ b/t/perf/run
@@ -137,6 +137,9 @@ run_subsection () {
 	get_var_from_env_or_config "GIT_PERF_MAKE_COMMAND" "perf" "makeCommand"
 	get_var_from_env_or_config "GIT_PERF_MAKE_OPTS" "perf" "makeOpts"
 
+	get_var_from_env_or_config "GIT_TEST_REPO_NAME" "perf" "repoName"
+	export GIT_PERF_REPO_NAME
+
 	GIT_PERF_AGGREGATING_LATER=t
 	export GIT_PERF_AGGREGATING_LATER
 
-- 
2.15.1.361.g8b07d831d0


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH 0/8] Codespeed perf results
  2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
                   ` (7 preceding siblings ...)
  2017-12-13 15:13 ` [PATCH 8/8] perf/run: read GIT_TEST_REPO_NAME from perf.repoName Christian Couder
@ 2017-12-13 16:02 ` Philip Oakley
  2017-12-13 16:18   ` Christian Couder
  8 siblings, 1 reply; 20+ messages in thread
From: Philip Oakley @ 2017-12-13 16:02 UTC (permalink / raw)
  To: Christian Couder, git
  Cc: Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

From: "Christian Couder" <christian.couder@gmail.com>
> This patch series is built on top of cc/perf-run-config which recently
> graduated to master.
>
> It makes it possible to send perf results to a Codespeed server. See
> https://github.com/tobami/codespeed/ and web sites like
> http://speed.pypy.org/ which are using Codespeed.
>
> The end goal would be to have such a server always available to track
> how the different git commands perform over time on different kind of
> repos (small, medium, large, ...) with different optimizations on and
> off (split-index, libpcre2, BLK_SHA1, ...)

Dumb question: is this expected to also be able to do a retrospective on the 
performance of appropriate past releases? That would allow immediate 
performance comparisons, rather than needing to wait for a few releases to 
see the trends.

Philip

>
> With this series and a config file like:
>
> $ cat perf.conf
> [perf]
>        dirsOrRevs = v2.12.0 v2.13.0
>        repeatCount = 10
> sendToCodespeed = http://localhost:8000
> repoName = Git repo
> [perf "with libpcre"]
>        makeOpts = "DEVELOPER=1 USE_LIBPCRE=YesPlease"
> [perf "without libpcre"]
>        makeOpts = "DEVELOPER=1"
>
> One should be able to just launch:
>
> $ ./run --config perf.conf p7810-grep.sh
>
> and then get nice graphs in a Codespeed instance running on
> http://localhost:8000.
>
> Caveat
> ~~~~~~
>
> For now one has to create the "Git repo" environment in the Codespeed
> admin interface. (We send the perf.repoName config variable in the
> "environment" Codespeed field.) This is because Codespeed requires the
> environment fields to be created and does not provide a simple way to
> create these fields programmatically.
>
> I might try to work around this problem in the future.
>
> Links
> ~~~~~
>
> This patch series is available here:
>
> https://github.com/chriscool/git/commits/codespeed
>
> The cc/perf-run-config patch series was discussed here:
>
> v1: 
> https://public-inbox.org/git/20170713065050.19215-1-chriscool@tuxfamily.org/
> v2: 
> https://public-inbox.org/git/CAP8UFD2j-UFh+9awz91gtZ-jusq7EUOExMgURO59vpf29jXS4A@mail.gmail.com/
>
> Christian Couder (8):
>  perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION}
>  perf/aggregate: refactor printing results
>  perf/aggregate: implement codespeed JSON output
>  perf/run: use $default_value instead of $4
>  perf/run: add conf_opts argument to get_var_from_env_or_config()
>  perf/run: learn about perf.codespeedOutput
>  perf/run: learn to send output to codespeed server
>  perf/run: read GIT_TEST_REPO_NAME from perf.repoName
>
> t/perf/aggregate.perl | 164 
> +++++++++++++++++++++++++++++++++++---------------
> t/perf/run            |  29 +++++++--
> 2 files changed, 140 insertions(+), 53 deletions(-)
>
> -- 
> 2.15.1.361.g8b07d831d0
> 


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 0/8] Codespeed perf results
  2017-12-13 16:02 ` [PATCH 0/8] Codespeed perf results Philip Oakley
@ 2017-12-13 16:18   ` Christian Couder
  0 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-13 16:18 UTC (permalink / raw)
  To: Philip Oakley
  Cc: git, Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

On Wed, Dec 13, 2017 at 5:02 PM, Philip Oakley <philipoakley@iee.org> wrote:

>> The end goal would be to have such a server always available to track
>> how the different git commands perform over time on different kind of
>> repos (small, medium, large, ...) with different optimizations on and
>> off (split-index, libpcre2, BLK_SHA1, ...)
>
> Dumb question: is this expected to also be able to do a retrospective on the
> performance of appropriate past releases? That would allow immediate
> performance comparisons, rather than needing to wait for a few releases to
> see the trends.

Yeah, sure. For example in the perf.conf file above there is
"dirsOrRevs = v2.12.0 v2.13.0" which means that tests will be
performed on v2.12.0 and v2.13.0.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/8] perf/aggregate: implement codespeed JSON output
  2017-12-13 15:13 ` [PATCH 3/8] perf/aggregate: implement codespeed JSON output Christian Couder
@ 2017-12-13 18:25   ` Eric Sunshine
  2017-12-14  8:47     ` Christian Couder
  2017-12-13 20:33   ` Junio C Hamano
  1 sibling, 1 reply; 20+ messages in thread
From: Eric Sunshine @ 2017-12-13 18:25 UTC (permalink / raw)
  To: Christian Couder
  Cc: Git List, Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

On Wed, Dec 13, 2017 at 10:13 AM, Christian Couder
<christian.couder@gmail.com> wrote:
> Codespeed (https://github.com/tobami/codespeed/) is an open source
> project that can be used to track how some software performs over
> time. It stores performance test results in a database and can show
> nice graphs and charts on a web interface.
>
> As it can be interesting to Codespeed to see how Git performance
> evolves over time and releases, let's implement a Codespeed output
> in "perf/aggregate.perl".
>
> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> ---
> diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
> @@ -174,6 +181,63 @@ sub print_default_results {
> +sub print_codespeed_results {
> +       my ($results_section) = @_;
> +
> +       my $project = "Git";
> +
> +       my $executable;
> +       if ($results_section eq "") {
> +               $executable = `uname -o -p`;

Option -o is not recognized on MacOS and, on at least a couple of my
Linux installations, -p returns only "unknown".

A combination, on the other hand, which seems to work nicely on MacOS,
Linux, and BSD is: uname -s -m

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/8] perf/aggregate: implement codespeed JSON output
  2017-12-13 15:13 ` [PATCH 3/8] perf/aggregate: implement codespeed JSON output Christian Couder
  2017-12-13 18:25   ` Eric Sunshine
@ 2017-12-13 20:33   ` Junio C Hamano
  2017-12-14  8:57     ` Christian Couder
  1 sibling, 1 reply; 20+ messages in thread
From: Junio C Hamano @ 2017-12-13 20:33 UTC (permalink / raw)
  To: Christian Couder
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Christian Couder <christian.couder@gmail.com> writes:

>  my $resultsdir = "test-results";
> +my $results_section = "";
>  if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne "") {
>  	$resultsdir .= "/" . $ENV{GIT_PERF_SUBSECTION};
> +	$results_section = $ENV{GIT_PERF_SUBSECTION};
>  }

...

> +	my $executable;
> +	if ($results_section eq "") {
> +		$executable = `uname -o -p`;
> +	} else {
> +		$executable = $results_section;
> +		chomp $executable;
> +	}

Aside from portability of 'uname -o' Eric raised, I wonder if the
platform information is still useful even when perf-subsection is
specified.  With the above code, we can identify that a single
result is for (say) MacOS only when we are not limiting to a single
subsection, but wouldn't it be equally a valid desire to be able to
track performance figures for a single subsection over time and
being able to say "On MacOS, subsection A's performance dropped
between release X and X+1 quite a bit, but on Linux x86-64, there
was no such change" or somesuch?

IOW, shouldn't the "executable" label always contain the platform
information, plus an optional subsection info when (and only when)
the result is limited to a subsection?

By the way, $results_section that is not an empty string at this
point must have come from $ENV{GIT_PERF_SUBSECTION}, and from the
way the environment variable is used in t/perf/run, e.g.

		(
			GIT_PERF_SUBSECTION="$subsec"
			export GIT_PERF_SUBSECTION
			echo "======== Run for subsection '$GIT_PERF_SUBSECTION' ========"
			run_subsection "$@"
		)

I do not think we expect it to have a trailing LF.  What's that
chomp doing there?

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 4/8] perf/run: use $default_value instead of $4
  2017-12-13 15:13 ` [PATCH 4/8] perf/run: use $default_value instead of $4 Christian Couder
@ 2017-12-13 20:40   ` Junio C Hamano
  2017-12-13 20:54     ` Junio C Hamano
  2017-12-14  8:59     ` Christian Couder
  0 siblings, 2 replies; 20+ messages in thread
From: Junio C Hamano @ 2017-12-13 20:40 UTC (permalink / raw)
  To: Christian Couder
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Christian Couder <christian.couder@gmail.com> writes:

> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
> ---
>  t/perf/run | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/t/perf/run b/t/perf/run
> index 43e4de49ef..bbd703dc4f 100755
> --- a/t/perf/run
> +++ b/t/perf/run
> @@ -105,7 +105,7 @@ get_var_from_env_or_config () {
>  	env_var="$1"
>  	conf_sec="$2"
>  	conf_var="$3"
> -	# $4 can be set to a default value
> +	default_value="$4" # optional
>  
>  	# Do nothing if the env variable is already set
>  	eval "test -z \"\${$env_var+x}\"" || return
> @@ -123,7 +123,7 @@ get_var_from_env_or_config () {
>  	conf_value=$(git config -f "$GIT_PERF_CONFIG_FILE" "$var") &&
>  	eval "$env_var=\"$conf_value\"" && return
>  
> -	test -n "${4+x}" && eval "$env_var=\"$4\""
> +	test -n "${default_value+x}" && eval "$env_var=\"$default_value\""

This conversion changes the behaviour.  Because default_value is
always set by your change in the previous hunk, we end up always
doing this eval.

The original says "If $4 is set, then ${4+x} becomes x and if $4 is
not set, ${4+x} is empty, so let's check if ${4+x} is a non-empty
string to see if $4 is set.  If ${4+x} is a non-empty string, that
means $4 was set so we do the eval.

If you want to be able to use this helper to specify a default value
of an empty string (which the orignal that used $4 did), then the
previous hunk must be corrected so that it does not unconditionally
set default_value to $4.  Perhaps like

	if test -n "${4+x}"
	then
		default_value=$4
	else
		unset default_value || :
	fi

or something.

>  }
>  
>  run_subsection () {

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 4/8] perf/run: use $default_value instead of $4
  2017-12-13 20:40   ` Junio C Hamano
@ 2017-12-13 20:54     ` Junio C Hamano
  2017-12-14  9:00       ` Christian Couder
  2017-12-14  8:59     ` Christian Couder
  1 sibling, 1 reply; 20+ messages in thread
From: Junio C Hamano @ 2017-12-13 20:54 UTC (permalink / raw)
  To: Christian Couder
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Junio C Hamano <gitster@pobox.com> writes:

> If you want to be able to use this helper to specify a default value
> of an empty string (which the orignal that used $4 did), then the
> previous hunk must be corrected so that it does not unconditionally
> set default_value to $4.  Perhaps like
>
> 	if test -n "${4+x}"
> 	then
> 		default_value=$4
> 	else
> 		unset default_value || :
> 	fi
>
> or something.

And if you do not care about passing an empty string as the default
value, then the earlier hunk can stay the same

	default_value=$4

but the eval part should become something like

	test -n "$default_value" && eval ...

Given that you are planning to add yet another optional argument to
the helper, and when you pass that variable, you'd probably need to
pass "" as the "default" that is not exported, perhaps this "give up
ability to pass an empty string as the default" approach may be the
only thing you could do.


^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/8] perf/aggregate: implement codespeed JSON output
  2017-12-13 18:25   ` Eric Sunshine
@ 2017-12-14  8:47     ` Christian Couder
  0 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-14  8:47 UTC (permalink / raw)
  To: Eric Sunshine
  Cc: Git List, Junio C Hamano, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

On Wed, Dec 13, 2017 at 7:25 PM, Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Wed, Dec 13, 2017 at 10:13 AM, Christian Couder
> <christian.couder@gmail.com> wrote:
>> Codespeed (https://github.com/tobami/codespeed/) is an open source
>> project that can be used to track how some software performs over
>> time. It stores performance test results in a database and can show
>> nice graphs and charts on a web interface.
>>
>> As it can be interesting to Codespeed to see how Git performance
>> evolves over time and releases, let's implement a Codespeed output
>> in "perf/aggregate.perl".
>>
>> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
>> ---
>> diff --git a/t/perf/aggregate.perl b/t/perf/aggregate.perl
>> @@ -174,6 +181,63 @@ sub print_default_results {
>> +sub print_codespeed_results {
>> +       my ($results_section) = @_;
>> +
>> +       my $project = "Git";
>> +
>> +       my $executable;
>> +       if ($results_section eq "") {
>> +               $executable = `uname -o -p`;
>
> Option -o is not recognized on MacOS and, on at least a couple of my
> Linux installations, -p returns only "unknown".
>
> A combination, on the other hand, which seems to work nicely on MacOS,
> Linux, and BSD is: uname -s -m

Ok I will take a look at this. Thanks!

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/8] perf/aggregate: implement codespeed JSON output
  2017-12-13 20:33   ` Junio C Hamano
@ 2017-12-14  8:57     ` Christian Couder
  2017-12-14 17:31       ` Junio C Hamano
  0 siblings, 1 reply; 20+ messages in thread
From: Christian Couder @ 2017-12-14  8:57 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

On Wed, Dec 13, 2017 at 9:33 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Christian Couder <christian.couder@gmail.com> writes:
>
>>  my $resultsdir = "test-results";
>> +my $results_section = "";
>>  if (exists $ENV{GIT_PERF_SUBSECTION} and $ENV{GIT_PERF_SUBSECTION} ne "") {
>>       $resultsdir .= "/" . $ENV{GIT_PERF_SUBSECTION};
>> +     $results_section = $ENV{GIT_PERF_SUBSECTION};
>>  }
>
> ...
>
>> +     my $executable;
>> +     if ($results_section eq "") {
>> +             $executable = `uname -o -p`;
>> +     } else {
>> +             $executable = $results_section;
>> +             chomp $executable;
>> +     }
>
> Aside from portability of 'uname -o' Eric raised, I wonder if the
> platform information is still useful even when perf-subsection is
> specified.  With the above code, we can identify that a single
> result is for (say) MacOS only when we are not limiting to a single
> subsection, but wouldn't it be equally a valid desire to be able to
> track performance figures for a single subsection over time and
> being able to say "On MacOS, subsection A's performance dropped
> between release X and X+1 quite a bit, but on Linux x86-64, there
> was no such change" or somesuch?

Yeah, I agree that it would be useful. Unfortunately it looks like the
number of fields in Codespeed is limited and I am not sure what will
be more important for people in general. Another option would be to
have "MacOS" in the "environment" field. Or maybe there is a need for
a generic way to customize this. For now I just tried to come up with
something sensible for what AEvar and me want to do.

> IOW, shouldn't the "executable" label always contain the platform
> information, plus an optional subsection info when (and only when)
> the result is limited to a subsection?
>
> By the way, $results_section that is not an empty string at this
> point must have come from $ENV{GIT_PERF_SUBSECTION}, and from the
> way the environment variable is used in t/perf/run, e.g.
>
>                 (
>                         GIT_PERF_SUBSECTION="$subsec"
>                         export GIT_PERF_SUBSECTION
>                         echo "======== Run for subsection '$GIT_PERF_SUBSECTION' ========"
>                         run_subsection "$@"
>                 )
>
> I do not think we expect it to have a trailing LF.  What's that
> chomp doing there?

It's a silly mistake I made when I reorganized the patches just before
sending them. The chomp should be after "$executable = `uname -o -p`;"
(so in the other branch of the "if ... else"). I will fix this in the
next version.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 4/8] perf/run: use $default_value instead of $4
  2017-12-13 20:40   ` Junio C Hamano
  2017-12-13 20:54     ` Junio C Hamano
@ 2017-12-14  8:59     ` Christian Couder
  1 sibling, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-14  8:59 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

On Wed, Dec 13, 2017 at 9:40 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Christian Couder <christian.couder@gmail.com> writes:
>
>> Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
>> ---
>>  t/perf/run | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/t/perf/run b/t/perf/run
>> index 43e4de49ef..bbd703dc4f 100755
>> --- a/t/perf/run
>> +++ b/t/perf/run
>> @@ -105,7 +105,7 @@ get_var_from_env_or_config () {
>>       env_var="$1"
>>       conf_sec="$2"
>>       conf_var="$3"
>> -     # $4 can be set to a default value
>> +     default_value="$4" # optional
>>
>>       # Do nothing if the env variable is already set
>>       eval "test -z \"\${$env_var+x}\"" || return
>> @@ -123,7 +123,7 @@ get_var_from_env_or_config () {
>>       conf_value=$(git config -f "$GIT_PERF_CONFIG_FILE" "$var") &&
>>       eval "$env_var=\"$conf_value\"" && return
>>
>> -     test -n "${4+x}" && eval "$env_var=\"$4\""
>> +     test -n "${default_value+x}" && eval "$env_var=\"$default_value\""
>
> This conversion changes the behaviour.  Because default_value is
> always set by your change in the previous hunk, we end up always
> doing this eval.
>
> The original says "If $4 is set, then ${4+x} becomes x and if $4 is
> not set, ${4+x} is empty, so let's check if ${4+x} is a non-empty
> string to see if $4 is set.  If ${4+x} is a non-empty string, that
> means $4 was set so we do the eval.
>
> If you want to be able to use this helper to specify a default value
> of an empty string (which the orignal that used $4 did), then the
> previous hunk must be corrected so that it does not unconditionally
> set default_value to $4.  Perhaps like
>
>         if test -n "${4+x}"
>         then
>                 default_value=$4
>         else
>                 unset default_value || :
>         fi
>
> or something.

Yeah, you are right I will fix this.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 4/8] perf/run: use $default_value instead of $4
  2017-12-13 20:54     ` Junio C Hamano
@ 2017-12-14  9:00       ` Christian Couder
  0 siblings, 0 replies; 20+ messages in thread
From: Christian Couder @ 2017-12-14  9:00 UTC (permalink / raw)
  To: Junio C Hamano
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

On Wed, Dec 13, 2017 at 9:54 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:
>
>> If you want to be able to use this helper to specify a default value
>> of an empty string (which the orignal that used $4 did), then the
>> previous hunk must be corrected so that it does not unconditionally
>> set default_value to $4.  Perhaps like
>>
>>       if test -n "${4+x}"
>>       then
>>               default_value=$4
>>       else
>>               unset default_value || :
>>       fi
>>
>> or something.
>
> And if you do not care about passing an empty string as the default
> value, then the earlier hunk can stay the same
>
>         default_value=$4
>
> but the eval part should become something like
>
>         test -n "$default_value" && eval ...
>
> Given that you are planning to add yet another optional argument to
> the helper, and when you pass that variable, you'd probably need to
> pass "" as the "default" that is not exported, perhaps this "give up
> ability to pass an empty string as the default" approach may be the
> only thing you could do.

Yeah, thanks for the explanations and suggestions.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/8] perf/aggregate: implement codespeed JSON output
  2017-12-14  8:57     ` Christian Couder
@ 2017-12-14 17:31       ` Junio C Hamano
  0 siblings, 0 replies; 20+ messages in thread
From: Junio C Hamano @ 2017-12-14 17:31 UTC (permalink / raw)
  To: Christian Couder
  Cc: git, Jeff King, Thomas Rast,
	Ævar Arnfjörð Bjarmason, Christian Couder

Christian Couder <christian.couder@gmail.com> writes:

>> Aside from portability of 'uname -o' Eric raised, I wonder if the
>> platform information is still useful even when perf-subsection is
>> specified.  With the above code, we can identify that a single
>> result is for (say) MacOS only when we are not limiting to a single
>> subsection, but wouldn't it be equally a valid desire to be able to
>> track performance figures for a single subsection over time and
>> being able to say "On MacOS, subsection A's performance dropped
>> between release X and X+1 quite a bit, but on Linux x86-64, there
>> was no such change" or somesuch?
>
> Yeah, I agree that it would be useful. Unfortunately it looks like the
> number of fields in Codespeed is limited and I am not sure what will
> be more important for people in general.

Is there a reason why such textual labels meant for human
consumption need to be two (or more) separate fields?  Would simply
concatenating them into a single string defeat whatever you wanted
to achieve by having this information in $executable in the first
place?


^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2017-12-14 17:31 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-12-13 15:13 [PATCH 0/8] Codespeed perf results Christian Couder
2017-12-13 15:13 ` [PATCH 1/8] perf/aggregate: fix checking ENV{GIT_PERF_SUBSECTION} Christian Couder
2017-12-13 15:13 ` [PATCH 2/8] perf/aggregate: refactor printing results Christian Couder
2017-12-13 15:13 ` [PATCH 3/8] perf/aggregate: implement codespeed JSON output Christian Couder
2017-12-13 18:25   ` Eric Sunshine
2017-12-14  8:47     ` Christian Couder
2017-12-13 20:33   ` Junio C Hamano
2017-12-14  8:57     ` Christian Couder
2017-12-14 17:31       ` Junio C Hamano
2017-12-13 15:13 ` [PATCH 4/8] perf/run: use $default_value instead of $4 Christian Couder
2017-12-13 20:40   ` Junio C Hamano
2017-12-13 20:54     ` Junio C Hamano
2017-12-14  9:00       ` Christian Couder
2017-12-14  8:59     ` Christian Couder
2017-12-13 15:13 ` [PATCH 5/8] perf/run: add conf_opts argument to get_var_from_env_or_config() Christian Couder
2017-12-13 15:13 ` [PATCH 6/8] perf/run: learn about perf.codespeedOutput Christian Couder
2017-12-13 15:13 ` [PATCH 7/8] perf/run: learn to send output to codespeed server Christian Couder
2017-12-13 15:13 ` [PATCH 8/8] perf/run: read GIT_TEST_REPO_NAME from perf.repoName Christian Couder
2017-12-13 16:02 ` [PATCH 0/8] Codespeed perf results Philip Oakley
2017-12-13 16:18   ` Christian Couder

Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).