From: Siddhesh Poyarekar <siddhesh@gotplt.org>
To: Naohiro Tamura <naohirot@fujitsu.com>, libc-alpha@sourceware.org
Subject: Re: [PATCH v5] benchtests: Fix validate_benchout.py exceptions
Date: Thu, 16 Sep 2021 09:18:28 +0530 [thread overview]
Message-ID: <430bdf8c-eb5c-72e6-b64e-59b0219669b7@gotplt.org> (raw)
In-Reply-To: <20210916022316.1101289-1-naohirot@fujitsu.com>
On 9/16/21 7:53 AM, Naohiro Tamura wrote:
> This patch fixed validate_benchout.py two exceptions,
> 1) AttributeError
> if benchout_strings.schema.json is specified, and
> 2) json.decoder.JSONDecodeError
> if benchout file is not JSON.
>
> $ ~/glibc/benchtests/scripts/validate_benchout.py bench-memset.out \
> ~/glibc/benchtests/scripts/benchout_strings.schema.json
> Traceback (most recent call last):
> File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in <module>
> sys.exit(main(sys.argv[1:]))
> File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main
> bench.parse_bench(args[0], args[1])
> File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 139, in parse_bench
> do_for_all_timings(bench, lambda b, f, v:
> File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 107, in do_for_all_timings
> if 'timings' not in bench['functions'][func][k].keys():
> AttributeError: 'str' object has no attribute 'keys'
>
> $ ~/glibc/benchtests/scripts/validate_benchout.py bench-math-inlines.out \
> ~/glibc/benchtests/scripts/benchout_strings.schema.json
> Traceback (most recent call last):
> File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in <module>
> sys.exit(main(sys.argv[1:]))
> File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main
> bench.parse_bench(args[0], args[1])
> File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 137, in parse_bench
> bench = json.load(benchfile)
> File "/usr/lib/python3.6/json/__init__.py", line 299, in load
> parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
> File "/usr/lib/python3.6/json/__init__.py", line 354, in loads
> return _default_decoder.decode(s)
> File "/usr/lib/python3.6/json/decoder.py", line 342, in decode
> raise JSONDecodeError("Extra data", s, end)
> json.decoder.JSONDecodeError: Extra data: line 1 column 17 (char 16)
>
> Reviewed-by: Siddhesh Poyarekar <siddhesh@gotplt.org>
Please don't add Reviewed-by until the reviewer specifically adds it
themselves. It is an indicator on patchwork that the patch is ready for
inclusion.
That said, LGTM, I'll push this :)
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
> ---
> benchtests/scripts/compare_bench.py | 4 ++++
> benchtests/scripts/import_bench.py | 2 --
> benchtests/scripts/validate_benchout.py | 6 +++++-
> 3 files changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/benchtests/scripts/compare_bench.py b/benchtests/scripts/compare_bench.py
> index 6fcbd0803808..1a497f8bb61b 100755
> --- a/benchtests/scripts/compare_bench.py
> +++ b/benchtests/scripts/compare_bench.py
> @@ -163,7 +163,11 @@ def plot_graphs(bench1, bench2):
>
> def main(bench1, bench2, schema, threshold, stats):
> bench1 = bench.parse_bench(bench1, schema)
> + bench.do_for_all_timings(bench1, lambda b, f, v:
> + b['functions'][f][v]['timings'].sort())
> bench2 = bench.parse_bench(bench2, schema)
> + bench.do_for_all_timings(bench2, lambda b, f, v:
> + b['functions'][f][v]['timings'].sort())
>
> plot_graphs(bench1, bench2)
>
> diff --git a/benchtests/scripts/import_bench.py b/benchtests/scripts/import_bench.py
> index a799b4e1b7dc..362c990fe153 100644
> --- a/benchtests/scripts/import_bench.py
> +++ b/benchtests/scripts/import_bench.py
> @@ -136,6 +136,4 @@ def parse_bench(filename, schema_filename):
> with open(filename, 'r') as benchfile:
> bench = json.load(benchfile)
> validator.validate(bench, schema)
> - do_for_all_timings(bench, lambda b, f, v:
> - b['functions'][f][v]['timings'].sort())
> return bench
> diff --git a/benchtests/scripts/validate_benchout.py b/benchtests/scripts/validate_benchout.py
> index 47df33ed0252..00d5fa0ee5eb 100755
> --- a/benchtests/scripts/validate_benchout.py
> +++ b/benchtests/scripts/validate_benchout.py
> @@ -73,11 +73,15 @@ def main(args):
>
> except bench.validator.ValidationError as e:
> return print_and_exit("Invalid benchmark output: %s" % e.message,
> - os.EX_DATAERR)
> + os.EX_DATAERR)
>
> except bench.validator.SchemaError as e:
> return print_and_exit("Invalid schema: %s" % e.message, os.EX_DATAERR)
>
> + except json.decoder.JSONDecodeError as e:
> + return print_and_exit("Benchmark output in %s is not JSON." % args[0],
> + os.EX_DATAERR)
> +
> print("Benchmark output in %s is valid." % args[0])
> return os.EX_OK
>
>
next prev parent reply other threads:[~2021-09-16 3:48 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-13 8:22 [PATCH] benchtests: Add memset zero fill benchmark tests Naohiro Tamura via Libc-alpha
2021-07-13 13:50 ` Lucas A. M. Magalhaes via Libc-alpha
2021-07-20 6:31 ` [PATCH v2 0/5] " Naohiro Tamura via Libc-alpha
2021-08-05 7:47 ` [PATCH v3 0/5] benchtests: Add memset zero fill benchmark test Naohiro Tamura via Libc-alpha
2021-08-05 7:49 ` [PATCH v3 1/5] benchtests: Enable scripts/plot_strings.py to read stdin Naohiro Tamura via Libc-alpha
2021-08-05 7:56 ` Siddhesh Poyarekar
2021-09-08 1:46 ` naohirot--- via Libc-alpha
2021-09-08 12:56 ` Siddhesh Poyarekar
2021-09-09 0:22 ` naohirot--- via Libc-alpha
2021-09-13 3:45 ` Siddhesh Poyarekar
2021-08-05 7:50 ` [PATCH v3 2/5] benchtests: Add memset zero fill benchtest Naohiro Tamura via Libc-alpha
2021-09-08 2:03 ` naohirot--- via Libc-alpha
2021-09-10 20:40 ` Lucas A. M. Magalhaes via Libc-alpha
2021-09-13 0:53 ` naohirot--- via Libc-alpha
2021-09-13 14:05 ` Lucas A. M. Magalhaes via Libc-alpha
2021-09-14 0:38 ` [PATCH v4] " Naohiro Tamura via Libc-alpha
2021-09-14 0:44 ` [PATCH v3 2/5] " naohirot--- via Libc-alpha
2021-09-14 14:02 ` Wilco Dijkstra via Libc-alpha
2021-09-15 8:24 ` naohirot--- via Libc-alpha
2021-09-21 1:27 ` naohirot--- via Libc-alpha
2021-09-21 11:09 ` Wilco Dijkstra via Libc-alpha
2021-09-22 1:05 ` [PATCH v5] " Naohiro Tamura via Libc-alpha
2023-02-09 17:23 ` Carlos O'Donell via Libc-alpha
2023-02-10 1:26 ` Siddhesh Poyarekar via Libc-alpha
2021-09-22 1:07 ` [PATCH v3 2/5] " naohirot--- via Libc-alpha
2021-09-28 1:40 ` naohirot--- via Libc-alpha
2021-09-30 0:55 ` Tamura, Naohiro/田村 直� via Libc-alpha
2021-10-18 12:57 ` Lucas A. M. Magalhaes via Libc-alpha
2021-10-20 13:44 ` Wilco Dijkstra via Libc-alpha
2021-10-20 15:35 ` Lucas A. M. Magalhaes via Libc-alpha
2021-10-20 17:47 ` Wilco Dijkstra via Libc-alpha
2021-10-22 13:08 ` Lucas A. M. Magalhaes via Libc-alpha
2021-08-05 7:51 ` [PATCH v3 3/5] benchtests: Remove redundant assert.h Naohiro Tamura via Libc-alpha
2021-09-08 1:59 ` naohirot--- via Libc-alpha
2021-09-13 3:36 ` Siddhesh Poyarekar
2021-08-05 7:51 ` [PATCH v3 4/5] benchtests: Fix validate_benchout.py exceptions Naohiro Tamura via Libc-alpha
2021-09-08 1:55 ` naohirot--- via Libc-alpha
2021-09-13 3:42 ` Siddhesh Poyarekar
2021-09-13 3:50 ` Siddhesh Poyarekar
2021-09-13 13:44 ` [PATCH v4] " Naohiro Tamura via Libc-alpha
2021-09-15 3:23 ` Siddhesh Poyarekar
2021-09-16 1:12 ` naohirot--- via Libc-alpha
2021-09-16 1:41 ` Siddhesh Poyarekar
2021-09-16 2:23 ` [PATCH v5] " Naohiro Tamura via Libc-alpha
2021-09-16 3:48 ` Siddhesh Poyarekar [this message]
2021-09-16 5:23 ` naohirot--- via Libc-alpha
2021-09-16 2:26 ` [PATCH v4] " naohirot--- via Libc-alpha
2021-09-13 13:46 ` [PATCH v3 4/5] " naohirot--- via Libc-alpha
2021-08-05 7:52 ` [PATCH v3 5/5] config: Rename HAVE_BUILTIN_MEMSET macro Naohiro Tamura via Libc-alpha
2021-08-11 20:34 ` Adhemerval Zanella via Libc-alpha
2021-07-20 6:34 ` [PATCH v2 1/5] benchtests: Enable scripts/plot_strings.py to read stdin Naohiro Tamura via Libc-alpha
2021-07-20 6:35 ` [PATCH v2 2/5] benchtests: Add memset zero fill benchtest Naohiro Tamura via Libc-alpha
2021-07-20 16:48 ` Noah Goldstein via Libc-alpha
2021-07-21 12:56 ` naohirot--- via Libc-alpha
2021-07-21 13:07 ` naohirot--- via Libc-alpha
2021-07-21 18:14 ` Noah Goldstein via Libc-alpha
2021-07-21 19:17 ` Wilco Dijkstra via Libc-alpha
2021-07-26 8:42 ` naohirot--- via Libc-alpha
2021-07-26 11:15 ` Wilco Dijkstra via Libc-alpha
2021-07-27 2:24 ` naohirot--- via Libc-alpha
2021-07-27 17:26 ` Wilco Dijkstra via Libc-alpha
2021-07-28 7:27 ` naohirot--- via Libc-alpha
2021-08-04 9:11 ` naohirot--- via Libc-alpha
2021-07-26 8:39 ` naohirot--- via Libc-alpha
2021-07-26 17:22 ` Noah Goldstein via Libc-alpha
2021-07-20 6:35 ` [PATCH v2 3/5] benchtests: Add a script to convert benchout string JSON to CSV Naohiro Tamura via Libc-alpha
2021-07-21 2:41 ` naohirot--- via Libc-alpha
2021-07-27 20:17 ` Joseph Myers
2021-07-29 1:56 ` naohirot--- via Libc-alpha
2021-07-29 4:42 ` Siddhesh Poyarekar
2021-07-30 7:05 ` naohirot--- via Libc-alpha
2021-07-31 10:47 ` Siddhesh Poyarekar
2021-07-20 6:36 ` [PATCH v2 4/5] benchtests: Remove redundant assert.h Naohiro Tamura via Libc-alpha
2021-07-20 6:37 ` [PATCH v2 5/5] benchtests: Fix validate_benchout.py exceptions Naohiro Tamura via Libc-alpha
2021-07-26 8:34 ` [PATCH] config: Remove HAVE_BUILTIN_MEMSET macro Naohiro Tamura via Libc-alpha
2021-07-26 8:48 ` naohirot--- via Libc-alpha
2021-07-26 8:49 ` Andreas Schwab
2021-07-26 9:42 ` naohirot--- via Libc-alpha
2021-07-26 9:51 ` Andreas Schwab
2021-07-26 13:16 ` naohirot--- via Libc-alpha
2021-07-26 8:35 ` [PATCH] benchtests: Add a script to merge two benchout string files Naohiro Tamura via Libc-alpha
2021-07-27 20:51 ` Joseph Myers
2021-07-30 7:04 ` naohirot--- via Libc-alpha
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/libc/involved.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=430bdf8c-eb5c-72e6-b64e-59b0219669b7@gotplt.org \
--to=siddhesh@gotplt.org \
--cc=libc-alpha@sourceware.org \
--cc=naohirot@fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).