From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-4.8 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_DNSWL_HI,SPF_HELO_PASS, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 15F541F8C8 for ; Thu, 16 Sep 2021 03:48:53 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 12F61385743C for ; Thu, 16 Sep 2021 03:48:52 +0000 (GMT) Received: from burlywood.elm.relay.mailchannels.net (burlywood.elm.relay.mailchannels.net [23.83.212.26]) by sourceware.org (Postfix) with ESMTPS id 55B0B3857419 for ; Thu, 16 Sep 2021 03:48:38 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 55B0B3857419 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=gotplt.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gotplt.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 3B7E29225CC; Thu, 16 Sep 2021 03:48:37 +0000 (UTC) Received: from pdx1-sub0-mail-a28.g.dreamhost.com (100-96-11-8.trex-nlb.outbound.svc.cluster.local [100.96.11.8]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 5C88792255B; Thu, 16 Sep 2021 03:48:36 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from pdx1-sub0-mail-a28.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.96.11.8 (trex/6.4.3); Thu, 16 Sep 2021 03:48:37 +0000 X-MC-Relay: Junk X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Left-Reaction: 30e3b17e1869ab70_1631764117082_2314737708 X-MC-Loop-Signature: 1631764117082:3308855072 X-MC-Ingress-Time: 1631764117082 Received: from pdx1-sub0-mail-a28.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a28.g.dreamhost.com (Postfix) with ESMTP id 1D1707F673; Wed, 15 Sep 2021 20:48:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gotplt.org; h=subject:to :references:from:message-id:date:mime-version:in-reply-to :content-type:content-transfer-encoding; s=gotplt.org; bh=vIwSba WgAm+TGyxZmFc7dnbcjkM=; b=c3pqG6MGLfjfyildSPR8ntGuOz6FggE0n/3j+m /7boe/VyrtWSeKr9a6ChSI/VVGw4uiTBO9CVTGasNUJGe3tBOzlBSaM+BYKGR4OC yL+qHzI7FWGsQSVL4Z0EVp5qSR/ZW+xdZUuaXtqW4fVhsGbMFcREmdRqGwivegWh LHe3k= Received: from [192.168.1.174] (unknown [1.186.224.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a28.g.dreamhost.com (Postfix) with ESMTPSA id 1FA027F66D; Wed, 15 Sep 2021 20:48:33 -0700 (PDT) Subject: Re: [PATCH v5] benchtests: Fix validate_benchout.py exceptions To: Naohiro Tamura , libc-alpha@sourceware.org References: <0eb9f04e-0db6-dcc8-3e76-36f18a39ba1d@gotplt.org> <20210916022316.1101289-1-naohirot@fujitsu.com> X-DH-BACKEND: pdx1-sub0-mail-a28 From: Siddhesh Poyarekar Message-ID: <430bdf8c-eb5c-72e6-b64e-59b0219669b7@gotplt.org> Date: Thu, 16 Sep 2021 09:18:28 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210916022316.1101289-1-naohirot@fujitsu.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" On 9/16/21 7:53 AM, Naohiro Tamura wrote: > This patch fixed validate_benchout.py two exceptions, > 1) AttributeError > if benchout_strings.schema.json is specified, and > 2) json.decoder.JSONDecodeError > if benchout file is not JSON. > > $ ~/glibc/benchtests/scripts/validate_benchout.py bench-memset.out \ > ~/glibc/benchtests/scripts/benchout_strings.schema.json > Traceback (most recent call last): > File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in > sys.exit(main(sys.argv[1:])) > File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main > bench.parse_bench(args[0], args[1]) > File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 139, in parse_bench > do_for_all_timings(bench, lambda b, f, v: > File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 107, in do_for_all_timings > if 'timings' not in bench['functions'][func][k].keys(): > AttributeError: 'str' object has no attribute 'keys' > > $ ~/glibc/benchtests/scripts/validate_benchout.py bench-math-inlines.out \ > ~/glibc/benchtests/scripts/benchout_strings.schema.json > Traceback (most recent call last): > File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 86, in > sys.exit(main(sys.argv[1:])) > File "/home/naohirot/glibc/benchtests/scripts/validate_benchout.py", line 69, in main > bench.parse_bench(args[0], args[1]) > File "/home/naohirot/glibc/benchtests/scripts/import_bench.py", line 137, in parse_bench > bench = json.load(benchfile) > File "/usr/lib/python3.6/json/__init__.py", line 299, in load > parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) > File "/usr/lib/python3.6/json/__init__.py", line 354, in loads > return _default_decoder.decode(s) > File "/usr/lib/python3.6/json/decoder.py", line 342, in decode > raise JSONDecodeError("Extra data", s, end) > json.decoder.JSONDecodeError: Extra data: line 1 column 17 (char 16) > > Reviewed-by: Siddhesh Poyarekar Please don't add Reviewed-by until the reviewer specifically adds it themselves. It is an indicator on patchwork that the patch is ready for inclusion. That said, LGTM, I'll push this :) Reviewed-by: Siddhesh Poyarekar > --- > benchtests/scripts/compare_bench.py | 4 ++++ > benchtests/scripts/import_bench.py | 2 -- > benchtests/scripts/validate_benchout.py | 6 +++++- > 3 files changed, 9 insertions(+), 3 deletions(-) > > diff --git a/benchtests/scripts/compare_bench.py b/benchtests/scripts/compare_bench.py > index 6fcbd0803808..1a497f8bb61b 100755 > --- a/benchtests/scripts/compare_bench.py > +++ b/benchtests/scripts/compare_bench.py > @@ -163,7 +163,11 @@ def plot_graphs(bench1, bench2): > > def main(bench1, bench2, schema, threshold, stats): > bench1 = bench.parse_bench(bench1, schema) > + bench.do_for_all_timings(bench1, lambda b, f, v: > + b['functions'][f][v]['timings'].sort()) > bench2 = bench.parse_bench(bench2, schema) > + bench.do_for_all_timings(bench2, lambda b, f, v: > + b['functions'][f][v]['timings'].sort()) > > plot_graphs(bench1, bench2) > > diff --git a/benchtests/scripts/import_bench.py b/benchtests/scripts/import_bench.py > index a799b4e1b7dc..362c990fe153 100644 > --- a/benchtests/scripts/import_bench.py > +++ b/benchtests/scripts/import_bench.py > @@ -136,6 +136,4 @@ def parse_bench(filename, schema_filename): > with open(filename, 'r') as benchfile: > bench = json.load(benchfile) > validator.validate(bench, schema) > - do_for_all_timings(bench, lambda b, f, v: > - b['functions'][f][v]['timings'].sort()) > return bench > diff --git a/benchtests/scripts/validate_benchout.py b/benchtests/scripts/validate_benchout.py > index 47df33ed0252..00d5fa0ee5eb 100755 > --- a/benchtests/scripts/validate_benchout.py > +++ b/benchtests/scripts/validate_benchout.py > @@ -73,11 +73,15 @@ def main(args): > > except bench.validator.ValidationError as e: > return print_and_exit("Invalid benchmark output: %s" % e.message, > - os.EX_DATAERR) > + os.EX_DATAERR) > > except bench.validator.SchemaError as e: > return print_and_exit("Invalid schema: %s" % e.message, os.EX_DATAERR) > > + except json.decoder.JSONDecodeError as e: > + return print_and_exit("Benchmark output in %s is not JSON." % args[0], > + os.EX_DATAERR) > + > print("Benchmark output in %s is valid." % args[0]) > return os.EX_OK > >