From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-4.8 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_DNSWL_MED,SPF_HELO_PASS, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 5FB091F8C8 for ; Thu, 16 Sep 2021 01:43:02 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 369B9385842A for ; Thu, 16 Sep 2021 01:43:01 +0000 (GMT) Received: from brown.birch.relay.mailchannels.net (brown.birch.relay.mailchannels.net [23.83.209.23]) by sourceware.org (Postfix) with ESMTPS id E38F03857802 for ; Thu, 16 Sep 2021 01:42:03 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org E38F03857802 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=gotplt.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gotplt.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D170D760896; Thu, 16 Sep 2021 01:42:02 +0000 (UTC) Received: from pdx1-sub0-mail-a96.g.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 69F37760838; Thu, 16 Sep 2021 01:42:02 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from pdx1-sub0-mail-a96.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.99.42.164 (trex/6.4.3); Thu, 16 Sep 2021 01:42:02 +0000 X-MC-Relay: Junk X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Stop-Snatch: 6117b3d37ace89fa_1631756522694_540317229 X-MC-Loop-Signature: 1631756522694:3260523515 X-MC-Ingress-Time: 1631756522694 Received: from pdx1-sub0-mail-a96.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a96.g.dreamhost.com (Postfix) with ESMTP id 1818E86767; Wed, 15 Sep 2021 18:42:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gotplt.org; h=subject:to :references:from:message-id:date:mime-version:in-reply-to :content-type:content-transfer-encoding; s=gotplt.org; bh=/XqMyK leDTzcTkA9qMR3S8QwdFw=; b=HUvqETI762YzqAO7uQXBqCGyZxmOlQhwBOKN30 xOQDRb9n39FAK3Q3hpkD3NOtT+kbk009KOtb49dxyRyIn8plcxT5uhgWdX5iax2g ucACJoNaw7nQFoLkUbND7MgXrmswYyyltR3uI4LshNqt3I9Xw6ZQCGsEsG79h0sW QymZo= Received: from [192.168.1.174] (unknown [1.186.224.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a96.g.dreamhost.com (Postfix) with ESMTPSA id 2E0F67E5FF; Wed, 15 Sep 2021 18:41:59 -0700 (PDT) Subject: Re: [PATCH v4] benchtests: Fix validate_benchout.py exceptions To: "naohirot@fujitsu.com" , "libc-alpha@sourceware.org" References: <2f0293c7-d0b2-0a3b-1ff9-48127837d3c5@gotplt.org> <20210913134412.751102-1-naohirot@fujitsu.com> <6fd1bad0-61bc-f114-4218-083832a693a8@gotplt.org> X-DH-BACKEND: pdx1-sub0-mail-a96 From: Siddhesh Poyarekar Message-ID: <0eb9f04e-0db6-dcc8-3e76-36f18a39ba1d@gotplt.org> Date: Thu, 16 Sep 2021 07:11:54 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=iso-2022-jp; format=flowed; delsp=yes Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" On 9/16/21 6:42 AM, naohirot@fujitsu.com wrote: > Hi Siddhesh, > > Thank you for the comment. > >> From: Siddhesh Poyarekar >> Sent: Wednesday, September 15, 2021 12:23 PM >> >> This will break the original use case, i.e. bench.out, since it doesn't >> have the 'results' nesting timings. The sorting doesn't seem necessary >> for validation, it's only necessary for compare_bench.py. You could >> move the do_for_all_timings call into compare_bench.py, which is >> specific to bench.out so that you don't have to modify do_for_all_timings. > > If do_for_all_timings is moved to compare_bench.py, parse_bench also has > to be moved to compare_bench.py and compare_strings.py. > Ahh no, I didn't suggest moving the implementation, only the call, like below. This way import_bench.parse_bench does not have to know about the specific json format. diff --git a/benchtests/scripts/compare_bench.py b/benchtests/scripts/compare_bench.py index 6fcbd08038..fa7481c76e 100755 --- a/benchtests/scripts/compare_bench.py +++ b/benchtests/scripts/compare_bench.py @@ -163,7 +163,11 @@ def plot_graphs(bench1, bench2): def main(bench1, bench2, schema, threshold, stats): bench1 = bench.parse_bench(bench1, schema) + bench.do_for_all_timings(bench1, lambda b, f, v: + b['functions'][f][v]['timings'].sort()) bench2 = bench.parse_bench(bench2, schema) + bench.do_for_all_timings(bench2, lambda b, f, v: + b['functions'][f][v]['timings'].sort()) plot_graphs(bench1, bench2) diff --git a/benchtests/scripts/import_bench.py b/benchtests/scripts/import_bench.py index a799b4e1b7..362c990fe1 100644 --- a/benchtests/scripts/import_bench.py +++ b/benchtests/scripts/import_bench.py @@ -136,6 +136,4 @@ def parse_bench(filename, schema_filename): with open(filename, 'r') as benchfile: bench = json.load(benchfile) validator.validate(bench, schema) - do_for_all_timings(bench, lambda b, f, v: - b['functions'][f][v]['timings'].sort()) return bench