From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS17314 8.43.84.0/22 X-Spam-Status: No, score=-4.4 required=3.0 tests=AWL,BAYES_00,BODY_8BITS, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,RCVD_IN_DNSWL_MED,RDNS_DYNAMIC, SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (ip-8-43-85-97.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id EE5731F8C6 for ; Mon, 13 Sep 2021 03:50:26 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 2CD6A3857C78 for ; Mon, 13 Sep 2021 03:50:26 +0000 (GMT) Received: from bumble.elm.relay.mailchannels.net (bumble.elm.relay.mailchannels.net [23.83.212.25]) by sourceware.org (Postfix) with ESMTPS id 5CAC63858D35 for ; Mon, 13 Sep 2021 03:50:14 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 5CAC63858D35 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=gotplt.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gotplt.org X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 1E8F46E098C; Mon, 13 Sep 2021 03:50:13 +0000 (UTC) Received: from pdx1-sub0-mail-a59.g.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id A70416E10DB; Mon, 13 Sep 2021 03:50:12 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from pdx1-sub0-mail-a59.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.112.83.61 (trex/6.4.3); Mon, 13 Sep 2021 03:50:13 +0000 X-MC-Relay: Junk X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Illustrious-Abortive: 0d63642b22cf1118_1631505012936_25243048 X-MC-Loop-Signature: 1631505012936:3070180358 X-MC-Ingress-Time: 1631505012936 Received: from pdx1-sub0-mail-a59.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a59.g.dreamhost.com (Postfix) with ESMTP id 0F0A78ADD3; Sun, 12 Sep 2021 20:50:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gotplt.org; h=subject:from :to:references:message-id:date:mime-version:in-reply-to :content-type:content-transfer-encoding; s=gotplt.org; bh=+/wg5m 5+I1y+dp2YWlOuIVL1wyM=; b=GZILIFhR7JxwoOs1xq9lxfBDfBAc930fhEZvxP xfdXfZot3wC6086kJZn8vy+eWVnl3AJWS4NLTefkKJ8HrcGpFZ1QMfTdF/Bhqnh+ IOKqez5y+ezYuKG6CKGUP4/4DfKVO8c/lvC6Hy/IyxWECjRHEO+no36gf4SBI5IV 8w5Zw= Received: from [192.168.1.167] (unknown [1.186.224.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a59.g.dreamhost.com (Postfix) with ESMTPSA id 6A7EF8ADD2; Sun, 12 Sep 2021 20:50:08 -0700 (PDT) Subject: Re: [PATCH v3 4/5] benchtests: Fix validate_benchout.py exceptions X-DH-BACKEND: pdx1-sub0-mail-a59 From: Siddhesh Poyarekar To: Naohiro Tamura , libc-alpha@sourceware.org References: <20210805074733.433430-1-naohirot@fujitsu.com> <20210805075144.433644-1-naohirot@fujitsu.com> Message-ID: <2f0293c7-d0b2-0a3b-1ff9-48127837d3c5@gotplt.org> Date: Mon, 13 Sep 2021 09:20:04 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" On 9/13/21 9:12 AM, Siddhesh Poyarekar wrote: >> --- a/benchtests/scripts/import_bench.py >> +++ b/benchtests/scripts/import_bench.py >> @@ -104,7 +104,10 @@ def do_for_all_timings(bench, callback): >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 """ >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for func in bench['functions'].keys(): >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for k in bench[= 'functions'][func].keys(): >> -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if= 'timings' not in bench['functions'][func][k].keys(): >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 tr= y: >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 if 'timings' not in bench['functions'][func][k].key= s(): >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 continue >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ex= cept AttributeError: >> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 continue >=20 > When do you get an AttributeError here? >=20 OK the one possibility I can think of is when=20 bench['functions'][func][k] is None. This implies the existence of a=20 benchmark output that has a function variant without any inputs and=20 hence, without any benchmark data. That should be invalid, in which=20 case the benchmark should be fixed, not the validator. Thanks, Siddhesh