From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-4.1 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server2.sourceware.org [IPv6:2620:52:3:1:0:246e:9693:128c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 498901F8C8 for ; Thu, 16 Sep 2021 01:12:56 +0000 (UTC) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id EE2FF3858020 for ; Thu, 16 Sep 2021 01:12:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org EE2FF3858020 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sourceware.org; s=default; t=1631754775; bh=RfOl31pLvkLnvsYj8/eX0coqs0sDpS2LPeC0ku6fDxM=; h=To:Subject:Date:References:In-Reply-To:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rXm2KA4nkcCYN3xjjS/jR10pqVdEEwgLqXVCzAU+YiE3/P2duLma+9GKMcgp+PD5a G9Tk5lcbTOfmtl8qw3y+LDXpLCTYOfSTyZKy2ewgmm4LonzO2J2XH5kPXq1pR73fHS aLrJTTJt7RIoqNiKtv0MGjsmUhVmJ3DDKnszUpDE= Received: from esa17.fujitsucc.c3s2.iphmx.com (esa17.fujitsucc.c3s2.iphmx.com [216.71.158.34]) by sourceware.org (Postfix) with ESMTPS id 409653858C39 for ; Thu, 16 Sep 2021 01:12:34 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 409653858C39 X-IronPort-AV: E=McAfee;i="6200,9189,10108"; a="39184517" X-IronPort-AV: E=Sophos;i="5.85,297,1624287600"; d="scan'208";a="39184517" Received: from mail-ty1jpn01lp2051.outbound.protection.outlook.com (HELO JPN01-TY1-obe.outbound.protection.outlook.com) ([104.47.93.51]) by ob1.fujitsucc.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Sep 2021 10:12:31 +0900 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kb9wgW3vsnmXvat8YMePcwixh6r4PkcnCFJZZA0BR0ti8f4iBU65UqDLXpGLUSitR0/sp9AOVx4IcGJbcaPjskd07RQg58HRMwd0QDg99WtOkwMMZ5iWivV7HZrw3IG+KR8CYjvbiNf2Vn+nVdNd72A8VuRooGykOFTYw9X3ospnhUiV67uXrtJ6lui4f4K7eAhb5IpiMMBDXRcZOPNkBh2Ss87SZ8o4zk45FBoViDg9yXQw88kxk8aDCMkrtJrEq8UXwwz5x1Wo6cKQ67h/7tJNACM1icbcZGxZlZA4E7ypT4K/In5xQ2CwvPy/z7wQvDJ76cjoQf9Z6NLu42x52A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=RfOl31pLvkLnvsYj8/eX0coqs0sDpS2LPeC0ku6fDxM=; b=JzOk+EMzWVl884SNKczjGUqgPL3+UM2q9L+OpczhC/1Fi+ZbQuuVqqPMY3McCfGld4dzLH1h+pC+o77hg458QuM5IA3jV33zYu9cO1qAC+F54tTY4VTnrSbAvHm2NDxAObUgmTrKYezLVAoakrJQO1JmT0lRVox/qjUWtsBph/31DPZiFGrKRimhw0PeR3weC7N0V2v3sClNhAEvdIIPBixyzgb6D8ODq1XotVuhwyNrbjkUvzlFDROn/w/RXEfKHpRkYHUDd5jTherFpw6nidDV8ctuBnXSkuOc/uf6RkKFb0E2GNqvvHIMYFdQkiMWUupJQKAAcJeYsqJrvlAQHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fujitsu.com; dmarc=pass action=none header.from=fujitsu.com; dkim=pass header.d=fujitsu.com; arc=none Received: from TYAPR01MB6025.jpnprd01.prod.outlook.com (2603:1096:402:36::13) by TY1PR01MB1689.jpnprd01.prod.outlook.com (2603:1096:403:7::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4500.18; Thu, 16 Sep 2021 01:12:27 +0000 Received: from TYAPR01MB6025.jpnprd01.prod.outlook.com ([fe80::cde6:2645:63a1:51b]) by TYAPR01MB6025.jpnprd01.prod.outlook.com ([fe80::cde6:2645:63a1:51b%9]) with mapi id 15.20.4500.019; Thu, 16 Sep 2021 01:12:27 +0000 To: 'Siddhesh Poyarekar' , "libc-alpha@sourceware.org" Subject: RE: [PATCH v4] benchtests: Fix validate_benchout.py exceptions Thread-Topic: [PATCH v4] benchtests: Fix validate_benchout.py exceptions Thread-Index: AQHXqKWBWrnHT5l32UCUMnlkrjHOGKukcMoAgABaVvA= Date: Thu, 16 Sep 2021 01:12:27 +0000 Message-ID: References: <2f0293c7-d0b2-0a3b-1ff9-48127837d3c5@gotplt.org> <20210913134412.751102-1-naohirot@fujitsu.com> <6fd1bad0-61bc-f114-4218-083832a693a8@gotplt.org> In-Reply-To: <6fd1bad0-61bc-f114-4218-083832a693a8@gotplt.org> Accept-Language: en-001, ja-JP, en-US Content-Language: ja-JP X-MS-Has-Attach: X-MS-TNEF-Correlator: x-securitypolicycheck: OK by SHieldMailChecker v2.6.3 x-shieldmailcheckermailid: 0b9ef15a37c94c07a4260b41559570ab msip_labels: MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Enabled=True; MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_SiteId=a19f121d-81e1-4858-a9d8-736e267fd4c7; MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_SetDate=2021-09-16T01:10:24.354Z; MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Name=FUJITSU-RESTRICTED; MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_ContentBits=0; MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Method=Standard; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 865a4a84-2b26-457e-ce1f-08d978af0cd3 x-ms-traffictypediagnostic: TY1PR01MB1689: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: +2u706Ew27LczIrx+eoijGc4eNFHeA4On5FZKnjOAQ8U2XLXfKMPefSJCd84j/TrAHTUHFYtS2ITV2jxxISdlac7IwmKkqDq1JDzezHxz4BH5OhKYUr0s8wmIZNL88Ke1ok+Bpcm1U7HA6kK4CQnfNmDgILskzIjUM6FTZvYE5Be4dUajuBTD7DHQ3ntgZE/ZK2i4qxmQEsT4l4tlrMtOp+ZI0T9et5NmWlplGtTtlLAbH/U2V9Ngjpackb6b2jBqw12/buBfGKntoLdomjsMEb6D+lkuQL7C6iS7rveMQ8mlAwvf6GGhFmZY9v2Bg1iNGVSE0Gg9ZcK8ho7DUcVyZh1eXzRuEPdghHowJ54al0+CjUf81TJOfr1ipyHHXzsMkqNpcJy5Y6qOWi/TKh3iO4EAu/kIOdPTYMPc19NmYwTYoSVFO7RGPJDwQCDRbPli4WO3a5FZNU2wFkQ+pSaVh2arBXOeOiEL6SYK2vqOZ427wyYfjY6SfjUvsFkuRWt8aI2D6ayMensrDwREnrPZX3K6VABF2R1JdQ5ENYXxHukPlmQ/KKxlB+29uA0MDtK07ZBcEI05fHqsOl2kSZi+5bvGOoU0W8iKyt+JKMZWb+eLNnQfq8TkAZYOB0T8mSQ6d5O7zr6V0Hira61ERcvtFEHGMqCGzghF/a0R0/K+kZTUV+QtCVcobrZtFTOcvFkLcU/bD2DlRv9K7wp+0qq2A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:TYAPR01MB6025.jpnprd01.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(136003)(366004)(376002)(346002)(39860400002)(6506007)(71200400001)(52536014)(8676002)(66476007)(66556008)(66446008)(5660300002)(2906002)(64756008)(478600001)(122000001)(85182001)(110136005)(316002)(9686003)(55016002)(33656002)(186003)(38070700005)(86362001)(38100700002)(66946007)(26005)(8936002)(76116006)(7696005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-2022-jp?B?OGNlZnRWSDE5Tm1VcGxGVzlmaEFXVWM3eFpKLzFJai9pRlBTMW94MGFJ?= =?iso-2022-jp?B?bkV6YnpPUlYyY0szTmNtUSs3c3NyQlR5UzlpZHZic0l3Q0JnMEwzSWhO?= =?iso-2022-jp?B?NXFmbWF5ZHNuOXdxRWd6L1ZZNEs0K2NiWGptOEkvajkzdHljRktWVEdZ?= =?iso-2022-jp?B?ci81VkRGaU9iUk9GNFNJUzlqUEFkMVlQV3hpQTlJYWg3aTFnRkltTmdj?= =?iso-2022-jp?B?MUM2Ymg0TWkvLzJXUGJQNGxDdDVYN3c2c2N3QVdNdUN0Z25CZ3dKcGhJ?= =?iso-2022-jp?B?TithMmV0UDlDRUV3SmVKQk5oc2VHdnc3RFFZNFhORTdFcW9kdkdNRlIz?= =?iso-2022-jp?B?bGFYS1lvUDQ5ZkRrRU1OS2lxZ0lwSHFFTzVidStQKzNUbW1HMHlKMVFL?= =?iso-2022-jp?B?cjJEd2YySUhUVm9rWUFnOUtOOEZSbW9DT05hRkxISk9hMS9CdFdCa0hi?= =?iso-2022-jp?B?NVFtSnZDZWxJNnB6b3kxeWpuODJnb05nUWtjMFBpd0M4bldKL3V4ejM5?= =?iso-2022-jp?B?QnJUUUNmelNEL0FXUWNSbWFQenN4SDdEMTRMT0ZKTXRldHZLNWVNR1Rz?= =?iso-2022-jp?B?SkhRT3JrbW1Xakp6Uk5rd1V0dm8zanpZaGpqMGVkZ0hwY1A0TzQ1aFNm?= =?iso-2022-jp?B?QkxYaGs4dWRnYktCZDk4R21VNVpjV2NhRWdvV2pNNXhkbWxUbmhEdWJs?= =?iso-2022-jp?B?Y1Zvcm1rWWlBOFEvY3VIWXZhZG1NcVB6MGRHNG9pL3VXTzRHZVQ3bTNU?= =?iso-2022-jp?B?YWpWVk1PR1h5SEd1VzRkdTVPNVZaTkZPZ2xyaGJhYmV0UkxxUVlMdksr?= =?iso-2022-jp?B?WG96OWIyN3ZJT01Jb2ZodGZ4UnhaUnc1VEZxTFNpYm01OVNBTVJyVzVr?= =?iso-2022-jp?B?SGM0bmdqS2VPb3FYeVBEQW9ORit3Zzd4TW5STUhzamxTSXBMSHdiTXRU?= =?iso-2022-jp?B?SDRvSmE4WUNocGN6ZUMxK0pody9RODNqZ0ttMTlJd3lDSEIvUDNWdi9L?= =?iso-2022-jp?B?RE94b3IwWHM0S1VSaTRDNUlXU2E0NldxamsxYkxVaml0SXZudHR0bXlS?= =?iso-2022-jp?B?R2psR3FVblNQMTVRUFlya3l0ZDJEQlg2U3ZlbE8rTTJTUmxBOHc1QzJt?= =?iso-2022-jp?B?UkFSVGFjK1FTOHQvOFVSREdvckZNc083bHBXMllLd3FTcEgwRk5NUjVJ?= =?iso-2022-jp?B?ZVRDRElTZ0V2Ry9TT2t3clZwbVFPVmFlQU9sTHBwL3lmTXMrcFFkcEV3?= =?iso-2022-jp?B?UkdJdjYxUkNlNjJ2dVhjNUJkY2JVWjBPR3JCcVFieDFYYlJUbHVOS2Yx?= =?iso-2022-jp?B?SEFsUks1QnROMzVqckh6ZTd5cjludHI4SkN6VmFDV2VuMWJBdWdWSmRu?= =?iso-2022-jp?B?a1htNkdWNkJGbzhOVTVzWUJmUUp5R010d3M2ZCtlajhFbDZmazFLMDFG?= =?iso-2022-jp?B?WWtEWFYvQjYxNmtaREJ3UHJON2JMeGVoZzg0bWE2SnVueHZGLzU2WXRj?= =?iso-2022-jp?B?M01WRkJWSG5LZmQreVRaWjRQUndrUXhxaEl6WnYrY1BSbFVkWkl5VU5H?= =?iso-2022-jp?B?aW8xQVIrb0VvQmJJb2xWZ0lQRmpCdkRUdDFadFIrS1F3Q1NzY2gvMXhG?= =?iso-2022-jp?B?d2hYQWJjTk1EcGV2SzM0TkdSV0drWWQxZEZEcEFUaGZsSTlqb1AvOGlk?= =?iso-2022-jp?B?U0FvTlNwQ0hmMGk5MUZrTUN6c296NVFqMmlGbm0wQSszUmRjZWFxU05G?= =?iso-2022-jp?B?L1djT0pWV0lqOHh6cHp2R1RNbk02cEJPNk8vZ1dyb0pKZDJtNzVibyti?= =?iso-2022-jp?B?WGM3WXMwN1JYazV3TTViRlRWZ3lCWG4rbmV5SHNuYXd3eCtiWm10cms2?= =?iso-2022-jp?B?clhxcVYwV0NzOThqaURJMzI0Zk1QMjJqa1ppM1FPMWpXNGkrUkhXWUg0?= Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: fujitsu.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYAPR01MB6025.jpnprd01.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 865a4a84-2b26-457e-ce1f-08d978af0cd3 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Sep 2021 01:12:27.8701 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j1TyB4m/qNwsHhUbrl1u5Kd8BpvWy9WteF6ZViVRldAod5+0Fq7NsvyVnCcza95rQSN70rbofngHmx1WJp8SPA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY1PR01MB1689 X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: naohirot--- via Libc-alpha Reply-To: "naohirot@fujitsu.com" Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org Sender: "Libc-alpha" Hi Siddhesh, Thank you for the comment.=20 > From: Siddhesh Poyarekar > Sent: Wednesday, September 15, 2021 12:23 PM >=20 > This will break the original use case, i.e. bench.out, since it doesn't > have the 'results' nesting timings. The sorting doesn't seem necessary > for validation, it's only necessary for compare_bench.py. You could > move the do_for_all_timings call into compare_bench.py, which is > specific to bench.out so that you don't have to modify do_for_all_timings= . If do_for_all_timings is moved to compare_bench.py, parse_bench also has to be moved to compare_bench.py and compare_strings.py. So I propose simply to catch AttributeError and return such as below. Because compare_strings.py is not affected. Can we agree? --- a/benchtests/scripts/import_bench.py +++ b/benchtests/scripts/import_bench.py @@ -104,10 +104,15 @@ def do_for_all_timings(bench, callback): """ for func in bench['functions'].keys(): for k in bench['functions'][func].keys(): - if 'timings' not in bench['functions'][func][k].keys(): - continue - - callback(bench, func, k) + try: + if 'timings' not in bench['functions'][func][k].keys(): + continue + + callback(bench, func, k) + # in case that conforms + # benchout_strings.schema.json + except AttributeError: + return Thanks. Naohiro