From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-3.3 required=3.0 tests=AWL,BAYES_00,BODY_8BITS, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 15DF71F461 for ; Thu, 16 May 2019 14:57:27 +0000 (UTC) DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id:references :in-reply-to:content-type:content-transfer-encoding :mime-version; q=dns; s=default; b=omb/UWu3qla1pOyK85iKOoJuD+4MU ObhsSB3okgYok5NrI6FKFmOSijZV+PRqiSnnH7C3fa8d9t6K/5ueZe2ti2leNCf1 mBwHhmP1X2s3Xt7uX7CBl3IwW60CX03BFOWDMwS3pplYEaqhHkjfzb3ods+V7ogM btchzA47ugiks8= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id:references :in-reply-to:content-type:content-transfer-encoding :mime-version; s=default; bh=F5yDxb11Vc2DUJEKaC9biu9JDxk=; b=kyI Nv6X01HGv2jMAkaM4L8hM5Ill1IZVqmCC/iXDuTpURQ4TVhSg1ZECTsU/u0Hp3ni v90dW7Hn9Md3iBfYCmtaiI7bczVkhvw+kTzigK+CuA014R6utsuyB8KSExIECHxo PFvBFHUEvQ4AAcSlrQhgNGCRWzor9a8zb4vnJG/Q= Received: (qmail 99410 invoked by alias); 16 May 2019 14:57:24 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 99400 invoked by uid 89); 16 May 2019 14:57:24 -0000 Authentication-Results: sourceware.org; auth=none X-HELO: EUR02-AM5-obe.outbound.protection.outlook.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rb6nZ/A4kIz6V6+8A7TQk1H8/o8z3AyI+IMo9LPC/os=; b=rMlkxH8c9tqSRM31VkdAWcZBmDotS8+eU/B7eS42jtbWtafEm3MBCVl9dQdxrB1g1JIrShqnWiaKQOxaE6YT4Y+gTQFASknIORNfIsidmnCogDHznkKbbLck2fKwN/JjE053j+Wc2raTqQTwZhm57BdD4Wgk9TQRTR2jAG0B3dQ= From: Wilco Dijkstra To: "libc-alpha@sourceware.org" CC: nd Subject: Re: [PATCH] Improve string benchtest timing Date: Thu, 16 May 2019 14:57:18 +0000 Message-ID: References: In-Reply-To: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Wilco.Dijkstra@arm.com; x-ms-oob-tlc-oobclassifiers: OLM:114; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED ping =A0=20 Improve string benchtest timing.=A0 Many tests run for 0.01s which is way t= oo short to give accurate results.=A0 Other tests take over 40 seconds which i= s way too long.=A0 Significantly increase the iterations of the short running tests.=A0 Reduce number of alignment variations in the long running memcpy = walk tests so they take less than 5 seconds. As a result most tests take at least 0.1s and all finish within 5 seconds. OK for commit? 2019-05-08=A0 Wilco Dijkstra=A0 =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-memcpy-random.c (do_one_test): Use= medium iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-memcpy-walk.c (test_main): Reduce = alignment tests. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-memmem.c (do_one_test): Use small = iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-memmove-walk.c (test_main): Reduce= alignment tests. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-memset-walk.c (test_main): Reduce = alignment tests. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-strcasestr.c (do_one_test): Use sm= all iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-string.h (INNER_LOOP_ITERS): Incre= ase iterations. =A0=A0=A0=A0=A0=A0=A0 (INNER_LOOP_ITERS_MEDIUM): New define. =A0=A0=A0=A0=A0=A0=A0 (INNER_LOOP_ITERS_SMALL): New define. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-strpbrk.c (do_one_test): Use mediu= m iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-strsep.c (do_one_test): Use small = iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-strspn.c (do_one_test): Use medium= iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-strstr.c (do_one_test): Use small = iterations. =A0=A0=A0=A0=A0=A0=A0 * benchtests/bench-strtok.c (do_one_test): Use small = iterations. -- diff --git a/benchtests/bench-memcpy-random.c b/benchtests/bench-memcpy-ran= dom.c index 873133d9a6988af8a251e1035ab8bded277d5775..f2c2e9dc3db29ca269514874a49= 76eec13d9d685 100644 --- a/benchtests/bench-memcpy-random.c +++ b/benchtests/bench-memcpy-random.c @@ -101,7 +101,7 @@ do_one_test (json_ctx_t *json_ctx, impl_t *impl, char *= dst, char *src, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 copy_t *copy, size_t n) =A0{ =A0=A0 timing_t start, stop, cur; -=A0 size_t iters =3D INNER_LOOP_ITERS * 20; +=A0 size_t iters =3D INNER_LOOP_ITERS_MEDIUM; =A0 =A0=A0 TIMING_NOW (start); =A0=A0 for (int i =3D 0; i < iters; ++i) diff --git a/benchtests/bench-memcpy-walk.c b/benchtests/bench-memcpy-walk.= c index 16b455f7796b9b3b9288fb6e5245bed006967970..397a1e98df4f7bd069865262470= c40a58665bcd6 100644 --- a/benchtests/bench-memcpy-walk.c +++ b/benchtests/bench-memcpy-walk.c @@ -103,12 +103,8 @@ test_main (void) =A0=A0 json_array_begin (&json_ctx, "results"); =A0=A0 for (size_t i =3D START_SIZE; i <=3D MIN_PAGE_SIZE; i <<=3D 1) =A0=A0=A0=A0 { -=A0=A0=A0=A0=A0 /* Test length alignments from 0-16 bytes.=A0 */ -=A0=A0=A0=A0=A0 for (int j =3D 0; j < 8; j++) -=A0=A0=A0=A0=A0=A0 { -=A0=A0=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, i + j); -=A0=A0=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, i + 16 - j); -=A0=A0=A0=A0=A0=A0 } +=A0=A0=A0=A0=A0 do_test (&json_ctx, i); +=A0=A0=A0=A0=A0 do_test (&json_ctx, i + 1); =A0=A0=A0=A0 } =A0 =A0=A0 json_array_end (&json_ctx); diff --git a/benchtests/bench-memmem.c b/benchtests/bench-memmem.c index b6b97f3d1f5f94a71c177cc516cba81c869d3865..42778b07658efbe9d8027e04dc4= 682aba5ab8797 100644 --- a/benchtests/bench-memmem.c +++ b/benchtests/bench-memmem.c @@ -105,7 +105,7 @@ static void =A0do_one_test (impl_t *impl, const void *haystack, size_t haystack_len, =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 const void *needle, size_t needle_l= en, const void *expected) =A0{ -=A0 size_t i, iters =3D INNER_LOOP_ITERS; +=A0 size_t i, iters =3D INNER_LOOP_ITERS_SMALL; =A0=A0 timing_t start, stop, cur; =A0 =A0=A0 TIMING_NOW (start); diff --git a/benchtests/bench-memmove-walk.c b/benchtests/bench-memmove-wal= k.c index 7b4c31e63f6f3d39beb8ec8da8bfffc82f24d1ac..2b4a7e4f2966b930a8488dd0264= 7639ffbfddc73 100644 --- a/benchtests/bench-memmove-walk.c +++ b/benchtests/bench-memmove-walk.c @@ -107,23 +107,15 @@ test_main (void) =A0=A0 /* Non-overlapping buffers.=A0 */ =A0=A0 for (size_t i =3D START_SIZE; i <=3D MIN_PAGE_SIZE; i <<=3D 1) =A0=A0=A0=A0 { -=A0=A0=A0=A0=A0 /* Test length alignments from 0-16 bytes.=A0 */ -=A0=A0=A0=A0=A0 for (int j =3D 0; j < 8; j++) -=A0=A0=A0=A0=A0=A0 { -=A0=A0=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, i + j, false); -=A0=A0=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, i + 16 - j, false); -=A0=A0=A0=A0=A0=A0 } +=A0=A0=A0=A0=A0 do_test (&json_ctx, i, false); +=A0=A0=A0=A0=A0 do_test (&json_ctx, i + 1, false); =A0=A0=A0=A0 } =A0 =A0=A0 /* Overlapping buffers.=A0 */ =A0=A0 for (size_t i =3D START_SIZE; i <=3D MIN_PAGE_SIZE; i <<=3D 1) =A0=A0=A0=A0 { -=A0=A0=A0=A0=A0 /* Test length alignments from 0-16 bytes.=A0 */ -=A0=A0=A0=A0=A0 for (int j =3D 0; j < 8; j++) -=A0=A0=A0=A0=A0=A0 { -=A0=A0=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, i + j, true); -=A0=A0=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, i + 16 - j, true); -=A0=A0=A0=A0=A0=A0 } +=A0=A0=A0=A0=A0 do_test (&json_ctx, i, true); +=A0=A0=A0=A0=A0 do_test (&json_ctx, i + 1, true); =A0=A0=A0=A0 } =A0 =A0=A0 json_array_end (&json_ctx); diff --git a/benchtests/bench-memset-walk.c b/benchtests/bench-memset-walk.= c index 5f43671216e2294856df35a865061740de7d7d67..930aae34d10603c80a7c40d123f= b0376cde56436 100644 --- a/benchtests/bench-memset-walk.c +++ b/benchtests/bench-memset-walk.c @@ -112,13 +112,16 @@ test_main (void) =A0 =A0=A0 json_array_begin (&json_ctx, "results"); =A0=A0 for (i =3D START_SIZE; i <=3D MIN_PAGE_SIZE; i <<=3D 1) -=A0=A0=A0=A0=A0 /* Test length alignments from 0-16 bytes.=A0 */ -=A0=A0=A0=A0=A0 for (int j =3D 0; j < i && j < 16; j++) -=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, 65, i + j); +=A0=A0=A0 { +=A0=A0=A0=A0=A0 do_test (&json_ctx, 65, i); +=A0=A0=A0=A0=A0 do_test (&json_ctx, 65, i + 1); +=A0=A0=A0 } =A0 =A0=A0 for (i =3D START_SIZE; i <=3D MIN_PAGE_SIZE; i <<=3D 1) -=A0=A0=A0=A0=A0 for (int j =3D 0; j < i && j < 16; j++) -=A0=A0=A0=A0=A0=A0 do_test (&json_ctx, 0, i + j); +=A0=A0=A0 { +=A0=A0=A0=A0=A0 do_test (&json_ctx, 0, i); +=A0=A0=A0=A0=A0 do_test (&json_ctx, 0, i + 1); +=A0=A0=A0 } =A0 =A0=A0 json_array_end (&json_ctx); =A0=A0 json_attr_object_end (&json_ctx); diff --git a/benchtests/bench-strcasestr.c b/benchtests/bench-strcasestr.c index 1458070d47cbfe5e27d40288f8ac7f83df735d08..d701ff4f364b35bd5223831fc8d= f7b468e183ceb 100644 --- a/benchtests/bench-strcasestr.c +++ b/benchtests/bench-strcasestr.c @@ -36,7 +36,7 @@ IMPL (strcasestr, 1) =A0static void =A0do_one_test (impl_t *impl, const char *s1, const char *s2, char *exp_res= ult) =A0{ -=A0 size_t i, iters =3D INNER_LOOP_ITERS; +=A0 size_t i, iters =3D INNER_LOOP_ITERS_SMALL; =A0=A0 timing_t start, stop, cur; =A0 =A0=A0 TIMING_NOW (start); diff --git a/benchtests/bench-string.h b/benchtests/bench-string.h index 9d3332a624c54b22884b2f90444c841a67f4baeb..e7fb015bc639d97e702e2c0e3c9= aee86c9537e2c 100644 --- a/benchtests/bench-string.h +++ b/benchtests/bench-string.h @@ -124,7 +124,9 @@ extern impl_t __start_impls[], __stop_impls[]; =A0# define OPT_RANDOM 10001 =A0# define OPT_SEED 10002 =A0 -# define INNER_LOOP_ITERS 64 +# define INNER_LOOP_ITERS 8192 +# define INNER_LOOP_ITERS_MEDIUM 1024 +# define INNER_LOOP_ITERS_SMALL 32 =A0 =A0int ret, do_srandom; =A0unsigned int seed; diff --git a/benchtests/bench-strpbrk.c b/benchtests/bench-strpbrk.c index 14e1bcb873fa86caa1f50a9013f52cd422e74185..e484ebd0c08df802e67578e6135= 35f65ad233bdc 100644 --- a/benchtests/bench-strpbrk.c +++ b/benchtests/bench-strpbrk.c @@ -66,7 +66,7 @@ static void =A0do_one_test (impl_t *impl, const CHAR *s, const CHAR *rej, RES_TYPE exp_= res) =A0{ =A0=A0 RES_TYPE res =3D CALL (impl, s, rej); -=A0 size_t i, iters =3D INNER_LOOP_ITERS; +=A0 size_t i, iters =3D INNER_LOOP_ITERS_MEDIUM; =A0=A0 timing_t start, stop, cur; =A0 =A0=A0 if (res !=3D exp_res) diff --git a/benchtests/bench-strsep.c b/benchtests/bench-strsep.c index fea7fbdfa6f2dcc8601354f56558dc3d83f07eef..96d3be52221f40ed1afeda18969= 21a04215a6b85 100644 --- a/benchtests/bench-strsep.c +++ b/benchtests/bench-strsep.c @@ -103,7 +103,7 @@ IMPL (oldstrsep, 2) =A0static void =A0do_one_test (impl_t * impl, const char *s1, const char *s2) =A0{ -=A0 size_t i, iters =3D INNER_LOOP_ITERS; +=A0 size_t i, iters =3D INNER_LOOP_ITERS_SMALL; =A0=A0 timing_t start, stop, cur; =A0 =A0=A0 TIMING_NOW (start); diff --git a/benchtests/bench-strspn.c b/benchtests/bench-strspn.c index ac874fe234af66ad98a6188099f32c10484452db..ed97c5fc6205825531cb7e0b6f3= 49223be0c3f66 100644 --- a/benchtests/bench-strspn.c +++ b/benchtests/bench-strspn.c @@ -60,7 +60,7 @@ SIMPLE_STRSPN (const CHAR *s, const CHAR *acc) =A0static void =A0do_one_test (impl_t *impl, const CHAR *s, const CHAR *acc, size_t exp_re= s) =A0{ -=A0 size_t res =3D CALL (impl, s, acc), i, iters =3D INNER_LOOP_ITERS; +=A0 size_t res =3D CALL (impl, s, acc), i, iters =3D INNER_LOOP_ITERS_MEDI= UM; =A0=A0 timing_t start, stop, cur; =A0 =A0=A0 if (res !=3D exp_res) diff --git a/benchtests/bench-strstr.c b/benchtests/bench-strstr.c index 6e45cb7c901fef4319ad62c8ba8ebb9109f12c28..1769dc2a102b3978ea51130e2dc= 7bb91c8b8947c 100644 --- a/benchtests/bench-strstr.c +++ b/benchtests/bench-strstr.c @@ -131,7 +131,7 @@ IMPL (basic_strstr, 0) =A0static void =A0do_one_test (impl_t *impl, const char *s1, const char *s2, char *exp_res= ult) =A0{ -=A0 size_t i, iters =3D INNER_LOOP_ITERS; +=A0 size_t i, iters =3D INNER_LOOP_ITERS_SMALL; =A0=A0 timing_t start, stop, cur; =A0=A0 char *res; =A0 diff --git a/benchtests/bench-strtok.c b/benchtests/bench-strtok.c index 15ae76e2373792781930a3d73418ef81fecfe645..f5ab587dd94ab5035632fbf78f4= 058cc62d72acb 100644 --- a/benchtests/bench-strtok.c +++ b/benchtests/bench-strtok.c @@ -60,7 +60,7 @@ IMPL (strtok, 1) =A0static void =A0do_one_test (impl_t * impl, const char *s1, const char *s2) =A0{ -=A0 size_t i, iters =3D INNER_LOOP_ITERS; +=A0 size_t i, iters =3D INNER_LOOP_ITERS_SMALL; =A0=A0 timing_t start, stop, cur; =A0=A0 TIMING_NOW (start); =A0=A0 for (i =3D 0; i < iters; ++i) =