From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on starla X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 43CBA1F44D for ; Mon, 22 Apr 2024 07:44:53 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=pass (2048-bit key; unprotected) header.d=vrull.eu header.i=@vrull.eu header.a=rsa-sha256 header.s=google header.b=Zb/DBFyk; dkim-atps=neutral Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 00BD0384AB64 for ; Mon, 22 Apr 2024 07:44:52 +0000 (GMT) Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) by sourceware.org (Postfix) with ESMTPS id EA709384AB61 for ; Mon, 22 Apr 2024 07:44:15 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org EA709384AB61 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=vrull.eu Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=vrull.eu ARC-Filter: OpenARC Filter v1.0.0 sourceware.org EA709384AB61 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2a00:1450:4864:20::12f ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713771858; cv=none; b=b38T9xahP/ISTcGVFGgcqtrOhAlLgnxhtls+17Xw/E0GiMl5yEe3KS3yQ4JgRCyCiylB2DcnuKuUOmlcDanFkyK5gTVBvEaGbp8h07JcbndV6ssL/4+i+9pjN8venGJmRgpp6ML0Apdj7KZ4Qia0oWDlUD7SKKdkkALTd/xnhj4= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1713771858; c=relaxed/simple; bh=xfxG74dcwYHPuac+l6Z7D3z7QOsCPn6nNqy4bv1cT4w=; h=DKIM-Signature:From:To:Subject:Date:Message-ID:MIME-Version; b=AaWC/bTE5g25tF4c8u6txKrjv2Kg+nSNZsggBmnUgUpAxvTx440oTM6h546fxzpZzG/u6y/mpQtkj5/7FzAluDYTdeg/Ydb073pQ+6Up8JD+0TA6cNLvnDuL0uW0YByTe2OLmL+1YKq2+66y5rUZ3YlE+BSMgi9Re90piH174Ew= ARC-Authentication-Results: i=1; server2.sourceware.org Received: by mail-lf1-x12f.google.com with SMTP id 2adb3069b0e04-518931f8d23so4137329e87.3 for ; Mon, 22 Apr 2024 00:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1713771854; x=1714376654; darn=sourceware.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UYhvnRUeQ3CrcEfjA4ORCLHqyO9rIzqltCntHPasd9o=; b=Zb/DBFykiBL6sDZLmDbeWrrlbx+/FwEchvbdZkI2nMGB6xv+L42UyfbGJEjy/bEi5j RKB5DFIRkW6zb2mOGspIga8ZoCuPGxn3jttJYlvvYqHZP+7bJJBIzrv05zRttpzAp/u5 ao1ENelj6EfEq9RxMxhNCHK0qsTvcNqg+hFc9cUIHMbXvuc7ktBr9m7dGE12ranBV0IC 4KhnQVxeKaa2k4HlyVcTgWgmFJd8/5UYimP71tt2RDk2vKVxxw/dzFlnJQHRLl3GjJol fP9GkTQeFZoicFcUhiyngfb1s3PlYDlst+eDsEDGKBz9FOngTKG+S7JcFzWM7K33gj7/ Pnnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713771854; x=1714376654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UYhvnRUeQ3CrcEfjA4ORCLHqyO9rIzqltCntHPasd9o=; b=fCt7NSpjDaQTspkNs1kwe+MutXs3Lkf04fu8KH0xX/LItuoFmj8mH7QutHHOifRkig XRdQvJQNC8L8lovjBI1L55Y7vZeKwoKusUFIj0zAKq7QrFKxdf2u4mZW0yHBGQUSooZM jegO5PltBwdePrXNvS6WE6P7MMdMm/3z5FYnQw6POu7h+srxk3KA+LKWYZBysHSXRRzz WBctnzYZ1t00cA8QuVgTDaHLiKfMaTid9GJUjjxyhaMygwPrEUC1yarecuNlgkL1/k2p Vm+pxFaowLyupQmPlDFzPYc2fwXhvt5K++DytQKZUhVDNDFRvQG0PutrkioLVPeiK8Fz mhjw== X-Gm-Message-State: AOJu0YylKcWuxUT4gKKAXwQsNDCrnhIo9Br/apwBX9RBn4PmriKLbOOv 5mNWexEndKYEW2+8/M8f2LMwmXvHqxJ+kEnptcbIB3r4V35cZLkXvVhk+/534EXWVj+lS4R/3+I O+4E= X-Google-Smtp-Source: AGHT+IEfJB8xJnETtyhgOzZO8bw3q/rO3rwIa37lNVLuHuHymcIUrM7MulZiqFgdaHTH6cSiaQ/Xrw== X-Received: by 2002:ac2:57ce:0:b0:516:d1bd:7743 with SMTP id k14-20020ac257ce000000b00516d1bd7743mr4639463lfo.64.1713771853744; Mon, 22 Apr 2024 00:44:13 -0700 (PDT) Received: from beast.fritz.box (static.239.130.217.95.clients.your-server.de. [95.217.130.239]) by smtp.gmail.com with ESMTPSA id v24-20020a197418000000b0051920234d6dsm1656756lfe.273.2024.04.22.00.44.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Apr 2024 00:44:13 -0700 (PDT) From: =?UTF-8?q?Christoph=20M=C3=BCllner?= To: libc-alpha@sourceware.org, Adhemerval Zanella , Palmer Dabbelt , Darius Rad , Andrew Waterman , Philipp Tomsich , Evan Green , DJ Delorie , Vineet Gupta , Kito Cheng , Jeff Law Cc: =?UTF-8?q?Christoph=20M=C3=BCllner?= Subject: [PATCH 2/7] RISC-V: Add Zbb optimized memchr as ifunc Date: Mon, 22 Apr 2024 09:43:58 +0200 Message-ID: <20240422074403.2399529-3-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240422074403.2399529-1-christoph.muellner@vrull.eu> References: <20240422074403.2399529-1-christoph.muellner@vrull.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libc-alpha-bounces+e=80x24.org@sourceware.org When building with Zbb enabled, memchr benefits from using orc.b in find_zero_all(). This patch changes the build system such, that a non-Zbb version as well as a Zbb version of this routine is built. Further, a ifunc resolver is provided that selects the right routine based on the outcome of extension probing via hwprobe(). Signed-off-by: Christoph Müllner --- sysdeps/riscv/multiarch/memchr-generic.c | 26 +++++++++ sysdeps/riscv/multiarch/memchr-zbb.c | 30 ++++++++++ .../unix/sysv/linux/riscv/multiarch/Makefile | 3 + .../linux/riscv/multiarch/ifunc-impl-list.c | 31 ++++++++-- .../unix/sysv/linux/riscv/multiarch/memchr.c | 57 +++++++++++++++++++ 5 files changed, 142 insertions(+), 5 deletions(-) create mode 100644 sysdeps/riscv/multiarch/memchr-generic.c create mode 100644 sysdeps/riscv/multiarch/memchr-zbb.c create mode 100644 sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c diff --git a/sysdeps/riscv/multiarch/memchr-generic.c b/sysdeps/riscv/multiarch/memchr-generic.c new file mode 100644 index 0000000000..a96c36398b --- /dev/null +++ b/sysdeps/riscv/multiarch/memchr-generic.c @@ -0,0 +1,26 @@ +/* Re-include the default memchr implementation. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMCHR __memchr_generic +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +#endif +#include diff --git a/sysdeps/riscv/multiarch/memchr-zbb.c b/sysdeps/riscv/multiarch/memchr-zbb.c new file mode 100644 index 0000000000..bead0335ae --- /dev/null +++ b/sysdeps/riscv/multiarch/memchr-zbb.c @@ -0,0 +1,30 @@ +/* Re-include the default memchr implementation for Zbb. + Copyright (C) 2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#if IS_IN(libc) +# define MEMCHR __memchr_zbb +# undef libc_hidden_builtin_def +# define libc_hidden_builtin_def(x) +#endif +/* Convince preprocessor to have Zbb instructions. */ +#ifndef __riscv_zbb +# define __riscv_zbb +#endif +#include diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile index fcef5659d4..5586d11c89 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/Makefile @@ -1,5 +1,8 @@ ifeq ($(subdir),string) sysdep_routines += \ + memchr \ + memchr-generic \ + memchr-zbb \ memcpy \ memcpy-generic \ memcpy_noalignment \ diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c index 9f806d7a9e..7321144a32 100644 --- a/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/ifunc-impl-list.c @@ -20,19 +20,40 @@ #include #include +#define ARRAY_SIZE(A) (sizeof (A) / sizeof ((A)[0])) + size_t __libc_ifunc_impl_list (const char *name, struct libc_ifunc_impl *array, size_t max) { size_t i = max; + struct riscv_hwprobe pairs[] = { + { .key = RISCV_HWPROBE_KEY_IMA_EXT_0 }, + { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }, + }; + bool has_zbb = false; bool fast_unaligned = false; - struct riscv_hwprobe pair = { .key = RISCV_HWPROBE_KEY_CPUPERF_0 }; - if (__riscv_hwprobe (&pair, 1, 0, NULL, 0) == 0 - && (pair.value & RISCV_HWPROBE_MISALIGNED_MASK) - == RISCV_HWPROBE_MISALIGNED_FAST) - fast_unaligned = true; + if (__riscv_hwprobe (pairs, ARRAY_SIZE (pairs), 0, NULL, 0) == 0) + { + struct riscv_hwprobe *pair; + + /* RISCV_HWPROBE_KEY_IMA_EXT_0 */ + pair = &pairs[0]; + if (pair->value & RISCV_HWPROBE_EXT_ZBB) + has_zbb = true; + + /* RISCV_HWPROBE_KEY_CPUPERF_0 */ + pair = &pairs[1]; + if ((pair->value & RISCV_HWPROBE_MISALIGNED_MASK) + == RISCV_HWPROBE_MISALIGNED_FAST) + fast_unaligned = true; + } + + IFUNC_IMPL (i, name, memchr, + IFUNC_IMPL_ADD (array, i, memchr, has_zbb, __memchr_zbb) + IFUNC_IMPL_ADD (array, i, memchr, 1, __memchr_generic)) IFUNC_IMPL (i, name, memcpy, IFUNC_IMPL_ADD (array, i, memcpy, fast_unaligned, diff --git a/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c b/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c new file mode 100644 index 0000000000..bc076cbf24 --- /dev/null +++ b/sysdeps/unix/sysv/linux/riscv/multiarch/memchr.c @@ -0,0 +1,57 @@ +/* Multiple versions of memchr. + All versions must be listed in ifunc-impl-list.c. + Copyright (C) 2017-2024 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#if IS_IN (libc) +/* Redefine memchr so that the compiler won't complain about the type + mismatch with the IFUNC selector in strong_alias, below. */ +# undef memchr +# define memchr __redirect_memchr +# include +# include +# include +# include +# include + +extern __typeof (__redirect_memchr) __libc_memchr; + +extern __typeof (__redirect_memchr) __memchr_generic attribute_hidden; +extern __typeof (__redirect_memchr) __memchr_zbb attribute_hidden; + +static inline __typeof (__redirect_memchr) * +select_memchr_ifunc (uint64_t dl_hwcap, __riscv_hwprobe_t hwprobe_func) +{ + unsigned long long int v; + if (__riscv_hwprobe_one (hwprobe_func, RISCV_HWPROBE_KEY_IMA_EXT_0, &v) == 0 + && (v & RISCV_HWPROBE_EXT_ZBB)) + return __memchr_zbb; + + return __memchr_generic; +} + +riscv_libc_ifunc (__libc_memchr, select_memchr_ifunc); + +# undef memchr +strong_alias (__libc_memchr, memchr); +# ifdef SHARED +__hidden_ver1 (memchr, __GI_memchr, __redirect_memchr) + __attribute__ ((visibility ("hidden"))) __attribute_copy__ (memchr); +# endif +#else +# include +#endif -- 2.44.0