* [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
@ 2020-12-25 16:24 H.J. Lu via Libc-alpha
2021-01-20 9:53 ` Florian Weimer via Libc-alpha
0 siblings, 1 reply; 10+ messages in thread
From: H.J. Lu via Libc-alpha @ 2020-12-25 16:24 UTC (permalink / raw)
To: libc-alpha
In <sys/platform/x86.h>, define CPU features as enum instead of using
the C preprocessor magic to make it easier to wrap this functionality
in other languages. Move the C preprocessor magic to internal header
for better GCC codegen when more than one features are checked in a
single expression as in x86-64 dl-hwcaps-subdirs.c.
Pass the feature index to __x86_get_cpu_features to work with the older
glibc binary with a smaller COMMON_CPUID_INDEX_MAX [BZ #27104].
256 bits are used for each CPUID leaf. Some leaves only contain a few
features. We can add exceptions to such leaves. But it will increase
code sizes and it is harder to provide backward/forward compatibilities
when new features are added to such leaves in the future.
When new leaves are added, _rtld_global_ro offsets will change which
leads to race condition during in-place updates. We may avoid in-place
updates by
1. Rename the old glibc.
2. Install the new glibc.
3. Remove the old glibc.
NB: A function, __x86_get_cpu_features, is used to avoid the copy
relocation issue with IFUNC resolver as shown in IFUNC resolver tests.
---
sysdeps/x86/Makefile | 3 +
sysdeps/x86/bits/platform/x86.h | 341 +++++++++
sysdeps/x86/dl-get-cpu-features.c | 4 +-
sysdeps/x86/include/cpu-features.h | 702 +++++++++++++++++-
sysdeps/x86/sys/platform/x86.h | 768 +-------------------
sysdeps/x86/tst-get-cpu-features-2-static.c | 1 +
sysdeps/x86/tst-get-cpu-features-2.c | 38 +
7 files changed, 1114 insertions(+), 743 deletions(-)
create mode 100644 sysdeps/x86/bits/platform/x86.h
create mode 100644 sysdeps/x86/tst-get-cpu-features-2-static.c
create mode 100644 sysdeps/x86/tst-get-cpu-features-2.c
diff --git a/sysdeps/x86/Makefile b/sysdeps/x86/Makefile
index 6f909a0f0c..81a27ba0b8 100644
--- a/sysdeps/x86/Makefile
+++ b/sysdeps/x86/Makefile
@@ -7,6 +7,8 @@ sysdep-dl-routines += dl-get-cpu-features
sysdep_headers += sys/platform/x86.h
tests += \
+ tst-get-cpu-features-2 \
+ tst-get-cpu-features-2-static \
tst-get-cpu-features \
tst-get-cpu-features-static \
tst-cpu-features-cpuinfo \
@@ -19,6 +21,7 @@ tests += \
tst-ifunc-platform-3-bindnow \
tst-ifunc-platform-3-lazy
tests-static += \
+ tst-get-cpu-features-2-static \
tst-get-cpu-features-static \
tst-ifunc-platform-1-static
diff --git a/sysdeps/x86/bits/platform/x86.h b/sysdeps/x86/bits/platform/x86.h
new file mode 100644
index 0000000000..104bf3ad53
--- /dev/null
+++ b/sysdeps/x86/bits/platform/x86.h
@@ -0,0 +1,341 @@
+/* Constants and data structures for x86 CPU features.
+ This file is part of the GNU C Library.
+ Copyright (C) 2008-2020 Free Software Foundation, Inc.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_PLATFORM_X86_H
+# error "Never include <bits/platform/x86.h> directly; use <sys/platform/x86.h> instead."
+#endif
+
+enum
+{
+ COMMON_CPUID_INDEX_1 = 0,
+ COMMON_CPUID_INDEX_7,
+ COMMON_CPUID_INDEX_80000001,
+ COMMON_CPUID_INDEX_D_ECX_1,
+ COMMON_CPUID_INDEX_80000007,
+ COMMON_CPUID_INDEX_80000008,
+ COMMON_CPUID_INDEX_7_ECX_1,
+ COMMON_CPUID_INDEX_19,
+ /* Keep the following line at the end. */
+ COMMON_CPUID_INDEX_MAX
+};
+
+struct cpuid_registers
+{
+ unsigned int eax;
+ unsigned int ebx;
+ unsigned int ecx;
+ unsigned int edx;
+};
+
+struct cpuid_feature
+{
+ union
+ {
+ unsigned int cpuid_array[4];
+ struct cpuid_registers cpuid;
+ };
+ union
+ {
+ unsigned int usable_array[4];
+ struct cpuid_registers usable;
+ };
+};
+
+enum cpuid_register_index
+{
+ cpuid_register_index_eax = 0,
+ cpuid_register_index_ebx,
+ cpuid_register_index_ecx,
+ cpuid_register_index_edx
+};
+
+enum cpu_features_kind
+{
+ arch_kind_unknown = 0,
+ arch_kind_intel,
+ arch_kind_amd,
+ arch_kind_zhaoxin,
+ arch_kind_other
+};
+
+struct cpu_features_basic
+{
+ enum cpu_features_kind kind;
+ int max_cpuid;
+ unsigned int family;
+ unsigned int model;
+ unsigned int stepping;
+};
+
+struct cpu_features
+{
+ struct cpu_features_basic basic;
+ struct cpuid_feature features[COMMON_CPUID_INDEX_MAX];
+};
+
+/* CPU features. */
+
+enum
+{
+ x86_cpu_index_1_ecx
+ = (COMMON_CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SSE3 = x86_cpu_index_1_ecx,
+ x86_cpu_PCLMULQDQ = x86_cpu_index_1_ecx + 1,
+ x86_cpu_DTES64 = x86_cpu_index_1_ecx + 2,
+ x86_cpu_MONITOR = x86_cpu_index_1_ecx + 3,
+ x86_cpu_DS_CPL = x86_cpu_index_1_ecx + 4,
+ x86_cpu_VMX = x86_cpu_index_1_ecx + 5,
+ x86_cpu_SMX = x86_cpu_index_1_ecx + 6,
+ x86_cpu_EIST = x86_cpu_index_1_ecx + 7,
+ x86_cpu_TM2 = x86_cpu_index_1_ecx + 8,
+ x86_cpu_SSSE3 = x86_cpu_index_1_ecx + 9,
+ x86_cpu_CNXT_ID = x86_cpu_index_1_ecx + 10,
+ x86_cpu_SDBG = x86_cpu_index_1_ecx + 11,
+ x86_cpu_FMA = x86_cpu_index_1_ecx + 12,
+ x86_cpu_CMPXCHG16B = x86_cpu_index_1_ecx + 13,
+ x86_cpu_XTPRUPDCTRL = x86_cpu_index_1_ecx + 14,
+ x86_cpu_PDCM = x86_cpu_index_1_ecx + 15,
+ x86_cpu_INDEX_1_ECX_16 = x86_cpu_index_1_ecx + 16,
+ x86_cpu_PCID = x86_cpu_index_1_ecx + 17,
+ x86_cpu_DCA = x86_cpu_index_1_ecx + 18,
+ x86_cpu_SSE4_1 = x86_cpu_index_1_ecx + 19,
+ x86_cpu_SSE4_2 = x86_cpu_index_1_ecx + 20,
+ x86_cpu_X2APIC = x86_cpu_index_1_ecx + 21,
+ x86_cpu_MOVBE = x86_cpu_index_1_ecx + 22,
+ x86_cpu_POPCNT = x86_cpu_index_1_ecx + 23,
+ x86_cpu_TSC_DEADLINE = x86_cpu_index_1_ecx + 24,
+ x86_cpu_AES = x86_cpu_index_1_ecx + 25,
+ x86_cpu_XSAVE = x86_cpu_index_1_ecx + 26,
+ x86_cpu_OSXSAVE = x86_cpu_index_1_ecx + 27,
+ x86_cpu_AVX = x86_cpu_index_1_ecx + 28,
+ x86_cpu_F16C = x86_cpu_index_1_ecx + 29,
+ x86_cpu_RDRAND = x86_cpu_index_1_ecx + 30,
+ x86_cpu_INDEX_1_ECX_31 = x86_cpu_index_1_ecx + 31,
+
+ x86_cpu_index_1_edx
+ = (COMMON_CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FPU = x86_cpu_index_1_edx,
+ x86_cpu_VME = x86_cpu_index_1_edx + 1,
+ x86_cpu_DE = x86_cpu_index_1_edx + 2,
+ x86_cpu_PSE = x86_cpu_index_1_edx + 3,
+ x86_cpu_TSC = x86_cpu_index_1_edx + 4,
+ x86_cpu_MSR = x86_cpu_index_1_edx + 5,
+ x86_cpu_PAE = x86_cpu_index_1_edx + 6,
+ x86_cpu_MCE = x86_cpu_index_1_edx + 7,
+ x86_cpu_CX8 = x86_cpu_index_1_edx + 8,
+ x86_cpu_APIC = x86_cpu_index_1_edx + 9,
+ x86_cpu_INDEX_1_EDX_10 = x86_cpu_index_1_edx + 10,
+ x86_cpu_SEP = x86_cpu_index_1_edx + 11,
+ x86_cpu_MTRR = x86_cpu_index_1_edx + 12,
+ x86_cpu_PGE = x86_cpu_index_1_edx + 13,
+ x86_cpu_MCA = x86_cpu_index_1_edx + 14,
+ x86_cpu_CMOV = x86_cpu_index_1_edx + 15,
+ x86_cpu_PAT = x86_cpu_index_1_edx + 16,
+ x86_cpu_PSE_36 = x86_cpu_index_1_edx + 17,
+ x86_cpu_PSN = x86_cpu_index_1_edx + 18,
+ x86_cpu_CLFSH = x86_cpu_index_1_edx + 19,
+ x86_cpu_INDEX_1_EDX_20 = x86_cpu_index_1_edx + 20,
+ x86_cpu_DS = x86_cpu_index_1_edx + 21,
+ x86_cpu_ACPI = x86_cpu_index_1_edx + 22,
+ x86_cpu_MMX = x86_cpu_index_1_edx + 23,
+ x86_cpu_FXSR = x86_cpu_index_1_edx + 24,
+ x86_cpu_SSE = x86_cpu_index_1_edx + 25,
+ x86_cpu_SSE2 = x86_cpu_index_1_edx + 26,
+ x86_cpu_SS = x86_cpu_index_1_edx + 27,
+ x86_cpu_HTT = x86_cpu_index_1_edx + 28,
+ x86_cpu_TM = x86_cpu_index_1_edx + 29,
+ x86_cpu_INDEX_1_EDX_30 = x86_cpu_index_1_edx + 30,
+ x86_cpu_PBE = x86_cpu_index_1_edx + 31,
+
+ x86_cpu_index_7_ebx
+ = (COMMON_CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FSGSBASE = x86_cpu_index_7_ebx,
+ x86_cpu_TSC_ADJUST = x86_cpu_index_7_ebx + 1,
+ x86_cpu_SGX = x86_cpu_index_7_ebx + 2,
+ x86_cpu_BMI1 = x86_cpu_index_7_ebx + 3,
+ x86_cpu_HLE = x86_cpu_index_7_ebx + 4,
+ x86_cpu_AVX2 = x86_cpu_index_7_ebx + 5,
+ x86_cpu_INDEX_7_EBX_6 = x86_cpu_index_7_ebx + 6,
+ x86_cpu_SMEP = x86_cpu_index_7_ebx + 7,
+ x86_cpu_BMI2 = x86_cpu_index_7_ebx + 8,
+ x86_cpu_ERMS = x86_cpu_index_7_ebx + 9,
+ x86_cpu_INVPCID = x86_cpu_index_7_ebx + 10,
+ x86_cpu_RTM = x86_cpu_index_7_ebx + 11,
+ x86_cpu_RDT_M = x86_cpu_index_7_ebx + 12,
+ x86_cpu_DEPR_FPU_CS_DS = x86_cpu_index_7_ebx + 13,
+ x86_cpu_MPX = x86_cpu_index_7_ebx + 14,
+ x86_cpu_RDT_A = x86_cpu_index_7_ebx + 15,
+ x86_cpu_AVX512F = x86_cpu_index_7_ebx + 16,
+ x86_cpu_AVX512DQ = x86_cpu_index_7_ebx + 17,
+ x86_cpu_RDSEED = x86_cpu_index_7_ebx + 18,
+ x86_cpu_ADX = x86_cpu_index_7_ebx + 19,
+ x86_cpu_SMAP = x86_cpu_index_7_ebx + 20,
+ x86_cpu_AVX512_IFMA = x86_cpu_index_7_ebx + 21,
+ x86_cpu_INDEX_7_EBX_22 = x86_cpu_index_7_ebx + 22,
+ x86_cpu_CLFLUSHOPT = x86_cpu_index_7_ebx + 23,
+ x86_cpu_CLWB = x86_cpu_index_7_ebx + 24,
+ x86_cpu_TRACE = x86_cpu_index_7_ebx + 25,
+ x86_cpu_AVX512PF = x86_cpu_index_7_ebx + 26,
+ x86_cpu_AVX512ER = x86_cpu_index_7_ebx + 27,
+ x86_cpu_AVX512CD = x86_cpu_index_7_ebx + 28,
+ x86_cpu_SHA = x86_cpu_index_7_ebx + 29,
+ x86_cpu_AVX512BW = x86_cpu_index_7_ebx + 30,
+ x86_cpu_AVX512VL = x86_cpu_index_7_ebx + 31,
+
+ x86_cpu_index_7_ecx
+ = (COMMON_CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_PREFETCHWT1 = x86_cpu_index_7_ecx,
+ x86_cpu_AVX512_VBMI = x86_cpu_index_7_ecx + 1,
+ x86_cpu_UMIP = x86_cpu_index_7_ecx + 2,
+ x86_cpu_PKU = x86_cpu_index_7_ecx + 3,
+ x86_cpu_OSPKE = x86_cpu_index_7_ecx + 4,
+ x86_cpu_WAITPKG = x86_cpu_index_7_ecx + 5,
+ x86_cpu_AVX512_VBMI2 = x86_cpu_index_7_ecx + 6,
+ x86_cpu_SHSTK = x86_cpu_index_7_ecx + 7,
+ x86_cpu_GFNI = x86_cpu_index_7_ecx + 8,
+ x86_cpu_VAES = x86_cpu_index_7_ecx + 9,
+ x86_cpu_VPCLMULQDQ = x86_cpu_index_7_ecx + 10,
+ x86_cpu_AVX512_VNNI = x86_cpu_index_7_ecx + 11,
+ x86_cpu_AVX512_BITALG = x86_cpu_index_7_ecx + 12,
+ x86_cpu_INDEX_7_ECX_13 = x86_cpu_index_7_ecx + 13,
+ x86_cpu_AVX512_VPOPCNTDQ = x86_cpu_index_7_ecx + 14,
+ x86_cpu_INDEX_7_ECX_1 = x86_cpu_index_7_ecx + 15,
+ x86_cpu_INDEX_7_ECX_16 = x86_cpu_index_7_ecx + 16,
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+ x86_cpu_RDPID = x86_cpu_index_7_ecx + 22,
+ x86_cpu_KL = x86_cpu_index_7_ecx + 23,
+ x86_cpu_INDEX_7_ECX_24 = x86_cpu_index_7_ecx + 24,
+ x86_cpu_CLDEMOTE = x86_cpu_index_7_ecx + 25,
+ x86_cpu_INDEX_7_ECX_26 = x86_cpu_index_7_ecx + 26,
+ x86_cpu_MOVDIRI = x86_cpu_index_7_ecx + 27,
+ x86_cpu_MOVDIR64B = x86_cpu_index_7_ecx + 28,
+ x86_cpu_ENQCMD = x86_cpu_index_7_ecx + 29,
+ x86_cpu_SGX_LC = x86_cpu_index_7_ecx + 30,
+ x86_cpu_PKS = x86_cpu_index_7_ecx + 31,
+
+ x86_cpu_index_7_edx
+ = (COMMON_CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INDEX_7_EDX_0 = x86_cpu_index_7_edx,
+ x86_cpu_INDEX_7_EDX_1 = x86_cpu_index_7_edx + 1,
+ x86_cpu_AVX512_4VNNIW = x86_cpu_index_7_edx + 2,
+ x86_cpu_AVX512_4FMAPS = x86_cpu_index_7_edx + 3,
+ x86_cpu_FSRM = x86_cpu_index_7_edx + 4,
+ x86_cpu_UINTR = x86_cpu_index_7_edx + 5,
+ x86_cpu_INDEX_7_EDX_6 = x86_cpu_index_7_edx + 6,
+ x86_cpu_INDEX_7_EDX_7 = x86_cpu_index_7_edx + 7,
+ x86_cpu_AVX512_VP2INTERSECT = x86_cpu_index_7_edx + 8,
+ x86_cpu_INDEX_7_EDX_9 = x86_cpu_index_7_edx + 9,
+ x86_cpu_MD_CLEAR = x86_cpu_index_7_edx + 10,
+ x86_cpu_INDEX_7_EDX_11 = x86_cpu_index_7_edx + 11,
+ x86_cpu_INDEX_7_EDX_12 = x86_cpu_index_7_edx + 12,
+ x86_cpu_INDEX_7_EDX_13 = x86_cpu_index_7_edx + 13,
+ x86_cpu_SERIALIZE = x86_cpu_index_7_edx + 14,
+ x86_cpu_HYBRID = x86_cpu_index_7_edx + 15,
+ x86_cpu_TSXLDTRK = x86_cpu_index_7_edx + 16,
+ x86_cpu_INDEX_7_EDX_17 = x86_cpu_index_7_edx + 17,
+ x86_cpu_PCONFIG = x86_cpu_index_7_edx + 18,
+ x86_cpu_INDEX_7_EDX_19 = x86_cpu_index_7_edx + 19,
+ x86_cpu_IBT = x86_cpu_index_7_edx + 20,
+ x86_cpu_INDEX_7_EDX_21 = x86_cpu_index_7_edx + 21,
+ x86_cpu_AMX_BF16 = x86_cpu_index_7_edx + 22,
+ x86_cpu_AVX512_FP16 = x86_cpu_index_7_edx + 23,
+ x86_cpu_AMX_TILE = x86_cpu_index_7_edx + 24,
+ x86_cpu_AMX_INT8 = x86_cpu_index_7_edx + 25,
+ x86_cpu_IBRS_IBPB = x86_cpu_index_7_edx + 26,
+ x86_cpu_STIBP = x86_cpu_index_7_edx + 27,
+ x86_cpu_L1D_FLUSH = x86_cpu_index_7_edx + 28,
+ x86_cpu_ARCH_CAPABILITIES = x86_cpu_index_7_edx + 29,
+ x86_cpu_CORE_CAPABILITIES = x86_cpu_index_7_edx + 30,
+ x86_cpu_SSBD = x86_cpu_index_7_edx + 31,
+
+ x86_cpu_index_80000001_ecx
+ = (COMMON_CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_LAHF64_SAHF64 = x86_cpu_index_80000001_ecx,
+ x86_cpu_SVM = x86_cpu_index_80000001_ecx + 2,
+ x86_cpu_LZCNT = x86_cpu_index_80000001_ecx + 5,
+ x86_cpu_SSE4A = x86_cpu_index_80000001_ecx + 6,
+ x86_cpu_PREFETCHW = x86_cpu_index_80000001_ecx + 8,
+ x86_cpu_XOP = x86_cpu_index_80000001_ecx + 11,
+ x86_cpu_LWP = x86_cpu_index_80000001_ecx + 15,
+ x86_cpu_FMA4 = x86_cpu_index_80000001_ecx + 16,
+ x86_cpu_TBM = x86_cpu_index_80000001_ecx + 20,
+
+ x86_cpu_index_80000001_edx
+ = (COMMON_CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SYSCALL_SYSRET = x86_cpu_index_80000001_edx + 11,
+ x86_cpu_NX = x86_cpu_index_80000001_edx + 20,
+ x86_cpu_PAGE1GB = x86_cpu_index_80000001_edx + 26,
+ x86_cpu_RDTSCP = x86_cpu_index_80000001_edx + 27,
+ x86_cpu_LM = x86_cpu_index_80000001_edx + 29,
+
+ x86_cpu_index_d_ecx_1_eax
+ = (COMMON_CPUID_INDEX_D_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_XSAVEOPT = x86_cpu_index_d_ecx_1_eax,
+ x86_cpu_XSAVEC = x86_cpu_index_d_ecx_1_eax + 1,
+ x86_cpu_XGETBV_ECX_1 = x86_cpu_index_d_ecx_1_eax + 2,
+ x86_cpu_XSAVES = x86_cpu_index_d_ecx_1_eax + 3,
+ x86_cpu_XFD = x86_cpu_index_d_ecx_1_eax + 4,
+
+ x86_cpu_index_80000007_edx
+ = (COMMON_CPUID_INDEX_80000007 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INVARIANT_TSC = x86_cpu_index_80000007_edx + 8,
+
+ x86_cpu_index_80000008_ebx
+ = (COMMON_CPUID_INDEX_80000008 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_WBNOINVD = x86_cpu_index_80000008_ebx + 9,
+
+ x86_cpu_index_7_ecx_1_eax
+ = (COMMON_CPUID_INDEX_7_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AVX_VNNI = x86_cpu_index_7_ecx_1_eax + 4,
+ x86_cpu_AVX512_BF16 = x86_cpu_index_7_ecx_1_eax + 5,
+ x86_cpu_FZLRM = x86_cpu_index_7_ecx_1_eax + 10,
+ x86_cpu_FSRS = x86_cpu_index_7_ecx_1_eax + 11,
+ x86_cpu_FSRCS = x86_cpu_index_7_ecx_1_eax + 12,
+ x86_cpu_HRESET = x86_cpu_index_7_ecx_1_eax + 22,
+ x86_cpu_LAM = x86_cpu_index_7_ecx_1_eax + 26,
+
+ x86_cpu_index_19_ebx
+ = (COMMON_CPUID_INDEX_19 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AESKLE = x86_cpu_index_19_ebx,
+ x86_cpu_WIDE_KL = x86_cpu_index_19_ebx + 2
+};
diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
index 349472d99f..4636d9f4a7 100644
--- a/sysdeps/x86/dl-get-cpu-features.c
+++ b/sysdeps/x86/dl-get-cpu-features.c
@@ -46,9 +46,9 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
#undef __x86_get_cpu_features
const struct cpu_features *
-__x86_get_cpu_features (unsigned int max)
+__x86_get_cpu_features (unsigned int index)
{
- if (max > COMMON_CPUID_INDEX_MAX)
+ if (index > COMMON_CPUID_INDEX_MAX * 8 * sizeof (unsigned int) * 4)
return NULL;
return &GLRO(dl_x86_cpu_features);
}
diff --git a/sysdeps/x86/include/cpu-features.h b/sysdeps/x86/include/cpu-features.h
index a53194a6ca..f9158932c2 100644
--- a/sysdeps/x86/include/cpu-features.h
+++ b/sysdeps/x86/include/cpu-features.h
@@ -26,13 +26,12 @@
/* Get most of the contents from the public header, but we define a
different `struct cpu_features' type for private use. */
#define cpu_features cpu_features_public
-#define __x86_get_cpu_features __x86_get_cpu_features_public
-#include <sysdeps/x86/sys/platform/x86.h>
+/* Get data structures without inline functions. */
+#define _SYS_PLATFORM_X86_H
+#include <bits/platform/x86.h>
#undef cpu_features
-#undef __x86_get_cpu_features
-#define __get_cpu_features() __x86_get_cpu_features (0)
enum
{
@@ -54,12 +53,18 @@ enum
#define CPU_FEATURE_PREFERRED_P(ptr, name) \
((ptr->preferred[index_arch_##name] & bit_arch_##name) != 0)
+#define CPU_FEATURE_CHECK_P(ptr, name, check) \
+ ((ptr->features[index_cpu_##name].check.reg_##name \
+ & bit_cpu_##name) != 0)
+#define CPU_FEATURE_CPU_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, cpuid)
+#define CPU_FEATURE_USABLE_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, usable)
+
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
-#undef HAS_CPU_FEATURE
#define HAS_CPU_FEATURE(name) \
CPU_FEATURE_CPU_P (__x86_get_cpu_features (0), name)
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
-#undef CPU_FEATURE_USABLE
#define CPU_FEATURE_USABLE(name) \
CPU_FEATURE_USABLE_P (__x86_get_cpu_features (0), name)
/* CPU_FEATURE_PREFER evaluates to true if we prefer the feature at
@@ -74,6 +79,685 @@ enum
#define HAS_ARCH_FEATURE(name) \
CPU_FEATURE_PREFERRED (name)
+/* CPU features. */
+
+/* COMMON_CPUID_INDEX_1. */
+
+/* ECX. */
+#define bit_cpu_SSE3 (1u << 0)
+#define bit_cpu_PCLMULQDQ (1u << 1)
+#define bit_cpu_DTES64 (1u << 2)
+#define bit_cpu_MONITOR (1u << 3)
+#define bit_cpu_DS_CPL (1u << 4)
+#define bit_cpu_VMX (1u << 5)
+#define bit_cpu_SMX (1u << 6)
+#define bit_cpu_EIST (1u << 7)
+#define bit_cpu_TM2 (1u << 8)
+#define bit_cpu_SSSE3 (1u << 9)
+#define bit_cpu_CNXT_ID (1u << 10)
+#define bit_cpu_SDBG (1u << 11)
+#define bit_cpu_FMA (1u << 12)
+#define bit_cpu_CMPXCHG16B (1u << 13)
+#define bit_cpu_XTPRUPDCTRL (1u << 14)
+#define bit_cpu_PDCM (1u << 15)
+#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
+#define bit_cpu_PCID (1u << 17)
+#define bit_cpu_DCA (1u << 18)
+#define bit_cpu_SSE4_1 (1u << 19)
+#define bit_cpu_SSE4_2 (1u << 20)
+#define bit_cpu_X2APIC (1u << 21)
+#define bit_cpu_MOVBE (1u << 22)
+#define bit_cpu_POPCNT (1u << 23)
+#define bit_cpu_TSC_DEADLINE (1u << 24)
+#define bit_cpu_AES (1u << 25)
+#define bit_cpu_XSAVE (1u << 26)
+#define bit_cpu_OSXSAVE (1u << 27)
+#define bit_cpu_AVX (1u << 28)
+#define bit_cpu_F16C (1u << 29)
+#define bit_cpu_RDRAND (1u << 30)
+#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
+
+/* EDX. */
+#define bit_cpu_FPU (1u << 0)
+#define bit_cpu_VME (1u << 1)
+#define bit_cpu_DE (1u << 2)
+#define bit_cpu_PSE (1u << 3)
+#define bit_cpu_TSC (1u << 4)
+#define bit_cpu_MSR (1u << 5)
+#define bit_cpu_PAE (1u << 6)
+#define bit_cpu_MCE (1u << 7)
+#define bit_cpu_CX8 (1u << 8)
+#define bit_cpu_APIC (1u << 9)
+#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
+#define bit_cpu_SEP (1u << 11)
+#define bit_cpu_MTRR (1u << 12)
+#define bit_cpu_PGE (1u << 13)
+#define bit_cpu_MCA (1u << 14)
+#define bit_cpu_CMOV (1u << 15)
+#define bit_cpu_PAT (1u << 16)
+#define bit_cpu_PSE_36 (1u << 17)
+#define bit_cpu_PSN (1u << 18)
+#define bit_cpu_CLFSH (1u << 19)
+#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
+#define bit_cpu_DS (1u << 21)
+#define bit_cpu_ACPI (1u << 22)
+#define bit_cpu_MMX (1u << 23)
+#define bit_cpu_FXSR (1u << 24)
+#define bit_cpu_SSE (1u << 25)
+#define bit_cpu_SSE2 (1u << 26)
+#define bit_cpu_SS (1u << 27)
+#define bit_cpu_HTT (1u << 28)
+#define bit_cpu_TM (1u << 29)
+#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
+#define bit_cpu_PBE (1u << 31)
+
+/* COMMON_CPUID_INDEX_7. */
+
+/* EBX. */
+#define bit_cpu_FSGSBASE (1u << 0)
+#define bit_cpu_TSC_ADJUST (1u << 1)
+#define bit_cpu_SGX (1u << 2)
+#define bit_cpu_BMI1 (1u << 3)
+#define bit_cpu_HLE (1u << 4)
+#define bit_cpu_AVX2 (1u << 5)
+#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
+#define bit_cpu_SMEP (1u << 7)
+#define bit_cpu_BMI2 (1u << 8)
+#define bit_cpu_ERMS (1u << 9)
+#define bit_cpu_INVPCID (1u << 10)
+#define bit_cpu_RTM (1u << 11)
+#define bit_cpu_RDT_M (1u << 12)
+#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
+#define bit_cpu_MPX (1u << 14)
+#define bit_cpu_RDT_A (1u << 15)
+#define bit_cpu_AVX512F (1u << 16)
+#define bit_cpu_AVX512DQ (1u << 17)
+#define bit_cpu_RDSEED (1u << 18)
+#define bit_cpu_ADX (1u << 19)
+#define bit_cpu_SMAP (1u << 20)
+#define bit_cpu_AVX512_IFMA (1u << 21)
+#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
+#define bit_cpu_CLFLUSHOPT (1u << 23)
+#define bit_cpu_CLWB (1u << 24)
+#define bit_cpu_TRACE (1u << 25)
+#define bit_cpu_AVX512PF (1u << 26)
+#define bit_cpu_AVX512ER (1u << 27)
+#define bit_cpu_AVX512CD (1u << 28)
+#define bit_cpu_SHA (1u << 29)
+#define bit_cpu_AVX512BW (1u << 30)
+#define bit_cpu_AVX512VL (1u << 31)
+
+/* ECX. */
+#define bit_cpu_PREFETCHWT1 (1u << 0)
+#define bit_cpu_AVX512_VBMI (1u << 1)
+#define bit_cpu_UMIP (1u << 2)
+#define bit_cpu_PKU (1u << 3)
+#define bit_cpu_OSPKE (1u << 4)
+#define bit_cpu_WAITPKG (1u << 5)
+#define bit_cpu_AVX512_VBMI2 (1u << 6)
+#define bit_cpu_SHSTK (1u << 7)
+#define bit_cpu_GFNI (1u << 8)
+#define bit_cpu_VAES (1u << 9)
+#define bit_cpu_VPCLMULQDQ (1u << 10)
+#define bit_cpu_AVX512_VNNI (1u << 11)
+#define bit_cpu_AVX512_BITALG (1u << 12)
+#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
+#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
+#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
+#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+#define bit_cpu_RDPID (1u << 22)
+#define bit_cpu_KL (1u << 23)
+#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
+#define bit_cpu_CLDEMOTE (1u << 25)
+#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
+#define bit_cpu_MOVDIRI (1u << 27)
+#define bit_cpu_MOVDIR64B (1u << 28)
+#define bit_cpu_ENQCMD (1u << 29)
+#define bit_cpu_SGX_LC (1u << 30)
+#define bit_cpu_PKS (1u << 31)
+
+/* EDX. */
+#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
+#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
+#define bit_cpu_AVX512_4VNNIW (1u << 2)
+#define bit_cpu_AVX512_4FMAPS (1u << 3)
+#define bit_cpu_FSRM (1u << 4)
+#define bit_cpu_UINTR (1u << 5)
+#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
+#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
+#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
+#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
+#define bit_cpu_MD_CLEAR (1u << 10)
+#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
+#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
+#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
+#define bit_cpu_SERIALIZE (1u << 14)
+#define bit_cpu_HYBRID (1u << 15)
+#define bit_cpu_TSXLDTRK (1u << 16)
+#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
+#define bit_cpu_PCONFIG (1u << 18)
+#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
+#define bit_cpu_IBT (1u << 20)
+#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
+#define bit_cpu_AMX_BF16 (1u << 22)
+#define bit_cpu_AVX512_FP16 (1u << 23)
+#define bit_cpu_AMX_TILE (1u << 24)
+#define bit_cpu_AMX_INT8 (1u << 25)
+#define bit_cpu_IBRS_IBPB (1u << 26)
+#define bit_cpu_STIBP (1u << 27)
+#define bit_cpu_L1D_FLUSH (1u << 28)
+#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
+#define bit_cpu_CORE_CAPABILITIES (1u << 30)
+#define bit_cpu_SSBD (1u << 31)
+
+/* COMMON_CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define bit_cpu_LAHF64_SAHF64 (1u << 0)
+#define bit_cpu_SVM (1u << 2)
+#define bit_cpu_LZCNT (1u << 5)
+#define bit_cpu_SSE4A (1u << 6)
+#define bit_cpu_PREFETCHW (1u << 8)
+#define bit_cpu_XOP (1u << 11)
+#define bit_cpu_LWP (1u << 15)
+#define bit_cpu_FMA4 (1u << 16)
+#define bit_cpu_TBM (1u << 21)
+
+/* EDX. */
+#define bit_cpu_SYSCALL_SYSRET (1u << 11)
+#define bit_cpu_NX (1u << 20)
+#define bit_cpu_PAGE1GB (1u << 26)
+#define bit_cpu_RDTSCP (1u << 27)
+#define bit_cpu_LM (1u << 29)
+
+/* COMMON_CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_XSAVEOPT (1u << 0)
+#define bit_cpu_XSAVEC (1u << 1)
+#define bit_cpu_XGETBV_ECX_1 (1u << 2)
+#define bit_cpu_XSAVES (1u << 3)
+#define bit_cpu_XFD (1u << 4)
+
+/* COMMON_CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define bit_cpu_INVARIANT_TSC (1u << 8)
+
+/* COMMON_CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define bit_cpu_WBNOINVD (1u << 9)
+
+/* COMMON_CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_AVX_VNNI (1u << 4)
+#define bit_cpu_AVX512_BF16 (1u << 5)
+#define bit_cpu_FZLRM (1u << 10)
+#define bit_cpu_FSRS (1u << 11)
+#define bit_cpu_FSRCS (1u << 12)
+#define bit_cpu_HRESET (1u << 22)
+#define bit_cpu_LAM (1u << 26)
+
+/* COMMON_CPUID_INDEX_19. */
+
+/* EBX. */
+#define bit_cpu_AESKLE (1u << 0)
+#define bit_cpu_WIDE_KL (1u << 2)
+
+/* COMMON_CPUID_INDEX_1. */
+
+/* ECX. */
+#define index_cpu_SSE3 COMMON_CPUID_INDEX_1
+#define index_cpu_PCLMULQDQ COMMON_CPUID_INDEX_1
+#define index_cpu_DTES64 COMMON_CPUID_INDEX_1
+#define index_cpu_MONITOR COMMON_CPUID_INDEX_1
+#define index_cpu_DS_CPL COMMON_CPUID_INDEX_1
+#define index_cpu_VMX COMMON_CPUID_INDEX_1
+#define index_cpu_SMX COMMON_CPUID_INDEX_1
+#define index_cpu_EIST COMMON_CPUID_INDEX_1
+#define index_cpu_TM2 COMMON_CPUID_INDEX_1
+#define index_cpu_SSSE3 COMMON_CPUID_INDEX_1
+#define index_cpu_CNXT_ID COMMON_CPUID_INDEX_1
+#define index_cpu_SDBG COMMON_CPUID_INDEX_1
+#define index_cpu_FMA COMMON_CPUID_INDEX_1
+#define index_cpu_CMPXCHG16B COMMON_CPUID_INDEX_1
+#define index_cpu_XTPRUPDCTRL COMMON_CPUID_INDEX_1
+#define index_cpu_PDCM COMMON_CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_16 COMMON_CPUID_INDEX_1
+#define index_cpu_PCID COMMON_CPUID_INDEX_1
+#define index_cpu_DCA COMMON_CPUID_INDEX_1
+#define index_cpu_SSE4_1 COMMON_CPUID_INDEX_1
+#define index_cpu_SSE4_2 COMMON_CPUID_INDEX_1
+#define index_cpu_X2APIC COMMON_CPUID_INDEX_1
+#define index_cpu_MOVBE COMMON_CPUID_INDEX_1
+#define index_cpu_POPCNT COMMON_CPUID_INDEX_1
+#define index_cpu_TSC_DEADLINE COMMON_CPUID_INDEX_1
+#define index_cpu_AES COMMON_CPUID_INDEX_1
+#define index_cpu_XSAVE COMMON_CPUID_INDEX_1
+#define index_cpu_OSXSAVE COMMON_CPUID_INDEX_1
+#define index_cpu_AVX COMMON_CPUID_INDEX_1
+#define index_cpu_F16C COMMON_CPUID_INDEX_1
+#define index_cpu_RDRAND COMMON_CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_31 COMMON_CPUID_INDEX_1
+
+/* ECX. */
+#define index_cpu_FPU COMMON_CPUID_INDEX_1
+#define index_cpu_VME COMMON_CPUID_INDEX_1
+#define index_cpu_DE COMMON_CPUID_INDEX_1
+#define index_cpu_PSE COMMON_CPUID_INDEX_1
+#define index_cpu_TSC COMMON_CPUID_INDEX_1
+#define index_cpu_MSR COMMON_CPUID_INDEX_1
+#define index_cpu_PAE COMMON_CPUID_INDEX_1
+#define index_cpu_MCE COMMON_CPUID_INDEX_1
+#define index_cpu_CX8 COMMON_CPUID_INDEX_1
+#define index_cpu_APIC COMMON_CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_10 COMMON_CPUID_INDEX_1
+#define index_cpu_SEP COMMON_CPUID_INDEX_1
+#define index_cpu_MTRR COMMON_CPUID_INDEX_1
+#define index_cpu_PGE COMMON_CPUID_INDEX_1
+#define index_cpu_MCA COMMON_CPUID_INDEX_1
+#define index_cpu_CMOV COMMON_CPUID_INDEX_1
+#define index_cpu_PAT COMMON_CPUID_INDEX_1
+#define index_cpu_PSE_36 COMMON_CPUID_INDEX_1
+#define index_cpu_PSN COMMON_CPUID_INDEX_1
+#define index_cpu_CLFSH COMMON_CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_20 COMMON_CPUID_INDEX_1
+#define index_cpu_DS COMMON_CPUID_INDEX_1
+#define index_cpu_ACPI COMMON_CPUID_INDEX_1
+#define index_cpu_MMX COMMON_CPUID_INDEX_1
+#define index_cpu_FXSR COMMON_CPUID_INDEX_1
+#define index_cpu_SSE COMMON_CPUID_INDEX_1
+#define index_cpu_SSE2 COMMON_CPUID_INDEX_1
+#define index_cpu_SS COMMON_CPUID_INDEX_1
+#define index_cpu_HTT COMMON_CPUID_INDEX_1
+#define index_cpu_TM COMMON_CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_30 COMMON_CPUID_INDEX_1
+#define index_cpu_PBE COMMON_CPUID_INDEX_1
+
+/* COMMON_CPUID_INDEX_7. */
+
+/* EBX. */
+#define index_cpu_FSGSBASE COMMON_CPUID_INDEX_7
+#define index_cpu_TSC_ADJUST COMMON_CPUID_INDEX_7
+#define index_cpu_SGX COMMON_CPUID_INDEX_7
+#define index_cpu_BMI1 COMMON_CPUID_INDEX_7
+#define index_cpu_HLE COMMON_CPUID_INDEX_7
+#define index_cpu_AVX2 COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_6 COMMON_CPUID_INDEX_7
+#define index_cpu_SMEP COMMON_CPUID_INDEX_7
+#define index_cpu_BMI2 COMMON_CPUID_INDEX_7
+#define index_cpu_ERMS COMMON_CPUID_INDEX_7
+#define index_cpu_INVPCID COMMON_CPUID_INDEX_7
+#define index_cpu_RTM COMMON_CPUID_INDEX_7
+#define index_cpu_RDT_M COMMON_CPUID_INDEX_7
+#define index_cpu_DEPR_FPU_CS_DS COMMON_CPUID_INDEX_7
+#define index_cpu_MPX COMMON_CPUID_INDEX_7
+#define index_cpu_RDT_A COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512F COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512DQ COMMON_CPUID_INDEX_7
+#define index_cpu_RDSEED COMMON_CPUID_INDEX_7
+#define index_cpu_ADX COMMON_CPUID_INDEX_7
+#define index_cpu_SMAP COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_IFMA COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_22 COMMON_CPUID_INDEX_7
+#define index_cpu_CLFLUSHOPT COMMON_CPUID_INDEX_7
+#define index_cpu_CLWB COMMON_CPUID_INDEX_7
+#define index_cpu_TRACE COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512PF COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512ER COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512CD COMMON_CPUID_INDEX_7
+#define index_cpu_SHA COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512BW COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512VL COMMON_CPUID_INDEX_7
+
+/* ECX. */
+#define index_cpu_PREFETCHWT1 COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI COMMON_CPUID_INDEX_7
+#define index_cpu_UMIP COMMON_CPUID_INDEX_7
+#define index_cpu_PKU COMMON_CPUID_INDEX_7
+#define index_cpu_OSPKE COMMON_CPUID_INDEX_7
+#define index_cpu_WAITPKG COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI2 COMMON_CPUID_INDEX_7
+#define index_cpu_SHSTK COMMON_CPUID_INDEX_7
+#define index_cpu_GFNI COMMON_CPUID_INDEX_7
+#define index_cpu_VAES COMMON_CPUID_INDEX_7
+#define index_cpu_VPCLMULQDQ COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_VNNI COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_BITALG COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_13 COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_VPOPCNTDQ COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_15 COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_16 COMMON_CPUID_INDEX_7
+#define index_cpu_RDPID COMMON_CPUID_INDEX_7
+#define index_cpu_KL COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_24 COMMON_CPUID_INDEX_7
+#define index_cpu_CLDEMOTE COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_26 COMMON_CPUID_INDEX_7
+#define index_cpu_MOVDIRI COMMON_CPUID_INDEX_7
+#define index_cpu_MOVDIR64B COMMON_CPUID_INDEX_7
+#define index_cpu_ENQCMD COMMON_CPUID_INDEX_7
+#define index_cpu_SGX_LC COMMON_CPUID_INDEX_7
+#define index_cpu_PKS COMMON_CPUID_INDEX_7
+
+/* EDX. */
+#define index_cpu_INDEX_7_EDX_0 COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_1 COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_4VNNIW COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_4FMAPS COMMON_CPUID_INDEX_7
+#define index_cpu_FSRM COMMON_CPUID_INDEX_7
+#define index_cpu_UINTR COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_6 COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_7 COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_VP2INTERSECT COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_9 COMMON_CPUID_INDEX_7
+#define index_cpu_MD_CLEAR COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_11 COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_12 COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_13 COMMON_CPUID_INDEX_7
+#define index_cpu_SERIALIZE COMMON_CPUID_INDEX_7
+#define index_cpu_HYBRID COMMON_CPUID_INDEX_7
+#define index_cpu_TSXLDTRK COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_17 COMMON_CPUID_INDEX_7
+#define index_cpu_PCONFIG COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_19 COMMON_CPUID_INDEX_7
+#define index_cpu_IBT COMMON_CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_21 COMMON_CPUID_INDEX_7
+#define index_cpu_AMX_BF16 COMMON_CPUID_INDEX_7
+#define index_cpu_AVX512_FP16 COMMON_CPUID_INDEX_7
+#define index_cpu_AMX_TILE COMMON_CPUID_INDEX_7
+#define index_cpu_AMX_INT8 COMMON_CPUID_INDEX_7
+#define index_cpu_IBRS_IBPB COMMON_CPUID_INDEX_7
+#define index_cpu_STIBP COMMON_CPUID_INDEX_7
+#define index_cpu_L1D_FLUSH COMMON_CPUID_INDEX_7
+#define index_cpu_ARCH_CAPABILITIES COMMON_CPUID_INDEX_7
+#define index_cpu_CORE_CAPABILITIES COMMON_CPUID_INDEX_7
+#define index_cpu_SSBD COMMON_CPUID_INDEX_7
+
+/* COMMON_CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define index_cpu_LAHF64_SAHF64 COMMON_CPUID_INDEX_80000001
+#define index_cpu_SVM COMMON_CPUID_INDEX_80000001
+#define index_cpu_LZCNT COMMON_CPUID_INDEX_80000001
+#define index_cpu_SSE4A COMMON_CPUID_INDEX_80000001
+#define index_cpu_PREFETCHW COMMON_CPUID_INDEX_80000001
+#define index_cpu_XOP COMMON_CPUID_INDEX_80000001
+#define index_cpu_LWP COMMON_CPUID_INDEX_80000001
+#define index_cpu_FMA4 COMMON_CPUID_INDEX_80000001
+#define index_cpu_TBM COMMON_CPUID_INDEX_80000001
+
+/* EDX. */
+#define index_cpu_SYSCALL_SYSRET COMMON_CPUID_INDEX_80000001
+#define index_cpu_NX COMMON_CPUID_INDEX_80000001
+#define index_cpu_PAGE1GB COMMON_CPUID_INDEX_80000001
+#define index_cpu_RDTSCP COMMON_CPUID_INDEX_80000001
+#define index_cpu_LM COMMON_CPUID_INDEX_80000001
+
+/* COMMON_CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define index_cpu_XSAVEOPT COMMON_CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVEC COMMON_CPUID_INDEX_D_ECX_1
+#define index_cpu_XGETBV_ECX_1 COMMON_CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVES COMMON_CPUID_INDEX_D_ECX_1
+#define index_cpu_XFD COMMON_CPUID_INDEX_D_ECX_1
+
+/* COMMON_CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define index_cpu_INVARIANT_TSC COMMON_CPUID_INDEX_80000007
+
+/* COMMON_CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define index_cpu_WBNOINVD COMMON_CPUID_INDEX_80000008
+
+/* COMMON_CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define index_cpu_AVX_VNNI COMMON_CPUID_INDEX_7_ECX_1
+#define index_cpu_AVX512_BF16 COMMON_CPUID_INDEX_7_ECX_1
+#define index_cpu_FZLRM COMMON_CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRS COMMON_CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRCS COMMON_CPUID_INDEX_7_ECX_1
+#define index_cpu_HRESET COMMON_CPUID_INDEX_7_ECX_1
+#define index_cpu_LAM COMMON_CPUID_INDEX_7_ECX_1
+
+/* COMMON_CPUID_INDEX_19. */
+
+/* EBX. */
+#define index_cpu_AESKLE COMMON_CPUID_INDEX_19
+#define index_cpu_WIDE_KL COMMON_CPUID_INDEX_19
+
+/* COMMON_CPUID_INDEX_1. */
+
+/* ECX. */
+#define reg_SSE3 ecx
+#define reg_PCLMULQDQ ecx
+#define reg_DTES64 ecx
+#define reg_MONITOR ecx
+#define reg_DS_CPL ecx
+#define reg_VMX ecx
+#define reg_SMX ecx
+#define reg_EIST ecx
+#define reg_TM2 ecx
+#define reg_SSSE3 ecx
+#define reg_CNXT_ID ecx
+#define reg_SDBG ecx
+#define reg_FMA ecx
+#define reg_CMPXCHG16B ecx
+#define reg_XTPRUPDCTRL ecx
+#define reg_PDCM ecx
+#define reg_INDEX_1_ECX_16 ecx
+#define reg_PCID ecx
+#define reg_DCA ecx
+#define reg_SSE4_1 ecx
+#define reg_SSE4_2 ecx
+#define reg_X2APIC ecx
+#define reg_MOVBE ecx
+#define reg_POPCNT ecx
+#define reg_TSC_DEADLINE ecx
+#define reg_AES ecx
+#define reg_XSAVE ecx
+#define reg_OSXSAVE ecx
+#define reg_AVX ecx
+#define reg_F16C ecx
+#define reg_RDRAND ecx
+#define reg_INDEX_1_ECX_31 ecx
+
+/* EDX. */
+#define reg_FPU edx
+#define reg_VME edx
+#define reg_DE edx
+#define reg_PSE edx
+#define reg_TSC edx
+#define reg_MSR edx
+#define reg_PAE edx
+#define reg_MCE edx
+#define reg_CX8 edx
+#define reg_APIC edx
+#define reg_INDEX_1_EDX_10 edx
+#define reg_SEP edx
+#define reg_MTRR edx
+#define reg_PGE edx
+#define reg_MCA edx
+#define reg_CMOV edx
+#define reg_PAT edx
+#define reg_PSE_36 edx
+#define reg_PSN edx
+#define reg_CLFSH edx
+#define reg_INDEX_1_EDX_20 edx
+#define reg_DS edx
+#define reg_ACPI edx
+#define reg_MMX edx
+#define reg_FXSR edx
+#define reg_SSE edx
+#define reg_SSE2 edx
+#define reg_SS edx
+#define reg_HTT edx
+#define reg_TM edx
+#define reg_INDEX_1_EDX_30 edx
+#define reg_PBE edx
+
+/* COMMON_CPUID_INDEX_7. */
+
+/* EBX. */
+#define reg_FSGSBASE ebx
+#define reg_TSC_ADJUST ebx
+#define reg_SGX ebx
+#define reg_BMI1 ebx
+#define reg_HLE ebx
+#define reg_BMI2 ebx
+#define reg_AVX2 ebx
+#define reg_INDEX_7_EBX_6 ebx
+#define reg_SMEP ebx
+#define reg_ERMS ebx
+#define reg_INVPCID ebx
+#define reg_RTM ebx
+#define reg_RDT_M ebx
+#define reg_DEPR_FPU_CS_DS ebx
+#define reg_MPX ebx
+#define reg_RDT_A ebx
+#define reg_AVX512F ebx
+#define reg_AVX512DQ ebx
+#define reg_RDSEED ebx
+#define reg_ADX ebx
+#define reg_SMAP ebx
+#define reg_AVX512_IFMA ebx
+#define reg_INDEX_7_EBX_22 ebx
+#define reg_CLFLUSHOPT ebx
+#define reg_CLWB ebx
+#define reg_TRACE ebx
+#define reg_AVX512PF ebx
+#define reg_AVX512ER ebx
+#define reg_AVX512CD ebx
+#define reg_SHA ebx
+#define reg_AVX512BW ebx
+#define reg_AVX512VL ebx
+
+/* ECX. */
+#define reg_PREFETCHWT1 ecx
+#define reg_AVX512_VBMI ecx
+#define reg_UMIP ecx
+#define reg_PKU ecx
+#define reg_OSPKE ecx
+#define reg_WAITPKG ecx
+#define reg_AVX512_VBMI2 ecx
+#define reg_SHSTK ecx
+#define reg_GFNI ecx
+#define reg_VAES ecx
+#define reg_VPCLMULQDQ ecx
+#define reg_AVX512_VNNI ecx
+#define reg_AVX512_BITALG ecx
+#define reg_INDEX_7_ECX_13 ecx
+#define reg_AVX512_VPOPCNTDQ ecx
+#define reg_INDEX_7_ECX_15 ecx
+#define reg_INDEX_7_ECX_16 ecx
+#define reg_RDPID ecx
+#define reg_KL ecx
+#define reg_INDEX_7_ECX_24 ecx
+#define reg_CLDEMOTE ecx
+#define reg_INDEX_7_ECX_26 ecx
+#define reg_MOVDIRI ecx
+#define reg_MOVDIR64B ecx
+#define reg_ENQCMD ecx
+#define reg_SGX_LC ecx
+#define reg_PKS ecx
+
+/* EDX. */
+#define reg_INDEX_7_EDX_0 edx
+#define reg_INDEX_7_EDX_1 edx
+#define reg_AVX512_4VNNIW edx
+#define reg_AVX512_4FMAPS edx
+#define reg_FSRM edx
+#define reg_UINTR edx
+#define reg_INDEX_7_EDX_6 edx
+#define reg_INDEX_7_EDX_7 edx
+#define reg_AVX512_VP2INTERSECT edx
+#define reg_INDEX_7_EDX_9 edx
+#define reg_MD_CLEAR edx
+#define reg_INDEX_7_EDX_11 edx
+#define reg_INDEX_7_EDX_12 edx
+#define reg_INDEX_7_EDX_13 edx
+#define reg_SERIALIZE edx
+#define reg_HYBRID edx
+#define reg_TSXLDTRK edx
+#define reg_INDEX_7_EDX_17 edx
+#define reg_PCONFIG edx
+#define reg_INDEX_7_EDX_19 edx
+#define reg_IBT edx
+#define reg_INDEX_7_EDX_21 edx
+#define reg_AMX_BF16 edx
+#define reg_AVX512_FP16 edx
+#define reg_AMX_TILE edx
+#define reg_AMX_INT8 edx
+#define reg_IBRS_IBPB edx
+#define reg_STIBP edx
+#define reg_L1D_FLUSH edx
+#define reg_ARCH_CAPABILITIES edx
+#define reg_CORE_CAPABILITIES edx
+#define reg_SSBD edx
+
+/* COMMON_CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define reg_LAHF64_SAHF64 ecx
+#define reg_SVM ecx
+#define reg_LZCNT ecx
+#define reg_SSE4A ecx
+#define reg_PREFETCHW ecx
+#define reg_XOP ecx
+#define reg_LWP ecx
+#define reg_FMA4 ecx
+#define reg_TBM ecx
+
+/* EDX. */
+#define reg_SYSCALL_SYSRET edx
+#define reg_NX edx
+#define reg_PAGE1GB edx
+#define reg_RDTSCP edx
+#define reg_LM edx
+
+/* COMMON_CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define reg_XSAVEOPT eax
+#define reg_XSAVEC eax
+#define reg_XGETBV_ECX_1 eax
+#define reg_XSAVES eax
+#define reg_XFD eax
+
+/* COMMON_CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define reg_INVARIANT_TSC edx
+
+/* COMMON_CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define reg_WBNOINVD ebx
+
+/* COMMON_CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define reg_AVX_VNNI eax
+#define reg_AVX512_BF16 eax
+#define reg_FZLRM eax
+#define reg_FSRS eax
+#define reg_FSRCS eax
+#define reg_HRESET eax
+#define reg_LAM eax
+
+/* COMMON_CPUID_INDEX_19. */
+
+/* EBX. */
+#define reg_AESKLE ebx
+#define reg_WIDE_KL ebx
+
/* PREFERRED_FEATURE_INDEX_1. */
#define bit_arch_I586 (1u << 0)
#define bit_arch_I686 (1u << 1)
@@ -121,7 +805,7 @@ enum
struct cpu_features
{
struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
+ struct cpuid_feature features[COMMON_CPUID_INDEX_MAX];
unsigned int preferred[PREFERRED_FEATURE_INDEX_MAX];
/* The state size for XSAVEC or XSAVE. The type must be unsigned long
int so that we use
@@ -149,10 +833,12 @@ struct cpu_features
unsigned long int rep_stosb_threshold;
};
+#define __get_cpu_features() __x86_get_cpu_features (0)
+
#if defined (_LIBC) && !IS_IN (nonlib)
/* Unused for x86. */
# define INIT_ARCH()
-# define __x86_get_cpu_features(max) (&GLRO(dl_x86_cpu_features))
+# define __x86_get_cpu_features(indx) (&GLRO(dl_x86_cpu_features))
extern void _dl_x86_init_cpu_features (void) attribute_hidden;
#endif
diff --git a/sysdeps/x86/sys/platform/x86.h b/sysdeps/x86/sys/platform/x86.h
index 99d8c9b0ab..6aa2fe45b2 100644
--- a/sysdeps/x86/sys/platform/x86.h
+++ b/sysdeps/x86/sys/platform/x86.h
@@ -19,760 +19,62 @@
#ifndef _SYS_PLATFORM_X86_H
#define _SYS_PLATFORM_X86_H
-enum
-{
- COMMON_CPUID_INDEX_1 = 0,
- COMMON_CPUID_INDEX_7,
- COMMON_CPUID_INDEX_80000001,
- COMMON_CPUID_INDEX_D_ECX_1,
- COMMON_CPUID_INDEX_80000007,
- COMMON_CPUID_INDEX_80000008,
- COMMON_CPUID_INDEX_7_ECX_1,
- COMMON_CPUID_INDEX_19,
- /* Keep the following line at the end. */
- COMMON_CPUID_INDEX_MAX
-};
+#include <features.h>
+#include <stdbool.h>
+#include <bits/platform/x86.h>
-struct cpuid_registers
-{
- unsigned int eax;
- unsigned int ebx;
- unsigned int ecx;
- unsigned int edx;
-};
+__BEGIN_DECLS
-struct cpuid_features
+static __inline__ _Bool
+x86_cpu_has_feature (const struct cpu_features *__ptr,
+ unsigned int __index)
{
- struct cpuid_registers cpuid;
- struct cpuid_registers usable;
-};
-
-enum cpu_features_kind
+ unsigned int __leaf
+ = __index / (8 * sizeof (unsigned int) * 4);
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
+
+ return __ptr->features[__leaf].cpuid_array[__reg] & (1 << __bit);
+}
+
+static __inline__ _Bool
+x86_cpu_is_usable (const struct cpu_features *__ptr,
+ unsigned int __index)
{
- arch_kind_unknown = 0,
- arch_kind_intel,
- arch_kind_amd,
- arch_kind_zhaoxin,
- arch_kind_other
-};
+ unsigned int __leaf
+ = __index / (8 * sizeof (unsigned int) * 4);
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
-struct cpu_features_basic
-{
- enum cpu_features_kind kind;
- int max_cpuid;
- unsigned int family;
- unsigned int model;
- unsigned int stepping;
-};
-
-struct cpu_features
-{
- struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
-};
+ return __ptr->features[__leaf].usable_array[__reg] & (1 << __bit);
+}
/* Get a pointer to the CPU features structure. */
extern const struct cpu_features *__x86_get_cpu_features (unsigned int)
__attribute__ ((const));
-#define CPU_FEATURE_CHECK_P(ptr, name, check) \
- ((ptr->features[index_cpu_##name].check.reg_##name \
- & bit_cpu_##name) != 0)
#define CPU_FEATURE_CPU_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, cpuid)
+ x86_cpu_has_feature ((ptr), x86_cpu_##name)
#define CPU_FEATURE_USABLE_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, usable)
+ x86_cpu_is_usable ((ptr), x86_cpu_##name)
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
#define HAS_CPU_FEATURE(name) \
(__extension__ \
({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_CPU_P (__ptr, name); }))
+ __x86_get_cpu_features (x86_cpu_##name); \
+ __ptr && x86_cpu_has_feature (__ptr, x86_cpu_##name); }))
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
#define CPU_FEATURE_USABLE(name) \
(__extension__ \
({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_USABLE_P (__ptr, name); }))
-
-/* CPU features. */
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define bit_cpu_SSE3 (1u << 0)
-#define bit_cpu_PCLMULQDQ (1u << 1)
-#define bit_cpu_DTES64 (1u << 2)
-#define bit_cpu_MONITOR (1u << 3)
-#define bit_cpu_DS_CPL (1u << 4)
-#define bit_cpu_VMX (1u << 5)
-#define bit_cpu_SMX (1u << 6)
-#define bit_cpu_EIST (1u << 7)
-#define bit_cpu_TM2 (1u << 8)
-#define bit_cpu_SSSE3 (1u << 9)
-#define bit_cpu_CNXT_ID (1u << 10)
-#define bit_cpu_SDBG (1u << 11)
-#define bit_cpu_FMA (1u << 12)
-#define bit_cpu_CMPXCHG16B (1u << 13)
-#define bit_cpu_XTPRUPDCTRL (1u << 14)
-#define bit_cpu_PDCM (1u << 15)
-#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
-#define bit_cpu_PCID (1u << 17)
-#define bit_cpu_DCA (1u << 18)
-#define bit_cpu_SSE4_1 (1u << 19)
-#define bit_cpu_SSE4_2 (1u << 20)
-#define bit_cpu_X2APIC (1u << 21)
-#define bit_cpu_MOVBE (1u << 22)
-#define bit_cpu_POPCNT (1u << 23)
-#define bit_cpu_TSC_DEADLINE (1u << 24)
-#define bit_cpu_AES (1u << 25)
-#define bit_cpu_XSAVE (1u << 26)
-#define bit_cpu_OSXSAVE (1u << 27)
-#define bit_cpu_AVX (1u << 28)
-#define bit_cpu_F16C (1u << 29)
-#define bit_cpu_RDRAND (1u << 30)
-#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
-
-/* EDX. */
-#define bit_cpu_FPU (1u << 0)
-#define bit_cpu_VME (1u << 1)
-#define bit_cpu_DE (1u << 2)
-#define bit_cpu_PSE (1u << 3)
-#define bit_cpu_TSC (1u << 4)
-#define bit_cpu_MSR (1u << 5)
-#define bit_cpu_PAE (1u << 6)
-#define bit_cpu_MCE (1u << 7)
-#define bit_cpu_CX8 (1u << 8)
-#define bit_cpu_APIC (1u << 9)
-#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
-#define bit_cpu_SEP (1u << 11)
-#define bit_cpu_MTRR (1u << 12)
-#define bit_cpu_PGE (1u << 13)
-#define bit_cpu_MCA (1u << 14)
-#define bit_cpu_CMOV (1u << 15)
-#define bit_cpu_PAT (1u << 16)
-#define bit_cpu_PSE_36 (1u << 17)
-#define bit_cpu_PSN (1u << 18)
-#define bit_cpu_CLFSH (1u << 19)
-#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
-#define bit_cpu_DS (1u << 21)
-#define bit_cpu_ACPI (1u << 22)
-#define bit_cpu_MMX (1u << 23)
-#define bit_cpu_FXSR (1u << 24)
-#define bit_cpu_SSE (1u << 25)
-#define bit_cpu_SSE2 (1u << 26)
-#define bit_cpu_SS (1u << 27)
-#define bit_cpu_HTT (1u << 28)
-#define bit_cpu_TM (1u << 29)
-#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
-#define bit_cpu_PBE (1u << 31)
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define bit_cpu_FSGSBASE (1u << 0)
-#define bit_cpu_TSC_ADJUST (1u << 1)
-#define bit_cpu_SGX (1u << 2)
-#define bit_cpu_BMI1 (1u << 3)
-#define bit_cpu_HLE (1u << 4)
-#define bit_cpu_AVX2 (1u << 5)
-#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
-#define bit_cpu_SMEP (1u << 7)
-#define bit_cpu_BMI2 (1u << 8)
-#define bit_cpu_ERMS (1u << 9)
-#define bit_cpu_INVPCID (1u << 10)
-#define bit_cpu_RTM (1u << 11)
-#define bit_cpu_RDT_M (1u << 12)
-#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
-#define bit_cpu_MPX (1u << 14)
-#define bit_cpu_RDT_A (1u << 15)
-#define bit_cpu_AVX512F (1u << 16)
-#define bit_cpu_AVX512DQ (1u << 17)
-#define bit_cpu_RDSEED (1u << 18)
-#define bit_cpu_ADX (1u << 19)
-#define bit_cpu_SMAP (1u << 20)
-#define bit_cpu_AVX512_IFMA (1u << 21)
-#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
-#define bit_cpu_CLFLUSHOPT (1u << 23)
-#define bit_cpu_CLWB (1u << 24)
-#define bit_cpu_TRACE (1u << 25)
-#define bit_cpu_AVX512PF (1u << 26)
-#define bit_cpu_AVX512ER (1u << 27)
-#define bit_cpu_AVX512CD (1u << 28)
-#define bit_cpu_SHA (1u << 29)
-#define bit_cpu_AVX512BW (1u << 30)
-#define bit_cpu_AVX512VL (1u << 31)
-
-/* ECX. */
-#define bit_cpu_PREFETCHWT1 (1u << 0)
-#define bit_cpu_AVX512_VBMI (1u << 1)
-#define bit_cpu_UMIP (1u << 2)
-#define bit_cpu_PKU (1u << 3)
-#define bit_cpu_OSPKE (1u << 4)
-#define bit_cpu_WAITPKG (1u << 5)
-#define bit_cpu_AVX512_VBMI2 (1u << 6)
-#define bit_cpu_SHSTK (1u << 7)
-#define bit_cpu_GFNI (1u << 8)
-#define bit_cpu_VAES (1u << 9)
-#define bit_cpu_VPCLMULQDQ (1u << 10)
-#define bit_cpu_AVX512_VNNI (1u << 11)
-#define bit_cpu_AVX512_BITALG (1u << 12)
-#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
-#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
-#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
-#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
-/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
- instructions in 64-bit mode. */
-#define bit_cpu_RDPID (1u << 22)
-#define bit_cpu_KL (1u << 23)
-#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
-#define bit_cpu_CLDEMOTE (1u << 25)
-#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
-#define bit_cpu_MOVDIRI (1u << 27)
-#define bit_cpu_MOVDIR64B (1u << 28)
-#define bit_cpu_ENQCMD (1u << 29)
-#define bit_cpu_SGX_LC (1u << 30)
-#define bit_cpu_PKS (1u << 31)
-
-/* EDX. */
-#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
-#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
-#define bit_cpu_AVX512_4VNNIW (1u << 2)
-#define bit_cpu_AVX512_4FMAPS (1u << 3)
-#define bit_cpu_FSRM (1u << 4)
-#define bit_cpu_UINTR (1u << 5)
-#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
-#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
-#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
-#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
-#define bit_cpu_MD_CLEAR (1u << 10)
-#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
-#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
-#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
-#define bit_cpu_SERIALIZE (1u << 14)
-#define bit_cpu_HYBRID (1u << 15)
-#define bit_cpu_TSXLDTRK (1u << 16)
-#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
-#define bit_cpu_PCONFIG (1u << 18)
-#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
-#define bit_cpu_IBT (1u << 20)
-#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
-#define bit_cpu_AMX_BF16 (1u << 22)
-#define bit_cpu_AVX512_FP16 (1u << 23)
-#define bit_cpu_AMX_TILE (1u << 24)
-#define bit_cpu_AMX_INT8 (1u << 25)
-#define bit_cpu_IBRS_IBPB (1u << 26)
-#define bit_cpu_STIBP (1u << 27)
-#define bit_cpu_L1D_FLUSH (1u << 28)
-#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
-#define bit_cpu_CORE_CAPABILITIES (1u << 30)
-#define bit_cpu_SSBD (1u << 31)
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define bit_cpu_LAHF64_SAHF64 (1u << 0)
-#define bit_cpu_SVM (1u << 2)
-#define bit_cpu_LZCNT (1u << 5)
-#define bit_cpu_SSE4A (1u << 6)
-#define bit_cpu_PREFETCHW (1u << 8)
-#define bit_cpu_XOP (1u << 11)
-#define bit_cpu_LWP (1u << 15)
-#define bit_cpu_FMA4 (1u << 16)
-#define bit_cpu_TBM (1u << 21)
-
-/* EDX. */
-#define bit_cpu_SYSCALL_SYSRET (1u << 11)
-#define bit_cpu_NX (1u << 20)
-#define bit_cpu_PAGE1GB (1u << 26)
-#define bit_cpu_RDTSCP (1u << 27)
-#define bit_cpu_LM (1u << 29)
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_XSAVEOPT (1u << 0)
-#define bit_cpu_XSAVEC (1u << 1)
-#define bit_cpu_XGETBV_ECX_1 (1u << 2)
-#define bit_cpu_XSAVES (1u << 3)
-#define bit_cpu_XFD (1u << 4)
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define bit_cpu_INVARIANT_TSC (1u << 8)
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define bit_cpu_WBNOINVD (1u << 9)
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_AVX_VNNI (1u << 4)
-#define bit_cpu_AVX512_BF16 (1u << 5)
-#define bit_cpu_FZLRM (1u << 10)
-#define bit_cpu_FSRS (1u << 11)
-#define bit_cpu_FSRCS (1u << 12)
-#define bit_cpu_HRESET (1u << 22)
-#define bit_cpu_LAM (1u << 26)
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define bit_cpu_AESKLE (1u << 0)
-#define bit_cpu_WIDE_KL (1u << 2)
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define index_cpu_SSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_PCLMULQDQ COMMON_CPUID_INDEX_1
-#define index_cpu_DTES64 COMMON_CPUID_INDEX_1
-#define index_cpu_MONITOR COMMON_CPUID_INDEX_1
-#define index_cpu_DS_CPL COMMON_CPUID_INDEX_1
-#define index_cpu_VMX COMMON_CPUID_INDEX_1
-#define index_cpu_SMX COMMON_CPUID_INDEX_1
-#define index_cpu_EIST COMMON_CPUID_INDEX_1
-#define index_cpu_TM2 COMMON_CPUID_INDEX_1
-#define index_cpu_SSSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_CNXT_ID COMMON_CPUID_INDEX_1
-#define index_cpu_SDBG COMMON_CPUID_INDEX_1
-#define index_cpu_FMA COMMON_CPUID_INDEX_1
-#define index_cpu_CMPXCHG16B COMMON_CPUID_INDEX_1
-#define index_cpu_XTPRUPDCTRL COMMON_CPUID_INDEX_1
-#define index_cpu_PDCM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_16 COMMON_CPUID_INDEX_1
-#define index_cpu_PCID COMMON_CPUID_INDEX_1
-#define index_cpu_DCA COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_1 COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_2 COMMON_CPUID_INDEX_1
-#define index_cpu_X2APIC COMMON_CPUID_INDEX_1
-#define index_cpu_MOVBE COMMON_CPUID_INDEX_1
-#define index_cpu_POPCNT COMMON_CPUID_INDEX_1
-#define index_cpu_TSC_DEADLINE COMMON_CPUID_INDEX_1
-#define index_cpu_AES COMMON_CPUID_INDEX_1
-#define index_cpu_XSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_OSXSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_AVX COMMON_CPUID_INDEX_1
-#define index_cpu_F16C COMMON_CPUID_INDEX_1
-#define index_cpu_RDRAND COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_31 COMMON_CPUID_INDEX_1
-
-/* ECX. */
-#define index_cpu_FPU COMMON_CPUID_INDEX_1
-#define index_cpu_VME COMMON_CPUID_INDEX_1
-#define index_cpu_DE COMMON_CPUID_INDEX_1
-#define index_cpu_PSE COMMON_CPUID_INDEX_1
-#define index_cpu_TSC COMMON_CPUID_INDEX_1
-#define index_cpu_MSR COMMON_CPUID_INDEX_1
-#define index_cpu_PAE COMMON_CPUID_INDEX_1
-#define index_cpu_MCE COMMON_CPUID_INDEX_1
-#define index_cpu_CX8 COMMON_CPUID_INDEX_1
-#define index_cpu_APIC COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_10 COMMON_CPUID_INDEX_1
-#define index_cpu_SEP COMMON_CPUID_INDEX_1
-#define index_cpu_MTRR COMMON_CPUID_INDEX_1
-#define index_cpu_PGE COMMON_CPUID_INDEX_1
-#define index_cpu_MCA COMMON_CPUID_INDEX_1
-#define index_cpu_CMOV COMMON_CPUID_INDEX_1
-#define index_cpu_PAT COMMON_CPUID_INDEX_1
-#define index_cpu_PSE_36 COMMON_CPUID_INDEX_1
-#define index_cpu_PSN COMMON_CPUID_INDEX_1
-#define index_cpu_CLFSH COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_20 COMMON_CPUID_INDEX_1
-#define index_cpu_DS COMMON_CPUID_INDEX_1
-#define index_cpu_ACPI COMMON_CPUID_INDEX_1
-#define index_cpu_MMX COMMON_CPUID_INDEX_1
-#define index_cpu_FXSR COMMON_CPUID_INDEX_1
-#define index_cpu_SSE COMMON_CPUID_INDEX_1
-#define index_cpu_SSE2 COMMON_CPUID_INDEX_1
-#define index_cpu_SS COMMON_CPUID_INDEX_1
-#define index_cpu_HTT COMMON_CPUID_INDEX_1
-#define index_cpu_TM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_30 COMMON_CPUID_INDEX_1
-#define index_cpu_PBE COMMON_CPUID_INDEX_1
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define index_cpu_FSGSBASE COMMON_CPUID_INDEX_7
-#define index_cpu_TSC_ADJUST COMMON_CPUID_INDEX_7
-#define index_cpu_SGX COMMON_CPUID_INDEX_7
-#define index_cpu_BMI1 COMMON_CPUID_INDEX_7
-#define index_cpu_HLE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX2 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_SMEP COMMON_CPUID_INDEX_7
-#define index_cpu_BMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_ERMS COMMON_CPUID_INDEX_7
-#define index_cpu_INVPCID COMMON_CPUID_INDEX_7
-#define index_cpu_RTM COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_M COMMON_CPUID_INDEX_7
-#define index_cpu_DEPR_FPU_CS_DS COMMON_CPUID_INDEX_7
-#define index_cpu_MPX COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_A COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512F COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512DQ COMMON_CPUID_INDEX_7
-#define index_cpu_RDSEED COMMON_CPUID_INDEX_7
-#define index_cpu_ADX COMMON_CPUID_INDEX_7
-#define index_cpu_SMAP COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_IFMA COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_22 COMMON_CPUID_INDEX_7
-#define index_cpu_CLFLUSHOPT COMMON_CPUID_INDEX_7
-#define index_cpu_CLWB COMMON_CPUID_INDEX_7
-#define index_cpu_TRACE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512PF COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512ER COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512CD COMMON_CPUID_INDEX_7
-#define index_cpu_SHA COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512BW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512VL COMMON_CPUID_INDEX_7
-
-/* ECX. */
-#define index_cpu_PREFETCHWT1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI COMMON_CPUID_INDEX_7
-#define index_cpu_UMIP COMMON_CPUID_INDEX_7
-#define index_cpu_PKU COMMON_CPUID_INDEX_7
-#define index_cpu_OSPKE COMMON_CPUID_INDEX_7
-#define index_cpu_WAITPKG COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_SHSTK COMMON_CPUID_INDEX_7
-#define index_cpu_GFNI COMMON_CPUID_INDEX_7
-#define index_cpu_VAES COMMON_CPUID_INDEX_7
-#define index_cpu_VPCLMULQDQ COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VNNI COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_BITALG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VPOPCNTDQ COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_15 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_16 COMMON_CPUID_INDEX_7
-#define index_cpu_RDPID COMMON_CPUID_INDEX_7
-#define index_cpu_KL COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_24 COMMON_CPUID_INDEX_7
-#define index_cpu_CLDEMOTE COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_26 COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIRI COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIR64B COMMON_CPUID_INDEX_7
-#define index_cpu_ENQCMD COMMON_CPUID_INDEX_7
-#define index_cpu_SGX_LC COMMON_CPUID_INDEX_7
-#define index_cpu_PKS COMMON_CPUID_INDEX_7
-
-/* EDX. */
-#define index_cpu_INDEX_7_EDX_0 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4VNNIW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4FMAPS COMMON_CPUID_INDEX_7
-#define index_cpu_FSRM COMMON_CPUID_INDEX_7
-#define index_cpu_UINTR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_7 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VP2INTERSECT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_9 COMMON_CPUID_INDEX_7
-#define index_cpu_MD_CLEAR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_11 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_12 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_SERIALIZE COMMON_CPUID_INDEX_7
-#define index_cpu_HYBRID COMMON_CPUID_INDEX_7
-#define index_cpu_TSXLDTRK COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_17 COMMON_CPUID_INDEX_7
-#define index_cpu_PCONFIG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_19 COMMON_CPUID_INDEX_7
-#define index_cpu_IBT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_21 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_BF16 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_FP16 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_TILE COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_INT8 COMMON_CPUID_INDEX_7
-#define index_cpu_IBRS_IBPB COMMON_CPUID_INDEX_7
-#define index_cpu_STIBP COMMON_CPUID_INDEX_7
-#define index_cpu_L1D_FLUSH COMMON_CPUID_INDEX_7
-#define index_cpu_ARCH_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_CORE_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_SSBD COMMON_CPUID_INDEX_7
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define index_cpu_LAHF64_SAHF64 COMMON_CPUID_INDEX_80000001
-#define index_cpu_SVM COMMON_CPUID_INDEX_80000001
-#define index_cpu_LZCNT COMMON_CPUID_INDEX_80000001
-#define index_cpu_SSE4A COMMON_CPUID_INDEX_80000001
-#define index_cpu_PREFETCHW COMMON_CPUID_INDEX_80000001
-#define index_cpu_XOP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LWP COMMON_CPUID_INDEX_80000001
-#define index_cpu_FMA4 COMMON_CPUID_INDEX_80000001
-#define index_cpu_TBM COMMON_CPUID_INDEX_80000001
-
-/* EDX. */
-#define index_cpu_SYSCALL_SYSRET COMMON_CPUID_INDEX_80000001
-#define index_cpu_NX COMMON_CPUID_INDEX_80000001
-#define index_cpu_PAGE1GB COMMON_CPUID_INDEX_80000001
-#define index_cpu_RDTSCP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LM COMMON_CPUID_INDEX_80000001
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define index_cpu_XSAVEOPT COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVEC COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XGETBV_ECX_1 COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVES COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XFD COMMON_CPUID_INDEX_D_ECX_1
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define index_cpu_INVARIANT_TSC COMMON_CPUID_INDEX_80000007
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define index_cpu_WBNOINVD COMMON_CPUID_INDEX_80000008
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define index_cpu_AVX_VNNI COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_AVX512_BF16 COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FZLRM COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRCS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_HRESET COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_LAM COMMON_CPUID_INDEX_7_ECX_1
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define index_cpu_AESKLE COMMON_CPUID_INDEX_19
-#define index_cpu_WIDE_KL COMMON_CPUID_INDEX_19
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define reg_SSE3 ecx
-#define reg_PCLMULQDQ ecx
-#define reg_DTES64 ecx
-#define reg_MONITOR ecx
-#define reg_DS_CPL ecx
-#define reg_VMX ecx
-#define reg_SMX ecx
-#define reg_EIST ecx
-#define reg_TM2 ecx
-#define reg_SSSE3 ecx
-#define reg_CNXT_ID ecx
-#define reg_SDBG ecx
-#define reg_FMA ecx
-#define reg_CMPXCHG16B ecx
-#define reg_XTPRUPDCTRL ecx
-#define reg_PDCM ecx
-#define reg_INDEX_1_ECX_16 ecx
-#define reg_PCID ecx
-#define reg_DCA ecx
-#define reg_SSE4_1 ecx
-#define reg_SSE4_2 ecx
-#define reg_X2APIC ecx
-#define reg_MOVBE ecx
-#define reg_POPCNT ecx
-#define reg_TSC_DEADLINE ecx
-#define reg_AES ecx
-#define reg_XSAVE ecx
-#define reg_OSXSAVE ecx
-#define reg_AVX ecx
-#define reg_F16C ecx
-#define reg_RDRAND ecx
-#define reg_INDEX_1_ECX_31 ecx
-
-/* EDX. */
-#define reg_FPU edx
-#define reg_VME edx
-#define reg_DE edx
-#define reg_PSE edx
-#define reg_TSC edx
-#define reg_MSR edx
-#define reg_PAE edx
-#define reg_MCE edx
-#define reg_CX8 edx
-#define reg_APIC edx
-#define reg_INDEX_1_EDX_10 edx
-#define reg_SEP edx
-#define reg_MTRR edx
-#define reg_PGE edx
-#define reg_MCA edx
-#define reg_CMOV edx
-#define reg_PAT edx
-#define reg_PSE_36 edx
-#define reg_PSN edx
-#define reg_CLFSH edx
-#define reg_INDEX_1_EDX_20 edx
-#define reg_DS edx
-#define reg_ACPI edx
-#define reg_MMX edx
-#define reg_FXSR edx
-#define reg_SSE edx
-#define reg_SSE2 edx
-#define reg_SS edx
-#define reg_HTT edx
-#define reg_TM edx
-#define reg_INDEX_1_EDX_30 edx
-#define reg_PBE edx
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define reg_FSGSBASE ebx
-#define reg_TSC_ADJUST ebx
-#define reg_SGX ebx
-#define reg_BMI1 ebx
-#define reg_HLE ebx
-#define reg_BMI2 ebx
-#define reg_AVX2 ebx
-#define reg_INDEX_7_EBX_6 ebx
-#define reg_SMEP ebx
-#define reg_ERMS ebx
-#define reg_INVPCID ebx
-#define reg_RTM ebx
-#define reg_RDT_M ebx
-#define reg_DEPR_FPU_CS_DS ebx
-#define reg_MPX ebx
-#define reg_RDT_A ebx
-#define reg_AVX512F ebx
-#define reg_AVX512DQ ebx
-#define reg_RDSEED ebx
-#define reg_ADX ebx
-#define reg_SMAP ebx
-#define reg_AVX512_IFMA ebx
-#define reg_INDEX_7_EBX_22 ebx
-#define reg_CLFLUSHOPT ebx
-#define reg_CLWB ebx
-#define reg_TRACE ebx
-#define reg_AVX512PF ebx
-#define reg_AVX512ER ebx
-#define reg_AVX512CD ebx
-#define reg_SHA ebx
-#define reg_AVX512BW ebx
-#define reg_AVX512VL ebx
-
-/* ECX. */
-#define reg_PREFETCHWT1 ecx
-#define reg_AVX512_VBMI ecx
-#define reg_UMIP ecx
-#define reg_PKU ecx
-#define reg_OSPKE ecx
-#define reg_WAITPKG ecx
-#define reg_AVX512_VBMI2 ecx
-#define reg_SHSTK ecx
-#define reg_GFNI ecx
-#define reg_VAES ecx
-#define reg_VPCLMULQDQ ecx
-#define reg_AVX512_VNNI ecx
-#define reg_AVX512_BITALG ecx
-#define reg_INDEX_7_ECX_13 ecx
-#define reg_AVX512_VPOPCNTDQ ecx
-#define reg_INDEX_7_ECX_15 ecx
-#define reg_INDEX_7_ECX_16 ecx
-#define reg_RDPID ecx
-#define reg_KL ecx
-#define reg_INDEX_7_ECX_24 ecx
-#define reg_CLDEMOTE ecx
-#define reg_INDEX_7_ECX_26 ecx
-#define reg_MOVDIRI ecx
-#define reg_MOVDIR64B ecx
-#define reg_ENQCMD ecx
-#define reg_SGX_LC ecx
-#define reg_PKS ecx
-
-/* EDX. */
-#define reg_INDEX_7_EDX_0 edx
-#define reg_INDEX_7_EDX_1 edx
-#define reg_AVX512_4VNNIW edx
-#define reg_AVX512_4FMAPS edx
-#define reg_FSRM edx
-#define reg_UINTR edx
-#define reg_INDEX_7_EDX_6 edx
-#define reg_INDEX_7_EDX_7 edx
-#define reg_AVX512_VP2INTERSECT edx
-#define reg_INDEX_7_EDX_9 edx
-#define reg_MD_CLEAR edx
-#define reg_INDEX_7_EDX_11 edx
-#define reg_INDEX_7_EDX_12 edx
-#define reg_INDEX_7_EDX_13 edx
-#define reg_SERIALIZE edx
-#define reg_HYBRID edx
-#define reg_TSXLDTRK edx
-#define reg_INDEX_7_EDX_17 edx
-#define reg_PCONFIG edx
-#define reg_INDEX_7_EDX_19 edx
-#define reg_IBT edx
-#define reg_INDEX_7_EDX_21 edx
-#define reg_AMX_BF16 edx
-#define reg_AVX512_FP16 edx
-#define reg_AMX_TILE edx
-#define reg_AMX_INT8 edx
-#define reg_IBRS_IBPB edx
-#define reg_STIBP edx
-#define reg_L1D_FLUSH edx
-#define reg_ARCH_CAPABILITIES edx
-#define reg_CORE_CAPABILITIES edx
-#define reg_SSBD edx
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define reg_LAHF64_SAHF64 ecx
-#define reg_SVM ecx
-#define reg_LZCNT ecx
-#define reg_SSE4A ecx
-#define reg_PREFETCHW ecx
-#define reg_XOP ecx
-#define reg_LWP ecx
-#define reg_FMA4 ecx
-#define reg_TBM ecx
-
-/* EDX. */
-#define reg_SYSCALL_SYSRET edx
-#define reg_NX edx
-#define reg_PAGE1GB edx
-#define reg_RDTSCP edx
-#define reg_LM edx
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define reg_XSAVEOPT eax
-#define reg_XSAVEC eax
-#define reg_XGETBV_ECX_1 eax
-#define reg_XSAVES eax
-#define reg_XFD eax
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define reg_INVARIANT_TSC edx
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define reg_WBNOINVD ebx
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define reg_AVX_VNNI eax
-#define reg_AVX512_BF16 eax
-#define reg_FZLRM eax
-#define reg_FSRS eax
-#define reg_FSRCS eax
-#define reg_HRESET eax
-#define reg_LAM eax
-
-/* COMMON_CPUID_INDEX_19. */
+ __x86_get_cpu_features (x86_cpu_##name); \
+ __ptr && x86_cpu_is_usable (__ptr, x86_cpu_##name); }))
-/* EBX. */
-#define reg_AESKLE ebx
-#define reg_WIDE_KL ebx
+__END_DECLS
#endif /* _SYS_PLATFORM_X86_H */
diff --git a/sysdeps/x86/tst-get-cpu-features-2-static.c b/sysdeps/x86/tst-get-cpu-features-2-static.c
new file mode 100644
index 0000000000..a86383ab62
--- /dev/null
+++ b/sysdeps/x86/tst-get-cpu-features-2-static.c
@@ -0,0 +1 @@
+#include "tst-get-cpu-features-2.c"
diff --git a/sysdeps/x86/tst-get-cpu-features-2.c b/sysdeps/x86/tst-get-cpu-features-2.c
new file mode 100644
index 0000000000..2c8cc1bf19
--- /dev/null
+++ b/sysdeps/x86/tst-get-cpu-features-2.c
@@ -0,0 +1,38 @@
+/* Forward compatibility test case for __x86_get_cpu_features interface
+ Copyright (C) 2020 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#include <stdlib.h>
+#include <stdio.h>
+#define COMMON_CPUID_INDEX_MAX __rename_COMMON_CPUID_INDEX_MAX
+#include <sys/platform/x86.h>
+#undef COMMON_CPUID_INDEX_MAX
+#define COMMON_CPUID_INDEX_MAX (__rename_COMMON_CPUID_INDEX_MAX + 1)
+#include <support/check.h>
+
+static int
+do_test (void)
+{
+#ifdef __SSE2__
+ TEST_VERIFY_EXIT (HAS_CPU_FEATURE (SSE2));
+ TEST_VERIFY_EXIT (CPU_FEATURE_USABLE (SSE2));
+#endif
+
+ return 0;
+}
+
+#include <support/test-driver.c>
--
2.29.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2020-12-25 16:24 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic H.J. Lu via Libc-alpha
@ 2021-01-20 9:53 ` Florian Weimer via Libc-alpha
2021-01-20 11:23 ` Florian Weimer via Libc-alpha
2021-01-20 14:43 ` V2 " H.J. Lu via Libc-alpha
0 siblings, 2 replies; 10+ messages in thread
From: Florian Weimer via Libc-alpha @ 2021-01-20 9:53 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha
* H. J. Lu via Libc-alpha:
> +enum
> +{
> + COMMON_CPUID_INDEX_1 = 0,
> + COMMON_CPUID_INDEX_7,
> + COMMON_CPUID_INDEX_80000001,
> + COMMON_CPUID_INDEX_D_ECX_1,
> + COMMON_CPUID_INDEX_80000007,
> + COMMON_CPUID_INDEX_80000008,
> + COMMON_CPUID_INDEX_7_ECX_1,
> + COMMON_CPUID_INDEX_19,
> + /* Keep the following line at the end. */
> + COMMON_CPUID_INDEX_MAX
> +};
COMMON_CPUID_INDEX_MAX should not be in the public header because it
subject to changes.
> +struct cpuid_registers
> +{
> + unsigned int eax;
> + unsigned int ebx;
> + unsigned int ecx;
> + unsigned int edx;
> +};
Should the non-public interfaces use __?
> +enum cpu_features_kind
> +{
> + arch_kind_unknown = 0,
> + arch_kind_intel,
> + arch_kind_amd,
> + arch_kind_zhaoxin,
> + arch_kind_other
> +};
> +
> +struct cpu_features_basic
> +{
> + enum cpu_features_kind kind;
> + int max_cpuid;
> + unsigned int family;
> + unsigned int model;
> + unsigned int stepping;
> +};
Should we really expose all this? It's not documented. enum
cpu_features_kind doesn't seem to be compatible with how people use
virtualization.
> diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
> index 349472d99f..4636d9f4a7 100644
> --- a/sysdeps/x86/dl-get-cpu-features.c
> +++ b/sysdeps/x86/dl-get-cpu-features.c
> @@ -46,9 +46,9 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
> #undef __x86_get_cpu_features
>
> const struct cpu_features *
> -__x86_get_cpu_features (unsigned int max)
> +__x86_get_cpu_features (unsigned int index)
> {
> - if (max > COMMON_CPUID_INDEX_MAX)
> + if (index > COMMON_CPUID_INDEX_MAX * 8 * sizeof (unsigned int) * 4)
> return NULL;
> return &GLRO(dl_x86_cpu_features);
> }
This should work in principle (but I haven't verified the boundary
condition).
Could you pass the COMMON_CPUID_INDEX_* value (without the register and
bit selectors)? Then more calls can be subject to common subexpression
elimination in the caller, leading to more compact code.
You could perhaps return a pointer to a zero struct in case the index is
too large. This way, the caller does not have to check for a null
pointer.
The result should be very close to what we had before in terms of
generated code, I hope.
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-20 9:53 ` Florian Weimer via Libc-alpha
@ 2021-01-20 11:23 ` Florian Weimer via Libc-alpha
2021-01-20 14:43 ` V2 " H.J. Lu via Libc-alpha
1 sibling, 0 replies; 10+ messages in thread
From: Florian Weimer via Libc-alpha @ 2021-01-20 11:23 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha
One more thing that I forgot: It should be okay to use a function for
this. If we ever get asymmetric multi-processing with vector
instructions inside rseq regions (so that if descheduled, they use a
non-vector fallback), anything based on a global variable will not work,
either. The feature check would have to look at per-CPU data, requiring
a different ABI anyway.
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 10+ messages in thread
* V2 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-20 9:53 ` Florian Weimer via Libc-alpha
2021-01-20 11:23 ` Florian Weimer via Libc-alpha
@ 2021-01-20 14:43 ` H.J. Lu via Libc-alpha
2021-01-20 15:58 ` Florian Weimer via Libc-alpha
1 sibling, 1 reply; 10+ messages in thread
From: H.J. Lu via Libc-alpha @ 2021-01-20 14:43 UTC (permalink / raw)
To: Florian Weimer; +Cc: libc-alpha
On Wed, Jan 20, 2021 at 10:53:07AM +0100, Florian Weimer wrote:
> * H. J. Lu via Libc-alpha:
>
> > +enum
> > +{
> > + COMMON_CPUID_INDEX_1 = 0,
> > + COMMON_CPUID_INDEX_7,
> > + COMMON_CPUID_INDEX_80000001,
> > + COMMON_CPUID_INDEX_D_ECX_1,
> > + COMMON_CPUID_INDEX_80000007,
> > + COMMON_CPUID_INDEX_80000008,
> > + COMMON_CPUID_INDEX_7_ECX_1,
> > + COMMON_CPUID_INDEX_19,
> > + /* Keep the following line at the end. */
> > + COMMON_CPUID_INDEX_MAX
> > +};
>
> COMMON_CPUID_INDEX_MAX should not be in the public header because it
> subject to changes.
Fixed.
>
> > +struct cpuid_registers
> > +{
> > + unsigned int eax;
> > + unsigned int ebx;
> > + unsigned int ecx;
> > + unsigned int edx;
> > +};
>
> Should the non-public interfaces use __?
Moved to sysdeps/x86/include/cpu-features.h.
>
> > +enum cpu_features_kind
> > +{
> > + arch_kind_unknown = 0,
> > + arch_kind_intel,
> > + arch_kind_amd,
> > + arch_kind_zhaoxin,
> > + arch_kind_other
> > +};
> > +
> > +struct cpu_features_basic
> > +{
> > + enum cpu_features_kind kind;
> > + int max_cpuid;
> > + unsigned int family;
> > + unsigned int model;
> > + unsigned int stepping;
> > +};
>
> Should we really expose all this? It's not documented. enum
> cpu_features_kind doesn't seem to be compatible with how people use
> virtualization.
Moved to sysdeps/x86/include/cpu-features.h.
>
> > diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
> > index 349472d99f..4636d9f4a7 100644
> > --- a/sysdeps/x86/dl-get-cpu-features.c
> > +++ b/sysdeps/x86/dl-get-cpu-features.c
> > @@ -46,9 +46,9 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
> > #undef __x86_get_cpu_features
> >
> > const struct cpu_features *
> > -__x86_get_cpu_features (unsigned int max)
> > +__x86_get_cpu_features (unsigned int index)
> > {
> > - if (max > COMMON_CPUID_INDEX_MAX)
> > + if (index > COMMON_CPUID_INDEX_MAX * 8 * sizeof (unsigned int) * 4)
> > return NULL;
> > return &GLRO(dl_x86_cpu_features);
> > }
>
> This should work in principle (but I haven't verified the boundary
> condition).
>
> Could you pass the COMMON_CPUID_INDEX_* value (without the register and
> bit selectors)? Then more calls can be subject to common subexpression
> elimination in the caller, leading to more compact code.
My current public API uses a single enum index for each CPU feature
detection. Use 2 indices for each CPU feature makes it less user
friendly. Internally, we use the C preprocessor magic so that more
calls can be subject to common subexpression elimination in the caller,
leading to more compact code for glibc.
>
> You could perhaps return a pointer to a zero struct in case the index is
> too large. This way, the caller does not have to check for a null
> pointer.
We can't do that since the size of struct increases when CPUID_INDEX_MAX
increases.
>
> The result should be very close to what we had before in terms of
> generated code, I hope.
>
Here is the updated patch. OK for master?
Thanks.
H.J.
---
In <sys/platform/x86.h>, define CPU features as enum instead of using
the C preprocessor magic to make it easier to wrap this functionality
in other languages. Move the C preprocessor magic to internal header
for better GCC codegen when more than one features are checked in a
single expression as in x86-64 dl-hwcaps-subdirs.c.
1. Rename COMMON_CPUID_INDEX_XXX to CPUID_INDEX_XXX.
2. Move CPUID_INDEX_MAX to sysdeps/x86/include/cpu-features.h.
3. Remove COMMON_CPUID_INDEX_MAX reference in manual/platform.texi.
4. Don't put struct cpu_features_basic in <sys/platform/x86.h>. Leave
only max_cpuid for struct cpu_features with flexible array.
5. Use a single enum index for each CPU feature detection.
6. Pass the feature index to __x86_get_cpu_features.
7. Return NULL to work with the older glibc binary with a smaller
CPUID_INDEX_MAX [BZ #27104].
8. Inside glibc, use the C preprocessor magic so that more calls can be
subject to common subexpression elimination in the caller, leading to
more compact code for glibc.
256 bits are used for each CPUID leaf. Some leaves only contain a few
features. We can add exceptions to such leaves. But it will increase
code sizes and it is harder to provide backward/forward compatibilities
when new features are added to such leaves in the future.
When new leaves are added, _rtld_global_ro offsets will change which
leads to race condition during in-place updates. We may avoid in-place
updates by
1. Rename the old glibc.
2. Install the new glibc.
3. Remove the old glibc.
NB: A function, __x86_get_cpu_features, is used to avoid the copy
relocation issue with IFUNC resolver as shown in IFUNC resolver tests.
---
manual/platform.texi | 9 +-
.../sysv/linux/x86_64/tst-glibc-hwcaps-2.c | 3 +-
sysdeps/x86/bits/platform/x86.h | 305 +++++++
sysdeps/x86/cpu-features.c | 78 +-
sysdeps/x86/dl-cacheinfo.h | 8 +-
sysdeps/x86/dl-get-cpu-features.c | 4 +-
sysdeps/x86/include/cpu-features.h | 749 ++++++++++++++++-
sysdeps/x86/sys/platform/x86.h | 768 +-----------------
sysdeps/x86/tst-get-cpu-features.c | 27 -
sysdeps/x86/tst-isa-level-1.c | 3 +-
sysdeps/x86_64/dl-hwcaps-subdirs.c | 3 +-
sysdeps/x86_64/tst-glibc-hwcaps.c | 3 +-
12 files changed, 1133 insertions(+), 827 deletions(-)
create mode 100644 sysdeps/x86/bits/platform/x86.h
diff --git a/manual/platform.texi b/manual/platform.texi
index b67683aeb3..9b8d51c41f 100644
--- a/manual/platform.texi
+++ b/manual/platform.texi
@@ -142,13 +142,12 @@ reserved.
Facilities specific to X86 that are not specific to a particular
operating system are declared in @file{sys/platform/x86.h}.
-@deftypefun {const struct cpu_features *} __x86_get_cpu_features (unsigned int @var{max})
+@deftypefun {const struct cpu_features *} __x86_get_cpu_features (unsigned int @var{leaf})
@safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}}
Return a pointer to x86 CPU feature structure used by query macros for x86
-CPU features. If @var{max} exceeds @code{COMMON_CPUID_INDEX_MAX} which
-is the limit of the CPUID leaves supported by @Theglibc{}, the function
-returns @code{NULL}, indicating that the queried processor feature is
-unsupported by @Theglibc{} run-time.
+CPU features in the CPUID @var{leaf}. If the function returns @code{NULL},
+indicating that the queried processor feature is unsupported by @Theglibc{}
+run-time.
@end deftypefun
@deftypefn Macro int HAS_CPU_FEATURE (@var{name})
diff --git a/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c b/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
index fe91bfd224..557cd9f876 100644
--- a/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
+++ b/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
@@ -29,8 +29,7 @@ extern int dso_isa_level (void);
static int
do_test (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
unsigned int isa_level = get_isa_level (cpu_features);
bool has_isa_baseline = ((isa_level & GNU_PROPERTY_X86_ISA_1_BASELINE)
== GNU_PROPERTY_X86_ISA_1_BASELINE);
diff --git a/sysdeps/x86/bits/platform/x86.h b/sysdeps/x86/bits/platform/x86.h
new file mode 100644
index 0000000000..b1a567ab51
--- /dev/null
+++ b/sysdeps/x86/bits/platform/x86.h
@@ -0,0 +1,305 @@
+/* Constants and data structures for x86 CPU features.
+ This file is part of the GNU C Library.
+ Copyright (C) 2008-2020 Free Software Foundation, Inc.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_PLATFORM_X86_H
+# error "Never include <bits/platform/x86.h> directly; use <sys/platform/x86.h> instead."
+#endif
+
+enum
+{
+ CPUID_INDEX_1 = 0,
+ CPUID_INDEX_7,
+ CPUID_INDEX_80000001,
+ CPUID_INDEX_D_ECX_1,
+ CPUID_INDEX_80000007,
+ CPUID_INDEX_80000008,
+ CPUID_INDEX_7_ECX_1,
+ CPUID_INDEX_19
+};
+
+struct cpuid_feature
+{
+ unsigned int cpuid_array[4];
+ unsigned int usable_array[4];
+};
+
+enum cpuid_register_index
+{
+ cpuid_register_index_eax = 0,
+ cpuid_register_index_ebx,
+ cpuid_register_index_ecx,
+ cpuid_register_index_edx
+};
+
+struct cpu_features
+{
+ int max_cpuid;
+ struct cpuid_feature features[];
+};
+
+/* CPU features. */
+
+enum
+{
+ x86_cpu_index_1_ecx
+ = (CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SSE3 = x86_cpu_index_1_ecx,
+ x86_cpu_PCLMULQDQ = x86_cpu_index_1_ecx + 1,
+ x86_cpu_DTES64 = x86_cpu_index_1_ecx + 2,
+ x86_cpu_MONITOR = x86_cpu_index_1_ecx + 3,
+ x86_cpu_DS_CPL = x86_cpu_index_1_ecx + 4,
+ x86_cpu_VMX = x86_cpu_index_1_ecx + 5,
+ x86_cpu_SMX = x86_cpu_index_1_ecx + 6,
+ x86_cpu_EIST = x86_cpu_index_1_ecx + 7,
+ x86_cpu_TM2 = x86_cpu_index_1_ecx + 8,
+ x86_cpu_SSSE3 = x86_cpu_index_1_ecx + 9,
+ x86_cpu_CNXT_ID = x86_cpu_index_1_ecx + 10,
+ x86_cpu_SDBG = x86_cpu_index_1_ecx + 11,
+ x86_cpu_FMA = x86_cpu_index_1_ecx + 12,
+ x86_cpu_CMPXCHG16B = x86_cpu_index_1_ecx + 13,
+ x86_cpu_XTPRUPDCTRL = x86_cpu_index_1_ecx + 14,
+ x86_cpu_PDCM = x86_cpu_index_1_ecx + 15,
+ x86_cpu_INDEX_1_ECX_16 = x86_cpu_index_1_ecx + 16,
+ x86_cpu_PCID = x86_cpu_index_1_ecx + 17,
+ x86_cpu_DCA = x86_cpu_index_1_ecx + 18,
+ x86_cpu_SSE4_1 = x86_cpu_index_1_ecx + 19,
+ x86_cpu_SSE4_2 = x86_cpu_index_1_ecx + 20,
+ x86_cpu_X2APIC = x86_cpu_index_1_ecx + 21,
+ x86_cpu_MOVBE = x86_cpu_index_1_ecx + 22,
+ x86_cpu_POPCNT = x86_cpu_index_1_ecx + 23,
+ x86_cpu_TSC_DEADLINE = x86_cpu_index_1_ecx + 24,
+ x86_cpu_AES = x86_cpu_index_1_ecx + 25,
+ x86_cpu_XSAVE = x86_cpu_index_1_ecx + 26,
+ x86_cpu_OSXSAVE = x86_cpu_index_1_ecx + 27,
+ x86_cpu_AVX = x86_cpu_index_1_ecx + 28,
+ x86_cpu_F16C = x86_cpu_index_1_ecx + 29,
+ x86_cpu_RDRAND = x86_cpu_index_1_ecx + 30,
+ x86_cpu_INDEX_1_ECX_31 = x86_cpu_index_1_ecx + 31,
+
+ x86_cpu_index_1_edx
+ = (CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FPU = x86_cpu_index_1_edx,
+ x86_cpu_VME = x86_cpu_index_1_edx + 1,
+ x86_cpu_DE = x86_cpu_index_1_edx + 2,
+ x86_cpu_PSE = x86_cpu_index_1_edx + 3,
+ x86_cpu_TSC = x86_cpu_index_1_edx + 4,
+ x86_cpu_MSR = x86_cpu_index_1_edx + 5,
+ x86_cpu_PAE = x86_cpu_index_1_edx + 6,
+ x86_cpu_MCE = x86_cpu_index_1_edx + 7,
+ x86_cpu_CX8 = x86_cpu_index_1_edx + 8,
+ x86_cpu_APIC = x86_cpu_index_1_edx + 9,
+ x86_cpu_INDEX_1_EDX_10 = x86_cpu_index_1_edx + 10,
+ x86_cpu_SEP = x86_cpu_index_1_edx + 11,
+ x86_cpu_MTRR = x86_cpu_index_1_edx + 12,
+ x86_cpu_PGE = x86_cpu_index_1_edx + 13,
+ x86_cpu_MCA = x86_cpu_index_1_edx + 14,
+ x86_cpu_CMOV = x86_cpu_index_1_edx + 15,
+ x86_cpu_PAT = x86_cpu_index_1_edx + 16,
+ x86_cpu_PSE_36 = x86_cpu_index_1_edx + 17,
+ x86_cpu_PSN = x86_cpu_index_1_edx + 18,
+ x86_cpu_CLFSH = x86_cpu_index_1_edx + 19,
+ x86_cpu_INDEX_1_EDX_20 = x86_cpu_index_1_edx + 20,
+ x86_cpu_DS = x86_cpu_index_1_edx + 21,
+ x86_cpu_ACPI = x86_cpu_index_1_edx + 22,
+ x86_cpu_MMX = x86_cpu_index_1_edx + 23,
+ x86_cpu_FXSR = x86_cpu_index_1_edx + 24,
+ x86_cpu_SSE = x86_cpu_index_1_edx + 25,
+ x86_cpu_SSE2 = x86_cpu_index_1_edx + 26,
+ x86_cpu_SS = x86_cpu_index_1_edx + 27,
+ x86_cpu_HTT = x86_cpu_index_1_edx + 28,
+ x86_cpu_TM = x86_cpu_index_1_edx + 29,
+ x86_cpu_INDEX_1_EDX_30 = x86_cpu_index_1_edx + 30,
+ x86_cpu_PBE = x86_cpu_index_1_edx + 31,
+
+ x86_cpu_index_7_ebx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FSGSBASE = x86_cpu_index_7_ebx,
+ x86_cpu_TSC_ADJUST = x86_cpu_index_7_ebx + 1,
+ x86_cpu_SGX = x86_cpu_index_7_ebx + 2,
+ x86_cpu_BMI1 = x86_cpu_index_7_ebx + 3,
+ x86_cpu_HLE = x86_cpu_index_7_ebx + 4,
+ x86_cpu_AVX2 = x86_cpu_index_7_ebx + 5,
+ x86_cpu_INDEX_7_EBX_6 = x86_cpu_index_7_ebx + 6,
+ x86_cpu_SMEP = x86_cpu_index_7_ebx + 7,
+ x86_cpu_BMI2 = x86_cpu_index_7_ebx + 8,
+ x86_cpu_ERMS = x86_cpu_index_7_ebx + 9,
+ x86_cpu_INVPCID = x86_cpu_index_7_ebx + 10,
+ x86_cpu_RTM = x86_cpu_index_7_ebx + 11,
+ x86_cpu_RDT_M = x86_cpu_index_7_ebx + 12,
+ x86_cpu_DEPR_FPU_CS_DS = x86_cpu_index_7_ebx + 13,
+ x86_cpu_MPX = x86_cpu_index_7_ebx + 14,
+ x86_cpu_RDT_A = x86_cpu_index_7_ebx + 15,
+ x86_cpu_AVX512F = x86_cpu_index_7_ebx + 16,
+ x86_cpu_AVX512DQ = x86_cpu_index_7_ebx + 17,
+ x86_cpu_RDSEED = x86_cpu_index_7_ebx + 18,
+ x86_cpu_ADX = x86_cpu_index_7_ebx + 19,
+ x86_cpu_SMAP = x86_cpu_index_7_ebx + 20,
+ x86_cpu_AVX512_IFMA = x86_cpu_index_7_ebx + 21,
+ x86_cpu_INDEX_7_EBX_22 = x86_cpu_index_7_ebx + 22,
+ x86_cpu_CLFLUSHOPT = x86_cpu_index_7_ebx + 23,
+ x86_cpu_CLWB = x86_cpu_index_7_ebx + 24,
+ x86_cpu_TRACE = x86_cpu_index_7_ebx + 25,
+ x86_cpu_AVX512PF = x86_cpu_index_7_ebx + 26,
+ x86_cpu_AVX512ER = x86_cpu_index_7_ebx + 27,
+ x86_cpu_AVX512CD = x86_cpu_index_7_ebx + 28,
+ x86_cpu_SHA = x86_cpu_index_7_ebx + 29,
+ x86_cpu_AVX512BW = x86_cpu_index_7_ebx + 30,
+ x86_cpu_AVX512VL = x86_cpu_index_7_ebx + 31,
+
+ x86_cpu_index_7_ecx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_PREFETCHWT1 = x86_cpu_index_7_ecx,
+ x86_cpu_AVX512_VBMI = x86_cpu_index_7_ecx + 1,
+ x86_cpu_UMIP = x86_cpu_index_7_ecx + 2,
+ x86_cpu_PKU = x86_cpu_index_7_ecx + 3,
+ x86_cpu_OSPKE = x86_cpu_index_7_ecx + 4,
+ x86_cpu_WAITPKG = x86_cpu_index_7_ecx + 5,
+ x86_cpu_AVX512_VBMI2 = x86_cpu_index_7_ecx + 6,
+ x86_cpu_SHSTK = x86_cpu_index_7_ecx + 7,
+ x86_cpu_GFNI = x86_cpu_index_7_ecx + 8,
+ x86_cpu_VAES = x86_cpu_index_7_ecx + 9,
+ x86_cpu_VPCLMULQDQ = x86_cpu_index_7_ecx + 10,
+ x86_cpu_AVX512_VNNI = x86_cpu_index_7_ecx + 11,
+ x86_cpu_AVX512_BITALG = x86_cpu_index_7_ecx + 12,
+ x86_cpu_INDEX_7_ECX_13 = x86_cpu_index_7_ecx + 13,
+ x86_cpu_AVX512_VPOPCNTDQ = x86_cpu_index_7_ecx + 14,
+ x86_cpu_INDEX_7_ECX_1 = x86_cpu_index_7_ecx + 15,
+ x86_cpu_INDEX_7_ECX_16 = x86_cpu_index_7_ecx + 16,
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+ x86_cpu_RDPID = x86_cpu_index_7_ecx + 22,
+ x86_cpu_KL = x86_cpu_index_7_ecx + 23,
+ x86_cpu_INDEX_7_ECX_24 = x86_cpu_index_7_ecx + 24,
+ x86_cpu_CLDEMOTE = x86_cpu_index_7_ecx + 25,
+ x86_cpu_INDEX_7_ECX_26 = x86_cpu_index_7_ecx + 26,
+ x86_cpu_MOVDIRI = x86_cpu_index_7_ecx + 27,
+ x86_cpu_MOVDIR64B = x86_cpu_index_7_ecx + 28,
+ x86_cpu_ENQCMD = x86_cpu_index_7_ecx + 29,
+ x86_cpu_SGX_LC = x86_cpu_index_7_ecx + 30,
+ x86_cpu_PKS = x86_cpu_index_7_ecx + 31,
+
+ x86_cpu_index_7_edx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INDEX_7_EDX_0 = x86_cpu_index_7_edx,
+ x86_cpu_INDEX_7_EDX_1 = x86_cpu_index_7_edx + 1,
+ x86_cpu_AVX512_4VNNIW = x86_cpu_index_7_edx + 2,
+ x86_cpu_AVX512_4FMAPS = x86_cpu_index_7_edx + 3,
+ x86_cpu_FSRM = x86_cpu_index_7_edx + 4,
+ x86_cpu_UINTR = x86_cpu_index_7_edx + 5,
+ x86_cpu_INDEX_7_EDX_6 = x86_cpu_index_7_edx + 6,
+ x86_cpu_INDEX_7_EDX_7 = x86_cpu_index_7_edx + 7,
+ x86_cpu_AVX512_VP2INTERSECT = x86_cpu_index_7_edx + 8,
+ x86_cpu_INDEX_7_EDX_9 = x86_cpu_index_7_edx + 9,
+ x86_cpu_MD_CLEAR = x86_cpu_index_7_edx + 10,
+ x86_cpu_INDEX_7_EDX_11 = x86_cpu_index_7_edx + 11,
+ x86_cpu_INDEX_7_EDX_12 = x86_cpu_index_7_edx + 12,
+ x86_cpu_INDEX_7_EDX_13 = x86_cpu_index_7_edx + 13,
+ x86_cpu_SERIALIZE = x86_cpu_index_7_edx + 14,
+ x86_cpu_HYBRID = x86_cpu_index_7_edx + 15,
+ x86_cpu_TSXLDTRK = x86_cpu_index_7_edx + 16,
+ x86_cpu_INDEX_7_EDX_17 = x86_cpu_index_7_edx + 17,
+ x86_cpu_PCONFIG = x86_cpu_index_7_edx + 18,
+ x86_cpu_INDEX_7_EDX_19 = x86_cpu_index_7_edx + 19,
+ x86_cpu_IBT = x86_cpu_index_7_edx + 20,
+ x86_cpu_INDEX_7_EDX_21 = x86_cpu_index_7_edx + 21,
+ x86_cpu_AMX_BF16 = x86_cpu_index_7_edx + 22,
+ x86_cpu_AVX512_FP16 = x86_cpu_index_7_edx + 23,
+ x86_cpu_AMX_TILE = x86_cpu_index_7_edx + 24,
+ x86_cpu_AMX_INT8 = x86_cpu_index_7_edx + 25,
+ x86_cpu_IBRS_IBPB = x86_cpu_index_7_edx + 26,
+ x86_cpu_STIBP = x86_cpu_index_7_edx + 27,
+ x86_cpu_L1D_FLUSH = x86_cpu_index_7_edx + 28,
+ x86_cpu_ARCH_CAPABILITIES = x86_cpu_index_7_edx + 29,
+ x86_cpu_CORE_CAPABILITIES = x86_cpu_index_7_edx + 30,
+ x86_cpu_SSBD = x86_cpu_index_7_edx + 31,
+
+ x86_cpu_index_80000001_ecx
+ = (CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_LAHF64_SAHF64 = x86_cpu_index_80000001_ecx,
+ x86_cpu_SVM = x86_cpu_index_80000001_ecx + 2,
+ x86_cpu_LZCNT = x86_cpu_index_80000001_ecx + 5,
+ x86_cpu_SSE4A = x86_cpu_index_80000001_ecx + 6,
+ x86_cpu_PREFETCHW = x86_cpu_index_80000001_ecx + 8,
+ x86_cpu_XOP = x86_cpu_index_80000001_ecx + 11,
+ x86_cpu_LWP = x86_cpu_index_80000001_ecx + 15,
+ x86_cpu_FMA4 = x86_cpu_index_80000001_ecx + 16,
+ x86_cpu_TBM = x86_cpu_index_80000001_ecx + 20,
+
+ x86_cpu_index_80000001_edx
+ = (CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SYSCALL_SYSRET = x86_cpu_index_80000001_edx + 11,
+ x86_cpu_NX = x86_cpu_index_80000001_edx + 20,
+ x86_cpu_PAGE1GB = x86_cpu_index_80000001_edx + 26,
+ x86_cpu_RDTSCP = x86_cpu_index_80000001_edx + 27,
+ x86_cpu_LM = x86_cpu_index_80000001_edx + 29,
+
+ x86_cpu_index_d_ecx_1_eax
+ = (CPUID_INDEX_D_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_XSAVEOPT = x86_cpu_index_d_ecx_1_eax,
+ x86_cpu_XSAVEC = x86_cpu_index_d_ecx_1_eax + 1,
+ x86_cpu_XGETBV_ECX_1 = x86_cpu_index_d_ecx_1_eax + 2,
+ x86_cpu_XSAVES = x86_cpu_index_d_ecx_1_eax + 3,
+ x86_cpu_XFD = x86_cpu_index_d_ecx_1_eax + 4,
+
+ x86_cpu_index_80000007_edx
+ = (CPUID_INDEX_80000007 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INVARIANT_TSC = x86_cpu_index_80000007_edx + 8,
+
+ x86_cpu_index_80000008_ebx
+ = (CPUID_INDEX_80000008 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_WBNOINVD = x86_cpu_index_80000008_ebx + 9,
+
+ x86_cpu_index_7_ecx_1_eax
+ = (CPUID_INDEX_7_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AVX_VNNI = x86_cpu_index_7_ecx_1_eax + 4,
+ x86_cpu_AVX512_BF16 = x86_cpu_index_7_ecx_1_eax + 5,
+ x86_cpu_FZLRM = x86_cpu_index_7_ecx_1_eax + 10,
+ x86_cpu_FSRS = x86_cpu_index_7_ecx_1_eax + 11,
+ x86_cpu_FSRCS = x86_cpu_index_7_ecx_1_eax + 12,
+ x86_cpu_HRESET = x86_cpu_index_7_ecx_1_eax + 22,
+ x86_cpu_LAM = x86_cpu_index_7_ecx_1_eax + 26,
+
+ x86_cpu_index_19_ebx
+ = (CPUID_INDEX_19 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AESKLE = x86_cpu_index_19_ebx,
+ x86_cpu_WIDE_KL = x86_cpu_index_19_ebx + 2
+};
diff --git a/sysdeps/x86/cpu-features.c b/sysdeps/x86/cpu-features.c
index 06e4307b71..148d8f1fd8 100644
--- a/sysdeps/x86/cpu-features.c
+++ b/sysdeps/x86/cpu-features.c
@@ -205,7 +205,7 @@ update_usable (struct cpu_features *cpu_features)
/* For _dl_runtime_resolve, set xsave_state_size to xsave area
size + integer register save size and align it to 64 bytes. */
- if (cpu_features->basic.max_cpuid >= 0xd)
+ if (cpu_features->max_cpuid >= 0xd)
{
unsigned int eax, ebx, ecx, edx;
@@ -293,22 +293,22 @@ get_extended_indices (struct cpu_features *cpu_features)
__cpuid (0x80000000, eax, ebx, ecx, edx);
if (eax >= 0x80000001)
__cpuid (0x80000001,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.edx);
if (eax >= 0x80000007)
__cpuid (0x80000007,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.edx);
if (eax >= 0x80000008)
__cpuid (0x80000008,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.edx);
}
static void
@@ -320,10 +320,10 @@ get_common_indices (struct cpu_features *cpu_features,
{
unsigned int eax;
__cpuid (1, eax,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.edx);
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.eax = eax;
+ cpu_features->features[CPUID_INDEX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_1].cpuid.eax = eax;
*family = (eax >> 8) & 0x0f;
*model = (eax >> 4) & 0x0f;
*extended_model = (eax >> 12) & 0xf0;
@@ -335,33 +335,33 @@ get_common_indices (struct cpu_features *cpu_features,
}
}
- if (cpu_features->basic.max_cpuid >= 7)
+ if (cpu_features->max_cpuid >= 7)
{
__cpuid_count (7, 0,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_7].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_7].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_7].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_7].cpuid.edx);
__cpuid_count (7, 1,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.edx);
}
- if (cpu_features->basic.max_cpuid >= 0xd)
+ if (cpu_features->max_cpuid >= 0xd)
__cpuid_count (0xd, 1,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.edx);
- if (cpu_features->basic.max_cpuid >= 0x19)
+ if (cpu_features->max_cpuid >= 0x19)
__cpuid_count (0x19, 0,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_19].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_19].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_19].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_19].cpuid.edx);
}
_Static_assert (((index_arch_Fast_Unaligned_Load
@@ -393,7 +393,7 @@ init_cpu_features (struct cpu_features *cpu_features)
}
#endif
- __cpuid (0, cpu_features->basic.max_cpuid, ebx, ecx, edx);
+ __cpuid (0, cpu_features->max_cpuid, ebx, ecx, edx);
/* This spells out "GenuineIntel". */
if (ebx == 0x756e6547 && ecx == 0x6c65746e && edx == 0x49656e69)
@@ -536,11 +536,11 @@ init_cpu_features (struct cpu_features *cpu_features)
update_usable (cpu_features);
- ecx = cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ecx;
+ ecx = cpu_features->features[CPUID_INDEX_1].cpuid.ecx;
if (CPU_FEATURE_USABLE_P (cpu_features, AVX))
{
- /* Since the FMA4 bit is in COMMON_CPUID_INDEX_80000001 and
+ /* Since the FMA4 bit is in CPUID_INDEX_80000001 and
FMA4 requires AVX, determine if FMA4 is usable here. */
CPU_FEATURE_SET_USABLE (cpu_features, FMA4);
}
diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h
index 96c49f2411..f92e4bc28b 100644
--- a/sysdeps/x86/dl-cacheinfo.h
+++ b/sysdeps/x86/dl-cacheinfo.h
@@ -249,7 +249,7 @@ intel_check_word (int name, unsigned int value, bool *has_level_2,
static long int __attribute__ ((noinline))
handle_intel (int name, const struct cpu_features *cpu_features)
{
- unsigned int maxidx = cpu_features->basic.max_cpuid;
+ unsigned int maxidx = cpu_features->max_cpuid;
/* Return -1 for older CPUs. */
if (maxidx < 2)
@@ -493,7 +493,7 @@ get_common_cache_info (long int *shared_ptr, unsigned int *threads_ptr,
int threads_l3;
const struct cpu_features *cpu_features = __get_cpu_features ();
- int max_cpuid = cpu_features->basic.max_cpuid;
+ int max_cpuid = cpu_features->max_cpuid;
unsigned int family = cpu_features->basic.family;
unsigned int model = cpu_features->basic.model;
long int shared = *shared_ptr;
@@ -672,8 +672,8 @@ intel_bug_no_cache_info:
/* Assume that all logical threads share the highest cache
level. */
threads
- = ((cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ebx
- >> 16) & 0xff);
+ = ((cpu_features->features[CPUID_INDEX_1].cpuid.ebx >> 16)
+ & 0xff);
}
/* Cap usage of highest cache level to the number of supported
diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
index 19fc4f1eeb..492c0881b9 100644
--- a/sysdeps/x86/dl-get-cpu-features.c
+++ b/sysdeps/x86/dl-get-cpu-features.c
@@ -46,9 +46,9 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
#undef __x86_get_cpu_features
const struct cpu_features *
-__x86_get_cpu_features (unsigned int max)
+__x86_get_cpu_features (unsigned int index)
{
- if (max > COMMON_CPUID_INDEX_MAX)
+ if (index > CPUID_INDEX_MAX * 8 * sizeof (unsigned int) * 4)
return NULL;
return &GLRO(dl_x86_cpu_features);
}
diff --git a/sysdeps/x86/include/cpu-features.h b/sysdeps/x86/include/cpu-features.h
index 50cb5a15f5..d88a84c9a0 100644
--- a/sysdeps/x86/include/cpu-features.h
+++ b/sysdeps/x86/include/cpu-features.h
@@ -26,13 +26,17 @@
/* Get most of the contents from the public header, but we define a
different `struct cpu_features' type for private use. */
#define cpu_features cpu_features_public
-#define __x86_get_cpu_features __x86_get_cpu_features_public
-#include <sysdeps/x86/sys/platform/x86.h>
+/* Get data structures without inline functions. */
+#define _SYS_PLATFORM_X86_H
+#include <bits/platform/x86.h>
+
+enum
+{
+ CPUID_INDEX_MAX = CPUID_INDEX_19 + 1
+};
#undef cpu_features
-#undef __x86_get_cpu_features
-#define __get_cpu_features() __x86_get_cpu_features (0)
enum
{
@@ -54,12 +58,18 @@ enum
#define CPU_FEATURE_PREFERRED_P(ptr, name) \
((ptr->preferred[index_arch_##name] & bit_arch_##name) != 0)
+#define CPU_FEATURE_CHECK_P(ptr, name, check) \
+ ((ptr->features[index_cpu_##name].check.reg_##name \
+ & bit_cpu_##name) != 0)
+#define CPU_FEATURE_CPU_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, cpuid)
+#define CPU_FEATURE_USABLE_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, usable)
+
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
-#undef HAS_CPU_FEATURE
#define HAS_CPU_FEATURE(name) \
CPU_FEATURE_CPU_P (__x86_get_cpu_features (0), name)
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
-#undef CPU_FEATURE_USABLE
#define CPU_FEATURE_USABLE(name) \
CPU_FEATURE_USABLE_P (__x86_get_cpu_features (0), name)
/* CPU_FEATURE_PREFER evaluates to true if we prefer the feature at
@@ -74,6 +84,685 @@ enum
#define HAS_ARCH_FEATURE(name) \
CPU_FEATURE_PREFERRED (name)
+/* CPU features. */
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define bit_cpu_SSE3 (1u << 0)
+#define bit_cpu_PCLMULQDQ (1u << 1)
+#define bit_cpu_DTES64 (1u << 2)
+#define bit_cpu_MONITOR (1u << 3)
+#define bit_cpu_DS_CPL (1u << 4)
+#define bit_cpu_VMX (1u << 5)
+#define bit_cpu_SMX (1u << 6)
+#define bit_cpu_EIST (1u << 7)
+#define bit_cpu_TM2 (1u << 8)
+#define bit_cpu_SSSE3 (1u << 9)
+#define bit_cpu_CNXT_ID (1u << 10)
+#define bit_cpu_SDBG (1u << 11)
+#define bit_cpu_FMA (1u << 12)
+#define bit_cpu_CMPXCHG16B (1u << 13)
+#define bit_cpu_XTPRUPDCTRL (1u << 14)
+#define bit_cpu_PDCM (1u << 15)
+#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
+#define bit_cpu_PCID (1u << 17)
+#define bit_cpu_DCA (1u << 18)
+#define bit_cpu_SSE4_1 (1u << 19)
+#define bit_cpu_SSE4_2 (1u << 20)
+#define bit_cpu_X2APIC (1u << 21)
+#define bit_cpu_MOVBE (1u << 22)
+#define bit_cpu_POPCNT (1u << 23)
+#define bit_cpu_TSC_DEADLINE (1u << 24)
+#define bit_cpu_AES (1u << 25)
+#define bit_cpu_XSAVE (1u << 26)
+#define bit_cpu_OSXSAVE (1u << 27)
+#define bit_cpu_AVX (1u << 28)
+#define bit_cpu_F16C (1u << 29)
+#define bit_cpu_RDRAND (1u << 30)
+#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
+
+/* EDX. */
+#define bit_cpu_FPU (1u << 0)
+#define bit_cpu_VME (1u << 1)
+#define bit_cpu_DE (1u << 2)
+#define bit_cpu_PSE (1u << 3)
+#define bit_cpu_TSC (1u << 4)
+#define bit_cpu_MSR (1u << 5)
+#define bit_cpu_PAE (1u << 6)
+#define bit_cpu_MCE (1u << 7)
+#define bit_cpu_CX8 (1u << 8)
+#define bit_cpu_APIC (1u << 9)
+#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
+#define bit_cpu_SEP (1u << 11)
+#define bit_cpu_MTRR (1u << 12)
+#define bit_cpu_PGE (1u << 13)
+#define bit_cpu_MCA (1u << 14)
+#define bit_cpu_CMOV (1u << 15)
+#define bit_cpu_PAT (1u << 16)
+#define bit_cpu_PSE_36 (1u << 17)
+#define bit_cpu_PSN (1u << 18)
+#define bit_cpu_CLFSH (1u << 19)
+#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
+#define bit_cpu_DS (1u << 21)
+#define bit_cpu_ACPI (1u << 22)
+#define bit_cpu_MMX (1u << 23)
+#define bit_cpu_FXSR (1u << 24)
+#define bit_cpu_SSE (1u << 25)
+#define bit_cpu_SSE2 (1u << 26)
+#define bit_cpu_SS (1u << 27)
+#define bit_cpu_HTT (1u << 28)
+#define bit_cpu_TM (1u << 29)
+#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
+#define bit_cpu_PBE (1u << 31)
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define bit_cpu_FSGSBASE (1u << 0)
+#define bit_cpu_TSC_ADJUST (1u << 1)
+#define bit_cpu_SGX (1u << 2)
+#define bit_cpu_BMI1 (1u << 3)
+#define bit_cpu_HLE (1u << 4)
+#define bit_cpu_AVX2 (1u << 5)
+#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
+#define bit_cpu_SMEP (1u << 7)
+#define bit_cpu_BMI2 (1u << 8)
+#define bit_cpu_ERMS (1u << 9)
+#define bit_cpu_INVPCID (1u << 10)
+#define bit_cpu_RTM (1u << 11)
+#define bit_cpu_RDT_M (1u << 12)
+#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
+#define bit_cpu_MPX (1u << 14)
+#define bit_cpu_RDT_A (1u << 15)
+#define bit_cpu_AVX512F (1u << 16)
+#define bit_cpu_AVX512DQ (1u << 17)
+#define bit_cpu_RDSEED (1u << 18)
+#define bit_cpu_ADX (1u << 19)
+#define bit_cpu_SMAP (1u << 20)
+#define bit_cpu_AVX512_IFMA (1u << 21)
+#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
+#define bit_cpu_CLFLUSHOPT (1u << 23)
+#define bit_cpu_CLWB (1u << 24)
+#define bit_cpu_TRACE (1u << 25)
+#define bit_cpu_AVX512PF (1u << 26)
+#define bit_cpu_AVX512ER (1u << 27)
+#define bit_cpu_AVX512CD (1u << 28)
+#define bit_cpu_SHA (1u << 29)
+#define bit_cpu_AVX512BW (1u << 30)
+#define bit_cpu_AVX512VL (1u << 31)
+
+/* ECX. */
+#define bit_cpu_PREFETCHWT1 (1u << 0)
+#define bit_cpu_AVX512_VBMI (1u << 1)
+#define bit_cpu_UMIP (1u << 2)
+#define bit_cpu_PKU (1u << 3)
+#define bit_cpu_OSPKE (1u << 4)
+#define bit_cpu_WAITPKG (1u << 5)
+#define bit_cpu_AVX512_VBMI2 (1u << 6)
+#define bit_cpu_SHSTK (1u << 7)
+#define bit_cpu_GFNI (1u << 8)
+#define bit_cpu_VAES (1u << 9)
+#define bit_cpu_VPCLMULQDQ (1u << 10)
+#define bit_cpu_AVX512_VNNI (1u << 11)
+#define bit_cpu_AVX512_BITALG (1u << 12)
+#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
+#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
+#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
+#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+#define bit_cpu_RDPID (1u << 22)
+#define bit_cpu_KL (1u << 23)
+#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
+#define bit_cpu_CLDEMOTE (1u << 25)
+#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
+#define bit_cpu_MOVDIRI (1u << 27)
+#define bit_cpu_MOVDIR64B (1u << 28)
+#define bit_cpu_ENQCMD (1u << 29)
+#define bit_cpu_SGX_LC (1u << 30)
+#define bit_cpu_PKS (1u << 31)
+
+/* EDX. */
+#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
+#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
+#define bit_cpu_AVX512_4VNNIW (1u << 2)
+#define bit_cpu_AVX512_4FMAPS (1u << 3)
+#define bit_cpu_FSRM (1u << 4)
+#define bit_cpu_UINTR (1u << 5)
+#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
+#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
+#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
+#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
+#define bit_cpu_MD_CLEAR (1u << 10)
+#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
+#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
+#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
+#define bit_cpu_SERIALIZE (1u << 14)
+#define bit_cpu_HYBRID (1u << 15)
+#define bit_cpu_TSXLDTRK (1u << 16)
+#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
+#define bit_cpu_PCONFIG (1u << 18)
+#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
+#define bit_cpu_IBT (1u << 20)
+#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
+#define bit_cpu_AMX_BF16 (1u << 22)
+#define bit_cpu_AVX512_FP16 (1u << 23)
+#define bit_cpu_AMX_TILE (1u << 24)
+#define bit_cpu_AMX_INT8 (1u << 25)
+#define bit_cpu_IBRS_IBPB (1u << 26)
+#define bit_cpu_STIBP (1u << 27)
+#define bit_cpu_L1D_FLUSH (1u << 28)
+#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
+#define bit_cpu_CORE_CAPABILITIES (1u << 30)
+#define bit_cpu_SSBD (1u << 31)
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define bit_cpu_LAHF64_SAHF64 (1u << 0)
+#define bit_cpu_SVM (1u << 2)
+#define bit_cpu_LZCNT (1u << 5)
+#define bit_cpu_SSE4A (1u << 6)
+#define bit_cpu_PREFETCHW (1u << 8)
+#define bit_cpu_XOP (1u << 11)
+#define bit_cpu_LWP (1u << 15)
+#define bit_cpu_FMA4 (1u << 16)
+#define bit_cpu_TBM (1u << 21)
+
+/* EDX. */
+#define bit_cpu_SYSCALL_SYSRET (1u << 11)
+#define bit_cpu_NX (1u << 20)
+#define bit_cpu_PAGE1GB (1u << 26)
+#define bit_cpu_RDTSCP (1u << 27)
+#define bit_cpu_LM (1u << 29)
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_XSAVEOPT (1u << 0)
+#define bit_cpu_XSAVEC (1u << 1)
+#define bit_cpu_XGETBV_ECX_1 (1u << 2)
+#define bit_cpu_XSAVES (1u << 3)
+#define bit_cpu_XFD (1u << 4)
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define bit_cpu_INVARIANT_TSC (1u << 8)
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define bit_cpu_WBNOINVD (1u << 9)
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_AVX_VNNI (1u << 4)
+#define bit_cpu_AVX512_BF16 (1u << 5)
+#define bit_cpu_FZLRM (1u << 10)
+#define bit_cpu_FSRS (1u << 11)
+#define bit_cpu_FSRCS (1u << 12)
+#define bit_cpu_HRESET (1u << 22)
+#define bit_cpu_LAM (1u << 26)
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define bit_cpu_AESKLE (1u << 0)
+#define bit_cpu_WIDE_KL (1u << 2)
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define index_cpu_SSE3 CPUID_INDEX_1
+#define index_cpu_PCLMULQDQ CPUID_INDEX_1
+#define index_cpu_DTES64 CPUID_INDEX_1
+#define index_cpu_MONITOR CPUID_INDEX_1
+#define index_cpu_DS_CPL CPUID_INDEX_1
+#define index_cpu_VMX CPUID_INDEX_1
+#define index_cpu_SMX CPUID_INDEX_1
+#define index_cpu_EIST CPUID_INDEX_1
+#define index_cpu_TM2 CPUID_INDEX_1
+#define index_cpu_SSSE3 CPUID_INDEX_1
+#define index_cpu_CNXT_ID CPUID_INDEX_1
+#define index_cpu_SDBG CPUID_INDEX_1
+#define index_cpu_FMA CPUID_INDEX_1
+#define index_cpu_CMPXCHG16B CPUID_INDEX_1
+#define index_cpu_XTPRUPDCTRL CPUID_INDEX_1
+#define index_cpu_PDCM CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_16 CPUID_INDEX_1
+#define index_cpu_PCID CPUID_INDEX_1
+#define index_cpu_DCA CPUID_INDEX_1
+#define index_cpu_SSE4_1 CPUID_INDEX_1
+#define index_cpu_SSE4_2 CPUID_INDEX_1
+#define index_cpu_X2APIC CPUID_INDEX_1
+#define index_cpu_MOVBE CPUID_INDEX_1
+#define index_cpu_POPCNT CPUID_INDEX_1
+#define index_cpu_TSC_DEADLINE CPUID_INDEX_1
+#define index_cpu_AES CPUID_INDEX_1
+#define index_cpu_XSAVE CPUID_INDEX_1
+#define index_cpu_OSXSAVE CPUID_INDEX_1
+#define index_cpu_AVX CPUID_INDEX_1
+#define index_cpu_F16C CPUID_INDEX_1
+#define index_cpu_RDRAND CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_31 CPUID_INDEX_1
+
+/* ECX. */
+#define index_cpu_FPU CPUID_INDEX_1
+#define index_cpu_VME CPUID_INDEX_1
+#define index_cpu_DE CPUID_INDEX_1
+#define index_cpu_PSE CPUID_INDEX_1
+#define index_cpu_TSC CPUID_INDEX_1
+#define index_cpu_MSR CPUID_INDEX_1
+#define index_cpu_PAE CPUID_INDEX_1
+#define index_cpu_MCE CPUID_INDEX_1
+#define index_cpu_CX8 CPUID_INDEX_1
+#define index_cpu_APIC CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_10 CPUID_INDEX_1
+#define index_cpu_SEP CPUID_INDEX_1
+#define index_cpu_MTRR CPUID_INDEX_1
+#define index_cpu_PGE CPUID_INDEX_1
+#define index_cpu_MCA CPUID_INDEX_1
+#define index_cpu_CMOV CPUID_INDEX_1
+#define index_cpu_PAT CPUID_INDEX_1
+#define index_cpu_PSE_36 CPUID_INDEX_1
+#define index_cpu_PSN CPUID_INDEX_1
+#define index_cpu_CLFSH CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_20 CPUID_INDEX_1
+#define index_cpu_DS CPUID_INDEX_1
+#define index_cpu_ACPI CPUID_INDEX_1
+#define index_cpu_MMX CPUID_INDEX_1
+#define index_cpu_FXSR CPUID_INDEX_1
+#define index_cpu_SSE CPUID_INDEX_1
+#define index_cpu_SSE2 CPUID_INDEX_1
+#define index_cpu_SS CPUID_INDEX_1
+#define index_cpu_HTT CPUID_INDEX_1
+#define index_cpu_TM CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_30 CPUID_INDEX_1
+#define index_cpu_PBE CPUID_INDEX_1
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define index_cpu_FSGSBASE CPUID_INDEX_7
+#define index_cpu_TSC_ADJUST CPUID_INDEX_7
+#define index_cpu_SGX CPUID_INDEX_7
+#define index_cpu_BMI1 CPUID_INDEX_7
+#define index_cpu_HLE CPUID_INDEX_7
+#define index_cpu_AVX2 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_6 CPUID_INDEX_7
+#define index_cpu_SMEP CPUID_INDEX_7
+#define index_cpu_BMI2 CPUID_INDEX_7
+#define index_cpu_ERMS CPUID_INDEX_7
+#define index_cpu_INVPCID CPUID_INDEX_7
+#define index_cpu_RTM CPUID_INDEX_7
+#define index_cpu_RDT_M CPUID_INDEX_7
+#define index_cpu_DEPR_FPU_CS_DS CPUID_INDEX_7
+#define index_cpu_MPX CPUID_INDEX_7
+#define index_cpu_RDT_A CPUID_INDEX_7
+#define index_cpu_AVX512F CPUID_INDEX_7
+#define index_cpu_AVX512DQ CPUID_INDEX_7
+#define index_cpu_RDSEED CPUID_INDEX_7
+#define index_cpu_ADX CPUID_INDEX_7
+#define index_cpu_SMAP CPUID_INDEX_7
+#define index_cpu_AVX512_IFMA CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_22 CPUID_INDEX_7
+#define index_cpu_CLFLUSHOPT CPUID_INDEX_7
+#define index_cpu_CLWB CPUID_INDEX_7
+#define index_cpu_TRACE CPUID_INDEX_7
+#define index_cpu_AVX512PF CPUID_INDEX_7
+#define index_cpu_AVX512ER CPUID_INDEX_7
+#define index_cpu_AVX512CD CPUID_INDEX_7
+#define index_cpu_SHA CPUID_INDEX_7
+#define index_cpu_AVX512BW CPUID_INDEX_7
+#define index_cpu_AVX512VL CPUID_INDEX_7
+
+/* ECX. */
+#define index_cpu_PREFETCHWT1 CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI CPUID_INDEX_7
+#define index_cpu_UMIP CPUID_INDEX_7
+#define index_cpu_PKU CPUID_INDEX_7
+#define index_cpu_OSPKE CPUID_INDEX_7
+#define index_cpu_WAITPKG CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI2 CPUID_INDEX_7
+#define index_cpu_SHSTK CPUID_INDEX_7
+#define index_cpu_GFNI CPUID_INDEX_7
+#define index_cpu_VAES CPUID_INDEX_7
+#define index_cpu_VPCLMULQDQ CPUID_INDEX_7
+#define index_cpu_AVX512_VNNI CPUID_INDEX_7
+#define index_cpu_AVX512_BITALG CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_13 CPUID_INDEX_7
+#define index_cpu_AVX512_VPOPCNTDQ CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_15 CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_16 CPUID_INDEX_7
+#define index_cpu_RDPID CPUID_INDEX_7
+#define index_cpu_KL CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_24 CPUID_INDEX_7
+#define index_cpu_CLDEMOTE CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_26 CPUID_INDEX_7
+#define index_cpu_MOVDIRI CPUID_INDEX_7
+#define index_cpu_MOVDIR64B CPUID_INDEX_7
+#define index_cpu_ENQCMD CPUID_INDEX_7
+#define index_cpu_SGX_LC CPUID_INDEX_7
+#define index_cpu_PKS CPUID_INDEX_7
+
+/* EDX. */
+#define index_cpu_INDEX_7_EDX_0 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_1 CPUID_INDEX_7
+#define index_cpu_AVX512_4VNNIW CPUID_INDEX_7
+#define index_cpu_AVX512_4FMAPS CPUID_INDEX_7
+#define index_cpu_FSRM CPUID_INDEX_7
+#define index_cpu_UINTR CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_6 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_7 CPUID_INDEX_7
+#define index_cpu_AVX512_VP2INTERSECT CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_9 CPUID_INDEX_7
+#define index_cpu_MD_CLEAR CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_11 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_12 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_13 CPUID_INDEX_7
+#define index_cpu_SERIALIZE CPUID_INDEX_7
+#define index_cpu_HYBRID CPUID_INDEX_7
+#define index_cpu_TSXLDTRK CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_17 CPUID_INDEX_7
+#define index_cpu_PCONFIG CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_19 CPUID_INDEX_7
+#define index_cpu_IBT CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_21 CPUID_INDEX_7
+#define index_cpu_AMX_BF16 CPUID_INDEX_7
+#define index_cpu_AVX512_FP16 CPUID_INDEX_7
+#define index_cpu_AMX_TILE CPUID_INDEX_7
+#define index_cpu_AMX_INT8 CPUID_INDEX_7
+#define index_cpu_IBRS_IBPB CPUID_INDEX_7
+#define index_cpu_STIBP CPUID_INDEX_7
+#define index_cpu_L1D_FLUSH CPUID_INDEX_7
+#define index_cpu_ARCH_CAPABILITIES CPUID_INDEX_7
+#define index_cpu_CORE_CAPABILITIES CPUID_INDEX_7
+#define index_cpu_SSBD CPUID_INDEX_7
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define index_cpu_LAHF64_SAHF64 CPUID_INDEX_80000001
+#define index_cpu_SVM CPUID_INDEX_80000001
+#define index_cpu_LZCNT CPUID_INDEX_80000001
+#define index_cpu_SSE4A CPUID_INDEX_80000001
+#define index_cpu_PREFETCHW CPUID_INDEX_80000001
+#define index_cpu_XOP CPUID_INDEX_80000001
+#define index_cpu_LWP CPUID_INDEX_80000001
+#define index_cpu_FMA4 CPUID_INDEX_80000001
+#define index_cpu_TBM CPUID_INDEX_80000001
+
+/* EDX. */
+#define index_cpu_SYSCALL_SYSRET CPUID_INDEX_80000001
+#define index_cpu_NX CPUID_INDEX_80000001
+#define index_cpu_PAGE1GB CPUID_INDEX_80000001
+#define index_cpu_RDTSCP CPUID_INDEX_80000001
+#define index_cpu_LM CPUID_INDEX_80000001
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define index_cpu_XSAVEOPT CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVEC CPUID_INDEX_D_ECX_1
+#define index_cpu_XGETBV_ECX_1 CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVES CPUID_INDEX_D_ECX_1
+#define index_cpu_XFD CPUID_INDEX_D_ECX_1
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define index_cpu_INVARIANT_TSC CPUID_INDEX_80000007
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define index_cpu_WBNOINVD CPUID_INDEX_80000008
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define index_cpu_AVX_VNNI CPUID_INDEX_7_ECX_1
+#define index_cpu_AVX512_BF16 CPUID_INDEX_7_ECX_1
+#define index_cpu_FZLRM CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRS CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRCS CPUID_INDEX_7_ECX_1
+#define index_cpu_HRESET CPUID_INDEX_7_ECX_1
+#define index_cpu_LAM CPUID_INDEX_7_ECX_1
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define index_cpu_AESKLE CPUID_INDEX_19
+#define index_cpu_WIDE_KL CPUID_INDEX_19
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define reg_SSE3 ecx
+#define reg_PCLMULQDQ ecx
+#define reg_DTES64 ecx
+#define reg_MONITOR ecx
+#define reg_DS_CPL ecx
+#define reg_VMX ecx
+#define reg_SMX ecx
+#define reg_EIST ecx
+#define reg_TM2 ecx
+#define reg_SSSE3 ecx
+#define reg_CNXT_ID ecx
+#define reg_SDBG ecx
+#define reg_FMA ecx
+#define reg_CMPXCHG16B ecx
+#define reg_XTPRUPDCTRL ecx
+#define reg_PDCM ecx
+#define reg_INDEX_1_ECX_16 ecx
+#define reg_PCID ecx
+#define reg_DCA ecx
+#define reg_SSE4_1 ecx
+#define reg_SSE4_2 ecx
+#define reg_X2APIC ecx
+#define reg_MOVBE ecx
+#define reg_POPCNT ecx
+#define reg_TSC_DEADLINE ecx
+#define reg_AES ecx
+#define reg_XSAVE ecx
+#define reg_OSXSAVE ecx
+#define reg_AVX ecx
+#define reg_F16C ecx
+#define reg_RDRAND ecx
+#define reg_INDEX_1_ECX_31 ecx
+
+/* EDX. */
+#define reg_FPU edx
+#define reg_VME edx
+#define reg_DE edx
+#define reg_PSE edx
+#define reg_TSC edx
+#define reg_MSR edx
+#define reg_PAE edx
+#define reg_MCE edx
+#define reg_CX8 edx
+#define reg_APIC edx
+#define reg_INDEX_1_EDX_10 edx
+#define reg_SEP edx
+#define reg_MTRR edx
+#define reg_PGE edx
+#define reg_MCA edx
+#define reg_CMOV edx
+#define reg_PAT edx
+#define reg_PSE_36 edx
+#define reg_PSN edx
+#define reg_CLFSH edx
+#define reg_INDEX_1_EDX_20 edx
+#define reg_DS edx
+#define reg_ACPI edx
+#define reg_MMX edx
+#define reg_FXSR edx
+#define reg_SSE edx
+#define reg_SSE2 edx
+#define reg_SS edx
+#define reg_HTT edx
+#define reg_TM edx
+#define reg_INDEX_1_EDX_30 edx
+#define reg_PBE edx
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define reg_FSGSBASE ebx
+#define reg_TSC_ADJUST ebx
+#define reg_SGX ebx
+#define reg_BMI1 ebx
+#define reg_HLE ebx
+#define reg_BMI2 ebx
+#define reg_AVX2 ebx
+#define reg_INDEX_7_EBX_6 ebx
+#define reg_SMEP ebx
+#define reg_ERMS ebx
+#define reg_INVPCID ebx
+#define reg_RTM ebx
+#define reg_RDT_M ebx
+#define reg_DEPR_FPU_CS_DS ebx
+#define reg_MPX ebx
+#define reg_RDT_A ebx
+#define reg_AVX512F ebx
+#define reg_AVX512DQ ebx
+#define reg_RDSEED ebx
+#define reg_ADX ebx
+#define reg_SMAP ebx
+#define reg_AVX512_IFMA ebx
+#define reg_INDEX_7_EBX_22 ebx
+#define reg_CLFLUSHOPT ebx
+#define reg_CLWB ebx
+#define reg_TRACE ebx
+#define reg_AVX512PF ebx
+#define reg_AVX512ER ebx
+#define reg_AVX512CD ebx
+#define reg_SHA ebx
+#define reg_AVX512BW ebx
+#define reg_AVX512VL ebx
+
+/* ECX. */
+#define reg_PREFETCHWT1 ecx
+#define reg_AVX512_VBMI ecx
+#define reg_UMIP ecx
+#define reg_PKU ecx
+#define reg_OSPKE ecx
+#define reg_WAITPKG ecx
+#define reg_AVX512_VBMI2 ecx
+#define reg_SHSTK ecx
+#define reg_GFNI ecx
+#define reg_VAES ecx
+#define reg_VPCLMULQDQ ecx
+#define reg_AVX512_VNNI ecx
+#define reg_AVX512_BITALG ecx
+#define reg_INDEX_7_ECX_13 ecx
+#define reg_AVX512_VPOPCNTDQ ecx
+#define reg_INDEX_7_ECX_15 ecx
+#define reg_INDEX_7_ECX_16 ecx
+#define reg_RDPID ecx
+#define reg_KL ecx
+#define reg_INDEX_7_ECX_24 ecx
+#define reg_CLDEMOTE ecx
+#define reg_INDEX_7_ECX_26 ecx
+#define reg_MOVDIRI ecx
+#define reg_MOVDIR64B ecx
+#define reg_ENQCMD ecx
+#define reg_SGX_LC ecx
+#define reg_PKS ecx
+
+/* EDX. */
+#define reg_INDEX_7_EDX_0 edx
+#define reg_INDEX_7_EDX_1 edx
+#define reg_AVX512_4VNNIW edx
+#define reg_AVX512_4FMAPS edx
+#define reg_FSRM edx
+#define reg_UINTR edx
+#define reg_INDEX_7_EDX_6 edx
+#define reg_INDEX_7_EDX_7 edx
+#define reg_AVX512_VP2INTERSECT edx
+#define reg_INDEX_7_EDX_9 edx
+#define reg_MD_CLEAR edx
+#define reg_INDEX_7_EDX_11 edx
+#define reg_INDEX_7_EDX_12 edx
+#define reg_INDEX_7_EDX_13 edx
+#define reg_SERIALIZE edx
+#define reg_HYBRID edx
+#define reg_TSXLDTRK edx
+#define reg_INDEX_7_EDX_17 edx
+#define reg_PCONFIG edx
+#define reg_INDEX_7_EDX_19 edx
+#define reg_IBT edx
+#define reg_INDEX_7_EDX_21 edx
+#define reg_AMX_BF16 edx
+#define reg_AVX512_FP16 edx
+#define reg_AMX_TILE edx
+#define reg_AMX_INT8 edx
+#define reg_IBRS_IBPB edx
+#define reg_STIBP edx
+#define reg_L1D_FLUSH edx
+#define reg_ARCH_CAPABILITIES edx
+#define reg_CORE_CAPABILITIES edx
+#define reg_SSBD edx
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define reg_LAHF64_SAHF64 ecx
+#define reg_SVM ecx
+#define reg_LZCNT ecx
+#define reg_SSE4A ecx
+#define reg_PREFETCHW ecx
+#define reg_XOP ecx
+#define reg_LWP ecx
+#define reg_FMA4 ecx
+#define reg_TBM ecx
+
+/* EDX. */
+#define reg_SYSCALL_SYSRET edx
+#define reg_NX edx
+#define reg_PAGE1GB edx
+#define reg_RDTSCP edx
+#define reg_LM edx
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define reg_XSAVEOPT eax
+#define reg_XSAVEC eax
+#define reg_XGETBV_ECX_1 eax
+#define reg_XSAVES eax
+#define reg_XFD eax
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define reg_INVARIANT_TSC edx
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define reg_WBNOINVD ebx
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define reg_AVX_VNNI eax
+#define reg_AVX512_BF16 eax
+#define reg_FZLRM eax
+#define reg_FSRS eax
+#define reg_FSRCS eax
+#define reg_HRESET eax
+#define reg_LAM eax
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define reg_AESKLE ebx
+#define reg_WIDE_KL ebx
+
/* PREFERRED_FEATURE_INDEX_1. */
#define bit_arch_I586 (1u << 0)
#define bit_arch_I686 (1u << 1)
@@ -118,11 +807,51 @@ enum
#define bit_XTILECFG_state (1u << 17)
#define bit_XTILEDATA_state (1u << 18)
+enum cpu_features_kind
+{
+ arch_kind_unknown = 0,
+ arch_kind_intel,
+ arch_kind_amd,
+ arch_kind_zhaoxin,
+ arch_kind_other
+};
+
+struct cpu_features_basic
+{
+ enum cpu_features_kind kind;
+ unsigned int family;
+ unsigned int model;
+ unsigned int stepping;
+};
+
+struct cpuid_registers
+{
+ unsigned int eax;
+ unsigned int ebx;
+ unsigned int ecx;
+ unsigned int edx;
+};
+
+struct cpuid_feature_internal
+{
+ union
+ {
+ unsigned int cpuid_array[4];
+ struct cpuid_registers cpuid;
+ };
+ union
+ {
+ unsigned int usable_array[4];
+ struct cpuid_registers usable;
+ };
+};
+
struct cpu_features
{
- struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
+ int max_cpuid;
+ struct cpuid_feature_internal features[CPUID_INDEX_MAX];
unsigned int preferred[PREFERRED_FEATURE_INDEX_MAX];
+ struct cpu_features_basic basic;
/* X86 micro-architecture ISA levels. */
unsigned int isa_1;
/* The state size for XSAVEC or XSAVE. The type must be unsigned long
@@ -173,10 +902,12 @@ struct cpu_features
unsigned long int level4_cache_size;
};
+#define __get_cpu_features() __x86_get_cpu_features (0)
+
#if defined (_LIBC) && !IS_IN (nonlib)
/* Unused for x86. */
# define INIT_ARCH()
-# define __x86_get_cpu_features(max) (&GLRO(dl_x86_cpu_features))
+# define __x86_get_cpu_features(indx) (&GLRO(dl_x86_cpu_features))
extern void _dl_x86_init_cpu_features (void) attribute_hidden;
#endif
diff --git a/sysdeps/x86/sys/platform/x86.h b/sysdeps/x86/sys/platform/x86.h
index 7c964ff907..d07e5e2af9 100644
--- a/sysdeps/x86/sys/platform/x86.h
+++ b/sysdeps/x86/sys/platform/x86.h
@@ -19,760 +19,62 @@
#ifndef _SYS_PLATFORM_X86_H
#define _SYS_PLATFORM_X86_H
-enum
-{
- COMMON_CPUID_INDEX_1 = 0,
- COMMON_CPUID_INDEX_7,
- COMMON_CPUID_INDEX_80000001,
- COMMON_CPUID_INDEX_D_ECX_1,
- COMMON_CPUID_INDEX_80000007,
- COMMON_CPUID_INDEX_80000008,
- COMMON_CPUID_INDEX_7_ECX_1,
- COMMON_CPUID_INDEX_19,
- /* Keep the following line at the end. */
- COMMON_CPUID_INDEX_MAX
-};
+#include <features.h>
+#include <stdbool.h>
+#include <bits/platform/x86.h>
-struct cpuid_registers
-{
- unsigned int eax;
- unsigned int ebx;
- unsigned int ecx;
- unsigned int edx;
-};
+__BEGIN_DECLS
-struct cpuid_features
+static __inline__ _Bool
+x86_cpu_has_feature (const struct cpu_features *__ptr,
+ unsigned int __index)
{
- struct cpuid_registers cpuid;
- struct cpuid_registers usable;
-};
-
-enum cpu_features_kind
+ unsigned int __leaf
+ = __index / (8 * sizeof (unsigned int) * 4);
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
+
+ return __ptr->features[__leaf].cpuid_array[__reg] & (1 << __bit);
+}
+
+static __inline__ _Bool
+x86_cpu_is_usable (const struct cpu_features *__ptr,
+ unsigned int __index)
{
- arch_kind_unknown = 0,
- arch_kind_intel,
- arch_kind_amd,
- arch_kind_zhaoxin,
- arch_kind_other
-};
+ unsigned int __leaf
+ = __index / (8 * sizeof (unsigned int) * 4);
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
-struct cpu_features_basic
-{
- enum cpu_features_kind kind;
- int max_cpuid;
- unsigned int family;
- unsigned int model;
- unsigned int stepping;
-};
-
-struct cpu_features
-{
- struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
-};
+ return __ptr->features[__leaf].usable_array[__reg] & (1 << __bit);
+}
/* Get a pointer to the CPU features structure. */
extern const struct cpu_features *__x86_get_cpu_features (unsigned int)
__attribute__ ((const));
-#define CPU_FEATURE_CHECK_P(ptr, name, check) \
- ((ptr->features[index_cpu_##name].check.reg_##name \
- & bit_cpu_##name) != 0)
#define CPU_FEATURE_CPU_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, cpuid)
+ x86_cpu_has_feature ((ptr), x86_cpu_##name)
#define CPU_FEATURE_USABLE_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, usable)
+ x86_cpu_is_usable ((ptr), x86_cpu_##name)
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
#define HAS_CPU_FEATURE(name) \
(__extension__ \
({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_CPU_P (__ptr, name); }))
+ __x86_get_cpu_features (x86_cpu_##name); \
+ __ptr && x86_cpu_has_feature (__ptr, x86_cpu_##name); }))
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
#define CPU_FEATURE_USABLE(name) \
(__extension__ \
({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_USABLE_P (__ptr, name); }))
-
-/* CPU features. */
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define bit_cpu_SSE3 (1u << 0)
-#define bit_cpu_PCLMULQDQ (1u << 1)
-#define bit_cpu_DTES64 (1u << 2)
-#define bit_cpu_MONITOR (1u << 3)
-#define bit_cpu_DS_CPL (1u << 4)
-#define bit_cpu_VMX (1u << 5)
-#define bit_cpu_SMX (1u << 6)
-#define bit_cpu_EIST (1u << 7)
-#define bit_cpu_TM2 (1u << 8)
-#define bit_cpu_SSSE3 (1u << 9)
-#define bit_cpu_CNXT_ID (1u << 10)
-#define bit_cpu_SDBG (1u << 11)
-#define bit_cpu_FMA (1u << 12)
-#define bit_cpu_CMPXCHG16B (1u << 13)
-#define bit_cpu_XTPRUPDCTRL (1u << 14)
-#define bit_cpu_PDCM (1u << 15)
-#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
-#define bit_cpu_PCID (1u << 17)
-#define bit_cpu_DCA (1u << 18)
-#define bit_cpu_SSE4_1 (1u << 19)
-#define bit_cpu_SSE4_2 (1u << 20)
-#define bit_cpu_X2APIC (1u << 21)
-#define bit_cpu_MOVBE (1u << 22)
-#define bit_cpu_POPCNT (1u << 23)
-#define bit_cpu_TSC_DEADLINE (1u << 24)
-#define bit_cpu_AES (1u << 25)
-#define bit_cpu_XSAVE (1u << 26)
-#define bit_cpu_OSXSAVE (1u << 27)
-#define bit_cpu_AVX (1u << 28)
-#define bit_cpu_F16C (1u << 29)
-#define bit_cpu_RDRAND (1u << 30)
-#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
-
-/* EDX. */
-#define bit_cpu_FPU (1u << 0)
-#define bit_cpu_VME (1u << 1)
-#define bit_cpu_DE (1u << 2)
-#define bit_cpu_PSE (1u << 3)
-#define bit_cpu_TSC (1u << 4)
-#define bit_cpu_MSR (1u << 5)
-#define bit_cpu_PAE (1u << 6)
-#define bit_cpu_MCE (1u << 7)
-#define bit_cpu_CX8 (1u << 8)
-#define bit_cpu_APIC (1u << 9)
-#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
-#define bit_cpu_SEP (1u << 11)
-#define bit_cpu_MTRR (1u << 12)
-#define bit_cpu_PGE (1u << 13)
-#define bit_cpu_MCA (1u << 14)
-#define bit_cpu_CMOV (1u << 15)
-#define bit_cpu_PAT (1u << 16)
-#define bit_cpu_PSE_36 (1u << 17)
-#define bit_cpu_PSN (1u << 18)
-#define bit_cpu_CLFSH (1u << 19)
-#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
-#define bit_cpu_DS (1u << 21)
-#define bit_cpu_ACPI (1u << 22)
-#define bit_cpu_MMX (1u << 23)
-#define bit_cpu_FXSR (1u << 24)
-#define bit_cpu_SSE (1u << 25)
-#define bit_cpu_SSE2 (1u << 26)
-#define bit_cpu_SS (1u << 27)
-#define bit_cpu_HTT (1u << 28)
-#define bit_cpu_TM (1u << 29)
-#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
-#define bit_cpu_PBE (1u << 31)
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define bit_cpu_FSGSBASE (1u << 0)
-#define bit_cpu_TSC_ADJUST (1u << 1)
-#define bit_cpu_SGX (1u << 2)
-#define bit_cpu_BMI1 (1u << 3)
-#define bit_cpu_HLE (1u << 4)
-#define bit_cpu_AVX2 (1u << 5)
-#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
-#define bit_cpu_SMEP (1u << 7)
-#define bit_cpu_BMI2 (1u << 8)
-#define bit_cpu_ERMS (1u << 9)
-#define bit_cpu_INVPCID (1u << 10)
-#define bit_cpu_RTM (1u << 11)
-#define bit_cpu_RDT_M (1u << 12)
-#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
-#define bit_cpu_MPX (1u << 14)
-#define bit_cpu_RDT_A (1u << 15)
-#define bit_cpu_AVX512F (1u << 16)
-#define bit_cpu_AVX512DQ (1u << 17)
-#define bit_cpu_RDSEED (1u << 18)
-#define bit_cpu_ADX (1u << 19)
-#define bit_cpu_SMAP (1u << 20)
-#define bit_cpu_AVX512_IFMA (1u << 21)
-#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
-#define bit_cpu_CLFLUSHOPT (1u << 23)
-#define bit_cpu_CLWB (1u << 24)
-#define bit_cpu_TRACE (1u << 25)
-#define bit_cpu_AVX512PF (1u << 26)
-#define bit_cpu_AVX512ER (1u << 27)
-#define bit_cpu_AVX512CD (1u << 28)
-#define bit_cpu_SHA (1u << 29)
-#define bit_cpu_AVX512BW (1u << 30)
-#define bit_cpu_AVX512VL (1u << 31)
-
-/* ECX. */
-#define bit_cpu_PREFETCHWT1 (1u << 0)
-#define bit_cpu_AVX512_VBMI (1u << 1)
-#define bit_cpu_UMIP (1u << 2)
-#define bit_cpu_PKU (1u << 3)
-#define bit_cpu_OSPKE (1u << 4)
-#define bit_cpu_WAITPKG (1u << 5)
-#define bit_cpu_AVX512_VBMI2 (1u << 6)
-#define bit_cpu_SHSTK (1u << 7)
-#define bit_cpu_GFNI (1u << 8)
-#define bit_cpu_VAES (1u << 9)
-#define bit_cpu_VPCLMULQDQ (1u << 10)
-#define bit_cpu_AVX512_VNNI (1u << 11)
-#define bit_cpu_AVX512_BITALG (1u << 12)
-#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
-#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
-#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
-#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
-/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
- instructions in 64-bit mode. */
-#define bit_cpu_RDPID (1u << 22)
-#define bit_cpu_KL (1u << 23)
-#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
-#define bit_cpu_CLDEMOTE (1u << 25)
-#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
-#define bit_cpu_MOVDIRI (1u << 27)
-#define bit_cpu_MOVDIR64B (1u << 28)
-#define bit_cpu_ENQCMD (1u << 29)
-#define bit_cpu_SGX_LC (1u << 30)
-#define bit_cpu_PKS (1u << 31)
-
-/* EDX. */
-#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
-#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
-#define bit_cpu_AVX512_4VNNIW (1u << 2)
-#define bit_cpu_AVX512_4FMAPS (1u << 3)
-#define bit_cpu_FSRM (1u << 4)
-#define bit_cpu_UINTR (1u << 5)
-#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
-#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
-#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
-#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
-#define bit_cpu_MD_CLEAR (1u << 10)
-#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
-#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
-#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
-#define bit_cpu_SERIALIZE (1u << 14)
-#define bit_cpu_HYBRID (1u << 15)
-#define bit_cpu_TSXLDTRK (1u << 16)
-#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
-#define bit_cpu_PCONFIG (1u << 18)
-#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
-#define bit_cpu_IBT (1u << 20)
-#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
-#define bit_cpu_AMX_BF16 (1u << 22)
-#define bit_cpu_AVX512_FP16 (1u << 23)
-#define bit_cpu_AMX_TILE (1u << 24)
-#define bit_cpu_AMX_INT8 (1u << 25)
-#define bit_cpu_IBRS_IBPB (1u << 26)
-#define bit_cpu_STIBP (1u << 27)
-#define bit_cpu_L1D_FLUSH (1u << 28)
-#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
-#define bit_cpu_CORE_CAPABILITIES (1u << 30)
-#define bit_cpu_SSBD (1u << 31)
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define bit_cpu_LAHF64_SAHF64 (1u << 0)
-#define bit_cpu_SVM (1u << 2)
-#define bit_cpu_LZCNT (1u << 5)
-#define bit_cpu_SSE4A (1u << 6)
-#define bit_cpu_PREFETCHW (1u << 8)
-#define bit_cpu_XOP (1u << 11)
-#define bit_cpu_LWP (1u << 15)
-#define bit_cpu_FMA4 (1u << 16)
-#define bit_cpu_TBM (1u << 21)
-
-/* EDX. */
-#define bit_cpu_SYSCALL_SYSRET (1u << 11)
-#define bit_cpu_NX (1u << 20)
-#define bit_cpu_PAGE1GB (1u << 26)
-#define bit_cpu_RDTSCP (1u << 27)
-#define bit_cpu_LM (1u << 29)
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_XSAVEOPT (1u << 0)
-#define bit_cpu_XSAVEC (1u << 1)
-#define bit_cpu_XGETBV_ECX_1 (1u << 2)
-#define bit_cpu_XSAVES (1u << 3)
-#define bit_cpu_XFD (1u << 4)
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define bit_cpu_INVARIANT_TSC (1u << 8)
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define bit_cpu_WBNOINVD (1u << 9)
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_AVX_VNNI (1u << 4)
-#define bit_cpu_AVX512_BF16 (1u << 5)
-#define bit_cpu_FZLRM (1u << 10)
-#define bit_cpu_FSRS (1u << 11)
-#define bit_cpu_FSRCS (1u << 12)
-#define bit_cpu_HRESET (1u << 22)
-#define bit_cpu_LAM (1u << 26)
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define bit_cpu_AESKLE (1u << 0)
-#define bit_cpu_WIDE_KL (1u << 2)
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define index_cpu_SSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_PCLMULQDQ COMMON_CPUID_INDEX_1
-#define index_cpu_DTES64 COMMON_CPUID_INDEX_1
-#define index_cpu_MONITOR COMMON_CPUID_INDEX_1
-#define index_cpu_DS_CPL COMMON_CPUID_INDEX_1
-#define index_cpu_VMX COMMON_CPUID_INDEX_1
-#define index_cpu_SMX COMMON_CPUID_INDEX_1
-#define index_cpu_EIST COMMON_CPUID_INDEX_1
-#define index_cpu_TM2 COMMON_CPUID_INDEX_1
-#define index_cpu_SSSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_CNXT_ID COMMON_CPUID_INDEX_1
-#define index_cpu_SDBG COMMON_CPUID_INDEX_1
-#define index_cpu_FMA COMMON_CPUID_INDEX_1
-#define index_cpu_CMPXCHG16B COMMON_CPUID_INDEX_1
-#define index_cpu_XTPRUPDCTRL COMMON_CPUID_INDEX_1
-#define index_cpu_PDCM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_16 COMMON_CPUID_INDEX_1
-#define index_cpu_PCID COMMON_CPUID_INDEX_1
-#define index_cpu_DCA COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_1 COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_2 COMMON_CPUID_INDEX_1
-#define index_cpu_X2APIC COMMON_CPUID_INDEX_1
-#define index_cpu_MOVBE COMMON_CPUID_INDEX_1
-#define index_cpu_POPCNT COMMON_CPUID_INDEX_1
-#define index_cpu_TSC_DEADLINE COMMON_CPUID_INDEX_1
-#define index_cpu_AES COMMON_CPUID_INDEX_1
-#define index_cpu_XSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_OSXSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_AVX COMMON_CPUID_INDEX_1
-#define index_cpu_F16C COMMON_CPUID_INDEX_1
-#define index_cpu_RDRAND COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_31 COMMON_CPUID_INDEX_1
-
-/* ECX. */
-#define index_cpu_FPU COMMON_CPUID_INDEX_1
-#define index_cpu_VME COMMON_CPUID_INDEX_1
-#define index_cpu_DE COMMON_CPUID_INDEX_1
-#define index_cpu_PSE COMMON_CPUID_INDEX_1
-#define index_cpu_TSC COMMON_CPUID_INDEX_1
-#define index_cpu_MSR COMMON_CPUID_INDEX_1
-#define index_cpu_PAE COMMON_CPUID_INDEX_1
-#define index_cpu_MCE COMMON_CPUID_INDEX_1
-#define index_cpu_CX8 COMMON_CPUID_INDEX_1
-#define index_cpu_APIC COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_10 COMMON_CPUID_INDEX_1
-#define index_cpu_SEP COMMON_CPUID_INDEX_1
-#define index_cpu_MTRR COMMON_CPUID_INDEX_1
-#define index_cpu_PGE COMMON_CPUID_INDEX_1
-#define index_cpu_MCA COMMON_CPUID_INDEX_1
-#define index_cpu_CMOV COMMON_CPUID_INDEX_1
-#define index_cpu_PAT COMMON_CPUID_INDEX_1
-#define index_cpu_PSE_36 COMMON_CPUID_INDEX_1
-#define index_cpu_PSN COMMON_CPUID_INDEX_1
-#define index_cpu_CLFSH COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_20 COMMON_CPUID_INDEX_1
-#define index_cpu_DS COMMON_CPUID_INDEX_1
-#define index_cpu_ACPI COMMON_CPUID_INDEX_1
-#define index_cpu_MMX COMMON_CPUID_INDEX_1
-#define index_cpu_FXSR COMMON_CPUID_INDEX_1
-#define index_cpu_SSE COMMON_CPUID_INDEX_1
-#define index_cpu_SSE2 COMMON_CPUID_INDEX_1
-#define index_cpu_SS COMMON_CPUID_INDEX_1
-#define index_cpu_HTT COMMON_CPUID_INDEX_1
-#define index_cpu_TM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_30 COMMON_CPUID_INDEX_1
-#define index_cpu_PBE COMMON_CPUID_INDEX_1
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define index_cpu_FSGSBASE COMMON_CPUID_INDEX_7
-#define index_cpu_TSC_ADJUST COMMON_CPUID_INDEX_7
-#define index_cpu_SGX COMMON_CPUID_INDEX_7
-#define index_cpu_BMI1 COMMON_CPUID_INDEX_7
-#define index_cpu_HLE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX2 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_SMEP COMMON_CPUID_INDEX_7
-#define index_cpu_BMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_ERMS COMMON_CPUID_INDEX_7
-#define index_cpu_INVPCID COMMON_CPUID_INDEX_7
-#define index_cpu_RTM COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_M COMMON_CPUID_INDEX_7
-#define index_cpu_DEPR_FPU_CS_DS COMMON_CPUID_INDEX_7
-#define index_cpu_MPX COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_A COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512F COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512DQ COMMON_CPUID_INDEX_7
-#define index_cpu_RDSEED COMMON_CPUID_INDEX_7
-#define index_cpu_ADX COMMON_CPUID_INDEX_7
-#define index_cpu_SMAP COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_IFMA COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_22 COMMON_CPUID_INDEX_7
-#define index_cpu_CLFLUSHOPT COMMON_CPUID_INDEX_7
-#define index_cpu_CLWB COMMON_CPUID_INDEX_7
-#define index_cpu_TRACE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512PF COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512ER COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512CD COMMON_CPUID_INDEX_7
-#define index_cpu_SHA COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512BW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512VL COMMON_CPUID_INDEX_7
-
-/* ECX. */
-#define index_cpu_PREFETCHWT1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI COMMON_CPUID_INDEX_7
-#define index_cpu_UMIP COMMON_CPUID_INDEX_7
-#define index_cpu_PKU COMMON_CPUID_INDEX_7
-#define index_cpu_OSPKE COMMON_CPUID_INDEX_7
-#define index_cpu_WAITPKG COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_SHSTK COMMON_CPUID_INDEX_7
-#define index_cpu_GFNI COMMON_CPUID_INDEX_7
-#define index_cpu_VAES COMMON_CPUID_INDEX_7
-#define index_cpu_VPCLMULQDQ COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VNNI COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_BITALG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VPOPCNTDQ COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_15 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_16 COMMON_CPUID_INDEX_7
-#define index_cpu_RDPID COMMON_CPUID_INDEX_7
-#define index_cpu_KL COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_24 COMMON_CPUID_INDEX_7
-#define index_cpu_CLDEMOTE COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_26 COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIRI COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIR64B COMMON_CPUID_INDEX_7
-#define index_cpu_ENQCMD COMMON_CPUID_INDEX_7
-#define index_cpu_SGX_LC COMMON_CPUID_INDEX_7
-#define index_cpu_PKS COMMON_CPUID_INDEX_7
-
-/* EDX. */
-#define index_cpu_INDEX_7_EDX_0 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4VNNIW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4FMAPS COMMON_CPUID_INDEX_7
-#define index_cpu_FSRM COMMON_CPUID_INDEX_7
-#define index_cpu_UINTR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_7 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VP2INTERSECT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_9 COMMON_CPUID_INDEX_7
-#define index_cpu_MD_CLEAR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_11 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_12 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_SERIALIZE COMMON_CPUID_INDEX_7
-#define index_cpu_HYBRID COMMON_CPUID_INDEX_7
-#define index_cpu_TSXLDTRK COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_17 COMMON_CPUID_INDEX_7
-#define index_cpu_PCONFIG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_19 COMMON_CPUID_INDEX_7
-#define index_cpu_IBT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_21 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_BF16 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_FP16 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_TILE COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_INT8 COMMON_CPUID_INDEX_7
-#define index_cpu_IBRS_IBPB COMMON_CPUID_INDEX_7
-#define index_cpu_STIBP COMMON_CPUID_INDEX_7
-#define index_cpu_L1D_FLUSH COMMON_CPUID_INDEX_7
-#define index_cpu_ARCH_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_CORE_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_SSBD COMMON_CPUID_INDEX_7
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define index_cpu_LAHF64_SAHF64 COMMON_CPUID_INDEX_80000001
-#define index_cpu_SVM COMMON_CPUID_INDEX_80000001
-#define index_cpu_LZCNT COMMON_CPUID_INDEX_80000001
-#define index_cpu_SSE4A COMMON_CPUID_INDEX_80000001
-#define index_cpu_PREFETCHW COMMON_CPUID_INDEX_80000001
-#define index_cpu_XOP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LWP COMMON_CPUID_INDEX_80000001
-#define index_cpu_FMA4 COMMON_CPUID_INDEX_80000001
-#define index_cpu_TBM COMMON_CPUID_INDEX_80000001
-
-/* EDX. */
-#define index_cpu_SYSCALL_SYSRET COMMON_CPUID_INDEX_80000001
-#define index_cpu_NX COMMON_CPUID_INDEX_80000001
-#define index_cpu_PAGE1GB COMMON_CPUID_INDEX_80000001
-#define index_cpu_RDTSCP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LM COMMON_CPUID_INDEX_80000001
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define index_cpu_XSAVEOPT COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVEC COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XGETBV_ECX_1 COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVES COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XFD COMMON_CPUID_INDEX_D_ECX_1
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define index_cpu_INVARIANT_TSC COMMON_CPUID_INDEX_80000007
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define index_cpu_WBNOINVD COMMON_CPUID_INDEX_80000008
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define index_cpu_AVX_VNNI COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_AVX512_BF16 COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FZLRM COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRCS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_HRESET COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_LAM COMMON_CPUID_INDEX_7_ECX_1
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define index_cpu_AESKLE COMMON_CPUID_INDEX_19
-#define index_cpu_WIDE_KL COMMON_CPUID_INDEX_19
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define reg_SSE3 ecx
-#define reg_PCLMULQDQ ecx
-#define reg_DTES64 ecx
-#define reg_MONITOR ecx
-#define reg_DS_CPL ecx
-#define reg_VMX ecx
-#define reg_SMX ecx
-#define reg_EIST ecx
-#define reg_TM2 ecx
-#define reg_SSSE3 ecx
-#define reg_CNXT_ID ecx
-#define reg_SDBG ecx
-#define reg_FMA ecx
-#define reg_CMPXCHG16B ecx
-#define reg_XTPRUPDCTRL ecx
-#define reg_PDCM ecx
-#define reg_INDEX_1_ECX_16 ecx
-#define reg_PCID ecx
-#define reg_DCA ecx
-#define reg_SSE4_1 ecx
-#define reg_SSE4_2 ecx
-#define reg_X2APIC ecx
-#define reg_MOVBE ecx
-#define reg_POPCNT ecx
-#define reg_TSC_DEADLINE ecx
-#define reg_AES ecx
-#define reg_XSAVE ecx
-#define reg_OSXSAVE ecx
-#define reg_AVX ecx
-#define reg_F16C ecx
-#define reg_RDRAND ecx
-#define reg_INDEX_1_ECX_31 ecx
-
-/* EDX. */
-#define reg_FPU edx
-#define reg_VME edx
-#define reg_DE edx
-#define reg_PSE edx
-#define reg_TSC edx
-#define reg_MSR edx
-#define reg_PAE edx
-#define reg_MCE edx
-#define reg_CX8 edx
-#define reg_APIC edx
-#define reg_INDEX_1_EDX_10 edx
-#define reg_SEP edx
-#define reg_MTRR edx
-#define reg_PGE edx
-#define reg_MCA edx
-#define reg_CMOV edx
-#define reg_PAT edx
-#define reg_PSE_36 edx
-#define reg_PSN edx
-#define reg_CLFSH edx
-#define reg_INDEX_1_EDX_20 edx
-#define reg_DS edx
-#define reg_ACPI edx
-#define reg_MMX edx
-#define reg_FXSR edx
-#define reg_SSE edx
-#define reg_SSE2 edx
-#define reg_SS edx
-#define reg_HTT edx
-#define reg_TM edx
-#define reg_INDEX_1_EDX_30 edx
-#define reg_PBE edx
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define reg_FSGSBASE ebx
-#define reg_TSC_ADJUST ebx
-#define reg_SGX ebx
-#define reg_BMI1 ebx
-#define reg_HLE ebx
-#define reg_BMI2 ebx
-#define reg_AVX2 ebx
-#define reg_INDEX_7_EBX_6 ebx
-#define reg_SMEP ebx
-#define reg_ERMS ebx
-#define reg_INVPCID ebx
-#define reg_RTM ebx
-#define reg_RDT_M ebx
-#define reg_DEPR_FPU_CS_DS ebx
-#define reg_MPX ebx
-#define reg_RDT_A ebx
-#define reg_AVX512F ebx
-#define reg_AVX512DQ ebx
-#define reg_RDSEED ebx
-#define reg_ADX ebx
-#define reg_SMAP ebx
-#define reg_AVX512_IFMA ebx
-#define reg_INDEX_7_EBX_22 ebx
-#define reg_CLFLUSHOPT ebx
-#define reg_CLWB ebx
-#define reg_TRACE ebx
-#define reg_AVX512PF ebx
-#define reg_AVX512ER ebx
-#define reg_AVX512CD ebx
-#define reg_SHA ebx
-#define reg_AVX512BW ebx
-#define reg_AVX512VL ebx
-
-/* ECX. */
-#define reg_PREFETCHWT1 ecx
-#define reg_AVX512_VBMI ecx
-#define reg_UMIP ecx
-#define reg_PKU ecx
-#define reg_OSPKE ecx
-#define reg_WAITPKG ecx
-#define reg_AVX512_VBMI2 ecx
-#define reg_SHSTK ecx
-#define reg_GFNI ecx
-#define reg_VAES ecx
-#define reg_VPCLMULQDQ ecx
-#define reg_AVX512_VNNI ecx
-#define reg_AVX512_BITALG ecx
-#define reg_INDEX_7_ECX_13 ecx
-#define reg_AVX512_VPOPCNTDQ ecx
-#define reg_INDEX_7_ECX_15 ecx
-#define reg_INDEX_7_ECX_16 ecx
-#define reg_RDPID ecx
-#define reg_KL ecx
-#define reg_INDEX_7_ECX_24 ecx
-#define reg_CLDEMOTE ecx
-#define reg_INDEX_7_ECX_26 ecx
-#define reg_MOVDIRI ecx
-#define reg_MOVDIR64B ecx
-#define reg_ENQCMD ecx
-#define reg_SGX_LC ecx
-#define reg_PKS ecx
-
-/* EDX. */
-#define reg_INDEX_7_EDX_0 edx
-#define reg_INDEX_7_EDX_1 edx
-#define reg_AVX512_4VNNIW edx
-#define reg_AVX512_4FMAPS edx
-#define reg_FSRM edx
-#define reg_UINTR edx
-#define reg_INDEX_7_EDX_6 edx
-#define reg_INDEX_7_EDX_7 edx
-#define reg_AVX512_VP2INTERSECT edx
-#define reg_INDEX_7_EDX_9 edx
-#define reg_MD_CLEAR edx
-#define reg_INDEX_7_EDX_11 edx
-#define reg_INDEX_7_EDX_12 edx
-#define reg_INDEX_7_EDX_13 edx
-#define reg_SERIALIZE edx
-#define reg_HYBRID edx
-#define reg_TSXLDTRK edx
-#define reg_INDEX_7_EDX_17 edx
-#define reg_PCONFIG edx
-#define reg_INDEX_7_EDX_19 edx
-#define reg_IBT edx
-#define reg_INDEX_7_EDX_21 edx
-#define reg_AMX_BF16 edx
-#define reg_AVX512_FP16 edx
-#define reg_AMX_TILE edx
-#define reg_AMX_INT8 edx
-#define reg_IBRS_IBPB edx
-#define reg_STIBP edx
-#define reg_L1D_FLUSH edx
-#define reg_ARCH_CAPABILITIES edx
-#define reg_CORE_CAPABILITIES edx
-#define reg_SSBD edx
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define reg_LAHF64_SAHF64 ecx
-#define reg_SVM ecx
-#define reg_LZCNT ecx
-#define reg_SSE4A ecx
-#define reg_PREFETCHW ecx
-#define reg_XOP ecx
-#define reg_LWP ecx
-#define reg_FMA4 ecx
-#define reg_TBM ecx
-
-/* EDX. */
-#define reg_SYSCALL_SYSRET edx
-#define reg_NX edx
-#define reg_PAGE1GB edx
-#define reg_RDTSCP edx
-#define reg_LM edx
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define reg_XSAVEOPT eax
-#define reg_XSAVEC eax
-#define reg_XGETBV_ECX_1 eax
-#define reg_XSAVES eax
-#define reg_XFD eax
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define reg_INVARIANT_TSC edx
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define reg_WBNOINVD ebx
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define reg_AVX_VNNI eax
-#define reg_AVX512_BF16 eax
-#define reg_FZLRM eax
-#define reg_FSRS eax
-#define reg_FSRCS eax
-#define reg_HRESET eax
-#define reg_LAM eax
-
-/* COMMON_CPUID_INDEX_19. */
+ __x86_get_cpu_features (x86_cpu_##name); \
+ __ptr && x86_cpu_is_usable (__ptr, x86_cpu_##name); }))
-/* EBX. */
-#define reg_AESKLE ebx
-#define reg_WIDE_KL ebx
+__END_DECLS
#endif /* _SYS_PLATFORM_X86_H */
diff --git a/sysdeps/x86/tst-get-cpu-features.c b/sysdeps/x86/tst-get-cpu-features.c
index 39f287d2a0..294340392d 100644
--- a/sysdeps/x86/tst-get-cpu-features.c
+++ b/sysdeps/x86/tst-get-cpu-features.c
@@ -33,36 +33,9 @@
printf (" " #name "\n"); \
}
-static const char * const cpu_kinds[] =
-{
- "Unknown",
- "Intel",
- "AMD",
- "ZHAOXIN",
- "Other",
-};
-
static int
do_test (void)
{
- const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
-
- switch (cpu_features->basic.kind)
- {
- case arch_kind_intel:
- case arch_kind_amd:
- case arch_kind_zhaoxin:
- case arch_kind_other:
- printf ("Vendor: %s\n", cpu_kinds[cpu_features->basic.kind]);
- printf ("Family: 0x%x\n", cpu_features->basic.family);
- printf ("Model: 0x%x\n", cpu_features->basic.model);
- printf ("Stepping: 0x%x\n", cpu_features->basic.stepping);
- break;
-
- default:
- abort ();
- }
-
#ifdef __SSE2__
TEST_VERIFY_EXIT (HAS_CPU_FEATURE (SSE2));
#endif
diff --git a/sysdeps/x86/tst-isa-level-1.c b/sysdeps/x86/tst-isa-level-1.c
index cf611e8b78..e4c2db3ca1 100644
--- a/sysdeps/x86/tst-isa-level-1.c
+++ b/sysdeps/x86/tst-isa-level-1.c
@@ -60,8 +60,7 @@ do_test_1 (const char *modname, bool fail)
static int
do_test (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
unsigned int isa_level = get_isa_level (cpu_features);
bool has_isa_baseline = ((isa_level & GNU_PROPERTY_X86_ISA_1_BASELINE)
== GNU_PROPERTY_X86_ISA_1_BASELINE);
diff --git a/sysdeps/x86_64/dl-hwcaps-subdirs.c b/sysdeps/x86_64/dl-hwcaps-subdirs.c
index 4acd343183..ab9c24e5fb 100644
--- a/sysdeps/x86_64/dl-hwcaps-subdirs.c
+++ b/sysdeps/x86_64/dl-hwcaps-subdirs.c
@@ -27,8 +27,7 @@ enum { subdirs_count = 3 }; /* Number of components in _dl_hwcaps_subdirs. */
uint32_t
_dl_hwcaps_subdirs_active (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
unsigned int isa_level = get_isa_level (cpu_features);
int active = 0;
diff --git a/sysdeps/x86_64/tst-glibc-hwcaps.c b/sysdeps/x86_64/tst-glibc-hwcaps.c
index 37b61d3c84..01bee6291d 100644
--- a/sysdeps/x86_64/tst-glibc-hwcaps.c
+++ b/sysdeps/x86_64/tst-glibc-hwcaps.c
@@ -30,8 +30,7 @@ extern int marker4 (void);
static int
compute_level (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
unsigned int isa_level = get_isa_level (cpu_features);
if (!(isa_level & GNU_PROPERTY_X86_ISA_1_V2))
--
2.29.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: V2 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-20 14:43 ` V2 " H.J. Lu via Libc-alpha
@ 2021-01-20 15:58 ` Florian Weimer via Libc-alpha
2021-01-21 0:19 ` V3 " H.J. Lu via Libc-alpha
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer via Libc-alpha @ 2021-01-20 15:58 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha
* H. J. Lu via Libc-alpha:
> On Wed, Jan 20, 2021 at 10:53:07AM +0100, Florian Weimer wrote:
>> * H. J. Lu via Libc-alpha:
>>
>> > +enum
>> > +{
>> > + COMMON_CPUID_INDEX_1 = 0,
>> > + COMMON_CPUID_INDEX_7,
>> > + COMMON_CPUID_INDEX_80000001,
>> > + COMMON_CPUID_INDEX_D_ECX_1,
>> > + COMMON_CPUID_INDEX_80000007,
>> > + COMMON_CPUID_INDEX_80000008,
>> > + COMMON_CPUID_INDEX_7_ECX_1,
>> > + COMMON_CPUID_INDEX_19,
>> > + /* Keep the following line at the end. */
>> > + COMMON_CPUID_INDEX_MAX
>> > +};
>>
>> COMMON_CPUID_INDEX_MAX should not be in the public header because it
>> subject to changes.
>
> Fixed.
>
>>
>> > +struct cpuid_registers
>> > +{
>> > + unsigned int eax;
>> > + unsigned int ebx;
>> > + unsigned int ecx;
>> > + unsigned int edx;
>> > +};
>>
>> Should the non-public interfaces use __?
>
> Moved to sysdeps/x86/include/cpu-features.h.
>
>>
>> > +enum cpu_features_kind
>> > +{
>> > + arch_kind_unknown = 0,
>> > + arch_kind_intel,
>> > + arch_kind_amd,
>> > + arch_kind_zhaoxin,
>> > + arch_kind_other
>> > +};
>> > +
>> > +struct cpu_features_basic
>> > +{
>> > + enum cpu_features_kind kind;
>> > + int max_cpuid;
>> > + unsigned int family;
>> > + unsigned int model;
>> > + unsigned int stepping;
>> > +};
>>
>> Should we really expose all this? It's not documented. enum
>> cpu_features_kind doesn't seem to be compatible with how people use
>> virtualization.
>
> Moved to sysdeps/x86/include/cpu-features.h.
>
>>
>> > diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
>> > index 349472d99f..4636d9f4a7 100644
>> > --- a/sysdeps/x86/dl-get-cpu-features.c
>> > +++ b/sysdeps/x86/dl-get-cpu-features.c
>> > @@ -46,9 +46,9 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
>> > #undef __x86_get_cpu_features
>> >
>> > const struct cpu_features *
>> > -__x86_get_cpu_features (unsigned int max)
>> > +__x86_get_cpu_features (unsigned int index)
>> > {
>> > - if (max > COMMON_CPUID_INDEX_MAX)
>> > + if (index > COMMON_CPUID_INDEX_MAX * 8 * sizeof (unsigned int) * 4)
>> > return NULL;
>> > return &GLRO(dl_x86_cpu_features);
>> > }
>>
>> This should work in principle (but I haven't verified the boundary
>> condition).
>>
>> Could you pass the COMMON_CPUID_INDEX_* value (without the register and
>> bit selectors)? Then more calls can be subject to common subexpression
>> elimination in the caller, leading to more compact code.
>
> My current public API uses a single enum index for each CPU feature
> detection. Use 2 indices for each CPU feature makes it less user
> friendly. Internally, we use the C preprocessor magic so that more
> calls can be subject to common subexpression elimination in the caller,
> leading to more compact code for glibc.
I had this in mind for the implementation:
struct cpuid_feature
{
unsigned int cpuid_array[4];
unsigned int usable_array[4];
};
const struct cpuid_feature *
__x86_get_cpu_features (unsigned int leaf)
{
const struct cpu_feature future = { };
if (leaf < CPUID_INDEX_MAX)
return &GLRO(dl_x86_cpu_features).features[leaf];
else
return &future;
}
And in the header:
unsigned int leaf = index / (8 * sizeof (unsigned int) * 4);
const struct cpuid_feature * __x86_get_cpu_features (unsigned int leaf)
__attribute__ ((pure));
static inline const struct cpuid_feature *
__x86_get_cpu_leaf (unsigned int index)
{
return __x86_get_cpu_features (index / (8 * sizeof (unsigned int) * 4));
}
#define HAS_CPU_FEATURE(name) \
(x86_cpu_has_feature (__x86_get_cpu_leaf (x86_cpu_##name), \
x86_cpu_##name))
#define HAS_CPU_FEATURE(name) \
(x86_cpu_is_usable (__x86_get_cpu_leaf (x86_cpu_##name), \
x86_cpu_##name))
I expect that the index arguments to __x86_get_cpu_features are
constant-folded by GCC, and many of the __x86_get_cpu_features calls can
be CSE'ed as a result.
In the public interface, the max_cpuid field appears redundant, so this
is why I think this will work.
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 10+ messages in thread
* V3 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-20 15:58 ` Florian Weimer via Libc-alpha
@ 2021-01-21 0:19 ` H.J. Lu via Libc-alpha
2021-01-21 11:19 ` Florian Weimer via Libc-alpha
0 siblings, 1 reply; 10+ messages in thread
From: H.J. Lu via Libc-alpha @ 2021-01-21 0:19 UTC (permalink / raw)
To: Florian Weimer; +Cc: H.J. Lu via Libc-alpha
[-- Attachment #1: Type: text/plain, Size: 4698 bytes --]
On Wed, Jan 20, 2021 at 7:58 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * H. J. Lu via Libc-alpha:
>
> > On Wed, Jan 20, 2021 at 10:53:07AM +0100, Florian Weimer wrote:
> >> * H. J. Lu via Libc-alpha:
> >>
> >> > +enum
> >> > +{
> >> > + COMMON_CPUID_INDEX_1 = 0,
> >> > + COMMON_CPUID_INDEX_7,
> >> > + COMMON_CPUID_INDEX_80000001,
> >> > + COMMON_CPUID_INDEX_D_ECX_1,
> >> > + COMMON_CPUID_INDEX_80000007,
> >> > + COMMON_CPUID_INDEX_80000008,
> >> > + COMMON_CPUID_INDEX_7_ECX_1,
> >> > + COMMON_CPUID_INDEX_19,
> >> > + /* Keep the following line at the end. */
> >> > + COMMON_CPUID_INDEX_MAX
> >> > +};
> >>
> >> COMMON_CPUID_INDEX_MAX should not be in the public header because it
> >> subject to changes.
> >
> > Fixed.
> >
> >>
> >> > +struct cpuid_registers
> >> > +{
> >> > + unsigned int eax;
> >> > + unsigned int ebx;
> >> > + unsigned int ecx;
> >> > + unsigned int edx;
> >> > +};
> >>
> >> Should the non-public interfaces use __?
> >
> > Moved to sysdeps/x86/include/cpu-features.h.
> >
> >>
> >> > +enum cpu_features_kind
> >> > +{
> >> > + arch_kind_unknown = 0,
> >> > + arch_kind_intel,
> >> > + arch_kind_amd,
> >> > + arch_kind_zhaoxin,
> >> > + arch_kind_other
> >> > +};
> >> > +
> >> > +struct cpu_features_basic
> >> > +{
> >> > + enum cpu_features_kind kind;
> >> > + int max_cpuid;
> >> > + unsigned int family;
> >> > + unsigned int model;
> >> > + unsigned int stepping;
> >> > +};
> >>
> >> Should we really expose all this? It's not documented. enum
> >> cpu_features_kind doesn't seem to be compatible with how people use
> >> virtualization.
> >
> > Moved to sysdeps/x86/include/cpu-features.h.
> >
> >>
> >> > diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
> >> > index 349472d99f..4636d9f4a7 100644
> >> > --- a/sysdeps/x86/dl-get-cpu-features.c
> >> > +++ b/sysdeps/x86/dl-get-cpu-features.c
> >> > @@ -46,9 +46,9 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
> >> > #undef __x86_get_cpu_features
> >> >
> >> > const struct cpu_features *
> >> > -__x86_get_cpu_features (unsigned int max)
> >> > +__x86_get_cpu_features (unsigned int index)
> >> > {
> >> > - if (max > COMMON_CPUID_INDEX_MAX)
> >> > + if (index > COMMON_CPUID_INDEX_MAX * 8 * sizeof (unsigned int) * 4)
> >> > return NULL;
> >> > return &GLRO(dl_x86_cpu_features);
> >> > }
> >>
> >> This should work in principle (but I haven't verified the boundary
> >> condition).
> >>
> >> Could you pass the COMMON_CPUID_INDEX_* value (without the register and
> >> bit selectors)? Then more calls can be subject to common subexpression
> >> elimination in the caller, leading to more compact code.
> >
> > My current public API uses a single enum index for each CPU feature
> > detection. Use 2 indices for each CPU feature makes it less user
> > friendly. Internally, we use the C preprocessor magic so that more
> > calls can be subject to common subexpression elimination in the caller,
> > leading to more compact code for glibc.
>
> I had this in mind for the implementation:
>
> struct cpuid_feature
> {
> unsigned int cpuid_array[4];
> unsigned int usable_array[4];
> };
>
> const struct cpuid_feature *
> __x86_get_cpu_features (unsigned int leaf)
> {
> const struct cpu_feature future = { };
> if (leaf < CPUID_INDEX_MAX)
> return &GLRO(dl_x86_cpu_features).features[leaf];
> else
> return &future;
> }
>
> And in the header:
>
> unsigned int leaf = index / (8 * sizeof (unsigned int) * 4);
>
> const struct cpuid_feature * __x86_get_cpu_features (unsigned int leaf)
> __attribute__ ((pure));
>
> static inline const struct cpuid_feature *
> __x86_get_cpu_leaf (unsigned int index)
> {
> return __x86_get_cpu_features (index / (8 * sizeof (unsigned int) * 4));
> }
> #define HAS_CPU_FEATURE(name) \
> (x86_cpu_has_feature (__x86_get_cpu_leaf (x86_cpu_##name), \
> x86_cpu_##name))
> #define HAS_CPU_FEATURE(name) \
> (x86_cpu_is_usable (__x86_get_cpu_leaf (x86_cpu_##name), \
> x86_cpu_##name))
>
>
> I expect that the index arguments to __x86_get_cpu_features are
> constant-folded by GCC, and many of the __x86_get_cpu_features calls can
> be CSE'ed as a result.
>
> In the public interface, the max_cpuid field appears redundant, so this
> is why I think this will work.
>
>
Here is the updated patch. The public function is changed to
/* Get a pointer to the CPU feature structure. */
extern const struct cpuid_feature *__x86_get_cpuid_feature_leaf (unsigned int)
__attribute__ ((pure));
OK for master?
Thanks.
--
H.J.
[-- Attachment #2: 0001-sys-platform-x86.h-Remove-the-C-preprocessor-magic.patch --]
[-- Type: text/x-patch, Size: 80586 bytes --]
From 8464eeadd70f3e41fa8313639884473c81803e7e Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Fri, 25 Dec 2020 07:30:46 -0800
Subject: [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
In <sys/platform/x86.h>, define CPU features as enum instead of using
the C preprocessor magic to make it easier to wrap this functionality
in other languages. Move the C preprocessor magic to internal header
for better GCC codegen when more than one features are checked in a
single expression as in x86-64 dl-hwcaps-subdirs.c.
1. Rename COMMON_CPUID_INDEX_XXX to CPUID_INDEX_XXX.
2. Move CPUID_INDEX_MAX to sysdeps/x86/include/cpu-features.h.
3. Remove struct cpu_features and __x86_get_cpu_features from
<sys/platform/x86.h>.
4. Add __x86_get_cpuid_feature_leaf to <sys/platform/x86.h>.
5. Make __get_cpu_features() private to glibc.
6. Replace __x86_get_cpu_features(N) with __get_cpu_features().
7. Add _dl_x86_get_cpu_features to GLIBC_PRIVATE.
8. Use a single enum index for each CPU feature detection.
9. Pass the CPUID feature leaf to __x86_get_cpuid_feature_leaf.
10. Return zero struct cpuid_feature for the older glibc binary with a
smaller CPUID_INDEX_MAX [BZ #27104].
11. Inside glibc, use the C preprocessor magic so that cpu_features data
can be loaded just once leading to more compact code for glibc.
256 bits are used for each CPUID leaf. Some leaves only contain a few
features. We can add exceptions to such leaves. But it will increase
code sizes and it is harder to provide backward/forward compatibilities
when new features are added to such leaves in the future.
When new leaves are added, _rtld_global_ro offsets will change which
leads to race condition during in-place updates. We may avoid in-place
updates by
1. Rename the old glibc.
2. Install the new glibc.
3. Remove the old glibc.
NB: A function, __x86_get_cpuid_feature_leaf , is used to avoid the copy
relocation issue with IFUNC resolver as shown in IFUNC resolver tests.
---
manual/platform.texi | 7 +-
sysdeps/mach/hurd/i386/ld.abilist | 2 +-
sysdeps/unix/sysv/linux/i386/ld.abilist | 2 +-
sysdeps/unix/sysv/linux/x86_64/64/ld.abilist | 2 +-
.../sysv/linux/x86_64/tst-glibc-hwcaps-2.c | 3 +-
sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist | 2 +-
sysdeps/x86/Versions | 5 +-
sysdeps/x86/bits/platform/x86.h | 299 +++++++
sysdeps/x86/cpu-features.c | 68 +-
sysdeps/x86/dl-cacheinfo.h | 4 +-
sysdeps/x86/dl-get-cpu-features.c | 17 +-
sysdeps/x86/get-isa-level.h | 2 +-
sysdeps/x86/include/cpu-features.h | 759 ++++++++++++++++-
sysdeps/x86/sys/platform/x86.h | 777 +-----------------
sysdeps/x86/tst-get-cpu-features.c | 29 +-
sysdeps/x86/tst-isa-level-1.c | 3 +-
sysdeps/x86_64/dl-hwcaps-subdirs.c | 3 +-
sysdeps/x86_64/tst-glibc-hwcaps.c | 3 +-
18 files changed, 1140 insertions(+), 847 deletions(-)
create mode 100644 sysdeps/x86/bits/platform/x86.h
diff --git a/manual/platform.texi b/manual/platform.texi
index b67683aeb3..6caf68d796 100644
--- a/manual/platform.texi
+++ b/manual/platform.texi
@@ -142,13 +142,10 @@ reserved.
Facilities specific to X86 that are not specific to a particular
operating system are declared in @file{sys/platform/x86.h}.
-@deftypefun {const struct cpu_features *} __x86_get_cpu_features (unsigned int @var{max})
+@deftypefun {const struct cpuid_feature *} __x86_get_cpuid_feature_leaf (unsigned int @var{leaf})
@safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}}
Return a pointer to x86 CPU feature structure used by query macros for x86
-CPU features. If @var{max} exceeds @code{COMMON_CPUID_INDEX_MAX} which
-is the limit of the CPUID leaves supported by @Theglibc{}, the function
-returns @code{NULL}, indicating that the queried processor feature is
-unsupported by @Theglibc{} run-time.
+CPU feature @var{leaf}.
@end deftypefun
@deftypefn Macro int HAS_CPU_FEATURE (@var{name})
diff --git a/sysdeps/mach/hurd/i386/ld.abilist b/sysdeps/mach/hurd/i386/ld.abilist
index 751664bcda..6f9ce00333 100644
--- a/sysdeps/mach/hurd/i386/ld.abilist
+++ b/sysdeps/mach/hurd/i386/ld.abilist
@@ -16,4 +16,4 @@ GLIBC_2.2.6 _r_debug D 0x14
GLIBC_2.2.6 abort F
GLIBC_2.3 ___tls_get_addr F
GLIBC_2.3 __tls_get_addr F
-GLIBC_2.33 __x86_get_cpu_features F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
diff --git a/sysdeps/unix/sysv/linux/i386/ld.abilist b/sysdeps/unix/sysv/linux/i386/ld.abilist
index 04655651b0..847b17f45a 100644
--- a/sysdeps/unix/sysv/linux/i386/ld.abilist
+++ b/sysdeps/unix/sysv/linux/i386/ld.abilist
@@ -3,4 +3,4 @@ GLIBC_2.1 __libc_stack_end D 0x4
GLIBC_2.1 _dl_mcount F
GLIBC_2.3 ___tls_get_addr F
GLIBC_2.3 __tls_get_addr F
-GLIBC_2.33 __x86_get_cpu_features F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
diff --git a/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist b/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
index e632c6198e..c7b4089c5c 100644
--- a/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
@@ -2,4 +2,4 @@ GLIBC_2.2.5 __libc_stack_end D 0x8
GLIBC_2.2.5 _dl_mcount F
GLIBC_2.2.5 _r_debug D 0x28
GLIBC_2.3 __tls_get_addr F
-GLIBC_2.33 __x86_get_cpu_features F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
diff --git a/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c b/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
index fe91bfd224..2d8f7ec462 100644
--- a/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
+++ b/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
@@ -29,8 +29,7 @@ extern int dso_isa_level (void);
static int
do_test (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
bool has_isa_baseline = ((isa_level & GNU_PROPERTY_X86_ISA_1_BASELINE)
== GNU_PROPERTY_X86_ISA_1_BASELINE);
diff --git a/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist b/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
index 9d9d0befde..02eaa9a347 100644
--- a/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
@@ -2,4 +2,4 @@ GLIBC_2.16 __libc_stack_end D 0x4
GLIBC_2.16 __tls_get_addr F
GLIBC_2.16 _dl_mcount F
GLIBC_2.16 _r_debug D 0x14
-GLIBC_2.33 __x86_get_cpu_features F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
diff --git a/sysdeps/x86/Versions b/sysdeps/x86/Versions
index 59db578a9d..8c6801238c 100644
--- a/sysdeps/x86/Versions
+++ b/sysdeps/x86/Versions
@@ -1,5 +1,8 @@
ld {
GLIBC_2.33 {
- __x86_get_cpu_features;
+ __x86_get_cpuid_feature_leaf;
+ }
+ GLIBC_PRIVATE {
+ _dl_x86_get_cpu_features;
}
}
diff --git a/sysdeps/x86/bits/platform/x86.h b/sysdeps/x86/bits/platform/x86.h
new file mode 100644
index 0000000000..8f423ae721
--- /dev/null
+++ b/sysdeps/x86/bits/platform/x86.h
@@ -0,0 +1,299 @@
+/* Constants and data structures for x86 CPU features.
+ This file is part of the GNU C Library.
+ Copyright (C) 2008-2020 Free Software Foundation, Inc.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_PLATFORM_X86_H
+# error "Never include <bits/platform/x86.h> directly; use <sys/platform/x86.h> instead."
+#endif
+
+enum
+{
+ CPUID_INDEX_1 = 0,
+ CPUID_INDEX_7,
+ CPUID_INDEX_80000001,
+ CPUID_INDEX_D_ECX_1,
+ CPUID_INDEX_80000007,
+ CPUID_INDEX_80000008,
+ CPUID_INDEX_7_ECX_1,
+ CPUID_INDEX_19
+};
+
+struct cpuid_feature
+{
+ unsigned int cpuid_array[4];
+ unsigned int usable_array[4];
+};
+
+enum cpuid_register_index
+{
+ cpuid_register_index_eax = 0,
+ cpuid_register_index_ebx,
+ cpuid_register_index_ecx,
+ cpuid_register_index_edx
+};
+
+/* CPU features. */
+
+enum
+{
+ x86_cpu_index_1_ecx
+ = (CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SSE3 = x86_cpu_index_1_ecx,
+ x86_cpu_PCLMULQDQ = x86_cpu_index_1_ecx + 1,
+ x86_cpu_DTES64 = x86_cpu_index_1_ecx + 2,
+ x86_cpu_MONITOR = x86_cpu_index_1_ecx + 3,
+ x86_cpu_DS_CPL = x86_cpu_index_1_ecx + 4,
+ x86_cpu_VMX = x86_cpu_index_1_ecx + 5,
+ x86_cpu_SMX = x86_cpu_index_1_ecx + 6,
+ x86_cpu_EIST = x86_cpu_index_1_ecx + 7,
+ x86_cpu_TM2 = x86_cpu_index_1_ecx + 8,
+ x86_cpu_SSSE3 = x86_cpu_index_1_ecx + 9,
+ x86_cpu_CNXT_ID = x86_cpu_index_1_ecx + 10,
+ x86_cpu_SDBG = x86_cpu_index_1_ecx + 11,
+ x86_cpu_FMA = x86_cpu_index_1_ecx + 12,
+ x86_cpu_CMPXCHG16B = x86_cpu_index_1_ecx + 13,
+ x86_cpu_XTPRUPDCTRL = x86_cpu_index_1_ecx + 14,
+ x86_cpu_PDCM = x86_cpu_index_1_ecx + 15,
+ x86_cpu_INDEX_1_ECX_16 = x86_cpu_index_1_ecx + 16,
+ x86_cpu_PCID = x86_cpu_index_1_ecx + 17,
+ x86_cpu_DCA = x86_cpu_index_1_ecx + 18,
+ x86_cpu_SSE4_1 = x86_cpu_index_1_ecx + 19,
+ x86_cpu_SSE4_2 = x86_cpu_index_1_ecx + 20,
+ x86_cpu_X2APIC = x86_cpu_index_1_ecx + 21,
+ x86_cpu_MOVBE = x86_cpu_index_1_ecx + 22,
+ x86_cpu_POPCNT = x86_cpu_index_1_ecx + 23,
+ x86_cpu_TSC_DEADLINE = x86_cpu_index_1_ecx + 24,
+ x86_cpu_AES = x86_cpu_index_1_ecx + 25,
+ x86_cpu_XSAVE = x86_cpu_index_1_ecx + 26,
+ x86_cpu_OSXSAVE = x86_cpu_index_1_ecx + 27,
+ x86_cpu_AVX = x86_cpu_index_1_ecx + 28,
+ x86_cpu_F16C = x86_cpu_index_1_ecx + 29,
+ x86_cpu_RDRAND = x86_cpu_index_1_ecx + 30,
+ x86_cpu_INDEX_1_ECX_31 = x86_cpu_index_1_ecx + 31,
+
+ x86_cpu_index_1_edx
+ = (CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FPU = x86_cpu_index_1_edx,
+ x86_cpu_VME = x86_cpu_index_1_edx + 1,
+ x86_cpu_DE = x86_cpu_index_1_edx + 2,
+ x86_cpu_PSE = x86_cpu_index_1_edx + 3,
+ x86_cpu_TSC = x86_cpu_index_1_edx + 4,
+ x86_cpu_MSR = x86_cpu_index_1_edx + 5,
+ x86_cpu_PAE = x86_cpu_index_1_edx + 6,
+ x86_cpu_MCE = x86_cpu_index_1_edx + 7,
+ x86_cpu_CX8 = x86_cpu_index_1_edx + 8,
+ x86_cpu_APIC = x86_cpu_index_1_edx + 9,
+ x86_cpu_INDEX_1_EDX_10 = x86_cpu_index_1_edx + 10,
+ x86_cpu_SEP = x86_cpu_index_1_edx + 11,
+ x86_cpu_MTRR = x86_cpu_index_1_edx + 12,
+ x86_cpu_PGE = x86_cpu_index_1_edx + 13,
+ x86_cpu_MCA = x86_cpu_index_1_edx + 14,
+ x86_cpu_CMOV = x86_cpu_index_1_edx + 15,
+ x86_cpu_PAT = x86_cpu_index_1_edx + 16,
+ x86_cpu_PSE_36 = x86_cpu_index_1_edx + 17,
+ x86_cpu_PSN = x86_cpu_index_1_edx + 18,
+ x86_cpu_CLFSH = x86_cpu_index_1_edx + 19,
+ x86_cpu_INDEX_1_EDX_20 = x86_cpu_index_1_edx + 20,
+ x86_cpu_DS = x86_cpu_index_1_edx + 21,
+ x86_cpu_ACPI = x86_cpu_index_1_edx + 22,
+ x86_cpu_MMX = x86_cpu_index_1_edx + 23,
+ x86_cpu_FXSR = x86_cpu_index_1_edx + 24,
+ x86_cpu_SSE = x86_cpu_index_1_edx + 25,
+ x86_cpu_SSE2 = x86_cpu_index_1_edx + 26,
+ x86_cpu_SS = x86_cpu_index_1_edx + 27,
+ x86_cpu_HTT = x86_cpu_index_1_edx + 28,
+ x86_cpu_TM = x86_cpu_index_1_edx + 29,
+ x86_cpu_INDEX_1_EDX_30 = x86_cpu_index_1_edx + 30,
+ x86_cpu_PBE = x86_cpu_index_1_edx + 31,
+
+ x86_cpu_index_7_ebx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FSGSBASE = x86_cpu_index_7_ebx,
+ x86_cpu_TSC_ADJUST = x86_cpu_index_7_ebx + 1,
+ x86_cpu_SGX = x86_cpu_index_7_ebx + 2,
+ x86_cpu_BMI1 = x86_cpu_index_7_ebx + 3,
+ x86_cpu_HLE = x86_cpu_index_7_ebx + 4,
+ x86_cpu_AVX2 = x86_cpu_index_7_ebx + 5,
+ x86_cpu_INDEX_7_EBX_6 = x86_cpu_index_7_ebx + 6,
+ x86_cpu_SMEP = x86_cpu_index_7_ebx + 7,
+ x86_cpu_BMI2 = x86_cpu_index_7_ebx + 8,
+ x86_cpu_ERMS = x86_cpu_index_7_ebx + 9,
+ x86_cpu_INVPCID = x86_cpu_index_7_ebx + 10,
+ x86_cpu_RTM = x86_cpu_index_7_ebx + 11,
+ x86_cpu_RDT_M = x86_cpu_index_7_ebx + 12,
+ x86_cpu_DEPR_FPU_CS_DS = x86_cpu_index_7_ebx + 13,
+ x86_cpu_MPX = x86_cpu_index_7_ebx + 14,
+ x86_cpu_RDT_A = x86_cpu_index_7_ebx + 15,
+ x86_cpu_AVX512F = x86_cpu_index_7_ebx + 16,
+ x86_cpu_AVX512DQ = x86_cpu_index_7_ebx + 17,
+ x86_cpu_RDSEED = x86_cpu_index_7_ebx + 18,
+ x86_cpu_ADX = x86_cpu_index_7_ebx + 19,
+ x86_cpu_SMAP = x86_cpu_index_7_ebx + 20,
+ x86_cpu_AVX512_IFMA = x86_cpu_index_7_ebx + 21,
+ x86_cpu_INDEX_7_EBX_22 = x86_cpu_index_7_ebx + 22,
+ x86_cpu_CLFLUSHOPT = x86_cpu_index_7_ebx + 23,
+ x86_cpu_CLWB = x86_cpu_index_7_ebx + 24,
+ x86_cpu_TRACE = x86_cpu_index_7_ebx + 25,
+ x86_cpu_AVX512PF = x86_cpu_index_7_ebx + 26,
+ x86_cpu_AVX512ER = x86_cpu_index_7_ebx + 27,
+ x86_cpu_AVX512CD = x86_cpu_index_7_ebx + 28,
+ x86_cpu_SHA = x86_cpu_index_7_ebx + 29,
+ x86_cpu_AVX512BW = x86_cpu_index_7_ebx + 30,
+ x86_cpu_AVX512VL = x86_cpu_index_7_ebx + 31,
+
+ x86_cpu_index_7_ecx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_PREFETCHWT1 = x86_cpu_index_7_ecx,
+ x86_cpu_AVX512_VBMI = x86_cpu_index_7_ecx + 1,
+ x86_cpu_UMIP = x86_cpu_index_7_ecx + 2,
+ x86_cpu_PKU = x86_cpu_index_7_ecx + 3,
+ x86_cpu_OSPKE = x86_cpu_index_7_ecx + 4,
+ x86_cpu_WAITPKG = x86_cpu_index_7_ecx + 5,
+ x86_cpu_AVX512_VBMI2 = x86_cpu_index_7_ecx + 6,
+ x86_cpu_SHSTK = x86_cpu_index_7_ecx + 7,
+ x86_cpu_GFNI = x86_cpu_index_7_ecx + 8,
+ x86_cpu_VAES = x86_cpu_index_7_ecx + 9,
+ x86_cpu_VPCLMULQDQ = x86_cpu_index_7_ecx + 10,
+ x86_cpu_AVX512_VNNI = x86_cpu_index_7_ecx + 11,
+ x86_cpu_AVX512_BITALG = x86_cpu_index_7_ecx + 12,
+ x86_cpu_INDEX_7_ECX_13 = x86_cpu_index_7_ecx + 13,
+ x86_cpu_AVX512_VPOPCNTDQ = x86_cpu_index_7_ecx + 14,
+ x86_cpu_INDEX_7_ECX_1 = x86_cpu_index_7_ecx + 15,
+ x86_cpu_INDEX_7_ECX_16 = x86_cpu_index_7_ecx + 16,
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+ x86_cpu_RDPID = x86_cpu_index_7_ecx + 22,
+ x86_cpu_KL = x86_cpu_index_7_ecx + 23,
+ x86_cpu_INDEX_7_ECX_24 = x86_cpu_index_7_ecx + 24,
+ x86_cpu_CLDEMOTE = x86_cpu_index_7_ecx + 25,
+ x86_cpu_INDEX_7_ECX_26 = x86_cpu_index_7_ecx + 26,
+ x86_cpu_MOVDIRI = x86_cpu_index_7_ecx + 27,
+ x86_cpu_MOVDIR64B = x86_cpu_index_7_ecx + 28,
+ x86_cpu_ENQCMD = x86_cpu_index_7_ecx + 29,
+ x86_cpu_SGX_LC = x86_cpu_index_7_ecx + 30,
+ x86_cpu_PKS = x86_cpu_index_7_ecx + 31,
+
+ x86_cpu_index_7_edx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INDEX_7_EDX_0 = x86_cpu_index_7_edx,
+ x86_cpu_INDEX_7_EDX_1 = x86_cpu_index_7_edx + 1,
+ x86_cpu_AVX512_4VNNIW = x86_cpu_index_7_edx + 2,
+ x86_cpu_AVX512_4FMAPS = x86_cpu_index_7_edx + 3,
+ x86_cpu_FSRM = x86_cpu_index_7_edx + 4,
+ x86_cpu_UINTR = x86_cpu_index_7_edx + 5,
+ x86_cpu_INDEX_7_EDX_6 = x86_cpu_index_7_edx + 6,
+ x86_cpu_INDEX_7_EDX_7 = x86_cpu_index_7_edx + 7,
+ x86_cpu_AVX512_VP2INTERSECT = x86_cpu_index_7_edx + 8,
+ x86_cpu_INDEX_7_EDX_9 = x86_cpu_index_7_edx + 9,
+ x86_cpu_MD_CLEAR = x86_cpu_index_7_edx + 10,
+ x86_cpu_INDEX_7_EDX_11 = x86_cpu_index_7_edx + 11,
+ x86_cpu_INDEX_7_EDX_12 = x86_cpu_index_7_edx + 12,
+ x86_cpu_INDEX_7_EDX_13 = x86_cpu_index_7_edx + 13,
+ x86_cpu_SERIALIZE = x86_cpu_index_7_edx + 14,
+ x86_cpu_HYBRID = x86_cpu_index_7_edx + 15,
+ x86_cpu_TSXLDTRK = x86_cpu_index_7_edx + 16,
+ x86_cpu_INDEX_7_EDX_17 = x86_cpu_index_7_edx + 17,
+ x86_cpu_PCONFIG = x86_cpu_index_7_edx + 18,
+ x86_cpu_INDEX_7_EDX_19 = x86_cpu_index_7_edx + 19,
+ x86_cpu_IBT = x86_cpu_index_7_edx + 20,
+ x86_cpu_INDEX_7_EDX_21 = x86_cpu_index_7_edx + 21,
+ x86_cpu_AMX_BF16 = x86_cpu_index_7_edx + 22,
+ x86_cpu_AVX512_FP16 = x86_cpu_index_7_edx + 23,
+ x86_cpu_AMX_TILE = x86_cpu_index_7_edx + 24,
+ x86_cpu_AMX_INT8 = x86_cpu_index_7_edx + 25,
+ x86_cpu_IBRS_IBPB = x86_cpu_index_7_edx + 26,
+ x86_cpu_STIBP = x86_cpu_index_7_edx + 27,
+ x86_cpu_L1D_FLUSH = x86_cpu_index_7_edx + 28,
+ x86_cpu_ARCH_CAPABILITIES = x86_cpu_index_7_edx + 29,
+ x86_cpu_CORE_CAPABILITIES = x86_cpu_index_7_edx + 30,
+ x86_cpu_SSBD = x86_cpu_index_7_edx + 31,
+
+ x86_cpu_index_80000001_ecx
+ = (CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_LAHF64_SAHF64 = x86_cpu_index_80000001_ecx,
+ x86_cpu_SVM = x86_cpu_index_80000001_ecx + 2,
+ x86_cpu_LZCNT = x86_cpu_index_80000001_ecx + 5,
+ x86_cpu_SSE4A = x86_cpu_index_80000001_ecx + 6,
+ x86_cpu_PREFETCHW = x86_cpu_index_80000001_ecx + 8,
+ x86_cpu_XOP = x86_cpu_index_80000001_ecx + 11,
+ x86_cpu_LWP = x86_cpu_index_80000001_ecx + 15,
+ x86_cpu_FMA4 = x86_cpu_index_80000001_ecx + 16,
+ x86_cpu_TBM = x86_cpu_index_80000001_ecx + 20,
+
+ x86_cpu_index_80000001_edx
+ = (CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SYSCALL_SYSRET = x86_cpu_index_80000001_edx + 11,
+ x86_cpu_NX = x86_cpu_index_80000001_edx + 20,
+ x86_cpu_PAGE1GB = x86_cpu_index_80000001_edx + 26,
+ x86_cpu_RDTSCP = x86_cpu_index_80000001_edx + 27,
+ x86_cpu_LM = x86_cpu_index_80000001_edx + 29,
+
+ x86_cpu_index_d_ecx_1_eax
+ = (CPUID_INDEX_D_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_XSAVEOPT = x86_cpu_index_d_ecx_1_eax,
+ x86_cpu_XSAVEC = x86_cpu_index_d_ecx_1_eax + 1,
+ x86_cpu_XGETBV_ECX_1 = x86_cpu_index_d_ecx_1_eax + 2,
+ x86_cpu_XSAVES = x86_cpu_index_d_ecx_1_eax + 3,
+ x86_cpu_XFD = x86_cpu_index_d_ecx_1_eax + 4,
+
+ x86_cpu_index_80000007_edx
+ = (CPUID_INDEX_80000007 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INVARIANT_TSC = x86_cpu_index_80000007_edx + 8,
+
+ x86_cpu_index_80000008_ebx
+ = (CPUID_INDEX_80000008 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_WBNOINVD = x86_cpu_index_80000008_ebx + 9,
+
+ x86_cpu_index_7_ecx_1_eax
+ = (CPUID_INDEX_7_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AVX_VNNI = x86_cpu_index_7_ecx_1_eax + 4,
+ x86_cpu_AVX512_BF16 = x86_cpu_index_7_ecx_1_eax + 5,
+ x86_cpu_FZLRM = x86_cpu_index_7_ecx_1_eax + 10,
+ x86_cpu_FSRS = x86_cpu_index_7_ecx_1_eax + 11,
+ x86_cpu_FSRCS = x86_cpu_index_7_ecx_1_eax + 12,
+ x86_cpu_HRESET = x86_cpu_index_7_ecx_1_eax + 22,
+ x86_cpu_LAM = x86_cpu_index_7_ecx_1_eax + 26,
+
+ x86_cpu_index_19_ebx
+ = (CPUID_INDEX_19 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AESKLE = x86_cpu_index_19_ebx,
+ x86_cpu_WIDE_KL = x86_cpu_index_19_ebx + 2
+};
diff --git a/sysdeps/x86/cpu-features.c b/sysdeps/x86/cpu-features.c
index 06e4307b71..6496512a0d 100644
--- a/sysdeps/x86/cpu-features.c
+++ b/sysdeps/x86/cpu-features.c
@@ -293,22 +293,22 @@ get_extended_indices (struct cpu_features *cpu_features)
__cpuid (0x80000000, eax, ebx, ecx, edx);
if (eax >= 0x80000001)
__cpuid (0x80000001,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.edx);
if (eax >= 0x80000007)
__cpuid (0x80000007,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.edx);
if (eax >= 0x80000008)
__cpuid (0x80000008,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.edx);
}
static void
@@ -320,10 +320,10 @@ get_common_indices (struct cpu_features *cpu_features,
{
unsigned int eax;
__cpuid (1, eax,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.edx);
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.eax = eax;
+ cpu_features->features[CPUID_INDEX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_1].cpuid.eax = eax;
*family = (eax >> 8) & 0x0f;
*model = (eax >> 4) & 0x0f;
*extended_model = (eax >> 12) & 0xf0;
@@ -338,30 +338,30 @@ get_common_indices (struct cpu_features *cpu_features,
if (cpu_features->basic.max_cpuid >= 7)
{
__cpuid_count (7, 0,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_7].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_7].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_7].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_7].cpuid.edx);
__cpuid_count (7, 1,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.edx);
}
if (cpu_features->basic.max_cpuid >= 0xd)
__cpuid_count (0xd, 1,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.edx);
if (cpu_features->basic.max_cpuid >= 0x19)
__cpuid_count (0x19, 0,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_19].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_19].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_19].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_19].cpuid.edx);
}
_Static_assert (((index_arch_Fast_Unaligned_Load
@@ -536,11 +536,11 @@ init_cpu_features (struct cpu_features *cpu_features)
update_usable (cpu_features);
- ecx = cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ecx;
+ ecx = cpu_features->features[CPUID_INDEX_1].cpuid.ecx;
if (CPU_FEATURE_USABLE_P (cpu_features, AVX))
{
- /* Since the FMA4 bit is in COMMON_CPUID_INDEX_80000001 and
+ /* Since the FMA4 bit is in CPUID_INDEX_80000001 and
FMA4 requires AVX, determine if FMA4 is usable here. */
CPU_FEATURE_SET_USABLE (cpu_features, FMA4);
}
diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h
index 96c49f2411..a31fa0783a 100644
--- a/sysdeps/x86/dl-cacheinfo.h
+++ b/sysdeps/x86/dl-cacheinfo.h
@@ -672,8 +672,8 @@ intel_bug_no_cache_info:
/* Assume that all logical threads share the highest cache
level. */
threads
- = ((cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ebx
- >> 16) & 0xff);
+ = ((cpu_features->features[CPUID_INDEX_1].cpuid.ebx >> 16)
+ & 0xff);
}
/* Cap usage of highest cache level to the number of supported
diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
index 19fc4f1eeb..d73f275dd3 100644
--- a/sysdeps/x86/dl-get-cpu-features.c
+++ b/sysdeps/x86/dl-get-cpu-features.c
@@ -43,12 +43,21 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
_dl_x86_init_cpu_features);
#endif
-#undef __x86_get_cpu_features
+#undef _dl_x86_get_cpu_features
const struct cpu_features *
-__x86_get_cpu_features (unsigned int max)
+_dl_x86_get_cpu_features (void)
{
- if (max > COMMON_CPUID_INDEX_MAX)
- return NULL;
return &GLRO(dl_x86_cpu_features);
}
+
+const struct cpuid_feature *
+__x86_get_cpuid_feature_leaf (unsigned int leaf)
+{
+ static const struct cpuid_feature_internal feature = {};
+ if (leaf < CPUID_INDEX_MAX)
+ return ((const struct cpuid_feature *)
+ &GLRO(dl_x86_cpu_features).features[leaf]);
+ else
+ return (const struct cpuid_feature *) &feature;
+}
diff --git a/sysdeps/x86/get-isa-level.h b/sysdeps/x86/get-isa-level.h
index 3f0e27ee59..aa80f56ca6 100644
--- a/sysdeps/x86/get-isa-level.h
+++ b/sysdeps/x86/get-isa-level.h
@@ -17,7 +17,7 @@
<https://www.gnu.org/licenses/>. */
#include <elf.h>
-#include <sys/platform/x86.h>
+#include <cpu-features.h>
/* Get GNU_PROPERTY_X86_ISA_1_BASELINE and GNU_PROPERTY_X86_ISA_1_V[234]
ISA level. */
diff --git a/sysdeps/x86/include/cpu-features.h b/sysdeps/x86/include/cpu-features.h
index 50cb5a15f5..624736b40e 100644
--- a/sysdeps/x86/include/cpu-features.h
+++ b/sysdeps/x86/include/cpu-features.h
@@ -23,16 +23,14 @@
# error this should be impossible
#endif
-/* Get most of the contents from the public header, but we define a
- different `struct cpu_features' type for private use. */
-#define cpu_features cpu_features_public
-#define __x86_get_cpu_features __x86_get_cpu_features_public
+/* Get data structures without inline functions. */
+#define _SYS_PLATFORM_X86_H
+#include <bits/platform/x86.h>
-#include <sysdeps/x86/sys/platform/x86.h>
-
-#undef cpu_features
-#undef __x86_get_cpu_features
-#define __get_cpu_features() __x86_get_cpu_features (0)
+enum
+{
+ CPUID_INDEX_MAX = CPUID_INDEX_19 + 1
+};
enum
{
@@ -54,14 +52,20 @@ enum
#define CPU_FEATURE_PREFERRED_P(ptr, name) \
((ptr->preferred[index_arch_##name] & bit_arch_##name) != 0)
+#define CPU_FEATURE_CHECK_P(ptr, name, check) \
+ ((ptr->features[index_cpu_##name].check.reg_##name \
+ & bit_cpu_##name) != 0)
+#define CPU_FEATURE_CPU_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, cpuid)
+#define CPU_FEATURE_USABLE_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, usable)
+
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
-#undef HAS_CPU_FEATURE
#define HAS_CPU_FEATURE(name) \
- CPU_FEATURE_CPU_P (__x86_get_cpu_features (0), name)
+ CPU_FEATURE_CPU_P (__get_cpu_features (), name)
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
-#undef CPU_FEATURE_USABLE
#define CPU_FEATURE_USABLE(name) \
- CPU_FEATURE_USABLE_P (__x86_get_cpu_features (0), name)
+ CPU_FEATURE_USABLE_P (__get_cpu_features (), name)
/* CPU_FEATURE_PREFER evaluates to true if we prefer the feature at
runtime. */
#define CPU_FEATURE_PREFERRED(name) \
@@ -74,6 +78,685 @@ enum
#define HAS_ARCH_FEATURE(name) \
CPU_FEATURE_PREFERRED (name)
+/* CPU features. */
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define bit_cpu_SSE3 (1u << 0)
+#define bit_cpu_PCLMULQDQ (1u << 1)
+#define bit_cpu_DTES64 (1u << 2)
+#define bit_cpu_MONITOR (1u << 3)
+#define bit_cpu_DS_CPL (1u << 4)
+#define bit_cpu_VMX (1u << 5)
+#define bit_cpu_SMX (1u << 6)
+#define bit_cpu_EIST (1u << 7)
+#define bit_cpu_TM2 (1u << 8)
+#define bit_cpu_SSSE3 (1u << 9)
+#define bit_cpu_CNXT_ID (1u << 10)
+#define bit_cpu_SDBG (1u << 11)
+#define bit_cpu_FMA (1u << 12)
+#define bit_cpu_CMPXCHG16B (1u << 13)
+#define bit_cpu_XTPRUPDCTRL (1u << 14)
+#define bit_cpu_PDCM (1u << 15)
+#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
+#define bit_cpu_PCID (1u << 17)
+#define bit_cpu_DCA (1u << 18)
+#define bit_cpu_SSE4_1 (1u << 19)
+#define bit_cpu_SSE4_2 (1u << 20)
+#define bit_cpu_X2APIC (1u << 21)
+#define bit_cpu_MOVBE (1u << 22)
+#define bit_cpu_POPCNT (1u << 23)
+#define bit_cpu_TSC_DEADLINE (1u << 24)
+#define bit_cpu_AES (1u << 25)
+#define bit_cpu_XSAVE (1u << 26)
+#define bit_cpu_OSXSAVE (1u << 27)
+#define bit_cpu_AVX (1u << 28)
+#define bit_cpu_F16C (1u << 29)
+#define bit_cpu_RDRAND (1u << 30)
+#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
+
+/* EDX. */
+#define bit_cpu_FPU (1u << 0)
+#define bit_cpu_VME (1u << 1)
+#define bit_cpu_DE (1u << 2)
+#define bit_cpu_PSE (1u << 3)
+#define bit_cpu_TSC (1u << 4)
+#define bit_cpu_MSR (1u << 5)
+#define bit_cpu_PAE (1u << 6)
+#define bit_cpu_MCE (1u << 7)
+#define bit_cpu_CX8 (1u << 8)
+#define bit_cpu_APIC (1u << 9)
+#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
+#define bit_cpu_SEP (1u << 11)
+#define bit_cpu_MTRR (1u << 12)
+#define bit_cpu_PGE (1u << 13)
+#define bit_cpu_MCA (1u << 14)
+#define bit_cpu_CMOV (1u << 15)
+#define bit_cpu_PAT (1u << 16)
+#define bit_cpu_PSE_36 (1u << 17)
+#define bit_cpu_PSN (1u << 18)
+#define bit_cpu_CLFSH (1u << 19)
+#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
+#define bit_cpu_DS (1u << 21)
+#define bit_cpu_ACPI (1u << 22)
+#define bit_cpu_MMX (1u << 23)
+#define bit_cpu_FXSR (1u << 24)
+#define bit_cpu_SSE (1u << 25)
+#define bit_cpu_SSE2 (1u << 26)
+#define bit_cpu_SS (1u << 27)
+#define bit_cpu_HTT (1u << 28)
+#define bit_cpu_TM (1u << 29)
+#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
+#define bit_cpu_PBE (1u << 31)
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define bit_cpu_FSGSBASE (1u << 0)
+#define bit_cpu_TSC_ADJUST (1u << 1)
+#define bit_cpu_SGX (1u << 2)
+#define bit_cpu_BMI1 (1u << 3)
+#define bit_cpu_HLE (1u << 4)
+#define bit_cpu_AVX2 (1u << 5)
+#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
+#define bit_cpu_SMEP (1u << 7)
+#define bit_cpu_BMI2 (1u << 8)
+#define bit_cpu_ERMS (1u << 9)
+#define bit_cpu_INVPCID (1u << 10)
+#define bit_cpu_RTM (1u << 11)
+#define bit_cpu_RDT_M (1u << 12)
+#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
+#define bit_cpu_MPX (1u << 14)
+#define bit_cpu_RDT_A (1u << 15)
+#define bit_cpu_AVX512F (1u << 16)
+#define bit_cpu_AVX512DQ (1u << 17)
+#define bit_cpu_RDSEED (1u << 18)
+#define bit_cpu_ADX (1u << 19)
+#define bit_cpu_SMAP (1u << 20)
+#define bit_cpu_AVX512_IFMA (1u << 21)
+#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
+#define bit_cpu_CLFLUSHOPT (1u << 23)
+#define bit_cpu_CLWB (1u << 24)
+#define bit_cpu_TRACE (1u << 25)
+#define bit_cpu_AVX512PF (1u << 26)
+#define bit_cpu_AVX512ER (1u << 27)
+#define bit_cpu_AVX512CD (1u << 28)
+#define bit_cpu_SHA (1u << 29)
+#define bit_cpu_AVX512BW (1u << 30)
+#define bit_cpu_AVX512VL (1u << 31)
+
+/* ECX. */
+#define bit_cpu_PREFETCHWT1 (1u << 0)
+#define bit_cpu_AVX512_VBMI (1u << 1)
+#define bit_cpu_UMIP (1u << 2)
+#define bit_cpu_PKU (1u << 3)
+#define bit_cpu_OSPKE (1u << 4)
+#define bit_cpu_WAITPKG (1u << 5)
+#define bit_cpu_AVX512_VBMI2 (1u << 6)
+#define bit_cpu_SHSTK (1u << 7)
+#define bit_cpu_GFNI (1u << 8)
+#define bit_cpu_VAES (1u << 9)
+#define bit_cpu_VPCLMULQDQ (1u << 10)
+#define bit_cpu_AVX512_VNNI (1u << 11)
+#define bit_cpu_AVX512_BITALG (1u << 12)
+#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
+#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
+#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
+#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+#define bit_cpu_RDPID (1u << 22)
+#define bit_cpu_KL (1u << 23)
+#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
+#define bit_cpu_CLDEMOTE (1u << 25)
+#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
+#define bit_cpu_MOVDIRI (1u << 27)
+#define bit_cpu_MOVDIR64B (1u << 28)
+#define bit_cpu_ENQCMD (1u << 29)
+#define bit_cpu_SGX_LC (1u << 30)
+#define bit_cpu_PKS (1u << 31)
+
+/* EDX. */
+#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
+#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
+#define bit_cpu_AVX512_4VNNIW (1u << 2)
+#define bit_cpu_AVX512_4FMAPS (1u << 3)
+#define bit_cpu_FSRM (1u << 4)
+#define bit_cpu_UINTR (1u << 5)
+#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
+#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
+#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
+#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
+#define bit_cpu_MD_CLEAR (1u << 10)
+#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
+#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
+#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
+#define bit_cpu_SERIALIZE (1u << 14)
+#define bit_cpu_HYBRID (1u << 15)
+#define bit_cpu_TSXLDTRK (1u << 16)
+#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
+#define bit_cpu_PCONFIG (1u << 18)
+#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
+#define bit_cpu_IBT (1u << 20)
+#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
+#define bit_cpu_AMX_BF16 (1u << 22)
+#define bit_cpu_AVX512_FP16 (1u << 23)
+#define bit_cpu_AMX_TILE (1u << 24)
+#define bit_cpu_AMX_INT8 (1u << 25)
+#define bit_cpu_IBRS_IBPB (1u << 26)
+#define bit_cpu_STIBP (1u << 27)
+#define bit_cpu_L1D_FLUSH (1u << 28)
+#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
+#define bit_cpu_CORE_CAPABILITIES (1u << 30)
+#define bit_cpu_SSBD (1u << 31)
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define bit_cpu_LAHF64_SAHF64 (1u << 0)
+#define bit_cpu_SVM (1u << 2)
+#define bit_cpu_LZCNT (1u << 5)
+#define bit_cpu_SSE4A (1u << 6)
+#define bit_cpu_PREFETCHW (1u << 8)
+#define bit_cpu_XOP (1u << 11)
+#define bit_cpu_LWP (1u << 15)
+#define bit_cpu_FMA4 (1u << 16)
+#define bit_cpu_TBM (1u << 21)
+
+/* EDX. */
+#define bit_cpu_SYSCALL_SYSRET (1u << 11)
+#define bit_cpu_NX (1u << 20)
+#define bit_cpu_PAGE1GB (1u << 26)
+#define bit_cpu_RDTSCP (1u << 27)
+#define bit_cpu_LM (1u << 29)
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_XSAVEOPT (1u << 0)
+#define bit_cpu_XSAVEC (1u << 1)
+#define bit_cpu_XGETBV_ECX_1 (1u << 2)
+#define bit_cpu_XSAVES (1u << 3)
+#define bit_cpu_XFD (1u << 4)
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define bit_cpu_INVARIANT_TSC (1u << 8)
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define bit_cpu_WBNOINVD (1u << 9)
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_AVX_VNNI (1u << 4)
+#define bit_cpu_AVX512_BF16 (1u << 5)
+#define bit_cpu_FZLRM (1u << 10)
+#define bit_cpu_FSRS (1u << 11)
+#define bit_cpu_FSRCS (1u << 12)
+#define bit_cpu_HRESET (1u << 22)
+#define bit_cpu_LAM (1u << 26)
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define bit_cpu_AESKLE (1u << 0)
+#define bit_cpu_WIDE_KL (1u << 2)
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define index_cpu_SSE3 CPUID_INDEX_1
+#define index_cpu_PCLMULQDQ CPUID_INDEX_1
+#define index_cpu_DTES64 CPUID_INDEX_1
+#define index_cpu_MONITOR CPUID_INDEX_1
+#define index_cpu_DS_CPL CPUID_INDEX_1
+#define index_cpu_VMX CPUID_INDEX_1
+#define index_cpu_SMX CPUID_INDEX_1
+#define index_cpu_EIST CPUID_INDEX_1
+#define index_cpu_TM2 CPUID_INDEX_1
+#define index_cpu_SSSE3 CPUID_INDEX_1
+#define index_cpu_CNXT_ID CPUID_INDEX_1
+#define index_cpu_SDBG CPUID_INDEX_1
+#define index_cpu_FMA CPUID_INDEX_1
+#define index_cpu_CMPXCHG16B CPUID_INDEX_1
+#define index_cpu_XTPRUPDCTRL CPUID_INDEX_1
+#define index_cpu_PDCM CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_16 CPUID_INDEX_1
+#define index_cpu_PCID CPUID_INDEX_1
+#define index_cpu_DCA CPUID_INDEX_1
+#define index_cpu_SSE4_1 CPUID_INDEX_1
+#define index_cpu_SSE4_2 CPUID_INDEX_1
+#define index_cpu_X2APIC CPUID_INDEX_1
+#define index_cpu_MOVBE CPUID_INDEX_1
+#define index_cpu_POPCNT CPUID_INDEX_1
+#define index_cpu_TSC_DEADLINE CPUID_INDEX_1
+#define index_cpu_AES CPUID_INDEX_1
+#define index_cpu_XSAVE CPUID_INDEX_1
+#define index_cpu_OSXSAVE CPUID_INDEX_1
+#define index_cpu_AVX CPUID_INDEX_1
+#define index_cpu_F16C CPUID_INDEX_1
+#define index_cpu_RDRAND CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_31 CPUID_INDEX_1
+
+/* ECX. */
+#define index_cpu_FPU CPUID_INDEX_1
+#define index_cpu_VME CPUID_INDEX_1
+#define index_cpu_DE CPUID_INDEX_1
+#define index_cpu_PSE CPUID_INDEX_1
+#define index_cpu_TSC CPUID_INDEX_1
+#define index_cpu_MSR CPUID_INDEX_1
+#define index_cpu_PAE CPUID_INDEX_1
+#define index_cpu_MCE CPUID_INDEX_1
+#define index_cpu_CX8 CPUID_INDEX_1
+#define index_cpu_APIC CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_10 CPUID_INDEX_1
+#define index_cpu_SEP CPUID_INDEX_1
+#define index_cpu_MTRR CPUID_INDEX_1
+#define index_cpu_PGE CPUID_INDEX_1
+#define index_cpu_MCA CPUID_INDEX_1
+#define index_cpu_CMOV CPUID_INDEX_1
+#define index_cpu_PAT CPUID_INDEX_1
+#define index_cpu_PSE_36 CPUID_INDEX_1
+#define index_cpu_PSN CPUID_INDEX_1
+#define index_cpu_CLFSH CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_20 CPUID_INDEX_1
+#define index_cpu_DS CPUID_INDEX_1
+#define index_cpu_ACPI CPUID_INDEX_1
+#define index_cpu_MMX CPUID_INDEX_1
+#define index_cpu_FXSR CPUID_INDEX_1
+#define index_cpu_SSE CPUID_INDEX_1
+#define index_cpu_SSE2 CPUID_INDEX_1
+#define index_cpu_SS CPUID_INDEX_1
+#define index_cpu_HTT CPUID_INDEX_1
+#define index_cpu_TM CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_30 CPUID_INDEX_1
+#define index_cpu_PBE CPUID_INDEX_1
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define index_cpu_FSGSBASE CPUID_INDEX_7
+#define index_cpu_TSC_ADJUST CPUID_INDEX_7
+#define index_cpu_SGX CPUID_INDEX_7
+#define index_cpu_BMI1 CPUID_INDEX_7
+#define index_cpu_HLE CPUID_INDEX_7
+#define index_cpu_AVX2 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_6 CPUID_INDEX_7
+#define index_cpu_SMEP CPUID_INDEX_7
+#define index_cpu_BMI2 CPUID_INDEX_7
+#define index_cpu_ERMS CPUID_INDEX_7
+#define index_cpu_INVPCID CPUID_INDEX_7
+#define index_cpu_RTM CPUID_INDEX_7
+#define index_cpu_RDT_M CPUID_INDEX_7
+#define index_cpu_DEPR_FPU_CS_DS CPUID_INDEX_7
+#define index_cpu_MPX CPUID_INDEX_7
+#define index_cpu_RDT_A CPUID_INDEX_7
+#define index_cpu_AVX512F CPUID_INDEX_7
+#define index_cpu_AVX512DQ CPUID_INDEX_7
+#define index_cpu_RDSEED CPUID_INDEX_7
+#define index_cpu_ADX CPUID_INDEX_7
+#define index_cpu_SMAP CPUID_INDEX_7
+#define index_cpu_AVX512_IFMA CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_22 CPUID_INDEX_7
+#define index_cpu_CLFLUSHOPT CPUID_INDEX_7
+#define index_cpu_CLWB CPUID_INDEX_7
+#define index_cpu_TRACE CPUID_INDEX_7
+#define index_cpu_AVX512PF CPUID_INDEX_7
+#define index_cpu_AVX512ER CPUID_INDEX_7
+#define index_cpu_AVX512CD CPUID_INDEX_7
+#define index_cpu_SHA CPUID_INDEX_7
+#define index_cpu_AVX512BW CPUID_INDEX_7
+#define index_cpu_AVX512VL CPUID_INDEX_7
+
+/* ECX. */
+#define index_cpu_PREFETCHWT1 CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI CPUID_INDEX_7
+#define index_cpu_UMIP CPUID_INDEX_7
+#define index_cpu_PKU CPUID_INDEX_7
+#define index_cpu_OSPKE CPUID_INDEX_7
+#define index_cpu_WAITPKG CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI2 CPUID_INDEX_7
+#define index_cpu_SHSTK CPUID_INDEX_7
+#define index_cpu_GFNI CPUID_INDEX_7
+#define index_cpu_VAES CPUID_INDEX_7
+#define index_cpu_VPCLMULQDQ CPUID_INDEX_7
+#define index_cpu_AVX512_VNNI CPUID_INDEX_7
+#define index_cpu_AVX512_BITALG CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_13 CPUID_INDEX_7
+#define index_cpu_AVX512_VPOPCNTDQ CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_15 CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_16 CPUID_INDEX_7
+#define index_cpu_RDPID CPUID_INDEX_7
+#define index_cpu_KL CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_24 CPUID_INDEX_7
+#define index_cpu_CLDEMOTE CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_26 CPUID_INDEX_7
+#define index_cpu_MOVDIRI CPUID_INDEX_7
+#define index_cpu_MOVDIR64B CPUID_INDEX_7
+#define index_cpu_ENQCMD CPUID_INDEX_7
+#define index_cpu_SGX_LC CPUID_INDEX_7
+#define index_cpu_PKS CPUID_INDEX_7
+
+/* EDX. */
+#define index_cpu_INDEX_7_EDX_0 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_1 CPUID_INDEX_7
+#define index_cpu_AVX512_4VNNIW CPUID_INDEX_7
+#define index_cpu_AVX512_4FMAPS CPUID_INDEX_7
+#define index_cpu_FSRM CPUID_INDEX_7
+#define index_cpu_UINTR CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_6 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_7 CPUID_INDEX_7
+#define index_cpu_AVX512_VP2INTERSECT CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_9 CPUID_INDEX_7
+#define index_cpu_MD_CLEAR CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_11 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_12 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_13 CPUID_INDEX_7
+#define index_cpu_SERIALIZE CPUID_INDEX_7
+#define index_cpu_HYBRID CPUID_INDEX_7
+#define index_cpu_TSXLDTRK CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_17 CPUID_INDEX_7
+#define index_cpu_PCONFIG CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_19 CPUID_INDEX_7
+#define index_cpu_IBT CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_21 CPUID_INDEX_7
+#define index_cpu_AMX_BF16 CPUID_INDEX_7
+#define index_cpu_AVX512_FP16 CPUID_INDEX_7
+#define index_cpu_AMX_TILE CPUID_INDEX_7
+#define index_cpu_AMX_INT8 CPUID_INDEX_7
+#define index_cpu_IBRS_IBPB CPUID_INDEX_7
+#define index_cpu_STIBP CPUID_INDEX_7
+#define index_cpu_L1D_FLUSH CPUID_INDEX_7
+#define index_cpu_ARCH_CAPABILITIES CPUID_INDEX_7
+#define index_cpu_CORE_CAPABILITIES CPUID_INDEX_7
+#define index_cpu_SSBD CPUID_INDEX_7
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define index_cpu_LAHF64_SAHF64 CPUID_INDEX_80000001
+#define index_cpu_SVM CPUID_INDEX_80000001
+#define index_cpu_LZCNT CPUID_INDEX_80000001
+#define index_cpu_SSE4A CPUID_INDEX_80000001
+#define index_cpu_PREFETCHW CPUID_INDEX_80000001
+#define index_cpu_XOP CPUID_INDEX_80000001
+#define index_cpu_LWP CPUID_INDEX_80000001
+#define index_cpu_FMA4 CPUID_INDEX_80000001
+#define index_cpu_TBM CPUID_INDEX_80000001
+
+/* EDX. */
+#define index_cpu_SYSCALL_SYSRET CPUID_INDEX_80000001
+#define index_cpu_NX CPUID_INDEX_80000001
+#define index_cpu_PAGE1GB CPUID_INDEX_80000001
+#define index_cpu_RDTSCP CPUID_INDEX_80000001
+#define index_cpu_LM CPUID_INDEX_80000001
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define index_cpu_XSAVEOPT CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVEC CPUID_INDEX_D_ECX_1
+#define index_cpu_XGETBV_ECX_1 CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVES CPUID_INDEX_D_ECX_1
+#define index_cpu_XFD CPUID_INDEX_D_ECX_1
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define index_cpu_INVARIANT_TSC CPUID_INDEX_80000007
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define index_cpu_WBNOINVD CPUID_INDEX_80000008
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define index_cpu_AVX_VNNI CPUID_INDEX_7_ECX_1
+#define index_cpu_AVX512_BF16 CPUID_INDEX_7_ECX_1
+#define index_cpu_FZLRM CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRS CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRCS CPUID_INDEX_7_ECX_1
+#define index_cpu_HRESET CPUID_INDEX_7_ECX_1
+#define index_cpu_LAM CPUID_INDEX_7_ECX_1
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define index_cpu_AESKLE CPUID_INDEX_19
+#define index_cpu_WIDE_KL CPUID_INDEX_19
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define reg_SSE3 ecx
+#define reg_PCLMULQDQ ecx
+#define reg_DTES64 ecx
+#define reg_MONITOR ecx
+#define reg_DS_CPL ecx
+#define reg_VMX ecx
+#define reg_SMX ecx
+#define reg_EIST ecx
+#define reg_TM2 ecx
+#define reg_SSSE3 ecx
+#define reg_CNXT_ID ecx
+#define reg_SDBG ecx
+#define reg_FMA ecx
+#define reg_CMPXCHG16B ecx
+#define reg_XTPRUPDCTRL ecx
+#define reg_PDCM ecx
+#define reg_INDEX_1_ECX_16 ecx
+#define reg_PCID ecx
+#define reg_DCA ecx
+#define reg_SSE4_1 ecx
+#define reg_SSE4_2 ecx
+#define reg_X2APIC ecx
+#define reg_MOVBE ecx
+#define reg_POPCNT ecx
+#define reg_TSC_DEADLINE ecx
+#define reg_AES ecx
+#define reg_XSAVE ecx
+#define reg_OSXSAVE ecx
+#define reg_AVX ecx
+#define reg_F16C ecx
+#define reg_RDRAND ecx
+#define reg_INDEX_1_ECX_31 ecx
+
+/* EDX. */
+#define reg_FPU edx
+#define reg_VME edx
+#define reg_DE edx
+#define reg_PSE edx
+#define reg_TSC edx
+#define reg_MSR edx
+#define reg_PAE edx
+#define reg_MCE edx
+#define reg_CX8 edx
+#define reg_APIC edx
+#define reg_INDEX_1_EDX_10 edx
+#define reg_SEP edx
+#define reg_MTRR edx
+#define reg_PGE edx
+#define reg_MCA edx
+#define reg_CMOV edx
+#define reg_PAT edx
+#define reg_PSE_36 edx
+#define reg_PSN edx
+#define reg_CLFSH edx
+#define reg_INDEX_1_EDX_20 edx
+#define reg_DS edx
+#define reg_ACPI edx
+#define reg_MMX edx
+#define reg_FXSR edx
+#define reg_SSE edx
+#define reg_SSE2 edx
+#define reg_SS edx
+#define reg_HTT edx
+#define reg_TM edx
+#define reg_INDEX_1_EDX_30 edx
+#define reg_PBE edx
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define reg_FSGSBASE ebx
+#define reg_TSC_ADJUST ebx
+#define reg_SGX ebx
+#define reg_BMI1 ebx
+#define reg_HLE ebx
+#define reg_BMI2 ebx
+#define reg_AVX2 ebx
+#define reg_INDEX_7_EBX_6 ebx
+#define reg_SMEP ebx
+#define reg_ERMS ebx
+#define reg_INVPCID ebx
+#define reg_RTM ebx
+#define reg_RDT_M ebx
+#define reg_DEPR_FPU_CS_DS ebx
+#define reg_MPX ebx
+#define reg_RDT_A ebx
+#define reg_AVX512F ebx
+#define reg_AVX512DQ ebx
+#define reg_RDSEED ebx
+#define reg_ADX ebx
+#define reg_SMAP ebx
+#define reg_AVX512_IFMA ebx
+#define reg_INDEX_7_EBX_22 ebx
+#define reg_CLFLUSHOPT ebx
+#define reg_CLWB ebx
+#define reg_TRACE ebx
+#define reg_AVX512PF ebx
+#define reg_AVX512ER ebx
+#define reg_AVX512CD ebx
+#define reg_SHA ebx
+#define reg_AVX512BW ebx
+#define reg_AVX512VL ebx
+
+/* ECX. */
+#define reg_PREFETCHWT1 ecx
+#define reg_AVX512_VBMI ecx
+#define reg_UMIP ecx
+#define reg_PKU ecx
+#define reg_OSPKE ecx
+#define reg_WAITPKG ecx
+#define reg_AVX512_VBMI2 ecx
+#define reg_SHSTK ecx
+#define reg_GFNI ecx
+#define reg_VAES ecx
+#define reg_VPCLMULQDQ ecx
+#define reg_AVX512_VNNI ecx
+#define reg_AVX512_BITALG ecx
+#define reg_INDEX_7_ECX_13 ecx
+#define reg_AVX512_VPOPCNTDQ ecx
+#define reg_INDEX_7_ECX_15 ecx
+#define reg_INDEX_7_ECX_16 ecx
+#define reg_RDPID ecx
+#define reg_KL ecx
+#define reg_INDEX_7_ECX_24 ecx
+#define reg_CLDEMOTE ecx
+#define reg_INDEX_7_ECX_26 ecx
+#define reg_MOVDIRI ecx
+#define reg_MOVDIR64B ecx
+#define reg_ENQCMD ecx
+#define reg_SGX_LC ecx
+#define reg_PKS ecx
+
+/* EDX. */
+#define reg_INDEX_7_EDX_0 edx
+#define reg_INDEX_7_EDX_1 edx
+#define reg_AVX512_4VNNIW edx
+#define reg_AVX512_4FMAPS edx
+#define reg_FSRM edx
+#define reg_UINTR edx
+#define reg_INDEX_7_EDX_6 edx
+#define reg_INDEX_7_EDX_7 edx
+#define reg_AVX512_VP2INTERSECT edx
+#define reg_INDEX_7_EDX_9 edx
+#define reg_MD_CLEAR edx
+#define reg_INDEX_7_EDX_11 edx
+#define reg_INDEX_7_EDX_12 edx
+#define reg_INDEX_7_EDX_13 edx
+#define reg_SERIALIZE edx
+#define reg_HYBRID edx
+#define reg_TSXLDTRK edx
+#define reg_INDEX_7_EDX_17 edx
+#define reg_PCONFIG edx
+#define reg_INDEX_7_EDX_19 edx
+#define reg_IBT edx
+#define reg_INDEX_7_EDX_21 edx
+#define reg_AMX_BF16 edx
+#define reg_AVX512_FP16 edx
+#define reg_AMX_TILE edx
+#define reg_AMX_INT8 edx
+#define reg_IBRS_IBPB edx
+#define reg_STIBP edx
+#define reg_L1D_FLUSH edx
+#define reg_ARCH_CAPABILITIES edx
+#define reg_CORE_CAPABILITIES edx
+#define reg_SSBD edx
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define reg_LAHF64_SAHF64 ecx
+#define reg_SVM ecx
+#define reg_LZCNT ecx
+#define reg_SSE4A ecx
+#define reg_PREFETCHW ecx
+#define reg_XOP ecx
+#define reg_LWP ecx
+#define reg_FMA4 ecx
+#define reg_TBM ecx
+
+/* EDX. */
+#define reg_SYSCALL_SYSRET edx
+#define reg_NX edx
+#define reg_PAGE1GB edx
+#define reg_RDTSCP edx
+#define reg_LM edx
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define reg_XSAVEOPT eax
+#define reg_XSAVEC eax
+#define reg_XGETBV_ECX_1 eax
+#define reg_XSAVES eax
+#define reg_XFD eax
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define reg_INVARIANT_TSC edx
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define reg_WBNOINVD ebx
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define reg_AVX_VNNI eax
+#define reg_AVX512_BF16 eax
+#define reg_FZLRM eax
+#define reg_FSRS eax
+#define reg_FSRCS eax
+#define reg_HRESET eax
+#define reg_LAM eax
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define reg_AESKLE ebx
+#define reg_WIDE_KL ebx
+
/* PREFERRED_FEATURE_INDEX_1. */
#define bit_arch_I586 (1u << 0)
#define bit_arch_I686 (1u << 1)
@@ -118,10 +801,50 @@ enum
#define bit_XTILECFG_state (1u << 17)
#define bit_XTILEDATA_state (1u << 18)
+enum cpu_features_kind
+{
+ arch_kind_unknown = 0,
+ arch_kind_intel,
+ arch_kind_amd,
+ arch_kind_zhaoxin,
+ arch_kind_other
+};
+
+struct cpu_features_basic
+{
+ enum cpu_features_kind kind;
+ int max_cpuid;
+ unsigned int family;
+ unsigned int model;
+ unsigned int stepping;
+};
+
+struct cpuid_registers
+{
+ unsigned int eax;
+ unsigned int ebx;
+ unsigned int ecx;
+ unsigned int edx;
+};
+
+struct cpuid_feature_internal
+{
+ union
+ {
+ unsigned int cpuid_array[4];
+ struct cpuid_registers cpuid;
+ };
+ union
+ {
+ unsigned int usable_array[4];
+ struct cpuid_registers usable;
+ };
+};
+
struct cpu_features
{
struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
+ struct cpuid_feature_internal features[CPUID_INDEX_MAX];
unsigned int preferred[PREFERRED_FEATURE_INDEX_MAX];
/* X86 micro-architecture ISA levels. */
unsigned int isa_1;
@@ -173,10 +896,16 @@ struct cpu_features
unsigned long int level4_cache_size;
};
+/* Get a pointer to the CPU features structure. */
+extern const struct cpu_features *_dl_x86_get_cpu_features (void)
+ __attribute__ ((pure));
+
+#define __get_cpu_features() _dl_x86_get_cpu_features()
+
#if defined (_LIBC) && !IS_IN (nonlib)
/* Unused for x86. */
# define INIT_ARCH()
-# define __x86_get_cpu_features(max) (&GLRO(dl_x86_cpu_features))
+# define _dl_x86_get_cpu_features() (&GLRO(dl_x86_cpu_features))
extern void _dl_x86_init_cpu_features (void) attribute_hidden;
#endif
diff --git a/sysdeps/x86/sys/platform/x86.h b/sysdeps/x86/sys/platform/x86.h
index 7c964ff907..e930219944 100644
--- a/sysdeps/x86/sys/platform/x86.h
+++ b/sysdeps/x86/sys/platform/x86.h
@@ -19,760 +19,47 @@
#ifndef _SYS_PLATFORM_X86_H
#define _SYS_PLATFORM_X86_H
-enum
-{
- COMMON_CPUID_INDEX_1 = 0,
- COMMON_CPUID_INDEX_7,
- COMMON_CPUID_INDEX_80000001,
- COMMON_CPUID_INDEX_D_ECX_1,
- COMMON_CPUID_INDEX_80000007,
- COMMON_CPUID_INDEX_80000008,
- COMMON_CPUID_INDEX_7_ECX_1,
- COMMON_CPUID_INDEX_19,
- /* Keep the following line at the end. */
- COMMON_CPUID_INDEX_MAX
-};
+#include <features.h>
+#include <stdbool.h>
+#include <bits/platform/x86.h>
-struct cpuid_registers
-{
- unsigned int eax;
- unsigned int ebx;
- unsigned int ecx;
- unsigned int edx;
-};
+__BEGIN_DECLS
-struct cpuid_features
-{
- struct cpuid_registers cpuid;
- struct cpuid_registers usable;
-};
+/* Get a pointer to the CPU feature structure. */
+extern const struct cpuid_feature *__x86_get_cpuid_feature_leaf (unsigned int)
+ __attribute__ ((pure));
-enum cpu_features_kind
+static __inline__ _Bool
+x86_cpu_has_feature (unsigned int __index)
{
- arch_kind_unknown = 0,
- arch_kind_intel,
- arch_kind_amd,
- arch_kind_zhaoxin,
- arch_kind_other
-};
-
-struct cpu_features_basic
+ const struct cpuid_feature *__ptr = __x86_get_cpuid_feature_leaf
+ (__index / (8 * sizeof (unsigned int) * 4));
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
+
+ return __ptr->cpuid_array[__reg] & (1 << __bit);
+}
+
+static __inline__ _Bool
+x86_cpu_is_usable (unsigned int __index)
{
- enum cpu_features_kind kind;
- int max_cpuid;
- unsigned int family;
- unsigned int model;
- unsigned int stepping;
-};
+ const struct cpuid_feature *__ptr = __x86_get_cpuid_feature_leaf
+ (__index / (8 * sizeof (unsigned int) * 4));
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
-struct cpu_features
-{
- struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
-};
-
-/* Get a pointer to the CPU features structure. */
-extern const struct cpu_features *__x86_get_cpu_features (unsigned int)
- __attribute__ ((const));
-
-#define CPU_FEATURE_CHECK_P(ptr, name, check) \
- ((ptr->features[index_cpu_##name].check.reg_##name \
- & bit_cpu_##name) != 0)
-#define CPU_FEATURE_CPU_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, cpuid)
-#define CPU_FEATURE_USABLE_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, usable)
+ return __ptr->usable_array[__reg] & (1 << __bit);
+}
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
-#define HAS_CPU_FEATURE(name) \
- (__extension__ \
- ({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_CPU_P (__ptr, name); }))
+#define HAS_CPU_FEATURE(name) x86_cpu_has_feature (x86_cpu_##name)
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
-#define CPU_FEATURE_USABLE(name) \
- (__extension__ \
- ({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_USABLE_P (__ptr, name); }))
-
-/* CPU features. */
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define bit_cpu_SSE3 (1u << 0)
-#define bit_cpu_PCLMULQDQ (1u << 1)
-#define bit_cpu_DTES64 (1u << 2)
-#define bit_cpu_MONITOR (1u << 3)
-#define bit_cpu_DS_CPL (1u << 4)
-#define bit_cpu_VMX (1u << 5)
-#define bit_cpu_SMX (1u << 6)
-#define bit_cpu_EIST (1u << 7)
-#define bit_cpu_TM2 (1u << 8)
-#define bit_cpu_SSSE3 (1u << 9)
-#define bit_cpu_CNXT_ID (1u << 10)
-#define bit_cpu_SDBG (1u << 11)
-#define bit_cpu_FMA (1u << 12)
-#define bit_cpu_CMPXCHG16B (1u << 13)
-#define bit_cpu_XTPRUPDCTRL (1u << 14)
-#define bit_cpu_PDCM (1u << 15)
-#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
-#define bit_cpu_PCID (1u << 17)
-#define bit_cpu_DCA (1u << 18)
-#define bit_cpu_SSE4_1 (1u << 19)
-#define bit_cpu_SSE4_2 (1u << 20)
-#define bit_cpu_X2APIC (1u << 21)
-#define bit_cpu_MOVBE (1u << 22)
-#define bit_cpu_POPCNT (1u << 23)
-#define bit_cpu_TSC_DEADLINE (1u << 24)
-#define bit_cpu_AES (1u << 25)
-#define bit_cpu_XSAVE (1u << 26)
-#define bit_cpu_OSXSAVE (1u << 27)
-#define bit_cpu_AVX (1u << 28)
-#define bit_cpu_F16C (1u << 29)
-#define bit_cpu_RDRAND (1u << 30)
-#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
-
-/* EDX. */
-#define bit_cpu_FPU (1u << 0)
-#define bit_cpu_VME (1u << 1)
-#define bit_cpu_DE (1u << 2)
-#define bit_cpu_PSE (1u << 3)
-#define bit_cpu_TSC (1u << 4)
-#define bit_cpu_MSR (1u << 5)
-#define bit_cpu_PAE (1u << 6)
-#define bit_cpu_MCE (1u << 7)
-#define bit_cpu_CX8 (1u << 8)
-#define bit_cpu_APIC (1u << 9)
-#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
-#define bit_cpu_SEP (1u << 11)
-#define bit_cpu_MTRR (1u << 12)
-#define bit_cpu_PGE (1u << 13)
-#define bit_cpu_MCA (1u << 14)
-#define bit_cpu_CMOV (1u << 15)
-#define bit_cpu_PAT (1u << 16)
-#define bit_cpu_PSE_36 (1u << 17)
-#define bit_cpu_PSN (1u << 18)
-#define bit_cpu_CLFSH (1u << 19)
-#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
-#define bit_cpu_DS (1u << 21)
-#define bit_cpu_ACPI (1u << 22)
-#define bit_cpu_MMX (1u << 23)
-#define bit_cpu_FXSR (1u << 24)
-#define bit_cpu_SSE (1u << 25)
-#define bit_cpu_SSE2 (1u << 26)
-#define bit_cpu_SS (1u << 27)
-#define bit_cpu_HTT (1u << 28)
-#define bit_cpu_TM (1u << 29)
-#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
-#define bit_cpu_PBE (1u << 31)
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define bit_cpu_FSGSBASE (1u << 0)
-#define bit_cpu_TSC_ADJUST (1u << 1)
-#define bit_cpu_SGX (1u << 2)
-#define bit_cpu_BMI1 (1u << 3)
-#define bit_cpu_HLE (1u << 4)
-#define bit_cpu_AVX2 (1u << 5)
-#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
-#define bit_cpu_SMEP (1u << 7)
-#define bit_cpu_BMI2 (1u << 8)
-#define bit_cpu_ERMS (1u << 9)
-#define bit_cpu_INVPCID (1u << 10)
-#define bit_cpu_RTM (1u << 11)
-#define bit_cpu_RDT_M (1u << 12)
-#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
-#define bit_cpu_MPX (1u << 14)
-#define bit_cpu_RDT_A (1u << 15)
-#define bit_cpu_AVX512F (1u << 16)
-#define bit_cpu_AVX512DQ (1u << 17)
-#define bit_cpu_RDSEED (1u << 18)
-#define bit_cpu_ADX (1u << 19)
-#define bit_cpu_SMAP (1u << 20)
-#define bit_cpu_AVX512_IFMA (1u << 21)
-#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
-#define bit_cpu_CLFLUSHOPT (1u << 23)
-#define bit_cpu_CLWB (1u << 24)
-#define bit_cpu_TRACE (1u << 25)
-#define bit_cpu_AVX512PF (1u << 26)
-#define bit_cpu_AVX512ER (1u << 27)
-#define bit_cpu_AVX512CD (1u << 28)
-#define bit_cpu_SHA (1u << 29)
-#define bit_cpu_AVX512BW (1u << 30)
-#define bit_cpu_AVX512VL (1u << 31)
-
-/* ECX. */
-#define bit_cpu_PREFETCHWT1 (1u << 0)
-#define bit_cpu_AVX512_VBMI (1u << 1)
-#define bit_cpu_UMIP (1u << 2)
-#define bit_cpu_PKU (1u << 3)
-#define bit_cpu_OSPKE (1u << 4)
-#define bit_cpu_WAITPKG (1u << 5)
-#define bit_cpu_AVX512_VBMI2 (1u << 6)
-#define bit_cpu_SHSTK (1u << 7)
-#define bit_cpu_GFNI (1u << 8)
-#define bit_cpu_VAES (1u << 9)
-#define bit_cpu_VPCLMULQDQ (1u << 10)
-#define bit_cpu_AVX512_VNNI (1u << 11)
-#define bit_cpu_AVX512_BITALG (1u << 12)
-#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
-#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
-#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
-#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
-/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
- instructions in 64-bit mode. */
-#define bit_cpu_RDPID (1u << 22)
-#define bit_cpu_KL (1u << 23)
-#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
-#define bit_cpu_CLDEMOTE (1u << 25)
-#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
-#define bit_cpu_MOVDIRI (1u << 27)
-#define bit_cpu_MOVDIR64B (1u << 28)
-#define bit_cpu_ENQCMD (1u << 29)
-#define bit_cpu_SGX_LC (1u << 30)
-#define bit_cpu_PKS (1u << 31)
-
-/* EDX. */
-#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
-#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
-#define bit_cpu_AVX512_4VNNIW (1u << 2)
-#define bit_cpu_AVX512_4FMAPS (1u << 3)
-#define bit_cpu_FSRM (1u << 4)
-#define bit_cpu_UINTR (1u << 5)
-#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
-#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
-#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
-#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
-#define bit_cpu_MD_CLEAR (1u << 10)
-#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
-#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
-#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
-#define bit_cpu_SERIALIZE (1u << 14)
-#define bit_cpu_HYBRID (1u << 15)
-#define bit_cpu_TSXLDTRK (1u << 16)
-#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
-#define bit_cpu_PCONFIG (1u << 18)
-#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
-#define bit_cpu_IBT (1u << 20)
-#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
-#define bit_cpu_AMX_BF16 (1u << 22)
-#define bit_cpu_AVX512_FP16 (1u << 23)
-#define bit_cpu_AMX_TILE (1u << 24)
-#define bit_cpu_AMX_INT8 (1u << 25)
-#define bit_cpu_IBRS_IBPB (1u << 26)
-#define bit_cpu_STIBP (1u << 27)
-#define bit_cpu_L1D_FLUSH (1u << 28)
-#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
-#define bit_cpu_CORE_CAPABILITIES (1u << 30)
-#define bit_cpu_SSBD (1u << 31)
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define bit_cpu_LAHF64_SAHF64 (1u << 0)
-#define bit_cpu_SVM (1u << 2)
-#define bit_cpu_LZCNT (1u << 5)
-#define bit_cpu_SSE4A (1u << 6)
-#define bit_cpu_PREFETCHW (1u << 8)
-#define bit_cpu_XOP (1u << 11)
-#define bit_cpu_LWP (1u << 15)
-#define bit_cpu_FMA4 (1u << 16)
-#define bit_cpu_TBM (1u << 21)
-
-/* EDX. */
-#define bit_cpu_SYSCALL_SYSRET (1u << 11)
-#define bit_cpu_NX (1u << 20)
-#define bit_cpu_PAGE1GB (1u << 26)
-#define bit_cpu_RDTSCP (1u << 27)
-#define bit_cpu_LM (1u << 29)
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_XSAVEOPT (1u << 0)
-#define bit_cpu_XSAVEC (1u << 1)
-#define bit_cpu_XGETBV_ECX_1 (1u << 2)
-#define bit_cpu_XSAVES (1u << 3)
-#define bit_cpu_XFD (1u << 4)
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define bit_cpu_INVARIANT_TSC (1u << 8)
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define bit_cpu_WBNOINVD (1u << 9)
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_AVX_VNNI (1u << 4)
-#define bit_cpu_AVX512_BF16 (1u << 5)
-#define bit_cpu_FZLRM (1u << 10)
-#define bit_cpu_FSRS (1u << 11)
-#define bit_cpu_FSRCS (1u << 12)
-#define bit_cpu_HRESET (1u << 22)
-#define bit_cpu_LAM (1u << 26)
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define bit_cpu_AESKLE (1u << 0)
-#define bit_cpu_WIDE_KL (1u << 2)
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define index_cpu_SSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_PCLMULQDQ COMMON_CPUID_INDEX_1
-#define index_cpu_DTES64 COMMON_CPUID_INDEX_1
-#define index_cpu_MONITOR COMMON_CPUID_INDEX_1
-#define index_cpu_DS_CPL COMMON_CPUID_INDEX_1
-#define index_cpu_VMX COMMON_CPUID_INDEX_1
-#define index_cpu_SMX COMMON_CPUID_INDEX_1
-#define index_cpu_EIST COMMON_CPUID_INDEX_1
-#define index_cpu_TM2 COMMON_CPUID_INDEX_1
-#define index_cpu_SSSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_CNXT_ID COMMON_CPUID_INDEX_1
-#define index_cpu_SDBG COMMON_CPUID_INDEX_1
-#define index_cpu_FMA COMMON_CPUID_INDEX_1
-#define index_cpu_CMPXCHG16B COMMON_CPUID_INDEX_1
-#define index_cpu_XTPRUPDCTRL COMMON_CPUID_INDEX_1
-#define index_cpu_PDCM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_16 COMMON_CPUID_INDEX_1
-#define index_cpu_PCID COMMON_CPUID_INDEX_1
-#define index_cpu_DCA COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_1 COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_2 COMMON_CPUID_INDEX_1
-#define index_cpu_X2APIC COMMON_CPUID_INDEX_1
-#define index_cpu_MOVBE COMMON_CPUID_INDEX_1
-#define index_cpu_POPCNT COMMON_CPUID_INDEX_1
-#define index_cpu_TSC_DEADLINE COMMON_CPUID_INDEX_1
-#define index_cpu_AES COMMON_CPUID_INDEX_1
-#define index_cpu_XSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_OSXSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_AVX COMMON_CPUID_INDEX_1
-#define index_cpu_F16C COMMON_CPUID_INDEX_1
-#define index_cpu_RDRAND COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_31 COMMON_CPUID_INDEX_1
-
-/* ECX. */
-#define index_cpu_FPU COMMON_CPUID_INDEX_1
-#define index_cpu_VME COMMON_CPUID_INDEX_1
-#define index_cpu_DE COMMON_CPUID_INDEX_1
-#define index_cpu_PSE COMMON_CPUID_INDEX_1
-#define index_cpu_TSC COMMON_CPUID_INDEX_1
-#define index_cpu_MSR COMMON_CPUID_INDEX_1
-#define index_cpu_PAE COMMON_CPUID_INDEX_1
-#define index_cpu_MCE COMMON_CPUID_INDEX_1
-#define index_cpu_CX8 COMMON_CPUID_INDEX_1
-#define index_cpu_APIC COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_10 COMMON_CPUID_INDEX_1
-#define index_cpu_SEP COMMON_CPUID_INDEX_1
-#define index_cpu_MTRR COMMON_CPUID_INDEX_1
-#define index_cpu_PGE COMMON_CPUID_INDEX_1
-#define index_cpu_MCA COMMON_CPUID_INDEX_1
-#define index_cpu_CMOV COMMON_CPUID_INDEX_1
-#define index_cpu_PAT COMMON_CPUID_INDEX_1
-#define index_cpu_PSE_36 COMMON_CPUID_INDEX_1
-#define index_cpu_PSN COMMON_CPUID_INDEX_1
-#define index_cpu_CLFSH COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_20 COMMON_CPUID_INDEX_1
-#define index_cpu_DS COMMON_CPUID_INDEX_1
-#define index_cpu_ACPI COMMON_CPUID_INDEX_1
-#define index_cpu_MMX COMMON_CPUID_INDEX_1
-#define index_cpu_FXSR COMMON_CPUID_INDEX_1
-#define index_cpu_SSE COMMON_CPUID_INDEX_1
-#define index_cpu_SSE2 COMMON_CPUID_INDEX_1
-#define index_cpu_SS COMMON_CPUID_INDEX_1
-#define index_cpu_HTT COMMON_CPUID_INDEX_1
-#define index_cpu_TM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_30 COMMON_CPUID_INDEX_1
-#define index_cpu_PBE COMMON_CPUID_INDEX_1
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define index_cpu_FSGSBASE COMMON_CPUID_INDEX_7
-#define index_cpu_TSC_ADJUST COMMON_CPUID_INDEX_7
-#define index_cpu_SGX COMMON_CPUID_INDEX_7
-#define index_cpu_BMI1 COMMON_CPUID_INDEX_7
-#define index_cpu_HLE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX2 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_SMEP COMMON_CPUID_INDEX_7
-#define index_cpu_BMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_ERMS COMMON_CPUID_INDEX_7
-#define index_cpu_INVPCID COMMON_CPUID_INDEX_7
-#define index_cpu_RTM COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_M COMMON_CPUID_INDEX_7
-#define index_cpu_DEPR_FPU_CS_DS COMMON_CPUID_INDEX_7
-#define index_cpu_MPX COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_A COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512F COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512DQ COMMON_CPUID_INDEX_7
-#define index_cpu_RDSEED COMMON_CPUID_INDEX_7
-#define index_cpu_ADX COMMON_CPUID_INDEX_7
-#define index_cpu_SMAP COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_IFMA COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_22 COMMON_CPUID_INDEX_7
-#define index_cpu_CLFLUSHOPT COMMON_CPUID_INDEX_7
-#define index_cpu_CLWB COMMON_CPUID_INDEX_7
-#define index_cpu_TRACE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512PF COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512ER COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512CD COMMON_CPUID_INDEX_7
-#define index_cpu_SHA COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512BW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512VL COMMON_CPUID_INDEX_7
-
-/* ECX. */
-#define index_cpu_PREFETCHWT1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI COMMON_CPUID_INDEX_7
-#define index_cpu_UMIP COMMON_CPUID_INDEX_7
-#define index_cpu_PKU COMMON_CPUID_INDEX_7
-#define index_cpu_OSPKE COMMON_CPUID_INDEX_7
-#define index_cpu_WAITPKG COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_SHSTK COMMON_CPUID_INDEX_7
-#define index_cpu_GFNI COMMON_CPUID_INDEX_7
-#define index_cpu_VAES COMMON_CPUID_INDEX_7
-#define index_cpu_VPCLMULQDQ COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VNNI COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_BITALG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VPOPCNTDQ COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_15 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_16 COMMON_CPUID_INDEX_7
-#define index_cpu_RDPID COMMON_CPUID_INDEX_7
-#define index_cpu_KL COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_24 COMMON_CPUID_INDEX_7
-#define index_cpu_CLDEMOTE COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_26 COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIRI COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIR64B COMMON_CPUID_INDEX_7
-#define index_cpu_ENQCMD COMMON_CPUID_INDEX_7
-#define index_cpu_SGX_LC COMMON_CPUID_INDEX_7
-#define index_cpu_PKS COMMON_CPUID_INDEX_7
-
-/* EDX. */
-#define index_cpu_INDEX_7_EDX_0 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4VNNIW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4FMAPS COMMON_CPUID_INDEX_7
-#define index_cpu_FSRM COMMON_CPUID_INDEX_7
-#define index_cpu_UINTR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_7 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VP2INTERSECT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_9 COMMON_CPUID_INDEX_7
-#define index_cpu_MD_CLEAR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_11 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_12 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_SERIALIZE COMMON_CPUID_INDEX_7
-#define index_cpu_HYBRID COMMON_CPUID_INDEX_7
-#define index_cpu_TSXLDTRK COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_17 COMMON_CPUID_INDEX_7
-#define index_cpu_PCONFIG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_19 COMMON_CPUID_INDEX_7
-#define index_cpu_IBT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_21 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_BF16 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_FP16 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_TILE COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_INT8 COMMON_CPUID_INDEX_7
-#define index_cpu_IBRS_IBPB COMMON_CPUID_INDEX_7
-#define index_cpu_STIBP COMMON_CPUID_INDEX_7
-#define index_cpu_L1D_FLUSH COMMON_CPUID_INDEX_7
-#define index_cpu_ARCH_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_CORE_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_SSBD COMMON_CPUID_INDEX_7
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define index_cpu_LAHF64_SAHF64 COMMON_CPUID_INDEX_80000001
-#define index_cpu_SVM COMMON_CPUID_INDEX_80000001
-#define index_cpu_LZCNT COMMON_CPUID_INDEX_80000001
-#define index_cpu_SSE4A COMMON_CPUID_INDEX_80000001
-#define index_cpu_PREFETCHW COMMON_CPUID_INDEX_80000001
-#define index_cpu_XOP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LWP COMMON_CPUID_INDEX_80000001
-#define index_cpu_FMA4 COMMON_CPUID_INDEX_80000001
-#define index_cpu_TBM COMMON_CPUID_INDEX_80000001
-
-/* EDX. */
-#define index_cpu_SYSCALL_SYSRET COMMON_CPUID_INDEX_80000001
-#define index_cpu_NX COMMON_CPUID_INDEX_80000001
-#define index_cpu_PAGE1GB COMMON_CPUID_INDEX_80000001
-#define index_cpu_RDTSCP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LM COMMON_CPUID_INDEX_80000001
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define index_cpu_XSAVEOPT COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVEC COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XGETBV_ECX_1 COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVES COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XFD COMMON_CPUID_INDEX_D_ECX_1
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define index_cpu_INVARIANT_TSC COMMON_CPUID_INDEX_80000007
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define index_cpu_WBNOINVD COMMON_CPUID_INDEX_80000008
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define index_cpu_AVX_VNNI COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_AVX512_BF16 COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FZLRM COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRCS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_HRESET COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_LAM COMMON_CPUID_INDEX_7_ECX_1
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define index_cpu_AESKLE COMMON_CPUID_INDEX_19
-#define index_cpu_WIDE_KL COMMON_CPUID_INDEX_19
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define reg_SSE3 ecx
-#define reg_PCLMULQDQ ecx
-#define reg_DTES64 ecx
-#define reg_MONITOR ecx
-#define reg_DS_CPL ecx
-#define reg_VMX ecx
-#define reg_SMX ecx
-#define reg_EIST ecx
-#define reg_TM2 ecx
-#define reg_SSSE3 ecx
-#define reg_CNXT_ID ecx
-#define reg_SDBG ecx
-#define reg_FMA ecx
-#define reg_CMPXCHG16B ecx
-#define reg_XTPRUPDCTRL ecx
-#define reg_PDCM ecx
-#define reg_INDEX_1_ECX_16 ecx
-#define reg_PCID ecx
-#define reg_DCA ecx
-#define reg_SSE4_1 ecx
-#define reg_SSE4_2 ecx
-#define reg_X2APIC ecx
-#define reg_MOVBE ecx
-#define reg_POPCNT ecx
-#define reg_TSC_DEADLINE ecx
-#define reg_AES ecx
-#define reg_XSAVE ecx
-#define reg_OSXSAVE ecx
-#define reg_AVX ecx
-#define reg_F16C ecx
-#define reg_RDRAND ecx
-#define reg_INDEX_1_ECX_31 ecx
-
-/* EDX. */
-#define reg_FPU edx
-#define reg_VME edx
-#define reg_DE edx
-#define reg_PSE edx
-#define reg_TSC edx
-#define reg_MSR edx
-#define reg_PAE edx
-#define reg_MCE edx
-#define reg_CX8 edx
-#define reg_APIC edx
-#define reg_INDEX_1_EDX_10 edx
-#define reg_SEP edx
-#define reg_MTRR edx
-#define reg_PGE edx
-#define reg_MCA edx
-#define reg_CMOV edx
-#define reg_PAT edx
-#define reg_PSE_36 edx
-#define reg_PSN edx
-#define reg_CLFSH edx
-#define reg_INDEX_1_EDX_20 edx
-#define reg_DS edx
-#define reg_ACPI edx
-#define reg_MMX edx
-#define reg_FXSR edx
-#define reg_SSE edx
-#define reg_SSE2 edx
-#define reg_SS edx
-#define reg_HTT edx
-#define reg_TM edx
-#define reg_INDEX_1_EDX_30 edx
-#define reg_PBE edx
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define reg_FSGSBASE ebx
-#define reg_TSC_ADJUST ebx
-#define reg_SGX ebx
-#define reg_BMI1 ebx
-#define reg_HLE ebx
-#define reg_BMI2 ebx
-#define reg_AVX2 ebx
-#define reg_INDEX_7_EBX_6 ebx
-#define reg_SMEP ebx
-#define reg_ERMS ebx
-#define reg_INVPCID ebx
-#define reg_RTM ebx
-#define reg_RDT_M ebx
-#define reg_DEPR_FPU_CS_DS ebx
-#define reg_MPX ebx
-#define reg_RDT_A ebx
-#define reg_AVX512F ebx
-#define reg_AVX512DQ ebx
-#define reg_RDSEED ebx
-#define reg_ADX ebx
-#define reg_SMAP ebx
-#define reg_AVX512_IFMA ebx
-#define reg_INDEX_7_EBX_22 ebx
-#define reg_CLFLUSHOPT ebx
-#define reg_CLWB ebx
-#define reg_TRACE ebx
-#define reg_AVX512PF ebx
-#define reg_AVX512ER ebx
-#define reg_AVX512CD ebx
-#define reg_SHA ebx
-#define reg_AVX512BW ebx
-#define reg_AVX512VL ebx
-
-/* ECX. */
-#define reg_PREFETCHWT1 ecx
-#define reg_AVX512_VBMI ecx
-#define reg_UMIP ecx
-#define reg_PKU ecx
-#define reg_OSPKE ecx
-#define reg_WAITPKG ecx
-#define reg_AVX512_VBMI2 ecx
-#define reg_SHSTK ecx
-#define reg_GFNI ecx
-#define reg_VAES ecx
-#define reg_VPCLMULQDQ ecx
-#define reg_AVX512_VNNI ecx
-#define reg_AVX512_BITALG ecx
-#define reg_INDEX_7_ECX_13 ecx
-#define reg_AVX512_VPOPCNTDQ ecx
-#define reg_INDEX_7_ECX_15 ecx
-#define reg_INDEX_7_ECX_16 ecx
-#define reg_RDPID ecx
-#define reg_KL ecx
-#define reg_INDEX_7_ECX_24 ecx
-#define reg_CLDEMOTE ecx
-#define reg_INDEX_7_ECX_26 ecx
-#define reg_MOVDIRI ecx
-#define reg_MOVDIR64B ecx
-#define reg_ENQCMD ecx
-#define reg_SGX_LC ecx
-#define reg_PKS ecx
-
-/* EDX. */
-#define reg_INDEX_7_EDX_0 edx
-#define reg_INDEX_7_EDX_1 edx
-#define reg_AVX512_4VNNIW edx
-#define reg_AVX512_4FMAPS edx
-#define reg_FSRM edx
-#define reg_UINTR edx
-#define reg_INDEX_7_EDX_6 edx
-#define reg_INDEX_7_EDX_7 edx
-#define reg_AVX512_VP2INTERSECT edx
-#define reg_INDEX_7_EDX_9 edx
-#define reg_MD_CLEAR edx
-#define reg_INDEX_7_EDX_11 edx
-#define reg_INDEX_7_EDX_12 edx
-#define reg_INDEX_7_EDX_13 edx
-#define reg_SERIALIZE edx
-#define reg_HYBRID edx
-#define reg_TSXLDTRK edx
-#define reg_INDEX_7_EDX_17 edx
-#define reg_PCONFIG edx
-#define reg_INDEX_7_EDX_19 edx
-#define reg_IBT edx
-#define reg_INDEX_7_EDX_21 edx
-#define reg_AMX_BF16 edx
-#define reg_AVX512_FP16 edx
-#define reg_AMX_TILE edx
-#define reg_AMX_INT8 edx
-#define reg_IBRS_IBPB edx
-#define reg_STIBP edx
-#define reg_L1D_FLUSH edx
-#define reg_ARCH_CAPABILITIES edx
-#define reg_CORE_CAPABILITIES edx
-#define reg_SSBD edx
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define reg_LAHF64_SAHF64 ecx
-#define reg_SVM ecx
-#define reg_LZCNT ecx
-#define reg_SSE4A ecx
-#define reg_PREFETCHW ecx
-#define reg_XOP ecx
-#define reg_LWP ecx
-#define reg_FMA4 ecx
-#define reg_TBM ecx
-
-/* EDX. */
-#define reg_SYSCALL_SYSRET edx
-#define reg_NX edx
-#define reg_PAGE1GB edx
-#define reg_RDTSCP edx
-#define reg_LM edx
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define reg_XSAVEOPT eax
-#define reg_XSAVEC eax
-#define reg_XGETBV_ECX_1 eax
-#define reg_XSAVES eax
-#define reg_XFD eax
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define reg_INVARIANT_TSC edx
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define reg_WBNOINVD ebx
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define reg_AVX_VNNI eax
-#define reg_AVX512_BF16 eax
-#define reg_FZLRM eax
-#define reg_FSRS eax
-#define reg_FSRCS eax
-#define reg_HRESET eax
-#define reg_LAM eax
-
-/* COMMON_CPUID_INDEX_19. */
+#define CPU_FEATURE_USABLE(name) x86_cpu_is_usable (x86_cpu_##name)
-/* EBX. */
-#define reg_AESKLE ebx
-#define reg_WIDE_KL ebx
+__END_DECLS
#endif /* _SYS_PLATFORM_X86_H */
diff --git a/sysdeps/x86/tst-get-cpu-features.c b/sysdeps/x86/tst-get-cpu-features.c
index 39f287d2a0..dcdb86bb93 100644
--- a/sysdeps/x86/tst-get-cpu-features.c
+++ b/sysdeps/x86/tst-get-cpu-features.c
@@ -1,4 +1,4 @@
-/* Test case for __x86_get_cpu_features interface
+/* Test case for <sys/platform/x86.h> interface
Copyright (C) 2015-2021 Free Software Foundation, Inc.
This file is part of the GNU C Library.
@@ -33,36 +33,9 @@
printf (" " #name "\n"); \
}
-static const char * const cpu_kinds[] =
-{
- "Unknown",
- "Intel",
- "AMD",
- "ZHAOXIN",
- "Other",
-};
-
static int
do_test (void)
{
- const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
-
- switch (cpu_features->basic.kind)
- {
- case arch_kind_intel:
- case arch_kind_amd:
- case arch_kind_zhaoxin:
- case arch_kind_other:
- printf ("Vendor: %s\n", cpu_kinds[cpu_features->basic.kind]);
- printf ("Family: 0x%x\n", cpu_features->basic.family);
- printf ("Model: 0x%x\n", cpu_features->basic.model);
- printf ("Stepping: 0x%x\n", cpu_features->basic.stepping);
- break;
-
- default:
- abort ();
- }
-
#ifdef __SSE2__
TEST_VERIFY_EXIT (HAS_CPU_FEATURE (SSE2));
#endif
diff --git a/sysdeps/x86/tst-isa-level-1.c b/sysdeps/x86/tst-isa-level-1.c
index cf611e8b78..38edabe75d 100644
--- a/sysdeps/x86/tst-isa-level-1.c
+++ b/sysdeps/x86/tst-isa-level-1.c
@@ -60,8 +60,7 @@ do_test_1 (const char *modname, bool fail)
static int
do_test (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
bool has_isa_baseline = ((isa_level & GNU_PROPERTY_X86_ISA_1_BASELINE)
== GNU_PROPERTY_X86_ISA_1_BASELINE);
diff --git a/sysdeps/x86_64/dl-hwcaps-subdirs.c b/sysdeps/x86_64/dl-hwcaps-subdirs.c
index 4acd343183..b8f27d7f98 100644
--- a/sysdeps/x86_64/dl-hwcaps-subdirs.c
+++ b/sysdeps/x86_64/dl-hwcaps-subdirs.c
@@ -27,8 +27,7 @@ enum { subdirs_count = 3 }; /* Number of components in _dl_hwcaps_subdirs. */
uint32_t
_dl_hwcaps_subdirs_active (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
int active = 0;
diff --git a/sysdeps/x86_64/tst-glibc-hwcaps.c b/sysdeps/x86_64/tst-glibc-hwcaps.c
index 37b61d3c84..4c8d24b3f2 100644
--- a/sysdeps/x86_64/tst-glibc-hwcaps.c
+++ b/sysdeps/x86_64/tst-glibc-hwcaps.c
@@ -30,8 +30,7 @@ extern int marker4 (void);
static int
compute_level (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
if (!(isa_level & GNU_PROPERTY_X86_ISA_1_V2))
--
2.29.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: V3 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-21 0:19 ` V3 " H.J. Lu via Libc-alpha
@ 2021-01-21 11:19 ` Florian Weimer via Libc-alpha
2021-01-21 13:29 ` V4 " H.J. Lu via Libc-alpha
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer via Libc-alpha @ 2021-01-21 11:19 UTC (permalink / raw)
To: H.J. Lu; +Cc: H.J. Lu via Libc-alpha
* H. J. Lu:
> Here is the updated patch. The public function is changed to
>
> /* Get a pointer to the CPU feature structure. */
> extern const struct cpuid_feature *__x86_get_cpuid_feature_leaf (unsigned int)
> __attribute__ ((pure));
>
> OK for master?
Are you okay with this version? I like the simplicity of the interface.
Can we move the GLIBC_2.33 symbol into libc.so.6? It will be very
unusual for an object not to be linked against libc.so.6, so I expect
that the IFUNC resolver already won't run until after libc.so.6
relocation (due to the relocation ordering check we have on x86).
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 10+ messages in thread
* V4 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-21 11:19 ` Florian Weimer via Libc-alpha
@ 2021-01-21 13:29 ` H.J. Lu via Libc-alpha
2021-01-21 13:57 ` Florian Weimer via Libc-alpha
0 siblings, 1 reply; 10+ messages in thread
From: H.J. Lu via Libc-alpha @ 2021-01-21 13:29 UTC (permalink / raw)
To: Florian Weimer; +Cc: H.J. Lu via Libc-alpha
[-- Attachment #1: Type: text/plain, Size: 828 bytes --]
On Thu, Jan 21, 2021 at 3:19 AM Florian Weimer <fweimer@redhat.com> wrote:
>
> * H. J. Lu:
>
> > Here is the updated patch. The public function is changed to
> >
> > /* Get a pointer to the CPU feature structure. */
> > extern const struct cpuid_feature *__x86_get_cpuid_feature_leaf (unsigned int)
> > __attribute__ ((pure));
> >
> > OK for master?
>
> Are you okay with this version? I like the simplicity of the interface.
Me too.
> Can we move the GLIBC_2.33 symbol into libc.so.6? It will be very
> unusual for an object not to be linked against libc.so.6, so I expect
> that the IFUNC resolver already won't run until after libc.so.6
> relocation (due to the relocation ordering check we have on x86).
Here is the updated patch to place __x86_get_cpuid_feature_leaf
in libc. OK for master?
Thanks.
--
H.J.
[-- Attachment #2: 0001-sys-platform-x86.h-Remove-the-C-preprocessor-magic.patch --]
[-- Type: text/x-patch, Size: 84078 bytes --]
From 72db84807ea6b10a511ecbc9cf6ebeb929eee92a Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Fri, 25 Dec 2020 07:30:46 -0800
Subject: [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
In <sys/platform/x86.h>, define CPU features as enum instead of using
the C preprocessor magic to make it easier to wrap this functionality
in other languages. Move the C preprocessor magic to internal header
for better GCC codegen when more than one features are checked in a
single expression as in x86-64 dl-hwcaps-subdirs.c.
1. Rename COMMON_CPUID_INDEX_XXX to CPUID_INDEX_XXX.
2. Move CPUID_INDEX_MAX to sysdeps/x86/include/cpu-features.h.
3. Remove struct cpu_features and __x86_get_cpu_features from
<sys/platform/x86.h>.
4. Add __x86_get_cpuid_feature_leaf to <sys/platform/x86.h> and put it
in libc.
5. Make __get_cpu_features() private to glibc.
6. Replace __x86_get_cpu_features(N) with __get_cpu_features().
7. Add _dl_x86_get_cpu_features to GLIBC_PRIVATE.
8. Use a single enum index for each CPU feature detection.
9. Pass the CPUID feature leaf to __x86_get_cpuid_feature_leaf.
10. Return zero struct cpuid_feature for the older glibc binary with a
smaller CPUID_INDEX_MAX [BZ #27104].
11. Inside glibc, use the C preprocessor magic so that cpu_features data
can be loaded just once leading to more compact code for glibc.
256 bits are used for each CPUID leaf. Some leaves only contain a few
features. We can add exceptions to such leaves. But it will increase
code sizes and it is harder to provide backward/forward compatibilities
when new features are added to such leaves in the future.
When new leaves are added, _rtld_global_ro offsets will change which
leads to race condition during in-place updates. We may avoid in-place
updates by
1. Rename the old glibc.
2. Install the new glibc.
3. Remove the old glibc.
NB: A function, __x86_get_cpuid_feature_leaf , is used to avoid the copy
relocation issue with IFUNC resolver as shown in IFUNC resolver tests.
---
manual/platform.texi | 7 +-
sysdeps/mach/hurd/i386/ld.abilist | 1 -
sysdeps/mach/hurd/i386/libc.abilist | 1 +
sysdeps/unix/sysv/linux/i386/ld.abilist | 1 -
sysdeps/unix/sysv/linux/i386/libc.abilist | 1 +
sysdeps/unix/sysv/linux/x86_64/64/ld.abilist | 1 -
.../unix/sysv/linux/x86_64/64/libc.abilist | 1 +
.../sysv/linux/x86_64/tst-glibc-hwcaps-2.c | 3 +-
sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist | 1 -
.../unix/sysv/linux/x86_64/x32/libc.abilist | 1 +
sysdeps/x86/Makefile | 1 +
sysdeps/x86/Versions | 7 +-
sysdeps/x86/bits/platform/x86.h | 299 +++++++
sysdeps/x86/cpu-features.c | 68 +-
sysdeps/x86/dl-cacheinfo.h | 4 +-
sysdeps/x86/dl-get-cpu-features.c | 6 +-
sysdeps/x86/get-cpuid-feature-leaf.c | 30 +
sysdeps/x86/get-isa-level.h | 2 +-
sysdeps/x86/include/cpu-features.h | 759 ++++++++++++++++-
sysdeps/x86/sys/platform/x86.h | 777 +-----------------
sysdeps/x86/tst-get-cpu-features.c | 29 +-
sysdeps/x86/tst-isa-level-1.c | 3 +-
sysdeps/x86_64/dl-hwcaps-subdirs.c | 3 +-
sysdeps/x86_64/tst-glibc-hwcaps.c | 3 +-
24 files changed, 1162 insertions(+), 847 deletions(-)
create mode 100644 sysdeps/x86/bits/platform/x86.h
create mode 100644 sysdeps/x86/get-cpuid-feature-leaf.c
diff --git a/manual/platform.texi b/manual/platform.texi
index b67683aeb3..6caf68d796 100644
--- a/manual/platform.texi
+++ b/manual/platform.texi
@@ -142,13 +142,10 @@ reserved.
Facilities specific to X86 that are not specific to a particular
operating system are declared in @file{sys/platform/x86.h}.
-@deftypefun {const struct cpu_features *} __x86_get_cpu_features (unsigned int @var{max})
+@deftypefun {const struct cpuid_feature *} __x86_get_cpuid_feature_leaf (unsigned int @var{leaf})
@safety{@prelim{}@mtsafe{}@assafe{}@acsafe{}}
Return a pointer to x86 CPU feature structure used by query macros for x86
-CPU features. If @var{max} exceeds @code{COMMON_CPUID_INDEX_MAX} which
-is the limit of the CPUID leaves supported by @Theglibc{}, the function
-returns @code{NULL}, indicating that the queried processor feature is
-unsupported by @Theglibc{} run-time.
+CPU feature @var{leaf}.
@end deftypefun
@deftypefn Macro int HAS_CPU_FEATURE (@var{name})
diff --git a/sysdeps/mach/hurd/i386/ld.abilist b/sysdeps/mach/hurd/i386/ld.abilist
index 751664bcda..7e20c5e7ce 100644
--- a/sysdeps/mach/hurd/i386/ld.abilist
+++ b/sysdeps/mach/hurd/i386/ld.abilist
@@ -16,4 +16,3 @@ GLIBC_2.2.6 _r_debug D 0x14
GLIBC_2.2.6 abort F
GLIBC_2.3 ___tls_get_addr F
GLIBC_2.3 __tls_get_addr F
-GLIBC_2.33 __x86_get_cpu_features F
diff --git a/sysdeps/mach/hurd/i386/libc.abilist b/sysdeps/mach/hurd/i386/libc.abilist
index 7a5eb66b85..509e9b7cb0 100644
--- a/sysdeps/mach/hurd/i386/libc.abilist
+++ b/sysdeps/mach/hurd/i386/libc.abilist
@@ -2191,6 +2191,7 @@ GLIBC_2.32 thrd_current F
GLIBC_2.32 thrd_equal F
GLIBC_2.32 thrd_sleep F
GLIBC_2.32 thrd_yield F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
GLIBC_2.33 fstat F
GLIBC_2.33 fstat64 F
GLIBC_2.33 fstatat F
diff --git a/sysdeps/unix/sysv/linux/i386/ld.abilist b/sysdeps/unix/sysv/linux/i386/ld.abilist
index 04655651b0..0478e22071 100644
--- a/sysdeps/unix/sysv/linux/i386/ld.abilist
+++ b/sysdeps/unix/sysv/linux/i386/ld.abilist
@@ -3,4 +3,3 @@ GLIBC_2.1 __libc_stack_end D 0x4
GLIBC_2.1 _dl_mcount F
GLIBC_2.3 ___tls_get_addr F
GLIBC_2.3 __tls_get_addr F
-GLIBC_2.33 __x86_get_cpu_features F
diff --git a/sysdeps/unix/sysv/linux/i386/libc.abilist b/sysdeps/unix/sysv/linux/i386/libc.abilist
index 4f0d3c1eb5..ddc5837059 100644
--- a/sysdeps/unix/sysv/linux/i386/libc.abilist
+++ b/sysdeps/unix/sysv/linux/i386/libc.abilist
@@ -2229,6 +2229,7 @@ GLIBC_2.32 sigabbrev_np F
GLIBC_2.32 sigdescr_np F
GLIBC_2.32 strerrordesc_np F
GLIBC_2.32 strerrorname_np F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
GLIBC_2.33 fstat F
GLIBC_2.33 fstat64 F
GLIBC_2.33 fstatat F
diff --git a/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist b/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
index e632c6198e..d3cdf7611e 100644
--- a/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/64/ld.abilist
@@ -2,4 +2,3 @@ GLIBC_2.2.5 __libc_stack_end D 0x8
GLIBC_2.2.5 _dl_mcount F
GLIBC_2.2.5 _r_debug D 0x28
GLIBC_2.3 __tls_get_addr F
-GLIBC_2.33 __x86_get_cpu_features F
diff --git a/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist b/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
index 4fff61818b..2744bba4af 100644
--- a/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/64/libc.abilist
@@ -2076,6 +2076,7 @@ GLIBC_2.32 sigabbrev_np F
GLIBC_2.32 sigdescr_np F
GLIBC_2.32 strerrordesc_np F
GLIBC_2.32 strerrorname_np F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
GLIBC_2.33 fstat F
GLIBC_2.33 fstat64 F
GLIBC_2.33 fstatat F
diff --git a/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c b/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
index fe91bfd224..2d8f7ec462 100644
--- a/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
+++ b/sysdeps/unix/sysv/linux/x86_64/tst-glibc-hwcaps-2.c
@@ -29,8 +29,7 @@ extern int dso_isa_level (void);
static int
do_test (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
bool has_isa_baseline = ((isa_level & GNU_PROPERTY_X86_ISA_1_BASELINE)
== GNU_PROPERTY_X86_ISA_1_BASELINE);
diff --git a/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist b/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
index 9d9d0befde..c70bccf782 100644
--- a/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/x32/ld.abilist
@@ -2,4 +2,3 @@ GLIBC_2.16 __libc_stack_end D 0x4
GLIBC_2.16 __tls_get_addr F
GLIBC_2.16 _dl_mcount F
GLIBC_2.16 _r_debug D 0x14
-GLIBC_2.33 __x86_get_cpu_features F
diff --git a/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist b/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist
index 102ed47a9c..ce2f4fb72b 100644
--- a/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist
+++ b/sysdeps/unix/sysv/linux/x86_64/x32/libc.abilist
@@ -2173,6 +2173,7 @@ GLIBC_2.32 sigabbrev_np F
GLIBC_2.32 sigdescr_np F
GLIBC_2.32 strerrordesc_np F
GLIBC_2.32 strerrorname_np F
+GLIBC_2.33 __x86_get_cpuid_feature_leaf F
GLIBC_2.33 fstat F
GLIBC_2.33 fstat64 F
GLIBC_2.33 fstatat F
diff --git a/sysdeps/x86/Makefile b/sysdeps/x86/Makefile
index 9ed559ace5..887aa9ff2b 100644
--- a/sysdeps/x86/Makefile
+++ b/sysdeps/x86/Makefile
@@ -3,6 +3,7 @@ gen-as-const-headers += cpu-features-offsets.sym
endif
ifeq ($(subdir),elf)
+sysdep_routines += get-cpuid-feature-leaf
sysdep-dl-routines += dl-get-cpu-features
sysdep_headers += sys/platform/x86.h
diff --git a/sysdeps/x86/Versions b/sysdeps/x86/Versions
index 59db578a9d..4b10c4b5d7 100644
--- a/sysdeps/x86/Versions
+++ b/sysdeps/x86/Versions
@@ -1,5 +1,10 @@
ld {
+ GLIBC_PRIVATE {
+ _dl_x86_get_cpu_features;
+ }
+}
+libc {
GLIBC_2.33 {
- __x86_get_cpu_features;
+ __x86_get_cpuid_feature_leaf;
}
}
diff --git a/sysdeps/x86/bits/platform/x86.h b/sysdeps/x86/bits/platform/x86.h
new file mode 100644
index 0000000000..8f423ae721
--- /dev/null
+++ b/sysdeps/x86/bits/platform/x86.h
@@ -0,0 +1,299 @@
+/* Constants and data structures for x86 CPU features.
+ This file is part of the GNU C Library.
+ Copyright (C) 2008-2020 Free Software Foundation, Inc.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+#ifndef _SYS_PLATFORM_X86_H
+# error "Never include <bits/platform/x86.h> directly; use <sys/platform/x86.h> instead."
+#endif
+
+enum
+{
+ CPUID_INDEX_1 = 0,
+ CPUID_INDEX_7,
+ CPUID_INDEX_80000001,
+ CPUID_INDEX_D_ECX_1,
+ CPUID_INDEX_80000007,
+ CPUID_INDEX_80000008,
+ CPUID_INDEX_7_ECX_1,
+ CPUID_INDEX_19
+};
+
+struct cpuid_feature
+{
+ unsigned int cpuid_array[4];
+ unsigned int usable_array[4];
+};
+
+enum cpuid_register_index
+{
+ cpuid_register_index_eax = 0,
+ cpuid_register_index_ebx,
+ cpuid_register_index_ecx,
+ cpuid_register_index_edx
+};
+
+/* CPU features. */
+
+enum
+{
+ x86_cpu_index_1_ecx
+ = (CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SSE3 = x86_cpu_index_1_ecx,
+ x86_cpu_PCLMULQDQ = x86_cpu_index_1_ecx + 1,
+ x86_cpu_DTES64 = x86_cpu_index_1_ecx + 2,
+ x86_cpu_MONITOR = x86_cpu_index_1_ecx + 3,
+ x86_cpu_DS_CPL = x86_cpu_index_1_ecx + 4,
+ x86_cpu_VMX = x86_cpu_index_1_ecx + 5,
+ x86_cpu_SMX = x86_cpu_index_1_ecx + 6,
+ x86_cpu_EIST = x86_cpu_index_1_ecx + 7,
+ x86_cpu_TM2 = x86_cpu_index_1_ecx + 8,
+ x86_cpu_SSSE3 = x86_cpu_index_1_ecx + 9,
+ x86_cpu_CNXT_ID = x86_cpu_index_1_ecx + 10,
+ x86_cpu_SDBG = x86_cpu_index_1_ecx + 11,
+ x86_cpu_FMA = x86_cpu_index_1_ecx + 12,
+ x86_cpu_CMPXCHG16B = x86_cpu_index_1_ecx + 13,
+ x86_cpu_XTPRUPDCTRL = x86_cpu_index_1_ecx + 14,
+ x86_cpu_PDCM = x86_cpu_index_1_ecx + 15,
+ x86_cpu_INDEX_1_ECX_16 = x86_cpu_index_1_ecx + 16,
+ x86_cpu_PCID = x86_cpu_index_1_ecx + 17,
+ x86_cpu_DCA = x86_cpu_index_1_ecx + 18,
+ x86_cpu_SSE4_1 = x86_cpu_index_1_ecx + 19,
+ x86_cpu_SSE4_2 = x86_cpu_index_1_ecx + 20,
+ x86_cpu_X2APIC = x86_cpu_index_1_ecx + 21,
+ x86_cpu_MOVBE = x86_cpu_index_1_ecx + 22,
+ x86_cpu_POPCNT = x86_cpu_index_1_ecx + 23,
+ x86_cpu_TSC_DEADLINE = x86_cpu_index_1_ecx + 24,
+ x86_cpu_AES = x86_cpu_index_1_ecx + 25,
+ x86_cpu_XSAVE = x86_cpu_index_1_ecx + 26,
+ x86_cpu_OSXSAVE = x86_cpu_index_1_ecx + 27,
+ x86_cpu_AVX = x86_cpu_index_1_ecx + 28,
+ x86_cpu_F16C = x86_cpu_index_1_ecx + 29,
+ x86_cpu_RDRAND = x86_cpu_index_1_ecx + 30,
+ x86_cpu_INDEX_1_ECX_31 = x86_cpu_index_1_ecx + 31,
+
+ x86_cpu_index_1_edx
+ = (CPUID_INDEX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FPU = x86_cpu_index_1_edx,
+ x86_cpu_VME = x86_cpu_index_1_edx + 1,
+ x86_cpu_DE = x86_cpu_index_1_edx + 2,
+ x86_cpu_PSE = x86_cpu_index_1_edx + 3,
+ x86_cpu_TSC = x86_cpu_index_1_edx + 4,
+ x86_cpu_MSR = x86_cpu_index_1_edx + 5,
+ x86_cpu_PAE = x86_cpu_index_1_edx + 6,
+ x86_cpu_MCE = x86_cpu_index_1_edx + 7,
+ x86_cpu_CX8 = x86_cpu_index_1_edx + 8,
+ x86_cpu_APIC = x86_cpu_index_1_edx + 9,
+ x86_cpu_INDEX_1_EDX_10 = x86_cpu_index_1_edx + 10,
+ x86_cpu_SEP = x86_cpu_index_1_edx + 11,
+ x86_cpu_MTRR = x86_cpu_index_1_edx + 12,
+ x86_cpu_PGE = x86_cpu_index_1_edx + 13,
+ x86_cpu_MCA = x86_cpu_index_1_edx + 14,
+ x86_cpu_CMOV = x86_cpu_index_1_edx + 15,
+ x86_cpu_PAT = x86_cpu_index_1_edx + 16,
+ x86_cpu_PSE_36 = x86_cpu_index_1_edx + 17,
+ x86_cpu_PSN = x86_cpu_index_1_edx + 18,
+ x86_cpu_CLFSH = x86_cpu_index_1_edx + 19,
+ x86_cpu_INDEX_1_EDX_20 = x86_cpu_index_1_edx + 20,
+ x86_cpu_DS = x86_cpu_index_1_edx + 21,
+ x86_cpu_ACPI = x86_cpu_index_1_edx + 22,
+ x86_cpu_MMX = x86_cpu_index_1_edx + 23,
+ x86_cpu_FXSR = x86_cpu_index_1_edx + 24,
+ x86_cpu_SSE = x86_cpu_index_1_edx + 25,
+ x86_cpu_SSE2 = x86_cpu_index_1_edx + 26,
+ x86_cpu_SS = x86_cpu_index_1_edx + 27,
+ x86_cpu_HTT = x86_cpu_index_1_edx + 28,
+ x86_cpu_TM = x86_cpu_index_1_edx + 29,
+ x86_cpu_INDEX_1_EDX_30 = x86_cpu_index_1_edx + 30,
+ x86_cpu_PBE = x86_cpu_index_1_edx + 31,
+
+ x86_cpu_index_7_ebx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_FSGSBASE = x86_cpu_index_7_ebx,
+ x86_cpu_TSC_ADJUST = x86_cpu_index_7_ebx + 1,
+ x86_cpu_SGX = x86_cpu_index_7_ebx + 2,
+ x86_cpu_BMI1 = x86_cpu_index_7_ebx + 3,
+ x86_cpu_HLE = x86_cpu_index_7_ebx + 4,
+ x86_cpu_AVX2 = x86_cpu_index_7_ebx + 5,
+ x86_cpu_INDEX_7_EBX_6 = x86_cpu_index_7_ebx + 6,
+ x86_cpu_SMEP = x86_cpu_index_7_ebx + 7,
+ x86_cpu_BMI2 = x86_cpu_index_7_ebx + 8,
+ x86_cpu_ERMS = x86_cpu_index_7_ebx + 9,
+ x86_cpu_INVPCID = x86_cpu_index_7_ebx + 10,
+ x86_cpu_RTM = x86_cpu_index_7_ebx + 11,
+ x86_cpu_RDT_M = x86_cpu_index_7_ebx + 12,
+ x86_cpu_DEPR_FPU_CS_DS = x86_cpu_index_7_ebx + 13,
+ x86_cpu_MPX = x86_cpu_index_7_ebx + 14,
+ x86_cpu_RDT_A = x86_cpu_index_7_ebx + 15,
+ x86_cpu_AVX512F = x86_cpu_index_7_ebx + 16,
+ x86_cpu_AVX512DQ = x86_cpu_index_7_ebx + 17,
+ x86_cpu_RDSEED = x86_cpu_index_7_ebx + 18,
+ x86_cpu_ADX = x86_cpu_index_7_ebx + 19,
+ x86_cpu_SMAP = x86_cpu_index_7_ebx + 20,
+ x86_cpu_AVX512_IFMA = x86_cpu_index_7_ebx + 21,
+ x86_cpu_INDEX_7_EBX_22 = x86_cpu_index_7_ebx + 22,
+ x86_cpu_CLFLUSHOPT = x86_cpu_index_7_ebx + 23,
+ x86_cpu_CLWB = x86_cpu_index_7_ebx + 24,
+ x86_cpu_TRACE = x86_cpu_index_7_ebx + 25,
+ x86_cpu_AVX512PF = x86_cpu_index_7_ebx + 26,
+ x86_cpu_AVX512ER = x86_cpu_index_7_ebx + 27,
+ x86_cpu_AVX512CD = x86_cpu_index_7_ebx + 28,
+ x86_cpu_SHA = x86_cpu_index_7_ebx + 29,
+ x86_cpu_AVX512BW = x86_cpu_index_7_ebx + 30,
+ x86_cpu_AVX512VL = x86_cpu_index_7_ebx + 31,
+
+ x86_cpu_index_7_ecx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_PREFETCHWT1 = x86_cpu_index_7_ecx,
+ x86_cpu_AVX512_VBMI = x86_cpu_index_7_ecx + 1,
+ x86_cpu_UMIP = x86_cpu_index_7_ecx + 2,
+ x86_cpu_PKU = x86_cpu_index_7_ecx + 3,
+ x86_cpu_OSPKE = x86_cpu_index_7_ecx + 4,
+ x86_cpu_WAITPKG = x86_cpu_index_7_ecx + 5,
+ x86_cpu_AVX512_VBMI2 = x86_cpu_index_7_ecx + 6,
+ x86_cpu_SHSTK = x86_cpu_index_7_ecx + 7,
+ x86_cpu_GFNI = x86_cpu_index_7_ecx + 8,
+ x86_cpu_VAES = x86_cpu_index_7_ecx + 9,
+ x86_cpu_VPCLMULQDQ = x86_cpu_index_7_ecx + 10,
+ x86_cpu_AVX512_VNNI = x86_cpu_index_7_ecx + 11,
+ x86_cpu_AVX512_BITALG = x86_cpu_index_7_ecx + 12,
+ x86_cpu_INDEX_7_ECX_13 = x86_cpu_index_7_ecx + 13,
+ x86_cpu_AVX512_VPOPCNTDQ = x86_cpu_index_7_ecx + 14,
+ x86_cpu_INDEX_7_ECX_1 = x86_cpu_index_7_ecx + 15,
+ x86_cpu_INDEX_7_ECX_16 = x86_cpu_index_7_ecx + 16,
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+ x86_cpu_RDPID = x86_cpu_index_7_ecx + 22,
+ x86_cpu_KL = x86_cpu_index_7_ecx + 23,
+ x86_cpu_INDEX_7_ECX_24 = x86_cpu_index_7_ecx + 24,
+ x86_cpu_CLDEMOTE = x86_cpu_index_7_ecx + 25,
+ x86_cpu_INDEX_7_ECX_26 = x86_cpu_index_7_ecx + 26,
+ x86_cpu_MOVDIRI = x86_cpu_index_7_ecx + 27,
+ x86_cpu_MOVDIR64B = x86_cpu_index_7_ecx + 28,
+ x86_cpu_ENQCMD = x86_cpu_index_7_ecx + 29,
+ x86_cpu_SGX_LC = x86_cpu_index_7_ecx + 30,
+ x86_cpu_PKS = x86_cpu_index_7_ecx + 31,
+
+ x86_cpu_index_7_edx
+ = (CPUID_INDEX_7 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INDEX_7_EDX_0 = x86_cpu_index_7_edx,
+ x86_cpu_INDEX_7_EDX_1 = x86_cpu_index_7_edx + 1,
+ x86_cpu_AVX512_4VNNIW = x86_cpu_index_7_edx + 2,
+ x86_cpu_AVX512_4FMAPS = x86_cpu_index_7_edx + 3,
+ x86_cpu_FSRM = x86_cpu_index_7_edx + 4,
+ x86_cpu_UINTR = x86_cpu_index_7_edx + 5,
+ x86_cpu_INDEX_7_EDX_6 = x86_cpu_index_7_edx + 6,
+ x86_cpu_INDEX_7_EDX_7 = x86_cpu_index_7_edx + 7,
+ x86_cpu_AVX512_VP2INTERSECT = x86_cpu_index_7_edx + 8,
+ x86_cpu_INDEX_7_EDX_9 = x86_cpu_index_7_edx + 9,
+ x86_cpu_MD_CLEAR = x86_cpu_index_7_edx + 10,
+ x86_cpu_INDEX_7_EDX_11 = x86_cpu_index_7_edx + 11,
+ x86_cpu_INDEX_7_EDX_12 = x86_cpu_index_7_edx + 12,
+ x86_cpu_INDEX_7_EDX_13 = x86_cpu_index_7_edx + 13,
+ x86_cpu_SERIALIZE = x86_cpu_index_7_edx + 14,
+ x86_cpu_HYBRID = x86_cpu_index_7_edx + 15,
+ x86_cpu_TSXLDTRK = x86_cpu_index_7_edx + 16,
+ x86_cpu_INDEX_7_EDX_17 = x86_cpu_index_7_edx + 17,
+ x86_cpu_PCONFIG = x86_cpu_index_7_edx + 18,
+ x86_cpu_INDEX_7_EDX_19 = x86_cpu_index_7_edx + 19,
+ x86_cpu_IBT = x86_cpu_index_7_edx + 20,
+ x86_cpu_INDEX_7_EDX_21 = x86_cpu_index_7_edx + 21,
+ x86_cpu_AMX_BF16 = x86_cpu_index_7_edx + 22,
+ x86_cpu_AVX512_FP16 = x86_cpu_index_7_edx + 23,
+ x86_cpu_AMX_TILE = x86_cpu_index_7_edx + 24,
+ x86_cpu_AMX_INT8 = x86_cpu_index_7_edx + 25,
+ x86_cpu_IBRS_IBPB = x86_cpu_index_7_edx + 26,
+ x86_cpu_STIBP = x86_cpu_index_7_edx + 27,
+ x86_cpu_L1D_FLUSH = x86_cpu_index_7_edx + 28,
+ x86_cpu_ARCH_CAPABILITIES = x86_cpu_index_7_edx + 29,
+ x86_cpu_CORE_CAPABILITIES = x86_cpu_index_7_edx + 30,
+ x86_cpu_SSBD = x86_cpu_index_7_edx + 31,
+
+ x86_cpu_index_80000001_ecx
+ = (CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ecx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_LAHF64_SAHF64 = x86_cpu_index_80000001_ecx,
+ x86_cpu_SVM = x86_cpu_index_80000001_ecx + 2,
+ x86_cpu_LZCNT = x86_cpu_index_80000001_ecx + 5,
+ x86_cpu_SSE4A = x86_cpu_index_80000001_ecx + 6,
+ x86_cpu_PREFETCHW = x86_cpu_index_80000001_ecx + 8,
+ x86_cpu_XOP = x86_cpu_index_80000001_ecx + 11,
+ x86_cpu_LWP = x86_cpu_index_80000001_ecx + 15,
+ x86_cpu_FMA4 = x86_cpu_index_80000001_ecx + 16,
+ x86_cpu_TBM = x86_cpu_index_80000001_ecx + 20,
+
+ x86_cpu_index_80000001_edx
+ = (CPUID_INDEX_80000001 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_SYSCALL_SYSRET = x86_cpu_index_80000001_edx + 11,
+ x86_cpu_NX = x86_cpu_index_80000001_edx + 20,
+ x86_cpu_PAGE1GB = x86_cpu_index_80000001_edx + 26,
+ x86_cpu_RDTSCP = x86_cpu_index_80000001_edx + 27,
+ x86_cpu_LM = x86_cpu_index_80000001_edx + 29,
+
+ x86_cpu_index_d_ecx_1_eax
+ = (CPUID_INDEX_D_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_XSAVEOPT = x86_cpu_index_d_ecx_1_eax,
+ x86_cpu_XSAVEC = x86_cpu_index_d_ecx_1_eax + 1,
+ x86_cpu_XGETBV_ECX_1 = x86_cpu_index_d_ecx_1_eax + 2,
+ x86_cpu_XSAVES = x86_cpu_index_d_ecx_1_eax + 3,
+ x86_cpu_XFD = x86_cpu_index_d_ecx_1_eax + 4,
+
+ x86_cpu_index_80000007_edx
+ = (CPUID_INDEX_80000007 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_edx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_INVARIANT_TSC = x86_cpu_index_80000007_edx + 8,
+
+ x86_cpu_index_80000008_ebx
+ = (CPUID_INDEX_80000008 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_WBNOINVD = x86_cpu_index_80000008_ebx + 9,
+
+ x86_cpu_index_7_ecx_1_eax
+ = (CPUID_INDEX_7_ECX_1 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_eax * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AVX_VNNI = x86_cpu_index_7_ecx_1_eax + 4,
+ x86_cpu_AVX512_BF16 = x86_cpu_index_7_ecx_1_eax + 5,
+ x86_cpu_FZLRM = x86_cpu_index_7_ecx_1_eax + 10,
+ x86_cpu_FSRS = x86_cpu_index_7_ecx_1_eax + 11,
+ x86_cpu_FSRCS = x86_cpu_index_7_ecx_1_eax + 12,
+ x86_cpu_HRESET = x86_cpu_index_7_ecx_1_eax + 22,
+ x86_cpu_LAM = x86_cpu_index_7_ecx_1_eax + 26,
+
+ x86_cpu_index_19_ebx
+ = (CPUID_INDEX_19 * 8 * 4 * sizeof (unsigned int)
+ + cpuid_register_index_ebx * 8 * sizeof (unsigned int)),
+
+ x86_cpu_AESKLE = x86_cpu_index_19_ebx,
+ x86_cpu_WIDE_KL = x86_cpu_index_19_ebx + 2
+};
diff --git a/sysdeps/x86/cpu-features.c b/sysdeps/x86/cpu-features.c
index 06e4307b71..6496512a0d 100644
--- a/sysdeps/x86/cpu-features.c
+++ b/sysdeps/x86/cpu-features.c
@@ -293,22 +293,22 @@ get_extended_indices (struct cpu_features *cpu_features)
__cpuid (0x80000000, eax, ebx, ecx, edx);
if (eax >= 0x80000001)
__cpuid (0x80000001,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000001].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000001].cpuid.edx);
if (eax >= 0x80000007)
__cpuid (0x80000007,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000007].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000007].cpuid.edx);
if (eax >= 0x80000008)
__cpuid (0x80000008,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_80000008].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_80000008].cpuid.edx);
}
static void
@@ -320,10 +320,10 @@ get_common_indices (struct cpu_features *cpu_features,
{
unsigned int eax;
__cpuid (1, eax,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.edx);
- cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.eax = eax;
+ cpu_features->features[CPUID_INDEX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_1].cpuid.eax = eax;
*family = (eax >> 8) & 0x0f;
*model = (eax >> 4) & 0x0f;
*extended_model = (eax >> 12) & 0xf0;
@@ -338,30 +338,30 @@ get_common_indices (struct cpu_features *cpu_features,
if (cpu_features->basic.max_cpuid >= 7)
{
__cpuid_count (7, 0,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_7].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_7].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_7].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_7].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_7].cpuid.edx);
__cpuid_count (7, 1,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_7_ECX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_7_ECX_1].cpuid.edx);
}
if (cpu_features->basic.max_cpuid >= 0xd)
__cpuid_count (0xd, 1,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_D_ECX_1].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_D_ECX_1].cpuid.edx);
if (cpu_features->basic.max_cpuid >= 0x19)
__cpuid_count (0x19, 0,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.eax,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.ebx,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.ecx,
- cpu_features->features[COMMON_CPUID_INDEX_19].cpuid.edx);
+ cpu_features->features[CPUID_INDEX_19].cpuid.eax,
+ cpu_features->features[CPUID_INDEX_19].cpuid.ebx,
+ cpu_features->features[CPUID_INDEX_19].cpuid.ecx,
+ cpu_features->features[CPUID_INDEX_19].cpuid.edx);
}
_Static_assert (((index_arch_Fast_Unaligned_Load
@@ -536,11 +536,11 @@ init_cpu_features (struct cpu_features *cpu_features)
update_usable (cpu_features);
- ecx = cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ecx;
+ ecx = cpu_features->features[CPUID_INDEX_1].cpuid.ecx;
if (CPU_FEATURE_USABLE_P (cpu_features, AVX))
{
- /* Since the FMA4 bit is in COMMON_CPUID_INDEX_80000001 and
+ /* Since the FMA4 bit is in CPUID_INDEX_80000001 and
FMA4 requires AVX, determine if FMA4 is usable here. */
CPU_FEATURE_SET_USABLE (cpu_features, FMA4);
}
diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h
index 96c49f2411..a31fa0783a 100644
--- a/sysdeps/x86/dl-cacheinfo.h
+++ b/sysdeps/x86/dl-cacheinfo.h
@@ -672,8 +672,8 @@ intel_bug_no_cache_info:
/* Assume that all logical threads share the highest cache
level. */
threads
- = ((cpu_features->features[COMMON_CPUID_INDEX_1].cpuid.ebx
- >> 16) & 0xff);
+ = ((cpu_features->features[CPUID_INDEX_1].cpuid.ebx >> 16)
+ & 0xff);
}
/* Cap usage of highest cache level to the number of supported
diff --git a/sysdeps/x86/dl-get-cpu-features.c b/sysdeps/x86/dl-get-cpu-features.c
index 19fc4f1eeb..99e3aa7d04 100644
--- a/sysdeps/x86/dl-get-cpu-features.c
+++ b/sysdeps/x86/dl-get-cpu-features.c
@@ -43,12 +43,10 @@ __ifunc (__x86_cpu_features, __x86_cpu_features, NULL, void,
_dl_x86_init_cpu_features);
#endif
-#undef __x86_get_cpu_features
+#undef _dl_x86_get_cpu_features
const struct cpu_features *
-__x86_get_cpu_features (unsigned int max)
+_dl_x86_get_cpu_features (void)
{
- if (max > COMMON_CPUID_INDEX_MAX)
- return NULL;
return &GLRO(dl_x86_cpu_features);
}
diff --git a/sysdeps/x86/get-cpuid-feature-leaf.c b/sysdeps/x86/get-cpuid-feature-leaf.c
new file mode 100644
index 0000000000..097e47c7b3
--- /dev/null
+++ b/sysdeps/x86/get-cpuid-feature-leaf.c
@@ -0,0 +1,30 @@
+/* Get CPUID feature leaf.
+ Copyright (C) 2021 Free Software Foundation, Inc.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <https://www.gnu.org/licenses/>. */
+
+
+#include <ldsodefs.h>
+
+const struct cpuid_feature *
+__x86_get_cpuid_feature_leaf (unsigned int leaf)
+{
+ static const struct cpuid_feature feature = {};
+ if (leaf < CPUID_INDEX_MAX)
+ return ((const struct cpuid_feature *)
+ &GLRO(dl_x86_cpu_features).features[leaf]);
+ else
+ return &feature;
+}
diff --git a/sysdeps/x86/get-isa-level.h b/sysdeps/x86/get-isa-level.h
index 3f0e27ee59..aa80f56ca6 100644
--- a/sysdeps/x86/get-isa-level.h
+++ b/sysdeps/x86/get-isa-level.h
@@ -17,7 +17,7 @@
<https://www.gnu.org/licenses/>. */
#include <elf.h>
-#include <sys/platform/x86.h>
+#include <cpu-features.h>
/* Get GNU_PROPERTY_X86_ISA_1_BASELINE and GNU_PROPERTY_X86_ISA_1_V[234]
ISA level. */
diff --git a/sysdeps/x86/include/cpu-features.h b/sysdeps/x86/include/cpu-features.h
index 50cb5a15f5..624736b40e 100644
--- a/sysdeps/x86/include/cpu-features.h
+++ b/sysdeps/x86/include/cpu-features.h
@@ -23,16 +23,14 @@
# error this should be impossible
#endif
-/* Get most of the contents from the public header, but we define a
- different `struct cpu_features' type for private use. */
-#define cpu_features cpu_features_public
-#define __x86_get_cpu_features __x86_get_cpu_features_public
+/* Get data structures without inline functions. */
+#define _SYS_PLATFORM_X86_H
+#include <bits/platform/x86.h>
-#include <sysdeps/x86/sys/platform/x86.h>
-
-#undef cpu_features
-#undef __x86_get_cpu_features
-#define __get_cpu_features() __x86_get_cpu_features (0)
+enum
+{
+ CPUID_INDEX_MAX = CPUID_INDEX_19 + 1
+};
enum
{
@@ -54,14 +52,20 @@ enum
#define CPU_FEATURE_PREFERRED_P(ptr, name) \
((ptr->preferred[index_arch_##name] & bit_arch_##name) != 0)
+#define CPU_FEATURE_CHECK_P(ptr, name, check) \
+ ((ptr->features[index_cpu_##name].check.reg_##name \
+ & bit_cpu_##name) != 0)
+#define CPU_FEATURE_CPU_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, cpuid)
+#define CPU_FEATURE_USABLE_P(ptr, name) \
+ CPU_FEATURE_CHECK_P (ptr, name, usable)
+
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
-#undef HAS_CPU_FEATURE
#define HAS_CPU_FEATURE(name) \
- CPU_FEATURE_CPU_P (__x86_get_cpu_features (0), name)
+ CPU_FEATURE_CPU_P (__get_cpu_features (), name)
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
-#undef CPU_FEATURE_USABLE
#define CPU_FEATURE_USABLE(name) \
- CPU_FEATURE_USABLE_P (__x86_get_cpu_features (0), name)
+ CPU_FEATURE_USABLE_P (__get_cpu_features (), name)
/* CPU_FEATURE_PREFER evaluates to true if we prefer the feature at
runtime. */
#define CPU_FEATURE_PREFERRED(name) \
@@ -74,6 +78,685 @@ enum
#define HAS_ARCH_FEATURE(name) \
CPU_FEATURE_PREFERRED (name)
+/* CPU features. */
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define bit_cpu_SSE3 (1u << 0)
+#define bit_cpu_PCLMULQDQ (1u << 1)
+#define bit_cpu_DTES64 (1u << 2)
+#define bit_cpu_MONITOR (1u << 3)
+#define bit_cpu_DS_CPL (1u << 4)
+#define bit_cpu_VMX (1u << 5)
+#define bit_cpu_SMX (1u << 6)
+#define bit_cpu_EIST (1u << 7)
+#define bit_cpu_TM2 (1u << 8)
+#define bit_cpu_SSSE3 (1u << 9)
+#define bit_cpu_CNXT_ID (1u << 10)
+#define bit_cpu_SDBG (1u << 11)
+#define bit_cpu_FMA (1u << 12)
+#define bit_cpu_CMPXCHG16B (1u << 13)
+#define bit_cpu_XTPRUPDCTRL (1u << 14)
+#define bit_cpu_PDCM (1u << 15)
+#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
+#define bit_cpu_PCID (1u << 17)
+#define bit_cpu_DCA (1u << 18)
+#define bit_cpu_SSE4_1 (1u << 19)
+#define bit_cpu_SSE4_2 (1u << 20)
+#define bit_cpu_X2APIC (1u << 21)
+#define bit_cpu_MOVBE (1u << 22)
+#define bit_cpu_POPCNT (1u << 23)
+#define bit_cpu_TSC_DEADLINE (1u << 24)
+#define bit_cpu_AES (1u << 25)
+#define bit_cpu_XSAVE (1u << 26)
+#define bit_cpu_OSXSAVE (1u << 27)
+#define bit_cpu_AVX (1u << 28)
+#define bit_cpu_F16C (1u << 29)
+#define bit_cpu_RDRAND (1u << 30)
+#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
+
+/* EDX. */
+#define bit_cpu_FPU (1u << 0)
+#define bit_cpu_VME (1u << 1)
+#define bit_cpu_DE (1u << 2)
+#define bit_cpu_PSE (1u << 3)
+#define bit_cpu_TSC (1u << 4)
+#define bit_cpu_MSR (1u << 5)
+#define bit_cpu_PAE (1u << 6)
+#define bit_cpu_MCE (1u << 7)
+#define bit_cpu_CX8 (1u << 8)
+#define bit_cpu_APIC (1u << 9)
+#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
+#define bit_cpu_SEP (1u << 11)
+#define bit_cpu_MTRR (1u << 12)
+#define bit_cpu_PGE (1u << 13)
+#define bit_cpu_MCA (1u << 14)
+#define bit_cpu_CMOV (1u << 15)
+#define bit_cpu_PAT (1u << 16)
+#define bit_cpu_PSE_36 (1u << 17)
+#define bit_cpu_PSN (1u << 18)
+#define bit_cpu_CLFSH (1u << 19)
+#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
+#define bit_cpu_DS (1u << 21)
+#define bit_cpu_ACPI (1u << 22)
+#define bit_cpu_MMX (1u << 23)
+#define bit_cpu_FXSR (1u << 24)
+#define bit_cpu_SSE (1u << 25)
+#define bit_cpu_SSE2 (1u << 26)
+#define bit_cpu_SS (1u << 27)
+#define bit_cpu_HTT (1u << 28)
+#define bit_cpu_TM (1u << 29)
+#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
+#define bit_cpu_PBE (1u << 31)
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define bit_cpu_FSGSBASE (1u << 0)
+#define bit_cpu_TSC_ADJUST (1u << 1)
+#define bit_cpu_SGX (1u << 2)
+#define bit_cpu_BMI1 (1u << 3)
+#define bit_cpu_HLE (1u << 4)
+#define bit_cpu_AVX2 (1u << 5)
+#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
+#define bit_cpu_SMEP (1u << 7)
+#define bit_cpu_BMI2 (1u << 8)
+#define bit_cpu_ERMS (1u << 9)
+#define bit_cpu_INVPCID (1u << 10)
+#define bit_cpu_RTM (1u << 11)
+#define bit_cpu_RDT_M (1u << 12)
+#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
+#define bit_cpu_MPX (1u << 14)
+#define bit_cpu_RDT_A (1u << 15)
+#define bit_cpu_AVX512F (1u << 16)
+#define bit_cpu_AVX512DQ (1u << 17)
+#define bit_cpu_RDSEED (1u << 18)
+#define bit_cpu_ADX (1u << 19)
+#define bit_cpu_SMAP (1u << 20)
+#define bit_cpu_AVX512_IFMA (1u << 21)
+#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
+#define bit_cpu_CLFLUSHOPT (1u << 23)
+#define bit_cpu_CLWB (1u << 24)
+#define bit_cpu_TRACE (1u << 25)
+#define bit_cpu_AVX512PF (1u << 26)
+#define bit_cpu_AVX512ER (1u << 27)
+#define bit_cpu_AVX512CD (1u << 28)
+#define bit_cpu_SHA (1u << 29)
+#define bit_cpu_AVX512BW (1u << 30)
+#define bit_cpu_AVX512VL (1u << 31)
+
+/* ECX. */
+#define bit_cpu_PREFETCHWT1 (1u << 0)
+#define bit_cpu_AVX512_VBMI (1u << 1)
+#define bit_cpu_UMIP (1u << 2)
+#define bit_cpu_PKU (1u << 3)
+#define bit_cpu_OSPKE (1u << 4)
+#define bit_cpu_WAITPKG (1u << 5)
+#define bit_cpu_AVX512_VBMI2 (1u << 6)
+#define bit_cpu_SHSTK (1u << 7)
+#define bit_cpu_GFNI (1u << 8)
+#define bit_cpu_VAES (1u << 9)
+#define bit_cpu_VPCLMULQDQ (1u << 10)
+#define bit_cpu_AVX512_VNNI (1u << 11)
+#define bit_cpu_AVX512_BITALG (1u << 12)
+#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
+#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
+#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
+#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
+/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
+ instructions in 64-bit mode. */
+#define bit_cpu_RDPID (1u << 22)
+#define bit_cpu_KL (1u << 23)
+#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
+#define bit_cpu_CLDEMOTE (1u << 25)
+#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
+#define bit_cpu_MOVDIRI (1u << 27)
+#define bit_cpu_MOVDIR64B (1u << 28)
+#define bit_cpu_ENQCMD (1u << 29)
+#define bit_cpu_SGX_LC (1u << 30)
+#define bit_cpu_PKS (1u << 31)
+
+/* EDX. */
+#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
+#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
+#define bit_cpu_AVX512_4VNNIW (1u << 2)
+#define bit_cpu_AVX512_4FMAPS (1u << 3)
+#define bit_cpu_FSRM (1u << 4)
+#define bit_cpu_UINTR (1u << 5)
+#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
+#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
+#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
+#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
+#define bit_cpu_MD_CLEAR (1u << 10)
+#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
+#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
+#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
+#define bit_cpu_SERIALIZE (1u << 14)
+#define bit_cpu_HYBRID (1u << 15)
+#define bit_cpu_TSXLDTRK (1u << 16)
+#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
+#define bit_cpu_PCONFIG (1u << 18)
+#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
+#define bit_cpu_IBT (1u << 20)
+#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
+#define bit_cpu_AMX_BF16 (1u << 22)
+#define bit_cpu_AVX512_FP16 (1u << 23)
+#define bit_cpu_AMX_TILE (1u << 24)
+#define bit_cpu_AMX_INT8 (1u << 25)
+#define bit_cpu_IBRS_IBPB (1u << 26)
+#define bit_cpu_STIBP (1u << 27)
+#define bit_cpu_L1D_FLUSH (1u << 28)
+#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
+#define bit_cpu_CORE_CAPABILITIES (1u << 30)
+#define bit_cpu_SSBD (1u << 31)
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define bit_cpu_LAHF64_SAHF64 (1u << 0)
+#define bit_cpu_SVM (1u << 2)
+#define bit_cpu_LZCNT (1u << 5)
+#define bit_cpu_SSE4A (1u << 6)
+#define bit_cpu_PREFETCHW (1u << 8)
+#define bit_cpu_XOP (1u << 11)
+#define bit_cpu_LWP (1u << 15)
+#define bit_cpu_FMA4 (1u << 16)
+#define bit_cpu_TBM (1u << 21)
+
+/* EDX. */
+#define bit_cpu_SYSCALL_SYSRET (1u << 11)
+#define bit_cpu_NX (1u << 20)
+#define bit_cpu_PAGE1GB (1u << 26)
+#define bit_cpu_RDTSCP (1u << 27)
+#define bit_cpu_LM (1u << 29)
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_XSAVEOPT (1u << 0)
+#define bit_cpu_XSAVEC (1u << 1)
+#define bit_cpu_XGETBV_ECX_1 (1u << 2)
+#define bit_cpu_XSAVES (1u << 3)
+#define bit_cpu_XFD (1u << 4)
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define bit_cpu_INVARIANT_TSC (1u << 8)
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define bit_cpu_WBNOINVD (1u << 9)
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define bit_cpu_AVX_VNNI (1u << 4)
+#define bit_cpu_AVX512_BF16 (1u << 5)
+#define bit_cpu_FZLRM (1u << 10)
+#define bit_cpu_FSRS (1u << 11)
+#define bit_cpu_FSRCS (1u << 12)
+#define bit_cpu_HRESET (1u << 22)
+#define bit_cpu_LAM (1u << 26)
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define bit_cpu_AESKLE (1u << 0)
+#define bit_cpu_WIDE_KL (1u << 2)
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define index_cpu_SSE3 CPUID_INDEX_1
+#define index_cpu_PCLMULQDQ CPUID_INDEX_1
+#define index_cpu_DTES64 CPUID_INDEX_1
+#define index_cpu_MONITOR CPUID_INDEX_1
+#define index_cpu_DS_CPL CPUID_INDEX_1
+#define index_cpu_VMX CPUID_INDEX_1
+#define index_cpu_SMX CPUID_INDEX_1
+#define index_cpu_EIST CPUID_INDEX_1
+#define index_cpu_TM2 CPUID_INDEX_1
+#define index_cpu_SSSE3 CPUID_INDEX_1
+#define index_cpu_CNXT_ID CPUID_INDEX_1
+#define index_cpu_SDBG CPUID_INDEX_1
+#define index_cpu_FMA CPUID_INDEX_1
+#define index_cpu_CMPXCHG16B CPUID_INDEX_1
+#define index_cpu_XTPRUPDCTRL CPUID_INDEX_1
+#define index_cpu_PDCM CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_16 CPUID_INDEX_1
+#define index_cpu_PCID CPUID_INDEX_1
+#define index_cpu_DCA CPUID_INDEX_1
+#define index_cpu_SSE4_1 CPUID_INDEX_1
+#define index_cpu_SSE4_2 CPUID_INDEX_1
+#define index_cpu_X2APIC CPUID_INDEX_1
+#define index_cpu_MOVBE CPUID_INDEX_1
+#define index_cpu_POPCNT CPUID_INDEX_1
+#define index_cpu_TSC_DEADLINE CPUID_INDEX_1
+#define index_cpu_AES CPUID_INDEX_1
+#define index_cpu_XSAVE CPUID_INDEX_1
+#define index_cpu_OSXSAVE CPUID_INDEX_1
+#define index_cpu_AVX CPUID_INDEX_1
+#define index_cpu_F16C CPUID_INDEX_1
+#define index_cpu_RDRAND CPUID_INDEX_1
+#define index_cpu_INDEX_1_ECX_31 CPUID_INDEX_1
+
+/* ECX. */
+#define index_cpu_FPU CPUID_INDEX_1
+#define index_cpu_VME CPUID_INDEX_1
+#define index_cpu_DE CPUID_INDEX_1
+#define index_cpu_PSE CPUID_INDEX_1
+#define index_cpu_TSC CPUID_INDEX_1
+#define index_cpu_MSR CPUID_INDEX_1
+#define index_cpu_PAE CPUID_INDEX_1
+#define index_cpu_MCE CPUID_INDEX_1
+#define index_cpu_CX8 CPUID_INDEX_1
+#define index_cpu_APIC CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_10 CPUID_INDEX_1
+#define index_cpu_SEP CPUID_INDEX_1
+#define index_cpu_MTRR CPUID_INDEX_1
+#define index_cpu_PGE CPUID_INDEX_1
+#define index_cpu_MCA CPUID_INDEX_1
+#define index_cpu_CMOV CPUID_INDEX_1
+#define index_cpu_PAT CPUID_INDEX_1
+#define index_cpu_PSE_36 CPUID_INDEX_1
+#define index_cpu_PSN CPUID_INDEX_1
+#define index_cpu_CLFSH CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_20 CPUID_INDEX_1
+#define index_cpu_DS CPUID_INDEX_1
+#define index_cpu_ACPI CPUID_INDEX_1
+#define index_cpu_MMX CPUID_INDEX_1
+#define index_cpu_FXSR CPUID_INDEX_1
+#define index_cpu_SSE CPUID_INDEX_1
+#define index_cpu_SSE2 CPUID_INDEX_1
+#define index_cpu_SS CPUID_INDEX_1
+#define index_cpu_HTT CPUID_INDEX_1
+#define index_cpu_TM CPUID_INDEX_1
+#define index_cpu_INDEX_1_EDX_30 CPUID_INDEX_1
+#define index_cpu_PBE CPUID_INDEX_1
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define index_cpu_FSGSBASE CPUID_INDEX_7
+#define index_cpu_TSC_ADJUST CPUID_INDEX_7
+#define index_cpu_SGX CPUID_INDEX_7
+#define index_cpu_BMI1 CPUID_INDEX_7
+#define index_cpu_HLE CPUID_INDEX_7
+#define index_cpu_AVX2 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_6 CPUID_INDEX_7
+#define index_cpu_SMEP CPUID_INDEX_7
+#define index_cpu_BMI2 CPUID_INDEX_7
+#define index_cpu_ERMS CPUID_INDEX_7
+#define index_cpu_INVPCID CPUID_INDEX_7
+#define index_cpu_RTM CPUID_INDEX_7
+#define index_cpu_RDT_M CPUID_INDEX_7
+#define index_cpu_DEPR_FPU_CS_DS CPUID_INDEX_7
+#define index_cpu_MPX CPUID_INDEX_7
+#define index_cpu_RDT_A CPUID_INDEX_7
+#define index_cpu_AVX512F CPUID_INDEX_7
+#define index_cpu_AVX512DQ CPUID_INDEX_7
+#define index_cpu_RDSEED CPUID_INDEX_7
+#define index_cpu_ADX CPUID_INDEX_7
+#define index_cpu_SMAP CPUID_INDEX_7
+#define index_cpu_AVX512_IFMA CPUID_INDEX_7
+#define index_cpu_INDEX_7_EBX_22 CPUID_INDEX_7
+#define index_cpu_CLFLUSHOPT CPUID_INDEX_7
+#define index_cpu_CLWB CPUID_INDEX_7
+#define index_cpu_TRACE CPUID_INDEX_7
+#define index_cpu_AVX512PF CPUID_INDEX_7
+#define index_cpu_AVX512ER CPUID_INDEX_7
+#define index_cpu_AVX512CD CPUID_INDEX_7
+#define index_cpu_SHA CPUID_INDEX_7
+#define index_cpu_AVX512BW CPUID_INDEX_7
+#define index_cpu_AVX512VL CPUID_INDEX_7
+
+/* ECX. */
+#define index_cpu_PREFETCHWT1 CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI CPUID_INDEX_7
+#define index_cpu_UMIP CPUID_INDEX_7
+#define index_cpu_PKU CPUID_INDEX_7
+#define index_cpu_OSPKE CPUID_INDEX_7
+#define index_cpu_WAITPKG CPUID_INDEX_7
+#define index_cpu_AVX512_VBMI2 CPUID_INDEX_7
+#define index_cpu_SHSTK CPUID_INDEX_7
+#define index_cpu_GFNI CPUID_INDEX_7
+#define index_cpu_VAES CPUID_INDEX_7
+#define index_cpu_VPCLMULQDQ CPUID_INDEX_7
+#define index_cpu_AVX512_VNNI CPUID_INDEX_7
+#define index_cpu_AVX512_BITALG CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_13 CPUID_INDEX_7
+#define index_cpu_AVX512_VPOPCNTDQ CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_15 CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_16 CPUID_INDEX_7
+#define index_cpu_RDPID CPUID_INDEX_7
+#define index_cpu_KL CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_24 CPUID_INDEX_7
+#define index_cpu_CLDEMOTE CPUID_INDEX_7
+#define index_cpu_INDEX_7_ECX_26 CPUID_INDEX_7
+#define index_cpu_MOVDIRI CPUID_INDEX_7
+#define index_cpu_MOVDIR64B CPUID_INDEX_7
+#define index_cpu_ENQCMD CPUID_INDEX_7
+#define index_cpu_SGX_LC CPUID_INDEX_7
+#define index_cpu_PKS CPUID_INDEX_7
+
+/* EDX. */
+#define index_cpu_INDEX_7_EDX_0 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_1 CPUID_INDEX_7
+#define index_cpu_AVX512_4VNNIW CPUID_INDEX_7
+#define index_cpu_AVX512_4FMAPS CPUID_INDEX_7
+#define index_cpu_FSRM CPUID_INDEX_7
+#define index_cpu_UINTR CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_6 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_7 CPUID_INDEX_7
+#define index_cpu_AVX512_VP2INTERSECT CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_9 CPUID_INDEX_7
+#define index_cpu_MD_CLEAR CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_11 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_12 CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_13 CPUID_INDEX_7
+#define index_cpu_SERIALIZE CPUID_INDEX_7
+#define index_cpu_HYBRID CPUID_INDEX_7
+#define index_cpu_TSXLDTRK CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_17 CPUID_INDEX_7
+#define index_cpu_PCONFIG CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_19 CPUID_INDEX_7
+#define index_cpu_IBT CPUID_INDEX_7
+#define index_cpu_INDEX_7_EDX_21 CPUID_INDEX_7
+#define index_cpu_AMX_BF16 CPUID_INDEX_7
+#define index_cpu_AVX512_FP16 CPUID_INDEX_7
+#define index_cpu_AMX_TILE CPUID_INDEX_7
+#define index_cpu_AMX_INT8 CPUID_INDEX_7
+#define index_cpu_IBRS_IBPB CPUID_INDEX_7
+#define index_cpu_STIBP CPUID_INDEX_7
+#define index_cpu_L1D_FLUSH CPUID_INDEX_7
+#define index_cpu_ARCH_CAPABILITIES CPUID_INDEX_7
+#define index_cpu_CORE_CAPABILITIES CPUID_INDEX_7
+#define index_cpu_SSBD CPUID_INDEX_7
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define index_cpu_LAHF64_SAHF64 CPUID_INDEX_80000001
+#define index_cpu_SVM CPUID_INDEX_80000001
+#define index_cpu_LZCNT CPUID_INDEX_80000001
+#define index_cpu_SSE4A CPUID_INDEX_80000001
+#define index_cpu_PREFETCHW CPUID_INDEX_80000001
+#define index_cpu_XOP CPUID_INDEX_80000001
+#define index_cpu_LWP CPUID_INDEX_80000001
+#define index_cpu_FMA4 CPUID_INDEX_80000001
+#define index_cpu_TBM CPUID_INDEX_80000001
+
+/* EDX. */
+#define index_cpu_SYSCALL_SYSRET CPUID_INDEX_80000001
+#define index_cpu_NX CPUID_INDEX_80000001
+#define index_cpu_PAGE1GB CPUID_INDEX_80000001
+#define index_cpu_RDTSCP CPUID_INDEX_80000001
+#define index_cpu_LM CPUID_INDEX_80000001
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define index_cpu_XSAVEOPT CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVEC CPUID_INDEX_D_ECX_1
+#define index_cpu_XGETBV_ECX_1 CPUID_INDEX_D_ECX_1
+#define index_cpu_XSAVES CPUID_INDEX_D_ECX_1
+#define index_cpu_XFD CPUID_INDEX_D_ECX_1
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define index_cpu_INVARIANT_TSC CPUID_INDEX_80000007
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define index_cpu_WBNOINVD CPUID_INDEX_80000008
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define index_cpu_AVX_VNNI CPUID_INDEX_7_ECX_1
+#define index_cpu_AVX512_BF16 CPUID_INDEX_7_ECX_1
+#define index_cpu_FZLRM CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRS CPUID_INDEX_7_ECX_1
+#define index_cpu_FSRCS CPUID_INDEX_7_ECX_1
+#define index_cpu_HRESET CPUID_INDEX_7_ECX_1
+#define index_cpu_LAM CPUID_INDEX_7_ECX_1
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define index_cpu_AESKLE CPUID_INDEX_19
+#define index_cpu_WIDE_KL CPUID_INDEX_19
+
+/* CPUID_INDEX_1. */
+
+/* ECX. */
+#define reg_SSE3 ecx
+#define reg_PCLMULQDQ ecx
+#define reg_DTES64 ecx
+#define reg_MONITOR ecx
+#define reg_DS_CPL ecx
+#define reg_VMX ecx
+#define reg_SMX ecx
+#define reg_EIST ecx
+#define reg_TM2 ecx
+#define reg_SSSE3 ecx
+#define reg_CNXT_ID ecx
+#define reg_SDBG ecx
+#define reg_FMA ecx
+#define reg_CMPXCHG16B ecx
+#define reg_XTPRUPDCTRL ecx
+#define reg_PDCM ecx
+#define reg_INDEX_1_ECX_16 ecx
+#define reg_PCID ecx
+#define reg_DCA ecx
+#define reg_SSE4_1 ecx
+#define reg_SSE4_2 ecx
+#define reg_X2APIC ecx
+#define reg_MOVBE ecx
+#define reg_POPCNT ecx
+#define reg_TSC_DEADLINE ecx
+#define reg_AES ecx
+#define reg_XSAVE ecx
+#define reg_OSXSAVE ecx
+#define reg_AVX ecx
+#define reg_F16C ecx
+#define reg_RDRAND ecx
+#define reg_INDEX_1_ECX_31 ecx
+
+/* EDX. */
+#define reg_FPU edx
+#define reg_VME edx
+#define reg_DE edx
+#define reg_PSE edx
+#define reg_TSC edx
+#define reg_MSR edx
+#define reg_PAE edx
+#define reg_MCE edx
+#define reg_CX8 edx
+#define reg_APIC edx
+#define reg_INDEX_1_EDX_10 edx
+#define reg_SEP edx
+#define reg_MTRR edx
+#define reg_PGE edx
+#define reg_MCA edx
+#define reg_CMOV edx
+#define reg_PAT edx
+#define reg_PSE_36 edx
+#define reg_PSN edx
+#define reg_CLFSH edx
+#define reg_INDEX_1_EDX_20 edx
+#define reg_DS edx
+#define reg_ACPI edx
+#define reg_MMX edx
+#define reg_FXSR edx
+#define reg_SSE edx
+#define reg_SSE2 edx
+#define reg_SS edx
+#define reg_HTT edx
+#define reg_TM edx
+#define reg_INDEX_1_EDX_30 edx
+#define reg_PBE edx
+
+/* CPUID_INDEX_7. */
+
+/* EBX. */
+#define reg_FSGSBASE ebx
+#define reg_TSC_ADJUST ebx
+#define reg_SGX ebx
+#define reg_BMI1 ebx
+#define reg_HLE ebx
+#define reg_BMI2 ebx
+#define reg_AVX2 ebx
+#define reg_INDEX_7_EBX_6 ebx
+#define reg_SMEP ebx
+#define reg_ERMS ebx
+#define reg_INVPCID ebx
+#define reg_RTM ebx
+#define reg_RDT_M ebx
+#define reg_DEPR_FPU_CS_DS ebx
+#define reg_MPX ebx
+#define reg_RDT_A ebx
+#define reg_AVX512F ebx
+#define reg_AVX512DQ ebx
+#define reg_RDSEED ebx
+#define reg_ADX ebx
+#define reg_SMAP ebx
+#define reg_AVX512_IFMA ebx
+#define reg_INDEX_7_EBX_22 ebx
+#define reg_CLFLUSHOPT ebx
+#define reg_CLWB ebx
+#define reg_TRACE ebx
+#define reg_AVX512PF ebx
+#define reg_AVX512ER ebx
+#define reg_AVX512CD ebx
+#define reg_SHA ebx
+#define reg_AVX512BW ebx
+#define reg_AVX512VL ebx
+
+/* ECX. */
+#define reg_PREFETCHWT1 ecx
+#define reg_AVX512_VBMI ecx
+#define reg_UMIP ecx
+#define reg_PKU ecx
+#define reg_OSPKE ecx
+#define reg_WAITPKG ecx
+#define reg_AVX512_VBMI2 ecx
+#define reg_SHSTK ecx
+#define reg_GFNI ecx
+#define reg_VAES ecx
+#define reg_VPCLMULQDQ ecx
+#define reg_AVX512_VNNI ecx
+#define reg_AVX512_BITALG ecx
+#define reg_INDEX_7_ECX_13 ecx
+#define reg_AVX512_VPOPCNTDQ ecx
+#define reg_INDEX_7_ECX_15 ecx
+#define reg_INDEX_7_ECX_16 ecx
+#define reg_RDPID ecx
+#define reg_KL ecx
+#define reg_INDEX_7_ECX_24 ecx
+#define reg_CLDEMOTE ecx
+#define reg_INDEX_7_ECX_26 ecx
+#define reg_MOVDIRI ecx
+#define reg_MOVDIR64B ecx
+#define reg_ENQCMD ecx
+#define reg_SGX_LC ecx
+#define reg_PKS ecx
+
+/* EDX. */
+#define reg_INDEX_7_EDX_0 edx
+#define reg_INDEX_7_EDX_1 edx
+#define reg_AVX512_4VNNIW edx
+#define reg_AVX512_4FMAPS edx
+#define reg_FSRM edx
+#define reg_UINTR edx
+#define reg_INDEX_7_EDX_6 edx
+#define reg_INDEX_7_EDX_7 edx
+#define reg_AVX512_VP2INTERSECT edx
+#define reg_INDEX_7_EDX_9 edx
+#define reg_MD_CLEAR edx
+#define reg_INDEX_7_EDX_11 edx
+#define reg_INDEX_7_EDX_12 edx
+#define reg_INDEX_7_EDX_13 edx
+#define reg_SERIALIZE edx
+#define reg_HYBRID edx
+#define reg_TSXLDTRK edx
+#define reg_INDEX_7_EDX_17 edx
+#define reg_PCONFIG edx
+#define reg_INDEX_7_EDX_19 edx
+#define reg_IBT edx
+#define reg_INDEX_7_EDX_21 edx
+#define reg_AMX_BF16 edx
+#define reg_AVX512_FP16 edx
+#define reg_AMX_TILE edx
+#define reg_AMX_INT8 edx
+#define reg_IBRS_IBPB edx
+#define reg_STIBP edx
+#define reg_L1D_FLUSH edx
+#define reg_ARCH_CAPABILITIES edx
+#define reg_CORE_CAPABILITIES edx
+#define reg_SSBD edx
+
+/* CPUID_INDEX_80000001. */
+
+/* ECX. */
+#define reg_LAHF64_SAHF64 ecx
+#define reg_SVM ecx
+#define reg_LZCNT ecx
+#define reg_SSE4A ecx
+#define reg_PREFETCHW ecx
+#define reg_XOP ecx
+#define reg_LWP ecx
+#define reg_FMA4 ecx
+#define reg_TBM ecx
+
+/* EDX. */
+#define reg_SYSCALL_SYSRET edx
+#define reg_NX edx
+#define reg_PAGE1GB edx
+#define reg_RDTSCP edx
+#define reg_LM edx
+
+/* CPUID_INDEX_D_ECX_1. */
+
+/* EAX. */
+#define reg_XSAVEOPT eax
+#define reg_XSAVEC eax
+#define reg_XGETBV_ECX_1 eax
+#define reg_XSAVES eax
+#define reg_XFD eax
+
+/* CPUID_INDEX_80000007. */
+
+/* EDX. */
+#define reg_INVARIANT_TSC edx
+
+/* CPUID_INDEX_80000008. */
+
+/* EBX. */
+#define reg_WBNOINVD ebx
+
+/* CPUID_INDEX_7_ECX_1. */
+
+/* EAX. */
+#define reg_AVX_VNNI eax
+#define reg_AVX512_BF16 eax
+#define reg_FZLRM eax
+#define reg_FSRS eax
+#define reg_FSRCS eax
+#define reg_HRESET eax
+#define reg_LAM eax
+
+/* CPUID_INDEX_19. */
+
+/* EBX. */
+#define reg_AESKLE ebx
+#define reg_WIDE_KL ebx
+
/* PREFERRED_FEATURE_INDEX_1. */
#define bit_arch_I586 (1u << 0)
#define bit_arch_I686 (1u << 1)
@@ -118,10 +801,50 @@ enum
#define bit_XTILECFG_state (1u << 17)
#define bit_XTILEDATA_state (1u << 18)
+enum cpu_features_kind
+{
+ arch_kind_unknown = 0,
+ arch_kind_intel,
+ arch_kind_amd,
+ arch_kind_zhaoxin,
+ arch_kind_other
+};
+
+struct cpu_features_basic
+{
+ enum cpu_features_kind kind;
+ int max_cpuid;
+ unsigned int family;
+ unsigned int model;
+ unsigned int stepping;
+};
+
+struct cpuid_registers
+{
+ unsigned int eax;
+ unsigned int ebx;
+ unsigned int ecx;
+ unsigned int edx;
+};
+
+struct cpuid_feature_internal
+{
+ union
+ {
+ unsigned int cpuid_array[4];
+ struct cpuid_registers cpuid;
+ };
+ union
+ {
+ unsigned int usable_array[4];
+ struct cpuid_registers usable;
+ };
+};
+
struct cpu_features
{
struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
+ struct cpuid_feature_internal features[CPUID_INDEX_MAX];
unsigned int preferred[PREFERRED_FEATURE_INDEX_MAX];
/* X86 micro-architecture ISA levels. */
unsigned int isa_1;
@@ -173,10 +896,16 @@ struct cpu_features
unsigned long int level4_cache_size;
};
+/* Get a pointer to the CPU features structure. */
+extern const struct cpu_features *_dl_x86_get_cpu_features (void)
+ __attribute__ ((pure));
+
+#define __get_cpu_features() _dl_x86_get_cpu_features()
+
#if defined (_LIBC) && !IS_IN (nonlib)
/* Unused for x86. */
# define INIT_ARCH()
-# define __x86_get_cpu_features(max) (&GLRO(dl_x86_cpu_features))
+# define _dl_x86_get_cpu_features() (&GLRO(dl_x86_cpu_features))
extern void _dl_x86_init_cpu_features (void) attribute_hidden;
#endif
diff --git a/sysdeps/x86/sys/platform/x86.h b/sysdeps/x86/sys/platform/x86.h
index 7c964ff907..e930219944 100644
--- a/sysdeps/x86/sys/platform/x86.h
+++ b/sysdeps/x86/sys/platform/x86.h
@@ -19,760 +19,47 @@
#ifndef _SYS_PLATFORM_X86_H
#define _SYS_PLATFORM_X86_H
-enum
-{
- COMMON_CPUID_INDEX_1 = 0,
- COMMON_CPUID_INDEX_7,
- COMMON_CPUID_INDEX_80000001,
- COMMON_CPUID_INDEX_D_ECX_1,
- COMMON_CPUID_INDEX_80000007,
- COMMON_CPUID_INDEX_80000008,
- COMMON_CPUID_INDEX_7_ECX_1,
- COMMON_CPUID_INDEX_19,
- /* Keep the following line at the end. */
- COMMON_CPUID_INDEX_MAX
-};
+#include <features.h>
+#include <stdbool.h>
+#include <bits/platform/x86.h>
-struct cpuid_registers
-{
- unsigned int eax;
- unsigned int ebx;
- unsigned int ecx;
- unsigned int edx;
-};
+__BEGIN_DECLS
-struct cpuid_features
-{
- struct cpuid_registers cpuid;
- struct cpuid_registers usable;
-};
+/* Get a pointer to the CPU feature structure. */
+extern const struct cpuid_feature *__x86_get_cpuid_feature_leaf (unsigned int)
+ __attribute__ ((pure));
-enum cpu_features_kind
+static __inline__ _Bool
+x86_cpu_has_feature (unsigned int __index)
{
- arch_kind_unknown = 0,
- arch_kind_intel,
- arch_kind_amd,
- arch_kind_zhaoxin,
- arch_kind_other
-};
-
-struct cpu_features_basic
+ const struct cpuid_feature *__ptr = __x86_get_cpuid_feature_leaf
+ (__index / (8 * sizeof (unsigned int) * 4));
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
+
+ return __ptr->cpuid_array[__reg] & (1 << __bit);
+}
+
+static __inline__ _Bool
+x86_cpu_is_usable (unsigned int __index)
{
- enum cpu_features_kind kind;
- int max_cpuid;
- unsigned int family;
- unsigned int model;
- unsigned int stepping;
-};
+ const struct cpuid_feature *__ptr = __x86_get_cpuid_feature_leaf
+ (__index / (8 * sizeof (unsigned int) * 4));
+ unsigned int __reg
+ = __index & (8 * sizeof (unsigned int) * 4 - 1);
+ unsigned int __bit = __reg & (8 * sizeof (unsigned int) - 1);
+ __reg /= 8 * sizeof (unsigned int);
-struct cpu_features
-{
- struct cpu_features_basic basic;
- struct cpuid_features features[COMMON_CPUID_INDEX_MAX];
-};
-
-/* Get a pointer to the CPU features structure. */
-extern const struct cpu_features *__x86_get_cpu_features (unsigned int)
- __attribute__ ((const));
-
-#define CPU_FEATURE_CHECK_P(ptr, name, check) \
- ((ptr->features[index_cpu_##name].check.reg_##name \
- & bit_cpu_##name) != 0)
-#define CPU_FEATURE_CPU_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, cpuid)
-#define CPU_FEATURE_USABLE_P(ptr, name) \
- CPU_FEATURE_CHECK_P (ptr, name, usable)
+ return __ptr->usable_array[__reg] & (1 << __bit);
+}
/* HAS_CPU_FEATURE evaluates to true if CPU supports the feature. */
-#define HAS_CPU_FEATURE(name) \
- (__extension__ \
- ({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_CPU_P (__ptr, name); }))
+#define HAS_CPU_FEATURE(name) x86_cpu_has_feature (x86_cpu_##name)
/* CPU_FEATURE_USABLE evaluates to true if the feature is usable. */
-#define CPU_FEATURE_USABLE(name) \
- (__extension__ \
- ({ const struct cpu_features *__ptr = \
- __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX); \
- __ptr && CPU_FEATURE_USABLE_P (__ptr, name); }))
-
-/* CPU features. */
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define bit_cpu_SSE3 (1u << 0)
-#define bit_cpu_PCLMULQDQ (1u << 1)
-#define bit_cpu_DTES64 (1u << 2)
-#define bit_cpu_MONITOR (1u << 3)
-#define bit_cpu_DS_CPL (1u << 4)
-#define bit_cpu_VMX (1u << 5)
-#define bit_cpu_SMX (1u << 6)
-#define bit_cpu_EIST (1u << 7)
-#define bit_cpu_TM2 (1u << 8)
-#define bit_cpu_SSSE3 (1u << 9)
-#define bit_cpu_CNXT_ID (1u << 10)
-#define bit_cpu_SDBG (1u << 11)
-#define bit_cpu_FMA (1u << 12)
-#define bit_cpu_CMPXCHG16B (1u << 13)
-#define bit_cpu_XTPRUPDCTRL (1u << 14)
-#define bit_cpu_PDCM (1u << 15)
-#define bit_cpu_INDEX_1_ECX_16 (1u << 16)
-#define bit_cpu_PCID (1u << 17)
-#define bit_cpu_DCA (1u << 18)
-#define bit_cpu_SSE4_1 (1u << 19)
-#define bit_cpu_SSE4_2 (1u << 20)
-#define bit_cpu_X2APIC (1u << 21)
-#define bit_cpu_MOVBE (1u << 22)
-#define bit_cpu_POPCNT (1u << 23)
-#define bit_cpu_TSC_DEADLINE (1u << 24)
-#define bit_cpu_AES (1u << 25)
-#define bit_cpu_XSAVE (1u << 26)
-#define bit_cpu_OSXSAVE (1u << 27)
-#define bit_cpu_AVX (1u << 28)
-#define bit_cpu_F16C (1u << 29)
-#define bit_cpu_RDRAND (1u << 30)
-#define bit_cpu_INDEX_1_ECX_31 (1u << 31)
-
-/* EDX. */
-#define bit_cpu_FPU (1u << 0)
-#define bit_cpu_VME (1u << 1)
-#define bit_cpu_DE (1u << 2)
-#define bit_cpu_PSE (1u << 3)
-#define bit_cpu_TSC (1u << 4)
-#define bit_cpu_MSR (1u << 5)
-#define bit_cpu_PAE (1u << 6)
-#define bit_cpu_MCE (1u << 7)
-#define bit_cpu_CX8 (1u << 8)
-#define bit_cpu_APIC (1u << 9)
-#define bit_cpu_INDEX_1_EDX_10 (1u << 10)
-#define bit_cpu_SEP (1u << 11)
-#define bit_cpu_MTRR (1u << 12)
-#define bit_cpu_PGE (1u << 13)
-#define bit_cpu_MCA (1u << 14)
-#define bit_cpu_CMOV (1u << 15)
-#define bit_cpu_PAT (1u << 16)
-#define bit_cpu_PSE_36 (1u << 17)
-#define bit_cpu_PSN (1u << 18)
-#define bit_cpu_CLFSH (1u << 19)
-#define bit_cpu_INDEX_1_EDX_20 (1u << 20)
-#define bit_cpu_DS (1u << 21)
-#define bit_cpu_ACPI (1u << 22)
-#define bit_cpu_MMX (1u << 23)
-#define bit_cpu_FXSR (1u << 24)
-#define bit_cpu_SSE (1u << 25)
-#define bit_cpu_SSE2 (1u << 26)
-#define bit_cpu_SS (1u << 27)
-#define bit_cpu_HTT (1u << 28)
-#define bit_cpu_TM (1u << 29)
-#define bit_cpu_INDEX_1_EDX_30 (1u << 30)
-#define bit_cpu_PBE (1u << 31)
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define bit_cpu_FSGSBASE (1u << 0)
-#define bit_cpu_TSC_ADJUST (1u << 1)
-#define bit_cpu_SGX (1u << 2)
-#define bit_cpu_BMI1 (1u << 3)
-#define bit_cpu_HLE (1u << 4)
-#define bit_cpu_AVX2 (1u << 5)
-#define bit_cpu_INDEX_7_EBX_6 (1u << 6)
-#define bit_cpu_SMEP (1u << 7)
-#define bit_cpu_BMI2 (1u << 8)
-#define bit_cpu_ERMS (1u << 9)
-#define bit_cpu_INVPCID (1u << 10)
-#define bit_cpu_RTM (1u << 11)
-#define bit_cpu_RDT_M (1u << 12)
-#define bit_cpu_DEPR_FPU_CS_DS (1u << 13)
-#define bit_cpu_MPX (1u << 14)
-#define bit_cpu_RDT_A (1u << 15)
-#define bit_cpu_AVX512F (1u << 16)
-#define bit_cpu_AVX512DQ (1u << 17)
-#define bit_cpu_RDSEED (1u << 18)
-#define bit_cpu_ADX (1u << 19)
-#define bit_cpu_SMAP (1u << 20)
-#define bit_cpu_AVX512_IFMA (1u << 21)
-#define bit_cpu_INDEX_7_EBX_22 (1u << 22)
-#define bit_cpu_CLFLUSHOPT (1u << 23)
-#define bit_cpu_CLWB (1u << 24)
-#define bit_cpu_TRACE (1u << 25)
-#define bit_cpu_AVX512PF (1u << 26)
-#define bit_cpu_AVX512ER (1u << 27)
-#define bit_cpu_AVX512CD (1u << 28)
-#define bit_cpu_SHA (1u << 29)
-#define bit_cpu_AVX512BW (1u << 30)
-#define bit_cpu_AVX512VL (1u << 31)
-
-/* ECX. */
-#define bit_cpu_PREFETCHWT1 (1u << 0)
-#define bit_cpu_AVX512_VBMI (1u << 1)
-#define bit_cpu_UMIP (1u << 2)
-#define bit_cpu_PKU (1u << 3)
-#define bit_cpu_OSPKE (1u << 4)
-#define bit_cpu_WAITPKG (1u << 5)
-#define bit_cpu_AVX512_VBMI2 (1u << 6)
-#define bit_cpu_SHSTK (1u << 7)
-#define bit_cpu_GFNI (1u << 8)
-#define bit_cpu_VAES (1u << 9)
-#define bit_cpu_VPCLMULQDQ (1u << 10)
-#define bit_cpu_AVX512_VNNI (1u << 11)
-#define bit_cpu_AVX512_BITALG (1u << 12)
-#define bit_cpu_INDEX_7_ECX_13 (1u << 13)
-#define bit_cpu_AVX512_VPOPCNTDQ (1u << 14)
-#define bit_cpu_INDEX_7_ECX_15 (1u << 15)
-#define bit_cpu_INDEX_7_ECX_16 (1u << 16)
-/* Note: Bits 17-21: The value of MAWAU used by the BNDLDX and BNDSTX
- instructions in 64-bit mode. */
-#define bit_cpu_RDPID (1u << 22)
-#define bit_cpu_KL (1u << 23)
-#define bit_cpu_INDEX_7_ECX_24 (1u << 24)
-#define bit_cpu_CLDEMOTE (1u << 25)
-#define bit_cpu_INDEX_7_ECX_26 (1u << 26)
-#define bit_cpu_MOVDIRI (1u << 27)
-#define bit_cpu_MOVDIR64B (1u << 28)
-#define bit_cpu_ENQCMD (1u << 29)
-#define bit_cpu_SGX_LC (1u << 30)
-#define bit_cpu_PKS (1u << 31)
-
-/* EDX. */
-#define bit_cpu_INDEX_7_EDX_0 (1u << 0)
-#define bit_cpu_INDEX_7_EDX_1 (1u << 1)
-#define bit_cpu_AVX512_4VNNIW (1u << 2)
-#define bit_cpu_AVX512_4FMAPS (1u << 3)
-#define bit_cpu_FSRM (1u << 4)
-#define bit_cpu_UINTR (1u << 5)
-#define bit_cpu_INDEX_7_EDX_6 (1u << 6)
-#define bit_cpu_INDEX_7_EDX_7 (1u << 7)
-#define bit_cpu_AVX512_VP2INTERSECT (1u << 8)
-#define bit_cpu_INDEX_7_EDX_9 (1u << 9)
-#define bit_cpu_MD_CLEAR (1u << 10)
-#define bit_cpu_INDEX_7_EDX_11 (1u << 11)
-#define bit_cpu_INDEX_7_EDX_12 (1u << 12)
-#define bit_cpu_INDEX_7_EDX_13 (1u << 13)
-#define bit_cpu_SERIALIZE (1u << 14)
-#define bit_cpu_HYBRID (1u << 15)
-#define bit_cpu_TSXLDTRK (1u << 16)
-#define bit_cpu_INDEX_7_EDX_17 (1u << 17)
-#define bit_cpu_PCONFIG (1u << 18)
-#define bit_cpu_INDEX_7_EDX_19 (1u << 19)
-#define bit_cpu_IBT (1u << 20)
-#define bit_cpu_INDEX_7_EDX_21 (1u << 21)
-#define bit_cpu_AMX_BF16 (1u << 22)
-#define bit_cpu_AVX512_FP16 (1u << 23)
-#define bit_cpu_AMX_TILE (1u << 24)
-#define bit_cpu_AMX_INT8 (1u << 25)
-#define bit_cpu_IBRS_IBPB (1u << 26)
-#define bit_cpu_STIBP (1u << 27)
-#define bit_cpu_L1D_FLUSH (1u << 28)
-#define bit_cpu_ARCH_CAPABILITIES (1u << 29)
-#define bit_cpu_CORE_CAPABILITIES (1u << 30)
-#define bit_cpu_SSBD (1u << 31)
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define bit_cpu_LAHF64_SAHF64 (1u << 0)
-#define bit_cpu_SVM (1u << 2)
-#define bit_cpu_LZCNT (1u << 5)
-#define bit_cpu_SSE4A (1u << 6)
-#define bit_cpu_PREFETCHW (1u << 8)
-#define bit_cpu_XOP (1u << 11)
-#define bit_cpu_LWP (1u << 15)
-#define bit_cpu_FMA4 (1u << 16)
-#define bit_cpu_TBM (1u << 21)
-
-/* EDX. */
-#define bit_cpu_SYSCALL_SYSRET (1u << 11)
-#define bit_cpu_NX (1u << 20)
-#define bit_cpu_PAGE1GB (1u << 26)
-#define bit_cpu_RDTSCP (1u << 27)
-#define bit_cpu_LM (1u << 29)
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_XSAVEOPT (1u << 0)
-#define bit_cpu_XSAVEC (1u << 1)
-#define bit_cpu_XGETBV_ECX_1 (1u << 2)
-#define bit_cpu_XSAVES (1u << 3)
-#define bit_cpu_XFD (1u << 4)
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define bit_cpu_INVARIANT_TSC (1u << 8)
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define bit_cpu_WBNOINVD (1u << 9)
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define bit_cpu_AVX_VNNI (1u << 4)
-#define bit_cpu_AVX512_BF16 (1u << 5)
-#define bit_cpu_FZLRM (1u << 10)
-#define bit_cpu_FSRS (1u << 11)
-#define bit_cpu_FSRCS (1u << 12)
-#define bit_cpu_HRESET (1u << 22)
-#define bit_cpu_LAM (1u << 26)
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define bit_cpu_AESKLE (1u << 0)
-#define bit_cpu_WIDE_KL (1u << 2)
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define index_cpu_SSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_PCLMULQDQ COMMON_CPUID_INDEX_1
-#define index_cpu_DTES64 COMMON_CPUID_INDEX_1
-#define index_cpu_MONITOR COMMON_CPUID_INDEX_1
-#define index_cpu_DS_CPL COMMON_CPUID_INDEX_1
-#define index_cpu_VMX COMMON_CPUID_INDEX_1
-#define index_cpu_SMX COMMON_CPUID_INDEX_1
-#define index_cpu_EIST COMMON_CPUID_INDEX_1
-#define index_cpu_TM2 COMMON_CPUID_INDEX_1
-#define index_cpu_SSSE3 COMMON_CPUID_INDEX_1
-#define index_cpu_CNXT_ID COMMON_CPUID_INDEX_1
-#define index_cpu_SDBG COMMON_CPUID_INDEX_1
-#define index_cpu_FMA COMMON_CPUID_INDEX_1
-#define index_cpu_CMPXCHG16B COMMON_CPUID_INDEX_1
-#define index_cpu_XTPRUPDCTRL COMMON_CPUID_INDEX_1
-#define index_cpu_PDCM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_16 COMMON_CPUID_INDEX_1
-#define index_cpu_PCID COMMON_CPUID_INDEX_1
-#define index_cpu_DCA COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_1 COMMON_CPUID_INDEX_1
-#define index_cpu_SSE4_2 COMMON_CPUID_INDEX_1
-#define index_cpu_X2APIC COMMON_CPUID_INDEX_1
-#define index_cpu_MOVBE COMMON_CPUID_INDEX_1
-#define index_cpu_POPCNT COMMON_CPUID_INDEX_1
-#define index_cpu_TSC_DEADLINE COMMON_CPUID_INDEX_1
-#define index_cpu_AES COMMON_CPUID_INDEX_1
-#define index_cpu_XSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_OSXSAVE COMMON_CPUID_INDEX_1
-#define index_cpu_AVX COMMON_CPUID_INDEX_1
-#define index_cpu_F16C COMMON_CPUID_INDEX_1
-#define index_cpu_RDRAND COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_ECX_31 COMMON_CPUID_INDEX_1
-
-/* ECX. */
-#define index_cpu_FPU COMMON_CPUID_INDEX_1
-#define index_cpu_VME COMMON_CPUID_INDEX_1
-#define index_cpu_DE COMMON_CPUID_INDEX_1
-#define index_cpu_PSE COMMON_CPUID_INDEX_1
-#define index_cpu_TSC COMMON_CPUID_INDEX_1
-#define index_cpu_MSR COMMON_CPUID_INDEX_1
-#define index_cpu_PAE COMMON_CPUID_INDEX_1
-#define index_cpu_MCE COMMON_CPUID_INDEX_1
-#define index_cpu_CX8 COMMON_CPUID_INDEX_1
-#define index_cpu_APIC COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_10 COMMON_CPUID_INDEX_1
-#define index_cpu_SEP COMMON_CPUID_INDEX_1
-#define index_cpu_MTRR COMMON_CPUID_INDEX_1
-#define index_cpu_PGE COMMON_CPUID_INDEX_1
-#define index_cpu_MCA COMMON_CPUID_INDEX_1
-#define index_cpu_CMOV COMMON_CPUID_INDEX_1
-#define index_cpu_PAT COMMON_CPUID_INDEX_1
-#define index_cpu_PSE_36 COMMON_CPUID_INDEX_1
-#define index_cpu_PSN COMMON_CPUID_INDEX_1
-#define index_cpu_CLFSH COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_20 COMMON_CPUID_INDEX_1
-#define index_cpu_DS COMMON_CPUID_INDEX_1
-#define index_cpu_ACPI COMMON_CPUID_INDEX_1
-#define index_cpu_MMX COMMON_CPUID_INDEX_1
-#define index_cpu_FXSR COMMON_CPUID_INDEX_1
-#define index_cpu_SSE COMMON_CPUID_INDEX_1
-#define index_cpu_SSE2 COMMON_CPUID_INDEX_1
-#define index_cpu_SS COMMON_CPUID_INDEX_1
-#define index_cpu_HTT COMMON_CPUID_INDEX_1
-#define index_cpu_TM COMMON_CPUID_INDEX_1
-#define index_cpu_INDEX_1_EDX_30 COMMON_CPUID_INDEX_1
-#define index_cpu_PBE COMMON_CPUID_INDEX_1
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define index_cpu_FSGSBASE COMMON_CPUID_INDEX_7
-#define index_cpu_TSC_ADJUST COMMON_CPUID_INDEX_7
-#define index_cpu_SGX COMMON_CPUID_INDEX_7
-#define index_cpu_BMI1 COMMON_CPUID_INDEX_7
-#define index_cpu_HLE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX2 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_SMEP COMMON_CPUID_INDEX_7
-#define index_cpu_BMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_ERMS COMMON_CPUID_INDEX_7
-#define index_cpu_INVPCID COMMON_CPUID_INDEX_7
-#define index_cpu_RTM COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_M COMMON_CPUID_INDEX_7
-#define index_cpu_DEPR_FPU_CS_DS COMMON_CPUID_INDEX_7
-#define index_cpu_MPX COMMON_CPUID_INDEX_7
-#define index_cpu_RDT_A COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512F COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512DQ COMMON_CPUID_INDEX_7
-#define index_cpu_RDSEED COMMON_CPUID_INDEX_7
-#define index_cpu_ADX COMMON_CPUID_INDEX_7
-#define index_cpu_SMAP COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_IFMA COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EBX_22 COMMON_CPUID_INDEX_7
-#define index_cpu_CLFLUSHOPT COMMON_CPUID_INDEX_7
-#define index_cpu_CLWB COMMON_CPUID_INDEX_7
-#define index_cpu_TRACE COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512PF COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512ER COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512CD COMMON_CPUID_INDEX_7
-#define index_cpu_SHA COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512BW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512VL COMMON_CPUID_INDEX_7
-
-/* ECX. */
-#define index_cpu_PREFETCHWT1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI COMMON_CPUID_INDEX_7
-#define index_cpu_UMIP COMMON_CPUID_INDEX_7
-#define index_cpu_PKU COMMON_CPUID_INDEX_7
-#define index_cpu_OSPKE COMMON_CPUID_INDEX_7
-#define index_cpu_WAITPKG COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VBMI2 COMMON_CPUID_INDEX_7
-#define index_cpu_SHSTK COMMON_CPUID_INDEX_7
-#define index_cpu_GFNI COMMON_CPUID_INDEX_7
-#define index_cpu_VAES COMMON_CPUID_INDEX_7
-#define index_cpu_VPCLMULQDQ COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VNNI COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_BITALG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VPOPCNTDQ COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_15 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_16 COMMON_CPUID_INDEX_7
-#define index_cpu_RDPID COMMON_CPUID_INDEX_7
-#define index_cpu_KL COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_24 COMMON_CPUID_INDEX_7
-#define index_cpu_CLDEMOTE COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_ECX_26 COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIRI COMMON_CPUID_INDEX_7
-#define index_cpu_MOVDIR64B COMMON_CPUID_INDEX_7
-#define index_cpu_ENQCMD COMMON_CPUID_INDEX_7
-#define index_cpu_SGX_LC COMMON_CPUID_INDEX_7
-#define index_cpu_PKS COMMON_CPUID_INDEX_7
-
-/* EDX. */
-#define index_cpu_INDEX_7_EDX_0 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_1 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4VNNIW COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_4FMAPS COMMON_CPUID_INDEX_7
-#define index_cpu_FSRM COMMON_CPUID_INDEX_7
-#define index_cpu_UINTR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_6 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_7 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_VP2INTERSECT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_9 COMMON_CPUID_INDEX_7
-#define index_cpu_MD_CLEAR COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_11 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_12 COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_13 COMMON_CPUID_INDEX_7
-#define index_cpu_SERIALIZE COMMON_CPUID_INDEX_7
-#define index_cpu_HYBRID COMMON_CPUID_INDEX_7
-#define index_cpu_TSXLDTRK COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_17 COMMON_CPUID_INDEX_7
-#define index_cpu_PCONFIG COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_19 COMMON_CPUID_INDEX_7
-#define index_cpu_IBT COMMON_CPUID_INDEX_7
-#define index_cpu_INDEX_7_EDX_21 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_BF16 COMMON_CPUID_INDEX_7
-#define index_cpu_AVX512_FP16 COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_TILE COMMON_CPUID_INDEX_7
-#define index_cpu_AMX_INT8 COMMON_CPUID_INDEX_7
-#define index_cpu_IBRS_IBPB COMMON_CPUID_INDEX_7
-#define index_cpu_STIBP COMMON_CPUID_INDEX_7
-#define index_cpu_L1D_FLUSH COMMON_CPUID_INDEX_7
-#define index_cpu_ARCH_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_CORE_CAPABILITIES COMMON_CPUID_INDEX_7
-#define index_cpu_SSBD COMMON_CPUID_INDEX_7
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define index_cpu_LAHF64_SAHF64 COMMON_CPUID_INDEX_80000001
-#define index_cpu_SVM COMMON_CPUID_INDEX_80000001
-#define index_cpu_LZCNT COMMON_CPUID_INDEX_80000001
-#define index_cpu_SSE4A COMMON_CPUID_INDEX_80000001
-#define index_cpu_PREFETCHW COMMON_CPUID_INDEX_80000001
-#define index_cpu_XOP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LWP COMMON_CPUID_INDEX_80000001
-#define index_cpu_FMA4 COMMON_CPUID_INDEX_80000001
-#define index_cpu_TBM COMMON_CPUID_INDEX_80000001
-
-/* EDX. */
-#define index_cpu_SYSCALL_SYSRET COMMON_CPUID_INDEX_80000001
-#define index_cpu_NX COMMON_CPUID_INDEX_80000001
-#define index_cpu_PAGE1GB COMMON_CPUID_INDEX_80000001
-#define index_cpu_RDTSCP COMMON_CPUID_INDEX_80000001
-#define index_cpu_LM COMMON_CPUID_INDEX_80000001
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define index_cpu_XSAVEOPT COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVEC COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XGETBV_ECX_1 COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XSAVES COMMON_CPUID_INDEX_D_ECX_1
-#define index_cpu_XFD COMMON_CPUID_INDEX_D_ECX_1
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define index_cpu_INVARIANT_TSC COMMON_CPUID_INDEX_80000007
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define index_cpu_WBNOINVD COMMON_CPUID_INDEX_80000008
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define index_cpu_AVX_VNNI COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_AVX512_BF16 COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FZLRM COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_FSRCS COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_HRESET COMMON_CPUID_INDEX_7_ECX_1
-#define index_cpu_LAM COMMON_CPUID_INDEX_7_ECX_1
-
-/* COMMON_CPUID_INDEX_19. */
-
-/* EBX. */
-#define index_cpu_AESKLE COMMON_CPUID_INDEX_19
-#define index_cpu_WIDE_KL COMMON_CPUID_INDEX_19
-
-/* COMMON_CPUID_INDEX_1. */
-
-/* ECX. */
-#define reg_SSE3 ecx
-#define reg_PCLMULQDQ ecx
-#define reg_DTES64 ecx
-#define reg_MONITOR ecx
-#define reg_DS_CPL ecx
-#define reg_VMX ecx
-#define reg_SMX ecx
-#define reg_EIST ecx
-#define reg_TM2 ecx
-#define reg_SSSE3 ecx
-#define reg_CNXT_ID ecx
-#define reg_SDBG ecx
-#define reg_FMA ecx
-#define reg_CMPXCHG16B ecx
-#define reg_XTPRUPDCTRL ecx
-#define reg_PDCM ecx
-#define reg_INDEX_1_ECX_16 ecx
-#define reg_PCID ecx
-#define reg_DCA ecx
-#define reg_SSE4_1 ecx
-#define reg_SSE4_2 ecx
-#define reg_X2APIC ecx
-#define reg_MOVBE ecx
-#define reg_POPCNT ecx
-#define reg_TSC_DEADLINE ecx
-#define reg_AES ecx
-#define reg_XSAVE ecx
-#define reg_OSXSAVE ecx
-#define reg_AVX ecx
-#define reg_F16C ecx
-#define reg_RDRAND ecx
-#define reg_INDEX_1_ECX_31 ecx
-
-/* EDX. */
-#define reg_FPU edx
-#define reg_VME edx
-#define reg_DE edx
-#define reg_PSE edx
-#define reg_TSC edx
-#define reg_MSR edx
-#define reg_PAE edx
-#define reg_MCE edx
-#define reg_CX8 edx
-#define reg_APIC edx
-#define reg_INDEX_1_EDX_10 edx
-#define reg_SEP edx
-#define reg_MTRR edx
-#define reg_PGE edx
-#define reg_MCA edx
-#define reg_CMOV edx
-#define reg_PAT edx
-#define reg_PSE_36 edx
-#define reg_PSN edx
-#define reg_CLFSH edx
-#define reg_INDEX_1_EDX_20 edx
-#define reg_DS edx
-#define reg_ACPI edx
-#define reg_MMX edx
-#define reg_FXSR edx
-#define reg_SSE edx
-#define reg_SSE2 edx
-#define reg_SS edx
-#define reg_HTT edx
-#define reg_TM edx
-#define reg_INDEX_1_EDX_30 edx
-#define reg_PBE edx
-
-/* COMMON_CPUID_INDEX_7. */
-
-/* EBX. */
-#define reg_FSGSBASE ebx
-#define reg_TSC_ADJUST ebx
-#define reg_SGX ebx
-#define reg_BMI1 ebx
-#define reg_HLE ebx
-#define reg_BMI2 ebx
-#define reg_AVX2 ebx
-#define reg_INDEX_7_EBX_6 ebx
-#define reg_SMEP ebx
-#define reg_ERMS ebx
-#define reg_INVPCID ebx
-#define reg_RTM ebx
-#define reg_RDT_M ebx
-#define reg_DEPR_FPU_CS_DS ebx
-#define reg_MPX ebx
-#define reg_RDT_A ebx
-#define reg_AVX512F ebx
-#define reg_AVX512DQ ebx
-#define reg_RDSEED ebx
-#define reg_ADX ebx
-#define reg_SMAP ebx
-#define reg_AVX512_IFMA ebx
-#define reg_INDEX_7_EBX_22 ebx
-#define reg_CLFLUSHOPT ebx
-#define reg_CLWB ebx
-#define reg_TRACE ebx
-#define reg_AVX512PF ebx
-#define reg_AVX512ER ebx
-#define reg_AVX512CD ebx
-#define reg_SHA ebx
-#define reg_AVX512BW ebx
-#define reg_AVX512VL ebx
-
-/* ECX. */
-#define reg_PREFETCHWT1 ecx
-#define reg_AVX512_VBMI ecx
-#define reg_UMIP ecx
-#define reg_PKU ecx
-#define reg_OSPKE ecx
-#define reg_WAITPKG ecx
-#define reg_AVX512_VBMI2 ecx
-#define reg_SHSTK ecx
-#define reg_GFNI ecx
-#define reg_VAES ecx
-#define reg_VPCLMULQDQ ecx
-#define reg_AVX512_VNNI ecx
-#define reg_AVX512_BITALG ecx
-#define reg_INDEX_7_ECX_13 ecx
-#define reg_AVX512_VPOPCNTDQ ecx
-#define reg_INDEX_7_ECX_15 ecx
-#define reg_INDEX_7_ECX_16 ecx
-#define reg_RDPID ecx
-#define reg_KL ecx
-#define reg_INDEX_7_ECX_24 ecx
-#define reg_CLDEMOTE ecx
-#define reg_INDEX_7_ECX_26 ecx
-#define reg_MOVDIRI ecx
-#define reg_MOVDIR64B ecx
-#define reg_ENQCMD ecx
-#define reg_SGX_LC ecx
-#define reg_PKS ecx
-
-/* EDX. */
-#define reg_INDEX_7_EDX_0 edx
-#define reg_INDEX_7_EDX_1 edx
-#define reg_AVX512_4VNNIW edx
-#define reg_AVX512_4FMAPS edx
-#define reg_FSRM edx
-#define reg_UINTR edx
-#define reg_INDEX_7_EDX_6 edx
-#define reg_INDEX_7_EDX_7 edx
-#define reg_AVX512_VP2INTERSECT edx
-#define reg_INDEX_7_EDX_9 edx
-#define reg_MD_CLEAR edx
-#define reg_INDEX_7_EDX_11 edx
-#define reg_INDEX_7_EDX_12 edx
-#define reg_INDEX_7_EDX_13 edx
-#define reg_SERIALIZE edx
-#define reg_HYBRID edx
-#define reg_TSXLDTRK edx
-#define reg_INDEX_7_EDX_17 edx
-#define reg_PCONFIG edx
-#define reg_INDEX_7_EDX_19 edx
-#define reg_IBT edx
-#define reg_INDEX_7_EDX_21 edx
-#define reg_AMX_BF16 edx
-#define reg_AVX512_FP16 edx
-#define reg_AMX_TILE edx
-#define reg_AMX_INT8 edx
-#define reg_IBRS_IBPB edx
-#define reg_STIBP edx
-#define reg_L1D_FLUSH edx
-#define reg_ARCH_CAPABILITIES edx
-#define reg_CORE_CAPABILITIES edx
-#define reg_SSBD edx
-
-/* COMMON_CPUID_INDEX_80000001. */
-
-/* ECX. */
-#define reg_LAHF64_SAHF64 ecx
-#define reg_SVM ecx
-#define reg_LZCNT ecx
-#define reg_SSE4A ecx
-#define reg_PREFETCHW ecx
-#define reg_XOP ecx
-#define reg_LWP ecx
-#define reg_FMA4 ecx
-#define reg_TBM ecx
-
-/* EDX. */
-#define reg_SYSCALL_SYSRET edx
-#define reg_NX edx
-#define reg_PAGE1GB edx
-#define reg_RDTSCP edx
-#define reg_LM edx
-
-/* COMMON_CPUID_INDEX_D_ECX_1. */
-
-/* EAX. */
-#define reg_XSAVEOPT eax
-#define reg_XSAVEC eax
-#define reg_XGETBV_ECX_1 eax
-#define reg_XSAVES eax
-#define reg_XFD eax
-
-/* COMMON_CPUID_INDEX_80000007. */
-
-/* EDX. */
-#define reg_INVARIANT_TSC edx
-
-/* COMMON_CPUID_INDEX_80000008. */
-
-/* EBX. */
-#define reg_WBNOINVD ebx
-
-/* COMMON_CPUID_INDEX_7_ECX_1. */
-
-/* EAX. */
-#define reg_AVX_VNNI eax
-#define reg_AVX512_BF16 eax
-#define reg_FZLRM eax
-#define reg_FSRS eax
-#define reg_FSRCS eax
-#define reg_HRESET eax
-#define reg_LAM eax
-
-/* COMMON_CPUID_INDEX_19. */
+#define CPU_FEATURE_USABLE(name) x86_cpu_is_usable (x86_cpu_##name)
-/* EBX. */
-#define reg_AESKLE ebx
-#define reg_WIDE_KL ebx
+__END_DECLS
#endif /* _SYS_PLATFORM_X86_H */
diff --git a/sysdeps/x86/tst-get-cpu-features.c b/sysdeps/x86/tst-get-cpu-features.c
index 39f287d2a0..dcdb86bb93 100644
--- a/sysdeps/x86/tst-get-cpu-features.c
+++ b/sysdeps/x86/tst-get-cpu-features.c
@@ -1,4 +1,4 @@
-/* Test case for __x86_get_cpu_features interface
+/* Test case for <sys/platform/x86.h> interface
Copyright (C) 2015-2021 Free Software Foundation, Inc.
This file is part of the GNU C Library.
@@ -33,36 +33,9 @@
printf (" " #name "\n"); \
}
-static const char * const cpu_kinds[] =
-{
- "Unknown",
- "Intel",
- "AMD",
- "ZHAOXIN",
- "Other",
-};
-
static int
do_test (void)
{
- const struct cpu_features *cpu_features = __x86_get_cpu_features (0);
-
- switch (cpu_features->basic.kind)
- {
- case arch_kind_intel:
- case arch_kind_amd:
- case arch_kind_zhaoxin:
- case arch_kind_other:
- printf ("Vendor: %s\n", cpu_kinds[cpu_features->basic.kind]);
- printf ("Family: 0x%x\n", cpu_features->basic.family);
- printf ("Model: 0x%x\n", cpu_features->basic.model);
- printf ("Stepping: 0x%x\n", cpu_features->basic.stepping);
- break;
-
- default:
- abort ();
- }
-
#ifdef __SSE2__
TEST_VERIFY_EXIT (HAS_CPU_FEATURE (SSE2));
#endif
diff --git a/sysdeps/x86/tst-isa-level-1.c b/sysdeps/x86/tst-isa-level-1.c
index cf611e8b78..38edabe75d 100644
--- a/sysdeps/x86/tst-isa-level-1.c
+++ b/sysdeps/x86/tst-isa-level-1.c
@@ -60,8 +60,7 @@ do_test_1 (const char *modname, bool fail)
static int
do_test (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
bool has_isa_baseline = ((isa_level & GNU_PROPERTY_X86_ISA_1_BASELINE)
== GNU_PROPERTY_X86_ISA_1_BASELINE);
diff --git a/sysdeps/x86_64/dl-hwcaps-subdirs.c b/sysdeps/x86_64/dl-hwcaps-subdirs.c
index 4acd343183..b8f27d7f98 100644
--- a/sysdeps/x86_64/dl-hwcaps-subdirs.c
+++ b/sysdeps/x86_64/dl-hwcaps-subdirs.c
@@ -27,8 +27,7 @@ enum { subdirs_count = 3 }; /* Number of components in _dl_hwcaps_subdirs. */
uint32_t
_dl_hwcaps_subdirs_active (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
int active = 0;
diff --git a/sysdeps/x86_64/tst-glibc-hwcaps.c b/sysdeps/x86_64/tst-glibc-hwcaps.c
index 37b61d3c84..4c8d24b3f2 100644
--- a/sysdeps/x86_64/tst-glibc-hwcaps.c
+++ b/sysdeps/x86_64/tst-glibc-hwcaps.c
@@ -30,8 +30,7 @@ extern int marker4 (void);
static int
compute_level (void)
{
- const struct cpu_features *cpu_features
- = __x86_get_cpu_features (COMMON_CPUID_INDEX_MAX);
+ const struct cpu_features *cpu_features = __get_cpu_features ();
unsigned int isa_level = get_isa_level (cpu_features);
if (!(isa_level & GNU_PROPERTY_X86_ISA_1_V2))
--
2.29.2
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: V4 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-21 13:29 ` V4 " H.J. Lu via Libc-alpha
@ 2021-01-21 13:57 ` Florian Weimer via Libc-alpha
2021-01-21 14:12 ` Adhemerval Zanella via Libc-alpha
0 siblings, 1 reply; 10+ messages in thread
From: Florian Weimer via Libc-alpha @ 2021-01-21 13:57 UTC (permalink / raw)
To: H.J. Lu via Libc-alpha
* H. J. Lu via Libc-alpha:
> Here is the updated patch to place __x86_get_cpuid_feature_leaf
> in libc. OK for master?
This version looks okay to me.
We should perhaps document more of the interface in the manual, but we
can discuss this separately. And I want to see the code changes go in
for release testing now.
Thanks,
Florian
--
Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn,
Commercial register: Amtsgericht Muenchen, HRB 153243,
Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: V4 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic
2021-01-21 13:57 ` Florian Weimer via Libc-alpha
@ 2021-01-21 14:12 ` Adhemerval Zanella via Libc-alpha
0 siblings, 0 replies; 10+ messages in thread
From: Adhemerval Zanella via Libc-alpha @ 2021-01-21 14:12 UTC (permalink / raw)
To: Florian Weimer, H.J. Lu via Libc-alpha
On 21/01/2021 10:57, Florian Weimer wrote:
> * H. J. Lu via Libc-alpha:
>
>> Here is the updated patch to place __x86_get_cpuid_feature_leaf
>> in libc. OK for master?
>
> This version looks okay to me.
>
> We should perhaps document more of the interface in the manual, but we
> can discuss this separately. And I want to see the code changes go in
> for release testing now.
Please do, this is ok for 2.33.
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-01-21 14:12 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-25 16:24 [PATCH] <sys/platform/x86.h>: Remove the C preprocessor magic H.J. Lu via Libc-alpha
2021-01-20 9:53 ` Florian Weimer via Libc-alpha
2021-01-20 11:23 ` Florian Weimer via Libc-alpha
2021-01-20 14:43 ` V2 " H.J. Lu via Libc-alpha
2021-01-20 15:58 ` Florian Weimer via Libc-alpha
2021-01-21 0:19 ` V3 " H.J. Lu via Libc-alpha
2021-01-21 11:19 ` Florian Weimer via Libc-alpha
2021-01-21 13:29 ` V4 " H.J. Lu via Libc-alpha
2021-01-21 13:57 ` Florian Weimer via Libc-alpha
2021-01-21 14:12 ` Adhemerval Zanella via Libc-alpha
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).