[PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[Prem Mallappa via Libc-alpha]

From: Prem Mallappa <Premachandra.Mallappa@amd.com>

Hello Glibc Community,

== (cross posting to libc-alpha, apologies for the spam) ==

This is in response to

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=24979
[2] https://sourceware.org/bugzilla/show_bug.cgi?id=24080
[3] https://sourceware.org/bugzilla/show_bug.cgi?id=23249

It is clear that there is no panacea here. However,
here is an attempt to address them in parts.

From [1], enable customers who already have
"haswell" libs and has seen perf benifits by loading
them on AMD Zen.
(Load libraries by placing them in LD_LIBRARY_PATH/zen
or by a symbolic link zen->haswell)

From [2] and [3]
And, A futuristic generic-avx2/generic-avx libs,
enables OS vendors to supply an optimized set.
And haswell/zen are really a superset, hence
keeping it made sense.

By this we would like to open it up for discussion
The haswell/zen can be intel/amd
(or any other name, and supply ifunc based loading
internally)

Prem Mallappa (3):
  x86: Refactor platform support in cpu_features
  x86: Add AMD Zen and AVX2/AVX platform support
  x86: test to load from PLATFORM path

 sysdeps/x86/cpu-features.c | 113 ++++++++++++++++++++++---------------
 sysdeps/x86_64/Makefile    |   3 +-
 2 files changed, 69 insertions(+), 47 deletions(-)

-- 
2.20.1

[PATCH 1/3] x86: Refactor platform support in cpu_features

[Prem Mallappa via Libc-alpha]

From: Prem Mallappa <Premachandra.Mallappa@amd.com>

This is a preliminary support to have platform
for AMD processors.

Signed-off-by: Prem Mallappa <Premachandra.Mallappa@amd.com>
---
 sysdeps/x86/cpu-features.c | 99 ++++++++++++++++++++------------------
 1 file changed, 53 insertions(+), 46 deletions(-)

diff --git a/sysdeps/x86/cpu-features.c b/sysdeps/x86/cpu-features.c
index 81a170a819..a36f385976 100644
--- a/sysdeps/x86/cpu-features.c
+++ b/sysdeps/x86/cpu-features.c
@@ -302,6 +302,58 @@ _Static_assert (((index_arch_Fast_Unaligned_Load
 		     == index_arch_Fast_Copy_Backward)),
 		"Incorrect index_arch_Fast_Unaligned_Load");
 
+static void
+set_platform (struct cpu_features *cpu_features)
+{
+#ifdef __x86_64__
+  const char *platform = NULL;
+
+  GLRO(dl_hwcap) = HWCAP_X86_64;
+
+  if (cpu_features->basic.kind == arch_kind_intel)
+    {
+      if (CPU_FEATURES_ARCH_P (cpu_features, AVX512F_Usable)
+	  && CPU_FEATURES_CPU_P (cpu_features, AVX512CD))
+	{
+	  if (CPU_FEATURES_CPU_P (cpu_features, AVX512ER))
+	    {
+	      if (CPU_FEATURES_CPU_P (cpu_features, AVX512PF))
+		platform = "xeon_phi";
+	    }
+	  else
+	    {
+	      if (CPU_FEATURES_CPU_P (cpu_features, AVX512BW)
+		  && CPU_FEATURES_CPU_P (cpu_features, AVX512DQ)
+		  && CPU_FEATURES_CPU_P (cpu_features, AVX512VL))
+		GLRO(dl_hwcap) |= HWCAP_X86_AVX512_1;
+	    }
+	}
+
+      if (platform == NULL
+	  && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)
+	  && CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
+	  && CPU_FEATURES_CPU_P (cpu_features, BMI1)
+	  && CPU_FEATURES_CPU_P (cpu_features, BMI2)
+	  && CPU_FEATURES_CPU_P (cpu_features, LZCNT)
+	  && CPU_FEATURES_CPU_P (cpu_features, MOVBE)
+	  && CPU_FEATURES_CPU_P (cpu_features, POPCNT))
+	platform = "haswell";
+    }
+
+  if (platform != NULL)
+    GLRO(dl_platform) = platform;
+#else
+  GLRO(dl_hwcap) = 0;
+  if (CPU_FEATURES_CPU_P (cpu_features, SSE2))
+    GLRO(dl_hwcap) |= HWCAP_X86_SSE2;
+
+  if (CPU_FEATURES_ARCH_P (cpu_features, I686))
+    GLRO(dl_platform) = "i686";
+  else if (CPU_FEATURES_ARCH_P (cpu_features, I586))
+    GLRO(dl_platform) = "i586";
+#endif
+}
+
 static inline void
 init_cpu_features (struct cpu_features *cpu_features)
 {
@@ -506,52 +558,7 @@ no_cpuid:
   GLRO(dl_hwcap_mask) = HWCAP_IMPORTANT;
 #endif
 
-#ifdef __x86_64__
-  GLRO(dl_hwcap) = HWCAP_X86_64;
-  if (cpu_features->basic.kind == arch_kind_intel)
-    {
-      const char *platform = NULL;
-
-      if (CPU_FEATURES_ARCH_P (cpu_features, AVX512F_Usable)
-	  && CPU_FEATURES_CPU_P (cpu_features, AVX512CD))
-	{
-	  if (CPU_FEATURES_CPU_P (cpu_features, AVX512ER))
-	    {
-	      if (CPU_FEATURES_CPU_P (cpu_features, AVX512PF))
-		platform = "xeon_phi";
-	    }
-	  else
-	    {
-	      if (CPU_FEATURES_CPU_P (cpu_features, AVX512BW)
-		  && CPU_FEATURES_CPU_P (cpu_features, AVX512DQ)
-		  && CPU_FEATURES_CPU_P (cpu_features, AVX512VL))
-		GLRO(dl_hwcap) |= HWCAP_X86_AVX512_1;
-	    }
-	}
-
-      if (platform == NULL
-	  && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)
-	  && CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
-	  && CPU_FEATURES_CPU_P (cpu_features, BMI1)
-	  && CPU_FEATURES_CPU_P (cpu_features, BMI2)
-	  && CPU_FEATURES_CPU_P (cpu_features, LZCNT)
-	  && CPU_FEATURES_CPU_P (cpu_features, MOVBE)
-	  && CPU_FEATURES_CPU_P (cpu_features, POPCNT))
-	platform = "haswell";
-
-      if (platform != NULL)
-	GLRO(dl_platform) = platform;
-    }
-#else
-  GLRO(dl_hwcap) = 0;
-  if (CPU_FEATURES_CPU_P (cpu_features, SSE2))
-    GLRO(dl_hwcap) |= HWCAP_X86_SSE2;
-
-  if (CPU_FEATURES_ARCH_P (cpu_features, I686))
-    GLRO(dl_platform) = "i686";
-  else if (CPU_FEATURES_ARCH_P (cpu_features, I586))
-    GLRO(dl_platform) = "i586";
-#endif
+  set_platform(cpu_features);
 
 #if CET_ENABLED
 # if HAVE_TUNABLES
-- 
2.20.1

[PATCH 2/3] x86: Add AMD Zen and AVX2/AVX platform support

[Prem Mallappa via Libc-alpha]

From: Prem Mallappa <Premachandra.Mallappa@amd.com>

Priority as below
- AVX512
- Haswell/Zen as they are superset of AVX2/AVX
- AVX2
- AVX

Signed-off-by: Prem Mallappa <Premachandra.Mallappa@amd.com>
---
 sysdeps/x86/cpu-features.c | 30 ++++++++++++++++++++++--------
 1 file changed, 22 insertions(+), 8 deletions(-)

diff --git a/sysdeps/x86/cpu-features.c b/sysdeps/x86/cpu-features.c
index a36f385976..2097107968 100644
--- a/sysdeps/x86/cpu-features.c
+++ b/sysdeps/x86/cpu-features.c
@@ -328,20 +328,34 @@ set_platform (struct cpu_features *cpu_features)
 		GLRO(dl_hwcap) |= HWCAP_X86_AVX512_1;
 	    }
 	}
+    }
 
-      if (platform == NULL
-	  && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)
-	  && CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
-	  && CPU_FEATURES_CPU_P (cpu_features, BMI1)
-	  && CPU_FEATURES_CPU_P (cpu_features, BMI2)
-	  && CPU_FEATURES_CPU_P (cpu_features, LZCNT)
-	  && CPU_FEATURES_CPU_P (cpu_features, MOVBE)
-	  && CPU_FEATURES_CPU_P (cpu_features, POPCNT))
+  if (platform == NULL
+      && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)
+      && CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
+      && CPU_FEATURES_CPU_P (cpu_features, BMI1)
+      && CPU_FEATURES_CPU_P (cpu_features, BMI2)
+      && CPU_FEATURES_CPU_P (cpu_features, LZCNT)
+      && CPU_FEATURES_CPU_P (cpu_features, MOVBE)
+      && CPU_FEATURES_CPU_P (cpu_features, POPCNT))
+    {
+      if (cpu_features->basic.kind == arch_kind_intel)
 	platform = "haswell";
+      else if (cpu_features->basic.kind == arch_kind_amd)
+	platform = "zen";
     }
 
+  if (platform == NULL
+      && CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable))
+    platform = "generic-avx2";
+
+  if (platform == NULL
+      && CPU_FEATURES_ARCH_P (cpu_features, AVX_Usable))
+    platform = "generic-avx";
+
   if (platform != NULL)
     GLRO(dl_platform) = platform;
+
 #else
   GLRO(dl_hwcap) = 0;
   if (CPU_FEATURES_CPU_P (cpu_features, SSE2))
-- 
2.20.1

[PATCH 3/3] x86: test to load from PLATFORM path

[Prem Mallappa via Libc-alpha]

From: Prem Mallappa <Premachandra.Mallappa@amd.com>

Test will load modifier library from new
PLATFORM path as well.

Signed-off-by: Prem Mallappa <Premachandra.Mallappa@amd.com>
---
 sysdeps/x86_64/Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/sysdeps/x86_64/Makefile b/sysdeps/x86_64/Makefile
index d51cf03ac9..b09881b2d7 100644
--- a/sysdeps/x86_64/Makefile
+++ b/sysdeps/x86_64/Makefile
@@ -71,6 +71,7 @@ CFLAGS-tst-platformmod-2.c = -mno-avx
 LDFLAGS-tst-platformmod-2.so = -Wl,-soname,tst-platformmod-2.so
 $(objpfx)tst-platform-1: $(objpfx)tst-platformmod-1.so
 $(objpfx)tst-platform-1.out: $(objpfx)x86_64/tst-platformmod-2.so
+$(objpfx)tst-platform-1.out: $(objpfx)zen/tst-platformmod-2.so
 # Turn off AVX512F_Usable and AVX2_Usable so that GLRO(dl_platform) is
 # always set to x86_64.
 tst-platform-1-ENV = LD_PRELOAD=$(objpfx)\$$PLATFORM/tst-platformmod-2.so \
@@ -161,7 +162,7 @@ do-tests-clean common-mostlyclean: tst-x86_64-1-clean
 tst-x86_64-1-clean:
 	-rm -rf $(objpfx)x86_64
 
-$(objpfx)x86_64/tst-platformmod-2.os: $(objpfx)tst-platformmod-2.os
+$(objpfx)%/tst-platformmod-2.os: $(objpfx)tst-platformmod-2.os
 	$(make-target-directory)
 	rm -f $@
 	ln $< $@
-- 
2.20.1

Re: [PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[Florian Weimer]

* Prem Mallappa via Libc-alpha:

> From: Prem Mallappa <Premachandra.Mallappa@amd.com>
>
> Hello Glibc Community,
>
> == (cross posting to libc-alpha, apologies for the spam) ==
>
> This is in response to
>
> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=24979
> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=24080
> [3] https://sourceware.org/bugzilla/show_bug.cgi?id=23249
>
> It is clear that there is no panacea here. However,
> here is an attempt to address them in parts.
>
> From [1], enable customers who already have
> "haswell" libs and has seen perf benifits by loading
> them on AMD Zen.
> (Load libraries by placing them in LD_LIBRARY_PATH/zen
> or by a symbolic link zen->haswell)
>
> From [2] and [3]
> And, A futuristic generic-avx2/generic-avx libs,
> enables OS vendors to supply an optimized set.
> And haswell/zen are really a superset, hence
> keeping it made sense.
>
> By this we would like to open it up for discussion
> The haswell/zen can be intel/amd
> (or any other name, and supply ifunc based loading
> internally)

I think we cannot use the platform subdirectory for that because there
is just a single one.  If we want a Intel/AMD split, we need to
enhance the dynamic loader to try the CPU vendor directory first, and
then fallback to a shared subdirectory.  Most distributions do not
want to test and ship binaries specific to Intel or AMD CPUs.

That's a generic loader change which will need some time to implement,
but we can work on something else in the meantime:

We need to check for *all* relevant CPU flags such code can use and,
and only enable a subdirectory if they are present.  This is necessary
because virtualization and microcode updates can disable individual
CPU features.

For the new shared subdirectory, I think we should not restrict
ourselves just to AVX2, but we should also include useful extensions
that are in practice always implemented in silicon along with AVX2,
but can be separately tweaked.

This seems to be a reasonable list of CPU feature flags to start with:

  3DNOW
  3DNOWEXT
  3DNOWPREFETCH
  ABM
  ADX
  AES
  AVX
  AVX2
  BMI
  BMI2
  CET
  CLFLUSH
  CLFLUSHOPT
  CLWB
  CLZERO
  CMPXCHG16B
  ERMS
  F16C
  FMA
  FMA4
  FSGSBASE
  FSRM
  FXSR
  HLE
  LAHF
  LZCNT
  MOVBE
  MWAITX
  PCLMUL
  PCOMMIT
  PKU
  POPCNT
  PREFETCHW
  RDPID
  RDRAND
  RDSEED
  RDTSCP
  RTM
  SHA
  SSE3
  SSE4.1
  SSE4.2
  SSE4A
  SSSE3
  TSC
  XGETBV
  XSAVE
  XSAVEC
  XSAVEOPT
  XSAVES

You (as in AMD) need to go through this list and come back with the
subset that you think should be enabled for current and future CPUs,
based on your internal roadmap and known errata for existing CPUs.  We
do not need a rationale for how you filter down the list, merely the
outcome.

(I already have the trimmed-down list from Intel.)

Re: [PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[Carlos O'Donell via Libc-alpha]

On 3/17/20 5:02 AM, Florian Weimer wrote:
> * Prem Mallappa via Libc-alpha:
> 
>> From: Prem Mallappa <Premachandra.Mallappa@amd.com>
>>
>> Hello Glibc Community,
>>
>> == (cross posting to libc-alpha, apologies for the spam) ==
>>
>> This is in response to
>>
>> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=24979
>> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=24080
>> [3] https://sourceware.org/bugzilla/show_bug.cgi?id=23249
>>
>> It is clear that there is no panacea here. However,
>> here is an attempt to address them in parts.
>>
>> From [1], enable customers who already have
>> "haswell" libs and has seen perf benifits by loading
>> them on AMD Zen.
>> (Load libraries by placing them in LD_LIBRARY_PATH/zen
>> or by a symbolic link zen->haswell)
>>
>> From [2] and [3]
>> And, A futuristic generic-avx2/generic-avx libs,
>> enables OS vendors to supply an optimized set.
>> And haswell/zen are really a superset, hence
>> keeping it made sense.
>>
>> By this we would like to open it up for discussion
>> The haswell/zen can be intel/amd
>> (or any other name, and supply ifunc based loading
>> internally)
> 
> I think we cannot use the platform subdirectory for that because there
> is just a single one.  If we want a Intel/AMD split, we need to
> enhance the dynamic loader to try the CPU vendor directory first, and
> then fallback to a shared subdirectory.  Most distributions do not
> want to test and ship binaries specific to Intel or AMD CPUs.

I agree. The additional burden on testing, maintaining, and supporting
distinct libraries is not feasible.

> That's a generic loader change which will need some time to implement,
> but we can work on something else in the meantime:
> 
> We need to check for *all* relevant CPU flags such code can use and,
> and only enable a subdirectory if they are present.  This is necessary
> because virtualization and microcode updates can disable individual
> CPU features.

Agreed. This is the only sensible plan. The platform directories already
imply some of this, but it's not well structured.

> For the new shared subdirectory, I think we should not restrict
> ourselves just to AVX2, but we should also include useful extensions
> that are in practice always implemented in silicon along with AVX2,
> but can be separately tweaked.

Agreed.

> This seems to be a reasonable list of CPU feature flags to start with:
> 
>   3DNOW
>   3DNOWEXT
>   3DNOWPREFETCH
>   ABM
>   ADX
>   AES
>   AVX
>   AVX2
>   BMI
>   BMI2
>   CET
>   CLFLUSH
>   CLFLUSHOPT
>   CLWB
>   CLZERO
>   CMPXCHG16B
>   ERMS
>   F16C
>   FMA
>   FMA4
>   FSGSBASE
>   FSRM
>   FXSR
>   HLE
>   LAHF
>   LZCNT
>   MOVBE
>   MWAITX
>   PCLMUL
>   PCOMMIT
>   PKU
>   POPCNT
>   PREFETCHW
>   RDPID
>   RDRAND
>   RDSEED
>   RDTSCP
>   RTM
>   SHA
>   SSE3
>   SSE4.1
>   SSE4.2
>   SSE4A
>   SSSE3
>   TSC
>   XGETBV
>   XSAVE
>   XSAVEC
>   XSAVEOPT
>   XSAVES
> 
> You (as in AMD) need to go through this list and come back with the
> subset that you think should be enabled for current and future CPUs,
> based on your internal roadmap and known errata for existing CPUs.  We
> do not need a rationale for how you filter down the list, merely the
> outcome.

And this is the hard part that we can't solve without AMD's help.

Even if you ignore "future CPUs" it would be useful to get this list
for all current CPUs given your architectural knowledge, errata, and
other factors like microcode, that covers the currently released CPUs.

> (I already have the trimmed-down list from Intel.)
> 


-- 
Cheers,
Carlos.

Re: [PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[Adhemerval Zanella via Libc-alpha]

On 17/03/2020 10:17, Carlos O'Donell via Libc-alpha wrote:
> On 3/17/20 5:02 AM, Florian Weimer wrote:
>> * Prem Mallappa via Libc-alpha:
>>
>>> From: Prem Mallappa <Premachandra.Mallappa@amd.com>
>>>
>>> Hello Glibc Community,
>>>
>>> == (cross posting to libc-alpha, apologies for the spam) ==
>>>
>>> This is in response to
>>>
>>> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=24979
>>> [2] https://sourceware.org/bugzilla/show_bug.cgi?id=24080
>>> [3] https://sourceware.org/bugzilla/show_bug.cgi?id=23249
>>>
>>> It is clear that there is no panacea here. However,
>>> here is an attempt to address them in parts.
>>>
>>> From [1], enable customers who already have
>>> "haswell" libs and has seen perf benifits by loading
>>> them on AMD Zen.
>>> (Load libraries by placing them in LD_LIBRARY_PATH/zen
>>> or by a symbolic link zen->haswell)
>>>
>>> From [2] and [3]
>>> And, A futuristic generic-avx2/generic-avx libs,
>>> enables OS vendors to supply an optimized set.
>>> And haswell/zen are really a superset, hence
>>> keeping it made sense.
>>>
>>> By this we would like to open it up for discussion
>>> The haswell/zen can be intel/amd
>>> (or any other name, and supply ifunc based loading
>>> internally)
>>
>> I think we cannot use the platform subdirectory for that because there
>> is just a single one.  If we want a Intel/AMD split, we need to
>> enhance the dynamic loader to try the CPU vendor directory first, and
>> then fallback to a shared subdirectory.  Most distributions do not
>> want to test and ship binaries specific to Intel or AMD CPUs.
> 
> I agree. The additional burden on testing, maintaining, and supporting
> distinct libraries is not feasible.
> 
>> That's a generic loader change which will need some time to implement,
>> but we can work on something else in the meantime:
>>
>> We need to check for *all* relevant CPU flags such code can use and,
>> and only enable a subdirectory if they are present.  This is necessary
>> because virtualization and microcode updates can disable individual
>> CPU features.
> 
> Agreed. This is the only sensible plan. The platform directories already
> imply some of this, but it's not well structured.

Which should be our policy regarding the platform name over releases?
Should the names set in previous release being supported in a 
compatibility manner or should it not be constraint (as for tunables)
and subject of change?

If the former, with a defined subset of the CPU features flags it might
be possible to share folder over x86 chips without using chips release
names.

> 
>> For the new shared subdirectory, I think we should not restrict
>> ourselves just to AVX2, but we should also include useful extensions
>> that are in practice always implemented in silicon along with AVX2,
>> but can be separately tweaked.
> 
> Agreed.
> 
>> This seems to be a reasonable list of CPU feature flags to start with:
>>
>>   3DNOW
>>   3DNOWEXT
>>   3DNOWPREFETCH
>>   ABM
>>   ADX
>>   AES
>>   AVX
>>   AVX2
>>   BMI
>>   BMI2
>>   CET
>>   CLFLUSH
>>   CLFLUSHOPT
>>   CLWB
>>   CLZERO
>>   CMPXCHG16B
>>   ERMS
>>   F16C
>>   FMA
>>   FMA4
>>   FSGSBASE
>>   FSRM
>>   FXSR
>>   HLE
>>   LAHF
>>   LZCNT
>>   MOVBE
>>   MWAITX
>>   PCLMUL
>>   PCOMMIT
>>   PKU
>>   POPCNT
>>   PREFETCHW
>>   RDPID
>>   RDRAND
>>   RDSEED
>>   RDTSCP
>>   RTM
>>   SHA
>>   SSE3
>>   SSE4.1
>>   SSE4.2
>>   SSE4A
>>   SSSE3
>>   TSC
>>   XGETBV
>>   XSAVE
>>   XSAVEC
>>   XSAVEOPT
>>   XSAVES
>>
>> You (as in AMD) need to go through this list and come back with the
>> subset that you think should be enabled for current and future CPUs,
>> based on your internal roadmap and known errata for existing CPUs.  We
>> do not need a rationale for how you filter down the list, merely the
>> outcome.
> 
> And this is the hard part that we can't solve without AMD's help.
> 
> Even if you ignore "future CPUs" it would be useful to get this list
> for all current CPUs given your architectural knowledge, errata, and
> other factors like microcode, that covers the currently released CPUs.

So the question is how should be move forward: let the chip vendor
(Intel, AMD, etc.) define its own naming scheme based on its own
chip roadmap, or create subsets of features to set a common naming
folder (as suggested by Richard Biener on BZ#24080 [1])?

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=24080

Re: [PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[Carlos O'Donell via Libc-alpha]

On 3/17/20 3:27 PM, Adhemerval Zanella via Libc-alpha wrote:
> On 17/03/2020 10:17, Carlos O'Donell via Libc-alpha wrote:
>> Agreed. This is the only sensible plan. The platform directories already
>> imply some of this, but it's not well structured.
> 
> Which should be our policy regarding the platform name over releases?
> Should the names set in previous release being supported in a 
> compatibility manner or should it not be constraint (as for tunables)
> and subject of change?

It should be subject to change just like tunables.

It should be an optimization, and not a requirement, and applications
should always provide a fallback implementaiton to allow the application
to load.

Failure to load the libraries *may* result in a failure to start the
application and we need to be OK with that. That is to say that a
particularly package may only ship an optimized library (going against
the recommendation).

We should verify that downstream distributions can use /etc/ld.so.conf
as a way to add back directories into the search of the existing 
additional multilib search directories e.g. Add back /lib64/haswell
for a few years.

Does that answer your question?
 
> If the former, with a defined subset of the CPU features flags it might
> be possible to share folder over x86 chips without using chips release
> names.

Yes.

>> And this is the hard part that we can't solve without AMD's help.
>>
>> Even if you ignore "future CPUs" it would be useful to get this list
>> for all current CPUs given your architectural knowledge, errata, and
>> other factors like microcode, that covers the currently released CPUs.
> 
> So the question is how should be move forward: let the chip vendor
> (Intel, AMD, etc.) define its own naming scheme based on its own
> chip roadmap, or create subsets of features to set a common naming
> folder (as suggested by Richard Biener on BZ#24080 [1])?

In the end I think we'll want:

(a) Try CPU vendor directories first.
- Each vendor should name their directories and the explicit
  compiler options to target them (printed by LD_DEBUG).

(b) Try shared directories second.
- Based on a common set of identified features.
  - Compiler options to target the shared set should be explicitly
    stated (printed by LD_DEBUG).

My understanding is that Florian is asking for help with (b)
to identify what things should be enabled for current CPUs, and
that we'll compare that list to the Intel list and make a common
shared directory that the downstream distributions can used
for the most optimized library we can have in common.

What we can do:
- Cleanup the generic code to allow a CPU vendor split?
  - We have a mix of hwcap bit handling and list handling
    for platforms. This is also a bit messy.
  - Allow vendors to drop in their CPU vendor search list
    ahead of the shared list, again based on feature presence.
- Prepare the code for a shared common directory based on
  some shared subset of features, and enable it only if
  those features are present. Today doing this is a bit
  messy in cpu-features.c.


> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=24080
 

-- 
Cheers,
Carlos.

Re: [PATCH 0/3] RFC: Platform Support for AMD Zen and AVX2/AVX

[Florian Weimer]

* Carlos O'Donell via Libc-alpha:

> On 3/17/20 3:27 PM, Adhemerval Zanella via Libc-alpha wrote:
>> On 17/03/2020 10:17, Carlos O'Donell via Libc-alpha wrote:
>>> Agreed. This is the only sensible plan. The platform directories already
>>> imply some of this, but it's not well structured.
>> 
>> Which should be our policy regarding the platform name over releases?
>> Should the names set in previous release being supported in a 
>> compatibility manner or should it not be constraint (as for tunables)
>> and subject of change?
>
> It should be subject to change just like tunables.

I disagree; for a subset of the directories, we should guarantee
stability.

> It should be an optimization, and not a requirement, and applications
> should always provide a fallback implementaiton to allow the application
> to load.

Agreed.  Programmers need to assume that future glibc versions may
stop selecting certain subdirectories.  However, I'm not sure if we
can suddenly start selecting directories on systems where we did not
do so before.

> We should verify that downstream distributions can use /etc/ld.so.conf
> as a way to add back directories into the search of the existing 
> additional multilib search directories e.g. Add back /lib64/haswell
> for a few years.

I don't think that works.

> In the end I think we'll want:
>
> (a) Try CPU vendor directories first.
> - Each vendor should name their directories and the explicit
>   compiler options to target them (printed by LD_DEBUG).
>
> (b) Try shared directories second.
> - Based on a common set of identified features.
>   - Compiler options to target the shared set should be explicitly
>     stated (printed by LD_DEBUG).
>
> My understanding is that Florian is asking for help with (b)
> to identify what things should be enabled for current CPUs, and
> that we'll compare that list to the Intel list and make a common
> shared directory that the downstream distributions can used
> for the most optimized library we can have in common.

The results for (b) also feed into (a) to some extent because if
research for (b) reveals that certain CPU features have been disabled
by microupdate updates, we probably do not want them for (a), either.