Re: tests failing on x86_64-linux (due to test-container?)

[Martin Sebor via Libc-alpha <libc-alpha@sourceware.org>]

On 1/17/22 11:04, Adhemerval Zanella wrote:
> 
> 
> On 14/01/2022 20:51, Martin Sebor via Libc-alpha wrote:
>> On 1/12/22 16:24, Martin Sebor wrote:
>>> On 1/12/22 15:13, H.J. Lu wrote:
>>>> On Wed, Jan 12, 2022 at 2:05 PM Martin Sebor via Libc-alpha
>>>> <libc-alpha@sourceware.org> wrote:
>>>>>
>>>>> On 5/13/21 15:29, Martin Sebor wrote:
>>>>>> On 4/28/21 5:54 PM, DJ Delorie wrote:
>>>>>>> Martin Sebor <msebor@gmail.com> writes:
>>>>>>>>> (which should be bind-mounted inside your test chroot, which would be
>>>>>>>>> /build/glibc/testroot.root/build/glibc/nss/tst-nss-test3)
>>>>>>>>
>>>>>>>> /build/glibc/testroot.root/build/glibc/ is empty.
>>>>>>>
>>>>>>> Only because you're not inside the container, where the mount exists.
>>>>>>> Containers are weird ;-)
>>>>>>>
>>>>>>>> error: test-container.c:235: can't mount /ssd/src/glibc onto
>>>>>>>> /build/glibc/testroot.root/ssd/src/glibc
>>>>>>>
>>>>>>> Sigh, you have "too many permissions" :-P  Again, containers are weird.
>>>>>>>
>>>>>>> You may be able to run test-container manually, but test-container
>>>>>>> itself is built against the just-built glibc
>>>>>>>
>>>>>>>>> Also, selinux might be interfering.
>>>>>>>>
>>>>>>>> That doesn't sound encouraging.  Anything I should look at/for?
>>>>>>>
>>>>>>> Dunno, I typically disable selinux.
>>>>>>
>>>>>> I haven't been able to make that work yet but I also haven't tried
>>>>>> too hard.  I'll keep at it.
>>>>>>
>>>>>>>
>>>>>>> Just for paranoia's sake... does all this work on a newer Fedora?
>>>>>>
>>>>>> Yes.  I just installed F33 on my laptop and it works fine there.
>>>>>> I haven't changed my main workstation though.  It was a pain to
>>>>>> set up.
>>>>>
>>>>> I finally bit the bullet and upgraded my workstation to Fedora
>>>>> 35, partly in hopes of resolving the Glibc test suite problems.
>>>>> I now get the failures below with the top of GCC 12 and Glibc
>>>>> trunk, even after disabling SELinux by running setenforce 0,
>>>>> as well as after permanently disabling it in /etc/selinux/config.
>>>>> Is there something else I should try?
>>>>>
>>>>> Thanks
>>>>> Martin
>>>>>
>>>>> UNSUPPORTED: assert/tst-assert-c++
>>>>> UNSUPPORTED: assert/tst-assert-g++
>>>>> UNSUPPORTED: debug/tst-chk4
>>>>> UNSUPPORTED: debug/tst-chk5
>>>>> UNSUPPORTED: debug/tst-chk6
>>>>> UNSUPPORTED: debug/tst-chk8
>>>>> UNSUPPORTED: debug/tst-lfschk4
>>>>> UNSUPPORTED: debug/tst-lfschk5
>>>>> UNSUPPORTED: debug/tst-lfschk6
>>>>> UNSUPPORTED: dlfcn/bug-atexit3
>>>>> UNSUPPORTED: elf/tst-audit10
>>>>> UNSUPPORTED: elf/tst-avx512
>>>>> FAIL: elf/tst-dlopen-self-container
>>>>> FAIL: elf/tst-dlopen-tlsmodid-container
>>>>> UNSUPPORTED: elf/tst-env-setuid
>>>>> UNSUPPORTED: elf/tst-env-setuid-tunables
>>>>> FAIL: elf/tst-glibc-hwcaps-2-cache
>>>>> FAIL: elf/tst-glibc-hwcaps-cache
>>>>> FAIL: elf/tst-glibc-hwcaps-prepend-cache
>>>>> XPASS: elf/tst-ifunc-isa-1
>>>>> XPASS: elf/tst-ifunc-isa-1-static
>>>>> XPASS: elf/tst-ifunc-isa-2
>>>>> XPASS: elf/tst-ifunc-isa-2-static
>>>>> FAIL: elf/tst-ldconfig-bad-aux-cache
>>>>> FAIL: elf/tst-ldconfig-ld_so_conf-update
>>>>> FAIL: elf/tst-pldd
>>>>> FAIL: elf/tst-preload-pthread-libc
>>>>> XPASS: elf/tst-protected1a
>>>>> XPASS: elf/tst-protected1b
>>>>> FAIL: grp/tst-initgroups1
>>>>> FAIL: grp/tst-initgroups2
>>>>> FAIL: locale/tst-localedef-path-norm
>>>>> FAIL: localedata/tst-localedef-hardlinks
>>>>> UNSUPPORTED: math/test-double-libmvec-acos-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-acosh-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-asin-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-asinh-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-atan-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-atan2-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-atanh-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-cbrt-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-cos-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-cosh-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-erf-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-erfc-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-exp-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-exp10-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-exp2-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-expm1-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-hypot-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-log-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-log10-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-log1p-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-log2-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-pow-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-sin-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-sincos-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-sinh-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-tan-avx512f
>>>>> UNSUPPORTED: math/test-double-libmvec-tanh-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-acosf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-acoshf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-asinf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-asinhf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-atan2f-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-atanf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-atanhf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-cbrtf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-cosf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-coshf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-erfcf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-erff-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-exp10f-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-exp2f-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-expf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-expm1f-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-hypotf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-log10f-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-log1pf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-log2f-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-logf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-powf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-sincosf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-sinf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-sinhf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-tanf-avx512f
>>>>> UNSUPPORTED: math/test-float-libmvec-tanhf-avx512f
>>>>> UNSUPPORTED: misc/tst-adjtimex
>>>>> UNSUPPORTED: misc/tst-clock_adjtime
>>>>> UNSUPPORTED: misc/tst-ntp_adjtime
>>>>> UNSUPPORTED: misc/tst-pkey
>>>>> UNSUPPORTED: nptl/tst-cancel24
>>>>> UNSUPPORTED: nptl/tst-cancel24-static
>>>>> UNSUPPORTED: nptl/tst-minstack-throw
>>>>> UNSUPPORTED: nptl/tst-once5
>>>>> FAIL: nptl/tst-pthread-getattr
>>>>> UNSUPPORTED: nptl/tst-thread-exit-clobber
>>>>> UNSUPPORTED: nptl/tst-thread_local1
>>>>> FAIL: nss/tst-nss-compat1
>>>>> FAIL: nss/tst-nss-db-endgrent
>>>>> FAIL: nss/tst-nss-db-endpwent
>>>>> FAIL: nss/tst-nss-files-hosts-long
>>>>> FAIL: nss/tst-nss-test3
>>>>> FAIL: nss/tst-reload1
>>>>> FAIL: nss/tst-reload2
>>>>> FAIL: posix/bug-ga2
>>>>> UNSUPPORTED: posix/bug-ga2-mem
>>>>> FAIL: posix/tst-vfork3
>>>>> UNSUPPORTED: posix/tst-vfork3-mem
>>>>> UNSUPPORTED: resolv/mtrace-tst-leaks2
>>>>> FAIL: resolv/tst-leaks2
>>>>> UNSUPPORTED: stdlib/tst-quick_exit
>>>>> UNSUPPORTED: stdlib/tst-secure-getenv
>>>>> FAIL: stdlib/tst-system
>>>>> UNSUPPORTED: stdlib/tst-thread-quick_exit
>>>>> UNSUPPORTED: string/tst-memchr-rtm
>>>>> UNSUPPORTED: string/tst-memcmp-rtm
>>>>> UNSUPPORTED: string/tst-memmove-rtm
>>>>> UNSUPPORTED: string/tst-memrchr-rtm
>>>>> UNSUPPORTED: string/tst-memset-rtm
>>>>> UNSUPPORTED: string/tst-strchr-rtm
>>>>> UNSUPPORTED: string/tst-strcpy-rtm
>>>>> FAIL: string/tst-strerror
>>>>> UNSUPPORTED: string/tst-strlen-rtm
>>>>> UNSUPPORTED: string/tst-strncmp-rtm
>>>>> UNSUPPORTED: string/tst-strrchr-rtm
>>>>> FAIL: string/tst-strsignal
>>>>> UNSUPPORTED: time/tst-clock_settime
>>>>> UNSUPPORTED: time/tst-settimeofday
>>>>> Summary of test results:
>>>>>         27 FAIL
>>>>>       4904 PASS
>>>>>         96 UNSUPPORTED
>>>>>         16 XFAIL
>>>>>          6 XPASS
>>>>
>>>> See:
>>>>
>>>> https://sourceware.org/bugzilla/show_bug.cgi?id=28455
>>>>
>>>> My last patch is at
>>>>
>>>> https://patchwork.sourceware.org/project/glibc/patch/20220104140009.1578867-1-hjl.tools@gmail.com/
>>> Thanks, but I still get the same results even with the patch
>>> applied and after rebuilding everything. The new test added
>>> by the patch, elf/tst-global2, passes.  Am I missing something
>>> else or is this still some container problem?
>>
>> The filesystem I'm using is mounted with these flags:
>>
>>    rw,nosuid,nodev,relatime,x-gvfs-show
>>
>> On a hunch I tried a different filesystem, one with these flags:
>>
>>    rw,relatime
>>
>> That changes things, making me wonder if one of the bits (nosuid?)
>> might be interfering with the container magic?
>>
>> I still don't get a clean build on the other filesystem, but I see
>> no failures, only unsupported results (below).  The exit status of
>> all those I checked is 77.  Are these results expected?
> 
> Yes, 77 is the internal exit magic number that states is was not
> a failure, but rather that the system lacks either proper tools or
> kernel/processor support.

Thanks a lot for the detailed analysis!  I figured out what 77 meant
but not yet why.  I'm not nearly as concerned about the unsupported
results as I was about failures but it would be nice to get a cleaner
bill of health.  Do you mind a few follow up questions?

> 
>>
>> Martin
>>
>> UNSUPPORTED: assert/tst-assert-c++
>> UNSUPPORTED: assert/tst-assert-g++
> 
> It means that the provided C++ compiler does not support thread_local
> or if CXX is not set.
> 
>> UNSUPPORTED: debug/tst-fortify-cc-default-1
>> UNSUPPORTED: debug/tst-fortify-cc-default-2
>> UNSUPPORTED: debug/tst-fortify-cc-default-3
>> UNSUPPORTED: debug/tst-fortify-cc-lfs-1
>> UNSUPPORTED: debug/tst-fortify-cc-lfs-2
>> UNSUPPORTED: debug/tst-fortify-cc-lfs-3
>> UNSUPPORTED: debug/tst-fortify-cc-nongnu-1
>> UNSUPPORTED: debug/tst-fortify-cc-nongnu-2
>> UNSUPPORTED: debug/tst-fortify-cc-nongnu-3
> 
> These are disabled if CXX is not set to a proper C++ compiler.

I set CXX to all the right options (all the C++ packages I've tried
compile) but I probably need to also set LD_LIBRARY_PATH to the new
libstdc++ for runtime testing.  Let me give that a try.

> 
>> UNSUPPORTED: dlfcn/bug-atexit3
> 
> Same.
> 
>> UNSUPPORTED: elf/tst-audit10
>> UNSUPPORTED: elf/tst-avx512
> 
> These requires a processor with support for AVX512.

I have an Intel Core i9-9900K CPU with the avx and avx2 bits set.
Is the AVX 512 something newer/different?

> 
>> UNSUPPORTED: elf/tst-env-setuid
>> UNSUPPORTED: elf/tst-env-setuid-tunables
> 
> These fail if the file system is mounted nosuid.

Just to be clear, this result is for a build on filesystem with
the nosuid bit clear:

/dev/mapper/fedora_bigbox-home on /home type ext4 (rw,relatime)

> 
>> XPASS: elf/tst-ifunc-isa-1
>> XPASS: elf/tst-ifunc-isa-1-static
>> XPASS: elf/tst-ifunc-isa-2
>> XPASS: elf/tst-ifunc-isa-2-static
> 
> These are marked as xfail to now thrown an issue if LLD is used
> instead of binutils.  I think maybe we mark as xfail iff lld is
> used.
> 
>> XPASS: elf/tst-protected1a
>> XPASS: elf/tst-protected1b
> 
> These are marked XFAIL because a gcc and binutils issue [1] [2].
> I think with current minimum supported versions we can remove
> the xfail.

Ack.

> 
>> UNSUPPORTED: math/test-double-libmvec-acos-avx512f
>> UNSUPPORTED: math/test-double-libmvec-acosh-avx512f
>> UNSUPPORTED: math/test-double-libmvec-asin-avx512f
>> UNSUPPORTED: math/test-double-libmvec-asinh-avx512f
>> UNSUPPORTED: math/test-double-libmvec-atan-avx512f
>> UNSUPPORTED: math/test-double-libmvec-atan2-avx512f
>> UNSUPPORTED: math/test-double-libmvec-atanh-avx512f
>> UNSUPPORTED: math/test-double-libmvec-cbrt-avx512f
>> UNSUPPORTED: math/test-double-libmvec-cos-avx512f
>> UNSUPPORTED: math/test-double-libmvec-cosh-avx512f
>> UNSUPPORTED: math/test-double-libmvec-erf-avx512f
>> UNSUPPORTED: math/test-double-libmvec-erfc-avx512f
>> UNSUPPORTED: math/test-double-libmvec-exp-avx512f
>> UNSUPPORTED: math/test-double-libmvec-exp10-avx512f
>> UNSUPPORTED: math/test-double-libmvec-exp2-avx512f
>> UNSUPPORTED: math/test-double-libmvec-expm1-avx512f
>> UNSUPPORTED: math/test-double-libmvec-hypot-avx512f
>> UNSUPPORTED: math/test-double-libmvec-log-avx512f
>> UNSUPPORTED: math/test-double-libmvec-log10-avx512f
>> UNSUPPORTED: math/test-double-libmvec-log1p-avx512f
>> UNSUPPORTED: math/test-double-libmvec-log2-avx512f
>> UNSUPPORTED: math/test-double-libmvec-pow-avx512f
>> UNSUPPORTED: math/test-double-libmvec-sin-avx512f
>> UNSUPPORTED: math/test-double-libmvec-sincos-avx512f
>> UNSUPPORTED: math/test-double-libmvec-sinh-avx512f
>> UNSUPPORTED: math/test-double-libmvec-tan-avx512f
>> UNSUPPORTED: math/test-double-libmvec-tanh-avx512f
>> UNSUPPORTED: math/test-float-libmvec-acosf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-acoshf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-asinf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-asinhf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-atan2f-avx512f
>> UNSUPPORTED: math/test-float-libmvec-atanf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-atanhf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-cbrtf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-cosf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-coshf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-erfcf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-erff-avx512f
>> UNSUPPORTED: math/test-float-libmvec-exp10f-avx512f
>> UNSUPPORTED: math/test-float-libmvec-exp2f-avx512f
>> UNSUPPORTED: math/test-float-libmvec-expf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-expm1f-avx512f
>> UNSUPPORTED: math/test-float-libmvec-hypotf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-log10f-avx512f
>> UNSUPPORTED: math/test-float-libmvec-log1pf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-log2f-avx512f
>> UNSUPPORTED: math/test-float-libmvec-logf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-powf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-sincosf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-sinf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-sinhf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-tanf-avx512f
>> UNSUPPORTED: math/test-float-libmvec-tanhf-avx512f
> 
> These all are require AVX512 as well.
> 
>> UNSUPPORTED: misc/tst-adjtimex
>> UNSUPPORTED: misc/tst-clock_adjtime
>> UNSUPPORTED: misc/tst-ntp_adjtime
> 
> There are ran only if GLIBC_TEST_ALLOW_TIME_SETTING env var is set
> (since they alter the system clock).

I see.

> 
>> UNSUPPORTED: misc/tst-pkey
> 
> It requires kernel support for pkey_alloc syscall.

I'm using Fedora 35 with linux 5.15.12-200.fc35.x86_64.  The man page
says the syscall was was added in 4.9 so it seems something must be
interfering with it.  Let me look into what that might be.

> 
>> UNSUPPORTED: nptl/tst-cancel24
>> UNSUPPORTED: nptl/tst-cancel24-static
>> UNSUPPORTED: nptl/tst-minstack-throw
>> UNSUPPORTED: nptl/tst-once5
>> UNSUPPORTED: nptl/tst-thread-exit-clobber
>> UNSUPPORTED: nptl/tst-thread_local1
>> UNSUPPORTED: stdlib/tst-quick_exit
>> UNSUPPORTED: stdlib/tst-secure-getenv
>> UNSUPPORTED: stdlib/tst-thread-quick_exit
> 
> These all require CXX to be properly set.

Ack.

> 
>> UNSUPPORTED: string/tst-memchr-rtm
>> UNSUPPORTED: string/tst-memcmp-rtm
>> UNSUPPORTED: string/tst-memmove-rtm
>> UNSUPPORTED: string/tst-memrchr-rtm
>> UNSUPPORTED: string/tst-memset-rtm
>> UNSUPPORTED: string/tst-strchr-rtm
>> UNSUPPORTED: string/tst-strcpy-rtm
>> UNSUPPORTED: string/tst-strlen-rtm
>> UNSUPPORTED: string/tst-strncmp-rtm
>> UNSUPPORTED: string/tst-strrchr-rtm
> 
> These require a CPU with TSX enabled.

Okay, I guess I'll just accept these.

> 
>> UNSUPPORTED: time/tst-clock_settime
>> UNSUPPORTED: time/tst-settimeofday
> 
> Same as GLIBC_TEST_ALLOW_TIME_SETTING.

Ack.

Thanks again!

Martin

> 
>> Summary of test results:
>>     4937 PASS
>>       95 UNSUPPORTED
>>       16 XFAIL
>>        6 XPASS
> 
> 
> [1] https://sourceware.org/bugzilla/show_bug.cgi?id=17709
> [2] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65248