Many float/double errors in math/ testsuite with gcc master branch

Many float/double errors in math/ testsuite with gcc master branch

[Romain GEISSLER via Libc-alpha]

Hi,

Trying to build currently a toolchain using glibc (current master branch) + gcc (current master branch) + binutils (release branch 2.39) on Linux x86_64 yields for me 53 test failure in the math/ folder. You can find the list of these tests at the end of this mail. Trying to downgrade to the latest commit of the current release branch of gcc 12 (so gcc 12.2.1) make all my glibc test failure disappear. Checking randomly couple of the .out files of the failing tests, it seems these test don’t work at all with gcc 13, there are many errors, and it doesn’t seem it’s due to a small change of precision. Am I the only one to experiences these test failure ? In case you don’t have such errors in the same conditions, how shall I report these errors for investigation ? Just the .out file would be enough ? I don’t know anything about these math/ tests, it’s the first time I see them failing like this.

Cheers,
Romain

FAIL: math/test-double-cosh
FAIL: math/test-double-exp10
FAIL: math/test-double-expm1
FAIL: math/test-double-iseqsig
FAIL: math/test-double-lgamma
FAIL: math/test-double-log1p
FAIL: math/test-double-tgamma
FAIL: math/test-float-cosh
FAIL: math/test-float-expm1
FAIL: math/test-float-iseqsig
FAIL: math/test-float-lgamma
FAIL: math/test-float-log1p
FAIL: math/test-float-tgamma
FAIL: math/test-float128-catan
FAIL: math/test-float128-catanh
FAIL: math/test-float128-cosh
FAIL: math/test-float128-exp10
FAIL: math/test-float128-iseqsig
FAIL: math/test-float128-lgamma
FAIL: math/test-float128-log
FAIL: math/test-float128-log1p
FAIL: math/test-float128-pow
FAIL: math/test-float128-tgamma
FAIL: math/test-float128-y0
FAIL: math/test-float128-y1
FAIL: math/test-float32-cosh
FAIL: math/test-float32-expm1
FAIL: math/test-float32-iseqsig
FAIL: math/test-float32-lgamma
FAIL: math/test-float32-log1p
FAIL: math/test-float32-tgamma
FAIL: math/test-float32x-cosh
FAIL: math/test-float32x-exp10
FAIL: math/test-float32x-expm1
FAIL: math/test-float32x-iseqsig
FAIL: math/test-float32x-lgamma
FAIL: math/test-float32x-log1p
FAIL: math/test-float32x-tgamma
FAIL: math/test-float64-cosh
FAIL: math/test-float64-exp10
FAIL: math/test-float64-expm1
FAIL: math/test-float64-iseqsig
FAIL: math/test-float64-lgamma
FAIL: math/test-float64-log1p
FAIL: math/test-float64-tgamma
FAIL: math/test-float64x-cosh
FAIL: math/test-float64x-iseqsig
FAIL: math/test-float64x-lgamma
FAIL: math/test-float64x-tgamma
FAIL: math/test-ldouble-cosh
FAIL: math/test-ldouble-iseqsig
FAIL: math/test-ldouble-lgamma
FAIL: math/test-ldouble-tgamma

Re: Many float/double errors in math/ testsuite with gcc master branch

[Joseph Myers]

On Tue, 29 Nov 2022, Romain GEISSLER via Libc-alpha wrote:

> Trying to build currently a toolchain using glibc (current master 
> branch) + gcc (current master branch) + binutils (release branch 2.39) 
> on Linux x86_64 yields for me 53 test failure in the math/ folder. You 
> can find the list of these tests at the end of this mail. Trying to 
> downgrade to the latest commit of the current release branch of gcc 12 
> (so gcc 12.2.1) make all my glibc test failure disappear. Checking 
> randomly couple of the .out files of the failing tests, it seems these 
> test don’t work at all with gcc 13, there are many errors, and it 
> doesn’t seem it’s due to a small change of precision. Am I the only one 
> to experiences these test failure ? In case you don’t have such errors 
> in the same conditions, how shall I report these errors for 
> investigation ? Just the .out file would be enough ? I don’t know 
> anything about these math/ tests, it’s the first time I see them failing 
> like this.

You should identify the specific test assertions failing - the inputs, 
outputs and what the test says is wrong - just the list of functions and 
types doesn't give much information.  An ulps failure is very different 
from a missing exception or missing errno setting, for example.  Missing 
exceptions could indicate a new optimization that loses exceptions as a 
result of optimizing some expression to a constant, for example.  Then, 
maybe identifying a specific GCC commit would help, or maybe it will be 
obvious from the details of the failures.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: Many float/double errors in math/ testsuite with gcc master branch

[Andrew Pinski via Libc-alpha]

On Mon, Nov 28, 2022 at 5:26 PM Romain GEISSLER via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> Hi,
>
> Trying to build currently a toolchain using glibc (current master branch) + gcc (current master branch) + binutils (release branch 2.39) on Linux x86_64 yields for me 53 test failure in the math/ folder. You can find the list of these tests at the end of this mail. Trying to downgrade to the latest commit of the current release branch of gcc 12 (so gcc 12.2.1) make all my glibc test failure disappear. Checking randomly couple of the .out files of the failing tests, it seems these test don’t work at all with gcc 13, there are many errors, and it doesn’t seem it’s due to a small change of precision. Am I the only one to experiences these test failure ? In case you don’t have such errors in the same conditions, how shall I report these errors for investigation ? Just the .out file would be enough ? I don’t know anything about these math/ tests, it’s the first time I see them failing like this.

This might be GCC bug 107879 (https://gcc.gnu.org/PR107879 ) or GCC
bug 107608 (https://gcc.gnu.org/PR107608 ) . I am suspecting the first
one. Jakub has a fix which he is I suspect will be submitting tomorrow
too.

Thanks,
Andrew Pinski

>
> Cheers,
> Romain
>
> FAIL: math/test-double-cosh
> FAIL: math/test-double-exp10
> FAIL: math/test-double-expm1
> FAIL: math/test-double-iseqsig
> FAIL: math/test-double-lgamma
> FAIL: math/test-double-log1p
> FAIL: math/test-double-tgamma
> FAIL: math/test-float-cosh
> FAIL: math/test-float-expm1
> FAIL: math/test-float-iseqsig
> FAIL: math/test-float-lgamma
> FAIL: math/test-float-log1p
> FAIL: math/test-float-tgamma
> FAIL: math/test-float128-catan
> FAIL: math/test-float128-catanh
> FAIL: math/test-float128-cosh
> FAIL: math/test-float128-exp10
> FAIL: math/test-float128-iseqsig
> FAIL: math/test-float128-lgamma
> FAIL: math/test-float128-log
> FAIL: math/test-float128-log1p
> FAIL: math/test-float128-pow
> FAIL: math/test-float128-tgamma
> FAIL: math/test-float128-y0
> FAIL: math/test-float128-y1
> FAIL: math/test-float32-cosh
> FAIL: math/test-float32-expm1
> FAIL: math/test-float32-iseqsig
> FAIL: math/test-float32-lgamma
> FAIL: math/test-float32-log1p
> FAIL: math/test-float32-tgamma
> FAIL: math/test-float32x-cosh
> FAIL: math/test-float32x-exp10
> FAIL: math/test-float32x-expm1
> FAIL: math/test-float32x-iseqsig
> FAIL: math/test-float32x-lgamma
> FAIL: math/test-float32x-log1p
> FAIL: math/test-float32x-tgamma
> FAIL: math/test-float64-cosh
> FAIL: math/test-float64-exp10
> FAIL: math/test-float64-expm1
> FAIL: math/test-float64-iseqsig
> FAIL: math/test-float64-lgamma
> FAIL: math/test-float64-log1p
> FAIL: math/test-float64-tgamma
> FAIL: math/test-float64x-cosh
> FAIL: math/test-float64x-iseqsig
> FAIL: math/test-float64x-lgamma
> FAIL: math/test-float64x-tgamma
> FAIL: math/test-ldouble-cosh
> FAIL: math/test-ldouble-iseqsig
> FAIL: math/test-ldouble-lgamma
> FAIL: math/test-ldouble-tgamma

Re: Many float/double errors in math/ testsuite with gcc master branch

[Romain GEISSLER via Libc-alpha]

Le 29 nov. 2022 à 04:38, Andrew Pinski <pinskia@gmail.com<mailto:pinskia@gmail.com>> a écrit :

This might be GCC bug 107879 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107879 ) or GCC
bug 107608 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107608 ) . I am suspecting the first
one. Jakub has a fix which he is I suspect will be submitting tomorrow
too.

Thanks,
Andrew Pinski

Hi,

Thanks for these pointers.

I tried to narrow a bit more today these regressions today. Most of them seems to have been introduced with the fixes for GCC bug 107569 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107569 ).

However all the iseqsig tests seems to fail because of another reason, it was already failing with the gcc master branch of 15th of October, and I couldn’t test earlier commits as I had build failures. I have found GCC bug 106805  (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106805 ) which might be related.

Cheers,
Romain

Re: Many float/double errors in math/ testsuite with gcc master branch

[Joseph Myers]

On Wed, 30 Nov 2022, Romain GEISSLER via Libc-alpha wrote:

> However all the iseqsig tests seems to fail because of another reason, 
> it was already failing with the gcc master branch of 15th of October, 
> and I couldn’t test earlier commits as I had build failures. I have 
> found GCC bug 106805 
> (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106805 ) which might be 
> related.

That ought to be marked as a regression if it appears with GCC 13 and not 
GCC 12.

-- 
Joseph S. Myers
joseph@codesourcery.com