Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[John Paul Adrian Glaubitz]

Hi!

It seems that git is failing its testsuite on all 64-bit Big-Endian targets,
full build log can be found in [1]. There seem to be multiple failures.

The issue first showed with 2.25-rc1 [2].

Thanks,
Adrian

> [1] https://buildd.debian.org/status/fetch.php?pkg=git&arch=s390x&ver=1%3A2.25.0%7Erc1-1&stamp=1578096339&raw=0
> [2] https://buildd.debian.org/status/logs.php?pkg=git&arch=s390x

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 1245 bytes --]

On 2020-01-04 at 20:14:31, John Paul Adrian Glaubitz wrote:
> Hi!
> 
> It seems that git is failing its testsuite on all 64-bit Big-Endian targets,
> full build log can be found in [1]. There seem to be multiple failures.
> 
> The issue first showed with 2.25-rc1 [2].

This is interesting, because there are different failures on different
platforms.  s390x shows failures only in t9100, which is one of the svn
tests.  From the diff between rc0 and rc1, I don't see anything which
should affect git-svn in that way.

ppc64 and sparc64 failing a lot more tests, mostly due to segfaults, but
from a cursory glance they all appear to be SVN-related.

Are we sure that some dependency of libsvn1 hasn't caused this problem?
I noticed that all of the broken builds seem to have libserf-1-1
1.3.9-8, which appears to have received an overhaul recently.  Of
course, it could be something else as well.

I got rid of my UltraSPARC hardware when I last moved, so I won't be
looking into this further, but it seems unlikely to me that the problem
is in Git when the only non-Windows C change between the two versions is
a one-line file name change.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 868 bytes --]

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[John Paul Adrian Glaubitz]

Hi!

On 1/4/20 9:58 PM, brian m. carlson wrote:
> I got rid of my UltraSPARC hardware when I last moved, so I won't be
> looking into this further, but it seems unlikely to me that the problem
> is in Git when the only non-Windows C change between the two versions is
> a one-line file name change.

You can get access to a very fast SPARC porterbox through the GCC compile
farm [1]. Anyone can request a free account.

Adrian

> [1] https://gcc.gnu.org/wiki/CompileFarm

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[John Paul Adrian Glaubitz]

Hi!

On 1/4/20 9:14 PM, John Paul Adrian Glaubitz wrote:
> It seems that git is failing its testsuite on all 64-bit Big-Endian targets,
> full build log can be found in [1]. There seem to be multiple failures.

Just checked out git with git from github (SCNR ;)) and built and ran the
testsuite on ppc64, s390x and sparc64.

ppc64:

make aggregate-results
make[3]: Entering directory '/srv/glaubitz/git-git/t'
for f in 'test-results'/t*-*.counts; do \
        echo "$f"; \
done | '/bin/sh' ./aggregate-results.sh
fixed   0
success 20683
failed  0
broken  239
total   21161

s390x:

make aggregate-results
make[3]: Entering directory '/home/glaubitz/git/t'
for f in 'test-results'/t*-*.counts; do \
        echo "$f"; \
done | '/bin/sh' ./aggregate-results.sh
fixed   0
success 21298
failed  0
broken  249
total   21760

sparc64:

make aggregate-results
make[3]: Entering directory '/home/glaubitz/git/t'
for f in 'test-results'/t*-*.counts; do \
        echo "$f"; \
done | '/bin/sh' ./aggregate-results.sh
fixed   0
success 20703
failed  0
broken  239
total   21176

So, it looks like the failures might be specific to the Debian package,
doesn't it? Where there maybe any recent commits that may have fixed
those issues?

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[brian m. carlson]

[-- Attachment #1: Type: text/plain, Size: 2207 bytes --]

On 2020-01-04 at 22:14:21, John Paul Adrian Glaubitz wrote:
> Hi!
> 
> On 1/4/20 9:14 PM, John Paul Adrian Glaubitz wrote:
> > It seems that git is failing its testsuite on all 64-bit Big-Endian targets,
> > full build log can be found in [1]. There seem to be multiple failures.
> 
> Just checked out git with git from github (SCNR ;)) and built and ran the
> testsuite on ppc64, s390x and sparc64.
> 
> ppc64:
> 
> make aggregate-results
> make[3]: Entering directory '/srv/glaubitz/git-git/t'
> for f in 'test-results'/t*-*.counts; do \
>         echo "$f"; \
> done | '/bin/sh' ./aggregate-results.sh
> fixed   0
> success 20683
> failed  0
> broken  239
> total   21161
> 
> s390x:
> 
> make aggregate-results
> make[3]: Entering directory '/home/glaubitz/git/t'
> for f in 'test-results'/t*-*.counts; do \
>         echo "$f"; \
> done | '/bin/sh' ./aggregate-results.sh
> fixed   0
> success 21298
> failed  0
> broken  249
> total   21760
> 
> sparc64:
> 
> make aggregate-results
> make[3]: Entering directory '/home/glaubitz/git/t'
> for f in 'test-results'/t*-*.counts; do \
>         echo "$f"; \
> done | '/bin/sh' ./aggregate-results.sh
> fixed   0
> success 20703
> failed  0
> broken  239
> total   21176
> 
> So, it looks like the failures might be specific to the Debian package,
> doesn't it? Where there maybe any recent commits that may have fixed
> those issues?

Did you build your version of Git with Subversion support, and if so,
which version of Subversion did you use (version of Debian package or
other source)?  The tests that were failing all require git-svn, which
in turn require libsvn-perl and subversion (/usr/bin/svn).  If you're
missing those packages, the Subversion tests will automatically be
skipped.

The latest version on master is 2.25-rc1, and that's failing in the
Debian package.  I haven't checked recently, but last I looked, the
Debian package wasn't applying any additional patches on top of Git, so
the version you're getting off GitHub is literally the same version that
you're getting in the Debian package.
-- 
brian m. carlson: Houston, Texas, US
OpenPGP: https://keybase.io/bk2204

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 868 bytes --]

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[John Paul Adrian Glaubitz]

Hi Brian!

On 1/4/20 11:48 PM, brian m. carlson wrote:
> Did you build your version of Git with Subversion support, and if so,
> which version of Subversion did you use (version of Debian package or
> other source)?  The tests that were failing all require git-svn, which
> in turn require libsvn-perl and subversion (/usr/bin/svn).  If you're
> missing those packages, the Subversion tests will automatically be
> skipped.

Good point. I just ran the configure script without any extra options.

I will have another look and make sure git-svn is enabled.

> The latest version on master is 2.25-rc1, and that's failing in the
> Debian package.  I haven't checked recently, but last I looked, the
> Debian package wasn't applying any additional patches on top of Git, so
> the version you're getting off GitHub is literally the same version that
> you're getting in the Debian package.

Okay. I wasn't sure whether there were any additional commits after the
2.25-rc1 tag, I hadn't checked yet as I was doing these tests on the
side while watching TV.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[Thomas Braun]

On 04.01.2020 21:58, brian m. carlson wrote:

> I got rid of my UltraSPARC hardware when I last moved, so I won't be
> looking into this further, but it seems unlikely to me that the problem
> is in Git when the only non-Windows C change between the two versions is
> a one-line file name change.
> 

Just in case someone is interested. Last year I've created a QEMU image
using MIPS big-endian hardware [1]. It is still on debian stretch and
like **slow** but does not require additional hardware.

[1]: https://github.com/t-b/debian-mips-qemu

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[John Paul Adrian Glaubitz]

On 1/7/20 12:31 AM, Thomas Braun wrote:
> On 04.01.2020 21:58, brian m. carlson wrote:
> 
>> I got rid of my UltraSPARC hardware when I last moved, so I won't be
>> looking into this further, but it seems unlikely to me that the problem
>> is in Git when the only non-Windows C change between the two versions is
>> a one-line file name change.
>>
> 
> Just in case someone is interested. Last year I've created a QEMU image
> using MIPS big-endian hardware [1]. It is still on debian stretch and
> like **slow** but does not require additional hardware.

FWIW, the GCC compile farm is supposed to get a new beefy MIPS server
soon as well so I have been told.

But as mentioned earlier, there is already a SPARC T5 in the GCC compile
farm which is open to use for any open-source developer [1].

Adrian

> [1] https://gcc.gnu.org/wiki/CompileFarm

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Testsuite failures on ppc64, sparc64 and s390x (64-bit BE)

[John Paul Adrian Glaubitz]

Hi!

On 1/4/20 11:48 PM, brian m. carlson wrote:
> Did you build your version of Git with Subversion support, and if so,
> which version of Subversion did you use (version of Debian package or
> other source)?  The tests that were failing all require git-svn, which
> in turn require libsvn-perl and subversion (/usr/bin/svn).  If you're
> missing those packages, the Subversion tests will automatically be
> skipped.
> 
> The latest version on master is 2.25-rc1, and that's failing in the
> Debian package.  I haven't checked recently, but last I looked, the
> Debian package wasn't applying any additional patches on top of Git, so
> the version you're getting off GitHub is literally the same version that
> you're getting in the Debian package.

I did some more testing and the failures are caused by sqlite3:

(sid_ppc64-dchroot)root@redpanda:/build/git-ceFUyK/git-2.25.0+next.20200130/t# ./t9108-git-svn-glob.sh
not ok 1 - test refspec globbing
#
#               mkdir -p trunk/src/a trunk/src/b trunk/doc &&
#               echo "hello world" > trunk/src/a/readme &&
#               echo "goodbye world" > trunk/src/b/readme &&
#               svn_cmd import -m "initial" trunk "$svnrepo"/trunk &&
#               svn_cmd co "$svnrepo" tmp &&
#               (
#                       cd tmp &&
#                       mkdir branches tags &&
#                       svn_cmd add branches tags &&
#                       svn_cmd cp trunk branches/start &&
#                       svn_cmd commit -m "start a new branch" &&
#                       svn_cmd up &&
#                       echo "hi" >> branches/start/src/b/readme &&
#                       poke branches/start/src/b/readme &&
#                       echo "hey" >> branches/start/src/a/readme &&
#                       poke branches/start/src/a/readme &&
#                       svn_cmd commit -m "hi" &&
#                       svn_cmd up &&
#                       svn_cmd cp branches/start tags/end &&
#                       echo "bye" >> tags/end/src/b/readme &&
#                       poke tags/end/src/b/readme &&
#                       echo "aye" >> tags/end/src/a/readme &&
#                       poke tags/end/src/a/readme &&
#                       svn_cmd commit -m "the end" &&
#                       echo "byebye" >> tags/end/src/b/readme &&
#                       poke tags/end/src/b/readme &&
#                       svn_cmd commit -m "nothing to see here"
#               ) &&
#               git config --add svn-remote.svn.url "$svnrepo" &&
#               git config --add svn-remote.svn.fetch \
#                                "trunk/src/a:refs/remotes/trunk" &&
#               git config --add svn-remote.svn.branches \
#                                "branches/*/src/a:refs/remotes/branches/*" &&
#               git config --add svn-remote.svn.tags\
#                                "tags/*/src/a:refs/remotes/tags/*" &&
#               git svn multi-fetch &&
#               git log --pretty=oneline refs/remotes/tags/end >actual &&
#               sed -e "s/^.\{41\}//" actual >output.end &&
#               test_cmp expect.end output.end &&
#               test "$(git rev-parse refs/remotes/tags/end~1)" = \
#                       "$(git rev-parse refs/remotes/branches/start)" &&
#               test "$(git rev-parse refs/remotes/branches/start~2)" = \
#                       "$(git rev-parse refs/remotes/trunk)" &&
#               test_must_fail git rev-parse refs/remotes/tags/end@3
#
not ok 2 - test left-hand-side only globbing
#
#               git config --add svn-remote.two.url "$svnrepo" &&
#               git config --add svn-remote.two.fetch trunk:refs/remotes/two/trunk &&
#               git config --add svn-remote.two.branches \
#                                "branches/*:refs/remotes/two/branches/*" &&
#               git config --add svn-remote.two.tags \
#                                "tags/*:refs/remotes/two/tags/*" &&
#               (
#                       cd tmp &&
#                       echo "try try" >> tags/end/src/b/readme &&
#                       poke tags/end/src/b/readme &&
#                       svn_cmd commit -m "try to try"
#               ) &&
#               git svn fetch two &&
#               git rev-list refs/remotes/two/tags/end >actual &&
#               test_line_count = 6 actual &&
#               git rev-list refs/remotes/two/branches/start >actual &&
#               test_line_count = 3 actual &&
#               test $(git rev-parse refs/remotes/two/branches/start~2) = \
#                    $(git rev-parse refs/remotes/two/trunk) &&
#               test $(git rev-parse refs/remotes/two/tags/end~3) = \
#                    $(git rev-parse refs/remotes/two/branches/start) &&
#               git log --pretty=oneline refs/remotes/two/tags/end >actual &&
#               sed -e "s/^.\{41\}//" actual >output.two &&
#               test_cmp expect.two output.two
#
ok 3 - prepare test disallow multi-globs
not ok 4 - test disallow multi-globs
#
#               git config --add svn-remote.three.url "$svnrepo" &&
#               git config --add svn-remote.three.fetch \
#                                trunk:refs/remotes/three/trunk &&
#               git config --add svn-remote.three.branches \
#                                "branches/*/t/*:refs/remotes/three/branches/*" &&
#               git config --add svn-remote.three.tags \
#                                "tags/*/*:refs/remotes/three/tags/*" &&
#               (
#                       cd tmp &&
#                       echo "try try" >> tags/end/src/b/readme &&
#                       poke tags/end/src/b/readme &&
#                       svn_cmd commit -m "try to try"
#               ) &&
#               test_must_fail git svn fetch three 2> stderr.three &&
#               test_cmp expect.three stderr.three
#
# failed 3 among 4 test(s)
1..4
(sid_ppc64-dchroot)root@redpanda:/build/git-ceFUyK/git-2.25.0+next.20200130/t# dpkg -i /libsqlite3-0_3.30.1-1_ppc64.deb
dpkg: warning: downgrading libsqlite3-0:ppc64 from 3.31.1-1 to 3.30.1-1
(Reading database ... 30145 files and directories currently installed.)
Preparing to unpack .../libsqlite3-0_3.30.1-1_ppc64.deb ...
Unpacking libsqlite3-0:ppc64 (3.30.1-1) over (3.31.1-1) ...
Setting up libsqlite3-0:ppc64 (3.30.1-1) ...
Processing triggers for libc-bin (2.29-10) ...
(sid_ppc64-dchroot)root@redpanda:/build/git-ceFUyK/git-2.25.0+next.20200130/t# ./t9108-git-svn-glob.sh
ok 1 - test refspec globbing
ok 2 - test left-hand-side only globbing
ok 3 - prepare test disallow multi-globs
ok 4 - test disallow multi-globs
# passed all 4 test(s)
1..4
(sid_ppc64-dchroot)root@redpanda:/build/git-ceFUyK/git-2.25.0+next.20200130/t#

I will bisect sqlite3 tomorrow to find out when the regression was introduced.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913