Git port to Debian SH4 may have trouble

Git port to Debian SH4 may have trouble

[Jeffrey Walton]

Hi Everyone,

This is a crummy bug report... I am  testing on Debian Unstable within
a SH4 Chroot. I'm seeing a hang during a git pull. Eventually I have
to CTRL+C, which results in "fatal: fetch-pack: invalid index-pack
output."

(sh4) debian-chroot:~# git clone -vv https://github.com/weidai11/cryptopp
Cloning into 'cryptopp'...
POST git-upload-pack (175 bytes)
want e9cbc4698a6b3ed067910862ad1355069f348d2b (HEAD)
want e9cbc4698a6b3ed067910862ad1355069f348d2b (refs/heads/master)
...
want 511806c0eba8ba5b5cedd4b4a814e96df92864a6 (refs/tags/CRYPTOPP_8_7_0)
POST git-upload-pack (gzip 1602 to 825 bytes)
remote: Enumerating objects: 28448, done.
remote: Counting objects: 100% (68/68), done.
remote: Compressing objects: 100% (50/50), done.
remote: Total 28448 (delta 29), reused 49 (delta 18), pack-reused 28380
Receiving objects: 100% (28448/28448), 27.01 MiB | 4.30 MiB/s, done.
Resolving deltas:   0% (0/20757)
^Cfatal: fetch-pack: invalid index-pack output
(sh4) debian-chroot:~#

(sh4) debian-chroot:~# git --version
git version 2.38.1

Jeff

Re: Git port to Debian SH4 may have trouble

[Jeff King]

On Tue, Nov 29, 2022 at 12:01:01PM -0500, Jeffrey Walton wrote:

> This is a crummy bug report... I am  testing on Debian Unstable within
> a SH4 Chroot. I'm seeing a hang during a git pull. Eventually I have
> to CTRL+C, which results in "fatal: fetch-pack: invalid index-pack
> output."

The message is probably a red herring. Once you terminate index-pack,
the fetch-pack process reading from it will see EOF before reading the
expected "pack ..." line, and generate that message.

So the hang is probably the only interesting part.

> (sh4) debian-chroot:~# git clone -vv https://github.com/weidai11/cryptopp
> Cloning into 'cryptopp'...
> POST git-upload-pack (175 bytes)
> want e9cbc4698a6b3ed067910862ad1355069f348d2b (HEAD)
> want e9cbc4698a6b3ed067910862ad1355069f348d2b (refs/heads/master)
> ...
> want 511806c0eba8ba5b5cedd4b4a814e96df92864a6 (refs/tags/CRYPTOPP_8_7_0)
> POST git-upload-pack (gzip 1602 to 825 bytes)
> remote: Enumerating objects: 28448, done.
> remote: Counting objects: 100% (68/68), done.
> remote: Compressing objects: 100% (50/50), done.
> remote: Total 28448 (delta 29), reused 49 (delta 18), pack-reused 28380
> Receiving objects: 100% (28448/28448), 27.01 MiB | 4.30 MiB/s, done.
> Resolving deltas:   0% (0/20757)
> ^Cfatal: fetch-pack: invalid index-pack output

If we're in the "resolving deltas" section, then that means we're
probably not hanging on receiving input from the remote side. At this
point we should have gotten all of the data and would be CPU-bound
reassembling deltas and computing their hashes.

Just a hunch, but does:

  git config --global pack.threads 1

help? The delta resolution is multi-threaded, but nothing else in the
clone should be.

-Peff

Re: Git port to Debian SH4 may have trouble

[Jeffrey Walton]

On Tue, Nov 29, 2022 at 2:02 PM Jeff King <peff@peff.net> wrote:
>
> On Tue, Nov 29, 2022 at 12:01:01PM -0500, Jeffrey Walton wrote:
>
> > This is a crummy bug report... I am  testing on Debian Unstable within
> > a SH4 Chroot. I'm seeing a hang during a git pull. Eventually I have
> > to CTRL+C, which results in "fatal: fetch-pack: invalid index-pack
> > output."
>
> The message is probably a red herring. Once you terminate index-pack,
> the fetch-pack process reading from it will see EOF before reading the
> expected "pack ..." line, and generate that message.
>
> So the hang is probably the only interesting part.
>
> > (sh4) debian-chroot:~# git clone -vv https://github.com/weidai11/cryptopp
> > Cloning into 'cryptopp'...
> > POST git-upload-pack (175 bytes)
> > want e9cbc4698a6b3ed067910862ad1355069f348d2b (HEAD)
> > want e9cbc4698a6b3ed067910862ad1355069f348d2b (refs/heads/master)
> > ...
> > want 511806c0eba8ba5b5cedd4b4a814e96df92864a6 (refs/tags/CRYPTOPP_8_7_0)
> > POST git-upload-pack (gzip 1602 to 825 bytes)
> > remote: Enumerating objects: 28448, done.
> > remote: Counting objects: 100% (68/68), done.
> > remote: Compressing objects: 100% (50/50), done.
> > remote: Total 28448 (delta 29), reused 49 (delta 18), pack-reused 28380
> > Receiving objects: 100% (28448/28448), 27.01 MiB | 4.30 MiB/s, done.
> > Resolving deltas:   0% (0/20757)
> > ^Cfatal: fetch-pack: invalid index-pack output
>
> If we're in the "resolving deltas" section, then that means we're
> probably not hanging on receiving input from the remote side. At this
> point we should have gotten all of the data and would be CPU-bound
> reassembling deltas and computing their hashes.
>
> Just a hunch, but does:
>
>   git config --global pack.threads 1
>
> help? The delta resolution is multi-threaded, but nothing else in the
> clone should be.

Yes, `git config --global pack.threads 1` allowed things to continue.
The check-out was successful.

Jeff

Re: Git port to Debian SH4 may have trouble

[Jeff King]

On Tue, Nov 29, 2022 at 04:49:30PM -0500, Jeffrey Walton wrote:

> > Just a hunch, but does:
> >
> >   git config --global pack.threads 1
> >
> > help? The delta resolution is multi-threaded, but nothing else in the
> > clone should be.
> 
> Yes, `git config --global pack.threads 1` allowed things to continue.
> The check-out was successful.

OK, that narrows it down. The question then is why threads don't work.
Is there something broken with threading or luck primitives on that
platform? Or are we doing something sketchy with concurrency that
happens to work on Intel but not elsewhere, and we end up in some kind
of deadlock?

I suspect if we want to know more, you'd need to use gdb to grab a
backtrace for each of the threads during the hang to see what they're
trying to do.

-Peff

Re: Git port to Debian SH4 may have trouble

[Jeffrey Walton]

On Tue, Nov 29, 2022 at 5:14 PM Jeff King <peff@peff.net> wrote:
>
> On Tue, Nov 29, 2022 at 04:49:30PM -0500, Jeffrey Walton wrote:
>
> > > Just a hunch, but does:
> > >
> > >   git config --global pack.threads 1
> > >
> > > help? The delta resolution is multi-threaded, but nothing else in the
> > > clone should be.
> >
> > Yes, `git config --global pack.threads 1` allowed things to continue.
> > The check-out was successful.
>
> OK, that narrows it down. The question then is why threads don't work.
> Is there something broken with threading or luck primitives on that
> platform? Or are we doing something sketchy with concurrency that
> happens to work on Intel but not elsewhere, and we end up in some kind
> of deadlock?

Probably something broken with the port. Or that's what I would put my money on.

> I suspect if we want to know more, you'd need to use gdb to grab a
> backtrace for each of the threads during the hang to see what they're
> trying to do.

Yeah, so GDB won't install at the moment. I tried earlier today. It
has some unsatisfied dependencies. Failure to build from source
(FTBFS), iirc.

Jeff