Re: public-inbox-convert hangs on systems using musl libc

[Eric Wong <e@80x24.org>]

Chris Brannon <chris@the-brannons.com> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> > Are you able to isolate whether $PIPE_BUFSIZ or MAX_INFLIGHT on
> > it's own fixes it?
> 
> MAX_INFLIGHT is sufficient.  In fact, $PIPE_BUFSIZ had no effect at all
> when I tried just now.

OK, good to know.  Thanks so far for your other testing and
reports.

> > Capping MAX_INFLIGHT to a smaller value is probably fine (maybe
> > 10 can work).
> 
> I tried with 10 just now, and that was fine also.

Oops, so POSIX::PIPE_BUF is actually 4096 on Linux.
(`perl -MPOSIX -E 'say POSIX::PIPE_BUF'` shows 4096 on the
Alpine cfarm machine)

But I'm still confused as to why the original code didn't
work on Linux since the pipe can't be smaller than 4096...

The following changes MAX_INFLIGHT to 23 for all platforms (and
explains things better).  I also wonder if you can find a
threshold for when things start failing (e.g. hardcoding 24
should fail on FreeBSD)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 882a9a4a..a1af776b 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -28,8 +28,10 @@ our $in_cleanup;
 our $RDTIMEO = 60_000; # milliseconds
 our $async_warn; # true in read-only daemons
 
-use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) /
-	65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1
+# 512: POSIX PIPE_BUF minimum (see pipe(7))
+# 3: @$inflight is flattened [ $OID, $cb, $arg ]
+# 65: SHA-256 hex size + "\n" in preparation for git using non-SHA1
+use constant MAX_INFLIGHT => 512 * 3 / 65;
 
 my %GIT_ESC = (
 	a => "\a",