From: Chris Brannon <chris@the-brannons.com>
To: Eric Wong <e@80x24.org>
Cc: meta@public-inbox.org
Subject: Re: public-inbox-convert hangs on systems using musl libc
Date: Wed, 21 Dec 2022 05:46:27 -0800 [thread overview]
Message-ID: <871qosna30.fsf@the-brannons.com> (raw)
In-Reply-To: <20221221122102.M600156@dcvr> (Eric Wong's message of "Wed, 21 Dec 2022 12:21:02 +0000")
[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]
Eric Wong <e@80x24.org> writes:
> Do you know which pipes are which? "lsof -p $PID +E" can help
> with connectivity checking, as can script/dtas-graph in
> https://80x24.org/dtas.git if you have Graph::Easy
Yes. I'm attaching my lsof output and a typescript.
The processes of interest here are 4849 public-inbox-convert and 4879
git cat-file.
PID 4849's FD 11 is the write end of a pipe, with 4879's stdin as the
read end.
PID 4849's FD 12 is the read end of a pipe, with 4879's stdout as the
write end. At the point of the hang, 4849 is trying to write a SHA1 to
FD 11, while 4879 is writing an email message to its stdout.
> Some shots in the dark:
>
> 2. Tweak $PIPE_BUFSIZ and/or MAX_INFLIGHT to smaller values. e.g.
>
> diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
> index 882a9a4a..ec40edd7 100644
> --- a/lib/PublicInbox/Git.pm
> +++ b/lib/PublicInbox/Git.pm
> @@ -23,13 +23,12 @@ use Carp qw(croak carp);
> use Digest::SHA ();
> use PublicInbox::DS qw(dwaitpid);
> our @EXPORT_OK = qw(git_unquote git_quote);
> -our $PIPE_BUFSIZ = 65536; # Linux default
> +our $PIPE_BUFSIZ = 4096; # Linux default
> our $in_cleanup;
> our $RDTIMEO = 60_000; # milliseconds
> our $async_warn; # true in read-only daemons
>
> -use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) /
> - 65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1
> +use constant MAX_INFLIGHT => 4;
>
> my %GIT_ESC = (
> a => "\a",
>
This right here seems to have fixed it, when testing locally.
PS. Thank you for that lsof command. I've never used lsof in that way;
I'll have to add that to my *nix debugging toolbelt.
-- Chris
[-- Attachment #2: lsofout.txt --]
[-- Type: text/plain, Size: 4867 bytes --]
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
screen 4028 chris 11u CHR 5,2 0t0 86 /dev/ptmx ->/dev/pts/7 2214,mksh,0u 2214,mksh,1u 2214,mksh,2u 4849,public-in,0u 4849,public-in,1u 4849,public-in,2u 4874,shard[0],0u 4874,shard[0],1u 4874,shard[0],2u 4879,git,2u
public-in 4849 chris cwd DIR 0,42 412 17211 /files/cmb-files/repos
public-in 4849 chris rtd DIR 0,26 28 34 /
public-in 4849 chris txt REG 0,26 14144 239824 /usr/bin/perl
public-in 4849 chris mem REG 0,26 2969912 227085 /usr/lib/libstdc++.so.6.0.30
public-in 4849 chris mem REG 0,26 100328 564900 /usr/lib/libz.so.1.2.13
public-in 4849 chris mem REG 0,26 30760 399826 /usr/lib/libuuid.so.1.3.0
public-in 4849 chris mem REG 0,26 121208 226756 /usr/lib/libgcc_s.so.1
public-in 4849 chris mem REG 0,26 2234600 563318 /usr/lib/libxapian.so.30.12.2
public-in 4849 chris mem REG 0,26 500184 219769 /usr/lib/perl5/vendor_perl/auto/Search/Xapian/Xapian.so
public-in 4849 chris mem REG 0,26 132928 239703 /usr/lib/perl5/core_perl/auto/Compress/Raw/Zlib/Zlib.so
public-in 4849 chris mem REG 0,26 1351488 347561 /usr/lib/libsqlite3.so.0.8.6
public-in 4849 chris mem REG 0,26 180112 239928 /usr/lib/perl5/vendor_perl/auto/DBD/SQLite/SQLite.so
public-in 4849 chris mem REG 0,26 138264 240017 /usr/lib/perl5/vendor_perl/auto/DBI/DBI.so
public-in 4849 chris mem REG 0,26 2310 4758 /usr/share/zoneinfo/PST8PDT
public-in 4849 chris mem REG 0,26 18168 239686 /usr/lib/perl5/core_perl/auto/MIME/Base64/Base64.so
public-in 4849 chris mem REG 0,26 657216 239665 /usr/lib/perl5/core_perl/auto/re/re.so
public-in 4849 chris mem REG 0,26 116640 239674 /usr/lib/perl5/core_perl/auto/Storable/Storable.so
public-in 4849 chris mem REG 0,26 46920 239702 /usr/lib/perl5/core_perl/auto/Encode/Encode.so
public-in 4849 chris mem REG 0,26 39608 218906 /usr/lib/perl5/vendor_perl/auto/Email/Address/XS/XS.so
public-in 4849 chris mem REG 0,26 46920 239675 /usr/lib/perl5/core_perl/auto/Socket/Socket.so
public-in 4849 chris mem REG 0,26 59128 239687 /usr/lib/perl5/core_perl/auto/List/Util/Util.so
public-in 4849 chris mem REG 0,26 46840 238715 /usr/lib/perl5/core_perl/auto/Digest/SHA/SHA.so
public-in 4849 chris mem REG 0,26 30456 239671 /usr/lib/perl5/core_perl/auto/Time/HiRes/HiRes.so
public-in 4849 chris mem REG 0,26 18168 239744 /usr/lib/perl5/core_perl/auto/Cwd/Cwd.so
public-in 4849 chris mem REG 0,26 30536 239695 /usr/lib/perl5/core_perl/auto/File/Glob/Glob.so
public-in 4849 chris mem REG 0,26 22264 239689 /usr/lib/perl5/core_perl/auto/IO/IO.so
public-in 4849 chris mem REG 0,26 108360 239682 /usr/lib/perl5/core_perl/auto/POSIX/POSIX.so
public-in 4849 chris mem REG 0,26 22344 239697 /usr/lib/perl5/core_perl/auto/Fcntl/Fcntl.so
public-in 4849 chris mem REG 0,26 3789544 239902 /usr/lib/libperl.so.5.36.0
public-in 4849 chris mem REG 0,26 4251720 228062 /usr/lib/libc.so
public-in 4849 chris 0u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u
public-in 4849 chris 1u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u
public-in 4849 chris 2u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u
public-in 4849 chris 3w CHR 1,3 0t0 4 /dev/null
public-in 4849 chris 4uW REG 0,42 0 2074958 /files/cmb-files/repos/metanew/inbox.lock
public-in 4849 chris 5ur REG 0,42 24576 2074742 /files/cmb-files/repos/metanew/msgmap.sqlite3
public-in 4849 chris 6w FIFO 0,12 0t0 26905684 pipe 4874,shard[0],5r
public-in 4849 chris 7r FIFO 0,12 0t0 26905685 pipe 4874,shard[0],8w
public-in 4849 chris 8ur REG 0,42 69632 2074960 /files/cmb-files/repos/metanew/xap15/over.sqlite3
public-in 4849 chris 9u REG 0,49 461301 410306 /tmp/PerlIO_LaapMm (deleted)
public-in 4849 chris 10u REG 0,42 24576 2074986 /files/cmb-files/repos/metanew/mm_tmp-4849-i60n
public-in 4849 chris 11w FIFO 0,12 0t0 26905179 pipe 4879,git,0r
public-in 4849 chris 12r FIFO 0,12 0t0 26905180 pipe 4879,git,1w
public-in 4849 chris 13u REG 0,42 16928 2074743 /files/cmb-files/repos/metanew/msgmap.sqlite3-journal
public-in 4849 chris 14u REG 0,42 66176 2074961 /files/cmb-files/repos/metanew/xap15/over.sqlite3-journal
shard[0] 4874 chris 5r FIFO 0,12 0t0 26905684 pipe 4849,public-in,6w
shard[0] 4874 chris 8w FIFO 0,12 0t0 26905685 pipe 4849,public-in,7r
git 4879 chris 0r FIFO 0,12 0t0 26905179 pipe 4849,public-in,11w
git 4879 chris 1w FIFO 0,12 0t0 26905180 pipe 4849,public-in,12r
[-- Attachment #3: typescript --]
[-- Type: text/plain, Size: 610 bytes --]
Script started on 2022-12-21 05:12:15-08:00 [TERM="screen.linux" TTY="/dev/pts/10" COLUMNS="80" LINES="30"]
chris@beast:/home/chris $ doas strace -p 4849
\rdoas (chris@beast) password:
strace: Process 4849 attached
write(11, "4187293ba385ec204df84742b7197f4d"..., 41^Cstrace: Process 4849 detached
<detached ...>
Interrupt
chris@beast:/home/chris $ doas strace -p 4879
strace: Process 4879 attached
write(1, "Return-Path: <e@80x24.org>\nX-Spa"..., 3793^Cstrace: Process 4879 detached
<detached ...>
Interrupt
chris@beast:/home/chris $ exit
Script done on 2022-12-21 05:12:43-08:00 [COMMAND_EXIT_CODE="130"]
next prev parent reply other threads:[~2022-12-21 13:46 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-21 11:28 public-inbox-convert hangs on systems using musl libc Chris Brannon
2022-12-21 12:21 ` Eric Wong
2022-12-21 13:46 ` Chris Brannon [this message]
2022-12-21 19:48 ` Eric Wong
2022-12-21 20:46 ` Chris Brannon
2022-12-21 21:11 ` Eric Wong
2022-12-21 22:17 ` Chris Brannon
2022-12-21 23:22 ` [PATCH] git: cap MAX_INFLIGHT value to POSIX minimum Eric Wong
2022-12-21 23:57 ` Chris Brannon
2023-01-04 3:49 ` [PATCH] git: fix asynchronous batching for deep pipelines Eric Wong
2023-01-05 1:08 ` Chris Brannon
2023-01-05 1:44 ` [PATCH] git: write_all: remove leftover debug messages Eric Wong
2023-01-05 7:32 ` Chris Brannon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=871qosna30.fsf@the-brannons.com \
--to=chris@the-brannons.com \
--cc=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).