user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 12/12] ds: drop dwaitpid, switch to waitpid(-1)
Date: Tue, 17 Jan 2023 07:19:11 +0000	[thread overview]
Message-ID: <20230117071911.1577890-13-e@80x24.org> (raw)
In-Reply-To: <20230117071911.1577890-1-e@80x24.org>

With no remaining users, we can drop dwaitpid and switch
awaitpid to rely on waitpid(-1) to save syscalls.
---
 Documentation/technical/ds.txt |  2 +-
 lib/PublicInbox/DS.pm          | 68 +++++++---------------------------
 2 files changed, 15 insertions(+), 55 deletions(-)

diff --git a/Documentation/technical/ds.txt b/Documentation/technical/ds.txt
index 5a1655a1..89cc05af 100644
--- a/Documentation/technical/ds.txt
+++ b/Documentation/technical/ds.txt
@@ -81,7 +81,7 @@ New features
 
 * IO::Socket::SSL support (for NNTPS, STARTTLS+NNTP, HTTPS)
 
-* dwaitpid (waitpid wrapper) support for reaping dead children
+* awaitpid (waitpid wrapper) support for reaping dead children
 
 * reliable signal wakeups are supported via signalfd on Linux,
   EVFILT_SIGNAL on *BSDs via IO::KQueue.
diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index 9563a1cb..c849f515 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -32,11 +32,10 @@ use PublicInbox::Syscall qw(:epoll);
 use PublicInbox::Tmpfile;
 use Errno qw(EAGAIN EINVAL);
 use Carp qw(carp croak);
-our @EXPORT_OK = qw(now msg_more dwaitpid awaitpid add_timer add_uniq_timer);
+our @EXPORT_OK = qw(now msg_more awaitpid add_timer add_uniq_timer);
 
 my %Stack;
 my $nextq; # queue for next_tick
-my $wait_pids; # list of [ pid, callback, callback_arg ]
 my $AWAIT_PIDS; # pid => [ $callback, @args ]
 my $reap_armed;
 my $ToClose; # sockets to close when event loop is done
@@ -75,11 +74,11 @@ sub Reset {
 		# we may be iterating inside one of these on our stack
 		my @q = delete @Stack{keys %Stack};
 		for my $q (@q) { @$q = () }
-		$AWAIT_PIDS = $wait_pids = $nextq = $ToClose = undef;
+		$AWAIT_PIDS = $nextq = $ToClose = undef;
 		$ep_io = undef; # closes real $Epoll FD
 		$Epoll = undef; # may call DSKQXS::DESTROY
-	} while (@Timers || keys(%Stack) || $nextq || $wait_pids ||
-		$ToClose || keys(%DescriptorMap) || $AWAIT_PIDS ||
+	} while (@Timers || keys(%Stack) || $nextq || $AWAIT_PIDS ||
+		$ToClose || keys(%DescriptorMap) ||
 		$PostLoopCallback || keys(%UniqTimer));
 
 	$reap_armed = undef;
@@ -209,43 +208,23 @@ sub await_cb ($;@) {
 	warn "E: awaitpid($pid): $@" if $@;
 }
 
-# We can't use waitpid(-1) safely here since it can hit ``, system(),
-# and other things.  So we scan the $wait_pids list, which is hopefully
-# not too big.  We keep $wait_pids small by not calling dwaitpid()
-# until we've hit EOF when reading the stdout of the child.
-
+# This relies on our Perl process is single-threaded, or at least
+# no threads are spawning and waiting on processes (``, system(), etc...)
+# Threads are officially discouraged by the Perl5 team, and I expect
+# that to remain the case.
 sub reap_pids {
 	$reap_armed = undef;
-	my $tmp = $wait_pids // [];
-	$wait_pids = undef;
-	$Stack{reap_runq} = $tmp;
 	my $oldset = block_signals();
-
-	# old API
-	foreach my $ary (@$tmp) {
-		my ($pid, $cb, $arg) = @$ary;
-		my $ret = waitpid($pid, WNOHANG);
-		if ($ret == 0) {
-			push @$wait_pids, $ary; # autovivifies @$wait_pids
-		} elsif ($ret == $pid) {
-			if ($cb) {
-				eval { $cb->($arg, $pid) };
-				warn "E: dwaitpid($pid) in_loop: $@" if $@;
-			}
+	while (1) {
+		my $pid = waitpid(-1, WNOHANG) // last;
+		last if $pid <= 0;
+		if (defined(my $cb_args = delete $AWAIT_PIDS->{$pid})) {
+			await_cb($pid, @$cb_args) if $cb_args;
 		} else {
-			warn "waitpid($pid, WNOHANG) = $ret, \$!=$!, \$?=$?";
+			warn "W: reaped unknown PID=$pid: \$?=$?\n";
 		}
 	}
-
-	# new API TODO: convert to waitpid(-1) in the future as long
-	# as we don't use threads
-	for my $pid (keys %$AWAIT_PIDS) {
-		my $wpid = waitpid($pid, WNOHANG) // next;
-		my $cb_args = delete $AWAIT_PIDS->{$wpid} or next;
-		await_cb($pid, @$cb_args);
-	}
 	sig_setmask($oldset);
-	delete $Stack{reap_runq};
 }
 
 # reentrant SIGCHLD handler (since reap_pids is not reentrant)
@@ -719,25 +698,6 @@ sub long_response ($$;@) {
 	undef;
 }
 
-sub dwaitpid ($;$$) {
-	my ($pid, $cb, $arg) = @_;
-	if ($in_loop) {
-		push @$wait_pids, [ $pid, $cb, $arg ];
-		# We could've just missed our SIGCHLD, cover it, here:
-		enqueue_reap();
-	} else {
-		my $ret = waitpid($pid, 0);
-		if ($ret == $pid) {
-			if ($cb) {
-				eval { $cb->($arg, $pid) };
-				carp "E: dwaitpid($pid) !in_loop: $@" if $@;
-			}
-		} else {
-			carp "waitpid($pid, 0) = $ret, \$!=$!, \$?=$?";
-		}
-	}
-}
-
 sub awaitpid {
 	my ($pid, @cb_args) = @_;
 	$AWAIT_PIDS->{$pid} //= @cb_args ? \@cb_args : 0;

      parent reply	other threads:[~2023-01-17  7:19 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-17  7:18 [PATCH 00/12] improve process reaping Eric Wong
2023-01-17  7:19 ` [PATCH 01/12] ipc: remove {-reap_async} field Eric Wong
2023-01-17  7:19 ` [PATCH 02/12] t/solver_git.t: fix test message Eric Wong
2023-01-17  7:19 ` [PATCH 03/12] qspawn: drop {psgi_env} deref Eric Wong
2023-01-17  7:19 ` [PATCH 04/12] ds: introduce awaitpid, switch ProcessPipe users Eric Wong
2023-01-18  2:10   ` [PATCH 13/12] qspawn: use ->DESTROY to force ->finalize Eric Wong
2023-01-17  7:19 ` [PATCH 05/12] git|gcf2: switch to awaitpid Eric Wong
2023-01-17  7:19 ` [PATCH 06/12] watch: " Eric Wong
2023-01-17  7:19 ` [PATCH 07/12] watch: simplify internal data structures Eric Wong
2023-01-17  7:19 ` [PATCH 08/12] eofpipe: drop {arg} support for now Eric Wong
2023-01-17  7:19 ` [PATCH 09/12] watch: IMAP and NNTP polling can use the same interval Eric Wong
2023-01-17  7:19 ` [PATCH 10/12] ipc: drop unused $args from ->ipc_worker_stop Eric Wong
2023-01-17  7:19 ` [PATCH 11/12] ipc+lei: switch to awaitpid Eric Wong
2023-01-17  7:19 ` Eric Wong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230117071911.1577890-13-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).