From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 21/36] ipc: use shutdown(2), base atfork* callback
Date: Thu, 31 Dec 2020 13:51:39 +0000 [thread overview]
Message-ID: <20201231135154.6070-22-e@80x24.org> (raw)
In-Reply-To: <20201231135154.6070-1-e@80x24.org>
shutdown(2) on a socket can be preferable if there's multiple
forked processes writing to a single worker and we really want
to shut things down ASAP.
It may also be good to provide an ipc_worker_exit method which
subclasses can override if needed for graceful shutdown. But we
won't need equivalents to atexit(3) since we can rely on DESTROY
handlers given this is Perl5.
---
lib/PublicInbox/IPC.pm | 49 ++++++++++++++++++++++++++++--------------
1 file changed, 33 insertions(+), 16 deletions(-)
diff --git a/lib/PublicInbox/IPC.pm b/lib/PublicInbox/IPC.pm
index 0baa218c..ed10cf44 100644
--- a/lib/PublicInbox/IPC.pm
+++ b/lib/PublicInbox/IPC.pm
@@ -56,8 +56,6 @@ sub ipc_return ($$$) {
sub ipc_worker_loop ($$) {
my ($self, $s2) = @_;
- $self->ipc_atfork_child if $self->can('ipc_atfork_child');
- $s2->autoflush(1);
while (my $rec = _get_rec($s2)) {
my ($wantarray, $sub, @args) = @$rec;
if (!defined($wantarray)) { # no waiting if client doesn't care
@@ -73,7 +71,7 @@ sub ipc_worker_loop ($$) {
}
}
-sub ipc_worker_spawn ($$$) {
+sub ipc_worker_spawn {
my ($self, $ident, $oldset) = @_;
return unless $enc;
my $pid = $self->{-ipc_worker_pid};
@@ -82,43 +80,62 @@ sub ipc_worker_spawn ($$$) {
my ($s1, $s2);
socketpair($s1, $s2, AF_UNIX, SOCK_STREAM, 0) or die "socketpair: $!";
my $sigset = $oldset // PublicInbox::Sigfd::block_signals();
+ my $parent = $$;
+ $self->ipc_atfork_parent;
defined($pid = fork) or die "fork: $!";
if ($pid == 0) {
- undef $s1;
- local $0 = $ident;
+ eval { PublicInbox::DS->Reset };
+ $self->{-ipc_parent_pid} = $parent;
+ close $s1 or die "close(\$s1): $!";
+ $s2->autoflush(1);
$SIG{$_} = 'IGNORE' for (qw(TERM INT QUIT));
+ local $0 = $ident;
PublicInbox::Sigfd::sig_setmask($oldset);
+ $self->ipc_atfork_child;
eval { ipc_worker_loop($self, $s2) };
- die "worker $ident died: $@\n" if $@;
- $self->ipc_at_worker_exit if $self->can('ipc_at_worker_exit');
+ die "worker $ident PID:$$ died: $@\n" if $@;
exit;
}
PublicInbox::Sigfd::sig_setmask($sigset) unless $oldset;
+ close $s2 or die "close(\$s2): $!";
$s1->autoflush(1);
$self->{-ipc_sock} = $s1;
$self->{-ipc_worker_pid} = $pid;
}
-sub ipc_reap_worker { # dwaitpid callback
+sub ipc_worker_reap { # dwaitpid callback
my ($self, $pid) = @_;
warn "PID:$pid died with \$?=$?\n" if $?;
}
+# for base class, override in superclasses
+sub ipc_atfork_parent {}
+sub ipc_atfork_child {}
+
+sub ipc_worker_exit {
+ my (undef, $code) = @_;
+ exit($code);
+}
+
sub ipc_worker_stop {
my ($self) = @_;
my $pid;
- if (delete $self->{-ipc_sock}) {
- $pid = delete $self->{-ipc_worker_pid} or die "no PID?";
- } else {
+ my $s1 = delete $self->{-ipc_sock} or do {
$pid = delete $self->{-ipc_worker_pid} and
- die "unexpected PID:$pid";
- }
- return unless $pid;
- eval { PublicInbox::DS::dwaitpid($pid, \&ipc_reap_worker, $self) };
+ die "unexpected PID:$pid without ipc_sock";
+ return;
+ };
+ $pid = delete $self->{-ipc_worker_pid} or die "no PID?";
+ _send_rec($s1, [ undef, 'ipc_worker_exit', 0 ]);
+ shutdown($s1, 2) or die "shutdown(\$s1) for PID:$pid";
+ eval {
+ my $reap = $self->can('ipc_worker_reap');
+ PublicInbox::DS::dwaitpid($pid, $reap, $self);
+ };
if ($@) {
my $wp = waitpid($pid, 0);
$pid == $wp or die "waitpid($pid) returned $wp: \$?=$?";
- ipc_reap_worker($self, $pid);
+ $self->ipc_worker_reap($pid);
}
}
next prev parent reply other threads:[~2020-12-31 13:51 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-31 13:51 [PATCH 00/36] another round of lei stuff Eric Wong
2020-12-31 13:51 ` [PATCH 01/36] import: respect init.defaultBranch Eric Wong
2020-12-31 13:51 ` [PATCH 02/36] lei_store: use per-machine refname as git HEAD Eric Wong
2020-12-31 13:51 ` [PATCH 03/36] revert "lei_store: use per-machine refname as git HEAD" Eric Wong
2020-12-31 13:51 ` [PATCH 04/36] lei_to_mail: initial implementation for writing mbox formats Eric Wong
2020-12-31 13:51 ` [PATCH 05/36] sharedkv: fork()-friendly key-value store Eric Wong
2020-12-31 13:51 ` [PATCH 06/36] sharedkv: split out index_values Eric Wong
2020-12-31 13:51 ` [PATCH 07/36] lei_to_mail: start atomic and compressed mbox writing Eric Wong
2020-12-31 13:51 ` [PATCH 08/36] mboxreader: new class for reading various mbox formats Eric Wong
2020-12-31 13:51 ` [PATCH 09/36] lei_to_mail: start --augment, dedupe, bz2 and xz Eric Wong
2020-12-31 13:51 ` [PATCH 10/36] lei: implement various deduplication strategies Eric Wong
2020-12-31 13:51 ` [PATCH 11/36] lei_to_mail: lazy-require LeiDedupe Eric Wong
2020-12-31 13:51 ` [PATCH 12/36] lei_to_mail: support for non-seekable outputs Eric Wong
2020-12-31 13:51 ` [PATCH 13/36] lei_to_mail: support Maildir, fix+test --augment Eric Wong
2020-12-31 13:51 ` [PATCH 14/36] ipc: generic IPC dispatch based on Storable Eric Wong
2020-12-31 13:51 ` [PATCH 15/36] ipc: support Sereal Eric Wong
2020-12-31 13:51 ` [PATCH 16/36] lei_store: add ->set_eml, ->add_eml can return smsg Eric Wong
2020-12-31 13:51 ` [PATCH 17/36] lei: rename "extinbox" => "external" Eric Wong
2020-12-31 13:51 ` [PATCH 18/36] mid: use defined-or with `push' for uniqueness check Eric Wong
2020-12-31 13:51 ` [PATCH 19/36] mid: hoist out mids_in sub Eric Wong
2020-12-31 13:51 ` [PATCH 20/36] lei_store: handle messages without Message-ID at all Eric Wong
2020-12-31 13:51 ` Eric Wong [this message]
2020-12-31 13:51 ` [PATCH 22/36] lei_to_mail: unlink mboxes if not augmenting Eric Wong
2020-12-31 13:51 ` [PATCH 23/36] lei: add --mfolder as an --output alias Eric Wong
2020-12-31 13:51 ` [PATCH 24/36] spawn: move run_die here from PublicInbox::Import Eric Wong
2020-12-31 13:51 ` [PATCH 25/36] init: remove embedded UnlinkMe package Eric Wong
2020-12-31 13:51 ` [PATCH 26/36] t/run: avoid uninitialized var on incomplete test Eric Wong
2020-12-31 13:51 ` [PATCH 27/36] gcf2client: reap process on DESTROY Eric Wong
2020-12-31 13:51 ` [PATCH 28/36] lei_to_mail: open FIFOs O_WRONLY so we block Eric Wong
2020-12-31 13:51 ` [PATCH 29/36] searchidxshard: call DS->Reset at worker start Eric Wong
2020-12-31 13:51 ` [PATCH 30/36] t/ipc.t: test for references via `die' Eric Wong
2020-12-31 13:51 ` [PATCH 31/36] use PublicInbox::DS for dwaitpid Eric Wong
2020-12-31 13:51 ` [PATCH 32/36] syscall: SFD_NONBLOCK can be a constant, again Eric Wong
2020-12-31 13:51 ` [PATCH 33/36] lei: avoid Spawn package when starting daemon Eric Wong
2020-12-31 13:51 ` [PATCH 34/36] avoid calling waitpid from children in DESTROY Eric Wong
2020-12-31 13:51 ` [PATCH 35/36] ds: clobber $in_loop first at reset Eric Wong
2020-12-31 13:51 ` [PATCH 36/36] on_destroy: support PID owner guard Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201231135154.6070-22-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).