* [PATCH] httpd/async: set O_NONBLOCK correctly
@ 2021-02-05 0:13 5% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2021-02-05 0:13 UTC (permalink / raw)
To: meta
While Perl tie is nice for some things, getting
IO::Handle->blocking to work transparently with it doesn't
seem possible at the moment.
Add some examples in t/spawn.t for future hackers.
Fixes: 22e51bd9da476fa9 ("qspawn: switch to ProcessPipe via popen_rd")
---
lib/PublicInbox/HTTPD/Async.pm | 3 ++-
t/spawn.t | 5 +++++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/lib/PublicInbox/HTTPD/Async.pm b/lib/PublicInbox/HTTPD/Async.pm
index bd1fd8fa..1de9501d 100644
--- a/lib/PublicInbox/HTTPD/Async.pm
+++ b/lib/PublicInbox/HTTPD/Async.pm
@@ -37,7 +37,8 @@ sub new {
arg => $arg, # arg for $cb
end_obj => $end_obj, # like END{}, can ->event_step
}, $class;
- IO::Handle::blocking($io, 0);
+ my $pp = tied *$io;
+ $pp->{fh}->blocking(0) // die "$io->blocking(0): $!";
$self->SUPER::new($io, EPOLLIN | EPOLLET);
}
diff --git a/t/spawn.t b/t/spawn.t
index 0eed79bb..6f811ec1 100644
--- a/t/spawn.t
+++ b/t/spawn.t
@@ -77,6 +77,11 @@ EOF
{
my $fh = popen_rd([qw(printf foo\nbar)]);
ok(fileno($fh) >= 0, 'tied fileno works');
+ my $tfh = (tied *$fh)->{fh};
+ is($tfh->blocking(0), 1, '->blocking was true');
+ is($tfh->blocking, 0, '->blocking is false');
+ is($tfh->blocking(1), 0, '->blocking was true');
+ is($tfh->blocking, 1, '->blocking is true');
my @line = <$fh>;
is_deeply(\@line, [ "foo\n", 'bar' ], 'wantarray works on readline');
}
^ permalink raw reply related [relevance 5%]
* [PATCH 6/6] qspawn: switch to ProcessPipe via popen_rd
2021-01-02 9:13 7% [PATCH 0/6] process pipe improvements Eric Wong
@ 2021-01-02 9:13 6% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2021-01-02 9:13 UTC (permalink / raw)
To: meta
ProcessPipe has a built-in mechanism to prevent siblings from
reaping children.
---
lib/PublicInbox/Qspawn.pm | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/lib/PublicInbox/Qspawn.pm b/lib/PublicInbox/Qspawn.pm
index 68b71112..7e50a59a 100644
--- a/lib/PublicInbox/Qspawn.pm
+++ b/lib/PublicInbox/Qspawn.pm
@@ -28,7 +28,6 @@ package PublicInbox::Qspawn;
use strict;
use PublicInbox::Spawn qw(popen_rd);
use PublicInbox::GzipFilter;
-use PublicInbox::DS qw(dwaitpid); # doesn't need event loop
# n.b.: we get EAGAIN with public-inbox-httpd, and EINTR on other PSGI servers
use Errno qw(EAGAIN EINTR);
@@ -58,9 +57,9 @@ sub _do_spawn {
$self->{cmd} = $o{quiet} ? undef : $cmd;
eval {
# popen_rd may die on EMFILE, ENFILE
- ($self->{rpipe}, $self->{pid}) = popen_rd($cmd, $cmd_env, \%o);
+ $self->{rpipe} = popen_rd($cmd, $cmd_env, \%o);
- die "E: $!" unless defined($self->{pid});
+ die "E: $!" unless defined($self->{rpipe});
$limiter->{running}++;
$start_cb->($self); # EPOLL_CTL_ADD may ENOSPC/ENOMEM
@@ -117,16 +116,14 @@ sub finalize ($$) {
}
}
-# callback for dwaitpid
+# callback for dwaitpid or ProcessPipe
sub waitpid_err { finalize($_[0], child_err($?)) }
sub finish ($;$) {
my ($self, $err) = @_;
- if (delete $self->{rpipe}) {
- dwaitpid $self->{pid}, \&waitpid_err, $self;
- } else {
- finalize($self, $err);
- }
+ my $tied_pp = delete($self->{rpipe}) or return finalize($self, $err);
+ my PublicInbox::ProcessPipe $pp = tied *$tied_pp;
+ @$pp{qw(cb arg)} = (\&waitpid_err, $self); # for ->DESTROY
}
sub start ($$$) {
^ permalink raw reply related [relevance 6%]
* [PATCH 0/6] process pipe improvements
@ 2021-01-02 9:13 7% Eric Wong
2021-01-02 9:13 6% ` [PATCH 6/6] qspawn: switch to ProcessPipe via popen_rd Eric Wong
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2021-01-02 9:13 UTC (permalink / raw)
To: meta
The DS::in_loop clobbering in MboxReader annoyed me and drove
this series.
I wanted to do it for the bidirectional pipes with
"git cat-file --batch", too, but there seems to be some
lifetime management issues which get hard-to-control
with things like the the waitpid(..,WNOHANG) call
in GitAsyncCat. Maybe switching --batch + DS to use UNIX
sockets can be done to save FDs (or I'm too brain-damaged
to figure this out).
But right now, all of our codebase is robust against children
attempting to reap siblings (or PIDs of former siblings)
Eric Wong (6):
processpipe: allow synchronous close to set $?
processpipe: lazy-require PublicInbox::DS for dwaitpid
git: qx: waitpid synchronously via ProcessPipe->CLOSE
import: switch to using ProcessPipe
git: manifest_entry: use ProcessPipe via popen_rd
qspawn: switch to ProcessPipe via popen_rd
lib/PublicInbox/Git.pm | 20 +++++++++-----
lib/PublicInbox/Import.pm | 23 +++++++---------
lib/PublicInbox/MboxReader.pm | 3 ---
lib/PublicInbox/ProcessPipe.pm | 49 +++++++++++++++++++++-------------
lib/PublicInbox/Qspawn.pm | 15 +++++------
lib/PublicInbox/Spawn.pm | 10 +++----
t/mbox_reader.t | 17 ++++++++++++
t/spawn.t | 38 ++++++++++++++++++++++++++
8 files changed, 119 insertions(+), 56 deletions(-)
^ permalink raw reply [relevance 7%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-01-02 9:13 7% [PATCH 0/6] process pipe improvements Eric Wong
2021-01-02 9:13 6% ` [PATCH 6/6] qspawn: switch to ProcessPipe via popen_rd Eric Wong
2021-02-05 0:13 5% [PATCH] httpd/async: set O_NONBLOCK correctly Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).