user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 5/9] lei: clean up pkt_op consumer on exception, too
Date: Wed, 24 Mar 2021 14:23:31 +0500	[thread overview]
Message-ID: <20210324092335.12345-6-e@80x24.org> (raw)
In-Reply-To: <20210324092335.12345-1-e@80x24.org>

We need to consistently ensure pkt_op_c doesn't lead to a
long-lived circular reference if an exception is thrown in
pre_augment.  Maybe the API could be better, but this fixes an
FD leak when attempting to --augment a FIFO.

Followup-to: b9524082ba39e665 ("lei_xsearch: cleanup {pkt_op_p} on exceptions")
---
 lib/PublicInbox/LEI.pm        | 22 ++++++++++++++++++++--
 lib/PublicInbox/LeiXSearch.pm |  9 ++-------
 2 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/lib/PublicInbox/LEI.pm b/lib/PublicInbox/LEI.pm
index 74372532..878685f1 100644
--- a/lib/PublicInbox/LEI.pm
+++ b/lib/PublicInbox/LEI.pm
@@ -482,6 +482,24 @@ sub _lei_atfork_child {
 	$current_lei = $persist ? undef : $self; # for SIG{__WARN__}
 }
 
+sub _delete_pkt_op { # OnDestroy callback to prevent leaks on die
+	my ($self) = @_;
+	if (my $op = delete $self->{pkt_op_c}) { # in case of die
+		$op->close; # PublicInbox::PktOp::close
+	}
+	my $unclosed_after_die = delete($self->{pkt_op_p}) or return;
+	close $unclosed_after_die;
+}
+
+sub pkt_op_pair {
+	my ($self, $ops) = @_;
+	require PublicInbox::OnDestroy;
+	require PublicInbox::PktOp;
+	my $end = PublicInbox::OnDestroy->new($$, \&_delete_pkt_op, $self);
+	@$self{qw(pkt_op_c pkt_op_p)} = PublicInbox::PktOp->pair($ops);
+	$end;
+}
+
 sub workers_start {
 	my ($lei, $wq, $ident, $jobs, $ops) = @_;
 	$ops = {
@@ -492,11 +510,11 @@ sub workers_start {
 		($ops ? %$ops : ()),
 	};
 	$ops->{''} //= [ \&dclose, $lei ];
-	require PublicInbox::PktOp;
-	($lei->{pkt_op_c}, $lei->{pkt_op_p}) = PublicInbox::PktOp->pair($ops);
+	my $end = $lei->pkt_op_pair($ops);
 	$wq->wq_workers_start($ident, $jobs, $lei->oldset, { lei => $lei });
 	delete $lei->{pkt_op_p};
 	my $op = delete $lei->{pkt_op_c};
+	@$end = ();
 	$lei->event_step_init;
 	# oneshot needs $op, daemon-mode uses DS->EventLoop to handle $op
 	$lei->{oneshot} ? $op : undef;
diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index c6b82eeb..58b6cfc0 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -406,11 +406,6 @@ sub ipc_atfork_child {
 	$self->SUPER::ipc_atfork_child;
 }
 
-sub delete_pkt_op { # OnDestroy callback
-	my $unclosed_after_die = delete($_[0])->{pkt_op_p} or return;
-	close $unclosed_after_die;
-}
-
 sub do_query {
 	my ($self, $lei) = @_;
 	my $l2m = $lei->{l2m};
@@ -426,8 +421,7 @@ sub do_query {
 		'incr_start_query' => [ \&incr_start_query, $self, $l2m ],
 	};
 	$lei->{auth}->op_merge($ops, $l2m) if $l2m && $lei->{auth};
-	my $od = PublicInbox::OnDestroy->new($$, \&delete_pkt_op, $lei);
-	($lei->{pkt_op_c}, $lei->{pkt_op_p}) = PublicInbox::PktOp->pair($ops);
+	my $end = $lei->pkt_op_pair($ops);
 	$lei->{1}->autoflush(1);
 	$lei->start_pager if delete $lei->{need_pager};
 	$lei->{ovv}->ovv_begin($lei);
@@ -446,6 +440,7 @@ sub do_query {
 				$lei->oldset, { lei => $lei });
 	my $op = delete $lei->{pkt_op_c};
 	delete $lei->{pkt_op_p};
+	@$end = ();
 	$self->{threads} = $lei->{opt}->{threads};
 	if ($l2m) {
 		$l2m->net_merge_complete unless $lei->{auth};

  parent reply	other threads:[~2021-03-24  9:23 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-24  9:23 [PATCH 0/9] lei: various corner case leak fixes Eric Wong
2021-03-24  9:23 ` [PATCH 1/9] ds: improve DS->Reset fork-safety Eric Wong
2021-03-24 23:01   ` [SQUASH] " Eric Wong
2021-03-24  9:23 ` [PATCH 2/9] mbox_lock: dotlock: chdir for relative lock paths Eric Wong
2021-03-24  9:23 ` [PATCH 3/9] lei: drop circular reference in lei_store process Eric Wong
2021-03-24  9:23 ` [PATCH 4/9] lei: update {3} after -C chdirs Eric Wong
2021-03-24  9:23 ` Eric Wong [this message]
2021-03-24  9:23 ` [PATCH 6/9] lei_store: give process a better name Eric Wong
2021-03-24  9:23 ` [PATCH 7/9] v2writable: cleanup SQLite handles on --xapian-only Eric Wong
2021-03-24  9:23 ` [PATCH 8/9] lei_mirror: fix circular reference Eric Wong
2021-03-24  9:23 ` [PATCH 9/9] lei-daemon: do not leak FDs on bogus requests Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210324092335.12345-6-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).