user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 7/9] inbox: keep DB handles if git processes are live
Date: Fri,  1 Oct 2021 09:54:43 +0000	[thread overview]
Message-ID: <20211001095445.9326-8-e@80x24.org> (raw)
In-Reply-To: <20211001095445.9326-1-e@80x24.org>

Having git processes outlive DB handles is likely to hurt
from a fragmentation perspective if the DB handle needs to
be recreated immediately due to a git->cat_async callback.
So only unref DB handles when we're sure there's no live
git users left, otherwise check the inodes.

We'll also avoid needless localization checks in git->cleanup
and make the return value more obvious since the pid fields are
unconditionally deleted nowadays.
---
 lib/PublicInbox/Git.pm   |  4 ++--
 lib/PublicInbox/Inbox.pm | 18 ++++++++++--------
 2 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 97c39aad7468..77783000573f 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -399,16 +399,16 @@ sub async_wait_all ($) {
 # returns true if there are pending "git cat-file" processes
 sub cleanup {
 	my ($self, $lazy) = @_;
-	local $in_cleanup = 1;
 	return 1 if $lazy && (scalar(@{$self->{inflight_c} // []}) ||
 				scalar(@{$self->{inflight} // []}));
+	local $in_cleanup = 1;
 	delete $self->{async_cat};
 	async_wait_all($self);
 	delete $self->{inflight};
 	delete $self->{inflight_c};
 	_destroy($self, qw(cat_rbuf in out pid));
 	_destroy($self, qw(chk_rbuf in_c out_c pid_c err_c));
-	defined($self->{pid}) || defined($self->{pid_c});
+	undef;
 }
 
 # assuming a well-maintained repo, this should be a somewhat
diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm
index 7c1c3afedf2d..724df50a7134 100644
--- a/lib/PublicInbox/Inbox.pm
+++ b/lib/PublicInbox/Inbox.pm
@@ -13,6 +13,10 @@ use Carp qw(croak);
 # returns true if further checking is required
 sub cleanup_shards { $_[0]->{search} ? $_[0]->{search}->cleanup_shards : undef }
 
+sub check_inodes ($) {
+	for (qw(over mm)) { $_[0]->{$_}->check_inodes if $_[0]->{$_} }
+}
+
 sub do_cleanup {
 	my ($ibx) = @_;
 	my $live;
@@ -21,11 +25,16 @@ sub do_cleanup {
 					: $ibx->{git}->cleanup_if_unlinked;
 		delete($ibx->{git}) unless $live;
 	}
+	if ($live) {
+		check_inodes($ibx);
+	} else {
+		delete(@$ibx{qw(over mm description cloneurl
+				-imap_url -nntp_url)});
+	}
 	$ibx->cleanup_shards and $live = 1;
 	for my $git (@{$ibx->{-repo_objs} // []}) {
 		$live = 1 if $git->cleanup(1);
 	}
-	delete(@$ibx{qw(over mm description cloneurl -imap_url -nntp_url)});
 	PublicInbox::DS::add_uniq_timer($ibx+0, 5, \&do_cleanup, $ibx) if $live;
 }
 
@@ -362,13 +371,6 @@ sub unsubscribe_unlock {
 	delete $self->{unlock_subs}->{$ident};
 }
 
-sub check_inodes ($) {
-	my ($self) = @_;
-	for (qw(over mm)) {
-		$self->{$_}->check_inodes if $self->{$_};
-	}
-}
-
 # called by inotify
 sub on_unlock {
 	my ($self) = @_;

  parent reply	other threads:[~2021-10-01  9:54 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-01  9:54 [PATCH 0/9] daemon-related things Eric Wong
2021-10-01  9:54 ` [PATCH 1/9] doc: lei-security: some more updates Eric Wong
2021-10-01  9:54 ` [PATCH 2/9] listener: switch to level-triggered epoll Eric Wong
2021-10-01  9:54 ` [PATCH 3/9] daemon: make SO_ACCEPTFILTER a shared variable Eric Wong
2021-10-01  9:54 ` [PATCH 4/9] ipc: run Net::SSLeay::randomize Eric Wong
2021-10-01  9:54 ` [PATCH 5/9] ds: simplify signalfd use Eric Wong
2021-10-01  9:54 ` [PATCH 6/9] inbox: inline and eliminate git_cleanup Eric Wong
2021-10-01  9:54 ` Eric Wong [this message]
2021-10-01  9:54 ` [PATCH 8/9] ds: inline set_cloexec Eric Wong
2021-10-01  9:54 ` [PATCH 9/9] doc: lei-daemon: new manpage Eric Wong
2021-10-02  0:19   ` Kyle Meyer
2021-10-02  8:08     ` Eric Wong
2021-10-04  3:16       ` Kyle Meyer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211001095445.9326-8-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    --subject='Re: [PATCH 7/9] inbox: keep DB handles if git processes are live' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).