user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH 0/9] daemon-related things
@ 2021-10-01  9:54  4% Eric Wong
  2021-10-01  9:54  7% ` [PATCH 7/9] inbox: keep DB handles if git processes are live Eric Wong
  0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2021-10-01  9:54 UTC (permalink / raw)
  To: meta

5/9 has been a long-time coming, a few other small things, too.
The diff stat for lib/ alone shows a nice reduction

Eric Wong (9):
  doc: lei-security: some more updates
  listener: switch to level-triggered epoll
  daemon: make SO_ACCEPTFILTER a shared variable
  ipc: run Net::SSLeay::randomize
  ds: simplify signalfd use
  inbox: inline and eliminate git_cleanup
  inbox: keep DB handles if git processes are live
  ds: inline set_cloexec
  doc: lei-daemon: new manpage

 Documentation/lei-daemon.pod       |  61 ++++++++++++++++
 Documentation/lei-security.pod     |   5 +-
 Documentation/lei.pod              |   9 ++-
 Documentation/lei_design_notes.txt |   2 +-
 MANIFEST                           |   1 +
 Makefile.PL                        |   2 +-
 lib/PublicInbox/ConfigIter.pm      |   2 +-
 lib/PublicInbox/DS.pm              | 110 ++++++++++++++---------------
 lib/PublicInbox/DSKQXS.pm          |  10 +--
 lib/PublicInbox/Daemon.pm          |  20 ++----
 lib/PublicInbox/ExtMsg.pm          |   2 +-
 lib/PublicInbox/ExtSearchIdx.pm    |  12 +---
 lib/PublicInbox/Gcf2Client.pm      |   4 +-
 lib/PublicInbox/Git.pm             |   4 +-
 lib/PublicInbox/IPC.pm             |   5 +-
 lib/PublicInbox/Inbox.pm           |  41 +++++------
 lib/PublicInbox/LEI.pm             |  17 +----
 lib/PublicInbox/Listener.pm        |   5 +-
 lib/PublicInbox/Qspawn.pm          |   2 +-
 lib/PublicInbox/Sigfd.pm           |  10 +--
 lib/PublicInbox/Syscall.pm         |  12 ++--
 lib/PublicInbox/Watch.pm           |   3 +-
 script/public-inbox-watch          |   9 ---
 t/dir_idle.t                       |   6 +-
 t/ds-leak.t                        |   4 +-
 t/httpd-corner.t                   |   4 +-
 t/httpd-https.t                    |   6 +-
 t/httpd.t                          |   6 +-
 t/imapd-tls.t                      |   7 +-
 t/imapd.t                          |   6 +-
 t/nntpd-tls.t                      |   7 +-
 t/nntpd.t                          |   2 +-
 t/sigfd.t                          |   7 +-
 t/watch_maildir.t                  |   2 +-
 xt/mem-imapd-tls.t                 |   6 +-
 xt/net_writer-imap.t               |   2 +-
 36 files changed, 221 insertions(+), 192 deletions(-)
 create mode 100644 Documentation/lei-daemon.pod

^ permalink raw reply	[relevance 4%]

* [PATCH 7/9] inbox: keep DB handles if git processes are live
  2021-10-01  9:54  4% [PATCH 0/9] daemon-related things Eric Wong
@ 2021-10-01  9:54  7% ` Eric Wong
  0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2021-10-01  9:54 UTC (permalink / raw)
  To: meta

Having git processes outlive DB handles is likely to hurt
from a fragmentation perspective if the DB handle needs to
be recreated immediately due to a git->cat_async callback.
So only unref DB handles when we're sure there's no live
git users left, otherwise check the inodes.

We'll also avoid needless localization checks in git->cleanup
and make the return value more obvious since the pid fields are
unconditionally deleted nowadays.
---
 lib/PublicInbox/Git.pm   |  4 ++--
 lib/PublicInbox/Inbox.pm | 18 ++++++++++--------
 2 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 97c39aad7468..77783000573f 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -399,16 +399,16 @@ sub async_wait_all ($) {
 # returns true if there are pending "git cat-file" processes
 sub cleanup {
 	my ($self, $lazy) = @_;
-	local $in_cleanup = 1;
 	return 1 if $lazy && (scalar(@{$self->{inflight_c} // []}) ||
 				scalar(@{$self->{inflight} // []}));
+	local $in_cleanup = 1;
 	delete $self->{async_cat};
 	async_wait_all($self);
 	delete $self->{inflight};
 	delete $self->{inflight_c};
 	_destroy($self, qw(cat_rbuf in out pid));
 	_destroy($self, qw(chk_rbuf in_c out_c pid_c err_c));
-	defined($self->{pid}) || defined($self->{pid_c});
+	undef;
 }
 
 # assuming a well-maintained repo, this should be a somewhat
diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm
index 7c1c3afedf2d..724df50a7134 100644
--- a/lib/PublicInbox/Inbox.pm
+++ b/lib/PublicInbox/Inbox.pm
@@ -13,6 +13,10 @@ use Carp qw(croak);
 # returns true if further checking is required
 sub cleanup_shards { $_[0]->{search} ? $_[0]->{search}->cleanup_shards : undef }
 
+sub check_inodes ($) {
+	for (qw(over mm)) { $_[0]->{$_}->check_inodes if $_[0]->{$_} }
+}
+
 sub do_cleanup {
 	my ($ibx) = @_;
 	my $live;
@@ -21,11 +25,16 @@ sub do_cleanup {
 					: $ibx->{git}->cleanup_if_unlinked;
 		delete($ibx->{git}) unless $live;
 	}
+	if ($live) {
+		check_inodes($ibx);
+	} else {
+		delete(@$ibx{qw(over mm description cloneurl
+				-imap_url -nntp_url)});
+	}
 	$ibx->cleanup_shards and $live = 1;
 	for my $git (@{$ibx->{-repo_objs} // []}) {
 		$live = 1 if $git->cleanup(1);
 	}
-	delete(@$ibx{qw(over mm description cloneurl -imap_url -nntp_url)});
 	PublicInbox::DS::add_uniq_timer($ibx+0, 5, \&do_cleanup, $ibx) if $live;
 }
 
@@ -362,13 +371,6 @@ sub unsubscribe_unlock {
 	delete $self->{unlock_subs}->{$ident};
 }
 
-sub check_inodes ($) {
-	my ($self) = @_;
-	for (qw(over mm)) {
-		$self->{$_}->check_inodes if $self->{$_};
-	}
-}
-
 # called by inotify
 sub on_unlock {
 	my ($self) = @_;

^ permalink raw reply related	[relevance 7%]

Results 1-2 of 2 | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2021-10-01  9:54  4% [PATCH 0/9] daemon-related things Eric Wong
2021-10-01  9:54  7% ` [PATCH 7/9] inbox: keep DB handles if git processes are live Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).