* [PATCH 0/9] daemon-related things
@ 2021-10-01 9:54 4% Eric Wong
2021-10-01 9:54 7% ` [PATCH 7/9] inbox: keep DB handles if git processes are live Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2021-10-01 9:54 UTC (permalink / raw)
To: meta
5/9 has been a long-time coming, a few other small things, too.
The diff stat for lib/ alone shows a nice reduction
Eric Wong (9):
doc: lei-security: some more updates
listener: switch to level-triggered epoll
daemon: make SO_ACCEPTFILTER a shared variable
ipc: run Net::SSLeay::randomize
ds: simplify signalfd use
inbox: inline and eliminate git_cleanup
inbox: keep DB handles if git processes are live
ds: inline set_cloexec
doc: lei-daemon: new manpage
Documentation/lei-daemon.pod | 61 ++++++++++++++++
Documentation/lei-security.pod | 5 +-
Documentation/lei.pod | 9 ++-
Documentation/lei_design_notes.txt | 2 +-
MANIFEST | 1 +
Makefile.PL | 2 +-
lib/PublicInbox/ConfigIter.pm | 2 +-
lib/PublicInbox/DS.pm | 110 ++++++++++++++---------------
lib/PublicInbox/DSKQXS.pm | 10 +--
lib/PublicInbox/Daemon.pm | 20 ++----
lib/PublicInbox/ExtMsg.pm | 2 +-
lib/PublicInbox/ExtSearchIdx.pm | 12 +---
lib/PublicInbox/Gcf2Client.pm | 4 +-
lib/PublicInbox/Git.pm | 4 +-
lib/PublicInbox/IPC.pm | 5 +-
lib/PublicInbox/Inbox.pm | 41 +++++------
lib/PublicInbox/LEI.pm | 17 +----
lib/PublicInbox/Listener.pm | 5 +-
lib/PublicInbox/Qspawn.pm | 2 +-
lib/PublicInbox/Sigfd.pm | 10 +--
lib/PublicInbox/Syscall.pm | 12 ++--
lib/PublicInbox/Watch.pm | 3 +-
script/public-inbox-watch | 9 ---
t/dir_idle.t | 6 +-
t/ds-leak.t | 4 +-
t/httpd-corner.t | 4 +-
t/httpd-https.t | 6 +-
t/httpd.t | 6 +-
t/imapd-tls.t | 7 +-
t/imapd.t | 6 +-
t/nntpd-tls.t | 7 +-
t/nntpd.t | 2 +-
t/sigfd.t | 7 +-
t/watch_maildir.t | 2 +-
xt/mem-imapd-tls.t | 6 +-
xt/net_writer-imap.t | 2 +-
36 files changed, 221 insertions(+), 192 deletions(-)
create mode 100644 Documentation/lei-daemon.pod
^ permalink raw reply [relevance 4%]
* [PATCH 7/9] inbox: keep DB handles if git processes are live
2021-10-01 9:54 4% [PATCH 0/9] daemon-related things Eric Wong
@ 2021-10-01 9:54 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2021-10-01 9:54 UTC (permalink / raw)
To: meta
Having git processes outlive DB handles is likely to hurt
from a fragmentation perspective if the DB handle needs to
be recreated immediately due to a git->cat_async callback.
So only unref DB handles when we're sure there's no live
git users left, otherwise check the inodes.
We'll also avoid needless localization checks in git->cleanup
and make the return value more obvious since the pid fields are
unconditionally deleted nowadays.
---
lib/PublicInbox/Git.pm | 4 ++--
lib/PublicInbox/Inbox.pm | 18 ++++++++++--------
2 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 97c39aad7468..77783000573f 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -399,16 +399,16 @@ sub async_wait_all ($) {
# returns true if there are pending "git cat-file" processes
sub cleanup {
my ($self, $lazy) = @_;
- local $in_cleanup = 1;
return 1 if $lazy && (scalar(@{$self->{inflight_c} // []}) ||
scalar(@{$self->{inflight} // []}));
+ local $in_cleanup = 1;
delete $self->{async_cat};
async_wait_all($self);
delete $self->{inflight};
delete $self->{inflight_c};
_destroy($self, qw(cat_rbuf in out pid));
_destroy($self, qw(chk_rbuf in_c out_c pid_c err_c));
- defined($self->{pid}) || defined($self->{pid_c});
+ undef;
}
# assuming a well-maintained repo, this should be a somewhat
diff --git a/lib/PublicInbox/Inbox.pm b/lib/PublicInbox/Inbox.pm
index 7c1c3afedf2d..724df50a7134 100644
--- a/lib/PublicInbox/Inbox.pm
+++ b/lib/PublicInbox/Inbox.pm
@@ -13,6 +13,10 @@ use Carp qw(croak);
# returns true if further checking is required
sub cleanup_shards { $_[0]->{search} ? $_[0]->{search}->cleanup_shards : undef }
+sub check_inodes ($) {
+ for (qw(over mm)) { $_[0]->{$_}->check_inodes if $_[0]->{$_} }
+}
+
sub do_cleanup {
my ($ibx) = @_;
my $live;
@@ -21,11 +25,16 @@ sub do_cleanup {
: $ibx->{git}->cleanup_if_unlinked;
delete($ibx->{git}) unless $live;
}
+ if ($live) {
+ check_inodes($ibx);
+ } else {
+ delete(@$ibx{qw(over mm description cloneurl
+ -imap_url -nntp_url)});
+ }
$ibx->cleanup_shards and $live = 1;
for my $git (@{$ibx->{-repo_objs} // []}) {
$live = 1 if $git->cleanup(1);
}
- delete(@$ibx{qw(over mm description cloneurl -imap_url -nntp_url)});
PublicInbox::DS::add_uniq_timer($ibx+0, 5, \&do_cleanup, $ibx) if $live;
}
@@ -362,13 +371,6 @@ sub unsubscribe_unlock {
delete $self->{unlock_subs}->{$ident};
}
-sub check_inodes ($) {
- my ($self) = @_;
- for (qw(over mm)) {
- $self->{$_}->check_inodes if $self->{$_};
- }
-}
-
# called by inotify
sub on_unlock {
my ($self) = @_;
^ permalink raw reply related [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-10-01 9:54 4% [PATCH 0/9] daemon-related things Eric Wong
2021-10-01 9:54 7% ` [PATCH 7/9] inbox: keep DB handles if git processes are live Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).