* [PATCH 10/11] ds: flatten $EXPMAP, delete entries on close
2020-01-12 21:17 7% [PATCH 00/11] ds: various cleanups and fixes Eric Wong
@ 2020-01-12 21:17 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-01-12 21:17 UTC (permalink / raw)
To: meta
We can reduce the amount of small arrayrefs in memory
by flattening $EXPMAP. This forces us to properly clean
up references during deferred close handling, so NNTP
(and soon HTTP) connections no longer linger until expiry.
---
lib/PublicInbox/DS.pm | 57 ++++++++++++++++++++++---------------------
1 file changed, 29 insertions(+), 28 deletions(-)
diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index fc3c82cc..c434464b 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -41,7 +41,7 @@ use Carp qw(confess carp);
my $nextq; # queue for next_tick
my $wait_pids; # list of [ pid, callback, callback_arg ]
my $later_queue; # list of callbacks to run at some later interval
-my $EXPMAP; # fd -> [ idle_time, $self ]
+my $EXPMAP; # fd -> idle_time
our $EXPTIME = 180; # 3 minutes
my ($later_timer, $reap_timer, $exp_timer);
my $ToClose; # sockets to close when event loop is done
@@ -259,9 +259,13 @@ sub PostEventLoop () {
# loop)
if (my $close_now = $ToClose) {
$ToClose = undef; # will be autovivified on push
+ @$close_now = map { fileno($_) } @$close_now;
+
+ # order matters, destroy expiry times, first:
+ delete @$EXPMAP{@$close_now};
+
# ->DESTROY methods may populate ToClose
- delete($DescriptorMap{fileno($_)}) for @$close_now;
- # let refcounting drop everything in $close_now at once
+ delete @DescriptorMap{@$close_now};
}
# by default we keep running, unless a postloop callback cancels it
@@ -642,37 +646,34 @@ sub later ($) {
}
sub expire_old () {
- my $now = now();
- my $exp = $EXPTIME;
- my $old = $now - $exp;
- my %new;
- while (my ($fd, $v) = each %$EXPMAP) {
- my ($idle_time, $ds_obj) = @$v;
- if ($idle_time < $old) {
- if (!$ds_obj->shutdn) {
- $new{$fd} = $v;
- }
- } else {
- $new{$fd} = $v;
- }
- }
- $EXPMAP = \%new;
- $exp_timer = scalar(keys %new) ? later(\&expire_old) : undef;
+ my $now = now();
+ my $exp = $EXPTIME;
+ my $old = $now - $exp;
+ my %new;
+ while (my ($fd, $idle_at) = each %$EXPMAP) {
+ if ($idle_at < $old) {
+ my $ds_obj = $DescriptorMap{$fd};
+ $new{$fd} = $idle_at if !$ds_obj->shutdn;
+ } else {
+ $new{$fd} = $idle_at;
+ }
+ }
+ $EXPMAP = \%new;
+ $exp_timer = scalar(keys %new) ? later(\&expire_old) : undef;
}
sub update_idle_time {
- my ($self) = @_;
- my $sock = $self->{sock} or return;
- $EXPMAP->{fileno($sock)} = [ now(), $self ];
- $exp_timer //= later(\&expire_old);
+ my ($self) = @_;
+ my $sock = $self->{sock} or return;
+ $EXPMAP->{fileno($sock)} = now();
+ $exp_timer //= later(\&expire_old);
}
sub not_idle_long {
- my ($self, $now) = @_;
- my $sock = $self->{sock} or return;
- my $ary = $EXPMAP->{fileno($sock)} or return;
- my $exp_at = $ary->[0] + $EXPTIME;
- $exp_at > $now;
+ my ($self, $now) = @_;
+ my $sock = $self->{sock} or return;
+ my $idle_at = $EXPMAP->{fileno($sock)} or return;
+ ($idle_at + $EXPTIME) > $now;
}
1;
^ permalink raw reply related [relevance 7%]
* [PATCH 00/11] ds: various cleanups and fixes
@ 2020-01-12 21:17 7% Eric Wong
2020-01-12 21:17 7% ` [PATCH 10/11] ds: flatten $EXPMAP, delete entries on close Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-01-12 21:17 UTC (permalink / raw)
To: meta
Start taking advantage of autovivification to simplify our code
and reduce allocations. Also, I believe it's my first time
using the return value of "push".
Eric Wong (11):
ds: new: avoid redundant check, make clobbering fatal
ds: add_timer: rename from AddTimer, remove a parameter
ds: add an in_loop() function for Inbox.pm use
ds: remove Timer->cancel and Timer class+bless
ds: PostEventLoop: guard ToClose against DESTROY side-effects
ds|http|nntp: simplify {wbuf} population
ds: rely on autovivification for nextq
ds: rely on autovivication for waitpid bits
ds: rely on autovivification for $later_queue
ds: flatten $EXPMAP, delete entries on close
sigfd: simplify loop and improve documentation
lib/PublicInbox/DS.pm | 200 +++++++++++++++++----------------------
lib/PublicInbox/HTTP.pm | 6 +-
lib/PublicInbox/Inbox.pm | 2 +-
lib/PublicInbox/NNTP.pm | 11 +--
lib/PublicInbox/Sigfd.pm | 13 +--
t/ds-leak.t | 2 +-
6 files changed, 105 insertions(+), 129 deletions(-)
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-01-12 21:17 7% [PATCH 00/11] ds: various cleanups and fixes Eric Wong
2020-01-12 21:17 7% ` [PATCH 10/11] ds: flatten $EXPMAP, delete entries on close Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).