user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 02/14] ds: replace FD map hash table with array
Date: Thu,  2 Nov 2023 09:35:27 +0000	[thread overview]
Message-ID: <20231102093539.2067470-3-e@80x24.org> (raw)
In-Reply-To: <20231102093539.2067470-1-e@80x24.org>

FDs are array indices into the kernel, anyways, so we can
take advantage of space savings and speedups because the
majority of FDs a big process has is going to end up in
the array, anyways.
---
 lib/PublicInbox/DS.pm          | 18 +++++++++---------
 lib/PublicInbox/LeiStoreErr.pm |  2 +-
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index ca1f0f81..8331da95 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -42,7 +42,7 @@ my $reap_armed;
 my @active; # FDs (or objs) returned by epoll/kqueue
 our (%AWAIT_PIDS, # pid => [ $callback, @args ]
 	$cur_runq, # only set inside next_tick
-     %DescriptorMap,             # fd (num) -> PublicInbox::DS object
+     @FD_MAP, # fd (num) -> PublicInbox::DS object
      $Poller, # global Select, Epoll, DSPoll, or DSKQXS ref
 
      @post_loop_do,              # subref + args to call at the end of each loop
@@ -74,7 +74,7 @@ sub Reset {
 		# clobbering $Poller may call DSKQXS::DESTROY,
 		# we must always have this set to something to avoid
 		# needing branches before ep_del/ep_mod calls (via ->close).
-		%DescriptorMap = (); # likely to call ep_del
+		@FD_MAP = ();
 		@Timers = ();
 		%UniqTimer = ();
 		@post_loop_do = ();
@@ -85,7 +85,7 @@ sub Reset {
 		$nextq = undef; # may call ep_del
 		%AWAIT_PIDS = ();
 	} while (@Timers || $nextq || keys(%AWAIT_PIDS) ||
-		@active || keys(%DescriptorMap) ||
+		@active || @FD_MAP ||
 		@post_loop_do || keys(%UniqTimer) ||
 		scalar(@{$cur_runq // []})); # do not vivify cur_runq
 
@@ -226,7 +226,7 @@ sub in_loop () { $in_loop }
 # use inside @post_loop_do, returns number of busy clients
 sub close_non_busy () {
 	my $n = 0;
-	for my $s (values %DescriptorMap) {
+	for my $s (grep defined, @FD_MAP) {
 		# close as much as possible, early as possible
 		($s->busy ? ++$n : $s->close) if $s->can('busy');
 	}
@@ -291,7 +291,7 @@ sub event_loop (;$$) {
 		$Poller->ep_wait($timeout, \@active);
 
 		# map all FDs to their associated Perl object
-		@active = @DescriptorMap{@active};
+		@active = @FD_MAP[@active];
 
 		while (my $obj = shift @active) {
 			$obj->event_step;
@@ -329,10 +329,10 @@ retry:
         }
         die "EPOLL_CTL_ADD $self/$sock/$fd: $!";
     }
-    croak("FD:$fd in use by $DescriptorMap{$fd} (for $self/$sock)")
-        if defined($DescriptorMap{$fd});
+    defined($FD_MAP[$fd]) and
+		croak("BUG: FD:$fd in use by $FD_MAP[$fd] (for $self/$sock)");
 
-    $DescriptorMap{$fd} = $self;
+    $FD_MAP[$fd] = $self;
 }
 
 # for IMAP, NNTP, and POP3 which greet clients upon connect
@@ -366,7 +366,7 @@ sub close {
 	# be self-referential closures (sub { $client->close })
 	# preventing the object from being destroyed
 	delete $self->{wbuf};
-	delete $DescriptorMap{fileno($sock)};
+	$FD_MAP[fileno($sock)] = undef;
 
 	!$Poller->ep_del($sock); # stop getting notifications
 }
diff --git a/lib/PublicInbox/LeiStoreErr.pm b/lib/PublicInbox/LeiStoreErr.pm
index fe4af51e..c8bc72b6 100644
--- a/lib/PublicInbox/LeiStoreErr.pm
+++ b/lib/PublicInbox/LeiStoreErr.pm
@@ -47,7 +47,7 @@ sub event_step {
 	return ($!{EAGAIN} ? 0 : $self->close) if !defined($n);
 	return $self->close if !$n;
 	my $printed;
-	for my $lei (values %PublicInbox::DS::DescriptorMap) {
+	for my $lei (grep defined, @PublicInbox::DS::FD_MAP) {
 		my $cb = $lei->can('store_path') // next;
 		next if $cb->($lei) ne $self->{store_path};
 		emit($lei->{2} // next, $buf) and $printed = 1;

  parent reply	other threads:[~2023-11-02  9:35 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-02  9:35 [PATCH 00/14] IO/IPC-related cleanups Eric Wong
2023-11-02  9:35 ` [PATCH 01/14] xap_helper.pm: use do_fork to Reset and reseed Eric Wong
2023-11-02  9:35 ` Eric Wong [this message]
2023-11-02  9:35 ` [PATCH 03/14] treewide: use ->close method rather than CORE::close Eric Wong
2023-11-02 21:35   ` [PATCH 15/14] ds: don't try ->close after ->accept_SSL failure Eric Wong
2023-11-02  9:35 ` [PATCH 04/14] cindex: drop redundant close on regular FH Eric Wong
2023-11-02  9:35 ` [PATCH 05/14] treewide: use ->close to call ProcessIO->CLOSE Eric Wong
2023-11-02  9:35 ` [PATCH 06/14] multi_git: use autodie Eric Wong
2023-11-02  9:35 ` [PATCH 07/14] git_credential: use autodie where appropriate Eric Wong
2023-11-02  9:35 ` [PATCH 08/14] replace ProcessIO with untied PublicInbox::IO Eric Wong
2023-11-02  9:35 ` [PATCH 09/14] io: introduce write_file helper sub Eric Wong
2023-11-02  9:35 ` [PATCH 10/14] spawn: support PerlIO layer in scalar redirects Eric Wong
2023-11-02  9:35 ` [PATCH 11/14] treewide: check alternates writes with eof + autodie Eric Wong
2023-11-02  9:35 ` [PATCH 12/14] treewide: use eof and close to detect readline errors Eric Wong
2023-11-02  9:35 ` [PATCH 13/14] move read_all, try_cat, and poll_in to PublicInbox::IO Eric Wong
2023-11-02 20:59   ` www: squash read_all usage fix Eric Wong
2023-11-02  9:35 ` [PATCH 14/14] t/cindex+extsearch: use write_file, autodie, etc Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231102093539.2067470-3-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).