user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH] dedupe inbox names, coderepo nicks + git dirs
Date: Mon,  4 Mar 2024 21:10:46 +0000	[thread overview]
Message-ID: <20240304211046.872347-1-e@80x24.org> (raw)

Inbox names, coderepo nicks, git_dir values are used heavily
as hash keys by the read-only coderepo WWW pieces.

Relying on CoW for mutable scalars on newer Perl doesn't work
well since CoW for those scalars are limited to 256 CoW references
and blow past that number when mapping thousands of coderepos
and inboxes to each other.  Instead, make the hash key up-front
and get the resulting string to point directly to the pointer
used by the hash key.
---
 This is only one teeny step in reducing memory usage.  It's
 really ridiculous with the coderepo stuff, heavy traffic and
 both jemalloc and glibc seem to struggle.  Oh, and the kernel
 is complaining about tcp_mem sysctl being too low on my 32-bit
 host with a whopping 1G of RAM...

 lib/PublicInbox/Config.pm | 6 ++++--
 lib/PublicInbox/Git.pm    | 3 ++-
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index 607197f6..d6300610 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -379,7 +379,8 @@ sub fill_coderepo {
 		$git->{cgit_url} = $cgits = _array($cgits);
 		$self->{"$pfx.cgiturl"} = $cgits;
 	}
-	$git->{nick} = $nick;
+	my %dedupe = ($nick => undef);
+	($git->{nick}) = keys %dedupe;
 	$git;
 }
 
@@ -486,7 +487,8 @@ sub _fill_ibx {
 	}
 
 	return unless valid_foo_name($name, 'publicinbox');
-	$ibx->{name} = $name;
+	my %dedupe = ($name => undef);
+	($ibx->{name}) = keys %dedupe; # used as a key everywhere
 	$ibx->{-pi_cfg} = $self;
 	$ibx = PublicInbox::Inbox->new($ibx);
 	foreach (@{$ibx->{address}}) {
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index f125b029..af12f141 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -96,7 +96,8 @@ sub new {
 	$git_dir =~ tr!/!/!s;
 	chop $git_dir;
 	# may contain {-tmp} field for File::Temp::Dir
-	bless { git_dir => $git_dir }, $class
+	my %dedupe = ($git_dir => undef);
+	bless { git_dir => (keys %dedupe)[0] }, $class
 }
 
 sub git_path ($$) {

                 reply	other threads:[~2024-03-04 21:10 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240304211046.872347-1-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).