From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id CE7951F44D for ; Mon, 4 Mar 2024 21:10:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1709586646; bh=3flbLsOkbHbop16Xf6HfsgQ1EA9fhOKyMsjV5OK9KtY=; h=From:To:Subject:Date:From; b=lnab0uFg8wKt2KZTIAWJ0kmXMaOzfkYOZs4OsSw7npue4QMghfjfzswbjvr2PsVTo 0sr+zTO6QYOW/EcgZKW+P82b3f297jV+eQMgxDm/i2OgSZWArny9Fxq9EmPH3Sparj qx12+iIAzqvNAay7PsuYeBk3WsMopcVAAzYlM7ZU= From: Eric Wong To: meta@public-inbox.org Subject: [PATCH] dedupe inbox names, coderepo nicks + git dirs Date: Mon, 4 Mar 2024 21:10:46 +0000 Message-Id: <20240304211046.872347-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Inbox names, coderepo nicks, git_dir values are used heavily as hash keys by the read-only coderepo WWW pieces. Relying on CoW for mutable scalars on newer Perl doesn't work well since CoW for those scalars are limited to 256 CoW references and blow past that number when mapping thousands of coderepos and inboxes to each other. Instead, make the hash key up-front and get the resulting string to point directly to the pointer used by the hash key. --- This is only one teeny step in reducing memory usage. It's really ridiculous with the coderepo stuff, heavy traffic and both jemalloc and glibc seem to struggle. Oh, and the kernel is complaining about tcp_mem sysctl being too low on my 32-bit host with a whopping 1G of RAM... lib/PublicInbox/Config.pm | 6 ++++-- lib/PublicInbox/Git.pm | 3 ++- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm index 607197f6..d6300610 100644 --- a/lib/PublicInbox/Config.pm +++ b/lib/PublicInbox/Config.pm @@ -379,7 +379,8 @@ sub fill_coderepo { $git->{cgit_url} = $cgits = _array($cgits); $self->{"$pfx.cgiturl"} = $cgits; } - $git->{nick} = $nick; + my %dedupe = ($nick => undef); + ($git->{nick}) = keys %dedupe; $git; } @@ -486,7 +487,8 @@ sub _fill_ibx { } return unless valid_foo_name($name, 'publicinbox'); - $ibx->{name} = $name; + my %dedupe = ($name => undef); + ($ibx->{name}) = keys %dedupe; # used as a key everywhere $ibx->{-pi_cfg} = $self; $ibx = PublicInbox::Inbox->new($ibx); foreach (@{$ibx->{address}}) { diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm index f125b029..af12f141 100644 --- a/lib/PublicInbox/Git.pm +++ b/lib/PublicInbox/Git.pm @@ -96,7 +96,8 @@ sub new { $git_dir =~ tr!/!/!s; chop $git_dir; # may contain {-tmp} field for File::Temp::Dir - bless { git_dir => $git_dir }, $class + my %dedupe = ($git_dir => undef); + bless { git_dir => (keys %dedupe)[0] }, $class } sub git_path ($$) {