* [PATCH 4/4] codesearch: deduplicate $git->{nick} field
@ 2024-03-11 19:40 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2024-03-11 19:40 UTC (permalink / raw)
To: meta
While PublicInbox::Config is responsible for some instances of
setting $git->{nick}, more PublicInbox::Git objects may be
created from loading the cindex and we should do our best to
reuse that memory, too.
Followup-to: 84ed7ec1c887 (dedupe inbox names, coderepo nicks + git dirs, 2024-03-04)
---
lib/PublicInbox/CodeSearch.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/PublicInbox/CodeSearch.pm b/lib/PublicInbox/CodeSearch.pm
index 48033bb5..e5fa4480 100644
--- a/lib/PublicInbox/CodeSearch.pm
+++ b/lib/PublicInbox/CodeSearch.pm
@@ -283,7 +283,8 @@ EOM
$nick =~ s!$lre!$nick_pfx!s or next;
$dir2cr{$p} = $coderepos->{$nick} //= do {
my $git = PublicInbox::Git->new($p);
- $git->{nick} = $nick; # for git->pub_urls
+ my %dedupe = ($nick => undef);
+ ($git->{nick}) = keys %dedupe; # for git->pub_urls
$git;
};
}
^ permalink raw reply related [relevance 6%]
* [PATCH] dedupe inbox names, coderepo nicks + git dirs
@ 2024-03-04 21:10 7% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2024-03-04 21:10 UTC (permalink / raw)
To: meta
Inbox names, coderepo nicks, git_dir values are used heavily
as hash keys by the read-only coderepo WWW pieces.
Relying on CoW for mutable scalars on newer Perl doesn't work
well since CoW for those scalars are limited to 256 CoW references
and blow past that number when mapping thousands of coderepos
and inboxes to each other. Instead, make the hash key up-front
and get the resulting string to point directly to the pointer
used by the hash key.
---
This is only one teeny step in reducing memory usage. It's
really ridiculous with the coderepo stuff, heavy traffic and
both jemalloc and glibc seem to struggle. Oh, and the kernel
is complaining about tcp_mem sysctl being too low on my 32-bit
host with a whopping 1G of RAM...
lib/PublicInbox/Config.pm | 6 ++++--
lib/PublicInbox/Git.pm | 3 ++-
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/lib/PublicInbox/Config.pm b/lib/PublicInbox/Config.pm
index 607197f6..d6300610 100644
--- a/lib/PublicInbox/Config.pm
+++ b/lib/PublicInbox/Config.pm
@@ -379,7 +379,8 @@ sub fill_coderepo {
$git->{cgit_url} = $cgits = _array($cgits);
$self->{"$pfx.cgiturl"} = $cgits;
}
- $git->{nick} = $nick;
+ my %dedupe = ($nick => undef);
+ ($git->{nick}) = keys %dedupe;
$git;
}
@@ -486,7 +487,8 @@ sub _fill_ibx {
}
return unless valid_foo_name($name, 'publicinbox');
- $ibx->{name} = $name;
+ my %dedupe = ($name => undef);
+ ($ibx->{name}) = keys %dedupe; # used as a key everywhere
$ibx->{-pi_cfg} = $self;
$ibx = PublicInbox::Inbox->new($ibx);
foreach (@{$ibx->{address}}) {
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index f125b029..af12f141 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -96,7 +96,8 @@ sub new {
$git_dir =~ tr!/!/!s;
chop $git_dir;
# may contain {-tmp} field for File::Temp::Dir
- bless { git_dir => $git_dir }, $class
+ my %dedupe = ($git_dir => undef);
+ bless { git_dir => (keys %dedupe)[0] }, $class
}
sub git_path ($$) {
^ permalink raw reply related [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2024-03-04 21:10 7% [PATCH] dedupe inbox names, coderepo nicks + git dirs Eric Wong
2024-03-11 19:40 [PATCH 0/4] memory reductions for WWW + solver Eric Wong
2024-03-11 19:40 6% ` [PATCH 4/4] codesearch: deduplicate $git->{nick} field Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).