diff options
author | Eric Wong <e@80x24.org> | 2017-02-10 03:30:40 +0000 |
---|---|---|
committer | Eric Wong <e@80x24.org> | 2017-02-10 03:32:30 +0000 |
commit | 7c3d8e174c15193851f811f22694ddb47cb11c65 (patch) | |
tree | 06f045ecb9feb9a09c80059903acd80f168ad59d /lib/PublicInbox | |
parent | 52f9edfb756676b471deac69e5d55df1933aa528 (diff) | |
download | public-inbox-7c3d8e174c15193851f811f22694ddb47cb11c65.tar.gz |
For certain repos, having too many active refs will cause memory usage problems. Mitigate the Xapian problems, for now, and consider a switch to GDBM_File or similar for repos with more refs.
Diffstat (limited to 'lib/PublicInbox')
-rw-r--r-- | lib/PublicInbox/RepoGitSearchIdx.pm | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/lib/PublicInbox/RepoGitSearchIdx.pm b/lib/PublicInbox/RepoGitSearchIdx.pm index 67d7ec3f..d2b4597e 100644 --- a/lib/PublicInbox/RepoGitSearchIdx.pm +++ b/lib/PublicInbox/RepoGitSearchIdx.pm @@ -345,10 +345,16 @@ sub index_top_ref ($$$) { print $progress "$refname => $tip\n" if $progress; replace_or_add($db, $doc_id, $ref_doc); } + $db->flush; # update all decorated refs which got snowballed into this one delete $active->{$refname}; + my $n = 100; foreach my $ref (keys %$active) { + if (--$n <= 0) { + $db->flush; + $n = 100; + } $ref_doc = get_doc($self, \$doc_id, 'ref', $ref); $ref_doc->set_data($active->{$ref}); if ($progress) { |