From: Eric Wong <e@yhbt.net>
To: meta@public-inbox.org
Subject: [PATCH 3/5] overidx: document the SQLite PRAGMA we use
Date: Sun, 10 May 2020 22:37:13 +0000 [thread overview]
Message-ID: <20200510223715.19254-4-e@yhbt.net> (raw)
In-Reply-To: <20200510223715.19254-1-e@yhbt.net>
This ought to prevent cargo-culting the cache_size PRAGMA
into smaller SQLite DBs we might use.
---
lib/PublicInbox/OverIdx.pm | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/lib/PublicInbox/OverIdx.pm b/lib/PublicInbox/OverIdx.pm
index acbf2c8de60..cb15baadf2b 100644
--- a/lib/PublicInbox/OverIdx.pm
+++ b/lib/PublicInbox/OverIdx.pm
@@ -21,8 +21,16 @@ use PublicInbox::Search;
sub dbh_new {
my ($self) = @_;
my $dbh = $self->SUPER::dbh_new(1);
+
+ # TRUNCATE reduces I/O compared to the default (DELETE)
$dbh->do('PRAGMA journal_mode = TRUNCATE');
+
+ # 80000 pages (80MiB on SQLite <3.12.0, 320MiB on 3.12.0+)
+ # was found to be good in 2018 during the large LKML import
+ # at the time. This ought to be configurable based on HW
+ # and inbox size; I suspect it's overkill for many inboxes.
$dbh->do('PRAGMA cache_size = 80000');
+
create_tables($dbh);
$dbh;
}
next prev parent reply other threads:[~2020-05-10 22:37 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-10 22:37 [PATCH 0/5] scattered dev/CLI-oriented changes Eric Wong
2020-05-10 22:37 ` [PATCH 1/5] xt/eml_check_limits: check limits against an inbox Eric Wong
2020-05-10 22:37 ` [PATCH 2/5] rename "ContentId" to "ContentHash" Eric Wong
2020-05-10 22:37 ` Eric Wong [this message]
2020-05-10 22:37 ` [PATCH 4/5] msgmap: use TRUNCATE for journal_mode, for now Eric Wong
2020-05-10 22:37 ` [PATCH 5/5] spawn: use ~/.cache/public-inbox/inline-c if writable Eric Wong
2020-05-11 0:29 ` Eric Wong
2020-05-11 4:27 ` [PATCH v2] " Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200510223715.19254-4-e@yhbt.net \
--to=e@yhbt.net \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).