Date | Commit message (Collapse) |
|
Determining storage device speed and latencies doesn't
seem portable or even possible with the wide variety
of storage layers in use.
This means we need to write a tuning document and hope
users read and improve on it :P
|
|
This gives better page cache utilization for Xapian indexing on
slow storage by improving locality for random I/O activity on
the Xapian DB.
Instead of doing a single-pass to index both SQLite and Xapian;
this indexes them separately. The first pass is identical to
indexlevel=basic: it indexes both over.sqlite3 and msgmap.sqlite3.
Subsequent passes only operate on a single Xapian shard for
documents belonging to that shard. Given enough shards, each
individual shard can be made small enough to fit into the kernel
page cache and avoid HDD seeks for read activity.
Doing rough tests with a busy system with a 7200 RPM HDD with ext4,
full indexing of LKML (9 epochs) goes from ~80 hours (-j0) to
~30 hours (-j8) with 16GB RAM with 7 shards configured and fsync(2)
disabled (--no-sync) and `--batch-size=10m'.
|
|
The old name may be confused with "Content-ID" as described in
RFC 2392, so use an alternate name to avoid confusing future
readers.
|
|
Since v2 inboxes contain multiple git repositories, avoid the
use of the word "repository" when referring to inboxes as a
whole in most places.
|
|
I didn't wait until September to do it, this year!
|
|
While both can be correct, the former seems more common,
is shorter, and is also consistent with the spelling found
in the AGPL-3.0 text.
|
|
We need to better ensure our manpage output is readable with a
standard terminal width. And fix some wording while we're at
it:
* use "inbox" instead of "list" for our storage
* replace the last "$PART" reference with "$SHARD"
|
|
This was being rendered as a paragraph, so line breaks weren't
preserved and it was unreadable in man.
|
|
For consistency with Xapian documentation (in the "master"
branch).
|
|
This is true as of e220b8b2ee5cfd458167dc2c6c92726352c4c80e
("Merge remote-tracking branch 'origin/xap-optional' into master")
|
|
-index documentation avoid redundant v1 information and refers
readers to apropriate v1/v2 manpages. Search::Xapian can also
be optional, now, as only the PSGI search interface uses it.
Favor "INBOX_DIR" where appropriate, since "REPO_DIR" can be
confused for code repos which we also support.
XAPIAN_FLUSH_THRESHOLD is documented for all relevant
bulk commands.
|
|
We're going to need copydatabase, too
|
|
|
|
Remove confusing documentation around ssoma now that we
have NNTP and downloadable mbox support.
Only lightly-checked for grammar and speling, and not yet
formatting. Edits, corrections and addendums expected :>
|