diff options
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/public-inbox-v2-format.pod | 10 | ||||
-rw-r--r-- | Documentation/public-inbox-xcpdb.pod | 11 |
2 files changed, 10 insertions, 11 deletions
diff --git a/Documentation/public-inbox-v2-format.pod b/Documentation/public-inbox-v2-format.pod index bdfe7abc..28d3550c 100644 --- a/Documentation/public-inbox-v2-format.pod +++ b/Documentation/public-inbox-v2-format.pod @@ -16,7 +16,7 @@ Message-IDs. The key change in v2 is the inbox is no longer a bare git repository, but a directory with two or more git repositories. v2 divides git repositories by time "epochs" and Xapian -databases for parallelism by "partitions". +databases for parallelism by "shards". =head2 INBOX OVERVIEW AND DEFINITIONS @@ -28,7 +28,7 @@ foo/ # assuming "foo" is the name of the list - inbox.lock # lock file (flock) to protect global state - git/$EPOCH.git # normal git repositories - all.git # empty git repo, alternates to git/$EPOCH.git -- xap$SCHEMA_VERSION/$PART # per-partition Xapian DB +- xap$SCHEMA_VERSION/$SHARD # per-shard Xapian DB - xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading - msgmap.sqlite3 # same the v1 msgmap @@ -95,16 +95,16 @@ are documented at: L<https://public-inbox.org/meta/20180209205140.GA11047@dcvr/> -=head2 XAPIAN PARTITIONS +=head2 XAPIAN SHARDS Another second scalability problem in v1 was the inability to utilize multiple CPU cores for Xapian indexing. This is -addressed by using partitions in Xapian to perform import +addressed by using shards in Xapian to perform import indexing in parallel. As with git alternates, Xapian natively supports a read-only interface which transparently abstracts away the knowledge of -multiple partitions. This allows us to simplify our read-only +multiple shards. This allows us to simplify our read-only code paths. The performance of the storage device is now the bottleneck on diff --git a/Documentation/public-inbox-xcpdb.pod b/Documentation/public-inbox-xcpdb.pod index fd8770a4..a13c4efa 100644 --- a/Documentation/public-inbox-xcpdb.pod +++ b/Documentation/public-inbox-xcpdb.pod @@ -21,7 +21,7 @@ L<public-inbox-watch(1)> or L<public-inbox-mda(1)>. =item --compact In addition to performing the copy operation, run L<xapian-compact(1)> -on each Xapian partition after copying but before finalizing it. +on each Xapian shard after copying but before finalizing it. Compared to the cost of copying a Xapian database, compacting a Xapian database takes only around 5% of the time required to copy. @@ -32,14 +32,13 @@ the compaction to take hours at-a-time. =item --reshard=N / -R N -Repartition the Xapian database on a L<v2|public-inbox-v2-format(5)> -inbox to C<N> partitions. Since L<xapian-compact(1)> is not suitable -for merging, users can rely on this switch to repartition the +Reshard the Xapian database on a L<v2|public-inbox-v2-format(5)> +inbox to C<N> shards . Since L<xapian-compact(1)> is not suitable +for merging, users can rely on this switch to reshard the existing Xapian database(s) to any positive value of C<N>. This is useful in case the Xapian DB was created with too few or -too many partitions given the capabilities of the current -hardware. +too many shards given the capabilities of the current hardware. =item --blocksize / --no-full / --fuller |