diff options
| author | Eric Wong <e@80x24.org> | 2019-06-16 01:04:28 +0000 |
|---|---|---|
| committer | Eric Wong <e@80x24.org> | 2019-06-16 01:04:28 +0000 |
| commit | c477bdd8a80eecc319b680764edfb24bd12cb7b2 (patch) | |
| tree | fb36e978b8dbe1f2228527891d47ef9d69b10591 /Documentation/public-inbox-v2-format.pod | |
| parent | 044b1d03c76246316d52de4bfd578745a8192398 (diff) | |
| parent | 27658d2c8b8e51fa64f523c873587273f4f16c46 (diff) | |
| download | public-inbox-c477bdd8a80eecc319b680764edfb24bd12cb7b2.tar.gz | |
* origin/newspeak:
comments: replace "partition" with "shard"
t/xcpdb-reshard: use 'shard' term in local variables
xapcmd: favor 'shard' over 'part' in local variables
search: use "shard" for local variable
v2writable: use "epoch" consistently when referring to git repos
adminedit: "part" => "shard" for local variables
v2writable: rename local vars to match Xapian terminology
v2writable: avoid "part" in internal subs and fields
search*: rename {partition} => {shard}
xapcmd: update comments referencing "partitions"
v2: rename SearchIdxPart => SearchIdxShard
inboxwritable: s/partitions/shards/ in local var
tests: change messages to use "shard" instead of partition
v2writable: rename {partitions} field to {shards}
v2writable: count_partitions => count_shards
searchidxpart: start using "shard" in user-visible places
rename reference to git epochs as "partitions"
admin|xapcmd: user-facing messages say "shard"
v2writable: update comments regarding xcpdb --reshard
doc: rename our Xapian "partitions" to "shards"
Diffstat (limited to 'Documentation/public-inbox-v2-format.pod')
| -rw-r--r-- | Documentation/public-inbox-v2-format.pod | 10 |
1 files changed, 5 insertions, 5 deletions
diff --git a/Documentation/public-inbox-v2-format.pod b/Documentation/public-inbox-v2-format.pod index bdfe7abc..28d3550c 100644 --- a/Documentation/public-inbox-v2-format.pod +++ b/Documentation/public-inbox-v2-format.pod @@ -16,7 +16,7 @@ Message-IDs. The key change in v2 is the inbox is no longer a bare git repository, but a directory with two or more git repositories. v2 divides git repositories by time "epochs" and Xapian -databases for parallelism by "partitions". +databases for parallelism by "shards". =head2 INBOX OVERVIEW AND DEFINITIONS @@ -28,7 +28,7 @@ foo/ # assuming "foo" is the name of the list - inbox.lock # lock file (flock) to protect global state - git/$EPOCH.git # normal git repositories - all.git # empty git repo, alternates to git/$EPOCH.git -- xap$SCHEMA_VERSION/$PART # per-partition Xapian DB +- xap$SCHEMA_VERSION/$SHARD # per-shard Xapian DB - xap$SCHEMA_VERSION/over.sqlite3 # OVER-view DB for NNTP and threading - msgmap.sqlite3 # same the v1 msgmap @@ -95,16 +95,16 @@ are documented at: L<https://public-inbox.org/meta/20180209205140.GA11047@dcvr/> -=head2 XAPIAN PARTITIONS +=head2 XAPIAN SHARDS Another second scalability problem in v1 was the inability to utilize multiple CPU cores for Xapian indexing. This is -addressed by using partitions in Xapian to perform import +addressed by using shards in Xapian to perform import indexing in parallel. As with git alternates, Xapian natively supports a read-only interface which transparently abstracts away the knowledge of -multiple partitions. This allows us to simplify our read-only +multiple shards. This allows us to simplify our read-only code paths. The performance of the storage device is now the bottleneck on |