user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH 0/9] convert: more fixes and tests
@ 2020-02-02  6:52  7% Eric Wong
  2020-02-02  6:52  5% ` [PATCH 8/9] doc: -convert: document switches Eric Wong
  0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-02-02  6:52 UTC (permalink / raw)
  To: meta

Nothing major, but SMP systems should be better utilized
now that -convert can take advantage of parallel shard
indexing in Xapian.  The switches for convert are also
now documented in the manpage.

Eric Wong (9):
  t/multi-mid.t: extra test for -convert highwater mark
  v2writable: nproc_shards: subtract 1 from given value
  v2writable: do not clobber {shards} or {parallel} if unset
  convert: remove unused variables capturing :from
  searchidxshard: rely on autoflush instead of ->flush
  convert: shift @ARGV explicitly
  convert: fix --no-index switch
  doc: -convert: document switches
  v2writable: more ways to detect online CPU count

 Documentation/public-inbox-convert.pod | 41 ++++++++++++++++-
 MANIFEST                               |  1 +
 lib/PublicInbox/SearchIdxShard.pm      |  4 +-
 lib/PublicInbox/V2Writable.pm          | 38 +++++++++++-----
 script/public-inbox-convert            | 16 ++-----
 t/convert-compact.t                    |  3 ++
 t/multi-mid.t                          | 61 ++++++++++++++++++++++++++
 t/xcpdb-reshard.t                      |  2 +-
 8 files changed, 137 insertions(+), 29 deletions(-)
 create mode 100644 t/multi-mid.t

^ permalink raw reply	[relevance 7%]

* [PATCH 8/9] doc: -convert: document switches
  2020-02-02  6:52  7% [PATCH 0/9] convert: more fixes and tests Eric Wong
@ 2020-02-02  6:52  5% ` Eric Wong
  0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-02-02  6:52 UTC (permalink / raw)
  To: meta

These switches have always been there, but were not
documented until now.
---
 Documentation/public-inbox-convert.pod | 41 ++++++++++++++++++++++++--
 1 file changed, 39 insertions(+), 2 deletions(-)

diff --git a/Documentation/public-inbox-convert.pod b/Documentation/public-inbox-convert.pod
index 4c790dee..417e5e4e 100644
--- a/Documentation/public-inbox-convert.pod
+++ b/Documentation/public-inbox-convert.pod
@@ -4,7 +4,7 @@ public-inbox-convert - convert v1 inboxes to v2
 
 =head1 SYNOPSIS
 
-	public-inbox-convert OLD_DIR NEW_DIR
+	public-inbox-convert [OPTIONS] OLD_DIR NEW_DIR
 
 =head1 DESCRIPTION
 
@@ -14,6 +14,27 @@ and users are expected to update the "inboxdir" path in
 L<public-inbox-config(5)> to point to the path of NEW_DIR
 once they are satisfied with the conversion.
 
+=head1 OPTIONS
+
+=over
+
+=item --no-index
+
+Disables Xapian and overview DB indexing on the new repository.
+By default, public-inbox-convert creates a new index in the v2
+repository and indexes all existing messages, a lengthy
+operation for large inboxes.
+
+=item -j JOBS, --jobs=JOBS
+
+Control the number of indexing jobs and Xapian shards of the v2
+inbox.  By default, this is the detected CPU count but capped
+at 4 due to various bottlenecks.  The number of Xapian shards
+will be 1 less than the JOBS value, since there is a single
+process which distributes work to the Xapian shards.
+
+=back
+
 =head1 ENVIRONMENT
 
 =over 8
@@ -27,6 +48,21 @@ See L<public-inbox-config(5)>
 
 =head1 UPGRADING
 
+Editing "~/.public-inbox/config" (or whatever C<PI_CONFIG> is
+set to) will be required to start using the new directory.
+
+=head1 BUGS
+
+Writes from L<public-inbox-mda(1)> or L<git-fetch(1)> to the v1
+inbox which occur after the start of the conversion will not
+be picked up in the v2 inbox.
+
+Users of L<public-inbox-watch(1)> do not have to worry about
+this.  They only need to update the config file to point to the
+v2 inbox, send C<SIGHUP> public-inbox-watch process to reload
+the config file, and then C<SIGUSR1> to rescan existing
+Maildirs.
+
 =head1 CONTACT
 
 Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>
@@ -42,4 +78,5 @@ License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>
 
 =head1 SEE ALSO
 
-L<public-inbox-init(1)>, L<public-inbox-index(1)>
+L<public-inbox-init(1)>, L<public-inbox-index(1)>, L<public-inbox-config(5)>,
+L<public-inbox-v1-format(5)>, L<public-inbox-v2-format(5)>

^ permalink raw reply related	[relevance 5%]

Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-02-02  6:52  7% [PATCH 0/9] convert: more fixes and tests Eric Wong
2020-02-02  6:52  5% ` [PATCH 8/9] doc: -convert: document switches Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).