user/dev discussion of public-inbox itself
 help / color / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 6/7] doc: xcpdb: add switch documentation
Date: Fri, 24 May 2019 02:57:57 +0000
Message-ID: <20190524025758.32261-7-e@80x24.org> (raw)
In-Reply-To: <20190524025758.32261-1-e@80x24.org>

In particular, the '--compact' switch is really useful since it
works without holding the inbox-wide lock for minutes at a time
on giant inboxes (inboxes where copies can take dozens, if not
hundreds of minutes).
---
 Documentation/public-inbox-compact.pod | 10 ++++++++
 Documentation/public-inbox-xcpdb.pod   | 34 ++++++++++++++++++++++----
 script/public-inbox-xcpdb              |  2 +-
 3 files changed, 40 insertions(+), 6 deletions(-)

diff --git a/Documentation/public-inbox-compact.pod b/Documentation/public-inbox-compact.pod
index 7d37f6f..47d950a 100644
--- a/Documentation/public-inbox-compact.pod
+++ b/Documentation/public-inbox-compact.pod
@@ -16,6 +16,16 @@ writing while it operates.
 It enforces the use of the C<--no-renumber> option of
 L<xapian-compact(1)>
 
+=head1 OPTIONS
+
+=over
+
+=item --blocksize / --no-full / --fuller
+
+These options are passed directly to L<xapian-compact(1)>.
+
+=back
+
 =head1 ENVIRONMENT
 
 =over 8
diff --git a/Documentation/public-inbox-xcpdb.pod b/Documentation/public-inbox-xcpdb.pod
index 5697dcd..e056b54 100644
--- a/Documentation/public-inbox-xcpdb.pod
+++ b/Documentation/public-inbox-xcpdb.pod
@@ -4,7 +4,7 @@ public-inbox-xcpdb - upgrade Xapian DB formats
 
 =head1 SYNOPSIS
 
-	public-inbox-xcpdb INBOX_DIR
+	public-inbox-xcpdb [OPTIONS] INBOX_DIR
 
 =head1 DESCRIPTION
 
@@ -14,10 +14,28 @@ upgrading to the latest database format supported by Xapian
 recover from Xapian database modifications from
 L<public-inbox-watch(1)> or L<public-inbox-mda(1)>.
 
-This is intended for upgrading the database format used by
-Xapian.  It DOES NOT upgrade the schema used by the
-public-inbox PSGI search interface (see
-L<public-inbox-index(1)>).
+=head1 OPTIONS
+
+=over
+
+=item --compact
+
+In addition to performing the copy operation, run L<xapian-compact(1)>
+on each Xapian partition after copying but before finalizing it.
+Compared to the cost of copying a Xapian database, compacting a
+Xapian database takes only around 5% of the time required to copy.
+
+Compared to L<public-inbox-compact(1)>, use of this option is
+preferable for gigantic inboxes where the coarse-grained lock
+currently required for L<public-inbox-compact(1)> can cause
+the compaction to take hours at-a-time.
+
+=item --blocksize / --no-full / --fuller
+
+These options are passed directly to L<xapian-compact(1)> when
+used with C<--compact>.
+
+=back
 
 =head1 ENVIRONMENT
 
@@ -40,6 +58,12 @@ Default: 10000
 
 =head1 UPGRADING
 
+This tool is intended for admins upgrading Xapian search databases
+used by public-inbox, NOT users upgrading public-inbox itself.
+
+In particular, it DOES NOT upgrade the schema used by the
+PSGI search interface (see L<public-inbox-index(1)>).
+
 =head1 CONTACT
 
 Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>
diff --git a/script/public-inbox-xcpdb b/script/public-inbox-xcpdb
index badb95e..ef64e58 100755
--- a/script/public-inbox-xcpdb
+++ b/script/public-inbox-xcpdb
@@ -7,7 +7,7 @@ use PublicInbox::InboxWritable;
 use PublicInbox::Xapcmd;
 use PublicInbox::Admin;
 PublicInbox::Admin::require_or_die('-search');
-my $usage = "Usage: public-inbox-xcpdb INBOX_DIR\n";
+my $usage = "Usage: public-inbox-xcpdb [--compact] INBOX_DIR\n";
 my $opt = {};
 GetOptions($opt, qw(compact), @PublicInbox::Xapcmd::COMPACT_OPT) or
 	die "bad command-line args\n$usage";
-- 
EW


  parent reply index

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-24  2:57 [PATCH 0/7] various documentation fixes Eric Wong
2019-05-24  2:57 ` [PATCH 1/7] doc: index: fix miscapitalization of "SQLite" Eric Wong
2019-05-24  2:57 ` [PATCH 2/7] doc: daemon: fix manpage section for nginx Eric Wong
2019-05-24  2:57 ` [PATCH 3/7] doc: don't barf on missing `git set-file-times' Eric Wong
2019-05-24  2:57 ` [PATCH 4/7] doc: sync .txt mtime to .pod mtime Eric Wong
2019-05-24  2:57 ` [PATCH 5/7] doc: generate manpages for some Xapian commands Eric Wong
2019-05-24  2:57 ` Eric Wong [this message]
2019-05-24  2:57 ` [PATCH 7/7] doc: add URLs for Xapian manpages Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190524025758.32261-7-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git