user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <>
Subject: [PATCH] index|compact|xcpdb: support --all switch
Date: Thu, 13 Aug 2020 08:04:37 +0000	[thread overview]
Message-ID: <> (raw)

For -index, this is a convenient way to quickly index all
inboxes after a grok-pull.  Might as well support it for
rarely used commands like -compact and -xcpdb, too.
 Documentation/public-inbox-compact.pod | 10 +++++++++-
 Documentation/public-inbox-index.pod   |  8 ++++++++
 Documentation/public-inbox-xcpdb.pod   | 10 +++++++++-
 script/public-inbox-compact            |  4 ++--
 script/public-inbox-index              |  5 +++--
 script/public-inbox-xcpdb              |  5 +++--
 6 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/Documentation/public-inbox-compact.pod b/Documentation/public-inbox-compact.pod
index d7a7dba0..8e463ab1 100644
--- a/Documentation/public-inbox-compact.pod
+++ b/Documentation/public-inbox-compact.pod
@@ -4,7 +4,9 @@ public-inbox-compact - compact Xapian DBs in an inbox
 =head1 SYNOPSIS
-	public-inbox-compact INBOX_DIR
+public-inbox-compact INBOX_DIR
+public-inbox-compact --all
@@ -21,6 +23,12 @@ rest of the public-inbox search code.
+=item --all
+Compact all inboxes configured in ~/.public-inbox/config.
+This is an alternative to specifying individual inboxes directories
+on the command-line.
 =item --blocksize / --no-full / --fuller
 These options are passed directly to L<xapian-compact(1)>.
diff --git a/Documentation/public-inbox-index.pod b/Documentation/public-inbox-index.pod
index b28db773..1ed9f5e7 100644
--- a/Documentation/public-inbox-index.pod
+++ b/Documentation/public-inbox-index.pod
@@ -6,6 +6,8 @@ public-inbox-index - create and update search indices
 public-inbox-index [OPTIONS] INBOX_DIR...
+public-inbox-index [OPTIONS] --all
 public-inbox-index creates and updates the search, overview and
@@ -81,6 +83,12 @@ This does not touch the NNTP article number database.
 It does not affect threading unless C<--rethread> is
+=item --all
+Index all inboxes configured in ~/.public-inbox/config.
+This is an alternative to specifying individual inboxes directories
+on the command-line.
 =item --rethread
 Regenerate internal THREADID and message thread associations
diff --git a/Documentation/public-inbox-xcpdb.pod b/Documentation/public-inbox-xcpdb.pod
index 62a28c0a..2b38f1da 100644
--- a/Documentation/public-inbox-xcpdb.pod
+++ b/Documentation/public-inbox-xcpdb.pod
@@ -4,7 +4,9 @@ public-inbox-xcpdb - upgrade Xapian DB formats
 =head1 SYNOPSIS
-	public-inbox-xcpdb [OPTIONS] INBOX_DIR
+public-inbox-xcpdb [OPTIONS] INBOX_DIR
+public-inbox-xcpdb [OPTIONS] --all
@@ -19,6 +21,12 @@ L<public-inbox-learn(1)>, and L<public-inbox-index(1)>.
+=item --all
+Copy all inboxes configured in ~/.public-inbox/config.
+This is an alternative to specifying individual inboxes directories
+on the command-line.
 =item -c, --compact
 In addition to performing the copy operation, run L<xapian-compact(1)>
diff --git a/script/public-inbox-compact b/script/public-inbox-compact
index 5c681466..b5fa0086 100755
--- a/script/public-inbox-compact
+++ b/script/public-inbox-compact
@@ -10,9 +10,9 @@ use PublicInbox::Admin;
 my $usage = "Usage: public-inbox-compact INBOX_DIR\n";
 my $opt = { compact => 1, -coarse_lock => 1 };
-GetOptions($opt, @PublicInbox::Xapcmd::COMPACT_OPT) or
+GetOptions($opt, qw(all), @PublicInbox::Xapcmd::COMPACT_OPT) or
 	die "bad command-line args\n$usage";
-my @ibxs = PublicInbox::Admin::resolve_inboxes(\@ARGV) or die $usage;
+my @ibxs = PublicInbox::Admin::resolve_inboxes(\@ARGV, $opt) or die $usage;
 foreach (@ibxs) {
 	my $ibx = PublicInbox::InboxWritable->new($_);
 	PublicInbox::Xapcmd::run($ibx, 'compact', $opt);
diff --git a/script/public-inbox-index b/script/public-inbox-index
index 14d3afd4..30d24838 100755
--- a/script/public-inbox-index
+++ b/script/public-inbox-index
@@ -18,6 +18,7 @@ options:
   --no-fsync          speed up indexing, risk corruption on power outage
   -L LEVEL            `basic', `medium', or `full' (default: full)
+  --all               index all configured inboxes
   --compact | -c      run public-inbox-compact(1) after indexing
   --sequential-shard  index Xapian shards sequentially for slow storage
   --jobs=NUM          set or disable parallelization (NUM=0)
@@ -38,7 +39,7 @@ GetOptions($opt, qw(verbose|v+ reindex rethread compact|c+ jobs|j=i prune
 		indexlevel|index-level|L=s max_size|max-size=s
-		help|?))
+		all help|?))
 	or die "bad command-line args\n$usage";
 if ($opt->{help}) { print $help; exit 0 };
 die "--jobs must be >= 0\n" if defined $opt->{jobs} && $opt->{jobs} < 0;
@@ -51,7 +52,7 @@ require PublicInbox::Admin;
 my $cfg = PublicInbox::Config->new; # Config is loaded by Admin
-my @ibxs = PublicInbox::Admin::resolve_inboxes(\@ARGV, undef, $cfg);
+my @ibxs = PublicInbox::Admin::resolve_inboxes(\@ARGV, $opt, $cfg);
 unless (@ibxs) { print STDERR "Usage: $usage\n"; exit 1 }
diff --git a/script/public-inbox-xcpdb b/script/public-inbox-xcpdb
index 718a34b7..2bfadc09 100755
--- a/script/public-inbox-xcpdb
+++ b/script/public-inbox-xcpdb
@@ -13,6 +13,7 @@ usage: $usage
   --compact | -c      run public-inbox-compact(1) after indexing
+  --all               copy all configured inboxes
   --reshard=NUM       change number the number of shards
   --jobs=NUM          limit parallelism to JOBS count
   --verbose | -v      increase verbosity (may be repeated)
@@ -34,7 +35,7 @@ GetOptions($opt, qw(
 	jobs|j=i quiet|q verbose|v
 	blocksize|b=s no-full|n fuller|F
-	help|?)) or die "bad command-line args\n$usage";
+	all help|?)) or die "bad command-line args\n$usage";
 if ($opt->{help}) { print $help; exit 0 };
 use PublicInbox::Admin;
@@ -42,7 +43,7 @@ PublicInbox::Admin::require_or_die('-search');
 require PublicInbox::Config;
 my $cfg = PublicInbox::Config->new;
-my @ibxs = PublicInbox::Admin::resolve_inboxes(\@ARGV, undef, $cfg) or
+my @ibxs = PublicInbox::Admin::resolve_inboxes(\@ARGV, $opt, $cfg) or
 	die $usage;
 my $idx_env = PublicInbox::Admin::index_prepare($opt, $cfg);

                 reply	other threads:[~2020-08-13  8:04 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \
    --subject='Re: [PATCH] index|compact|xcpdb: support --all switch' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

This inbox may be cloned and mirrored by anyone:

	git clone --mirror
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 meta meta/ \
	public-inbox-index meta

Example config snippet for mirrors.
Newsgroups are available over NNTP:
 note: .onion URLs require Tor:

code repositories for project(s) associated with this inbox:

AGPL code for this site: git clone