* [PATCH 09/10] index: support --verbose option
2019-05-29 8:28 7% [PATCH 00/10] index: support progress output Eric Wong
@ 2019-05-29 8:28 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-05-29 8:28 UTC (permalink / raw)
To: meta
It doesn't implement progress of batches, yet, but it wires
up the parsing of the command-line while preserving output
compatibility.
This output is NOT meant to be stable.
---
lib/PublicInbox/Admin.pm | 18 ++++++++++++++++++
lib/PublicInbox/Xapcmd.pm | 14 +-------------
script/public-inbox-index | 5 +++--
3 files changed, 22 insertions(+), 15 deletions(-)
diff --git a/lib/PublicInbox/Admin.pm b/lib/PublicInbox/Admin.pm
index 34aa312..07d8b57 100644
--- a/lib/PublicInbox/Admin.pm
+++ b/lib/PublicInbox/Admin.pm
@@ -167,4 +167,22 @@ sub index_inbox {
}
}
+sub progress_prepare ($) {
+ my ($opt) = @_;
+
+ # public-inbox-index defaults to quiet, -xcpdb and -compact do not
+ if (defined($opt->{quiet}) && $opt->{quiet} < 0) {
+ $opt->{quiet} = !$opt->{verbose};
+ }
+ if ($opt->{quiet}) {
+ open my $null, '>', '/dev/null' or
+ die "failed to open /dev/null: $!\n";
+ $opt->{1} = fileno($null); # suitable for spawn() redirect
+ $opt->{-dev_null} = $null;
+ } else {
+ $opt->{verbose} ||= 1;
+ $opt->{-progress} = sub { print STDERR @_ };
+ }
+}
+
1;
diff --git a/lib/PublicInbox/Xapcmd.pm b/lib/PublicInbox/Xapcmd.pm
index 999819c..7e3d47f 100644
--- a/lib/PublicInbox/Xapcmd.pm
+++ b/lib/PublicInbox/Xapcmd.pm
@@ -86,18 +86,6 @@ sub prepare_reindex ($$) {
}
}
-sub progress_prepare ($) {
- my ($opt) = @_;
- if ($opt->{quiet}) {
- open my $null, '>', '/dev/null' or
- die "failed to open /dev/null: $!\n";
- $opt->{1} = fileno($null);
- $opt->{-dev_null} = $null;
- } else {
- $opt->{-progress} = sub { print STDERR @_ };
- }
-}
-
sub same_fs_or_die ($$) {
my ($x, $y) = @_;
return if ((stat($x))[0] == (stat($y))[0]); # 0 - st_dev
@@ -132,7 +120,7 @@ sub process_queue {
sub run {
my ($ibx, $task, $opt) = @_; # task = 'cpdb' or 'compact'
my $cb = \&${\"PublicInbox::Xapcmd::$task"};
- progress_prepare($opt ||= {});
+ PublicInbox::Admin::progress_prepare($opt ||= {});
my $dir = $ibx->{mainrepo} or die "no mainrepo in inbox\n";
runnable_or_die($XAPIAN_COMPACT) if $opt->{compact};
my $reindex; # v1:{ from => $x40 }, v2:{ from => [ $x40, $x40, .. ] } }
diff --git a/script/public-inbox-index b/script/public-inbox-index
index 40187b3..c0d637b 100755
--- a/script/public-inbox-index
+++ b/script/public-inbox-index
@@ -13,8 +13,8 @@ my $usage = "public-inbox-index INBOX_DIR";
use PublicInbox::Admin;
PublicInbox::Admin::require_or_die('-index');
-my $opt = {};
-GetOptions($opt, qw(reindex jobs|j=i prune indexlevel|L=s))
+my $opt = { quiet => -1 };
+GetOptions($opt, qw(verbose|v+ reindex jobs|j=i prune indexlevel|L=s))
or die "bad command-line args\n$usage";
die "--jobs must be positive\n" if defined $opt->{jobs} && $opt->{jobs} <= 0;
@@ -35,4 +35,5 @@ foreach my $ibx (@ibxs) {
}
PublicInbox::Admin::require_or_die(keys %$mods);
+PublicInbox::Admin::progress_prepare($opt);
PublicInbox::Admin::index_inbox($_, $opt) for @ibxs;
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 00/10] index: support progress output
@ 2019-05-29 8:28 7% Eric Wong
2019-05-29 8:28 7% ` [PATCH 09/10] index: support --verbose option Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-05-29 8:28 UTC (permalink / raw)
To: meta
While progress support was implemented for -xcpdb, there was no
way to enable it for the -index command; and v2 progress was
lacking compared to v1.
This series fills in those functionality gaps and cleans up some
of the v2 indexing code at the same time.
Eric Wong (10):
v2writable: introduce $sync state and put mm_tmp in it
v2writable: sync: move delete markers into $sync state
v2writable: move {reindex} field to $sync state
v2writable: move {regen} into $sync state
v2writable: move {ranges} into $sync state
v2writable: localize unindex-range.$EPOCH to $sync state
v2writable: use prototypes for internal subs
v2writable: move index_sync options to sync state
index: support --verbose option
v2writable: show progress updates for index_sync
lib/PublicInbox/Admin.pm | 18 ++++
lib/PublicInbox/V2Writable.pm | 178 +++++++++++++++++++---------------
lib/PublicInbox/Xapcmd.pm | 14 +--
script/public-inbox-index | 5 +-
4 files changed, 124 insertions(+), 91 deletions(-)
--
EW
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-05-29 8:28 7% [PATCH 00/10] index: support progress output Eric Wong
2019-05-29 8:28 7% ` [PATCH 09/10] index: support --verbose option Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).