miscsearch: a new Xapian sub-DB for extindex

This will be used to index and search Inbox objects and perhaps
individual git repositories/epochs for grokmirror manifest.js.gz
generation.  There is no sharding planned for this at the moment
since inbox count should remain low (~100K to 1M) compared to
message count.

Folding this into the existing sharded DBs could be possible;
but would likely increase query and maintenance costs, as well
as development complexity.  So we'll use a few more inodes and
FDs at runtime, instead.

1 files changed, 5 insertions, 0 deletions

diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index ba7cef13..afba0220 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -631,6 +631,9 @@ sub checkpoint ($;$) {
                         $_->shard_commit for @$shards;
                 }
 
+                my $midx = $self->{midx}; # misc index
+                $midx->commit_txn if $midx;
+
                 # last_commit is special, don't commit these until
                 # Xapian shards are done:
                 $dbh->begin_work if $dbh;
@@ -639,6 +642,7 @@ sub checkpoint ($;$) {
                         $dbh->commit;
                         $dbh->begin_work;
                 }
+                $midx->begin_txn if $midx;
         }
         $self->{total_bytes} += $self->{transact_bytes};
         $self->{transact_bytes} = 0;
@@ -678,6 +682,7 @@ sub done {
         }
         eval { $self->{oidx}->dbh_close };
         $err .= "over close: $@\n" if $@;
+        delete $self->{midx};
         delete $self->{bnote};
         my $nbytes = $self->{total_bytes};
         $self->{total_bytes} = 0;