isearch: emulate per-inbox search with ->ALL

Using "eidx_key:" boolean prefix to limit results to a given
inbox, we can use ->ALL to emulate and replace per-Inbox
xap15/[0-9] search indices.

With this change, the presence of "extindex.all.topdir" in the
$PI_CONFIG will cause the WWW code to use that extindex and
ignore per-inbox Xapian DBs in xap15/[0-9].

Unfortunately IMAP search still requires old per-inbox indices,
for now.  Mapping extindex Xapian docids to per-Inbox UIDs and
vice-versa is proving tricky.  Fortunately, IMAP search is
rarely used and optional.  The RFCs don't specify expensive
phrase search, either, so `indexlevel=medium' can be used in
per-inbox Xapian indices to save space.

For primarily WWW (and future JMAP) users; this should result in
significant disk space, FD, and page cache footprint savings for
large instances with many inboxes and many cross-posted
messages.

1 files changed, 24 insertions, 1 deletions

diff --git a/t/extsearch.t b/t/extsearch.t
index 2b8b88ea..97786b21 100644
--- a/t/extsearch.t
+++ b/t/extsearch.t
@@ -130,9 +130,32 @@ my $es = PublicInbox::ExtSearch->new("$home/extindex");
         is($mset->size, 1, 'new message found');
         $mset = $es->mset('b:"test message"');
         is($mset->size, 1, 'old message found');
-
         delete @$es{qw(git over xdb)}; # fork preparation
 
+        my $pi_cfg = PublicInbox::Config->new;
+        $pi_cfg->fill_all;
+        is(scalar($pi_cfg->ALL->mset('s:Testing')->items), 2,
+                '2 results in ->ALL');
+        my $res = {};
+        my $nr = 0;
+        $pi_cfg->each_inbox(sub {
+                $nr++;
+                my ($ibx) = @_;
+                local $SIG{__WARN__} = sub {}; # FIXME support --reindex
+                my $mset = $ibx->isrch->mset('s:Testing');
+                $res->{$ibx->eidx_key} = $ibx->isrch->mset_to_smsg($ibx, $mset);
+        });
+        is($nr, 2, 'two inboxes');
+        my $exp = {};
+        for my $v (qw(v1 v2)) {
+                my $ibx = $pi_cfg->lookup_newsgroup("$v.example");
+                my $smsg = $ibx->over->get_art(1);
+                $smsg->psgi_cull;
+                $exp->{"$v.example"} = [ $smsg ];
+        }
+        is_deeply($res, $exp, 'isearch limited results');
+        $pi_cfg = $res = $exp = undef;
+
         open my $rmfh, '+>', undef or BAIL_OUT $!;
         $rmfh->autoflush(1);
         print $rmfh $eml2->as_string or BAIL_OUT $!;