imap: support isearch and reduce Xapian queries

Since IMAP search (either with Isearch or traditional per-Inbox
search) only returns UIDs, we can safely set the limit to the
UID slice size(*).  With isearch, we can also trust the Xapian
result to fit any docid range we specify.

Limiting Xapian results to 1000 was making ->ALL docid <=>
per-Inbox UID impossible since results could overlap between
ranges unpredictably.

Finally, we can map the ->ALL docids into per-Inbox UIDs and
show them to the client in the UID order of the Inbox, not the
docid order of the ->ALL extindex.

This also lets us get rid of the "uid:" query parser prefix
and use the Xapian::Query API directly to reduce our search
prefix footprint.

For mbox.gz downloads in WWW, we'll also make a best effort to
preserve the order from the Inbox, not the order of extindex;
though it's possible large result sets can have non-overlapping
windows.

(*) by definition, UID slice size is a "safe" value which
    shouldn't OOM either the server or clients.

1 files changed, 3 insertions, 3 deletions

diff --git a/lib/PublicInbox/Mbox.pm b/lib/PublicInbox/Mbox.pm
index 19459150..0df31e7f 100644
--- a/lib/PublicInbox/Mbox.pm
+++ b/lib/PublicInbox/Mbox.pm
@@ -222,7 +222,7 @@ sub results_cb {
                 my $mset = $srch->mset($ctx->{query}, $ctx->{qopts});
                 my $size = $mset->size or return;
                 $ctx->{qopts}->{offset} += $size;
-                $ctx->{ids} = $srch->mset_to_artnums($mset);
+                $ctx->{ids} = $srch->mset_to_artnums($mset, $ctx->{qopts});
         }
 }
 
@@ -244,7 +244,7 @@ sub results_thread_cb {
                 my $mset = $srch->mset($ctx->{query}, $ctx->{qopts});
                 my $size = $mset->size or return;
                 $ctx->{qopts}->{offset} += $size;
-                $ctx->{ids} = $srch->mset_to_artnums($mset);
+                $ctx->{ids} = $srch->mset_to_artnums($mset, $ctx->{qopts});
         }
 
 }
@@ -265,7 +265,7 @@ sub mbox_all {
                         return [404, [qw(Content-Type text/plain)],
                                 ["No results found\n"]];
         $ctx->{query} = $q_string;
-        $ctx->{ids} = $srch->mset_to_artnums($mset);
+        $ctx->{ids} = $srch->mset_to_artnums($mset, $qopts);
         require PublicInbox::MboxGz;
         my $fn;
         if ($q->{t} && $srch->has_threadid) {