From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 05/12] lei lcat: use single queue for ordering
Date: Tue, 21 Sep 2021 07:41:52 +0000 [thread overview]
Message-ID: <20210921074159.20052-6-e@80x24.org> (raw)
In-Reply-To: <20210921074159.20052-1-e@80x24.org>
If lcat-ing multiple argument types (blobs vs folders),
maintain the original order of the arguments instead of
dumping all blobs before folder contents.
---
lib/PublicInbox/LeiLcat.pm | 13 ++++++-------
lib/PublicInbox/LeiXSearch.pm | 16 ++++++++--------
2 files changed, 14 insertions(+), 15 deletions(-)
diff --git a/lib/PublicInbox/LeiLcat.pm b/lib/PublicInbox/LeiLcat.pm
index ccb1823d..1a4a988e 100644
--- a/lib/PublicInbox/LeiLcat.pm
+++ b/lib/PublicInbox/LeiLcat.pm
@@ -21,7 +21,7 @@ sub lcat_folder ($$$) {
} else {
for my $f (@$folders) {
my $fid = $lms->fid_for($f);
- push @{$lei->{lcat_fid}}, $fid;
+ push @{$lei->{lcat_todo}}, { fid => $fid };
}
}
}
@@ -31,10 +31,9 @@ sub lcat_imap_uri ($$) {
my $lms = $lei->lms or return;
# cf. LeiXsearch->lcat_dump
if (defined $uri->uid) {
- my @oidhex = $lms->imap_oidhex($lei, $uri);
- push @{$lei->{lcat_blob}}, @oidhex;
+ push @{$lei->{lcat_todo}}, $lms->imap_oidhex($lei, $uri);
} elsif (defined(my $fid = $lms->fid_for($$uri))) {
- push @{$lei->{lcat_fid}}, $fid;
+ push @{$lei->{lcat_todo}}, { fid => $fid };
} else {
lcat_folder($lei, $lms, $$uri);
}
@@ -46,10 +45,10 @@ sub extract_1 ($$) {
my $u = $1;
require PublicInbox::URIimap;
lcat_imap_uri($lei, PublicInbox::URIimap->new($u));
- '""'; # blank query, using {lcat_blob} or {lcat_fid}
+ '""'; # blank query, using {lcat_todo}
} elsif ($x =~ m!\b(maildir:.+)!i) {
lcat_folder($lei, undef, $1);
- '""'; # blank query, using {lcat_blob} or {lcat_fid}
+ '""'; # blank query, using {lcat_todo}
} elsif ($x =~ m!\b([a-z]+?://\S+)!i) {
my $u = $1;
$u =~ s/[\>\]\)\,\.\;]+\z//;
@@ -82,7 +81,7 @@ sub extract_1 ($$) {
} elsif ($x =~ /\bid:(\S+)/) { # notmuch convention
"mid:$1";
} elsif ($x =~ /\bblob:([0-9a-f]{7,})\b/) {
- push @{$lei->{lcat_blob}}, $1; # cf. LeiToMail->wq_atexit_child
+ push @{$lei->{lcat_todo}}, $1; # cf. LeiToMail->wq_atexit_child
'""'; # blank query
} else {
undef;
diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index 756183a9..3ce8f32d 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -570,7 +570,7 @@ sub do_query {
@$end = ();
$self->{opt_threads} = $lei->{opt}->{threads};
$self->{opt_sort} = $lei->{opt}->{'sort'};
- $self->{-do_lcat} = $lei->{lcat_blob} // $lei->{lcat_fid};
+ $self->{-do_lcat} = !!(delete $lei->{lcat_todo});
if ($l2m) {
$l2m->net_merge_all_done unless $lei->{auth};
} else {
@@ -646,13 +646,13 @@ sub lcat_dump { # via wq_io_do
$git->cat_async($smsg->{blob}, \&_lcat2smsg, $smsg);
};
}
- for my $oid (@{$lei->{lcat_blob} // []}) {
- $each_smsg->({ blob => $oid, pct => 100 });
- }
- if (my $fids = delete $lei->{lcat_fid}) {
- my $lms = $lei->{lse}->lms;
- for my $fid (@$fids) {
- $lms->each_src({fid => $fid}, \&_lcat_i, $each_smsg);
+ my $lms;
+ for my $ent (@{$lei->{lcat_todo}}) {
+ if (ref $ent eq 'HASH') { # { fid => $fid ,.. }
+ $lms //= $lei->{lse}->lms;
+ $lms->each_src($ent, \&_lcat_i, $each_smsg);
+ } else { # oidhex
+ $each_smsg->({ blob => $ent, pct => 100 });
}
}
$git->async_wait_all;
next prev parent reply other threads:[~2021-09-21 7:41 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-21 7:41 [PATCH 00/12] lei: fix various annoyances Eric Wong
2021-09-21 7:41 ` [PATCH 01/12] lei inspect: convert to WQ worker Eric Wong
2021-09-21 7:41 ` [PATCH 02/12] lei inspect: support NNTP URLs Eric Wong
2021-09-21 7:41 ` [PATCH 03/12] lei_mail_sync: account for non-unique cases Eric Wong
2021-09-21 7:41 ` [PATCH 04/12] lei: simplify internal arg2folder usage Eric Wong
2021-09-21 7:41 ` Eric Wong [this message]
2021-09-21 7:41 ` [PATCH 06/12] doc: lei-security: section for WIP auth methods Eric Wong
2021-09-21 7:41 ` [PATCH 07/12] lei lcat: support NNTP URLs Eric Wong
2021-09-21 7:41 ` [PATCH 08/12] lei: various completion improvements Eric Wong
2021-09-21 7:41 ` [PATCH 09/12] lei q: show progress on >1s preparation phase Eric Wong
2021-09-21 7:41 ` [PATCH 10/12] search: drop reopen retry message Eric Wong
2021-09-21 7:41 ` [PATCH 11/12] lei q: update messages to reflect --save default Eric Wong
2021-09-21 7:41 ` [PATCH 12/12] lei q: improve --limit behavior and progress Eric Wong
2021-09-21 9:29 ` [PATCH 0/3] lei: a few more annoyances fixed Eric Wong
2021-09-21 9:29 ` [PATCH 1/3] t/lei-up: use '-q' to silence non-redirected test Eric Wong
2021-09-21 9:29 ` [PATCH 2/3] script/lei: handle SIGTSTP and SIGCONT Eric Wong
2021-09-21 9:29 ` [PATCH 3/3] lei: umask(077) before opening errors.log Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210921074159.20052-6-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).