From 10b523eb017162240b1ac3647f8dcbbf2be348a7 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Tue, 8 Jun 2021 09:50:21 +0000 Subject: lei import: speed up repeated Maildir imports On a 4-core CPU, this speeds up "lei import" on a largish Maildir inbox with 75K messages from ~8 minutes down to ~40s. Parallelizing alone did not bring any improvement and may even hurt performance slightly, depending on CPU availability. However, creating the index on the "fid" and "name" columns in blob2name yields us the same speedup we got. Parallelizing IMAP makes more sense due to the fact most IMAP stores are non-local and subject to network latency. Followup-to: bdecd7ed8e0dcf0b45491b947cd737ba8cfe38a3 ("lei import: speed up kw updates for old IMAP messages") --- MANIFEST | 1 + 1 file changed, 1 insertion(+) (limited to 'MANIFEST') diff --git a/MANIFEST b/MANIFEST index 5a70a144..7bdbf252 100644 --- a/MANIFEST +++ b/MANIFEST @@ -221,6 +221,7 @@ lib/PublicInbox/LeiMailSync.pm lib/PublicInbox/LeiMirror.pm lib/PublicInbox/LeiOverview.pm lib/PublicInbox/LeiP2q.pm +lib/PublicInbox/LeiPmdir.pm lib/PublicInbox/LeiQuery.pm lib/PublicInbox/LeiRediff.pm lib/PublicInbox/LeiRemote.pm -- cgit v1.2.3-24-ge0c7