about summary refs log tree commit homepage
path: root/lib/PublicInbox/MdirReader.pm
DateCommit message (Collapse)
2021-06-09mdir_reader: maildir_each_file: pass flags, skip Trash
This is a slight behavior change for "lei q": Trashed (but not-yet-expunged) messages no longer get unlinked when --output is used without --augment.
2021-06-08lei import: speed up repeated Maildir imports
On a 4-core CPU, this speeds up "lei import" on a largish Maildir inbox with 75K messages from ~8 minutes down to ~40s. Parallelizing alone did not bring any improvement and may even hurt performance slightly, depending on CPU availability. However, creating the index on the "fid" and "name" columns in blob2name yields us the same speedup we got. Parallelizing IMAP makes more sense due to the fact most IMAP stores are non-local and subject to network latency. Followup-to: bdecd7ed8e0dcf0b45491b947cd737ba8cfe38a3 ("lei import: speed up kw updates for old IMAP messages")
2021-05-23lei export-kw: new command to export keywords to Maildirs
IMAP will eventually be supported.
2021-05-19lei: relax rules for "new" in Maildir
mbsync and offlineimap both use ":2," suffixes for filenames in "new/", however my interpretation of the Maildir spec at <https://cr.yp.to/proto/maildir.html> is that ":2," is only for files in "cur/". My interpretation also matches that of doveecot, but we'll allow what mbsync and offlineimap do given their popularity.
2021-04-05lei: maildir: move shard support to MdirReader
We'll eventually want lei_input users like "lei import" and "lei tag" to support parallel reads.
2021-04-01lei: maildir: handle "forwarded" keyword as "P"
mbox and IMAP seem to have no way of describing this keyword. but Maildir does with the "P" flagged (for "passed").
2021-03-20maildir: avoid redundant slashes
Redundant slashes look ugly in strace(1) output.
2021-03-10lei import: skip trashed Maildir messages
This matches IMAP behavior in NetReader in skipping \\Deleted messages. Since lei may be used for personal, non-public mail; Draft messages are NOT skipped by "lei import".
2021-03-04lei: use maildir_each_eml in more places
This saves us some code and redundant callsites for eml_from_path. We'll change maildir_each_eml to include the filename to facilitate an upcoming change to "lei q" without --augment
2021-02-18lei convert: mail format conversion sub-command
This will make testing IMAP support for other commands easier, as it doesn't write to lei/store at all. Like the pager and MUA, "git credential" is always spawned by script/lei (and not lei-daemon) so it has a controlling terminal for password prompts. v2: fix missing requires, correct test ordering v3: ensure config exists for IMAP auth
2021-02-10use MdirReader in -watch and InboxWritable
MdirReader now handles files in "$MAILDIR/new" properly and is stricter about what it accepts. eml_from_path is also made robust against FIFOs while eliminating TOCTOU races with between stat(2) and open(2) calls.
2021-02-10lei: split out MdirReader package, lazy-require earlier
We'll do more requires in the top-level lei-daemon process to save work in workers. We can also work towards aborting on user errors in lei-daemon rather than worker processes. "lei import -f mbox*" is finally tested inside t/lei_to_mail.t