about summary refs log tree commit homepage
path: root/t/eml.t
DateCommit message (Collapse)
2021-10-26eml: keep body if no headers are found
This easily allows us to treat "git diff" output as header-less "messages" for commands such as "lei p2q".
2021-03-21lei q: support vmd for external-only messages
"lei q" now preserves changes per-message keywords across invocations when it's --output (Maildir or mbox) is reused (with or without --augment). In the future, these changes will be monitored via inotify, EVFILT_VNODE or IMAP IDLE, too. Unfortunately, this currently prevents "lei import" from ever importing a message that's in an external. That will be fixed in a future change.
2021-01-01update copyrights for 2021
Using "make update-copyrights" after setting GNULIB_PATH in my config.mak
2020-11-15t/eml.t: workaround newer Email::MIME* behavior
Recent (2020) versions of Email::MIME (and/or dependencies) have different behavior than historical versions which seem to be less DWIM and perhaps technically more correct. We'll retain historical behavior for now, since it doesn't seem to cause real problems and DWIM-ness is often required to make sense of historical mail. Tested on a FreeBSD 11.4 VM with the following packages: p5-Email-MIME-1.949 p5-Email-MIME-ContentType-1.024_1 p5-Email-MIME-Encodings-1.315_2
2020-06-13eml: each_part: single part $idx is 1
Instead of counts starting at 0, we start the single-part message at 1 like we do with subparts of a multipart message. This will make it easier to map offsets for "BODY[$SECTION]" when using IMAP FETCH, since $SECTION must contain non-zero numbers according to RFC 3501. This doesn't make any difference for WWW URLs, since single part messages cannot have downloadable attachments.
2020-05-24t/eml.t: favor ->header over ->header_str
This test may still run against ancient versions of Email::MIME for comparisons.
2020-05-17confine Email::MIME use even further
To avoid confusing future readers and users, recommend PublicInbox::Eml in our Import POD and refer to PublicInbox::Eml comments at the top of PublicInbox::MIME. mime_load() confined to t/eml.t, since we won't be using it anywhere else in our tests.
2020-05-17descend into message/(rfc822|news|global) parts
Email::MIME never supported this properly, but there's real instances of forwarded messages as message/rfc822 attachments. message/news is legacy thing which we'll see in archives, and message/global appears to be the new thing. gmime also supports message/rfc2822, so we'll support it anyways despite lacking other evidence of its existence. Existing attachments remain downloadable as a whole message, but individual attachments of subparts are now downloadable and can be displayed in HTML, too. Furthermore, ensure Xapian can now search for common headers inside those messages as well as the message bodies.
2020-05-10eml: rename limits to match postfix names
They're still part of our internal API at this point, but reusing the same names as those used by postfix makes sense for now to reduce cognitive overheads of learning new things. There's no "mime_parts_limit", but the name is consistent with "mime_nesting_limit".
2020-05-10eml: enforce a maximum header length
While our header processing is more efficient than Email::*::Header, capping the maximum size for a `m//g' match still limits memory growth on a header we care for. Use the same limit as postfix (header_size_limit=102400), since messages fetched via git/HTTP/NNTP/etc can bypass MTA limits.
2020-05-09eml: pure-Perl replacement for Email::MIME
Email::MIME eats memory, wastes time parsing out all the headers, and some problems can't be fixed without breaking compatibility for other projects which depend on it. Informal benchmarks show a ~2x improvement in general stats gathering scripts and ~10% improvement in HTML view rendering. We also don't need the ability to create MIME messages, just parse them and maybe drop an attachment. While this isn't the zero-copy or streaming MIME parser of my dreams; it's still an improvement in that it doesn't keep a scalar copy of the raw body around along with subparts. It also doesn't parse subparts up front, so it can also replace our uses of Email::Simple.