* [PATCH 2/3] eml: speed up common LF-only emails
2020-05-09 8:27 6% [PATCH 0/3] eml: some fixes and speedups Eric Wong
@ 2020-05-09 8:27 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-05-09 8:27 UTC (permalink / raw)
To: meta
Emails a *nix MTA are typically LF-only, so we don't need the
complexity of the RE engine when a simple index() works. We
still need to ensure there's no "\r\n\r\n" before the first
"\n\n", but two calls to index() is still faster than a RE
match.
This gives a 2-5% speedup in some informal tests and saves ~30MB
when scanning a 30MB spam message on newer versions of Perl.
I'll have to diagnose why Perl wastes so much memory doing
RE matches on giant strings, though.
---
lib/PublicInbox/Eml.pm | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/Eml.pm b/lib/PublicInbox/Eml.pm
index 80e7c1af..f022516c 100644
--- a/lib/PublicInbox/Eml.pm
+++ b/lib/PublicInbox/Eml.pm
@@ -71,10 +71,18 @@ sub re_memo ($) {
# compatible with our uses of Email::MIME
sub new {
my $ref = ref($_[1]) ? $_[1] : \(my $cpy = $_[1]);
- if ($$ref =~ /\r?\n(\r?\n)/s) { # likely
- # This can modify $$ref in-place and to avoid memcpy/memmove
- # on a potentially large $$ref. It does need to make a
- # copy for $hdr, though. Idea stolen from Email::Simple
+ # substr() can modify the first arg in-place and to avoid
+ # memcpy/memmove on a potentially large scalar. It does need
+ # to make a copy for $hdr, though. Idea stolen from Email::Simple.
+
+ # We also prefer index() on common LFLF emails since it's faster
+ # and re scan can bump RSS by length($$ref) on big strings
+ if (index($$ref, "\r\n") < 0 && (my $pos = index($$ref, "\n\n")) >= 0) {
+ # likely on *nix
+ my $hdr = substr($$ref, 0, $pos + 2, ''); # sv_chop on $$ref
+ chop($hdr); # lower SvCUR
+ bless { hdr => \$hdr, crlf => "\n", bdy => $ref }, __PACKAGE__;
+ } elsif ($$ref =~ /\r?\n(\r?\n)/s) {
my $hdr = substr($$ref, 0, $+[0], ''); # sv_chop on $$ref
substr($hdr, -(length($1))) = ''; # lower SvCUR
bless { hdr => \$hdr, crlf => $1, bdy => $ref }, __PACKAGE__;
^ permalink raw reply related [relevance 7%]
* [PATCH 0/3] eml: some fixes and speedups
@ 2020-05-09 8:27 6% Eric Wong
2020-05-09 8:27 7% ` [PATCH 2/3] eml: speed up common LF-only emails Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-05-09 8:27 UTC (permalink / raw)
To: meta
Nothing major and the 1/3 side effect fix never affected
any of our use cases.
Eric Wong (3):
eml: reduce RE captures and possible side effects
eml: speed up common LF-only emails
emlcontentfoo: avoid undefined warning on missing attributes
lib/PublicInbox/Eml.pm | 26 +++++++++++++++++---------
lib/PublicInbox/EmlContentFoo.pm | 2 +-
2 files changed, 18 insertions(+), 10 deletions(-)
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-05-09 8:27 6% [PATCH 0/3] eml: some fixes and speedups Eric Wong
2020-05-09 8:27 7% ` [PATCH 2/3] eml: speed up common LF-only emails Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).