* [PATCH 10/13] smsg: get rid of remaining {mime} users
2020-06-01 10:06 6% [PATCH 00/13] smsg: remove tricky {mime} field Eric Wong
@ 2020-06-01 10:06 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-06-01 10:06 UTC (permalink / raw)
To: meta
We'll let $smsg->populate take care of everything all at once
without hanging onto the header object for too long.
---
lib/PublicInbox/OverIdx.pm | 1 -
lib/PublicInbox/Smsg.pm | 33 +++------------------------------
2 files changed, 3 insertions(+), 31 deletions(-)
diff --git a/lib/PublicInbox/OverIdx.pm b/lib/PublicInbox/OverIdx.pm
index cb15baadf2b..a078f80451f 100644
--- a/lib/PublicInbox/OverIdx.pm
+++ b/lib/PublicInbox/OverIdx.pm
@@ -254,7 +254,6 @@ sub subject_path ($) {
sub add_overview {
my ($self, $mime, $smsg) = @_;
$smsg->{lines} = $mime->body_raw =~ tr!\n!\n!;
- $smsg->{mime} = $mime; # XXX temporary?
my $hdr = $mime->header_obj;
my $mids = mids_for_index($hdr);
my $refs = parse_references($smsg, $hdr, $mids);
diff --git a/lib/PublicInbox/Smsg.pm b/lib/PublicInbox/Smsg.pm
index 9688c5592a2..9e363a112c0 100644
--- a/lib/PublicInbox/Smsg.pm
+++ b/lib/PublicInbox/Smsg.pm
@@ -12,7 +12,7 @@ use strict;
use warnings;
use base qw(Exporter);
our @EXPORT_OK = qw(subject_normalized);
-use PublicInbox::MID qw(mid_mime mids);
+use PublicInbox::MID qw(mids);
use PublicInbox::Address;
use PublicInbox::MsgTime qw(msg_timestamp msg_datestamp);
use Time::Local qw(timegm);
@@ -96,13 +96,7 @@ sub lines ($) { $_[0]->{lines} }
sub __hdr ($$) {
my ($self, $field) = @_;
- $self->{lc($field)} //= do {
- my $mime = $self->{mime} or return;
- my $val = join(', ', $mime->header($field));
- $val =~ tr/\r//d;
- $val =~ tr/\t\n/ /;
- $val;
- };
+ $self->{lc($field)};
}
# for Import and v1 non-SQLite WWW code paths
@@ -174,34 +168,13 @@ sub from_name {
$self->{from_name};
}
-sub ts {
- my ($self) = @_;
- $self->{ts} ||= eval { msg_timestamp($self->{mime}->header_obj) } || 0;
-}
-
-sub ds {
- my ($self) = @_;
- $self->{ds} ||= eval { msg_datestamp($self->{mime}->header_obj); } || 0;
-}
-
sub references {
my ($self) = @_;
my $x = $self->{references};
defined $x ? $x : '';
}
-sub mid ($;$) {
- my ($self, $mid) = @_;
-
- if (defined $mid) {
- $self->{mid} = $mid;
- } elsif (defined(my $rv = $self->{mid})) {
- $rv;
- } else {
- die "NO {mime} for mid\n" unless $self->{mime};
- mid_mime($self->{mime}) # v1 w/o Xapian
- }
-}
+sub mid { $_[0]->{mid} }
our $REPLY_RE = qr/^re:\s+/i;
^ permalink raw reply related [relevance 7%]
* [PATCH 00/13] smsg: remove tricky {mime} field
@ 2020-06-01 10:06 6% Eric Wong
2020-06-01 10:06 7% ` [PATCH 10/13] smsg: get rid of remaining {mime} users Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-06-01 10:06 UTC (permalink / raw)
To: meta
Storing a large PublicInbox::Eml (or in the past, Email::MIME)
object inside a small PublicInbox::Smsg object has historically
been bloat-prone[1] since there may be many small smsgs in
memory at once
Hundreds or thousands of $smsg objects can linger in memory due
to search results and message threading operations. So keep
$eml and $smsg objects independent of each other, for now.
Instead, we'll introduce a $smsg->populate($eml) API to handle
filling in the keys for the importer, indexer, and
non-SQLite-using WWW users.
Furthermore, $smsg->$field dispatch has always been measurably
faster than $smsg->{$field} access in NNTP. Since $smsg->$field
became read-only with the removal of $smsg->{mime}, we can
abandon the $smsg->$field invocations and favor of direct hash
access.
[1] the prime example being what commit 7d02b9e64455831d fixed
("view: stop storing all MIME objects on large threads")
Eric Wong (13):
inbox: introduce smsg_eml method
wwwatomstream: convert callers to use smsg_eml
v2writable: fix non-sensical interpolation in BUG message
import: modernize to use Perl 5.10 features
smsg: introduce ->populate method
smsg: get rid of ->wrap initializer, too
inbox: msg_by_*: remove $(size)ref args
www: remove smsg_mime API and adjust callers
nntp: smsg_range_i: favor ->{$field} lookups when possible
smsg: get rid of remaining {mime} users
smsg: remove ->bytes and ->lines methods
smsg: remove remaining accessor methods
wwwatomstream: drop smsg->{mid} fallback for non-SQLite
Documentation/mknews.perl | 7 +-
lib/PublicInbox/ExtMsg.pm | 2 +-
lib/PublicInbox/Feed.pm | 8 +-
lib/PublicInbox/Import.pm | 69 ++++++++---------
lib/PublicInbox/Inbox.pm | 32 ++++----
lib/PublicInbox/Mbox.pm | 2 +-
lib/PublicInbox/NNTP.pm | 14 +++-
lib/PublicInbox/OverIdx.pm | 3 +-
lib/PublicInbox/SearchIdx.pm | 33 ++++-----
lib/PublicInbox/SearchView.pm | 6 +-
lib/PublicInbox/Smsg.pm | 123 +++++++++++--------------------
lib/PublicInbox/SolverGit.pm | 4 +-
lib/PublicInbox/V2Writable.pm | 11 +--
lib/PublicInbox/View.pm | 63 ++++++++--------
lib/PublicInbox/WwwAtomStream.pm | 8 +-
t/altid.t | 3 +-
t/altid_v2.t | 3 +-
t/import.t | 3 +-
t/search.t | 46 +++++++-----
t/v2mda.t | 4 +-
t/v2writable.t | 5 +-
21 files changed, 207 insertions(+), 242 deletions(-)
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-06-01 10:06 6% [PATCH 00/13] smsg: remove tricky {mime} field Eric Wong
2020-06-01 10:06 7% ` [PATCH 10/13] smsg: get rid of remaining {mime} users Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).