user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 8/9] drop bodies and messages ASAP after processing
Date: Mon, 17 Aug 2015 03:38:16 +0000	[thread overview]
Message-ID: <1439782697-16412-9-git-send-email-e@80x24.org> (raw)
In-Reply-To: <1439782697-16412-1-git-send-email-e@80x24.org>

We can rely on reference counting to lower memory usage for
big messages.
---
 lib/PublicInbox/Feed.pm | 10 ++++++----
 lib/PublicInbox/View.pm |  2 ++
 t/view.t                | 12 +++++++-----
 3 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/lib/PublicInbox/Feed.pm b/lib/PublicInbox/Feed.pm
index 226c50e..95bde4f 100644
--- a/lib/PublicInbox/Feed.pm
+++ b/lib/PublicInbox/Feed.pm
@@ -243,23 +243,25 @@ sub add_to_feed {
 	my $midurl = $feed_opts->{midurl} || 'http://example.com/m/';
 	my $fullurl = $feed_opts->{fullurl} || 'http://example.com/f/';
 
-	my $mid = $mime->header_obj->header_raw('Message-ID');
+	my $header_obj = $mime->header_obj;
+	my $mid = $header_obj->header_raw('Message-ID');
 	defined $mid or return 0;
 	$mid = PublicInbox::Hval->new_msgid($mid);
 	my $href = $mid->as_href . '.html';
 	my $content = PublicInbox::View->feed_entry($mime, $fullurl . $href);
 	defined($content) or return 0;
+	$mime = undef;
 
-	my $subject = mime_header($mime, 'Subject') or return 0;
+	my $subject = mime_header($header_obj, 'Subject') or return 0;
 
-	my $from = mime_header($mime, 'From') or return 0;
+	my $from = mime_header($header_obj, 'From') or return 0;
 	my @from = Email::Address->parse($from);
 	my $name = $from[0]->name;
 	defined $name or $name = "";
 	my $email = $from[0]->address;
 	defined $email or $email = "";
 
-	my $date = $mime->header('Date');
+	my $date = $header_obj->header('Date');
 	$date = PublicInbox::Hval->new_oneline($date);
 	$date = feed_date($date->raw) or return 0;
 	$add =~ tr!/!!d;
diff --git a/lib/PublicInbox/View.pm b/lib/PublicInbox/View.pm
index 4f2833b..55e12f2 100644
--- a/lib/PublicInbox/View.pm
+++ b/lib/PublicInbox/View.pm
@@ -116,6 +116,7 @@ sub index_entry {
 				  \$more);
 		$part_nr++;
 	});
+	$mime->body_set('');
 
 	$rv .= "\n$pfx<a\nhref=\"$mhref\">$more</a> ";
 	my $txt = "${path}m/$href.txt";
@@ -277,6 +278,7 @@ sub multipart_text_as_html {
 		$rv .= "\n" unless $rv =~ /\n\z/s;
 		++$part_nr;
 	});
+	$mime->body_set('');
 	$rv;
 }
 
diff --git a/t/view.t b/t/view.t
index 63de49b..3107285 100644
--- a/t/view.t
+++ b/t/view.t
@@ -39,9 +39,9 @@ EOF
 			Subject => 'this is a subject',
 		],
 		body => $body,
-	);
-	$s = Email::MIME->new($s->as_string);
-	my $html = PublicInbox::View->msg_html($s);
+	)->as_string;
+	my $mime = Email::MIME->new($s);
+	my $html = PublicInbox::View->msg_html($mime);
 
 	# ghetto tests
 	like($html, qr!<a\nhref="\.\./m/hello%40!s, "MID link present");
@@ -52,7 +52,8 @@ EOF
 
 	# short page
 	my $pfx = "http://example.com/test/f";
-	my $short = PublicInbox::View->msg_html($s, $pfx);
+	$mime = Email::MIME->new($s);
+	my $short = PublicInbox::View->msg_html($mime, $pfx);
 	like($short, qr!<a\nhref="hello%40!s, "MID link present");
 	like($short, qr/\n&gt; keep this inline/,
 		"short quoted text is inline");
@@ -137,8 +138,9 @@ EOF
 		parts => $parts,
 	);
 
+	my $orig = $mime->body_raw;
 	my $html = PublicInbox::View->msg_html($mime);
-	like($mime->body_raw, qr/hi =3D bye=/, "our test used QP correctly");
+	like($orig, qr/hi =3D bye=/, "our test used QP correctly");
 	like($html, qr/\bhi = bye\b/, "HTML output decoded QP");
 }
 
-- 
EW


  parent reply	other threads:[~2015-08-17  3:38 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-08-17  3:38 [PATCH 0/9] search pushed and deployed to master Eric Wong
2015-08-17  3:38 ` [PATCH 1/9] feed: remove unnecesary time paramenter in index state Eric Wong
2015-08-17  3:38 ` [PATCH 2/9] favor /t/ to /s/, since subjects may change mid-thread Eric Wong
2015-08-17  3:38 ` [PATCH 3/9] WWW: eliminate "top" parameter for feeds Eric Wong
2015-08-17  3:38 ` [PATCH 4/9] www: simplify parameter passing to feed Eric Wong
2015-08-17  3:38 ` [PATCH 5/9] terminology: replies => followups Eric Wong
2015-08-17  3:38 ` [PATCH 6/9] search: use raw headers without MIME decoding Eric Wong
2015-08-17  3:38 ` [PATCH 7/9] feed: disable the generator statement Eric Wong
2015-08-17  3:38 ` Eric Wong [this message]
2015-08-17  3:38 ` [PATCH 9/9] search: apply mid_compression to subject paths, too Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1439782697-16412-9-git-send-email-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).