From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-3.4 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, RP_MATCHES_RCVD shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: meta@public-inbox.org Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 3E8621F8B9 for ; Thu, 20 Aug 2015 02:57:24 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 06/11] search: avoid needless decode Date: Thu, 20 Aug 2015 02:57:18 +0000 Message-Id: <1440039443-27052-6-git-send-email-e@80x24.org> In-Reply-To: <1440039443-27052-1-git-send-email-e@80x24.org> References: <1440039443-27052-1-git-send-email-e@80x24.org> List-Id: Email::MIME should handle everything for us and make things work nicely with Xapian (assuming I understand how encoding works in Perl). While we're at it, reduce temporary strings and arrays by using destructive operations and clobbering parts as we iterate through them. --- lib/PublicInbox/Search.pm | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm index c28401b..aa29ae5 100644 --- a/lib/PublicInbox/Search.pm +++ b/lib/PublicInbox/Search.pm @@ -6,7 +6,6 @@ use strict; use warnings; use PublicInbox::SearchMsg; use Search::Xapian qw/:standard/; -require PublicInbox::View; use Email::MIME; use PublicInbox::MID qw/mid_clean mid_compressed/; @@ -88,7 +87,6 @@ sub add_message { my $mid = mid_compressed($mid_orig); my $was_ghost = 0; my $ct_msg = $mime->header('Content-Type') || 'text/plain'; - my $enc_msg = PublicInbox::View::enc_for($ct_msg); eval { my $smsg = $self->lookup_message($mid); @@ -151,9 +149,11 @@ sub add_message { # account for filter bugs... $ct =~ m!\btext/plain\b!i or return; - my $enc = PublicInbox::View::enc_for($ct, $enc_msg); my (@orig, @quot); - foreach my $l (split(/\n/, $enc->decode($part->body))) { + my $body = $part->body; + $part->body_set(''); + my @lines = split(/\n/, $body); + while (defined(my $l = shift @lines)) { if ($l =~ /^\s*>/) { push @quot, $l; } else { @@ -162,10 +162,12 @@ sub add_message { } if (@quot) { $tg->index_text(join("\n", @quot), 0); + @quot = (); $tg->increase_termpos; } if (@orig) { $tg->index_text(join("\n", @orig)); + @orig = (); $tg->increase_termpos; } }); -- EW