From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-2.9 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00 shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: meta@public-inbox.org Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 83ED820102 for ; Mon, 21 Sep 2015 11:11:15 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 3/6] nntp: speed up xover slightly Date: Mon, 21 Sep 2015 11:11:09 +0000 Message-Id: <20150921111112.18873-4-e@80x24.org> In-Reply-To: <20150921111112.18873-1-e@80x24.org> References: <20150921111112.18873-1-e@80x24.org> List-Id: Reserializing the message to a string to check size wastes considerable time and should be able to get by with slightly less accuracy. --- lib/PublicInbox/GitCatFile.pm | 3 ++- lib/PublicInbox/NNTP.pm | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/PublicInbox/GitCatFile.pm b/lib/PublicInbox/GitCatFile.pm index 4f16762..dd3f291 100644 --- a/lib/PublicInbox/GitCatFile.pm +++ b/lib/PublicInbox/GitCatFile.pm @@ -38,7 +38,7 @@ sub _cat_file_begin { } sub cat_file { - my ($self, $object) = @_; + my ($self, $object, $sizeref) = @_; $object .= "\n"; my $len = bytes::length($object); @@ -58,6 +58,7 @@ sub cat_file { die "Unexpected result from git cat-file: $head\n"; my $size = $1; + $$sizeref = $size if $sizeref; my $bytes_left = $size; my $offset = 0; my $rv = ''; diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm index d5eb497..fb93330 100644 --- a/lib/PublicInbox/NNTP.pm +++ b/lib/PublicInbox/NNTP.pm @@ -366,7 +366,8 @@ find_mid: } found: my $o = 'HEAD:' . mid2path($mid); - my $s = eval { Email::Simple->new($ng->gcf->cat_file($o)) }; + my $bytes; + my $s = eval { Email::Simple->new($ng->gcf->cat_file($o, \$bytes)) }; return $err unless $s; if ($set_headers) { $s->header_set('Newsgroups', $ng->{name}); @@ -375,7 +376,7 @@ found: # must be last if ($set_headers == 2) { - $s->header_set('Bytes', bytes::length($s->as_string)); + $s->header_set('Bytes', $bytes); $s->body_set(''); } } -- EW