user/dev discussion of public-inbox itself
 help / color / Atom feed
From: Eric Wong <e@yhbt.net>
To: meta@public-inbox.org
Subject: Re: [PATCH 3/4] ds: tmpio: store offsets per-buffer
Date: Fri, 24 Jan 2020 19:07:12 +0000
Message-ID: <20200124190712.GA10143@dcvr> (raw)
In-Reply-To: <20200124094352.19437-4-e@yhbt.net>

Eric Wong <e@yhbt.net> wrote:
> diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
> index 970061fd..a9ac7fcd 100644
> --- a/lib/PublicInbox/DS.pm
> +++ b/lib/PublicInbox/DS.pm
> @@ -490,12 +491,13 @@ sub drop {
>  # PerlIO::mmap or PerlIO::scalar if needed
>  sub tmpio ($$$) {
>      my ($self, $bref, $off) = @_;
> -    my $fh = tmpfile('wbuf', $self->{sock}, 1) or
> +    my $fh = tmpfile('wbuf', $self->{sock}, O_APPEND) or
>          return drop($self, "tmpfile $!");
>      $fh->autoflush(1);
> +    binmode $fh, ':unix'; # reduce syscalls for read() >8192 bytes

That binmode :unix call triggers a leak in Perl[1], going to
have to squash the patch below to workaround it:

diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index a9ac7fcd..c76a5038 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -494,7 +494,6 @@ sub tmpio ($$$) {
     my $fh = tmpfile('wbuf', $self->{sock}, O_APPEND) or
         return drop($self, "tmpfile $!");
     $fh->autoflush(1);
-    binmode $fh, ':unix'; # reduce syscalls for read() >8192 bytes
     my $len = bytes::length($$bref) - $off;
     $fh->write($$bref, $len, $off) or return drop($self, "write ($len): $!");
     [ $fh, 0 ] # [1] = offset, [2] = length, not set by us


... And perhaps just use sysread() instead of read(), since
I'm not seeing a good reason to keep compatibility with
PerlIO::scalar for buffering, after all.

[1] http://nntp.perl.org/group/perl.perl5.porters/256918

  reply index

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-24  9:43 [PATCH 0/4] -httpd static file improvements Eric Wong
2020-01-24  9:43 ` [PATCH 1/4] http: eliminate short-lived cyclic ref for psgix.io Eric Wong
2020-01-24  9:43 ` [PATCH 2/4] wwwstatic: offload error handling to PSGI server Eric Wong
2020-01-24  9:43 ` [PATCH 3/4] ds: tmpio: store offsets per-buffer Eric Wong
2020-01-24 19:07   ` Eric Wong [this message]
2020-01-24  9:43 ` [PATCH 4/4] wwwstatic: wire up buffer bypass for -httpd Eric Wong
2020-01-25 19:27   ` Eric Wong
2020-01-25 19:34   ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200124190712.GA10143@dcvr \
    --to=e@yhbt.net \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git