user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 2/2] httpd: disable Deflater middleware by default on Perl <5.18
Date: Thu, 26 Sep 2019 01:50:38 +0000	[thread overview]
Message-ID: <20190926015038.29770-3-e@80x24.org> (raw)
In-Reply-To: <20190926015038.29770-1-e@80x24.org>

Testing with perl-5.16.3-294.el7_6 RPM package on RHEL/CentOS 7,
the Deflater middleware triggers a leak when used in conjunction
with our push-based responses from PublicInbox::Qspawn.

I could not find another solution to workaround the memory leak
in this case, and I could not find a specific leak fix in
the perl5180delta manpage[1] which looked like it would
solve our problem.

Attempting to workaround the issue proved futile.  Using
internal Deflater-specific keys to prevent deflating in
GitHTTPBackend and Qspawn did not solve the problem:

        $env->{"plack.skip-deflater"} = 1;
        $env->{"psgix.no-compress"} = 1;

Nor did forcing an invalid encoding via "git fetch":

	git -c http.extraheader=Accept-Encoding:gzap fetch

So this appears to be a problem with Plack::Util::response_cb
somewhere.

This does NOT appear to be a problem with ref() leaking as in
DS::next_tick[2], since I couldn't find where
Plack::Middleware::Deflater or Plack::Util::response_cb would be
calling ref() on a blessed reference to trigger a leak.

Also, oddly enough, the ref() use for backwards compatibility at
the top of PublicInbox::GitHTTPBackend::serve does NOT seem to
trigger a leak on 5.16.3 due to [2]:

	# XXX compatibility... ugh, can we stop supporting this?
	$git = PublicInbox::Git->new($git) unless ref($git);

[1] https://perldoc.perl.org/perl5180delta.html
[2] https://rt.perl.org/Public/Bug/Display.html?id=114340
---
 script/public-inbox-httpd | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/script/public-inbox-httpd b/script/public-inbox-httpd
index b2464f4..9b869f9 100755
--- a/script/public-inbox-httpd
+++ b/script/public-inbox-httpd
@@ -24,7 +24,13 @@ my $refresh = sub {
 		my $www = PublicInbox::WWW->new;
 		$www->preload;
 		$app = builder {
-			eval {
+			# Perl 5.16.3 leaks in our "push" response code path
+			# (e.g. Qspawn) due to something in
+			# Plack::Util::response_cb, regardless of whether the
+			# client is sending Accept-Encoding:gzip requests.
+			# perl5180delta documents many leak fixes, so assume
+			# 5.18+ is safe for now and bump the check as-need:
+			$] >= 5.018000 and eval {
 				enable 'Deflater',
 					content_type => [ qw(
 						text/html
-- 
EW


  parent reply	other threads:[~2019-09-26  1:50 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-26  1:50 [PATCH 0/2] leak workarounds for Perl 5.16 on CentOS/RHEL 7 Eric Wong
2019-09-26  1:50 ` [PATCH 1/2] ds: workaround a memory leak in Perl 5.16.x Eric Wong
2019-09-26  1:50 ` Eric Wong [this message]
2019-09-26 12:36 ` [PATCH 0/2] leak workarounds for Perl 5.16 on CentOS/RHEL 7 Konstantin Ryabitsev
2019-09-26 21:16   ` Eric Wong
2019-09-27 21:01 ` [PATCH 3/2] qspawn: workaround Perl 5.16.3 leak, re-enable Deflater Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190926015038.29770-3-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).