user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH] shrink low-bandwidth pipes under Linux
@ 2019-01-05 21:52  7% Eric Wong
  0 siblings, 0 replies; 1+ results
From: Eric Wong @ 2019-01-05 21:52 UTC (permalink / raw)
  To: meta

I've hit /proc/sys/fs/pipe-user-pages-* limits on some systems.
So stop hogging resources on pipes which don't benefit from
giant sizes.

Some of these can use eventfd in the future to further reduce
resource use.
---
 lib/PublicInbox/Daemon.pm     | 5 +++++
 lib/PublicInbox/EvCleanup.pm  | 4 ++++
 lib/PublicInbox/Git.pm        | 4 ++++
 lib/PublicInbox/V2Writable.pm | 3 +++
 4 files changed, 16 insertions(+)

diff --git a/lib/PublicInbox/Daemon.pm b/lib/PublicInbox/Daemon.pm
index 4629aad..6d2ae81 100644
--- a/lib/PublicInbox/Daemon.pm
+++ b/lib/PublicInbox/Daemon.pm
@@ -356,6 +356,11 @@ sub unlink_pid_file_safe_ish ($$) {
 sub master_loop {
 	pipe(my ($p0, $p1)) or die "failed to create parent-pipe: $!";
 	pipe(my ($r, $w)) or die "failed to create self-pipe: $!";
+
+	if ($^O eq 'linux') { # 1031: F_SETPIPE_SZ = 1031
+		fcntl($_, 1031, 4096) for ($w, $p1);
+	}
+
 	IO::Handle::blocking($w, 0);
 	my $set_workers = $worker_processes;
 	my @caught;
diff --git a/lib/PublicInbox/EvCleanup.pm b/lib/PublicInbox/EvCleanup.pm
index 1a3a3d5..1a2bdb2 100644
--- a/lib/PublicInbox/EvCleanup.pm
+++ b/lib/PublicInbox/EvCleanup.pm
@@ -19,7 +19,11 @@ my $laterq = [ [], undef ];
 sub once_init () {
 	my $self = fields::new('PublicInbox::EvCleanup');
 	my ($r, $w);
+
+	# This is a dummy pipe which is always writable so it can always
+	# fires in the next event loop iteration.
 	pipe($r, $w) or die "pipe: $!";
+	fcntl($w, 1031, 4096) if $^O eq 'linux'; # 1031: F_SETPIPE_SZ
 	$self->SUPER::new($w);
 	$self->{rd} = $r; # never read, since we never write..
 	$self;
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 95df52e..1611727 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -37,6 +37,10 @@ sub _bidi_pipe {
 
 	pipe($in_r, $in_w) or fail($self, "pipe failed: $!");
 	pipe($out_r, $out_w) or fail($self, "pipe failed: $!");
+	if ($^O eq 'linux') { # 1031: F_SETPIPE_SZ
+		fcntl($out_w, 1031, 4096);
+		fcntl($in_w, 1031, 4096) if $batch eq '--batch-check';
+	}
 
 	my @cmd = ('git', "--git-dir=$self->{git_dir}", qw(cat-file), $batch);
 	my $redir = { 0 => fileno($out_r), 1 => fileno($in_w) };
diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index 93babed..bfbe195 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -243,6 +243,9 @@ sub idx_init {
 
 	if ($self->{parallel}) {
 		pipe(my ($r, $w)) or die "pipe failed: $!";
+		# pipe for barrier notifications doesn't need to be big,
+		# 1031: F_SETPIPE_SZ
+		fcntl($w, 1031, 4096) if $^O eq 'linux';
 		$self->{bnote} = [ $r, $w ];
 		$w->autoflush(1);
 	}
-- 
EW


^ permalink raw reply related	[relevance 7%]

Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-01-05 21:52  7% [PATCH] shrink low-bandwidth pipes under Linux Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).