* [PATCH 11/12] nntp: log to FDs given by the Nntpd module
2015-09-19 2:03 5% [PATCH 0/12] nntp: misc updates Eric Wong
@ 2015-09-19 2:03 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2015-09-19 2:03 UTC (permalink / raw)
To: meta
This will allow us to redirect stdout/stderr more easily
for logging.
---
lib/PublicInbox/NNTP.pm | 29 ++++++++++++++++++-----------
public-inbox-nntpd | 2 ++
2 files changed, 20 insertions(+), 11 deletions(-)
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 01039ba..80adb65 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -61,7 +61,7 @@ sub process_line ($$) {
my $err = $@;
if ($err && !$self->{closed}) {
chomp($l = Dumper(\$l));
- warning('error from: ', $l, ' ', $err);
+ err($self, "error from: $l $err");
$res = '503 program fault - command not performed';
}
return 0 unless defined $res;
@@ -502,13 +502,13 @@ sub long_response ($$$$) {
$self->{long_res} = undef;
if ($err) {
- warning("$err during long response[$fd] - ".
- sprintf('%0.6', now() - $t0));
+ err($self,
+ "$err during long response[$fd] - %0.6f",
+ now() - $t0);
}
if ($self->{closed}) {
- printf(STDERR
- " deferred[$fd] aborted - %0.6f\n",
- now() - $t0);
+ out($self, " deferred[$fd] aborted - %0.6f",
+ now() - $t0);
} else {
$self->watch_read(1);
}
@@ -522,8 +522,7 @@ sub long_response ($$$$) {
$self->{long_res} = undef;
$self->watch_read(1);
res($self, '.');
- printf(STDERR " deferred[$fd] done - %0.6f\n",
- now() - $t0);
+ out($self, " deferred[$fd] done - %0.6f", now() - $t0);
}
};
$self->{long_res}->(); # kick off!
@@ -713,6 +712,16 @@ sub do_write ($$) {
$done;
}
+sub err ($$;@) {
+ my ($self, $fmt, @args) = @_;
+ printf { $self->{nntpd}->{err} } $fmt."\n", @args;
+}
+
+sub out ($$;@) {
+ my ($self, $fmt, @args) = @_;
+ printf { $self->{nntpd}->{out} } $fmt."\n", @args;
+}
+
use constant MSG_MORE => ($^O eq 'linux') ? 0x8000 : 0;
sub do_more ($$) {
@@ -750,7 +759,7 @@ sub event_read {
$r = eval { $self->process_line($line) };
my $d = $self->{long_res} ?
' deferred['.fileno($self->{sock}).']' : '';
- printf(STDERR "$line - %0.6f$d\n", now() - $t0);
+ out($self, "$line - %0.6f$d", now() - $t0);
}
return $self->close if $r < 0;
my $len = bytes::length($$buf);
@@ -758,6 +767,4 @@ sub event_read {
$self->push_back_read($buf) if ($len);
}
-sub warning { print STDERR 'W: ', @_, "\n" }
-
1;
diff --git a/public-inbox-nntpd b/public-inbox-nntpd
index 7fec840..0c221fa 100644
--- a/public-inbox-nntpd
+++ b/public-inbox-nntpd
@@ -44,6 +44,8 @@ sub new {
my ($class) = @_;
my $self = fields::new($class);
$self->{groups} = {};
+ $self->{err} = \*STDERR;
+ $self->{out} = \*STDOUT;
$self;
}
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 0/12] nntp: misc updates
@ 2015-09-19 2:03 5% Eric Wong
2015-09-19 2:03 7% ` [PATCH 11/12] nntp: log to FDs given by the Nntpd module Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2015-09-19 2:03 UTC (permalink / raw)
To: meta
Still not changing or adding new caches or modifying our data layout,
but things seem to work well for the common case.
The most important change is the new long response API which will help
us even after any future optimizations we make. We should be able to
stream millions of messages without excessive buffering and memory
usage.
Eric Wong (12):
nntp: use write_buf_size instead write_buf
nntp: introduce long response API for streaming
nntp: use long response API for LISTGROUP
nntp: implement command argument checking
nntp: XOVER does not require range
nntp: speed up XHDR for the Message-ID case
nntp: implement XROVER, speed up XHDR for some cases
nntp: implement XPATH
nntp: fix logging of long responses
nntp: fix ARTICLE/HEAD/BODY/STAT
nntp: log to FDs given by the Nntpd module
nntp: article lookups by Message-ID may cross newsgroups
lib/PublicInbox/Msgmap.pm | 24 +--
lib/PublicInbox/NNTP.pm | 419 ++++++++++++++++++++++++++++++++-----------
lib/PublicInbox/SearchMsg.pm | 1 +
public-inbox-nntpd | 2 +
4 files changed, 330 insertions(+), 116 deletions(-)
^ permalink raw reply [relevance 5%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2015-09-19 2:03 5% [PATCH 0/12] nntp: misc updates Eric Wong
2015-09-19 2:03 7% ` [PATCH 11/12] nntp: log to FDs given by the Nntpd module Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).