| Date | Commit message (Collapse) |
|---|
|
It may be hard to tell what command triggered an error,
otherwise.
|
|
Oops
|
|
We don't want to waste precious server memory on clients which
have been idle for longer than 3 minutes.
|
|
The document data of a search message already contains a good chunk
of the information needed to respond to OVER/XOVER commands quickly.
Expand on that and use the document data to implement OVER/XOVER
quickly.
This adds a dependency on Xapian being available for nntpd usage,
but is probably alright since nntpd is esoteric enough that anybody
willing to run nntpd will also want search functionality offered
by Xapian.
This also speeds up XHDR/HDR with the To: and Cc: headers and
:bytes/:lines article metadata used by some clients for header
displays and marking messages as read/unread.
|
|
We shall remove slow, unoptimized headers in XHDR/HDR to avoid
becoming an easy DoS target.
|
|
Redundantly confirm to clients we do not accept posting with the
MODE READER command.
ref: RFC 3977 5.3.1
|
|
No point in sending such a short, bounded response with
multiple syscalls.
|
|
This is stipulated by RFC 3977 8.5.1, but apparently I misread it.
|
|
Using a signal-based timer can hurt throughput on a machine that's
overloaded. Ensure there's always forward progress and reduce the
number of syscalls we make, too.
|
|
This can help us track down what request patterns clients
will perform when we have multiple clients.
|
|
Oops, we must increment our range even if we yield or
get blocked on output buffering.
|
|
This is required by RFC 3977, section 3.2.1
|
|
RFC 3977, section 8.5.2 states metadata lookups can be done
with HDR.
|
|
Oops :x
|
|
This is better encapsulated and hopefully more readable.
While we're at it, check for being inside a long response, too.
|
|
Oops, we need to test commands more closely :x
Add a missing prototype while we're at it for extra
checking.
|
|
This is similar to XHDR, but differs in how it handles Message-ID
lookups.
|
|
This is allowed by RFC 2980 and HDR (to-be-implemented) in
RFC 3977 supports it, too.
|
|
We'll require some modifications for HDR support, though.
|
|
This is just like the XOVER command, but allows a single Message-ID
to be given.
|
|
These are internal metadata should be calculated, so avoid
leaking it into the header.
|
|
It's common for mail bodies to end with LF-only, so end them with
CRLF to avoid triggering errors in clients.
|
|
RFC 3977 supports YYYYMMDD dates while retaining backwards
compatibility for old YYMMDD dates.
|
|
RFC 3977 stipulates the use of UTF-8 as the default charset,
so we shall try using that and hopefully not mangle things.
|
|
Apparently, my mental model of Perl internals is still incorrect
after all these years. I am but a simple *nix programmer:
everything is a bag of bytes to me.
This fixes a problem with UTF-8 headers from Xapian (via
"XHDR Subject [range]") triggering partial writes and writing an
extra newline to the outputs.
|
|
Reserializing the message to a string to check size wastes
considerable time and should be able to get by with slightly
less accuracy.
|
|
We could also start displaying Xref in XOVER as rtin seems to
prefer it. Anyways this is nearly 100 times faster now and
requires no DB changes.
|
|
Using non-hierarchical mailing list names for newsgroups
might confuse traditional newsreader software and perhaps
some humans. Allow administrators to configure newsgroups
names and hierarchies to their liking.
Sorting the grouplist alphabetically should probably be
done anyways to improve usability for some clients which
won't sort themselves.
|
|
We cannot use the push_back_read functionality of Danga::Socket
since it will trigger event_read on buffered data. This would
allow a malicious (or badly implemented) client to burn CPU
without actually sending us anything.
So we still do buffering ourselves and play some tricks with
timers re-enable readability.
|
|
Long responses may leave the buffer momentarily empty,
but we must prevent read events from firing at that point.
|
|
Lynx seems to rely on this behavior for "ARTICLE <message-id>"
Tested with Lynx Version 2.8.8dev.12 (22 Feb 2012) on Debian wheezy.
|
|
This will allow us to redirect stdout/stderr more easily
for logging.
|
|
Article number is optional, but we need to update the
article number of the client connection if it was specified
(but not if it was given a Message-ID) as stipulated by
RFC 977
|
|
It's misleading to show short times on long responses.
Instead, show the long response as a separate entry on
completion or failure.
|
|
This may be helpful for sorting out duplicates.
|
|
Using Xapian allows us to implement XROVER without forking
new processes.
|
|
We can use our msgmap database to implement "XHDR Message-ID [range]"
commands quickly. This helps immensely with slrnpull, which prefers
XHDR to LISTGROUP for some reason...
|
|
XOVER uses the current article if no range is given as
stipulated in RFC 2980.
|
|
Validate commands to make sure we do not accidentally screw up
some things or leave out some argument checking.
|
|
LISTGROUP can be expensive for giant groups, too. Use the
long response API to improve fairness and prevent excessive
buffering.
|
|
XOVER, NEWNEWS, XHDR responses may be arbitrarily long and cause
memory usage via buffering. This problem would exist even if we
were to optimize them by caching headers for fast retrieval and
search.
Introduce a generic API to handle all of these commands fairly
without triggering excessive buffering and unfairness of the
existing implementation.
Generating these responses is still expensive for now, but at least
the implementation is fair to other clients and prevents one client
from using one of these commands to monopolize resources away from
other clients.
|
|
It is shorter code and probably faster than checking an
array.
|
|
Implementing NEWNEWS, XHDR, XOVER efficiently will require
additional caching on top of msgmap.
This seems to work with lynx and slrnpull, haven't tried clients.
DO NOT run in production, yet, denial-of-service vulnerabilities
await!
|