about summary refs log tree commit homepage
path: root/lib/PublicInbox/NNTP.pm
DateCommit message (Collapse)
2015-09-20nntp: add "newsgroup" parameter and sort grouplist
Using non-hierarchical mailing list names for newsgroups might confuse traditional newsreader software and perhaps some humans. Allow administrators to configure newsgroups names and hierarchies to their liking. Sorting the grouplist alphabetically should probably be done anyways to improve usability for some clients which won't sort themselves.
2015-09-20nntp: fix handling of trickled responses
We cannot use the push_back_read functionality of Danga::Socket since it will trigger event_read on buffered data. This would allow a malicious (or badly implemented) client to burn CPU without actually sending us anything. So we still do buffering ourselves and play some tricks with timers re-enable readability.
2015-09-20nntp: do not re-enable reads during long responses
Long responses may leave the buffer momentarily empty, but we must prevent read events from firing at that point.
2015-09-19nntp: article lookups by Message-ID may cross newsgroups
Lynx seems to rely on this behavior for "ARTICLE <message-id>" Tested with Lynx Version 2.8.8dev.12 (22 Feb 2012) on Debian wheezy.
2015-09-19nntp: log to FDs given by the Nntpd module
This will allow us to redirect stdout/stderr more easily for logging.
2015-09-19nntp: fix ARTICLE/HEAD/BODY/STAT
Article number is optional, but we need to update the article number of the client connection if it was specified (but not if it was given a Message-ID) as stipulated by RFC 977
2015-09-19nntp: fix logging of long responses
It's misleading to show short times on long responses. Instead, show the long response as a separate entry on completion or failure.
2015-09-19nntp: implement XPATH
This may be helpful for sorting out duplicates.
2015-09-19nntp: implement XROVER, speed up XHDR for some cases
Using Xapian allows us to implement XROVER without forking new processes.
2015-09-19nntp: speed up XHDR for the Message-ID case
We can use our msgmap database to implement "XHDR Message-ID [range]" commands quickly. This helps immensely with slrnpull, which prefers XHDR to LISTGROUP for some reason...
2015-09-19nntp: XOVER does not require range
XOVER uses the current article if no range is given as stipulated in RFC 2980.
2015-09-19nntp: implement command argument checking
Validate commands to make sure we do not accidentally screw up some things or leave out some argument checking.
2015-09-19nntp: use long response API for LISTGROUP
LISTGROUP can be expensive for giant groups, too. Use the long response API to improve fairness and prevent excessive buffering.
2015-09-19nntp: introduce long response API for streaming
XOVER, NEWNEWS, XHDR responses may be arbitrarily long and cause memory usage via buffering. This problem would exist even if we were to optimize them by caching headers for fast retrieval and search. Introduce a generic API to handle all of these commands fairly without triggering excessive buffering and unfairness of the existing implementation. Generating these responses is still expensive for now, but at least the implementation is fair to other clients and prevents one client from using one of these commands to monopolize resources away from other clients.
2015-09-19nntp: use write_buf_size instead write_buf
It is shorter code and probably faster than checking an array.
2015-09-18read-only NNTP server
Implementing NEWNEWS, XHDR, XOVER efficiently will require additional caching on top of msgmap. This seems to work with lynx and slrnpull, haven't tried clients. DO NOT run in production, yet, denial-of-service vulnerabilities await!