about summary refs log tree commit homepage
path: root/lib/PublicInbox/NNTP.pm
DateCommit message (Collapse)
2016-12-13nntp: avoid useless use of strftime
There's no need to use strftime if we'll be converting the date by hand, anyways.
2016-09-09nntp: cleanup: move use statements out of sub scope
This clarifies the code somewhat, and we don't care to lazy-load in NNTP.pm anyways since this is only used for a long-lived daemon.
2016-08-14www: do not unecessarily escape some chars in paths
Based on reading RFC 3986, it seems '@', ':', '!', '$', '&', "'", '; '(', ')', '*', '+', ',', ';', '=' are all allowed in path-absolute where we have the Message-ID. In any case, it seems '@' is fairly common in path components nowadays and too common in Message-IDs.
2016-07-27localize $/ when using chomp
Callers may have localized $/ to something else, so make sure we chomp the expected character(s) when calling chomp.
2016-07-09nntp: return if a client drops on us
Danga::Socket::write will set the closed flag on a socket, automatically, and we do not need to bring down an entire server when one client breaks the connection :P
2016-07-07inbox: cleanup and consolidate object weakening
This fixes some layering violations and consolidates the cleanup into the inbox object itself. Keeping in mind weakening does not work at all without our PSGI server.
2016-07-02nntp: respect 3 minute idle time for shutdown
This avoids breaking clients on graceful shutdown since NNTP responses should usually be quick.
2016-07-02nntp: simplify update_idle_time
This ought to make things easier when we add TLS support.
2016-06-20nntp: use lookup_mail instead of lookup_message
lookup_mail is safer since it won't inadvertently load ghosts.
2016-06-20feed: various object-orientation cleanups
Favor Inbox objects as our primary source of truth to simplify our code. This increases our coupling with PSGI to make it easier to write tests in the future. A lot of this code was originally designed to be usable standalone without PSGI or CGI at all; but that might increase development effort.
2016-06-14nntp: do not double-encode UTF-8 body
Or whatever the appropriate Perl terminology, is... And we will need to do something appropriate for other encodings, too. I still barely understand Perl Unicode despite attempting to understand the docs over the years..
2016-05-30use utf8::{encode,decode} for in-place transforms
No need to duplicate the string when transforming it; learned from studying SpamAssassin 3.4.1
2016-05-29nntp: fix for missing articles/bodies/heads
Oops, we totally forgot to automate testing for this :x
2016-05-28remove redundant NewsGroup class
Most of its functionality is in the PublicInbox::Inbox class. While we're at it, we no longer auto-create newsgroup names based on the inbox name, since newsgroup names probably deserve some thought when it comes to hierarchy.
2016-05-24standardize timer-related event-loop code
Standardize the code we have in place to avoid creating too many timer objects. We do not need exact timers for things that don't need to be run ASAP, so we can play things fast and loose to avoid wasting power with unnecessary wakeups. We only need two classes of timers: * asap - run this on the next loop tick, after operating on @Danga::Socket::ToClose to close remaining sockets * later - run at some point in the future. It could be as soon as immediately (like "asap"), and as late as 60s into the future. In the future, we support an "emergency" switch to fire "later" timers immediately.
2016-05-18nntpd: reject control characters entirely
There's no place for them in the commands and we don't take messages; potentially printing them into a log opened in a terminal is too dangerous. Hoist out read_til_dot in the test while we're at it.
2016-05-14nntp: use "newsgroup" instead of "name"
This reduces the cognitive overhead for mapping names of configuration values to internal field names of our classes. Further changes along these lines coming...
2016-05-13nntp: fixup "Wide character" warnings
We need Perl to believe everything we send is UTF-8, make it so, even if it may not be. Fixes: 265e79ff82ce 'Revert "nntp: proper UTF-8 support (hopefully?)"'
2016-05-13Revert "nntp: proper UTF-8 support (hopefully?)"
This reverts commit f81ad477cb013d05b9b11fa051a9ebc5983a5be6. The raw, undecoded body is probably what should be sent over the wire anyways for clients to deal with. We'll need this to avoid deprecation warnings with Perl 5.24+ since we use send()/recv()/sysread().
2016-05-02nntp: append Archived-At and List-Archive headers
For readers using NNTP, we should do our best to advertise the clonable HTTP/HTTPS URLs and the message permalink URL for ease-of-referencing messages, since we don't want the NNTP server and it's sequential article numbers to be relied on.
2016-05-01daemon: reduce timer-related allocations
We can reduce the allocation and overhead needed for Danga::Socket timers for immediately-executed responses by combining identical timers and reducing anonymous sub creation.
2016-04-25nntp: reduce timers for weakening
Danga::Socket timers are not cheap, so avoid creating up to 3 timers per-newsgroup by batching resource weakening. This lets us reduce resource consumption for scheduing additional resource consumption reduction :)
2016-04-25nntp: remove unused hdr_val subroutine
hdr_val has not been used since commit 1d236e649df1 ("nntp: implement OVER/XOVER summary in search document")
2016-02-29favor procedural calls for most private functions
This makes for better compile-time checking and also helps document which calls are private for HTTP and NNTP. While we're at it, use IO::Handle::* functions procedurally, too, since we know we're working with native glob handles.
2016-02-29distinguish error messages intended for users vs developers
For error messages intended to show user error (e.g. giving invalid options), we add a newline ("\n") at the end to polluting the output with location information. However, for diagnosing non-user-triggered errors, we should show the location of where the error occured.
2016-02-28http: support graceful shutdown like nntp
HTTP responses may be long-running or requests may be slow or pipelined. Ensure we don't kill them off prematurely.
2015-12-22rename 'GitCatFile' package to 'Git'
We'll be using it for more than just cat-file. Adding a `popen' API for internal use allows us to save a bunch of code in other places.
2015-11-20various internal documentation updates
Hopefully this gives new hackers a better overview of how the components relate to each other.
2015-11-18nntp: fix printf warnings
Error messages and request lines may contain '%' which would throw off Perl printf.
2015-10-01nntp: better delimit error message
It may be hard to tell what command triggered an error, otherwise.
2015-10-01nntp: remove reference to non-existent function
Oops
2015-10-01nntp: implement idle time expiration
We don't want to waste precious server memory on clients which have been idle for longer than 3 minutes.
2015-09-30nntp: implement OVER/XOVER summary in search document
The document data of a search message already contains a good chunk of the information needed to respond to OVER/XOVER commands quickly. Expand on that and use the document data to implement OVER/XOVER quickly. This adds a dependency on Xapian being available for nntpd usage, but is probably alright since nntpd is esoteric enough that anybody willing to run nntpd will also want search functionality offered by Xapian. This also speeds up XHDR/HDR with the To: and Cc: headers and :bytes/:lines article metadata used by some clients for header displays and marking messages as read/unread.
2015-09-30nntp: implement LIST HEADERS
We shall remove slow, unoptimized headers in XHDR/HDR to avoid becoming an easy DoS target.
2015-09-30nntp: MODE READER denies posting
Redundantly confirm to clients we do not accept posting with the MODE READER command. ref: RFC 3977 5.3.1
2015-09-30nntp: reduce syscalls for LIST OVERVIEW.FMT
No point in sending such a short, bounded response with multiple syscalls.
2015-09-30nntp: HDR returns 225, not 224
This is stipulated by RFC 3977 8.5.1, but apparently I misread it.
2015-09-25nntp: avoid signals for long responses
Using a signal-based timer can hurt throughput on a machine that's overloaded. Ensure there's always forward progress and reduce the number of syscalls we make, too.
2015-09-25nntp: prefix FD on every log line
This can help us track down what request patterns clients will perform when we have multiple clients.
2015-09-25nntp: do not repeat result on blocked write
Oops, we must increment our range even if we yield or get blocked on output buffering.
2015-09-25nntp: consistently use 501 for unsupported LIST
This is required by RFC 3977, section 3.2.1
2015-09-25nntp: HDR allows metadata prefixed with ':'
RFC 3977, section 8.5.2 states metadata lookups can be done with HDR.
2015-09-24nntp: avoid infinite loop on partial read
Oops :x
2015-09-24nntpd: move busy check to NNTP package
This is better encapsulated and hopefully more readable. While we're at it, check for being inside a long response, too.
2015-09-24nntp: fix XOVER command
Oops, we need to test commands more closely :x Add a missing prototype while we're at it for extra checking.
2015-09-23nntp: support HDR command from RFC 3977
This is similar to XHDR, but differs in how it handles Message-ID lookups.
2015-09-22nntp: XHDR lookups by Message-ID may cross groups
This is allowed by RFC 2980 and HDR (to-be-implemented) in RFC 3977 supports it, too.
2015-09-22nntp: XHDR fixes for Message-ID lookups
We'll require some modifications for HDR support, though.
2015-09-22nntp: implement OVER from RFC 3977
This is just like the XOVER command, but allows a single Message-ID to be given.
2015-09-22nntp: avoid setting Bytes and Lines headers
These are internal metadata should be calculated, so avoid leaking it into the header.