about summary refs log tree commit homepage
path: root/lib/PublicInbox
DateCommit message (Collapse)
2016-04-02www: various style changes and comment updates
Reduce stack depth of arguments and rely more on state hashref to store response state. We may end up shoving everything in ctx eventually.
2016-03-17daemon: expand @ARGV paths for running in '/'
We also require --stdout/--stderr/--pid-file to be absolute paths for USR2 usage. However, allow PSGI files for -httpd to be relative paths for ease-of-use.
2016-03-12feed: fix brain farts in new_oneline removal
Ugh... Fixes: 476fc666c223 (reduce "PublicInbox::Hval->new_oneline" use)
2016-03-12searchmsg: preserve hard tabs, but drop CR (\r)
Hard tabs *may* be searchable, so preserve them since they do not take up any more space than a normal space. However, CR (carriage return) is worthless and likely a sign of a buggy mail (or spam) client anyways.
2016-03-12reduce "PublicInbox::Hval->new_oneline" use
It's probably a bad idea to strip extraneous whitespace from some headers as an extra space may convey useful information. Newlines don't seem to be preserved by Email::MIME or Email::Simple anyways, so there's no danger in breaking formatting.
2016-03-12http: use Plack::HTTPParser for HTTP parsing
This allows us to reduce installation dependencies while retaining performance as it favors HTTP::Parser::XS when it is installed and available. PLACK_HTTP_PARSER_PP may be set to 1 to force a pure Perl parser for testing.
2016-03-12http: prevent zero-byte writes
Plack::Middleware::Deflater (and perhaps other middleware) triggers zero-byte writes which wastes syscalls when they get passed to Danga::Socket. This may also trigger problems when we introduce TLS support in the future.
2016-03-11daemon: fixup usage of the '-l' switch with IP/INET6 sockets
We need to ensure $sock_pkg is preserved outside of the loop. The variable passed to "for" or "foreach" is implicitly local and restores the previous value when the loop exits. This is documented in the perlsyn manpage in the "Foreach Loops" section. Fixes: ea1b6cbd422b ("daemon: allow using IO::Socket::IP over INET6")
2016-03-08daemon: allow using IO::Socket::IP over INET6
IO::Socket::IP is bundled with newer versions of Perl, so it is more likely to be available. There should be no differences between these with our use cases.
2016-03-06http: reject excessively large HTTP request bodies
We cannot risk using all of a users' disk space buffering gigantic requests. Use the defaults git gives us since we primarily host git repositories.
2016-03-06http: ensure errors are printable before PSGI env
We cannot rely on a client socket having a PSGI env before headers are fully-parsed as we seek to avoid storing hashes for idle clients. Sso print errors to the psgi.errors value which belongs to the httpd listener, instead.
2016-03-06http: reject excessive headers
HTTP::Parser::XS::PP does not reject excessively large headers like the XS version. Ensure we reject headers over 16K since public-inbox should never need such large request headers.
2016-03-05daemon: sockname detects listeners correctly
This means we can avoid false-positives when inheriting multiple Unix domain sockets.
2016-03-05daemon: document optional Net::Server dependency
Non-socket activation users will want to install Net::Server for daemonization, pid file writing, and user/group switching.
2016-03-05feed: remove unnecessary encoding lookup
We handle encoding-related things elsewhere.
2016-03-05daemon: simplify parent death handling
No need to create a new sub which kill ourselves $$ when we can invoke worker_quit directly.
2016-03-05daemon: avoid cyclic references for once-used callbacks
Not that these subs are repeatedly created, but this makes the code easier-to-review and these callbacks are idempotent anyways.
2016-03-05daemon: drop listener sockets ASAP on termination
We do not want to be accepting connections during graceful shutdown because another new process is likely taking over. This also allows us to free up the listener case another (independent) process wants to claim it.
2016-03-05git-http-backend: favor sysread for regular files
We do not need line buffering, here; so favor sysread to bypass extra copies which may be done by normal read.
2016-03-04daemon: simplify socket inheriting, slightly
IO::Handle->new_from_fd has existed since at least 1996, so it should be safe to depend on at this point.
2016-03-04daemon: support listening on Unix domain sockets
Listening on Unix domain sockets can be convenient for running behind reverse proxies, avoiding port conflicts, limiting access, or avoiding the overhead (if any) of TCP over loopback.
2016-03-03daemon: introduce host_with_port for identifying sockets
This allows us to share more code between daemons and avoids having to make additional syscalls for preparing REMOTE_HOST and REMOTE_PORT in the PSGI env in -httpd. This will also make supporting HTTP (and NNTP) over Unix sockets easier in a future commit.
2016-03-03daemon: avoid polluting the main package
We've distilled the daemon code into one public function ("run"), so avoid polluting the main namespace and just have users prefix with the full package name for this rarely-used class.
2016-03-03view: fix stupid typo in inline_dump
Ugh, this enabled-iff-xapian-is-available code really needs better testing...
2016-03-03use raw header for Message-ID
Message-IDs should not be MIME encoded, but in case they are, use the raw form for compatibility with ssoma and possibly other tools. This prevents a potential problem where a malicious client could confuse our storage layer into indexing incorrect contents.
2016-03-01http: better error handling for EMFILE/ENFILE
Better to throw the error back to the client ASAP if we're out-of-descriptors. We will need to implement idle client expiration for long-lived HTTP connections.
2016-03-01httpd: document pi-httpd.async as totally unstable
We'll have to use it some more before deciding it is a public interface. I do hope for it to be a usable public interface one day for other users.
2016-03-01processpipe: preserve native close behavior
We need to ensure close on handles tied to this class get the same errors a normal "close" in Perl gets.
2016-03-01linkify: do not capture trailing '.' or ';' in URLs
It seems common for users to end statements with URLs, while it is rare for a URL itself to end with a '.' or ';'. So make a guess and assume the URL was intended to not include the trailing '.' or ';'
2016-03-01extract linkification code to a separate package
This will allow us to more easily reuse it elsewhere.
2016-03-01view: consolidate whitespace stripping from messages
We now keep intermediate blank lines in messages, since it could be used to denote logical gaps in the message (such as giving readers a chance to opt out of "spoiler" information). However leading blank lines, trailing blank lines, and trailing whitespace have no useful value we can discern; so drop those entirely to prevent clients from eating up vertical whitespace.
2016-03-01view: do not hide patches or signatures
It's often not that much information and may be useful to reduce HTTP requests a reader will want to make.
2016-02-29fixup Plack-related requires
We do not need to load Plack::Request outside of WWW anymore.
2016-02-29git-http-backend: fixes for mod_perl
Apache2 mod_perl does not give us a real file handle, so we must translate that before giving that to git-http-backend(1). Also, parse the Status: correctly for errors since we failed to set %ENV properly before the previous fix for SpawnPP
2016-02-29spawnpp: use env(1) for mod_perl compatibility
We cannot modify %ENV directly under mod_perl (even after forking!), so use env(1) instead to pass the environment.
2016-02-29git-http-backend: stricter parsing of CRLF
It is not needed as we know git uses CRLF termination.
2016-02-29favor procedural calls for most private functions
This makes for better compile-time checking and also helps document which calls are private for HTTP and NNTP. While we're at it, use IO::Handle::* functions procedurally, too, since we know we're working with native glob handles.
2016-02-29distinguish error messages intended for users vs developers
For error messages intended to show user error (e.g. giving invalid options), we add a newline ("\n") at the end to polluting the output with location information. However, for diagnosing non-user-triggered errors, we should show the location of where the error occured.
2016-02-29http: avoid needless time2str calls
Checking the time is nearly free on modern systems with vDSO/vsyscall/similar while sprintf is always expensive.
2016-02-29http: document event_write usage
It may not be obvious where we are when we enter the event_write callback. Hopefully this clarifies things.
2016-02-29http: error check for sysseek on input
Just in case we screwed up somewhere, we need to match up syswrite to sysseek and we also favor procedural calls for native types.
2016-02-28spawnpp: die instead of exit on exec failure
Perl may complain about exit not being executed, but not die.
2016-02-28newswww: set Content-Type properly
Oops :x
2016-02-28http: support graceful shutdown like nntp
HTTP responses may be long-running or requests may be slow or pipelined. Ensure we don't kill them off prematurely.
2016-02-28reduce calls to close unless error checks are needed
We can rely on timely auto-destruction based on reference counting; reducing the chance of redundant close(2) calls which may hit the wront FD. We do care about certain close calls (e.g. writing to a buffered IO handle) if we require error-checking for write-integrity. In other cases, let things go out-of-scope so it can be freed automatically after use.
2016-02-28searchidx: use defined for checking EOF behavior
While empty or "0" should never appear, this allows the reviewer to think and know less about the context in which this check is done.
2016-02-28searchmsg: update + fix license header
Not sure how, but this should've always been AGPL-3.0+ like the rest of the code, not GPL-3.0+
2016-02-28spawn: disable popen optimization for non-vfork
This is necessary since we want to be able to do arbitrary redirects via the popen interface. Oh well, we'll be a little slower for now for users without vfork. vfork users will get all the performance benefits.
2016-02-28spawnpp: fix error message for stderr redirect failing
Oops :x
2016-02-27daemon: refresh before forking
This means we always load the PSGI server code early for -httpd. This may make things less compatible with existing PSGI/Plack apps, but we prioritize our httpd for the uses of public-inbox itself, first. And any existing PSGI/Plack app which wants to may adapt themselves to being preload-friendly.