about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2016-06-17import: auto-update index when done
This prevents multiple update processes from stepping over each other while called under the lock, and also allows the new -watch process to update the index iff indexing was desired.
2016-06-17watch: quiet down rejected header matches
People may use this directive because they prefer to merge several mailing lists into one local mailbox, so there may be many messages and we should not needlessly clutter logs for this.
2016-06-17address: no commas in email addresses
We only do loose parsing, here, and I don't think I've seen a comma in a valid email address, so lets not support them.
2016-06-17search: increase limit for thread search
Some threads are easily over 100 messages, so the 50 limit is not enough. It is likely that 1000 messages is not enough, either, and we will need to tune our threading to handle more messages and supply options for configurability.
2016-06-17watch: introduce watch directive
This will allow users to run importers off existing mail accounts where they may not have access to run -mda. Currently, we only support Maildirs, but IMAP ought to be doable.
2016-06-17filter: split out scrub method from delivery
We will scrub for importing archives, so ensure it is usable outside of the delivery routine.
2016-06-17searchidx: disable Email::MIME::ContentType::STRICT_PARAMS
Disable this since we handle imperfect data from an imperfect world.
2016-06-17msg_iter: support read-only elements
Apparently, it's possible to have read-only bodies in Email::MIME objects. Haven't gotten a chance to reliably reproduce it, though...
2016-06-15unsubscribe: archive_url may be undefined
We'll show a nasty warning in the UI instead of triggering a perl warning about an undefined variable.
2016-06-15inbox: allow undef return value for base_url
It should be possible to serve the contents of a public-inbox over NNTP but not HTTP.
2016-06-15mda: hook up new filter functionality
This removes the Email::Filter dependency as well as the signature-breaking scrubber code. We now prefer to reject unacceptable messages and grudgingly (and blindly) mirror messages we're not the primary endpoint for.
2016-06-15emergency: implement new emergency Maildir delivery
This is transactional and hopefully safer in case we hit SIGSEGV or SIGKILL during processing, as the tmp/ copy will remain on the FS even if DESTROY/END handlers are not called.
2016-06-15filter: begin work on a new filter API
This filter API should be independent of Email::Filter and hopefully less intrusive to long running processes.
2016-06-15mda: precheck no longer depends on Email::Filter
Email::Filter doesn't offer any functionality we need, here; and our dependency on Email::Filter will gradually be removed since it (and Email::LocalDelivery) seem abandoned and we can have more-fine-grained control by rolling our own Maildir delivery which can work transactionally.
2016-06-15drop dependency on File::Path::Expand
We still pull it in via Email::LocalDelivery, but that dependency will go away, soon.
2016-06-14nntp: do not double-encode UTF-8 body
Or whatever the appropriate Perl terminology, is... And we will need to do something appropriate for other encodings, too. I still barely understand Perl Unicode despite attempting to understand the docs over the years..
2016-06-13view: msg_html uses getline body to reduce latency
We need to ensure we show the message body ASAP since the thread generation via Xapian could take a while and maybe even raise an exception or crash.
2016-06-11daemon: reset unused signal handlers to default in child
They're effectively noops anyways, and we don't want to be holding a reference to the read end of the parent pipe.
2016-06-10unsubscribe: HTML encode undecryptable username
Otherwise, URLs can be crafted to inject HTML.
2016-06-07unsubscribe: fix off-by-one error
Oops, pesky users of single-character email addresses!
2016-06-07Merge branch 'unsubscribe'
* unsubscribe: unsubscribe.milter: use default postfork dispatcher unsubscribe: prevent decrypt from showing random crap examples/unsubscribe-psgi@.service: disable worker processes unsubscribe: bad URL fixup unsubscribe: get off mah lawn^H^H^Hist
2016-06-07view: be sure reply text describes plain-text
While we may end up mirroring lists which allow HTML mail, encourage plain-text for compatibility since all current inboxes we host are text-only.
2016-06-07view: remove trailing whitespace from reply command
Oops, needless waste of space.
2016-06-07view: escape From name properly for title
Oops :x Add an additional test for live data for any unprintable characters, too, since this could be a dangerous source of HTML injection.
2016-06-05view: inline message reply into message view
This should reduce link following for replies and improve visibility. This should also reduce cache overhead/footprint for crawlers.
2016-06-02www: force two element key-value pairs in query
Oops, this quiets down a warning seen in logs.
2016-05-30use utf8::{encode,decode} for in-place transforms
No need to duplicate the string when transforming it; learned from studying SpamAssassin 3.4.1
2016-05-30http: yield body->getline running time
We cannot let a client monopolize the single-threaded server even if it can drain the socket buffer faster than we can emit data. While we're at it, acknowledge the this behavior (which happens naturally) in httpd/async. The same idea is present in NNTP for the long_response code. This is the HTTP followup to: commit 0d0fde0bff97 ("nntp: introduce long response API for streaming") commit 79d8bfedcdd2 ("nntp: avoid signals for long responses")
2016-05-30www: remove a few more Plack::Request dependencies
Still a work in progress, but SearchView no longer depends on Plack::Request at all and Feed is getting there. We now parse all query parameters up front, but we may do that lazily again in the future.
2016-05-30www: remove gratuitous use of Plack::Request methods
Accessing $env directly is faster and we will eventually remove all Plack::Request dependencies.
2016-05-30git-http-backend: remove dependency on Plack::Request
Plack::Request is unnecessary overhead for this given the strictness of git-http-backend. Furthermore, having to make commit 311c2adc8c63 ("avoid Plack::Request parsing body") to avoid tempfiles should not have been necessary.
2016-05-29nntp: fix for missing articles/bodies/heads
Oops, we totally forgot to automate testing for this :x
2016-05-29inbox: drop references ASAP for search and msgmap
We can't leave them lingering in the parent process at all due to the risk of corruption with multiple processes.
2016-05-29searchmsg: all timestamps stored in Xapian are UTC
We cannot have strftime using the local timezone for %z. This fixes output when a server is not running UTC.
2016-05-28remove redundant NewsGroup class
Most of its functionality is in the PublicInbox::Inbox class. While we're at it, we no longer auto-create newsgroup names based on the inbox name, since newsgroup names probably deserve some thought when it comes to hierarchy.
2016-05-28config: remove try_cat
It's moved into the Inbox module and we no longer use it in WWW
2016-05-28www: remove footer_html support
I haven't used it in a while and the existing "description" is probably good enough. If we support it again, it should be plain-text + auto-linkified for ease-of-maintenance and consistency.
2016-05-28http: clarify comments about layering violation
It's a low priority, but acknowledge it.
2016-05-27config: fix NewsWWW fallback for newsgroups in HTTP URLs
Oops, added a test to prevent regressions while we're at it.
2016-05-27git-http-backend: close pipe for generic PSGI on errors
The generic PSGI code needs to avoid resource leaks if smart cloning is disabled (due to resource contraints).
2016-05-27git-http-backend: move real close to GetlineBody
This makes more sense as it keeps management of rpipe nice and neat.
2016-05-27httpd/async: do not needlessly weaken
The restart_read callback has no chance of circular reference, and weakening $self before we create it can cause $self to be undefined inside the callback (seen during stress testing). Fixes: 395406118cb2 ("httpd/async: prevent circular reference")
2016-05-27git-http-backend: fix aborts for generic PSGI clone
We need to avoid circular references in the generic PSGI layer, do it by abusing DESTROY.
2016-05-27http: avoid circular reference for getline responses
Lightly tested, this seems to work when mass-aborting responses. Will still need to automate the testing...
2016-05-27httpd/async: prevent circular reference
We must avoid circular references which can cause leaks in long-running processes. This callback is dangerous since it may never be called to properly terminate everything.
2016-05-25remove Email::Address dependency
git has stricter requirements for ident names (no '<>') which Email::Address allows. Even in 1.908, Email::Address also has an incomplete fix for CVE-2015-7686 with a DoS-able regexp for comments. Since we don't care for or need all the RFC compliance of Email::Address, avoiding it entirely may be preferable. Email::Address will still be installed as a requirement for Email::MIME, but it is only used by the Email::MIME::header_str_set which we do not use
2016-05-24git-http-backend: use qspawn to limit running processes
Having an excessive amount of git-pack-objects processes is dangerous to the health of the server. Queue up process spawning for long-running responses and serve them sequentially, instead.
2016-05-24http: fix various race conditions
We no longer override Danga::Socket::event_write and instead re-enable reads by queuing up another callback in the $close response callback. This is necessary because event_write may not be completely done writing a response, only the existing buffered data. Furthermore, the {closed} field can almost be set at any time when writing, so we must check it before acting on pipelined requests as well as during write callbacks in more().
2016-05-24standardize timer-related event-loop code
Standardize the code we have in place to avoid creating too many timer objects. We do not need exact timers for things that don't need to be run ASAP, so we can play things fast and loose to avoid wasting power with unnecessary wakeups. We only need two classes of timers: * asap - run this on the next loop tick, after operating on @Danga::Socket::ToClose to close remaining sockets * later - run at some point in the future. It could be as soon as immediately (like "asap"), and as late as 60s into the future. In the future, we support an "emergency" switch to fire "later" timers immediately.
2016-05-23http: avoid uninitialized variable
Oops, really gotta start checking logs in tests :x Fixes: bb38f0fcce739 ("http: chunk in the server, not middleware")