about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2017-01-10introduce PublicInbox::MIME wrapper class
This should fix problems with multipart messages where text/plain parts lack a header. cf. git clone --mirror https://github.com/rjbs/Email-MIME.git refs/pull/28/head In the future, we may still introduce as streaming interface to reduce memory usage on large emails.
2017-01-07inbox: properly register cleanup timer for git processes
We still need to cleanup git processes occasionally, since "git cat-file --batch" does not release old packs (and git processes are fairly expensive). For SQLite and Xapian file handles, they should be capable of managing themselves without too much trouble, so lets try keeping them for the lifetime of a process.
2017-01-07search: remove subject_summary
Apparently it never actually got used, and the world seems fine without it, so we can drop it. While we're at it, consider removing our subject_path usage from existence, too. We are not using fancy subject-line based URLs, here.
2017-01-07searchmsg: favor direct hash access over accessor methods
This is faster, smaller, and more straighforward to me with fewer layers of indirection.
2017-01-07remove incorrect comment about strftime + locales
We only need strftime to be locale-independent when generating dates for email and HTTP headers. Purely numeric dates can use strftime for ease-of-readability.
2017-01-07config: allow per-inbox nntpserver
This allows certain inboxes to override the global nntpserver (perhaps under a different domain).
2017-01-07inbox: eliminate weaken usage entirely
We can do a better job initializing the data structure so we no longer need to rely on weak references to cleanup when we ditch the config on reload.
2017-01-07inbox: describe the full key name
Hopefully make this easier for future generations to understand.
2017-01-07config: remove unused get() method
This seems like an unnecessary abstraction, or an abstraction on the wrong level.
2017-01-07config: always use namespaced "publicinboxlimiter"
I'm not sure if we'll ever support sharing a config file with other tools, but maybe we will, and "limiter" is too generic.
2017-01-07qspawn: prepare to support runtime reloading of Limiter
We may allow the {max} value of a limiter to be changed in the future, so lets start accounting for it before we spawn followup processes.
2017-01-04http: remove weaken usage, reduce anonsub capture scope
Avoiding weaken here is no more dangerous than the existing circular refs (e.g. psgix.io) we create and manage throughout the lifetime of the connection. So, trust ourselves to maintain the data structure properly and avoid triggering extra memory usage. While we're at it, avoid having anonymous subroutines capture more variables than necessary to simplify reference auditing.
2017-01-04httpd/async: remove weaken usage
We do not need to use weaken() here, so avoid it to simplify our interactions with Perl; as weaken requires additional storage and (it seems) time complexity.
2017-01-04http: fix spelling error
Oops. And we'll be fixing circular references from now...
2017-01-02watch: watchspam affects all configured inboxes
If a message is spam in one mailbox, it is spam in all others a particular user/group will care about.
2016-12-26evcleanup: ensure deferred close from timers are handled ASAP
Danga::Socket defers close() syscalls until the end of the event loop to avoid FD recycling. Unfortunately, this is dependent on IO events firing and waking the process up from poll/kevent/epoll_wait. Without any I/O activity, a socket could remain in the @Danga::Socket::ToClose array indefinitely. Thus, we will trigger a fake IO event after running all timers to trigger the deferred close in Danga::Socket::PostEventLoop.
2016-12-25httpd/async: improve variable naming
We only refer to PublicInbox::HTTP objects here, so '$io' was a bad name.
2016-12-25githttpbackend: minor cleanups to improve readability
Fewer returns improves readability and the diffstat agrees.
2016-12-25githttpbackend: simplify compatibility code
Fewer conditionals means theres fewer code paths to test and makes things easier-to-read.
2016-12-25githttpbackend: minor readability improvement
Use a more meaningful variable name for the Qspawn object, since this module is the reference for its use.
2016-12-25http: fix clobbering of $null_io
Oops, this would be disatrous if we started handling bigger request bodies or slow clients. Fixes: c008654229a9 ("avoid IO::File for anonymous temporary files")
2016-12-24linkify: modify argument in place
This results in over 1% speedup doing $MESSAGE_ID/T/ HTML generation for a 368-message thread.
2016-12-24view: do not modify array during iteration
This results in a half percent speedup or so doing $MESSAGE_ID/T/ HTML generation for a 368 message thread.
2016-12-24view: stop chomping off whitespace at ends of messages
This allows a 3-4% speedup in $MESSAGE_ID/T/ page generation speed for a 368+ message thread. It also more faithfully preserves the message as intended; even if the it makes the sender look like a space-wasting slob :P
2016-12-24view: remove unused parameter
And add a comment about it to remind our future selves.
2016-12-22search: lookup_mail handles modified DBs
We call lookup_mail all over the place, be sure we can handle database modifications in those cases.
2016-12-22doc: various comments on async handling
Notes for future developers (myself included) since we can't assume people can read my mind.
2016-12-21searchthread: simplify API and remove needless OO
This simplifies callers to prevent errors and avoids needless object-orientation in favor of a single procedure call to handle threading and ordering.
2016-12-21searchthread: update comment about loop prevention
It definitely is necessary to prevent looping with the %seen hash.
2016-12-20searchmsg: remove ensure_metadata
Instead, only preload the ->mid field for threading, as we only need ->thread and ->path once in Search->get_thread (but we will need the ->mid field repeatedly). This more than doubles View->load_results performance on according to thread-all on an inbox with over 300K messages.
2016-12-17searchmsg: do not memoize {date} field
We only generate the ->date once in NNTP, so creating the hash entry is a waste.
2016-12-17searchmsg: remove locale-dependency for ->date
strftime is locale-dependent, which can cause surprising failures for some users.
2016-12-17wwwtext: link to RFC4685 (Atom Threading)
This should give this feature some more visibility.
2016-12-17atom: implement message threading per RFC 4685
This will allows certain feed readers to render a message thread as described in <https://www.jwz.org/doc/threading.html>. Feed readers with knowledge of of RFC 4685 are unknown to us at this time, but perhaps this will encourage future implementations. Existing feed readers I've tested (newsbeuter, feed2imap) seem to ignore these tags gracefully without degradation.
2016-12-17feed: support publicinbox.<name>.feedmax
This allows users to customize by using smaller or larger Atom feeds than the default value of 25 entries.
2016-12-14wwwtext: remove outdated comment
I originally envisioned wwwtext being more flexible and able to serve arbitrary blobs; but at this point I consider it redundant and public-inbox is not wiki software.
2016-12-13searchmsg: remove unused EPOCH_822 constant
This hasn't been needed since our Email::Abstract removal for message threading.
2016-12-13nntp: avoid useless use of strftime
There's no need to use strftime if we'll be converting the date by hand, anyways.
2016-12-12daemon: set $now time for NNTP shutdown
commit 6e238ee3396719e578d6a90e177a71ce9f8c1ca0 ("nntp: respect 3 minute idle time for shutdown") was incomplete, and needed this change to Daemon to be effective. In the future, there will be more common code between NNTP.pm and HTTP.pm
2016-12-10search: retry document loading from Xapian
In addition to needing to retry enquire queries, we also need to protect document loading from the Xapian DB and retry on modification, as it seems to throw the same errors. Checking the $@ ref for Search::Xapian::DatabaseModifiedError is actually in the test suite for both the XS and SWIG Xapian bindings, so we should be good as far as forward/backwards compatibility.
2016-12-10search: always sort thread results in ascending time order
This makes life easier for the threading algorithm, as we can use the implied ordering of timestamps to avoid temporary ghosts and resulting container vivication. This would've also allowed us to hide the bug (in most cases) fixed by the patch titled "thread: last Reference always wins", in case that needs to be reverted due to infinite looping.
2016-12-10thread: last Reference always wins
Since we use SearchMsg from Xapian data, we can be assured we do not get self-referential {references} field. However, we may need to be more careful when checking has_descendent for loops, as blindly calling add_child could open us up to that possibility...
2016-12-10view: skip ghosts with no direct children
Otherwise, a malicious or broken client could populate the thread skeleton with invalid References. We only care about ghosts which messages correctly refer to, not totally bogus ones which may be the result of long line or token truncation + wrapping in MUA headers.
2016-12-10view: reduce indentation for skeleton generation
This should reduce the number of subroutine calls needed for the common case of real (non-ghost) messages as well as shortening code.
2016-12-10thread: fix comment describing its existence
Mail::Thread is UNavailable on many distros, meaning ordinary users will have to rely on CPAN, a Perl-specific packaging tool.
2016-12-10view: favor SearchMsg for In-Reply-To over Email::MIME
This should avoid warnings during thread skeleton generation if ever the Xapian database disagrees with View.pm about which is the proper direct parent of a message. We will treat the data in Xapian as the truth (if Xapian is available).
2016-12-10search: favor In-Reply-To over last References iff IRT exists
Some email clients set the References headers backwards, so trust the In-Reply-To header if (and only if) it exists and is parseable as direct parent of the current message. For affected repos, this will require reindexing (via "public-inbox-index --reindex"), but there will be no version bump for this bugfix.
2016-12-06linkify: implement Markdown link compatibility (again)
Although unescaped parentheses in URLs are technically allowed, they are uncommon. However, Markdown-like syntaxes are unfortunately common for URLs, so we might as well support them. This fixes parentheses detection at sentence endings, as seen in practice on emails.
2016-12-06Revert "linkify: implement Markdown link compatibility"
This reverts commit 130d0c4e33c5c73dc69e270fc698735d49e0f159.
2016-12-06linkify: implement Markdown link compatibility
Although unescaped parentheses in URLs are technically allowed, they are uncommon. However, Markdown-like syntaxes are unfortunately common for URLs, so we might as well support them.