about summary refs log tree commit homepage
path: root/lib/PublicInbox
DateCommit message (Collapse)
2017-01-07config: remove unused get() method
This seems like an unnecessary abstraction, or an abstraction on the wrong level.
2017-01-07config: always use namespaced "publicinboxlimiter"
I'm not sure if we'll ever support sharing a config file with other tools, but maybe we will, and "limiter" is too generic.
2017-01-07qspawn: prepare to support runtime reloading of Limiter
We may allow the {max} value of a limiter to be changed in the future, so lets start accounting for it before we spawn followup processes.
2017-01-04http: remove weaken usage, reduce anonsub capture scope
Avoiding weaken here is no more dangerous than the existing circular refs (e.g. psgix.io) we create and manage throughout the lifetime of the connection. So, trust ourselves to maintain the data structure properly and avoid triggering extra memory usage. While we're at it, avoid having anonymous subroutines capture more variables than necessary to simplify reference auditing.
2017-01-04httpd/async: remove weaken usage
We do not need to use weaken() here, so avoid it to simplify our interactions with Perl; as weaken requires additional storage and (it seems) time complexity.
2017-01-04http: fix spelling error
Oops. And we'll be fixing circular references from now...
2017-01-02watch: watchspam affects all configured inboxes
If a message is spam in one mailbox, it is spam in all others a particular user/group will care about.
2017-01-01repobrowse: avoid empty pathspecs for future git compatibility
At the moment, we always set expath, so it will always be defined.
2016-12-26spawn: remove non-blocking support, here
It is never used, and inappropriate to support in generic code. HTTPD::Async already sets non-blocking, and it's better to do it in -httpd-specific code since we know our -httpd can handle it.
2016-12-26repobrowse: port git snapshot over to qspawn
This is expensive, so we will utilize the qspawn system to prevent excessive overhead.
2016-12-26repobrowse: port patch generation over to qspawn
And start generalizing the qspawn usage code for PSGI with psgi_return.
2016-12-26git: lazy initialization of error output
We may not keep this feature after all, but for now we'll hold off on creating it to cheapen instantiation.
2016-12-26Merge remote-tracking branch 'origin/master' into repobrowse
* origin/master: (25 commits) evcleanup: ensure deferred close from timers are handled ASAP httpd/async: improve variable naming githttpbackend: minor cleanups to improve readability githttpbackend: simplify compatibility code githttpbackend: minor readability improvement http: fix clobbering of $null_io linkify: modify argument in place view: do not modify array during iteration view: stop chomping off whitespace at ends of messages view: remove unused parameter search: lookup_mail handles modified DBs doc: various comments on async handling searchthread: simplify API and remove needless OO searchthread: update comment about loop prevention searchmsg: remove ensure_metadata tests: add thread-all testing for benchmarking searchmsg: do not memoize {date} field searchmsg: remove locale-dependency for ->date t/config.t: fix feedmax default wwwtext: link to RFC4685 (Atom Threading) ...
2016-12-26evcleanup: ensure deferred close from timers are handled ASAP
Danga::Socket defers close() syscalls until the end of the event loop to avoid FD recycling. Unfortunately, this is dependent on IO events firing and waking the process up from poll/kevent/epoll_wait. Without any I/O activity, a socket could remain in the @Danga::Socket::ToClose array indefinitely. Thus, we will trigger a fake IO event after running all timers to trigger the deferred close in Danga::Socket::PostEventLoop.
2016-12-25httpd/async: improve variable naming
We only refer to PublicInbox::HTTP objects here, so '$io' was a bad name.
2016-12-25githttpbackend: minor cleanups to improve readability
Fewer returns improves readability and the diffstat agrees.
2016-12-25githttpbackend: simplify compatibility code
Fewer conditionals means theres fewer code paths to test and makes things easier-to-read.
2016-12-25githttpbackend: minor readability improvement
Use a more meaningful variable name for the Qspawn object, since this module is the reference for its use.
2016-12-25http: fix clobbering of $null_io
Oops, this would be disatrous if we started handling bigger request bodies or slow clients. Fixes: c008654229a9 ("avoid IO::File for anonymous temporary files")
2016-12-24linkify: modify argument in place
This results in over 1% speedup doing $MESSAGE_ID/T/ HTML generation for a 368-message thread.
2016-12-24view: do not modify array during iteration
This results in a half percent speedup or so doing $MESSAGE_ID/T/ HTML generation for a 368 message thread.
2016-12-24view: stop chomping off whitespace at ends of messages
This allows a 3-4% speedup in $MESSAGE_ID/T/ page generation speed for a 368+ message thread. It also more faithfully preserves the message as intended; even if the it makes the sender look like a space-wasting slob :P
2016-12-24view: remove unused parameter
And add a comment about it to remind our future selves.
2016-12-22search: lookup_mail handles modified DBs
We call lookup_mail all over the place, be sure we can handle database modifications in those cases.
2016-12-22doc: various comments on async handling
Notes for future developers (myself included) since we can't assume people can read my mind.
2016-12-22repobrowse: remove Plack::Request dependency
This does not make installation easier, but lightens runtime a bit. Plack::Request is unnecessary bloat and indirection which does things behind our back. $env has all the stuff we need.
2016-12-21searchthread: simplify API and remove needless OO
This simplifies callers to prevent errors and avoids needless object-orientation in favor of a single procedure call to handle threading and ordering.
2016-12-21searchthread: update comment about loop prevention
It definitely is necessary to prevent looping with the %seen hash.
2016-12-20repobrowse: git plain: fixup request handling
Leftover from commit d4cec3bc3e439ab6ea9a2f89da5f98033bd95b4f ("repobrowse: start reducing Plack::Request dependencies")
2016-12-20searchmsg: remove ensure_metadata
Instead, only preload the ->mid field for threading, as we only need ->thread and ->path once in Search->get_thread (but we will need the ->mid field repeatedly). This more than doubles View->load_results performance on according to thread-all on an inbox with over 300K messages.
2016-12-17searchmsg: do not memoize {date} field
We only generate the ->date once in NNTP, so creating the hash entry is a waste.
2016-12-17searchmsg: remove locale-dependency for ->date
strftime is locale-dependent, which can cause surprising failures for some users.
2016-12-17wwwtext: link to RFC4685 (Atom Threading)
This should give this feature some more visibility.
2016-12-17atom: implement message threading per RFC 4685
This will allows certain feed readers to render a message thread as described in <https://www.jwz.org/doc/threading.html>. Feed readers with knowledge of of RFC 4685 are unknown to us at this time, but perhaps this will encourage future implementations. Existing feed readers I've tested (newsbeuter, feed2imap) seem to ignore these tags gracefully without degradation.
2016-12-17feed: support publicinbox.<name>.feedmax
This allows users to customize by using smaller or larger Atom feeds than the default value of 25 entries.
2016-12-14wwwtext: remove outdated comment
I originally envisioned wwwtext being more flexible and able to serve arbitrary blobs; but at this point I consider it redundant and public-inbox is not wiki software.
2016-12-14repobrowse: start reducing Plack::Request dependencies
We'll be migrating away from it to minimize dependencies and surprises, just like the rest of public-inbox did several months ago.
2016-12-14Merge remote-tracking branch 'origin/repobrowse' into repobrowse
* origin/repobrowse: (98 commits) t/repobrowse_git_httpd.t: ensure signature exists for split t/repobrowse_git_tree.t: fix test for lack of bold repobrowse: fix alignment of gitlink entries repobrowse: show invalid type for tree views repobrowse: do not bold directory names in tree view repobrowse: reduce checks for response fh repobrowse: larger, short-lived buffer for reading patches repobrowse: reduce risk of callback reference cycles repobrowse: snapshot support for cgit compatibility test: disable warning for Plack::Test::Impl repobrowse: avoid confusing linkification for "diff" repobrowse: git commit view uses pi-httpd.async repobrowse: more consistent variable naming for /commit/ repobrowse: show roughly equivalent "diff-tree" invocation repobrowse: reduce local variables for state management repobrowse: summary handles multiple README types repobrowse: remove bold decorations from diff view repobrowse: common git diff parsing code repobrowse: implement diff view for compatibility examples/repobrowse.psgi: disable Chunked response by default ...
2016-12-13searchmsg: remove unused EPOCH_822 constant
This hasn't been needed since our Email::Abstract removal for message threading.
2016-12-13nntp: avoid useless use of strftime
There's no need to use strftime if we'll be converting the date by hand, anyways.
2016-12-12daemon: set $now time for NNTP shutdown
commit 6e238ee3396719e578d6a90e177a71ce9f8c1ca0 ("nntp: respect 3 minute idle time for shutdown") was incomplete, and needed this change to Daemon to be effective. In the future, there will be more common code between NNTP.pm and HTTP.pm
2016-12-10search: retry document loading from Xapian
In addition to needing to retry enquire queries, we also need to protect document loading from the Xapian DB and retry on modification, as it seems to throw the same errors. Checking the $@ ref for Search::Xapian::DatabaseModifiedError is actually in the test suite for both the XS and SWIG Xapian bindings, so we should be good as far as forward/backwards compatibility.
2016-12-10search: always sort thread results in ascending time order
This makes life easier for the threading algorithm, as we can use the implied ordering of timestamps to avoid temporary ghosts and resulting container vivication. This would've also allowed us to hide the bug (in most cases) fixed by the patch titled "thread: last Reference always wins", in case that needs to be reverted due to infinite looping.
2016-12-10thread: last Reference always wins
Since we use SearchMsg from Xapian data, we can be assured we do not get self-referential {references} field. However, we may need to be more careful when checking has_descendent for loops, as blindly calling add_child could open us up to that possibility...
2016-12-10view: skip ghosts with no direct children
Otherwise, a malicious or broken client could populate the thread skeleton with invalid References. We only care about ghosts which messages correctly refer to, not totally bogus ones which may be the result of long line or token truncation + wrapping in MUA headers.
2016-12-10view: reduce indentation for skeleton generation
This should reduce the number of subroutine calls needed for the common case of real (non-ghost) messages as well as shortening code.
2016-12-10thread: fix comment describing its existence
Mail::Thread is UNavailable on many distros, meaning ordinary users will have to rely on CPAN, a Perl-specific packaging tool.
2016-12-10view: favor SearchMsg for In-Reply-To over Email::MIME
This should avoid warnings during thread skeleton generation if ever the Xapian database disagrees with View.pm about which is the proper direct parent of a message. We will treat the data in Xapian as the truth (if Xapian is available).
2016-12-10search: favor In-Reply-To over last References iff IRT exists
Some email clients set the References headers backwards, so trust the In-Reply-To header if (and only if) it exists and is parseable as direct parent of the current message. For affected repos, this will require reindexing (via "public-inbox-index --reindex"), but there will be no version bump for this bugfix.
2016-12-06linkify: implement Markdown link compatibility (again)
Although unescaped parentheses in URLs are technically allowed, they are uncommon. However, Markdown-like syntaxes are unfortunately common for URLs, so we might as well support them. This fixes parentheses detection at sentence endings, as seen in practice on emails.