about summary refs log tree commit homepage
path: root/lib/PublicInbox/WWW.pm
DateCommit message (Collapse)
2017-02-14www: do not unescape PATH_INFO twice
PSGI specs already require PATH_INFO to be unescaped; so our tests were wrong, too.
2017-01-10introduce PublicInbox::MIME wrapper class
This should fix problems with multipart messages where text/plain parts lack a header. cf. git clone --mirror https://github.com/rjbs/Email-MIME.git refs/pull/28/head In the future, we may still introduce as streaming interface to reduce memory usage on large emails.
2016-10-05thread: remove Mail::Thread dependency
Introduce our own SearchThread class for threading messages. This should allow us to specialize and optimize away objects in future commits.
2016-08-18www: implement generic help text
Begin documenting some basic help functionality. I may tweak the anchor names of the various HTML endpoints to be more consistent with each other (old ones will be supported for a short while), so I'm not documenting those, for now. This may become part of a builtin key-value store for basic texts, but this probably shouldn't become a wiki engine, either.
2016-08-14www: do not unecessarily escape some chars in paths
Based on reading RFC 3986, it seems '@', ':', '!', '$', '&', "'", '; '(', ')', '*', '+', ',', ';', '=' are all allowed in path-absolute where we have the Message-ID. In any case, it seems '@' is fairly common in path components nowadays and too common in Message-IDs.
2016-08-09www: avoid misinterpreting '&' and ';' in query parameters
Oops, we must unescape each key=value pair in a QUERY_STRING individually; otherwise we cannot interpret '&' or ';' in query parameter values.
2016-07-09www: cleanup parameter passing
Reduce the size of hashes a bit and drops some unneeded hash lookups for uncommon paths.
2016-07-09www: drop unused constants
We no longer generate our footer, here. We are not currently advertising ssoma, here.
2016-07-07www: remove old footer generation code and normalize new.html
We now generate all of our HTML using WwwStream which forces us to have consistent headers and footers in the HTML itself. This also makes the search-capable vs search-less installs go to the new.html endpoint to maintain consistency (in case an admin decides to enable Xapian).
2016-07-07inbox: cleanup and consolidate object weakening
This fixes some layering violations and consolidates the cleanup into the inbox object itself. Keeping in mind weakening does not work at all without our PSGI server.
2016-07-02www: remove Plack::Request dependency entirely
Lighter and ever-so-slightly faster! Most importantly, this won't do non-obvious stuff behind our backs like trying to parse a POST request body for a query string param.
2016-07-02www: use PSGI env directly
More work on on the Plack::Request/CGI.pm removal front, No need to access the PSGI env through an extra hash lookup.
2016-07-02inbox: base_url method takes PSGI env hashref instead
This is lighter and we can work further towards eliminating our Plack::Request dependency entirely.
2016-06-30www_stream: add response wrapper sub
This encapsulates an entire PSGI response array, hopefully making it easier to generate responses and avoid typos when setting the Content-Type.
2016-06-30www: reinstate old thread view as an option
This hybrid view is better than the old flat, but can still fall down compared to the old threaded view in some cases.
2016-06-30feed: add $INBOX/new.html endpoint
This acts like the Atom feed; but should be viewable directly from browsers.
2016-06-30www: use WwwStream for dumping thread and search views
This allows us the HTTP server to react to backpressure from slow clients when writing. As a side effect, this also makes it easier for us to maintain a consistent header/footer across our HTML.
2016-06-30www: implement hybrid flat+thread conversation view
This should be more accessible to readers on narrow terminals (or giant fonts) while providing a chronological view which is also aware of message threading relationships.
2016-06-26www: unescape '+' in query parameter to space
Fixes: fbcb7de93884b ("www: remove a few more Plack::Request dependencies")
2016-06-20feed: various object-orientation cleanups
Favor Inbox objects as our primary source of truth to simplify our code. This increases our coupling with PSGI to make it easier to write tests in the future. A lot of this code was originally designed to be usable standalone without PSGI or CGI at all; but that might increase development effort.
2016-06-17www: undefined query string values are empty strings
We use very short query parameters for search, so "&r" without a '=' implies truth for 'r' (relevance).
2016-06-17www: escape HTML in footer description
This isn't a security vulnerability since $GIT_DIR/description is controlled by the admin; but it causes the footer to misrender.
2016-06-13view: msg_html uses getline body to reduce latency
We need to ensure we show the message body ASAP since the thread generation via Xapian could take a while and maybe even raise an exception or crash.
2016-06-05view: inline message reply into message view
This should reduce link following for replies and improve visibility. This should also reduce cache overhead/footprint for crawlers.
2016-06-02www: force two element key-value pairs in query
Oops, this quiets down a warning seen in logs.
2016-05-30www: remove a few more Plack::Request dependencies
Still a work in progress, but SearchView no longer depends on Plack::Request at all and Feed is getting there. We now parse all query parameters up front, but we may do that lazily again in the future.
2016-05-30www: remove gratuitous use of Plack::Request methods
Accessing $env directly is faster and we will eventually remove all Plack::Request dependencies.
2016-05-30git-http-backend: remove dependency on Plack::Request
Plack::Request is unnecessary overhead for this given the strictness of git-http-backend. Furthermore, having to make commit 311c2adc8c63 ("avoid Plack::Request parsing body") to avoid tempfiles should not have been necessary.
2016-05-28remove redundant NewsGroup class
Most of its functionality is in the PublicInbox::Inbox class. While we're at it, we no longer auto-create newsgroup names based on the inbox name, since newsgroup names probably deserve some thought when it comes to hierarchy.
2016-05-28config: remove try_cat
It's moved into the Inbox module and we no longer use it in WWW
2016-05-28www: remove footer_html support
I haven't used it in a while and the existing "description" is probably good enough. If we support it again, it should be plain-text + auto-linkified for ease-of-maintenance and consistency.
2016-05-19www: tighten up allowable filenames for attachments
Having a file start with '.' or '-' can be confusing and for users, so do not allow it.
2016-05-19www: validate and check filenames in URLs
We shall ensure links continue working for this.
2016-05-19www: support downloading attachments
This can be useful for lists where the convention is to attach (rather than inline) patches into the message body.
2016-05-17http: release resources when idle
This lets us release old git processes so unlinked packs (leftover from repacking) can be released. This may also be helpful for Xapian as indices get rebuilt for tuning. For SQLite (msgmap), the there may be no benefit besides reducing FD pressure. Followup changes will unify the Inbox and NewsGroup classes and allow better code-sharing between NNTP and HTTP classes (as well as the planned POP3 class).
2016-05-16www: fix for running under mount paths
We try to avoid issues like these by using relative URLs in hrefs, but we can't avoid the problem with Location: for redirects and Atom feeds which are likely to be rehosted elsewhere. We also reorder some of the code to work around a weird issue on the psgi-plack mailing list: <20160516073750.GA11931@dcvr.yhbt.net> (Somewhere on https://groups.google.com/group/psgi-plack but it's probably not bookmarkable)
2016-05-16declare Inbox object for reusability
From the beginning, we've avoided objects here in favor of faster startup time; but it may not be worth it since a persistent httpd/nntpd is faster and -mda isn't hit as often.
2016-05-15mbox: support /$INBOX/all.mbox.gz endpoint
Allows easily downloading the entire archive without special tools. In any case, it's not yet advertised to via HTML until we can test it better. It'll also support range queries in the future to avoid wasting bandwidth.
2016-05-14rename most instances of "list" to "inbox"
A public-inbox is NOT necessarily a mailing list, but it could serve as an input point for zero, one, or infinite mailing lists :D
2016-04-15www: redirect /$MESSAGE_ID/f/ endpoints
Quote-folding was a major design mistake pre-1.0. Since this project is still in its infancy and unlikely to be in wide use at the moment, redirect the /f/ endpoints back to the plain message.
2016-04-02www: more explicit "git clone" usage
Little harm in having the entire command-line for users and avoiding the cognitive overhead of figuring out $URL.
2016-03-03use raw header for Message-ID
Message-IDs should not be MIME encoded, but in case they are, use the raw form for compatibility with ssoma and possibly other tools. This prevents a potential problem where a malicious client could confuse our storage layer into indexing incorrect contents.
2016-02-29fixup Plack-related requires
We do not need to load Plack::Request outside of WWW anymore.
2016-02-29distinguish error messages intended for users vs developers
For error messages intended to show user error (e.g. giving invalid options), we add a newline ("\n") at the end to polluting the output with location information. However, for diagnosing non-user-triggered errors, we should show the location of where the error occured.
2016-02-26www: add News* wrappers to preload
We want to preload as much as possible in -httpd when forking to save memory via CoW.
2016-02-26www: workaround for malformed NNTP links
Some linkifiers to create invalid HTTP links when it sees a link intended for NNTP services. This means we may see links to news.public-inbox.org/inbox.comp.mail.public-inbox.meta point to "http://" on port 80 instead of 119. Try to redirect users to http://public-inbox.org/meta/ in this case.
2016-02-25hval: implement common UI for protocol-relative URLs
This allows users to avoid HTTPS -> HTTP downgrade warnings, but we will also avoid encouraging them towards HTTPS, for now. IMHO: the CA system gives a false sense of security, TLS libraries (e.g. OpenSSL) can introduce new bugs and problems (even to attack clients), and TLS libraries also eats memory on cheap servers.
2016-02-25www: make interface more OO
This allows multiple instances the WWW app from running within the same process space
2016-02-25remove direct CGI.pm support
Relying on Plack::Handler::CGI is much easier for long-term maintenance and development. Nowadays, we even include our own httpd implementation to facilitate easier deployment with PSGI/Plack.
2016-02-24www: support $MESSAGE_ID/R/ endpoint for replies
Setting the "In-Reply-To:" header via mailto: links is not well-supported and should probably not be encouraged unless the client situation improves. So instead, teach users more widely-supported ways of setting the In-Reply-To: header to ensure proper threading of replies.