about summary refs log tree commit homepage
path: root/lib/PublicInbox/WWW.pm
DateCommit message (Collapse)
2020-02-06treewide: run update-copyrights from gnulib for 2019
I didn't wait until September to do it, this year!
2020-02-04www: serve $INBOX_DIR/description as $INBOX_URL/description
Instead of serving $INBOX_DIR/all.git/description, since $INBOX_DIR/all.git/description is not described in the default message when it's missing.
2020-02-04www: stricter regexp for 405 errors
We want to match "GET" and "HEAD" exactly, not requests which start with "GET" or end with "HEAD". This doesn't seem like a real problem for public-inboxes which are actually public data anyways.
2020-01-01wwwstatic: add directory listing + index.html support
It's now possible to use WwwStatic as a standalone PSGI app to serve static files and recreate the award-winning web design of https://public-inbox.org/ :>
2020-01-01wwwstatic: move r(...) functions here
Remove redundant "r" functions for generating short error responses. These responses will no longer be cached by clients, which is probably a good thing since most errors ought to be transient, anyways. This also fixes error responses for our cgit wrapper when static files are missing.
2020-01-01www: move more logic into path_info_raw
It'll be easier to reuse in future code.
2019-12-27www: lazy load Plack::Util
cgit users won't need Plack::Util, here.
2019-10-22www: remove unused ctx_get sub
This hasn't been used since commit 48b21cb662c1e17b7 in 2016: ("declare Inbox object for reusability")
2019-09-09run update-copyrights from gnulib for 2019
2019-06-14rename reference to git epochs as "partitions"
Try to remain consistent with our own documentation regarding v2 git "epochs", first.
2019-06-09www: support $INBOX/git/$EPOCH.git for v2 cloning
And use it in manifest.js. To ease maintaining mirrors with grokmirror(1), we can accept a "git/" directory prefix before the epoch, and ".git" suffix after the epoch number. We maintain compatibility with "$INBOX/$EPOCH" cloning, of course, and it's still easier-to-type on the command-line.
2019-06-09www: wire up /$INBOX/manifest.js.gz, too
I can imagine myself just wanting to clone a single v2 inbox and all its epochs without thinking about include/exclude rules in a grokmirror config file.
2019-06-09wwwlisting: generate grokmirror-compatible manifest.js.gz
Support on-demand generation of "/manifest.js.gz" for inboxes. By default, this matches inboxes with URLs matching the given request hostname by default. This makes it easier to create full mirrors of several inboxes without needing to configure static file serving. cf. https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git
2019-06-04www: require ASCII word characters for CSS filenames
Allowing admins to set non-ASCII CSS filenames could cause unnecessary problems for client and proxies.
2019-06-04www: require ASCII range for mbox downloads
We do not support many mboxrd download range specifications at the moment; but parsing non-ASCII characters isn't planned. This makes no difference aside from being able to return 404 slightly earlier than we would've in the past.
2019-06-04www: require ASCII digit for git epoch
Don't inadvertantly serve git repos containing non-ASCII digit characters.
2019-06-04www: require ASCII filenames in git blob downloads
Our Hval::to_filename sub has always been strict about emitting ASCII-only characters for ViewVCS "raw" links. However, somebody could manually generate a filename with non-ASCII words for somebody else to download (we have no cheap and fast way of mapping filenames back to blobs for validation).
2019-06-04www: only emit ASCII chars in attachment filenames
We don't want to emit funky URLs which can be lost in translation or cause problems with non-Unicode-aware clients. Then, don't accept non-ASCII filenames in URLs, since a manually-generated URL/filename in attachment downloads could be used for Unicode homographs to confuse folks who down the attachment.
2019-05-21Merge remote-tracking branch 'origin/xap-optional' into master
* origin/xap-optional: admin: improve warnings and errors for missing modules searchidx: do not create empty Xapian partitions for basic lazy load Xapian and make it optional for v2 www: use Inbox->over where appropriate nntp: use Inbox->over directly inbox: add ->over method to ease access
2019-05-16www: unescape '+' => ' ' before general URI unescape
This allows searching for terms with "+" in them properly.
2019-05-15lazy load Xapian and make it optional for v2
More tests work without Search::Xapian, now. Usability issues still need to be fixed
2019-05-15www: use Inbox->over where appropriate
We don't need to rely on Xapian search functionality for the majority of the WWW code, even. subject_normalized is moved to SearchMsg, where it (probably) makes more sense, anyways.
2019-04-19www: support listing of inboxes
We will still return a 404 by default to '/' for compatibility with users of Plack::App::Cascade or similar. Inboxes are sorted by modification times to help users detect activity (similar to the /$INBOX/ topic view). New configuration options: * publicinbox.wwwlisting - configure the listing type * publicinbox.<name>.hide - hide a particular inbox from the listing See changes to public-inbox-config.pod for full descriptions of the new options. Requested-by: Leah Neukirchen <leah@vuxu.org> https://public-inbox.org/meta/871sdfzy80.fsf@gmail.com/
2019-04-19start depending on Perl 5.10.1+
I mainly want to start using the '//' (defined-or) operator to simplify code, and Perl 5.10.1 is roughly a decade old at this point. "given/when" would've be nice, but it's future is in doubt AFAIK. I also started using the 'parent' module in WwwHighlight, and 'autodie' in UserContent.pm, both of which were only distributed with Perl since 5.10.1; and testing with ancient versions/distros is time-consuming. Anyways, I think this a small-enough jump to not break any existing installations, given we already depend on fairly recent versions of git and Xapian. Maybe we can use more newish Perl features in the future...
2019-04-16cleanup: use '$ibx' consistently when referring to Inbox refs
'$inbox' is more human-readable, so that is for the more human-readable name in most cases. Making our variable naming more consistent should make the code easier-to-review and harder to screw up.
2019-04-16www: remove unnecessary Git object reference
We access the Git object via the Inbox object nowadays, so there's no point in having a shortcut to it, anymore.
2019-04-04www: fix missing cgit fallback after legacy redirects
We need to instate our cgit handler everywhere we use NewsWWW to catch wildcard requests which our normal endpoints do not handle.
2019-04-04www: wire up cgit as a 404 handler if cgitrc is configured
Requests intended for cgit are unlikely to conflict with requests to inboxes. So we can safely hand those requests off to cgit.cgi.
2019-02-23www: prevent '!important' in BOFH-specified CSS
CSS specified by the BOFH must never take precedence over what a user sets in userContent.css.
2019-02-13ensure bytes::length is available to callers
We were relying on Danga::Socket using the "bytes" pragma, previously. Nowadays, the "bytes" pragma is not recommended in general, but bytes::length remains acceptable for getting the byte-size of a scalar.
2019-01-20$INBOX/_/text/color/ and sample user-side CSS
Since we now support more CSS classes for coloring, give this feature more visibility.
2019-01-20www: admin-configurable CSS via "publicinbox.css"
Maybe we'll default to a dark theme to promote energy savings... See contrib/css/README for details
2019-01-20view: enforce trailing slash for /$INBOX/$OID/s/ endpoints
As with our use of the trailing slash in $MESSAGE_ID/T/ and '$MESSAGE_ID/t/' endpoints, this for 'wget -r --mirror' compatibility as well as allowing sysadmins to quickly stand up a static directory with "index.html" in it to reduce load.
2019-01-19view: wire up diff and vcs viewers with solver
2019-01-15config: inbox name checking matches git.git more closely
Actually, it turns out git.git/remote.c::valid_remote_nick rules alone are insufficient. More checking is performed as part of the refname in the git.git/refs.c::check_refname_component I also considered rejecting URL-unfriendly inbox names entirely, but realized some users may intentionally configure names not handled by our WWW endpoint for archives they don't want accessible over HTTP.
2018-06-26www: use undecoded paths for Message-ID extraction
In PSGI, PATH_INFO contains URI-decoded paths which cause problems when Message-IDs contain ambiguous characters for used for routing. Instead, extract the undecoded path from REQUEST_URI and use that. Reported-by: Leah Neukirchen <leah@vuxu.org> https://public-inbox.org/meta/8736xsb5s5.fsf@vuxu.org/
2018-03-29www: cleanup expensive fallback for legacy URLs
Back in the day, we compressed long Message-IDs to SHA-1 hexdigests for the URL. This now redirects to a 301 in the hopes we can remove these checks some day to reduce overhead.
2018-03-27www: support cloning individual v2 git partitions
This will require multiple client invocations, but should reduce load on the server and make it easier for readers to only clone the latest data. Unfortunately, supporting a cloneurl file for externally-hosted repos will be more difficult as we cannot easily know if the clones use v1 or v2 repositories, or how many git partitions they have.
2018-03-27view: depend on SearchMsg for Message-ID
Since we need to handle messages with multiple and duplicate Message-ID headers, our thread skeleton display must account for that. Since we have a "preferred" Message-ID in case of conflicts, use it as the UUID in an Atom feed so readers do not get confused by conflicts.
2018-03-27www: get rid of unnecessary 'inbox' name reference
We use the actual Inbox object everywhere else and don't need the name of the inbox separated from the object.
2018-03-27view: permalink (per-message) view shows multiple messages
This needs tests and further refinement, but current tests pass.
2018-03-23www: $MESSAGE_ID/raw endpoint supports "duplicates"
Since v2 supports duplicate messages, we need to support looking up different messages with the same Message-Id. Fortunately, our "raw" endpoint has always been mboxrd, so users won't need to change their parsing tools.
2018-02-16www: stop assuming mainrepo == git_dir
It won't be in v2
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2017-12-08search: force large mbox result downloads to POST
This should prevent crawlers (including most robots.txt ignoring ones) from burning our CPU time without severely compromising usability for humans.
2017-05-23www: do not mangle characters from search queries
Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> https://public-inbox.org/meta/CACBZZX5Gnow08r=0A1J_kt3a=zpGyMfvsqu8nAN7kacNnDm+dg@mail.gmail.com/
2017-05-09www: avoid undefined warnings for query string parsing
Sometimes bots generate malformed queries with sequential "&" and ";" characters.
2017-02-14www: do not unescape PATH_INFO twice
PSGI specs already require PATH_INFO to be unescaped; so our tests were wrong, too.
2017-01-10introduce PublicInbox::MIME wrapper class
This should fix problems with multipart messages where text/plain parts lack a header. cf. git clone --mirror https://github.com/rjbs/Email-MIME.git refs/pull/28/head In the future, we may still introduce as streaming interface to reduce memory usage on large emails.
2016-10-05thread: remove Mail::Thread dependency
Introduce our own SearchThread class for threading messages. This should allow us to specialize and optimize away objects in future commits.