about summary refs log tree commit homepage
path: root/lib/PublicInbox/WwwStream.pm
DateCommit message (Collapse)
2020-08-01www: rework async_* to use method table
Although the ->async_next method does not take $self as a receiver, but rather a PublicInbox::HTTP object, we may still retrieve it to be called with the HTTP object via UNIVERSAL->can.
2020-07-06www: update internal docs
We no longer favor getline+close for streaming PSGI responses when using public-inbox-httpd. We still support it for other PSGI servers, though.
2020-07-06wwwstream: eliminate ::response, use html_oneshot
All of our streaming responses use ::aresponse, now, and our synchronous responses use html_oneshot. So there's no need for the old WwwStream::response.
2020-07-06view: make /$INBOX/$MSGID/ permalink async
This will allow -httpd to handle other requusts if waiting on an HDD seek or git to decode a blob.
2020-07-06wwwstream: subclass off GzipFilter
This makes WwwStream closer to MboxGz and WwwAtomStream and will eventually allow us to follow the same patterns.
2020-07-06wwwstream: use parent.pm and no warnings
parent.pm is leaner than base and we'll rely on `-w' for warnings during development.
2020-07-06wwwstream: reduce blob fetch paths for ->getline
This will make it easier to support asynchronous blob retrievals. The `$ctx->{nr}' counter is no longer implicitly supplied since many users didn't care for it, so stack overhead is slightly reduced.
2020-07-06wwwstream: reduce object graph depth
Like with WwwAtomStream and MboxGz, we can bless the existing $ctx object directly to avoid allocating a new hashref. We'll also switch from "->" to "::" to reduce stack utilization.
2020-07-06gzipfilter: replace Compress::Raw::Deflate usages
The new ->zmore and ->zflush APIs make it possible to replace existing verbose usages of Compress::Raw::Deflate and simplify buffering logic for streaming large gzipped data. One potentially user visible change is we now break the mbox.gz response on zlib failures, instead of silently continuing onto the next message. zlib only seems to fail on OOM, which should be rare; so it's ideal we drop the connection anyways.
2020-07-06www*stream: gzip ->getline responses
Our most common endpoints deserve to be gzipped.
2020-07-06wwwstream: oneshot: perform gzip without middleware
Plack::Middleware::Deflater forces us to use a memory-intensive closure. Instead, work towards building compressed strings in memory to reduce the overhead of buffering large HTML output.
2020-03-30wwwstream::oneshot => html_oneshot
And use Exporter to make our life easier, since WwwAltId was using a non-existent PublicInbox::WwwResponse namespace in error paths which doesn't get noticed by `perl -c' or exercised by tests on normal systems. Fixes: 6512b1245ebc6fe3 ("www: add endpoint to retrieve altid dumps")
2020-03-25wwwstream: oneshot sets content-length
PublicInbox::HTTP will chunk, otherwise, and that's extra overhead which isn't needed.
2020-03-25extmsg: use WwwResponse::oneshot
No reason to use the ->getline interface for small responses.
2020-03-25wwwstream: introduce oneshot API to avoid ->getline
The ->getline API is only useful for limiting memory use when streaming responses containing multiple emails or log messages. However it's unnecessary complexity and overhead for callers (PublicInbox::HTTP) when there's only a single message.
2020-02-06treewide: run update-copyrights from gnulib for 2019
I didn't wait until September to do it, this year!
2020-01-27wwwstream: discard single-use $ctx fields after use
This should make it clear that we only use these elements once and can discard them. While we're in the area, avoid escaping '"' by using qq() instead of "" to quote strings requiring interpolation.
2020-01-27www*stream: favor \&close instead of *close
Be explicit that we're making a code reference, and not a reference to a scalar, array, hash, or IO...
2020-01-24wwwstream: shorten cloneurl uniquification
Another place where List::Scalar::uniq doesn't make sense, but there's a small op reduction to be had anyways.
2020-01-06hval: export prurl and add prototype
This allows to do some compile-time checking and fills in a missing "use" in PublicInbox::NewsWWW, allowing it to be used standalone and independently of PublicInbox::WWW
2019-10-17Merge remote-tracking branch 'origin/inboxdir'
* origin/inboxdir: config: remove redundant inboxdir check config: support "inboxdir" in addition to "mainrepo" examples/grok-pull.post_update_hook: use "inbox_dir"
2019-10-17doc: avoid [<directory>] arg for git-clone(1)
While it is possible to host source code from the root of a URL using git-http-backend(1), the lack of pathname in the URL can also be confusing to users. So just add the path name of the project into the URL itself so users can invoke "git clone" with one command-line argument instead of two. Of course, previously documented URLs continue to work as normal.
2019-10-16config: support "inboxdir" in addition to "mainrepo"
"mainrepo" ws a bad name and artifact from the early days when I intended for there to be a "spamrepo" (now just the ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be especially confusing, since v2 needs at least two git repositories (epoch + all.git) to function and we shouldn't confuse users by having them point to a git repository for v2. Much of our documentation already references "INBOX_DIR" for command-line arguments, so use "inboxdir" as the git-config(1)-friendly variant for that. "mainrepo" remains supported indefinitely for compatibility. Users may need to revert to old versions, or may be referring to old documentation and must not be forced to change config files to account for this change. So if you're using "mainrepo" today, I do NOT recommend changing it right away because other bugs can lurk. Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
2019-10-01www: fix absolute URLs when mounted under a subdir
While we avoid generating absolute URLs in most cases, our "git clone" instructions and URL headers in mboxrd files contain full URLs. So do the same thing we do for WwwAtomStream and pre-generate the full URL before Plack::App::URLMap changes $env->{PATH_INFO} and $env->{SCRIPT_NAME} back to their original values. Reported-by: edef <edef@edef.eu> Link: https://public-inbox.org/meta/cover.0f97c47bb88db8b875be7497289d8fedd3b11991.1569296942.git-series.edef@edef.eu/
2019-09-27wwwtext: support $INBOX_URL/_/text/config/raw
This returns a git-config(1)-compatible file to make it easier to get started on mirroring an existing public-inbox. Omitting the "raw" from the URL works, as well, but I'm not sure if it's very useful.
2019-09-09run update-copyrights from gnulib for 2019
2019-06-14rename reference to git epochs as "partitions"
Try to remain consistent with our own documentation regarding v2 git "epochs", first.
2019-05-07wwwstream: do not load URI.pm
It's unneeded since commit e358bd7a3833f8c5 (2016-07-02) ("inbox: base_url method takes PSGI env hashref instead") So we only depend on URI::Escape from the "URI" CPAN distribution, at the moment.
2019-05-05Merge remote-tracking branch 'origin/wwwlisting'
* origin/wwwlisting: www: support listing of inboxes start depending on Perl 5.10.1+
2019-05-04wwwstream: drop tor2web URL
It was a MITM hazard and been killed off by the DoS subsystem last July: https://marc.info/?i=87d0vwwkbs.fsf@riseup.net
2019-04-19www: support listing of inboxes
We will still return a 404 by default to '/' for compatibility with users of Plack::App::Cascade or similar. Inboxes are sorted by modification times to help users detect activity (similar to the /$INBOX/ topic view). New configuration options: * publicinbox.wwwlisting - configure the listing type * publicinbox.<name>.hide - hide a particular inbox from the listing See changes to public-inbox-config.pod for full descriptions of the new options. Requested-by: Leah Neukirchen <leah@vuxu.org> https://public-inbox.org/meta/871sdfzy80.fsf@gmail.com/
2019-04-16cleanup: use '$ibx' consistently when referring to Inbox refs
'$inbox' is more human-readable, so that is for the more human-readable name in most cases. Making our variable naming more consistent should make the code easier-to-review and harder to screw up.
2019-01-20$INBOX/_/text/color/ and sample user-side CSS
Since we now support more CSS classes for coloring, give this feature more visibility.
2019-01-20www: admin-configurable CSS via "publicinbox.css"
Maybe we'll default to a dark theme to promote energy savings... See contrib/css/README for details
2019-01-09doc: various overview-level module comments
Hopefully this helps people familiarize themselves with the source code.
2018-12-28wwwstream: always show multi-line cloning instructions
Unfortunately, long inbox names and URLs don't really display well with my gigantic fonts...
2018-03-30wwwstream: flesh out clone instructions for v2
Relying solely on git for v2 repos is probably not so useful, so add pointers to public-inbox-init/index commands.
2018-03-27www: support cloning individual v2 git partitions
This will require multiple client invocations, but should reduce load on the server and make it easier for readers to only clone the latest data. Unfortunately, supporting a cloneurl file for externally-hosted repos will be more difficult as we cannot easily know if the clones use v1 or v2 repositories, or how many git partitions they have.
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2016-12-03wwwstream: improve documentation and variable naming
Hopefully this makes the code more readable for newbies.
2016-11-29note the source code is AGPL for cloning
This should be adequate warning for folks who may be uncomfortable or uncertain about even possessing AGPL source code due to employer agreements and such. Disclaimer: I remain completely in favor of AGPL and strong copyleft, and am more than willing to risk my own future on it. However, I refuse to even nudge people into downloading AGPL source code if it presents any legal risk to them.
2016-08-23www: give tor2web some exposure, too
Not everybody can run Tor, hopefully more can use Tor2web even if it compromises their privacy. This should help make system more resilient for users unable to use Tor.
2016-08-18www: implement generic help text
Begin documenting some basic help functionality. I may tweak the anchor names of the various HTML endpoints to be more consistent with each other (old ones will be supported for a short while), so I'm not documenting those, for now. This may become part of a builtin key-value store for basic texts, but this probably shouldn't become a wiki engine, either.
2016-08-12www: allow including links to NNTP sites in HTML footer
Improve the discoverability of NNTP endpoints for users who still know what NNTP is. ==> ~/.public-inbox/config <== ; aliases for the locally-run nntpd can be specified in ; the "publicinbox" section: [publicinbox] nntpserver = nntp://ou63pmih66umazou.onion/ nntpserver = news.public-inbox.org ; NNTPS is not supported natively, yet, ; but one can use haproxy or similar ; nntpserver = nntps://news.public-inbox.invalid/ ; mirrors for specific inboxes may be specified either as full ; NNTP (or NNTPS) URLs, or with the server name only if the ; newsgroup name is specfied for a local NNTP server [publicinbox "git"] ... newsgroup = inbox.a.b.c nntpmirror = nntp://czquwvybam4bgbro.onion/ nntpmirror = hjrcffqmbrq6wope.onion ; there may be a mirror on a different server with a ; different name: nntpmirror = nntp://news.example.com/differently.named.group ; (And I really need to write manpages for all this...)
2016-08-02wwwstream: prioritize search in top title bar
search is probably more useful so users should be able to select it sooner. Put it on its own line so it won't get scrolled off the edge for non-CSS users. Fix a minor spacing bug in the input tag while we're at it, too
2016-07-21www: redefinable project name and URL
This should make it easier for folks to run their own forks.
2016-07-21www: a note .onion URLs require the usage of Tor
Not everybody knows what .onion URLs are, so refer them to Tor.
2016-07-21www: label sections and hopefully improve navigation
Clearly label "Thread overview" and "Reply instructions" so users can quickly skip stuff they're not interested in. Additionally, note the fact the thread view allows quick navigation within the thread to avoid extra network requests and improve the display for single-message threads. Finally, use <hr> to better-delineate sections of each page.
2016-07-06wwwstream: allow undef noop callback
This can be use to specify a small response using -html_tip.
2016-07-02wwwstream: wording/grammar tweaks in trailer
git.git documentation uses "clonable" so that's probably a better term than "clone-able". Also, shorten the section for retrieving our code and remove an obvious typo.