about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2020-01-01wwwstatic: add directory listing + index.html support
It's now possible to use WwwStatic as a standalone PSGI app to serve static files and recreate the award-winning web design of https://public-inbox.org/ :>
2020-01-01wwwstatic: avoid TOCTTOU for FIFO check
We can use Perl's sysopen function to pass O_NONBLOCK to open(2) and avoid blocking on FIFOs. This avoids a TOCTTOU race where somebody can change a regular to FIFO in between the stat(2) and open(2) syscalls.
2020-01-01wwwstatic: do not open() files for HEAD requests
open() is a much more expensive syscall than stat(), so avoid it
2020-01-01wwwstatic: move r(...) functions here
Remove redundant "r" functions for generating short error responses. These responses will no longer be cached by clients, which is probably a good thing since most errors ought to be transient, anyways. This also fixes error responses for our cgit wrapper when static files are missing.
2020-01-01www: move more logic into path_info_raw
It'll be easier to reuse in future code.
2020-01-01wwwstatic: implement Last-Modified and If-Modified-Since
We're already serving static files for cgit, and will serve more static files, soon.
2020-01-01cgit: type declaration for PublicInbox::Git
AFAIK this doesn't do anything for Perl internally since PublicInbox::Git doesn't "use fields", but it makes it easier for humans readers to follow and ensure we're not passing unblessed or non-ref scalars to PublicInbox::GitHTTPBackend::serve.
2020-01-01filter/base: export REJECT as a constant
And update callers to use it, as it makes the code a bit cleaner. Probably irrelvant, but it should be faster, too, as "perl -I lib -w -MO=Deparse $FILE" shows REJECT() calls are constant-folded.
2020-01-01githttpbackend: remove ancient compatibility check
The ref() call could be hitting memory leaks on Perl 5.16.x. It's been 3 years (2016-12-25) since 292ca34140489da2 ("githttpbackend: simplify compatibility code") back when this project was barely known and probably nobody used examples/public-inbox.psgi...
2020-01-01http: update comment about psgix.io usage
We've been using async_pass for a while.
2020-01-01wwwstatic: getline: die on missing psgix.io
"psgix." extensions aren't guaranteed, so make we should try and support some theoretical generic PSGI servers without "psgix.io" on errors by die-ing. While we're at it, make the error handling path more obvious by sharing more code between the EOF and errno ($!) cases.
2020-01-01spamcheck/spamc: pass GLOB handles instead of FD numbers
The spawn() interface improvements[1] propagate to popen_rd, too, so we can avoid weird dances to keep the GLOB handle references live and just pass the handle around. [1] commit 267371b1273b518215939e817e53733584b68af7 ("spawn: allow passing GLOB handles for redirects")
2020-01-01nntp: handle 2-digit year "70" properly
Time::Local has the concept of a "rolling century" which is defined at 50 years on either side of the current year. Since it's now 2020 and >50 years since the Unix epoch, the year "70" gets interpreted by Time::Local as 2070-01-01 instead of 1970-01-01. Since NNTP servers are unlikely to store messages from the future, we'll feed 4-digit year to Time::Local::{timegm,timelocal} and hopefully not have to worry about things until Y10K. This fixes test failures on t/v2writable.t and t/nntpd.t since 2020-01-01.
2019-12-30spawn: better error handling
Since vfork always shares memory between the child and parent, we can propagate errors to the parent errno using shared memory instead of just dumping to stderr and hoping somebody sees it.
2019-12-30spawn: support chdir via -C option
This simplifies our admin module a bit and allows solver to be used with v1 inboxes using git versions prior to v1.8.5 (but still >= git v1.8.0).
2019-12-30spawn: allow passing GLOB handles for redirects
We can save callers the trouble of {-hold} and {-dev_null} refs as well as the trouble of calling fileno().
2019-12-29search: load_xapian: return true on success
This was causing -xcpdb and other admin modules to fail outside of tests (or when testing with the slow TEST_RUN_MODE=0).
2019-12-28Merge branch 'no-closure'
* no-closure: (30 commits) search: retry_reopen passes user arg to callback solvergit: allow passing arg to user-supplied callback viewvcs: avoid anonymous sub for HTML response wwwattach: avoid anonymous sub for msg_iter view: msg_iter calls add_body_text directly searchview: remove anonymous sub when sorting threads by relevance view: thread_html: pass named sub to WwwStream searchview: pass named subs to Www*Stream wwwtext: avoid anonymous sub in response contentid: no anonymous sub view: msg_html: stop using an anonymous sub view: avoid anon sub in stream_thread config: each_inbox: pass user arg to callback feed: avoid anonymous subs mboxgz: pass $ctx to callback to avoid anon subs www: lazy load Plack::Util githttpbackend: split out wwwstatic qspawn: psgi_return: allow non-anon parse_hdr callback qspawn: drop "qspawn.filter" support, for now qspawn: psgi_qx: eliminate anonymous subs ...
2019-12-28ds: use MSG_MORE when wbuf is empty during long responses
HTTP::getline_pull and NNTP::long_step will both populate {wbuf} manually to avoid recursion, so we need to account for an empty-but-present {wbuf} while dispatching msg_more().
2019-12-28search: retry_reopen passes user arg to callback
This allows callers to pass named (not anonymous) subs. Update all retry_reopen callers to use this feature, and fix some places where we failed to use retry_reopen :x
2019-12-28solvergit: allow passing arg to user-supplied callback
This allows us to get rid of the requirement to capture on-stack variables with an anonymous sub, as illustrated with the update to viewvcs to take advantage of this. v2: fix error handling for missing OIDs
2019-12-27viewvcs: avoid anonymous sub for HTML response
No need to create a new sub for every HTML page we render with our VCS viewer.
2019-12-27wwwattach: avoid anonymous sub for msg_iter
We can pass arguments to msg_iter for msg_iter to pass to our user-supplied callback, now.
2019-12-27view: msg_iter calls add_body_text directly
No need to waste several kilobytes creating an anonymous sub for every invocation of msg_iter.
2019-12-27searchview: remove anonymous sub when sorting threads by relevance
We don't need to return a closure or have a separate hash for sorting threads by relevance. Instead, we can stuff the relevance {pct} into the SearchMsg object itself and use that. Note: upon reviewing this code, the sort-by-relevance seems bogus as it only considers the relevance of the topmost message. Instead, it would make more sense to the user to sort by the highest relevance of all messages in that particular thread.
2019-12-27view: thread_html: pass named sub to WwwStream
We can pass everything we need into the WWW $ctx to avoid allocating kilobytes of memory for an anonymous sub for every $MESSAGE_ID/t/ request.
2019-12-27searchview: pass named subs to Www*Stream
Both WwwStream and WwwAtomStream ->response pass the WWW $ctx to the callback nowadays, so we can pass named subs to them.
2019-12-27wwwtext: avoid anonymous sub in response
We can pass arbitrary local variables via WWW $ctx, so just pass that into the one-off _do_linkify sub which already exists.
2019-12-27contentid: no anonymous sub
msg_iter now passes a user specified arg into the supplied callback, so we can use that to pass the Digest object into the \&content_dig_i callback.
2019-12-27view: msg_html: stop using an anonymous sub
Stash 5 local variables into the WWW $ctx hash table instead of allocating several kilobytes for an anonymous sub.
2019-12-27view: avoid anon sub in stream_thread
WwwStream already passes the WWW $ctx to the callback sub, so we don't need to create a new sub every call to capture local variables for the callback.
2019-12-27config: each_inbox: pass user arg to callback
Another place where we can replace anonymous subs with named subs by passing a user-supplied arg.
2019-12-27feed: avoid anonymous subs
WwwStream already passes the WWW $ctx to the user-supplied callback, and it's a trivial change for WwwAtomStream to do the same. Callers in Feed.pm can now take advantage of that to save a few kilobytes of memory on every response.
2019-12-27mboxgz: pass $ctx to callback to avoid anon subs
Another place where we can rid ourselves of most anonymous subs by passing the $ctx arg to the callback.
2019-12-27www: lazy load Plack::Util
cgit users won't need Plack::Util, here.
2019-12-27githttpbackend: split out wwwstatic
Make it easier to share code between our GitHTTPBackend and Cgit packages, for now, and possibly other packages in the future. We can avoid inline_object and anonymous subs at the same time, reducing per-request memory overhead.
2019-12-26qspawn: psgi_return: allow non-anon parse_hdr callback
Callers can supply an arg to parse_hdr, now, eliminating the need for closures to capture local variables.
2019-12-26qspawn: drop "qspawn.filter" support, for now
This feature was added in preparation for future changes that have yet to materialize after nearly 3 years. We can re-add it if needed in the future.
2019-12-26qspawn: psgi_qx: eliminate anonymous subs
We can follow what we did in psgi_return to make psgi_qx allocate less memory on each call.
2019-12-26qspawn: psgi_return_start: hoist out from psgi_return
Instead of just passing the rpipe to the start_cb, pass the entire qspawn ref to start_cb. Update existing callers to avoid circular refs.
2019-12-26qspawn: psgi_return: initial cb can be named
We can take advantage of HTTPD::Async being able to pass user-supplied args to callbacks to get rid of one (of many) anonymous subs in the code path.
2019-12-26httpd/async: get rid of ephemeral main_cb
Cheaper to use up two hash table slots than creating a new sub.
2019-12-26qspawn: reduce local vars, de-anonymize rd_hdr
rd_hdr() now becomes a named subroutine instead of a per-call local variable, so kilobytes of memory will not have to be allocated for it on every ->psgi_return call.
2019-12-26msg_iter: provide means to stop using anonymous subs
And remove the last anonymous sub in SolverGit itself.
2019-12-26qspawn: replace anonymous $end callbacks w/ event_step
This will tie into the DS event loop if that's used, but event_step an be called directly without relying on the event loop from Apache or other HTTP servers (or PSGI tests).
2019-12-26qspawn: disambiguate command vs PSGI env
Make things easier-to-follow and paves the way for future work to reduce dependencies on anonymous subs capturing local variables.
2019-12-26qspawn: remove some anonymous subs for psgi_qx
By passing a user-supplied arg to $qx_cb, we can eliminate the callers' need to capture on-stack variables with a closure. This saves several kilobytes of memory allocation at the expense of some extra hash table lookups in user-supplied callbacks. It also reduces the risk of memory leaks by eliminating a common source of circular references.
2019-12-26httpd/async: support passing arg to callbacks
Another step towards removing anonymous subs to eliminate a possible source of memory leaks and high memory use.
2019-12-26git: allow async_cat to pass arg to callback
This allows callers to avoid allocating several KB for for every call to ->async_cat.
2019-12-26wwwlisting: do not rely on $? after ProcessPipe::CLOSE
ProcessPipe::CLOSE won't reliably set $? inside the event loop if waitpid(..., WNOHANG) isn't successful. So use a blocking waitpid() call, here, and hope "git show-ref" exits promptly since we've already drained its stdout.