about summary refs log tree commit homepage
DateCommit message (Collapse)
2019-05-08DS: epoll: fix misordered EPOLL_CTL_DEL call
Any operations on an fd after POSIX::close() are invalid, so epoll_ctl will fail. Worse off, in a multi-threaded Perl, the fd may be reused by another thread and EPOLL_CTL_DEL can hit the wrong file description as a result. cf. https://rt.cpan.org/Ticket/Display.html?id=129487
2019-05-08DS: drop unused "_undef" sub
No longer used since we removed the *_ip_string fields
2019-05-08syscall: drop readahead wrapper
No backwards compatibility to worry about for us; and fadvise is superior anyways.
2019-05-08build: do not manify DS and Syscall pods
We don't need to increase our install footprint with documentation from our internals (which will surely change).
2019-05-08DS: handle EINTR in IO::Poll path, too
IO::Poll::_poll returns -1, which is "true" to Perl. cf. https://rt.cpan.org/Ticket/Display.html?id=129484
2019-05-05DS: workaround IO::Kqueue EINTR (mis-)handling
IO::Kqueue seems unmaintained, so workaround a long-standing bug where it falls over on signals: https://rt.cpan.org/Ticket/Display.html?id=116615
2019-05-05DS: drop profiling support
There's other ways to profile and we don't need to add runtime branches to do this.
2019-05-05DS: remove unused fields and functions
More will likely be dropped in the future, but drop the obvious ones we aren't using, for now; especially since some of them are set at ->new time and unavoidable. This saves 579 bytes per-client on my 64-bit Debian stable system as measured by Devel::Size::total_size from PublicInbox::HTTP::event_read. This adds up in C10K or C100K situations. Things we drop are: * corked - MSG_MORE requires fewer syscalls * read_push_back - tried to use it, ate CPU with slow clients * IP/port fields - accept() already returns what we care about
2019-05-05listener: use EPOLLEXCLUSIVE for listen sockets
Since our listen sockets are non-blocking and we may run multiple httpd|nntpd processes; we need a way to avoid thundering herds when there are multiple httpd|nntpd worker processes. EPOLLEXCLUSIVE was added just for that in Linux 4.5
2019-05-04bundle Danga::Socket and Sys::Syscall
These modules are unmaintained upstream at the moment, but I'll be able to help with the intended maintainer once/if CPAN ownership is transferred. OTOH, we've been waiting for that transfer for several years, now... Changes I intend to make: * EPOLLEXCLUSIVE for Linux * remove unused fields wasting memory * kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615 * accept4 support And some lower priority experiments: * switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes) * nginx-style buffering to tmpfile instead of string array * sendfile off tmpfile buffers * io_uring maybe?
2019-05-04wwwstream: drop tor2web URL
It was a MITM hazard and been killed off by the DoS subsystem last July: https://marc.info/?i=87d0vwwkbs.fsf@riseup.net
2019-04-29doc/standards: add a link to PSGI specs
2019-04-27extmsg: escape ampersands in @EXT_URL array
We already escape the user-provided Message-IDs (so there's no security problem AFAIK), but the URL templates which exist in our source code were not escaped properly. This quiets down tidy(1).
2019-04-26viewdiff: do not break out of DSTATE_CTX on /^$/
It seems a common case for mangled patches is editors or MUAs dropping trailing whitespace, and lines matching /^ $/ gets the space dropped to only match /^$/.
2019-04-25cgit: improve handling of cgit data path
Document `publicinbox.cgitdata' config directive, but allow it to be unspecified and/or missing for installations which do not wish to serve static data at all. For users installing cgit from source to their home directory, we can usually infer the cgit data path based on the cgit.cgi binary path, even.
2019-04-25examples/cgit-commit-filter.lua: some doc updates
It's been a while since I wrote this, and it needs to be kept up-to-date with some advances in our Perl code.
2019-04-25examples: cgit filter for use with WwwHighlight
I'm using this as the cgit about-filter and source-filter in https://80x24.org/public-inbox.git
2019-04-24wwwhighlight: deal with no-op highlight
Not all inputs are highlight-able, so reuse the original input and just linkify it if it can't be highlighted.
2019-04-24examples/cgit-commit-filter: remove unused variable
2019-04-24wwwhighlight: do linkification, too
Otherwise, there's no reason to use this API over highlight(1). Maybe this can be an option in the future; but I'm struggling to find a reason to not do it by default.
2019-04-23view: avoid "1+ messages" in per-message footer of /t/ and /T/
Try to appear gramatically correct and state: "only message in thread" when there's only one known (to us) message in the thread.
2019-04-23doc: add standards reference
Incomplete at the moment, but this ought to be a handy reference for both implementers and users alike.
2019-04-19t/hl_mod: workaround w3m not handling '
This fixes a test failure on my Debian buster system. Bug report filed for w3m to handle "'": https://bugs.debian.org/927409 and for "highlight" to favor "'" in case other browsers fail: https://bugs.debian.org/927410
2019-04-18linkify: require parentheses pairs in URLs
Dangling parentheses with trailing punctuation usually means the parentheses is not intended as part of the URL.
2019-04-18view: show "(no subject)" consistently in HTML
Empty subjects ("") and undefined Subjects: are now both displayed as "(no subject)" for now.
2019-04-18doc: config: fix braino/typo :x
2019-04-18linkify: don't get confused by URLs in Perl code, at least
The URLs at the top of WwwStream.pm weren't getting linkified correctly.
2019-04-18inbox: add `modified' sub
For inboxes with SQLite enabled (all v2, and probably most v1); we can use the overview DB to get the timestamp of the latest message. It's faster than scanning git branches for commit times, but not always the same.
2019-04-18git: calculate modified time of repository
This will be used for generating an HTML listing for v1 inboxes, at least. The logic for this follows that of grokmirror, and we may dynamically generate manifest.js.gz natively...
2019-04-18config: use '$ibx' instead of '$rv' to denote Inbox objects
Followup-to: 6e6f7999361925e4 ("cleanup: use '$ibx' consistently when referring to Inbox refs")
2019-04-17TODO: add an item for hooks support
and add a note for grokmirror support/integration, too
2019-04-16HACKING: stop advertising mailing list subscription info
It's still available, but no point in advertising something which increases the dependency on a centralized subscriber list. In fact, it's actively against the goal of promoting resistance against centralization.
2019-04-16import: prefix 'inbox' with '-' to denote Inbox ref
This is for consistency with other fields which follow this pattern w.r.t. field-naming when referring to internal fields.
2019-04-16cleanup: use '$ibx' consistently when referring to Inbox refs
'$inbox' is more human-readable, so that is for the more human-readable name in most cases. Making our variable naming more consistent should make the code easier-to-review and harder to screw up.
2019-04-16www: remove unnecessary Git object reference
We access the Git object via the Inbox object nowadays, so there's no point in having a shortcut to it, anymore.
2019-04-16INSTALL: document Crypt:CBC for PublicInbox::Unsubscribe
I basically forgot about this module, but "make syntax" on a new chroot trips it up.
2019-04-15config: fix regression in repo.path => coderepo.dir mapping
We parse cgitrc for "repo.path", while we use "coderepo.dir" to mean the same thing for non-cgit users. So I ended up confusing myself, here. But then again, git uses "--git-dir" and "GIT_DIR", so I suspect "dir" is the better choice than "path", here
2019-04-15viewdiff: document constants
We'll be building off of this for showing diffs in the coderepo views.
2019-04-15doc/config: update cgit.cgi scan location
We account for the upstream default location as well as the Debian-installed one.
2019-04-15config: support more cgit directives for project lists
Hopefully this gets us closer to matching cgit upstream behavior (which also lacks tests). We'll still need to support macro expansion at some point for compatibility...
2019-04-15cgit: serve static css, logo, favicon directly
We can reduce the configuration needed to run cgit by reusing the static file handling logic of the dumb git HTTP protocol. I hate logos and icons, so don't expect public-inbox.org or 80x24.org to ever have those to waste users' bandwidth with :P But I expect other users to find this useful.
2019-04-15config: support cgit scan-path and scan-hidden-path
project_list support still needs to be done And tests need to be written... :<
2019-04-11doc: fix a few minor typos
2019-04-04www: fix missing cgit fallback after legacy redirects
We need to instate our cgit handler everywhere we use NewsWWW to catch wildcard requests which our normal endpoints do not handle.
2019-04-04spawn: require soft and hard entries in RLIMIT_* handling
Our high-level config already treats single limits as a soft==hard limit for limiters; so stop handling that redundant in the low-level spawn() sub.
2019-04-04cgit: use a dedicated named limiter
I mainly need this to enforce RLIMIT_CPU (and RLIMIT_CORE) when requests come which generate giant, unrealistic diffs. Per-coderepo limiters may be added in the future. But for now, I need to prevent cgit from monopolizing resources on my dinky server.
2019-04-04qspawn: wire up RLIMIT_* handling to limiters
This allows users to configure RLIMIT_{CORE,CPU,DATA} using our "limiter" config directive when spawning external processes.
2019-04-04www: wire up cgit as a 404 handler if cgitrc is configured
Requests intended for cgit are unlikely to conflict with requests to inboxes. So we can safely hand those requests off to cgit.cgi.
2019-04-04cgit: support running cgit as a standalone CGI
We depend on git-http-backend for smart HTTP clone support, however; since cgit does not support smart clones natively. WWW.pm will be able to cascade down to this as a 404 handler in the future.
2019-04-04spawn: support absolute paths
cgit (and most other CGI executables) is not typically installed for use via $PATH, so we'll need to support absolute paths to run it.