about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2019-05-07wwwstream: do not load URI.pm
It's unneeded since commit e358bd7a3833f8c5 (2016-07-02) ("inbox: base_url method takes PSGI env hashref instead") So we only depend on URI::Escape from the "URI" CPAN distribution, at the moment.
2019-05-07spawn (Inline::C): fix off-by-one error
Noticed while testing on FreeBSD 11.2 amd64 with the optional Inline::C extension using clang 6.0.0. The end result on FreeBSD was spawning processes failed badly and things were immediately unusable with this enabled. av_len is a misleading API, and I failed to read the API comments in perl:/av.c which state: > Note that, unlike what the name implies, it returns > the highest index in the array, so to get the size of > the array you need to use "av_len(av) + 1". > This is unlike "sv_len", which returns what you would expect. If this bug affected anybody, it would've only affected users using both the optional Inline::C module AND set the PERL_INLINE_DIRECTORY environment variable. That said, I've never seen any evidence of it on Debian GNU/Linux + gcc on any x86 variant. That includes full 64-bit systems, a full 32-bit system, a 64-bit system with 32-bit userspace, across multiple gcc versions since 2016.
2019-05-06index: warn with info about the message as context
This can help users track down the source of warnings when presented with imperfect emails. While we're at it, make the __WARN__ callback in t/v2writable.t a no-op since we don't check for warnings, there.
2019-05-05Merge remote-tracking branch 'origin/wwwlisting'
* origin/wwwlisting: www: support listing of inboxes start depending on Perl 5.10.1+
2019-05-04wwwstream: drop tor2web URL
It was a MITM hazard and been killed off by the DoS subsystem last July: https://marc.info/?i=87d0vwwkbs.fsf@riseup.net
2019-04-27extmsg: escape ampersands in @EXT_URL array
We already escape the user-provided Message-IDs (so there's no security problem AFAIK), but the URL templates which exist in our source code were not escaped properly. This quiets down tidy(1).
2019-04-26viewdiff: do not break out of DSTATE_CTX on /^$/
It seems a common case for mangled patches is editors or MUAs dropping trailing whitespace, and lines matching /^ $/ gets the space dropped to only match /^$/.
2019-04-25cgit: improve handling of cgit data path
Document `publicinbox.cgitdata' config directive, but allow it to be unspecified and/or missing for installations which do not wish to serve static data at all. For users installing cgit from source to their home directory, we can usually infer the cgit data path based on the cgit.cgi binary path, even.
2019-04-24wwwhighlight: deal with no-op highlight
Not all inputs are highlight-able, so reuse the original input and just linkify it if it can't be highlighted.
2019-04-24wwwhighlight: do linkification, too
Otherwise, there's no reason to use this API over highlight(1). Maybe this can be an option in the future; but I'm struggling to find a reason to not do it by default.
2019-04-23view: avoid "1+ messages" in per-message footer of /t/ and /T/
Try to appear gramatically correct and state: "only message in thread" when there's only one known (to us) message in the thread.
2019-04-19www: support listing of inboxes
We will still return a 404 by default to '/' for compatibility with users of Plack::App::Cascade or similar. Inboxes are sorted by modification times to help users detect activity (similar to the /$INBOX/ topic view). New configuration options: * publicinbox.wwwlisting - configure the listing type * publicinbox.<name>.hide - hide a particular inbox from the listing See changes to public-inbox-config.pod for full descriptions of the new options. Requested-by: Leah Neukirchen <leah@vuxu.org> https://public-inbox.org/meta/871sdfzy80.fsf@gmail.com/
2019-04-19start depending on Perl 5.10.1+
I mainly want to start using the '//' (defined-or) operator to simplify code, and Perl 5.10.1 is roughly a decade old at this point. "given/when" would've be nice, but it's future is in doubt AFAIK. I also started using the 'parent' module in WwwHighlight, and 'autodie' in UserContent.pm, both of which were only distributed with Perl since 5.10.1; and testing with ancient versions/distros is time-consuming. Anyways, I think this a small-enough jump to not break any existing installations, given we already depend on fairly recent versions of git and Xapian. Maybe we can use more newish Perl features in the future...
2019-04-18linkify: require parentheses pairs in URLs
Dangling parentheses with trailing punctuation usually means the parentheses is not intended as part of the URL.
2019-04-18view: show "(no subject)" consistently in HTML
Empty subjects ("") and undefined Subjects: are now both displayed as "(no subject)" for now.
2019-04-18linkify: don't get confused by URLs in Perl code, at least
The URLs at the top of WwwStream.pm weren't getting linkified correctly.
2019-04-18inbox: add `modified' sub
For inboxes with SQLite enabled (all v2, and probably most v1); we can use the overview DB to get the timestamp of the latest message. It's faster than scanning git branches for commit times, but not always the same.
2019-04-18git: calculate modified time of repository
This will be used for generating an HTML listing for v1 inboxes, at least. The logic for this follows that of grokmirror, and we may dynamically generate manifest.js.gz natively...
2019-04-18config: use '$ibx' instead of '$rv' to denote Inbox objects
Followup-to: 6e6f7999361925e4 ("cleanup: use '$ibx' consistently when referring to Inbox refs")
2019-04-16import: prefix 'inbox' with '-' to denote Inbox ref
This is for consistency with other fields which follow this pattern w.r.t. field-naming when referring to internal fields.
2019-04-16cleanup: use '$ibx' consistently when referring to Inbox refs
'$inbox' is more human-readable, so that is for the more human-readable name in most cases. Making our variable naming more consistent should make the code easier-to-review and harder to screw up.
2019-04-16www: remove unnecessary Git object reference
We access the Git object via the Inbox object nowadays, so there's no point in having a shortcut to it, anymore.
2019-04-15config: fix regression in repo.path => coderepo.dir mapping
We parse cgitrc for "repo.path", while we use "coderepo.dir" to mean the same thing for non-cgit users. So I ended up confusing myself, here. But then again, git uses "--git-dir" and "GIT_DIR", so I suspect "dir" is the better choice than "path", here
2019-04-15viewdiff: document constants
We'll be building off of this for showing diffs in the coderepo views.
2019-04-15config: support more cgit directives for project lists
Hopefully this gets us closer to matching cgit upstream behavior (which also lacks tests). We'll still need to support macro expansion at some point for compatibility...
2019-04-15cgit: serve static css, logo, favicon directly
We can reduce the configuration needed to run cgit by reusing the static file handling logic of the dumb git HTTP protocol. I hate logos and icons, so don't expect public-inbox.org or 80x24.org to ever have those to waste users' bandwidth with :P But I expect other users to find this useful.
2019-04-15config: support cgit scan-path and scan-hidden-path
project_list support still needs to be done And tests need to be written... :<
2019-04-04www: fix missing cgit fallback after legacy redirects
We need to instate our cgit handler everywhere we use NewsWWW to catch wildcard requests which our normal endpoints do not handle.
2019-04-04spawn: require soft and hard entries in RLIMIT_* handling
Our high-level config already treats single limits as a soft==hard limit for limiters; so stop handling that redundant in the low-level spawn() sub.
2019-04-04cgit: use a dedicated named limiter
I mainly need this to enforce RLIMIT_CPU (and RLIMIT_CORE) when requests come which generate giant, unrealistic diffs. Per-coderepo limiters may be added in the future. But for now, I need to prevent cgit from monopolizing resources on my dinky server.
2019-04-04qspawn: wire up RLIMIT_* handling to limiters
This allows users to configure RLIMIT_{CORE,CPU,DATA} using our "limiter" config directive when spawning external processes.
2019-04-04www: wire up cgit as a 404 handler if cgitrc is configured
Requests intended for cgit are unlikely to conflict with requests to inboxes. So we can safely hand those requests off to cgit.cgi.
2019-04-04cgit: support running cgit as a standalone CGI
We depend on git-http-backend for smart HTTP clone support, however; since cgit does not support smart clones natively. WWW.pm will be able to cascade down to this as a 404 handler in the future.
2019-04-04spawn: support absolute paths
cgit (and most other CGI executables) is not typically installed for use via $PATH, so we'll need to support absolute paths to run it.
2019-04-04githttpbackend: check for other errors and relax CRLF check
Reads to git-http-backend(1) could fail or EOF prematurely, so we must be ready for that case. Furthermore, cgit (and possibly other CGI) uses LF instead of CRLF, so support those programs, too.
2019-04-04githttpbackend: move more psgi.input handling into subroutine
This will be useful for other CGI wrappers we make. This also fixes a bug with some PSGI servers which did not present a real IO::Handle in the psgi.input env field.
2019-04-04support publicinbox.cgitrc directive
We can save admins the trouble of declaring [coderepo "..."] sections in the public-inbox config by parsing the cgitrc directly. Macro expansion (e.g. $HTTP_HOST) expansion is not supported, yet; but may be in the future.
2019-04-04spawn: support RLIMIT_CPU, RLIMIT_DATA and RLIMIT_CORE
We'll be spawning cgit and git-diff, which can take gigantic amounts of CPU time and/or heap given the right (ermm... wrong) input. Limit the damage that large/expensive diffs can cause.
2019-04-04viewvcs: match 8000-byte lookup for git
No need to scan the entire string, but prefer to match git behavior. This might be faster if/when Perl can create substrings efficiently using CoW. Fix a 80-column violation while we're at it.
2019-04-04viewvcs: preliminary support for showing non-blobs
Eventually, we'll have special displays for various git objects (commit, tree, tag). But for now, we'll just use git-show to spew whatever comes from git.
2019-04-04git: add "commit_title" method
This will be useful for extracting titles/subjects from commit objects when displaying commits.
2019-04-02githttpbackend: serve $GIT_DIR/info/attributes
This will be useful for reproducibility when mirroring coderepos and generating diffs.
2019-04-02viewvcs: check for premature EOF from git-cat-file
Not entirely sure what is causing this, but it appears to be causing infinite loops when attempting to display certain blobs. Fortunately, the fair scheduling of public-inbox-httpd prevented this from becoming a real problem aside from increasing CPU usage.
2019-03-12config: ignore missing config files
There's no reason for us to have git-config(1) warn users when a config file is entirely missing.
2019-03-08v2writable: index_sync adds new epochs to alternates
Newly-cloned epochs need to be in alternates file of all.git for the web and NNTP interfaces to work. So allow invocations of "public-inbox-index" to idempotently ensure the epoch is visible from the all.git repo.
2019-03-08v2writable: hoist out fill_alternates
We'll be using this sub to fill $GIT_DIR/objects/info/alternates if somebody uses clone --mirror, too
2019-02-27v2writable: fix epoch rollover on incremental imports
All of our internal epoch rollover calculations are done using the estimated unpacked (and uncompressed) size of the repo. The importer instance needs to check that unpacked size before selecting an epoch when an epoch already has packed data. This bug did not impact the initial mass imports since we only initialize the Import instance once-per-epoch and did not need to take existing epochs into account. Tested manually with -mda on a local clone of LKML Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2019-02-23www: prevent '!important' in BOFH-specified CSS
CSS specified by the BOFH must never take precedence over what a user sets in userContent.css.
2019-02-23usercontent.pm: set "!important" to override BOFH prefs
The sample userContent.css needs a higher priority than what the BOFH specifies. In other words, user preference must ALWAYS take precedence. Reported-by: Dmitry Alexandrov <321942@gmail.com> cf. https://public-inbox.org/meta/87mumn4kx8.fsf@gmail.com/
2019-02-13ensure bytes::length is available to callers
We were relying on Danga::Socket using the "bytes" pragma, previously. Nowadays, the "bytes" pragma is not recommended in general, but bytes::length remains acceptable for getting the byte-size of a scalar.