about summary refs log tree commit homepage
DateCommit message (Collapse)
2019-01-29css/216dark: add comments and tweak highlight colors
Overkill, but "highlight" supports single-line comments (slc) independently of multi-line comments (com); but we'll use the same color for that. We'll also use #0f0 instead of #0ff for "kwb" (keyword class "b") since blue shades are prevalent in <a> links and comments, while green was unused.
2019-01-29viewvcs: avoid segfault with highlight.pm at shutdown
Proper ordering of destruction seems required to avoid segfaults at shutdown.
2019-01-29solvergit: do not solve blobs twice
In some cases, a file may ping-pong between blob IDs in the same message when reverts occur. So break out of this early. This doesn't account for different abbreviations, but the limited variations of abbreviations should alleviate the problem.
2019-01-29t/check-www-inbox: disable history
WWW::Mechanize keeps an infinitely large stack, which was leading to OOM errors on my system.
2019-01-29t/check-www-inbox: don't follow mboxes
They can be extremely large with no limit, so can lead to OOM errors.
2019-01-29t/check-www-inbox: replace IPC::Run with PublicInbox::Spawn
Because WWW::Mechanize uses truckload of memory, fork needs to prepare all that memory for CoW, which ends up bailing with ENOMEM.
2019-01-28hval: add src_escape for highlight post-processing
We need to post-process "highlight" output to ensure it doesn't contain odd bytes which cause "wide character" warnings or require odd glyphs in source form.
2019-01-27viewvcs: wire up syntax-highlighting for blobs
And update 216dark.css to match a color scheme I'm used to; which is fairly minimal and doesn't use all the classes "highlight" provides.
2019-01-27hlmod: disable enclosing <pre> tag
We already have a <pre> tag in ViewVCS, and nesting <pre> inside the pre-existing <pre> overrides the "white-space:pre" we use to align line numbers.
2019-01-27t/hl_mod: extra check to ensure we escape HTML
Otherwise, it's open season on our users :<
2019-01-27wwwhighlight: read_in_full returns undef on errors
The return value of "print" is not undef for Perl IO::Handle.
2019-01-27solver: crank up max patches to 9999
Might as well, since the only constraint is filesystem space for temporary files for public-inbox-httpd users. -httpd can fairly share work across clients with our use of psgi_qx; and there's a recent patch series in git@vger with 64 patches in sequence.
2019-01-27viewvcs: do not show final error message twice
SolverGit::ERR already writes the exception to the debug log before calling {user_cb}, so there's no need for viewvcs to append it.
2019-01-27qspawn: decode $? for user-friendliness
The raw value of $? isn't very useful, generally.
2019-01-27solver: reduce "git apply" invocations
"git apply" is capable of applying multiple patches in one invocation, so give it multiple patches on the command-line now that we no longer rely on anonymous file handles to hold patches. This cuts down a 64-patch series on git@vger from ~1s to ~800ms with vfork spawn enabled using Inline::C.
2019-01-27solver: hold patches in temporary directory
We can avoid bumping up RLIMIT_NOFILE too much by storing patches in a temporary directory. And we can share this top-level directory with our temporary git repository. Since we no longer rely on a working-tree for git, we are free to rearrange the layout and avoid relying on the ".git" convention and relying on "git -C" for chdir. This may also ease porting public-inbox to older systems where git does not support "-C" for chdir.
2019-01-26solver: rewrite to use Qspawn->psgi_qx and pi-httpd.async
The psgi_qx routine in the now-abandoned "repobrowse" branch allows us to break down blob-solving at each process execution point. It reuses the Qspawn facility for git-http-backend(1), allowing us to limit parallel subprocesses independently of Perl worker count. This is actually a 2-3% slower a fully-synchronous execution; but it is fair to other clients as it won't monopolize the server for hundreds of milliseconds (or even seconds) at a time.
2019-01-26view: swap CRLF for LF in HTML output
It makes no difference to browsers aside from saving a few bytes; and this means we won't have to worry about extra '%0D' showing up in links to solver.
2019-01-26t/qspawn.t: psgi_qx stderr test
2019-01-22qspawn: implement psgi_qx
This new asynchronous API, will allow us to take advantage of non-blocking I/O from even small commands; as those may still need to wait for slow operations.
2019-01-22httpd/async: stop running command if client disconnects
If an HTTP client disconnects while we're piping the output of a process to them, break the pipe of the process to reclaim resources as soon as possible.
2019-01-22qspawn|httpd/async: improve and fix out-of-date comments
2019-01-22qspawn|getlinebody: support streaming filters
This is intended for wrapping "git show" and "git diff" processes in the future and to prevent it from monopolizing callers. This will us to better handle backpressure from gigantic commits.
2019-01-22qspawn: implement psgi_return and use it for githttpbackend
Was: ("repobrowse: port patch generation over to qspawn") We'll be using it for githttpbackend and maybe other things.
2019-01-22httpd/async: remove needless sysread wrapper
We don't appear to be using it anywhere
2019-01-21t/check-www-inbox: trap SIGINT for File::Temp destruction
Otherwise, temporary GDBM files don't get unlinked when I SIGINT the process.
2019-01-21hval: split out escape sequences to a separate table
We'll want to handle those escape sequences independently, "highlight" already does HTML escaping.
2019-01-21highlight: initial wrapper and PSGI service
I'll probably expose the PSGI service for cgit; but it could be useful to others as well.
2019-01-21t/check-www-inbox: warn on missing Content-Type
Oops, I might've left it out, somewhere.
2019-01-20config: each_inbox iteration preserves config order
For cross-inbox Message-ID resolution; having some sort of stable ordering makes the most sense. Relying on the order of the config file seems most natural and allows us to avoid introducing yet another configuration knob.
2019-01-20solver: remove extra "^index $OID..$OID" line
It was harmless, besides wasting space and memory.
2019-01-20solver: force quoted-printable bodies to LF
..if the Email::MIME ->crlf is LF. Email::MIME::Encodings forces everything to CRLF on quoted-printable messages for RFC-compliance; and git-apply --ignore-whitespace seems to miss a context line which is just "\r\n" (w/o leading space).
2019-01-20viewvcs: disable white-space prewrap in blob view
We need to keep line-numbers from <a> tags synced to the actual line numbers in the code when working in smaller viewports. Maybe I only work on reasonable projects, but excessively long lines seem to be less of a problem in code than they are in emails.
2019-01-20viewdiff: do not link to 0{7,40} blobs (again)
We must reset diff context when starting a new file; and we must check for all-zeroes object_ids as the post-image correctly.
2019-01-20t/check-www-inbox: use xmlstarlet to validate Atom if available
I almost forgot about this script; but remembering to test it against real-world data can be useful to hunt for bugs.
2019-01-20viewdiff: quote attributes for Atom feed
We still need to use XHTML the Atom feed, and XHTML requires attributes to be quoted, whereas HTML 5 does not.
2019-01-20viewdiff: cleanup state transitions a bit
This makes things less error-prone and allows us to only highlight the "@@ -\S+ \+\S+ @@" part of the hunk header line, without highlighting the function context. This more closely matches the coloring behavior of git-diff(1)
2019-01-20viewdiff: support diff-highlighting w/o coderepo
Having diff highlighting alone is still useful, even if blob-resolution/recreation is too expensive or unfeasible.
2019-01-20$INBOX/_/text/color/ and sample user-side CSS
Since we now support more CSS classes for coloring, give this feature more visibility.
2019-01-20www: admin-configurable CSS via "publicinbox.css"
Maybe we'll default to a dark theme to promote energy savings... See contrib/css/README for details
2019-01-20solver: restore diagnostics and deal with CRLF
Apparently Email::MIME returns quoted-printable text with CRLF. So use --ignore-whitespace with git-apply(1) and ensure we don't capture '\r' in pathnames from those emails. And restore "$@" dumping when we die while solving.
2019-01-20view: enforce trailing slash for /$INBOX/$OID/s/ endpoints
As with our use of the trailing slash in $MESSAGE_ID/T/ and '$MESSAGE_ID/t/' endpoints, this for 'wget -r --mirror' compatibility as well as allowing sysadmins to quickly stand up a static directory with "index.html" in it to reduce load.
2019-01-20solver: add a TODO note about making this fully evented
Applying a 100+ patch series can be a pain and lead to a wayward client monopolizing the connection. On the other hand, we'll also need to be careful and limit the number of in-flight file descriptors and parallel git-apply processes when we move to an evented model, here.
2019-01-20solver: note the synchronous nature of index preparation
It's not likely to be worth our time to support a callback-driven model for something which happens once per patch series.
2019-01-20solver: break @todo loop into a callback
This will allow each patch search via Xapian to "yield" the current client in favor of another client in the PSGI web interface for fairness.
2019-01-20solver: simplify control flow for initial loop
We'll be breaking this up into several steps, too; since searching inboxes for patch blobs can take 10s of milliseconds for me.
2019-01-20solver: switch patch application to use a callback
A bit messy at the moment, but we need to break this up into smaller steps for fairness with other clients, as applying dozens of patches can take several hundred milliseconds.
2019-01-20solver: break up patch application steps
We want more fine-grained scheduling for PSGI use, as the patch application step can take hundreds of milliseconds on my modest hardware
2019-01-20solver: more verbose blob resolution
Help users find out where each step of the resolution came from. Also, we must clean abort the process if we have missing blobs. And refine the output to avoid unnecessary braces, too.
2019-01-20git: support 'ambiguous' result from --batch-check
David Turner's patch to return "ambiguous" seems like a reasonable patch for future versions of git: https://public-inbox.org/git/672a6fb9e480becbfcb5df23ae37193784811b6b.camel@novalis.org/