about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2019-04-04viewvcs: preliminary support for showing non-blobs
Eventually, we'll have special displays for various git objects (commit, tree, tag). But for now, we'll just use git-show to spew whatever comes from git.
2019-04-04git: add "commit_title" method
This will be useful for extracting titles/subjects from commit objects when displaying commits.
2019-04-02githttpbackend: serve $GIT_DIR/info/attributes
This will be useful for reproducibility when mirroring coderepos and generating diffs.
2019-04-02viewvcs: check for premature EOF from git-cat-file
Not entirely sure what is causing this, but it appears to be causing infinite loops when attempting to display certain blobs. Fortunately, the fair scheduling of public-inbox-httpd prevented this from becoming a real problem aside from increasing CPU usage.
2019-03-12config: ignore missing config files
There's no reason for us to have git-config(1) warn users when a config file is entirely missing.
2019-03-08v2writable: index_sync adds new epochs to alternates
Newly-cloned epochs need to be in alternates file of all.git for the web and NNTP interfaces to work. So allow invocations of "public-inbox-index" to idempotently ensure the epoch is visible from the all.git repo.
2019-03-08v2writable: hoist out fill_alternates
We'll be using this sub to fill $GIT_DIR/objects/info/alternates if somebody uses clone --mirror, too
2019-02-27v2writable: fix epoch rollover on incremental imports
All of our internal epoch rollover calculations are done using the estimated unpacked (and uncompressed) size of the repo. The importer instance needs to check that unpacked size before selecting an epoch when an epoch already has packed data. This bug did not impact the initial mass imports since we only initialize the Import instance once-per-epoch and did not need to take existing epochs into account. Tested manually with -mda on a local clone of LKML Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
2019-02-23www: prevent '!important' in BOFH-specified CSS
CSS specified by the BOFH must never take precedence over what a user sets in userContent.css.
2019-02-23usercontent.pm: set "!important" to override BOFH prefs
The sample userContent.css needs a higher priority than what the BOFH specifies. In other words, user preference must ALWAYS take precedence. Reported-by: Dmitry Alexandrov <321942@gmail.com> cf. https://public-inbox.org/meta/87mumn4kx8.fsf@gmail.com/
2019-02-13ensure bytes::length is available to callers
We were relying on Danga::Socket using the "bytes" pragma, previously. Nowadays, the "bytes" pragma is not recommended in general, but bytes::length remains acceptable for getting the byte-size of a scalar.
2019-02-07nntp: get rid of long_response_limit constant
Unused since commit b8c41362f2a5c8fcc6b1846a79c72bfa77565297 ("nntp: simplify the long_response API")
2019-02-07http: cleanup partial-write handling on readonly values
Don't bother assigning to $_[1]; just let Danga::Socket do its thing since $_[1] should be out-of-scope soon.
2019-02-07daemon: favor Socket.pm in Perl 5.14+ for IPv6
Users on Perl 5.14+ are common, so we can try the bundled Socket (not "Socket6") module before attempting Socket6 for IPv6.
2019-02-07daemon: die properly when Socket6 is missing
We need to keep Unix-socket-only httpd instances working without Socket6. This fixes t/httpd-unix.t with Socket6 uninstalled.
2019-02-05Merge remote-tracking branch 'origin/help-color'
* origin/help-color: wwwtext: inline sample CSS and use highlight hlmod: support "```$LANG" blocks in text hlmod: do_hl* performs src_escape immediately hlmod: make into a singleton hlmod: hoist out do_hl_lang sub viewvcs: cleanup utf8 handling
2019-02-05solvergit: include the $oid_want tmpdir name
This can help admins diagnose problems with SolverGit, since qspawn logs the failed "git apply" command-line in stderr. (or it can waste admins' time because sometimes there's crap mail clients which mangle patches)
2019-02-05wwwtext: inline sample CSS and use highlight
For user documentation regarding CSS; showing users the sample CSS with comments is probably more helpful than having standalone documentation on CSS classes.
2019-02-05hlmod: support "```$LANG" blocks in text
This is compatible with Markdown; but we still keep the WYSIWYG nature of plain-text with this. This is only intended for use with our documentation. Enabling any type of Markdown support for emails can lead to incompatibilities or interopability problems with alternative implementations.
2019-02-05hlmod: do_hl* performs src_escape immediately
We want to be able to take advantage of this in other modules
2019-02-05hlmod: make into a singleton
It turns out there's no point in having multiple instances of this or having to worry about destruction or destruction ordering. This will make it easier to reuse the one instance we have across different modules.
2019-02-05hlmod: hoist out do_hl_lang sub
We'll want to use to support highlighting syntax used by Markdown and possibly other markup languages (while retaining the raw plain-text layout and formatting).
2019-02-05viewvcs: cleanup utf8 handling
Favor in-place utf8::decode since it's a bit faster without method dispatch overhead; and don't care about validity just yet. HlMod->do_hl itself should return "utf8" strings, since other parts of our code can use it, so it's not the job of ViewVCS to post-process HlMod output.
2019-02-04viewdiff: group path match to not be confused by "/dev/null"
Leaving out parentheses caused transitions to state="del" or state="add" to be misidentified. cf. https://public-inbox.org/meta/20190204105454.GG10587@szeder.dev/ Reported-by: SZEDER Gábor <szeder.dev@gmail.com>
2019-02-01newswww: add /$MESSAGE_ID global redirector endpoint
This is the fallback for the normal WWW endpoint. Adding this to the top-level seems to be alright, since lynx and w3m both understand nntp://<HOSTNAME>/<Message-ID> anyways. If newsgroup and inbox names conflict, then consider it the fault of the original sender. Since NewsWWW is intended to support buggy linkifiers in mail clients, they can interpret nntp:// URLs as http://<HOSTNAME>/<Message-ID> Inbox ordering from the config file is preserved since commit cfa8ff7c256e20f3240aed5f98d155c019788e3b ("config: each_inbox iteration preserves config order"), so admins can rely on that to configure how scanning works. Requested-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org> cf. https://public-inbox.org/meta/20190107190719.GE9442@pure.paranoia.local/ nntp://news.public-inbox.org/20190107190719.GE9442@pure.paranoia.local
2019-02-01viewdiff: support renames and long paths in diffstat anchors
This is best-effort, but works well-enough in practice for projects which use shell-friendly filenames as well as the long path names for some Linux kernel selftests.
2019-02-01viewdiff: escape HTML ampersand for renames
For URLs we generate, we need to escape '&' in query parameters for correctness.
2019-02-01viewvcs: allow '0' as a valid filename for blob downloads
Only to be pedantic...
2019-02-01linkify: support proto://hostname without trailing slash
Sometimes users will write "http://example.com" without the trailing slash, which every browser and tool I've tested seems to understand.
2019-02-01view: simplify quote splitting
Perl "split" can capture and group in the regexp itself, so rely on that to shorten our code. Comparing the /T/ HTML output of a thread from hell (on LKML with 1356 messages) reveals no difference in the rendered result. Only the HTML source differs in newline placement before/after the closing </span> This allows a minor speedup on my X32 Thinkpad @ 1.6GHz with the aforementioned LKML thread from hell: before: 3.67s after: 3.55s
2019-02-01view: fix broken hunk header hrefs in Atom feeds
We use absolute URLs in the Atom feeds (to ease syndication/mirroring), so hunk headers need to point to the solver URLs.
2019-02-01view: diffstat anchors for multi-message/attachment views
diffstat <-> ^diff anchors work within the same attachment or message while in HTML views which display multiple messages.
2019-02-01viewdiff: diffstat links to diff anchors
This can be helpful for reviewing larger patches which span across several files on the permalink (/$MESSAGE_ID/) HTML page. More work will be needed to get this working for the /T/ and /t/ pages which show multiple emails, as the filename-based anchors will conflict at the moment.
2019-02-01hval: routines for attribute escaping
We'll use HTML attributes + anchor links to link to filenames in coming commits.
2019-01-31Merge remote-tracking branch 'origin/purge'
* origin/purge: implement public-inbox-purge tool v2writable: read epoch on purge v2writable: cleanup processes when done v2writable: purge ignores non-existent git epoch directories v2writable: ->purge returns undef on no-op import: purge: reap fast-export process hoist out resolve_repo_dir from -index
2019-01-31doc: remove completed TODO items
2019-01-31config: tiny cleanup to use _array() sub
2019-01-31qspawn: documentation updates
This will become critical for future changes to display git commits, diffs, and trees. Use "qspawn.wcb" instead of "qspawn.response" to enhance readability.
2019-01-31inbox: drop psgi.url_scheme requirement from base_url
This will make it easier to make command-line tools from SolverGit.
2019-01-31viewvcs: support streaming large blobs
Forking off git-cat-file here for streaming large blobs is reasonably efficient, at least no worse than using git-http-backend for serving clones. So let our limiter framework deal with it. git itself isn't great for large files, and AFAIK there's no stable/widely-available mechanisms for reading smaller chunks of giant blobs in git itself. Tested with some giant GPU headers in the Linux kernel.
2019-01-31solvergit: allow shorter-than-necessary OIDs from user
We can rely on git to disambiguate, here; because sometimes shorter OIDs can be unambiguous even if we only resolved the longer one.
2019-01-31solvergit: allow searching on longer-than-needed OIDs
public-inbox can only index the abbreviated object_ids in emails, not the full or even longer-than-necessary object_ids. So retry failed object_ids if they're longer than 7 hex characters.
2019-01-31inbox: perform cleanup of Git objects for coderepos
Otherwise, long-running but idle git processes may keep unlinked packs around indefinitely and waste disk space.
2019-01-30solvergit: don't confuse Xapian with ".." in filenames
Xapian will interpret ".." as ranges, even quoted phrases. So break up words on ".." since punctuation (AFAIK) is not searchable, anyways.
2019-01-30git: use "git rev-parse --git-path"
Using git worktrees was causing t/solver_git.t to fail on me.
2019-01-30Merge remote-tracking branch 'origin/viewvcs' into master
* origin/viewvcs: (66 commits) solvergit: deal with alternative diff prefixes solvergit: extract mode from diff headers properly solvergit: avoid "Wide character" warnings solvergit: do not show full path names to "git apply" css/216dark: add comments and tweak highlight colors viewvcs: avoid segfault with highlight.pm at shutdown solvergit: do not solve blobs twice t/check-www-inbox: disable history t/check-www-inbox: don't follow mboxes t/check-www-inbox: replace IPC::Run with PublicInbox::Spawn hval: add src_escape for highlight post-processing viewvcs: wire up syntax-highlighting for blobs hlmod: disable enclosing <pre> tag t/hl_mod: extra check to ensure we escape HTML wwwhighlight: read_in_full returns undef on errors solver: crank up max patches to 9999 viewvcs: do not show final error message twice qspawn: decode $? for user-friendliness solver: reduce "git apply" invocations solver: hold patches in temporary directory ...
2019-01-30view: remove unused _msg_date sub
Not needed since commit 956abe9ad5f13a0d1755262be412d6a54fda72e9 ("view: depend on SearchMsg for Message-ID")
2019-01-30httpd: a few comments about some fields we set
Removing 'psgix.input.buffered' could be a possibility in the future.
2019-01-30solvergit: deal with alternative diff prefixes
At least, without extra directory levels, since git-diff supports --src-prefix and --dst-prefix, and /git/6aa8857a11/s/ uses it...
2019-01-30solvergit: extract mode from diff headers properly
grep() won't set $1, so use "=~", instead.