about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2017-01-26repobrowse: simplify command generation for git commands
This shortens the code quite a bit at a negligible performance cost, and the diffstat agrees.
2017-01-26add filter for Subject: tags
Some mailing lists add annoying tags into the Subject line which discourages readers from doing proper mail organization on the client side. They also waste precious screen space and attention span. Remove them from our archives to reduce clutter.
2017-01-26watchmaildir: allow arguments for filters
We'll want to allow some degree of configuration for various mailing lists.
2017-01-22repobrowse: git summary view uses psgi_qx
This reduces one synchronous dependency from the hot path, and psgi_return will be used in the future.
2017-01-21repobrowse: preserve newlines in Atom feed
Commit messages are assumed to be displayed in a terminal with a fixed width font, so we must preserve newlines and all whitespace as-is so ASCII art may be displayed properly.
2017-01-21repobrowse: simplify git log parsing implementation
Based on what was done for the Atom feed, this will allow us to simplify state management through metaprogramming and avoid placeholder characters ('D' for decoration) for empty fields.
2017-01-21repobrowse: fix full URL generation in Atom feed
We must not drop the leading slash in the URI. This regression was introduced when we dropped Plack::Request dependency.
2017-01-21repobrowse: avoid extra hash assignments for Atom feed
This should make the code somewhat easier-to-follow.
2017-01-21repobrowse: git Atom feed uses Qspawn->psgi_return
This allows us to wait on "git log" output in a non-blocking manner while being able to throttle on backpressure from slow clients when used with pi-httpd.
2017-01-21repobrowse: git Atom feed uses Qspawn->psgi_qx
This allows pi-httpd to service other I/O while we wait on "git symbolic-ref" to run. And psgi_return will be used in the next commit...
2017-01-21qspawn: better annotate where $qx_cb is called
Hopefully this makes the code easier-to-follow for random readers. This requires a small amount of modification to our one caller, but this is a new, unstable API (as is nearly all of our code).
2017-01-19watchmaildir: limit live importer processes
We don't want to be triggering OOM or swapping on weaker systems when we have dozens of inboxes as potential targets.
2017-01-18mime: avoid SUPER usage in Email::MIME subclass
We must call Email::Simple methods directly in our monkey patch for Email::MIME to call the intended method. Using SUPER in our subclass would instead hit a different, unintended method in Email::MIME. Reported-by: Junio C Hamano <gitster@pobox.com> <xmqq4m0wb43w.fsf@gitster.mtv.corp.google.com>
2017-01-18repobrowse: expath is always defined
Remove an outdated comment while we're at it, too.
2017-01-18http: cast a wider net to prevent circular references
We can more effectly nuke circular references by clearing the entire PSGI $env, not just particular keys, when there are self-referential fields such as "qspawn.response" in our environment.
2017-01-18repobrowse: git snapshot waits for all commands asynchronously
This new asynchronous API, psgi_qx, will allow us to take advantage of non-blocking I/O from even small commands; as those may still need to wait for slow operations.
2017-01-17qspawn: better description
We'll probably use this in a lot of places...
2017-01-17repobrowse: verbose git tree display uses qspawn for ls-tree
For now, qspawn provides resource management for dealing with expensive "git ls-tree" processes.
2017-01-15repobrowse: use qspawn for plain tree views
We may eventually handle tree parsing ourselves (since we already git cat-file), but for now we can rely on ls-tree to give good output and qspawn to manage resource allocation.
2017-01-15repobrowse: git: drop unused diff parsing routines
We don't need these legacy routines anymore and use the newer stream-friendly _sed interface.
2017-01-13httpd/async: stop running command if client disconnects
If an HTTP client disconnects while we're piping the output of a process to them, break the pipe of the process to reclaim resources as soon as possible.
2017-01-13repobrowse: simplify conditional for cat-file input
expath is always defined, even to an empty string, so simplify the conditional for checking it.
2017-01-13rename "GitAsyncRd" to "GitAsync"
This wrapper class actually does both reading and writing, and a shorter name is nicer.
2017-01-13gitasyncrd: pass a reference to Danga::Socket::write
D::S creates a reference for this, anyways, so avoid the extra work by doing it ourselves.
2017-01-13repobrowse: comment describing Git wrapper creation
Metaprogramming can be difficult-to-read after several months, so leave comments in place to describe common usage results of.
2017-01-13repobrowse: port git log view to qspawn streaming interface
This will prevent too many processes from being spawned at once while also allowing us to respond to backpressure from slow clients.
2017-01-11inbox: reinstate periodic cleanup of Xapian and SQLite objects
We may need to do this even more aggressively, since the Xapian database does not always give the latest results. This time, we'll do it without relying on weak references, and instead check refcounts.
2017-01-11repobrowse: make git diff output use qspawn
This is a potentially expensive operation, so we may want to give it it's own limiter channel.
2017-01-11diff: note the dangers of gigantic anchors hash
2017-01-11async: improve and fix out-of-date comments
2017-01-11repobrowse: qspawn + streaming for git commit display
This prevents "git show" processes from monopolizing the system and allows us to better handle backpressure from gigantic commits.
2017-01-11qspawn: fix bad error reporting on errors
Oops :x
2017-01-10introduce PublicInbox::MIME wrapper class
This should fix problems with multipart messages where text/plain parts lack a header. cf. git clone --mirror https://github.com/rjbs/Email-MIME.git refs/pull/28/head In the future, we may still introduce as streaming interface to reduce memory usage on large emails.
2017-01-08githttpbackend: use psgi_return shortcut
This drastically cuts down the amount of duplicate code we have in this branch.
2017-01-08httpd/async: remove needless sysread wrapper
We don't appear to be using it anywhere
2017-01-08Merge remote-tracking branch 'origin/master' into repobrowse
* origin/master: inbox: properly register cleanup timer for git processes search: remove subject_summary searchmsg: favor direct hash access over accessor methods remove incorrect comment about strftime + locales config: allow per-inbox nntpserver inbox: eliminate weaken usage entirely inbox: describe the full key name config: remove unused get() method config: always use namespaced "publicinboxlimiter" qspawn: prepare to support runtime reloading of Limiter http: remove weaken usage, reduce anonsub capture scope httpd/async: remove weaken usage http: fix spelling error watch: watchspam affects all configured inboxes doc: minor updates to design notes
2017-01-07initial git async work
This will allow us to handle network operations while waiting on "git cat-file" to seek and unpack things.
2017-01-07inbox: drop $ref arg for writing destination buffer
We never used this feature, so lets drop it for now since we can have fine-grained memory release with reference counting, anyways.
2017-01-07inbox: properly register cleanup timer for git processes
We still need to cleanup git processes occasionally, since "git cat-file --batch" does not release old packs (and git processes are fairly expensive). For SQLite and Xapian file handles, they should be capable of managing themselves without too much trouble, so lets try keeping them for the lifetime of a process.
2017-01-07search: remove subject_summary
Apparently it never actually got used, and the world seems fine without it, so we can drop it. While we're at it, consider removing our subject_path usage from existence, too. We are not using fancy subject-line based URLs, here.
2017-01-07searchmsg: favor direct hash access over accessor methods
This is faster, smaller, and more straighforward to me with fewer layers of indirection.
2017-01-07remove incorrect comment about strftime + locales
We only need strftime to be locale-independent when generating dates for email and HTTP headers. Purely numeric dates can use strftime for ease-of-readability.
2017-01-07config: allow per-inbox nntpserver
This allows certain inboxes to override the global nntpserver (perhaps under a different domain).
2017-01-07inbox: eliminate weaken usage entirely
We can do a better job initializing the data structure so we no longer need to rely on weak references to cleanup when we ditch the config on reload.
2017-01-07inbox: describe the full key name
Hopefully make this easier for future generations to understand.
2017-01-07config: remove unused get() method
This seems like an unnecessary abstraction, or an abstraction on the wrong level.
2017-01-07config: always use namespaced "publicinboxlimiter"
I'm not sure if we'll ever support sharing a config file with other tools, but maybe we will, and "limiter" is too generic.
2017-01-07qspawn: prepare to support runtime reloading of Limiter
We may allow the {max} value of a limiter to be changed in the future, so lets start accounting for it before we spawn followup processes.
2017-01-04http: remove weaken usage, reduce anonsub capture scope
Avoiding weaken here is no more dangerous than the existing circular refs (e.g. psgix.io) we create and manage throughout the lifetime of the connection. So, trust ourselves to maintain the data structure properly and avoid triggering extra memory usage. While we're at it, avoid having anonymous subroutines capture more variables than necessary to simplify reference auditing.
2017-01-04httpd/async: remove weaken usage
We do not need to use weaken() here, so avoid it to simplify our interactions with Perl; as weaken requires additional storage and (it seems) time complexity.