about summary refs log tree commit homepage
path: root/lib/PublicInbox
DateCommit message (Collapse)
2016-06-25www_stream: linkify cloneurl entries if they're HTTP/HTTPS
They may be other public-inbox instances which are browseable, so provide a link to them to encourage their use as clones.
2016-06-25view: safer and optional quoting for --in-reply-to arg
Angle brackets around the --in-reply-to= arg for git send-email has been optional since git v1.5.3.2, so strip them and make the command-line argument easier-to-type.
2016-06-25address: remove Address::from_name
Address::names is sufficient to handle what from_name did.
2016-06-25view: show To/Cc destinations in conversation view
It is important to show the decentralized nature of communication in our web views.
2016-06-25address: beef up the module with name list extaction
We may remove from_name in the future. ...And disallow quotes in email addresses. Technically I believe they're allowed, but they're definitely uncommon and unlikely to show up in legitimate mail.
2016-06-25inbox: do not weaken already-weak refs
This quiets a (hopefully harmless) warning when a ref remains alive through several expiry timeouts.
2016-06-25http: cork chunked responses for small savings
This only affects Linux users with MSG_MORE support. We can avoid extra TCP overhead for sub-optimal chunk sizes by using MSG_MORE even with chunk trailers under Linux. This breaks real-time apps which require <= 200ms latency for streaming small packets (e.g. implementing "tail -F"), but the public-inbox WWW code does not (and will never) do such things.
2016-06-24mbox: reduce small packets for gzipped mboxes
We want to avoid sending 10 or 20-byte gzip headers as separate TCP packets to reduce syscalls and avoid wasting bandwidth.
2016-06-24githttpbackend: shallow clone workaround
Apparently git-http-backend exits with a non-zero status on shallow clones (due to git-upload-pack), so there is a to-be-fixed bug in git.git http://mid.gmane.org/20160621112303.GA21973@dcvr.yhbt.net http://mid.gmane.org/20160621121041.GA29156@sigill.intra.peff.net
2016-06-24evcleanup: micro-optimize asap function
Instead of relying on a timer with immediate callback, arm a pipe to watch for writability, ensuring the callback always fires.
2016-06-24http: always yield on getline/body
We want to maximize fairness for large responses which may download the entire mbox.
2016-06-24watch_maildir: ignore Trash and Drafts, support Dovecot
Trashed messages and drafts are probably not intended for importing, so do not import them. Dovecot uses extra flags via lowercase letters, so we must support those (as that's the server I use).
2016-06-24watch_maildir: implement optional spam checking
Mailing lists I watch and mirror may not have the best spam filtering, and an extra layer should not hurt.
2016-06-24watch_maildir: rename _check_spam => _remove_spam
We do not actually do spam checking, here; but will do spam checking before adding a message in the future.
2016-06-24document Filesys::Notify::Simple dependency
And improve documentation for existing dependencies, too.
2016-06-24split out spamcheck/spamc to its own module.
This should hopefully make it easier to try other anti-spam systems (or none at all) in the future.
2016-06-24implement ListMirror SpamAssassin plugin
When running mailing list mirrors, one needs to be careful spammers do not try to sidestep the list server we want to mirror from and inject email into our mail directly by setting the appropriate list headers (e.g. "X-Mailing-List" or "List-Id"). We trust the top-most Received: header is the one our own mail server got the mail from. Bcc:-ing a public mailing list is a very likely indicator of spam in my experience, so throw in an extra rule mark it. While public-inbox-mda rejects Bcc: entirely, public-inbox-watch needs to mirror lists which allow Bcc. ==> list_mirror.cf <== loadplugin PublicInbox::SaPlugin::ListMirror ifplugin PublicInbox::SaPlugin::ListMirror header LIST_MIRROR_RECEIVED eval:check_list_mirror_received() describe LIST_MIRROR_RECEIVED Received does not match trusted list server score LIST_MIRROR_RECEIVED 10 header LIST_MIRROR_BCC eval:check_list_mirror_bcc() describe LIST_MIRROR_BCC Mailing list was Bcc-ed score LIST_MIRROR_BCC 1 endif ==> ~/.spamassassin/user_prefs <== ifplugin PublicInbox::SaPlugin::ListMirror list_mirror X-Mailing-List git@vger.kernel.org *.kernel.org git@vger.kernel.org endif
2016-06-23view: update git-send-email URL
I've setup my own mirror on https://git-htmldocs.bogomips.org/ since kernel.org is no longer updated, and I don't believe in endorsing commercial hosting sites or interests. Unfortunately, it still contains JavaScript and CSS, but it's generated by AsciiDoc. Oh well, it's all Free Software, at least. Junio C Hamano writes: > On Wed, Jun 22, 2016 at 12:00 PM, Eric Wong <e@80x24.org> wrote: > > Just wondering, who updates > > https://kernel.org/pub/software/scm/git/docs/ > > and why hasn't it been updated in a while? > > (currently it says Last updated 2015-06-06 at the bottom) > > Nobody. It is too cumbersome to use their upload tool to update many > files (it is geared towards updating a handful of tarballs at a time). ref: http://mid.gmane.org/20160622190018.GA786@dcvr.yhbt.net/ http://mid.gmane.org/CAPc5daUiUv-EEv7ouQ=K+Q8S64QVV5wn4H6+TuF0wLeo123K5Q@mail.gmail.com http://mid.gmane.org/20160622214811.GA19633@dcvr.yhbt.net
2016-06-22view: allow thread generation to EOF on errors
We want to ensure the actual message gets shown, and less important info does not destroy things.
2016-06-22view: fix topic threading when ghosts are present
This fixes a bug where a message replying to a ghost would accidentally be added to the wrong topic in the index/topic view. Before commit 76d8f68dc273e54809ad69cfe49e141003f790ef ("view: avoid recursion in topic index"), we would refuse to indent a topic which started with a ghost which hid the problem. Now we inform the user the thread started elsewhere.
2016-06-21view: support non-existent Subjects for permalink titles
Showing "(no subject)" seems like a common fallback for messages without a Subject header.
2016-06-21search: support Subject:-less messages
Some mailing lists allow empty Subject headers and we shall support searching and indexing them.
2016-06-21daemon: disable SIGWINCH unless explicitly daemonized
Checking stdin/stdout/stderr is not sufficient as the daemon without setsid can still be under the control of a terminal. Unfortunately this means systemd users cannot use SIGWINCH, either.
2016-06-21searchidx: merge old thread id from ghosts
We failed to discard old thread IDs when vivifying ghosts due to out-of-order message arrival. This rectifies the failure and will trigger a re-index.
2016-06-21searchidx: simplify ghost creation
Remove some worthless parameters and redundant no-ops to make the next (important) patch easier-to-review.
2016-06-21view: common thread walking interface
Since we have a common pattern, for walking threads, extract it into a function and reduce the amount of code we haev. This will make it easier to switch to an event-driven interface for getline, too.
2016-06-21view: avoid recursion in topic index
Recursion can cause problems, so do our best to avoid it even in the topic index.
2016-06-21searchview: remove recursion from thread view
As before, recursion can cause problems sooner than unshifting objects into the head of a queue.
2016-06-21view: remove recursion from expanded thread view
This should let us generate HTML for arbitrarily deep threads without blowing the stack. How it renders on the client side is another matter...
2016-06-21view: remove recursion from thread skeleton dump
This should help prevent OOM errors from arbitrarily deep threads and will make our streaming interface easier-to-implement.
2016-06-21view: remove dst parameter from thread skeleton dump
We can stuff this into the state hash to reduce stack size and hopefully improve readability.
2016-06-21view: remove upfx parameter from thread skeleton dump
This makes the string creation somewhat simpler hopefully makes the code easier-to-reason with.
2016-06-21spawn: improve error checking for fork failures
fork failures are unfortunately common when Xapian has gigabytes and gigabytes mmapped.
2016-06-20searchview: use inbox->msg_by_mid
This abstracts out the path lookup logic and and allow us potentially allow different heads in the same repository. We may also bypass slow tree name lookups in the future by storing the raw blob ID in the Xapian document. Followup-to: 4b313dc74bc9 ("feed: various object-orientation cleanups")
2016-06-20searchview: fix Atom dump
Ugh, and I will still need to write better tests for this (and a billion other things :x) Fixes: 4b313dc74bc9 ("feed: various object-orientation cleanups")
2016-06-20nntp: use lookup_mail instead of lookup_message
lookup_mail is safer since it won't inadvertently load ghosts.
2016-06-20www: improve topic view by scanning for ghosts
This should help avoid having too many fake top-level messages in the topic view since we only have a partial window for threading results.
2016-06-20address: remove quotes around names
They're needless for actual display once outside of email headers. But we will still show them when displaying mock headers in the permalink view.
2016-06-20inbox: move field population logic to initializer
Inboxes are normally created by Config, but having the population logic in Inbox should make it easier to mock for testing.
2016-06-20feed: various object-orientation cleanups
Favor Inbox objects as our primary source of truth to simplify our code. This increases our coupling with PSGI to make it easier to write tests in the future. A lot of this code was originally designed to be usable standalone without PSGI or CGI at all; but that might increase development effort.
2016-06-20mbox: avoid write dependency for streaming
Prefer to return strings instead, so Content-Length can be calculated for caching and such.
2016-06-20mbox: remove feed dependency
We do not need feed options there (or anywhere, hopefully).
2016-06-20feed: remove dependence on fh->write for streaming
We'll be switching to a getline/close response body to give the HTTP server more control when dealing with slow clients.
2016-06-20feed: avoid needless method dispatches on 404
We overuse streaming, here. Allow Content-Length to be calculated in this case.
2016-06-19search: reopen and retry on updated databases
This seems like a nasty thing which breaks downloads of large mailboxes.
2016-06-19http: constrain getline/close responses by time
This allows us to yield control to other clients gracefully if getline takes too long to generate a chunk. This is more expensive but should not cost a syscall on modern 64-bit systems.
2016-06-19http: avoid recursion when hitting write count limit
Use the EvCleanup::asap handler to reschedule our writes after yielding to other clients.
2016-06-19mbox: set gzip timestamp to the Unix epoch
This allows consistency between different invocations from roughly the same period and is no worse for caching any any of our existing HTML and Atom feeds. We cannot set the timestamp to the end date since messages may be added to the repository while we are iterating (and this streaming mechanism will pick them up).
2016-06-19watch_maildir: tighten up path checks
Only mark seen messages as spam, otherwise it could be too aggressive and cause problems or over training. We wouldn't want a wayward FIFO ruining our day, either :)
2016-06-19import: allow messages without subject
Because our WatchMaildir module is liberal about what it accepts, we can potentially have messages without a subject.