about summary refs log tree commit homepage
DateCommit message (Collapse)
2015-08-27wire up shorter, less ambiguous URLs
We will prefer URLs without suffixes for now to avoid ambiguity in case a Message-ID ends with ".html", ".txt", ".mbox.gz" or any other suffix we may use. Static file compatibility is preserved by using a trailing slash as most servers can/will fall back to an index.html file in this case. For raw text files, we will follow gmane's lead with "/raw"
2015-08-27www: minor cleanups to shorten code
Less scrolling is more efficient.
2015-08-27www: reduce unused arguments in internal API
Less code is easier-to-manage, although we make a few extra hash insertions now.
2015-08-26mbox: close file handle for single mbox
This doesn't seem needed for actual server use, but Plack tests complain about it
2015-08-25feed: fix "created" vs "updated" diff in topics
This fixes a regression introduced in commit 72c0f7c71ff28de9755dc4aee8b6ce6f0e4f2ed7 (feed: merge subjects regardless of "[PATCH vN]")
2015-08-25filter: loosen regexp on type matches
part_type still contains the filename, unfortunately, so PGP signatures were truly stripped. Oh well, nobody cares to verify PGP signatures anyways.
2015-08-25feed: merge subjects regardless of "[PATCH vN]"
This normalizes rerolled patches with identical topics, but does not normalize different patches even if they are in the same thread (for now).
2015-08-25search: implement subject summarization
We ought to summarize subjects to avoid exploding line lengths in the web interface.
2015-08-25Makefile.PL: add "syntax" target to check syntax
This is necessary since Xapian may not be installed and we may hide a lot of errors this way.
2015-08-25mid: mid_compressed => mid_compress
Consistently name mid_* functions as verbs.
2015-08-25search: only sort by relevance if requested
Many of our internal search queries do not care about relevance, but is used for proper thread displays.
2015-08-24view: refactor $state as a hash
Using hash means we no longer have to document and remember what every field does. The original array form was insane premature optimization and crazy. Who wrote that? Oh wait, I was on drugs :<
2015-08-24mda: remove unnecessary import
Relying on Email::MIME means encoding is handled transparently for us.
2015-08-23view: simplify root filtering in followup display
The root message-ID may be too long to compare. Instead, check fields based on the consistency of our DB.
2015-08-23view: case-insensitive "Re:" check for replies
This is to match what Mail::Thread nad our own search relies on. However, we will be more lenient on spaces, though.
2015-08-23cleanup calls to header_obj
Dereference header_obj only once when performance may be critical, or simplify our code by calling "header" directly on the Email::{Simple,MIME} object if not.
2015-08-23hopefully fix broken permissions for search
We must preserve the umask for the entirety of the indexing operation, as Xapian transactions replace entire files atomically instead of writing them in place.
2015-08-23cleanup Email::Address usage
Email::Address::name never fails assuming it was able to parse anything.
2015-08-23search: respect core.sharedRepository in for Xapian DB
Extend the purpose of core.sharedRepository to apply to the $GIT_DIR/public-inbox/xapian* directory.
2015-08-23feed: r= parameter supports "HEAD" as a revision
public-inbox git repositories require a "HEAD" ref to function correctly anyways.
2015-08-23GitCatFile: use offset for read instead of appending
There is no need to perform string appends when the "read" and "sysread" functions take an offset argument to append to the given buffer. This avoid needless string creation.
2015-08-23mbox: clarify our use of the the mboxrd variant
Commenting it in the From: line seems appropriate and reduces compatibility problems in case a MUA cannot handle trailing comments after the timestamp.
2015-08-23mbox: use mboxrd quoting rules
This redundantly quotes >From from to prevent losing information as described by qmail
2015-08-23.txt links return an mbox instead
This improves compatibility and allows individual messages to be concatenated into an existing mbox without further modifications. "git format-patch" does something similar (but does not do "From " line escaping(!))
2015-08-22view: wire up mbox.gz links
To reduce clutter, we will not link to uncompressed versions. Users should be able to download entire threads for offline reading, enable this feature for them.
2015-08-22mbox: support uncompressed mbox
Some folks may want to view the mbox inline as a string of raw text, when guessing URLs. Let them do this...
2015-08-22search: consistently pass options and flags
Most of our special query functions require exact matches, so none of the flags we normally use are necessary for query parsing.
2015-08-22view: reference total followups
In case there's huge threads, readers should know about them even though we currently lack the navigation to display them.
2015-08-22view: misc cleanups and simplifications
Less code should be easier-to-read.
2015-08-22search: split search indexing to a separate file
This makes organization easier and reduces the amount of code loaded for a PSGI, mod_perl or CGI instance.
2015-08-22view: prevent 'once' warnings for sub ref
Perl seems to incorrectly warn for this, workaround it.
2015-08-22remove XML::Atom::SimpleFeed dependency
We will attempt to generate Atom feeds "by hand" as the XML::Atom::SimpleFeed API does not support streaming output. Since email is large and servers are small, this should prevent wasting memory when we generate larger feeds. Of course, we hope clients use SAX parsers capable of handling large streams without slurping.
2015-08-22www: enable and expand preload from mod_perl2
Hopefully this saves us some memory with CoW on *nix.
2015-08-22INSTALL: document IO::Compress::Gzip dependency
Otherwise folks won't get downloadable mboxes
2015-08-22cgi: remove static file generation support for now
We may not support this after all, CGI.pm is already legacy-enough and far more powerful.
2015-08-22stream HTML views as much as possible
This should allow progressive rendering on the client and reduce memory usage on the server. Unfortunately XML::Atom::SimpleFeed does not yet support streaming, so we may not use it in the future.
2015-08-21search: s/count/total/ for results
This is hopefully less ambiguous, as the word "count" confused me, too.
2015-08-21mbox: drop unnecessary imports
These are not necessary, anymore
2015-08-21switch to gzipped mboxes
Mboxes may be huge, so only support downloading gzipped mboxes to save bandwidth and to get free checksumming. Streaming output means we should not be wasting too much memory on this unless the chosen server sucks.
2015-08-21mbox: stream entire thread, regardless of size
Since mbox is usually downloaded, support fetching infinitely large responses via streaming.
2015-08-21support dumping thread as an mbox
Some folks may not want to download and install Perl code like ssoma, so allow downloading an mbox containing the entire thread.
2015-08-21view: "next" link in thread view goes to next Subject line
It's a bit disconcerting to jump to the authorship line.
2015-08-21view: cleanup and reduce duplication
This also avoids incorrectly incrementing $part_nr when we skip a part due to bad Content-Type.
2015-08-20feed: fix extra, unnecessary quote
Oops!
2015-08-20search: preserve References: order in document data
We need proper ordering of References to thread messages correctly. We would lose this order if we load the terms from the database, so set it directly document data. Do not bother with a separate In-Reply-To, since Mail::Thread just merges the IRT into References. This bumps our schema version once again.
2015-08-20avoid using header_raw for Message-ID retrieval
This is for consistency with ssoma. I doubt it makes a difference in practice, but in case somebody decides any of the Message-ID-containing headers should have strange characters, we'll decode and attempt to thread them. This isn't an attack vector, just a way to make messages thread improperly which is pointless...
2015-08-20view: simplify message threading dumpers
2015-08-20dead code cleanup
We may not be using subject_path after all.
2015-08-20www: remove useless no-op assignment statement
Oops
2015-08-20misc documentation updates
Threading in Xapian is mostly supported by now; so start documenting things.