about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2015-08-20feed: remove threading from index
We'll be making the index smarter for people with search support enabled. Otherwise, it'll be chronological and a bit dumb. At least it'll use less memory.
2015-08-19www: redirect /f/$MESSAGE_ID.txt links to /m/$MESSAGE_ID.txt
Some people (e.g. myself :p) may try to guess URLs and hit a 404. Redirect to the /m/ version. Note: we prefer to redirect to canonical URLs to improve caching.
2015-08-19view: return empty string to avoid undefined values
Sometimes we have filter bugs and let HTML slip through...
2015-08-19view: fix spacing on missing ghosts
We must not prematurely indent if we have no message header to display.
2015-08-18view: close anchor tag correctly before starting another
Noticed by tidy
2015-08-18thread: another workaround for a Mail::Thread bug
Yay for monkey patching! ref: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795913 ref: https://rt.cpan.org/Ticket/Display.html?id=106498
2015-08-18search: bump SCHEMA_VERSION to 4
The following two commits affect indexing behavior, so change the schema version to avoid compatibility problems or missing messages: search: common Subject: normalization for Re: prefixes search: avoid creating ghosts for circular References
2015-08-18search: expose $PublicInbox::Search::LANG variable
This makes it easier to reconfigure for non-English users
2015-08-18search: common Subject: normalization for Re: prefixes
Drop German ("Aw:") support since it's non-standard and is not supported by Mail::Thread and non-English prefixes are more likely to conflict with prefixes used in Free Software development where ("subsection:") prefixes are common and English is the common language. Anyways we don't filter "Vs: " (Finnish) or "Sv: " (Norwegian, Swedish, Danish, Icelandic), either. ref: https://en.wikipedia.org/wiki/RE_(e-mail)#Abbreviations_in_other_languages
2015-08-18search: avoid creating ghosts for circular References
Some mail software incorrectly creates circular references and causes us to create ghosts before the actual mail doc is created.
2015-08-18view: cleaner Message-ID filtering for References
Avoid compiling a weird and potentially fragile regexp every time and use the same logic as our search module to dedupe References.
2015-08-17view: do not recompress already-compressed MID for anchors
This is merely for display, so on the off chance somebody does send a 40-byte MID with nothing but hexadecimal characters, the worst that could happen is we repeat an anchor name in the rendered HTML. This has no impact on git archival or Xapian indexing.
2015-08-17search: simplify indexing operation
There's no need to make a transaction for each message when doing incremental indexing against a git repository. While we're at it, simplify the interface for callers, too and do not auto-create the Xapian database if it was not explicitly enabled.
2015-08-17view: always compress Message-IDs for anchors
Valid URLs do not make valid anchor ids.
2015-08-17search: bump schema version for '%' compression change
commit 0fea7793b22efd2596983283947ee43687e0cfac ("mid: compress Message-IDs with '%' in them") requires re-indexing of repositories with '%' in Message-IDs :<
2015-08-17mid: compress Message-IDs with '%' in them
Some HTTP servers (apache2 2.2.22-13+deb7u5) on my system apparently do not handle "%25" correctly. I'm not yet sure if it's something weird with my rewrite rules or what....
2015-08-17search: apply mid_compression to subject paths, too
Otherwise we'll be wasting space in our index for long subjects.
2015-08-17drop bodies and messages ASAP after processing
We can rely on reference counting to lower memory usage for big messages.
2015-08-17feed: disable the generator statement
No need to waste bandwidth, here
2015-08-17search: use raw headers without MIME decoding
This should be less error-prone in case somebody tries to screw with us and our thread_id mechanism or somehow waste our resources. Unfortunately Mail::Thread isn't smart enough for this, yet, so we may need to downgrade to Email::Simple objects as a workaround. Or simply not worry about the display so much if somebody is intentionally trying to make it thread badly/incorrectly.
2015-08-17terminology: replies => followups
Replies are only direct replies, but followups could be any message further down the thread. The latter is more useful.
2015-08-17www: simplify parameter passing to feed
No need to create a new hash when we can reuse the existing one more.
2015-08-17WWW: eliminate "top" parameter for feeds
This parameter hasn't been used since commit 5adf8d639e9b5abd4cbac975d70ddc0fb76541fc ("feed: dead code elimination around dropped endpoints")
2015-08-17favor /t/ to /s/, since subjects may change mid-thread
/t/ always falls back to subject path searching anyways, so there's little lost besides perhaps more readable URLs. Unfortunately people still use non-compliant mail clients which fail to set In-Reply-To or References headers :<
2015-08-17feed: remove unnecesary time paramenter in index state
We no longer do "smart" time displays as of commit ea0e8649f90d1fd0850a41c0ca16642faadf4f14 ("view: simplify timestamp generation"). In retrospect, that commit also made us more cache-friendly, too.
2015-08-17Merge remote-tracking branch 'origin/search'
* origin/search: view: deduplicate common code for loading search results SearchMsg: ensure metadata for ghost messages mid implement /s/$SUBJECT_PATH.html lookups search: remove unnecessary xpfx export www: /t/$MESSAGE_ID.html for threads view: hoist out index_walk function view: reply threading adjustment thread: common sorting code view: display replies in per-message view search: make search results more OO extract redundant Message-ID handling code search: implement index_sync to fixup indexer initial search backend implementation
2015-08-16view: kill leading empty lines correctly
Was too sleepy to be coding last night :x
2015-08-16view: cleaner killing of leading/trailing whitespace
No point in wasting bytes even if gets compressed over the wire, it'll use more memory when rendering on the client.
2015-08-16view: hoist out index_walk function
We will reuse it for thread views when powered by Xapian.
2015-08-16view: deduplicate common code for loading search results
More to come later.
2015-08-16SearchMsg: ensure metadata for ghost messages mid
Ghosts have no document data in them. Perhaps we should just rely on terms for Message-ID and avoid storing that in the document data...
2015-08-16implement /s/$SUBJECT_PATH.html lookups
Quick-and-dirty wiring up of to Subject: paths. This may prove more memorizable and easier-to-share than /t/$MESSAGE_ID.html links, but less strict. This changes our schema version to 1, since we now use lower-case subject paths.
2015-08-16search: remove unnecessary xpfx export
SearchMsg calls it with the full module path anyways.
2015-08-16www: /t/$MESSAGE_ID.html for threads
This should bring up nearly the entire thread a given Message-ID is linked to.
2015-08-16view: hoist out index_walk function
We will reuse it for thread views when powered by Xapian.
2015-08-16view: reply threading adjustment
Give changes in subject their own line to reduce line wrapping, but avoid showing any redundant subjects by maintaining a hash of subjects already displayed.
2015-08-15thread: common sorting code
We'll be sharing the same threading, so it makes sense to sort replies using the same code and message headers without repeating ourselves. This also standardizes on sorting on X-PI-TS (Unix epoch in seconds) instead over using X-PI-Date differently in two different places
2015-08-15view: display replies in per-message view
This can be used to quickly scan for replies in a message without displaying an entire thread.
2015-08-15search: make search results more OO
This will relieve callers of the need to decode the data we store internally in Xapian
2015-08-15extract redundant Message-ID handling code
Quit repeating ourselves and use a common MID module instead.
2015-08-15search: implement index_sync to fixup indexer
We need to make the indexer executable and installable while we're at it.
2015-08-13initial search backend implementation
This shall allow us to search for replies/threads more easily.
2015-08-12view: consistent ordering of Cc: addresses
This fixes a minor test failure in t/cgi.t Tested with perl 5.18.2-2ubuntu1 on Ubuntu 14.04.3 LTS
2015-08-05view: remove unused $enc_mime Encoding object
Unneeded since commit e022d3377fd2c50fd9931bf96394728958a90bf3 ("huge refactor of encoding handling")
2015-08-05view: pass fallback encoding to enc_for
This fixes the fallback to message encoding if the message itself was not UTF-8
2015-07-29view: simplify timestamp generation
It's seems less ambiguous to parse a consistent in quiet lists where messages are sparse.
2015-07-20feed: extract subroutines for threading
We'll be using this in the future for displaying per-thread views.
2015-07-14reject HTML loudly and automatically
This should hopefully reduce the delay between when a user fails to send plain-text to when an admin such as myself notices the HTML mail in a sea of spam. Unfortunately, this can lead to backscatter, so avoid doing it until its passed through spamc, at least.
2015-07-06feed: compile regexps only once
This avoids some runtime penalties associated with recompiling a regexp based on a constant local variable.
2015-07-06view: reduce empty <a>, use "id" instead of "name" attributes
This is probably more compliant, and saves us a few bytes on the uncompressed HTML.