about summary refs log tree commit homepage
path: root/lib
DateCommit message (Collapse)
2015-08-20replace references to lynx
Table rendering in lynx is crap compared to w3m and links. However, we still use it for filtering HTML since the renderer is otherwise nice...
2015-08-20search: index_sync allows specifying alternate HEAD
This should allow us to sync the index to a temporary head to update the Xapian index before we update the real HEAD index.
2015-08-20view: do not fold top-level messages in thread
This hopefully reduces clicking. We may drop folding entirely since we can use Xapian to make searching easier.
2015-08-20index: layout fix + title and Atom feed links at top
Add some spacing between topics to improve readability when scanning or in case a subject gets too long. The title and Atom feed may not be highly-visible otherwise. While we're at it, use the proper "Atom feed" terminology since some folks may not understand just what "atom" means.
2015-08-20search: bump schema version to 5 for subject_path
In "index: simplify main landing page if search-enabled", subject normalization went a little farther to drop trailing '.' characters, so we will need to re-index.
2015-08-20view: reduce memory usage when displaying large threads
We want to minimize the time any large objects or strings are referenced. We can do threading entirely from the mini_mime-generated messages and lazilly load full messages when rendering the display.
2015-08-20search: reject ghosts in all cases
We do not need ghost messages in any of our thread views
2015-08-20search: avoid needless decode
Email::MIME should handle everything for us and make things work nicely with Xapian (assuming I understand how encoding works in Perl). While we're at it, reduce temporary strings and arrays by using destructive operations and clobbering parts as we iterate through them.
2015-08-20index: simplify main landing page if search-enabled
We can display /t/$MESSAGE_ID.html easily with a Xapian search index, so rely on it instead of trying to display messages inline.
2015-08-20view: avoid nesting <a> tags from auto-linkification
It is wrong HTML to have <a> tags nested due to auto-linkification.
2015-08-20use tables for rendering comment nesting
This is more space efficient since we don't need to place padding bytes in front of every line. While this unfortunately does not render well on lynx; w3m, links, elinks can all render tables sanely. Tables are also superior for long lines which require wrapping inside <pre> containers.
2015-08-20feed: move timestamp parsing to view
We don't need share duplicate logic across both files.
2015-08-20feed: remove threading from index
We'll be making the index smarter for people with search support enabled. Otherwise, it'll be chronological and a bit dumb. At least it'll use less memory.
2015-08-19www: redirect /f/$MESSAGE_ID.txt links to /m/$MESSAGE_ID.txt
Some people (e.g. myself :p) may try to guess URLs and hit a 404. Redirect to the /m/ version. Note: we prefer to redirect to canonical URLs to improve caching.
2015-08-19view: return empty string to avoid undefined values
Sometimes we have filter bugs and let HTML slip through...
2015-08-19view: fix spacing on missing ghosts
We must not prematurely indent if we have no message header to display.
2015-08-18view: close anchor tag correctly before starting another
Noticed by tidy
2015-08-18thread: another workaround for a Mail::Thread bug
Yay for monkey patching! ref: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795913 ref: https://rt.cpan.org/Ticket/Display.html?id=106498
2015-08-18search: bump SCHEMA_VERSION to 4
The following two commits affect indexing behavior, so change the schema version to avoid compatibility problems or missing messages: search: common Subject: normalization for Re: prefixes search: avoid creating ghosts for circular References
2015-08-18search: expose $PublicInbox::Search::LANG variable
This makes it easier to reconfigure for non-English users
2015-08-18search: common Subject: normalization for Re: prefixes
Drop German ("Aw:") support since it's non-standard and is not supported by Mail::Thread and non-English prefixes are more likely to conflict with prefixes used in Free Software development where ("subsection:") prefixes are common and English is the common language. Anyways we don't filter "Vs: " (Finnish) or "Sv: " (Norwegian, Swedish, Danish, Icelandic), either. ref: https://en.wikipedia.org/wiki/RE_(e-mail)#Abbreviations_in_other_languages
2015-08-18search: avoid creating ghosts for circular References
Some mail software incorrectly creates circular references and causes us to create ghosts before the actual mail doc is created.
2015-08-18view: cleaner Message-ID filtering for References
Avoid compiling a weird and potentially fragile regexp every time and use the same logic as our search module to dedupe References.
2015-08-17view: do not recompress already-compressed MID for anchors
This is merely for display, so on the off chance somebody does send a 40-byte MID with nothing but hexadecimal characters, the worst that could happen is we repeat an anchor name in the rendered HTML. This has no impact on git archival or Xapian indexing.
2015-08-17search: simplify indexing operation
There's no need to make a transaction for each message when doing incremental indexing against a git repository. While we're at it, simplify the interface for callers, too and do not auto-create the Xapian database if it was not explicitly enabled.
2015-08-17view: always compress Message-IDs for anchors
Valid URLs do not make valid anchor ids.
2015-08-17search: bump schema version for '%' compression change
commit 0fea7793b22efd2596983283947ee43687e0cfac ("mid: compress Message-IDs with '%' in them") requires re-indexing of repositories with '%' in Message-IDs :<
2015-08-17mid: compress Message-IDs with '%' in them
Some HTTP servers (apache2 2.2.22-13+deb7u5) on my system apparently do not handle "%25" correctly. I'm not yet sure if it's something weird with my rewrite rules or what....
2015-08-17search: apply mid_compression to subject paths, too
Otherwise we'll be wasting space in our index for long subjects.
2015-08-17drop bodies and messages ASAP after processing
We can rely on reference counting to lower memory usage for big messages.
2015-08-17feed: disable the generator statement
No need to waste bandwidth, here
2015-08-17search: use raw headers without MIME decoding
This should be less error-prone in case somebody tries to screw with us and our thread_id mechanism or somehow waste our resources. Unfortunately Mail::Thread isn't smart enough for this, yet, so we may need to downgrade to Email::Simple objects as a workaround. Or simply not worry about the display so much if somebody is intentionally trying to make it thread badly/incorrectly.
2015-08-17terminology: replies => followups
Replies are only direct replies, but followups could be any message further down the thread. The latter is more useful.
2015-08-17www: simplify parameter passing to feed
No need to create a new hash when we can reuse the existing one more.
2015-08-17WWW: eliminate "top" parameter for feeds
This parameter hasn't been used since commit 5adf8d639e9b5abd4cbac975d70ddc0fb76541fc ("feed: dead code elimination around dropped endpoints")
2015-08-17favor /t/ to /s/, since subjects may change mid-thread
/t/ always falls back to subject path searching anyways, so there's little lost besides perhaps more readable URLs. Unfortunately people still use non-compliant mail clients which fail to set In-Reply-To or References headers :<
2015-08-17feed: remove unnecesary time paramenter in index state
We no longer do "smart" time displays as of commit ea0e8649f90d1fd0850a41c0ca16642faadf4f14 ("view: simplify timestamp generation"). In retrospect, that commit also made us more cache-friendly, too.
2015-08-17Merge remote-tracking branch 'origin/search'
* origin/search: view: deduplicate common code for loading search results SearchMsg: ensure metadata for ghost messages mid implement /s/$SUBJECT_PATH.html lookups search: remove unnecessary xpfx export www: /t/$MESSAGE_ID.html for threads view: hoist out index_walk function view: reply threading adjustment thread: common sorting code view: display replies in per-message view search: make search results more OO extract redundant Message-ID handling code search: implement index_sync to fixup indexer initial search backend implementation
2015-08-16view: kill leading empty lines correctly
Was too sleepy to be coding last night :x
2015-08-16view: cleaner killing of leading/trailing whitespace
No point in wasting bytes even if gets compressed over the wire, it'll use more memory when rendering on the client.
2015-08-16view: hoist out index_walk function
We will reuse it for thread views when powered by Xapian.
2015-08-16view: deduplicate common code for loading search results
More to come later.
2015-08-16SearchMsg: ensure metadata for ghost messages mid
Ghosts have no document data in them. Perhaps we should just rely on terms for Message-ID and avoid storing that in the document data...
2015-08-16implement /s/$SUBJECT_PATH.html lookups
Quick-and-dirty wiring up of to Subject: paths. This may prove more memorizable and easier-to-share than /t/$MESSAGE_ID.html links, but less strict. This changes our schema version to 1, since we now use lower-case subject paths.
2015-08-16search: remove unnecessary xpfx export
SearchMsg calls it with the full module path anyways.
2015-08-16www: /t/$MESSAGE_ID.html for threads
This should bring up nearly the entire thread a given Message-ID is linked to.
2015-08-16view: hoist out index_walk function
We will reuse it for thread views when powered by Xapian.
2015-08-16view: reply threading adjustment
Give changes in subject their own line to reduce line wrapping, but avoid showing any redundant subjects by maintaining a hash of subjects already displayed.
2015-08-15thread: common sorting code
We'll be sharing the same threading, so it makes sense to sort replies using the same code and message headers without repeating ourselves. This also standardizes on sorting on X-PI-TS (Unix epoch in seconds) instead over using X-PI-Date differently in two different places
2015-08-15view: display replies in per-message view
This can be used to quickly scan for replies in a message without displaying an entire thread.