about summary refs log tree commit homepage
path: root/lib/PublicInbox/SearchView.pm
DateCommit message (Collapse)
2019-04-18view: show "(no subject)" consistently in HTML
Empty subjects ("") and undefined Subjects: are now both displayed as "(no subject)" for now.
2019-04-16cleanup: use '$ibx' consistently when referring to Inbox refs
'$inbox' is more human-readable, so that is for the more human-readable name in most cases. Making our variable naming more consistent should make the code easier-to-review and harder to screw up.
2019-01-08searchview: drop unused {seen} hashref
Unused since commit 5f09452bb7e6cf49fb6eb7e6cf166a7c3cdc5433 ("view: cull redundant phrases in subjects")
2018-04-23searchview: do not blindly append "l" parameter to URL
It's ugly and all of our other parameters are omitted when values are not the default.
2018-04-18Merge remote-tracking branch 'origin/master' into v2
* origin/master: nntp: allow and ignore empty commands mbox: do not barf on queries which return no results nntp: fix NEWNEWS command searchview: fix non-numeric comparison Allow specification of the number of search results to return githttpbackend: avoid infinite loop on generic PSGI servers http: fix modification of read-only value extmsg: use news.gmane.org for Message-ID lookups extmsg: rework partial MID matching to favor current inbox Update the installation instructions with Fedora package names nntp: do not drain rbuf if there is a command pending nntp: improve fairness during XOVER and similar commands searchidx: do not modify Xapian DB while iterating Don't use LIMIT in UPDATE statements
2018-04-05searchview: minor cleanup
$mset->size is probably more obvious than relying on a tied array and saves us a line.
2018-04-03mbox: do not barf on queries which return no results
Having zero search results means we never get a chance to populate the Content-Disposition header for mbox downloads.
2018-04-01searchview: fix non-numeric comparison
We don't want non-fully-numeric limits being compared and tripping warnings. While we're at it, avoid hard-coding '200' and reuse $LIM as the default.
2018-04-01Allow specification of the number of search results to return
Add an "l=" parameter to the search query syntax to specify how many results should be returned.
2018-03-29search: get rid of most lookup_* subroutines
Too many similar functions doing the same basic thing was redundant and misleading, especially since Message-ID is no longer treated as a truly unique identifier. For displaying threads in the HTML, this makes it clear that we favor the primary Message-ID mapped to an NNTP article number if a message cannot be found.
2018-03-27view: depend on SearchMsg for Message-ID
Since we need to handle messages with multiple and duplicate Message-ID headers, our thread skeleton display must account for that. Since we have a "preferred" Message-ID in case of conflicts, use it as the UUID in an Atom feed so readers do not get confused by conflicts.
2018-03-27searchview: remove unnecessary imports from MID module
We do not need many of these, anymore.
2018-03-22use both Date: and Received: times
We want to rely on Date: to sort messages within individual threads since it keeps messages from git-send-email(1) sorted. However, since developers occasionally have the clock set wrong on their machines, sort overall messages by the newest date in a Received: header so the landing page isn't forever polluted by messages from the future. This also gives us determinism for commit times in most cases, as we'll used the Received: timestamp there, as well.
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2017-12-08search: force large mbox result downloads to POST
This should prevent crawlers (including most robots.txt ignoring ones) from burning our CPU time without severely compromising usability for humans.
2017-12-07searchview: nofollow on mbox downloads
Some search results are gigantic, and search engines are unlikely to be able to handle gzipped mboxes anyways.
2017-12-01search: allow downloading search results as mbox
Allowing downloading of all search results as an gzipped mboxrd file can be convenient for some users.
2017-11-29searchview: s/threaded/nested/
We want to be consistent with the view change in commit b223e6f49debb99b9132bc85d97a065ebcee00b9
2017-10-03search: try to fill in ghosts when generating thread skeleton
Since we attempt to fill in threads by Subject, our thread skeletons can cross actual thread IDs, leading to the possibility of false ghosts showing up in the skeleton. Try to fill in the ghosts as well as possible by performing a message lookup.
2017-06-23allow admins to configure non-obfuscated addresses/domains
We will also treat all known list addresses as non-obfuscated. By setting publicinbox.noObfuscate in ~/.public-inbox/config, this will allow users to disable address obfuscation on a per-domain or per-address basis.
2017-06-16view: implement optional address obfuscation
This is lightly-tested and seems to work. I'm still hesitant to support this, but the alternative of receiving death threats for displaying unobfuscated addresses seems to be not worth it.
2017-05-23searchview: retry queries if uri_unescape-able
It is possible to have double-escaped queries when copy and pasting into browsers, so try to help users work around this common error by automatically retrying after unescaping once. Of course, we must inform the user when doing this results in success, in case they really meant to search for a double-escaped term which resulted in nothing. Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> https://public-inbox.org/meta/CACBZZX5Gnow08r=0A1J_kt3a=zpGyMfvsqu8nAN7kacNnDm+dg@mail.gmail.com/
2017-05-23www: do not mangle characters from search queries
Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> https://public-inbox.org/meta/CACBZZX5Gnow08r=0A1J_kt3a=zpGyMfvsqu8nAN7kacNnDm+dg@mail.gmail.com/
2017-03-24searchview: show full (&x=t) messages in ascending chronlogical order
When displaying search results with full messages, it makes more sense to show them in ascending chronological order when going by date. Reverse chronological order makes more sense for search results which only show the subject.
2017-03-24searchview: add "t" id to link to thread overview
At least for the thread view (&x=t); this will make it easy to link to the overview.
2017-02-06searchview: increase limit for displaying search results
We are in no danger of excessive buffering or OOM-ing, the main page for every inbox already loads 200 results; and thread page views even load 1000! Increase this to 200 for now.
2017-02-06searchview: clarify numeric summary at bottom
Xapian can only give estimated results when a result limit is given to it, so make clear it is an estimate to avoid showing non-sensical ranges when no results are returned.
2017-01-10introduce PublicInbox::MIME wrapper class
This should fix problems with multipart messages where text/plain parts lack a header. cf. git clone --mirror https://github.com/rjbs/Email-MIME.git refs/pull/28/head In the future, we may still introduce as streaming interface to reduce memory usage on large emails.
2016-12-21searchthread: simplify API and remove needless OO
This simplifies callers to prevent errors and avoids needless object-orientation in favor of a single procedure call to handle threading and ordering.
2016-12-10search: retry document loading from Xapian
In addition to needing to retry enquire queries, we also need to protect document loading from the Xapian DB and retry on modification, as it seems to throw the same errors. Checking the $@ ref for Search::Xapian::DatabaseModifiedError is actually in the test suite for both the XS and SWIG Xapian bindings, so we should be good as far as forward/backwards compatibility.
2016-12-03atom: switch to getline/close for response bodies
This will let us stream larger Atom documents bodies without wasting too much memory and reduce the amount of round-trip requests needed to get necessary information. Hopefully clients are using streaming (SAX) parsers, too. This is the final transition in the core public-inbox code to allow migrating to a "pull"-based body streaming scheme which allows a HTTP server to respond appropriately to backpressure from slow clients.
2016-12-03searchview: fix <title> tag in Atom feed
This only affects the Atom feed for search results. "xmlstarlet val" failed to detect or warn about this, and I only noticed this bug while working on another patch.
2016-10-14thread: reinstates stable ordering when ghosts are present
This reverts commit 3c9dd6619f825f0515e7e4afa1bd55c99c1a68d3 ("thread: fix sorting without topmost") and reinstates the "topmost" routine for sorting purposes.
2016-10-05thread: fix sorting without topmost
This bug was hidden, and we may not be able to efficiently implement a topmost subroutine with the hash-based (vs linked-list) based container for threading in the next commit.
2016-10-05thread: remove Email::Abstract wrapping
This roughly doubles performance due to the reduction in object creation and abstraction layers.
2016-10-05thread: pass array refs instead of entire arrays
Copying large arrays is expensive, so avoid it. This reduces /$INBOX/ time by around 1%.
2016-10-05thread: remove Mail::Thread dependency
Introduce our own SearchThread class for threading messages. This should allow us to specialize and optimize away objects in future commits.
2016-08-18searchview: link to internal help text
The internal help text links to the Xapian query parser documentation anyways, but also provides information on which prefixes exist.
2016-08-14www: do not unecessarily escape some chars in paths
Based on reading RFC 3986, it seems '@', ':', '!', '$', '&', "'", '; '(', ')', '*', '+', ',', ';', '=' are all allowed in path-absolute where we have the Message-ID. In any case, it seems '@' is fairly common in path components nowadays and too common in Message-IDs.
2016-07-06www: use HTML <hr> instead of XHTML <hr />
We only need XHTML-compatibility inside Atom feeds, as anecdotally, feed readers are stricter than normal browsers and some do not support HTML, only XHTML. So we will continue to accomodate them. However we favor HTML elsewhere since it tends to be smaller than the equivalent well-formed XHTML.
2016-07-02www: use PSGI env directly
More work on on the Plack::Request/CGI.pm removal front, No need to access the PSGI env through an extra hash lookup.
2016-07-01searchview: add missing newline in search results
Hrm... is there a more obvious way to do an internal API for this while still being streamable?
2016-06-30searchview: show result count in thread index, for now
I'm not sure what to show here, actually; but it's better than triggering an uninitialized variable warning.
2016-06-30www_stream: add response wrapper sub
This encapsulates an entire PSGI response array, hopefully making it easier to generate responses and avoid typos when setting the Content-Type.
2016-06-30view: merge $state hash with existing $ctx
This reduces the level of indirection to reach certain objects within the hash and there are no namespace or lifetime conflicts anyways.
2016-06-30view: show thread context in the thread-aware flat view
This lets user have a small window of the context of the current message relative to other threads.
2016-06-30www: use WwwStream for dumping thread and search views
This allows us the HTTP server to react to backpressure from slow clients when writing. As a side effect, this also makes it easier for us to maintain a consistent header/footer across our HTML.
2016-06-30www: implement hybrid flat+thread conversation view
This should be more accessible to readers on narrow terminals (or giant fonts) while providing a chronological view which is also aware of message threading relationships.
2016-06-21view: common thread walking interface
Since we have a common pattern, for walking threads, extract it into a function and reduce the amount of code we haev. This will make it easier to switch to an event-driven interface for getline, too.
2016-06-21searchview: remove recursion from thread view
As before, recursion can cause problems sooner than unshifting objects into the head of a queue.