* [PATCH 1/7] search: favor In-Reply-To over last References iff IRT exists
2016-12-10 3:42 6% [PATCH 0/7] message threading fixes for WWW UI Eric Wong
@ 2016-12-10 3:42 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2016-12-10 3:42 UTC (permalink / raw)
To: meta
Some email clients set the References headers backwards, so
trust the In-Reply-To header if (and only if) it exists and
is parseable as direct parent of the current message.
For affected repos, this will require reindexing (via
"public-inbox-index --reindex"), but there will be no
version bump for this bugfix.
---
lib/PublicInbox/SearchIdx.pm | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index 4aac028..832d1cb 100644
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -293,10 +293,10 @@ sub link_message {
my $hdr = $mime->header_obj;
my $refs = $hdr->header_raw('References');
my @refs = $refs ? ($refs =~ /<([^>]+)>/g) : ();
- if (my $irt = $hdr->header_raw('In-Reply-To')) {
- # last References should be $irt
- # we will de-dupe later
- push @refs, mid_clean($irt);
+ my $irt = $hdr->header_raw('In-Reply-To');
+ if (defined $irt) {
+ $irt = mid_clean($irt);
+ $irt = undef if $mid eq $irt;
}
my $tid;
@@ -305,6 +305,15 @@ sub link_message {
my @orig_refs = @refs;
@refs = ();
+ if (defined $irt) {
+ # to check MAX_MID_SIZE
+ push @orig_refs, $irt;
+
+ # below, we will ensure IRT (if specified)
+ # is the last References
+ $uniq{$irt} = 1;
+ }
+
# prevent circular references via References: here:
foreach my $ref (@orig_refs) {
if (length($ref) > MAX_MID_SIZE) {
@@ -315,6 +324,11 @@ sub link_message {
push @refs, $ref;
}
}
+
+ # last References should be IRT, but some mail clients do things
+ # out of order, so trust IRT over References iff IRT exists
+ push @refs, $irt if defined $irt;
+
if (@refs) {
$smsg->{references} = '<'.join('> <', @refs).'>';
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 0/7] message threading fixes for WWW UI
@ 2016-12-10 3:42 6% Eric Wong
2016-12-10 3:42 7% ` [PATCH 1/7] search: favor In-Reply-To over last References iff IRT exists Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2016-12-10 3:42 UTC (permalink / raw)
To: meta
This series improves thread handling in several oddball
cases.
In the Xapian search indexing phase, the In-Reply-To header
is always considered the last (direct) parent of a message.
This is necessary in cases where a MUA specifies References
in an invalid order. This is also what our View.pm display
has done for generating "reply" links.
Not many repos are affected by this, but
"public-inbox-index --reindex" will make those consistent
(there is no incompatible Xapian DB version bump).
We will now prune ghosts without children before display, as
they are sometimes the result of buggy (or malicious) MUAs
inserting spaces or otherwise mangling References: headers.
Ghosts with valid children remain shown, as they are likely to
be legitimate (but lost) messages.
Deploying over the next few hours, .onions first!
Currently reindexing git@vger mirror:
http://czquwvybam4bgbro.onion/meta
Up next:
http://hjrcffqmbrq6wope.onion/meta
Last: (also public-inbox.org)
http://ou63pmih66umazou.onion/meta
Eric Wong (7):
search: favor In-Reply-To over last References iff IRT exists
view: favor SearchMsg for In-Reply-To over Email::MIME
thread: fix comment describing its existence
view: reduce indentation for skeleton generation
view: skip ghosts with no direct children
thread: last Reference always wins
search: always sort thread results in ascending time order
lib/PublicInbox/Mbox.pm | 2 +-
lib/PublicInbox/Search.pm | 5 ++++
lib/PublicInbox/SearchIdx.pm | 22 ++++++++++++---
lib/PublicInbox/SearchThread.pm | 30 ++++++++++++++------
lib/PublicInbox/View.pm | 61 +++++++++++++++++++++--------------------
t/thread-cycle.t | 8 ------
6 files changed, 76 insertions(+), 52 deletions(-)
--
EW
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2016-12-10 3:42 6% [PATCH 0/7] message threading fixes for WWW UI Eric Wong
2016-12-10 3:42 7% ` [PATCH 1/7] search: favor In-Reply-To over last References iff IRT exists Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).