From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00, T_FILL_THIS_FORM_SHORT shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id D761A1F9FD; Mon, 8 Mar 2021 08:08:16 +0000 (UTC) Date: Mon, 8 Mar 2021 08:08:16 +0000 From: Eric Wong To: Kyle Meyer Cc: meta@public-inbox.org Subject: [PATCH] lei q: remove angle brackets around Message-IDs Message-ID: <20210308080816.GA11470@dcvr> References: <20210304184348.GA19350@dcvr> <87mtvg9m0p.fsf@kyleam.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <87mtvg9m0p.fsf@kyleam.com> List-Id: Kyle Meyer wrote: > Eric Wong writes: > > > I'm thinking these shouldn't include angle brackets: > > > > "m": "<20210228122528.18552-2-e@80x24.org>", > > "refs": ["<20210228122528.18552-1-e@80x24.org>"], > > > > Using angle brackets on the command-line requires quoting to > > disambiguate against redirects, so it's a pain. Leaving the > > brackets in still works because of how Xapian's query parser > > works, not because of anything we do on our end. > > I think it'd be nice to drop the brackets from a noise perspective too. Yes, we don't include them in name email address pairs, either. > Also, does m: work with brackets? Trying it out with a recent message > ID: > > $ lei q -q -I https://public-inbox.org/meta/ -f ldjson \ > '20210304203352.pd5mcg5pw4u2epzl@pengutronix.de' > {"blob":"87304c8a8cae8ce400443b56309427aeee601505",...} > > $ lei q -q -I https://public-inbox.org/meta/ -f ldjson \ > m:'20210304203352.pd5mcg5pw4u2epzl@pengutronix.de' > {"blob":"87304c8a8cae8ce400443b56309427aeee601505",...} > > $ lei q -q -I https://public-inbox.org/meta/ -f ldjson \ > '<20210304203352.pd5mcg5pw4u2epzl@pengutronix.de>' > {"blob":"87304c8a8cae8ce400443b56309427aeee601505",...} > > $ lei q -q -I https://public-inbox.org/meta/ -f ldjson \ > m:'<20210304203352.pd5mcg5pw4u2epzl@pengutronix.de>' > # no results Odd, I'm not sure about that one... It's probably something the Xapian query parser is doing internally and nothing on our end... > > Since the actual headers are "Message-ID" and "References", (and > > not "m" or "refs"), I think it's clear that we don't have to > > match the raw mail contents exactly. We RFC 2047 decode > > "f|t|c|s" fields anyways instead of showing the raw values, > > so more precedence for leaving out <>. > > Fwiw I don't think leaving out the brackets would be a source of > confusion. Agreed. And I'm now wondering if we should start indexing "References:" to be a searchable header with the "refs:" prefix (but also wary about increasing disk space usage as a result...) In any case, this denoises the output a bit: ------------8<---------- Subject: [PATCH] lei q: remove angle brackets around Message-IDs They're unnecessary visual noise, and angle brackets don't always work as intended when going through Xapian's query parser. Since we already use "m:" and "refs:" instead of the actual header names, it should be obvious we're at liberty to abbreviate such things Link: https://public-inbox.org/meta/20210304184348.GA19350@dcvr/ --- lib/PublicInbox/LeiOverview.pm | 5 ++--- t/lei-externals.t | 2 +- t/lei-q-thread.t | 8 ++++---- xt/net_writer-imap.t | 2 +- 4 files changed, 8 insertions(+), 9 deletions(-) diff --git a/lib/PublicInbox/LeiOverview.pm b/lib/PublicInbox/LeiOverview.pm index 4db1d8c8..01556273 100644 --- a/lib/PublicInbox/LeiOverview.pm +++ b/lib/PublicInbox/LeiOverview.pm @@ -141,17 +141,16 @@ sub _unbless_smsg { $smsg->{dt} = _iso8601(delete $smsg->{ds}); # JMAP UTCDate $smsg->{pct} = get_pct($mitem) if $mitem; if (my $r = delete $smsg->{references}) { - $smsg->{refs} = [ map { "<$_>" } ($r =~ m/$MID_EXTRACT/go) ]; + $smsg->{refs} = [ map { $_ } ($r =~ m/$MID_EXTRACT/go) ]; } if (my $m = delete($smsg->{mid})) { - $smsg->{'m'} = "<$m>"; + $smsg->{'m'} = $m; } for my $f (qw(from to cc)) { my $v = delete $smsg->{$f} or next; $smsg->{substr($f, 0, 1)} = pairs($v); } $smsg->{'s'} = delete $smsg->{subject}; - # can we be bothered to parse From/To/Cc into arrays? scalar { %$smsg }; # unbless } diff --git a/t/lei-externals.t b/t/lei-externals.t index 29667640..2a92d101 100644 --- a/t/lei-externals.t +++ b/t/lei-externals.t @@ -25,7 +25,7 @@ SKIP: { lei_ok(@cmd, \"query $url"); is($lei_err, '', "no errors on $url"); my $res = json_utf8->decode($lei_out); - is($res->[0]->{'m'}, "<$mid>", "got expected mid from $url") or + is($res->[0]->{'m'}, $mid, "got expected mid from $url") or skip 'further remote tests', 1; lei_ok(@cmd, 'd:..20101002', \'no results, no error'); is($lei_err, '', 'no output on 404, matching local FS behavior'); diff --git a/t/lei-q-thread.t b/t/lei-q-thread.t index 28c639f5..e24fb2cb 100644 --- a/t/lei-q-thread.t +++ b/t/lei-q-thread.t @@ -27,9 +27,9 @@ test_lei(sub { is(scalar(@$res), 3, 'got 2 results'); pop @$res; my %m = map { $_->{'m'} => $_ } @$res; - is_deeply($m{''}->{kw}, ['seen'], + is_deeply($m{'testmessage@example.com'}->{kw}, ['seen'], 'flag set in direct hit'); - 'TODO' or is_deeply($m{''}->{kw}, ['draft'], + 'TODO' or is_deeply($m{'a-reply@miss'}->{kw}, ['draft'], 'flag set in thread hit'); lei_ok qw(q -t -t m:testmessage@example.com); @@ -37,9 +37,9 @@ test_lei(sub { is(scalar(@$res), 3, 'got 2 results with -t -t'); pop @$res; %m = map { $_->{'m'} => $_ } @$res; - is_deeply($m{''}->{kw}, ['flagged', 'seen'], + is_deeply($m{'testmessage@example.com'}->{kw}, ['flagged', 'seen'], 'flagged set in direct hit'); - 'TODO' or is_deeply($m{''}->{kw}, ['draft'], + 'TODO' or is_deeply($m{'testmessage@example.com'}->{kw}, ['draft'], 'flagged set in direct hit'); lei_ok qw(q -tt m:testmessage@example.com --only), "$ro_home/t2"; $res = json_utf8->decode($lei_out); diff --git a/xt/net_writer-imap.t b/xt/net_writer-imap.t index c24fa993..3631d932 100644 --- a/xt/net_writer-imap.t +++ b/xt/net_writer-imap.t @@ -168,7 +168,7 @@ test_lei(sub { is_deeply($empty, [], 'clobbered folder'); lei_ok qw(q -o /dev/stdout m:testmessage@example.com --no-external); $res = json_utf8->decode($lei_out)->[0]; - is_deeply([@$res{qw(m kw)}], ['', ['seen']], + is_deeply([@$res{qw(m kw)}], ['testmessage@example.com', ['seen']], 'kw set'); });