* [PATCH 8/8] lei import: ignore Status headers in "eml" messages
2021-03-22 7:53 7% [PATCH 0/8] lei input handling improvements Eric Wong
@ 2021-03-22 7:54 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2021-03-22 7:54 UTC (permalink / raw)
To: meta
Those headers only have meaning with for mboxes. Don't surprise
users by trying to make sense of a header that is defined for mboxes.
It's possible to send email with (Status|X-Status) headers and
have those headers show up in a recipient's IMAP mailbox.
This was bad because an IMAP user may want to import a single
message through their MUA and pipe its contents to "lei import"
without noticing a mischievious sender stuck "X-Status: F"
(flagged/important) in there.
---
lib/PublicInbox/LeiImport.pm | 14 +++++++-------
t/lei-import.t | 27 ++++++++++++++++++++++-----
2 files changed, 29 insertions(+), 12 deletions(-)
diff --git a/lib/PublicInbox/LeiImport.pm b/lib/PublicInbox/LeiImport.pm
index 767cae60..9ad2ff12 100644
--- a/lib/PublicInbox/LeiImport.pm
+++ b/lib/PublicInbox/LeiImport.pm
@@ -10,19 +10,19 @@ use PublicInbox::Eml;
use PublicInbox::PktOp qw(pkt_do);
sub eml_cb { # used by PublicInbox::LeiInput::input_fh
- my ($self, $eml) = @_;
- my $vmd;
- if ($self->{-import_kw}) { # FIXME
- my $kw = PublicInbox::MboxReader::mbox_keywords($eml);
- $vmd = { kw => $kw } if scalar(@$kw);
- }
+ my ($self, $eml, $vmd) = @_;
my $xoids = $self->{lei}->{ale}->xoids_for($eml);
$self->{lei}->{sto}->ipc_do('set_eml', $eml, $vmd, $xoids);
}
sub mbox_cb { # MboxReader callback used by PublicInbox::LeiInput::input_fh
my ($eml, $self) = @_;
- eml_cb($self, $eml);
+ my $vmd;
+ if ($self->{-import_kw}) {
+ my $kw = PublicInbox::MboxReader::mbox_keywords($eml);
+ $vmd = { kw => $kw } if scalar(@$kw);
+ }
+ eml_cb($self, $eml, $vmd);
}
sub import_done_wait { # dwaitpid callback
diff --git a/t/lei-import.t b/t/lei-import.t
index eef1e4e2..a697d756 100644
--- a/t/lei-import.t
+++ b/t/lei-import.t
@@ -39,27 +39,44 @@ lei_ok(qw(q m:testmessage@example.com));
is(json_utf8->decode($lei_out)->[0]->{'blob'}, $oid,
'got expected OID w/o From');
-my $str = <<'';
+my $eml_str = <<'';
From: a@b
Message-ID: <x@y>
Status: RO
-my $opt = { %$lei_opt, 0 => \$str };
+my $opt = { %$lei_opt, 0 => \$eml_str };
lei_ok([qw(import -F eml -)], undef, $opt,
\'import single file with keywords from stdin');
lei_ok(qw(q m:x@y));
my $res = json_utf8->decode($lei_out);
is($res->[1], undef, 'only one result');
-is_deeply($res->[0]->{kw}, ['seen'], "message `seen' keyword set");
+is($res->[0]->{'m'}, 'x@y', 'got expected message');
+is($res->[0]->{kw}, undef, 'Status ignored for eml');
+lei_ok(qw(q -f mboxrd m:x@y));
+unlike($lei_out, qr/^Status:/, 'no Status: in imported message');
-$str =~ tr/x/v/; # v@y
-lei_ok([qw(import --no-kw -F eml -)], undef, $opt,
+
+$eml->header_set('Message-ID', '<v@y>');
+$eml->header_set('Status', 'RO');
+$in = 'From v@y Fri Oct 2 00:00:00 1993'."\n".$eml->as_string;
+lei_ok([qw(import --no-kw -F mboxrd -)], undef, { %$lei_opt, 0 => \$in },
\'import single file with --no-kw from stdin');
lei(qw(q m:v@y));
$res = json_utf8->decode($lei_out);
is($res->[1], undef, 'only one result');
+is($res->[0]->{'m'}, 'v@y', 'got expected message');
is($res->[0]->{kw}, undef, 'no keywords set');
+$eml->header_set('Message-ID', '<k@y>');
+$in = 'From k@y Fri Oct 2 00:00:00 1993'."\n".$eml->as_string;
+lei_ok([qw(import -F mboxrd -)], undef, { %$lei_opt, 0 => \$in },
+ \'import single file with --kw (default) from stdin');
+lei(qw(q m:k@y));
+$res = json_utf8->decode($lei_out);
+is($res->[1], undef, 'only one result');
+is($res->[0]->{'m'}, 'k@y', 'got expected message');
+is_deeply($res->[0]->{kw}, ['seen'], "`seen' keywords set");
+
# see t/lei_to_mail.t for "import -F mbox*"
});
done_testing;
^ permalink raw reply related [relevance 6%]
* [PATCH 0/8] lei input handling improvements
@ 2021-03-22 7:53 7% Eric Wong
2021-03-22 7:54 6% ` [PATCH 8/8] lei import: ignore Status headers in "eml" messages Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2021-03-22 7:53 UTC (permalink / raw)
To: meta
lei <convert|import> share a bit more code, now; and being
able to set "-c imap.debug" on the command-line should make
future work easier.
All this should set us up nicely for implementing "lei mark"
to add/remove keywords and labels.
Eric Wong (8):
lei: support -c <name>=<value> to overrides
net_reader: escape nasty chars from Net::NNTP->message
lei: share input code between convert and import
lei: simplify workers_start and callers
mbox_reader: add ->reads method to avoid nonsensical formats
lei_input: common filehandle reader for eml + mbox
lei_input: drop "From " line on single "eml" (message/rfc822)
lei import: ignore Status headers in "eml" messages
MANIFEST | 1 +
lib/PublicInbox/InboxWritable.pm | 2 +-
lib/PublicInbox/LEI.pm | 137 ++++++++++++++++++-------------
lib/PublicInbox/LeiConvert.pm | 94 ++++-----------------
lib/PublicInbox/LeiExternal.pm | 2 +-
lib/PublicInbox/LeiImport.pm | 107 +++++-------------------
lib/PublicInbox/LeiInput.pm | 106 ++++++++++++++++++++++++
lib/PublicInbox/LeiP2q.pm | 4 +-
lib/PublicInbox/MboxReader.pm | 5 ++
lib/PublicInbox/NetReader.pm | 10 ++-
t/lei-import.t | 37 +++++++--
t/lei.t | 9 ++
12 files changed, 278 insertions(+), 236 deletions(-)
create mode 100644 lib/PublicInbox/LeiInput.pm
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-03-22 7:53 7% [PATCH 0/8] lei input handling improvements Eric Wong
2021-03-22 7:54 6% ` [PATCH 8/8] lei import: ignore Status headers in "eml" messages Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).