* [PATCH 0/4] Various bits to support import_imap_mailbox
@ 2019-10-09 8:15 5% ` Eric W. Biederman
2019-10-09 8:16 6% ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman
0 siblings, 1 reply; 4+ results
From: Eric W. Biederman @ 2019-10-09 8:15 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
Eric Wong,
These should all of my generic patches to support my import_imap_mailbox
script. The really important patch that adds to the support for List-ID
to public inbox configuration file I have already sent.
I haven't written tests and I get the following test failure when I run
make test
> t/config.t ................. 1/?
> # Failed test 'lookup matches expected output'
> # at t/config.t line 26.
> # Structures begin differing at:
> # $got->{listid} = ARRAY(0x55c1d4e3b6a8)
> # $expected->{listid} = Does not exist
>
> # Failed test 'lookup matches expected output for test'
> # at t/config.t line 42.
> # Structures begin differing at:
> # $got->{listid} = ARRAY(0x55c1d508d8d0)
> # $expected->{listid} = Does not exist
> # Looks like you failed 2 tests of 69.
> t/config.t ................. Dubious, test returned 2 (wstat 512, 0x200)
I haven't looked into what is happening there.
Eric Biederman (4):
PublicInbox::Import Smuggle a raw message into add
PublicInbox::Config: Process mailboxes in sorted order
Config.pm: Add support for looking up repos by their directories
IMAPTracker: Add a helper to track our place in reading imap mailboxes
lib/PublicInbox/Config.pm | 19 ++++++++++-
lib/PublicInbox/IMAPTracker.pm | 73 ++++++++++++++++++++++++++++++++++++++++++
lib/PublicInbox/Import.pm | 9 +++---
3 files changed, 96 insertions(+), 5 deletions(-)
Eric
^ permalink raw reply [relevance 5%]
* [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add
2019-10-09 8:15 5% ` [PATCH 0/4] Various bits to support import_imap_mailbox Eric W. Biederman
@ 2019-10-09 8:16 6% ` Eric W. Biederman
2019-10-15 20:26 7% ` Eric Wong
0 siblings, 1 reply; 4+ results
From: Eric W. Biederman @ 2019-10-09 8:16 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
Date: Tue, 15 Jan 2019 16:36:42 -0600
I don't trust the MIME type to not munge my email messages in horrible
ways upon occasion. Therefore allow for passing in the raw message
value instead of trusting the mime object to preserve it.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
---
As we discussed last time I was working to merge my imap import script.
lib/PublicInbox/Import.pm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index 137b2b7800c4..b17c9d5c9fa8 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -394,16 +394,17 @@ sub add {
}
my $blob = $self->{mark}++;
- my $str = $mime->as_string;
- my $n = length($str);
+ my $raw_email = $mime->{-public_inbox_raw};
+ $raw_email ||= $mime->as_string;
+ my $n = length($raw_email);
$self->{bytes_added} += $n;
print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail;
- print $w $str, "\n" or wfail;
+ print $w $raw_email, "\n" or wfail;
# v2: we need this for Xapian
if ($self->{want_object_info}) {
my $oid = $self->get_mark(":$blob");
- $self->{last_object} = [ $oid, $n, \$str ];
+ $self->{last_object} = [ $oid, $n, \$raw_email ];
}
my $ref = $self->{ref};
my $commit = $self->{mark}++;
--
2.23.0
^ permalink raw reply related [relevance 6%]
* Re: [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add
2019-10-09 8:16 6% ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman
@ 2019-10-15 20:26 7% ` Eric Wong
2019-10-15 23:05 7% ` Eric W. Biederman
0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2019-10-15 20:26 UTC (permalink / raw)
To: Eric W. Biederman; +Cc: meta
"Eric W. Biederman" <ebiederm@xmission.com> wrote:
>
> Date: Tue, 15 Jan 2019 16:36:42 -0600
>
> I don't trust the MIME type to not munge my email messages in horrible
> ways upon occasion. Therefore allow for passing in the raw message
> value instead of trusting the mime object to preserve it.
>
> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
> ---
>
> As we discussed last time I was working to merge my imap import script.
Thanks, pushed with the following interdiff:
diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index b17c9d5c..e1f48771 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -394,8 +394,7 @@ sub add {
}
my $blob = $self->{mark}++;
- my $raw_email = $mime->{-public_inbox_raw};
- $raw_email ||= $mime->as_string;
+ my $raw_email = $mime->{-public_inbox_raw} // $mime->as_string;
my $n = length($raw_email);
$self->{bytes_added} += $n;
print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail;
^ permalink raw reply [relevance 7%]
* Re: [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add
2019-10-15 20:26 7% ` Eric Wong
@ 2019-10-15 23:05 7% ` Eric W. Biederman
0 siblings, 0 replies; 4+ results
From: Eric W. Biederman @ 2019-10-15 23:05 UTC (permalink / raw)
To: Eric Wong; +Cc: meta
Eric Wong <e@80x24.org> writes:
> "Eric W. Biederman" <ebiederm@xmission.com> wrote:
>>
>> Date: Tue, 15 Jan 2019 16:36:42 -0600
>>
>> I don't trust the MIME type to not munge my email messages in horrible
>> ways upon occasion. Therefore allow for passing in the raw message
>> value instead of trusting the mime object to preserve it.
>>
>> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
>> ---
>>
>> As we discussed last time I was working to merge my imap import script.
>
> Thanks, pushed with the following interdiff:
>
> diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
> index b17c9d5c..e1f48771 100644
> --- a/lib/PublicInbox/Import.pm
> +++ b/lib/PublicInbox/Import.pm
> @@ -394,8 +394,7 @@ sub add {
> }
>
> my $blob = $self->{mark}++;
> - my $raw_email = $mime->{-public_inbox_raw};
> - $raw_email ||= $mime->as_string;
> + my $raw_email = $mime->{-public_inbox_raw} // $mime->as_string;
> my $n = length($raw_email);
> $self->{bytes_added} += $n;
> print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail;
Looks good. I hadn't realized that // was an operator in perl.
It is nice to see.
Eric
^ permalink raw reply [relevance 7%]
Results 1-4 of 4 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-10-07 22:13 Do I need multiple publicinbox.<name>.address values? Alyssa Ross
2019-10-08 0:10 ` Eric Wong
2019-10-08 12:18 ` Eric W. Biederman
2019-10-08 22:11 ` Eric Wong
2019-10-08 22:24 ` Eric W. Biederman
2019-10-08 22:41 ` Eric Wong
2019-10-09 7:58 ` Eric W. Biederman
2019-10-09 8:15 5% ` [PATCH 0/4] Various bits to support import_imap_mailbox Eric W. Biederman
2019-10-09 8:16 6% ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman
2019-10-15 20:26 7% ` Eric Wong
2019-10-15 23:05 7% ` Eric W. Biederman
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).