user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* Re: [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add
  2019-10-15 20:26  7%                 ` Eric Wong
@ 2019-10-15 23:05  7%                   ` Eric W. Biederman
  0 siblings, 0 replies; 4+ results
From: Eric W. Biederman @ 2019-10-15 23:05 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta

Eric Wong <e@80x24.org> writes:

> "Eric W. Biederman" <ebiederm@xmission.com> wrote:
>> 
>> Date: Tue, 15 Jan 2019 16:36:42 -0600
>> 
>> I don't trust the MIME type to not munge my email messages in horrible
>> ways upon occasion.  Therefore  allow for passing in the raw message
>> value instead of trusting the mime object to preserve it.
>> 
>> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
>> ---
>> 
>> As we discussed last time I was working to merge my imap import script.
>
> Thanks, pushed with the following interdiff:
>
>   diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
>   index b17c9d5c..e1f48771 100644
>   --- a/lib/PublicInbox/Import.pm
>   +++ b/lib/PublicInbox/Import.pm
>   @@ -394,8 +394,7 @@ sub add {
>    	}
>    
>    	my $blob = $self->{mark}++;
>   -	my $raw_email = $mime->{-public_inbox_raw};
>   -	$raw_email ||= $mime->as_string;
>   +	my $raw_email = $mime->{-public_inbox_raw} // $mime->as_string;
>    	my $n = length($raw_email);
>    	$self->{bytes_added} += $n;
>    	print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail;

Looks good.  I hadn't realized that // was an operator in perl.
It is nice to see.

Eric

^ permalink raw reply	[relevance 7%]

* Re: [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add
  2019-10-09  8:16  6%               ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman
@ 2019-10-15 20:26  7%                 ` Eric Wong
  2019-10-15 23:05  7%                   ` Eric W. Biederman
  0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2019-10-15 20:26 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: meta

"Eric W. Biederman" <ebiederm@xmission.com> wrote:
> 
> Date: Tue, 15 Jan 2019 16:36:42 -0600
> 
> I don't trust the MIME type to not munge my email messages in horrible
> ways upon occasion.  Therefore  allow for passing in the raw message
> value instead of trusting the mime object to preserve it.
> 
> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
> ---
> 
> As we discussed last time I was working to merge my imap import script.

Thanks, pushed with the following interdiff:

  diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
  index b17c9d5c..e1f48771 100644
  --- a/lib/PublicInbox/Import.pm
  +++ b/lib/PublicInbox/Import.pm
  @@ -394,8 +394,7 @@ sub add {
   	}
   
   	my $blob = $self->{mark}++;
  -	my $raw_email = $mime->{-public_inbox_raw};
  -	$raw_email ||= $mime->as_string;
  +	my $raw_email = $mime->{-public_inbox_raw} // $mime->as_string;
   	my $n = length($raw_email);
   	$self->{bytes_added} += $n;
   	print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail;

^ permalink raw reply	[relevance 7%]

* [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add
  2019-10-09  8:15  5%             ` [PATCH 0/4] Various bits to support import_imap_mailbox Eric W. Biederman
@ 2019-10-09  8:16  6%               ` Eric W. Biederman
  2019-10-15 20:26  7%                 ` Eric Wong
  0 siblings, 1 reply; 4+ results
From: Eric W. Biederman @ 2019-10-09  8:16 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta


Date: Tue, 15 Jan 2019 16:36:42 -0600

I don't trust the MIME type to not munge my email messages in horrible
ways upon occasion.  Therefore  allow for passing in the raw message
value instead of trusting the mime object to preserve it.

Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
---

As we discussed last time I was working to merge my imap import script.

 lib/PublicInbox/Import.pm | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index 137b2b7800c4..b17c9d5c9fa8 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -394,16 +394,17 @@ sub add {
 	}
 
 	my $blob = $self->{mark}++;
-	my $str = $mime->as_string;
-	my $n = length($str);
+	my $raw_email = $mime->{-public_inbox_raw};
+	$raw_email ||= $mime->as_string;
+	my $n = length($raw_email);
 	$self->{bytes_added} += $n;
 	print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail;
-	print $w $str, "\n" or wfail;
+	print $w $raw_email, "\n" or wfail;
 
 	# v2: we need this for Xapian
 	if ($self->{want_object_info}) {
 		my $oid = $self->get_mark(":$blob");
-		$self->{last_object} = [ $oid, $n, \$str ];
+		$self->{last_object} = [ $oid, $n, \$raw_email ];
 	}
 	my $ref = $self->{ref};
 	my $commit = $self->{mark}++;
-- 
2.23.0


^ permalink raw reply	[relevance 6%]

* [PATCH 0/4] Various bits to support import_imap_mailbox
  @ 2019-10-09  8:15  5%             ` Eric W. Biederman
  2019-10-09  8:16  6%               ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman
  0 siblings, 1 reply; 4+ results
From: Eric W. Biederman @ 2019-10-09  8:15 UTC (permalink / raw)
  To: Eric Wong; +Cc: meta


Eric Wong,

These should all of my generic patches to support my import_imap_mailbox
script.  The really important patch that adds to the support for List-ID
to public inbox configuration file I have already sent.

I haven't written tests and I get the following test failure when I run
make test

> t/config.t ................. 1/? 
> #   Failed test 'lookup matches expected output'
> #   at t/config.t line 26.
> #     Structures begin differing at:
> #          $got->{listid} = ARRAY(0x55c1d4e3b6a8)
> #     $expected->{listid} = Does not exist
> 
> #   Failed test 'lookup matches expected output for test'
> #   at t/config.t line 42.
> #     Structures begin differing at:
> #          $got->{listid} = ARRAY(0x55c1d508d8d0)
> #     $expected->{listid} = Does not exist
> # Looks like you failed 2 tests of 69.
> t/config.t ................. Dubious, test returned 2 (wstat 512, 0x200)

I haven't looked into what is happening there.

Eric Biederman (4):
      PublicInbox::Import  Smuggle a raw message into add
      PublicInbox::Config: Process mailboxes in sorted order
      Config.pm: Add support for looking up repos by their directories
      IMAPTracker:  Add a helper to track our place in reading imap mailboxes

 lib/PublicInbox/Config.pm      | 19 ++++++++++-
 lib/PublicInbox/IMAPTracker.pm | 73 ++++++++++++++++++++++++++++++++++++++++++
 lib/PublicInbox/Import.pm      |  9 +++---
 3 files changed, 96 insertions(+), 5 deletions(-)

Eric

^ permalink raw reply	[relevance 5%]

Results 1-4 of 4 | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2019-10-07 22:13     Do I need multiple publicinbox.<name>.address values? Alyssa Ross
2019-10-08  0:10     ` Eric Wong
2019-10-08 12:18       ` Eric W. Biederman
2019-10-08 22:11         ` Eric Wong
2019-10-08 22:24           ` Eric W. Biederman
2019-10-08 22:41             ` Eric Wong
2019-10-09  7:58               ` Eric W. Biederman
2019-10-09  8:15  5%             ` [PATCH 0/4] Various bits to support import_imap_mailbox Eric W. Biederman
2019-10-09  8:16  6%               ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman
2019-10-15 20:26  7%                 ` Eric Wong
2019-10-15 23:05  7%                   ` Eric W. Biederman

Code repositories for project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).