user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: meta@public-inbox.org
Subject: Re: [PATCH] PublicInbox::Import Extend add with a optional raw message parameter
Date: Sat, 18 May 2019 08:03:00 +0000	[thread overview]
Message-ID: <20190518080300.q2klxia2uymnoxyi@dcvr> (raw)
In-Reply-To: <87h89tzvp5.fsf@xmission.com>

"Eric W. Biederman" <ebiederm@xmission.com> wrote:
> 
> I don't trust the MIME type to not munge my email messages in horrible
> ways upon occasion.  Therefore  allow for passing in the raw message
> value instead of trusting the mime object to preserve it.
> 
> Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>

I've had the same concern in the past about Email::MIME and
Email::Simple.  But after reading the code for Email::MIME,
Email::Simple and Email::{MIME,Simple}::Header, I don't think
the implementation of Email::MIME->as_string and all methods it
calls does anything unreasonable.

The only notable munging they seem to do is make irrelevant
whitespace changes in headers and maybe fix quoting in headers.
No body changes AFAIK.

> The context here is because the only copy of messages that I save
> I save in public-inbox I don't want to have to worry about losing
> information.  So I just pass the raw email_str to add.
> 
> I expect if I were to export these lists public I would want to do
> some more but for now I am just putting them in public-inbox
> so that I can read and archive the lists locally.

I worry about public archives getting badly munged, too.

>  lib/PublicInbox/Import.pm     | 10 +++++-----
>  lib/PublicInbox/V2Writable.pm |  8 ++++----
>  2 files changed, 9 insertions(+), 9 deletions(-)

Did you have plans to modify -mda/-watch or another script to
use this?

> diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
> index 81a38fb6987d..0a63784414f2 100644
> --- a/lib/PublicInbox/Import.pm
> +++ b/lib/PublicInbox/Import.pm
> @@ -359,7 +359,7 @@ sub clean_tree_v2 ($$$) {
>  # returns undef on duplicate
>  # returns the :MARK of the most recent commit
>  sub add {
> -	my ($self, $mime, $check_cb) = @_; # mime = Email::MIME
> +	my ($self, $mime, $check_cb, $email_str) = @_; # mime = Email::MIME

I usually place callback args at the end of the arg list so
it's easy to write:

	$im->add($mime, sub {
		# ...
	});

So having a parameter after the sub{} is a bit ugly...
If I had to support this, I think I'd accept $mime being
a plain hashref:

	if (ref($mime) eq 'HASH') {
		$raw = $mime->{raw};
		$mime = $mime->{mime};
	} else {
		$raw = $mime->as_string;
	}

But, I'm still on the fence about the idea...

Side note: I'm also taking the opportunity to use "$raw" instead
of "$str", because I've been bitten by the difference header_raw
vs header_str in the Email::MIME API, so consistency with
that API would be good, here.

  reply	other threads:[~2019-05-18  8:03 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-17  2:00 Eric W. Biederman
2019-05-18  8:03 ` Eric Wong [this message]
2019-05-18 15:09   ` Eric W. Biederman
2019-05-18 21:39     ` Eric Wong
2019-05-19 18:14       ` Eric W. Biederman
2019-05-19 22:04         ` Eric Wong
2019-05-24 11:44           ` TLS support and event loops Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190518080300.q2klxia2uymnoxyi@dcvr \
    --to=e@80x24.org \
    --cc=ebiederm@xmission.com \
    --cc=meta@public-inbox.org \
    --subject='Re: [PATCH] PublicInbox::Import Extend add with a optional raw message parameter' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).