From: ebiederm@xmission.com (Eric W. Biederman) To: Eric Wong <e@80x24.org> Cc: meta@public-inbox.org Subject: [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Date: Wed, 09 Oct 2019 03:16:58 -0500 Message-ID: <87muea9vut.fsf_-_@x220.int.ebiederm.org> (raw) In-Reply-To: <87wode9vxw.fsf_-_@x220.int.ebiederm.org> (Eric W. Biederman's message of "Wed, 09 Oct 2019 03:15:07 -0500") Date: Tue, 15 Jan 2019 16:36:42 -0600 I don't trust the MIME type to not munge my email messages in horrible ways upon occasion. Therefore allow for passing in the raw message value instead of trusting the mime object to preserve it. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com> --- As we discussed last time I was working to merge my imap import script. lib/PublicInbox/Import.pm | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm index 137b2b7800c4..b17c9d5c9fa8 100644 --- a/lib/PublicInbox/Import.pm +++ b/lib/PublicInbox/Import.pm @@ -394,16 +394,17 @@ sub add { } my $blob = $self->{mark}++; - my $str = $mime->as_string; - my $n = length($str); + my $raw_email = $mime->{-public_inbox_raw}; + $raw_email ||= $mime->as_string; + my $n = length($raw_email); $self->{bytes_added} += $n; print $w "blob\nmark :$blob\ndata ", $n, "\n" or wfail; - print $w $str, "\n" or wfail; + print $w $raw_email, "\n" or wfail; # v2: we need this for Xapian if ($self->{want_object_info}) { my $oid = $self->get_mark(":$blob"); - $self->{last_object} = [ $oid, $n, \$str ]; + $self->{last_object} = [ $oid, $n, \$raw_email ]; } my $ref = $self->{ref}; my $commit = $self->{mark}++; -- 2.23.0
next prev parent reply other threads:[~2019-10-09 8:17 UTC|newest] Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-07 22:13 Do I need multiple publicinbox.<name>.address values? Alyssa Ross 2019-10-08 0:10 ` Eric Wong 2019-10-08 12:18 ` Eric W. Biederman 2019-10-08 12:23 ` [PATCH] Config.pm: Add support for mailing list information Eric W. Biederman 2019-10-08 22:11 ` Do I need multiple publicinbox.<name>.address values? Eric Wong 2019-10-08 22:24 ` Eric W. Biederman 2019-10-08 22:41 ` Eric Wong 2019-10-09 7:58 ` Eric W. Biederman 2019-10-09 8:15 ` [PATCH 0/4] Various bits to support import_imap_mailbox Eric W. Biederman 2019-10-09 8:16 ` Eric W. Biederman [this message] 2019-10-15 20:26 ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric Wong 2019-10-15 23:05 ` Eric W. Biederman 2019-10-09 8:17 ` [PATCH 2/4] PublicInbox::Config: Process mailboxes in sorted order Eric W. Biederman 2019-10-10 9:43 ` Eric Wong 2019-10-10 11:05 ` Eric W. Biederman 2019-10-09 8:23 ` [PATCH 3/4] Config.pm: Add support for looking up repos by their directories Eric W. Biederman 2019-10-09 8:25 ` [PATCH 4/4] IMAPTracker: Add a helper to track our place in reading imap mailboxes Eric W. Biederman 2019-10-10 19:08 ` ibx->{listid} autoviv fixup [was: [PATCH 0/4] Various bits to support import_imap_mailbox] Eric Wong 2019-10-10 21:23 ` Eric W. Biederman 2019-10-10 8:31 ` Do I need multiple publicinbox.<name>.address values? Eric Wong 2019-10-10 10:56 ` Eric W. Biederman 2019-10-09 11:59 ` Alyssa Ross 2019-10-10 10:06 ` Eric Wong
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://public-inbox.org/README * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=87muea9vut.fsf_-_@x220.int.ebiederm.org \ --to=ebiederm@xmission.com \ --cc=e@80x24.org \ --cc=meta@public-inbox.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
user/dev discussion of public-inbox itself This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/meta git clone --mirror http://czquwvybam4bgbro.onion/meta git clone --mirror http://hjrcffqmbrq6wope.onion/meta git clone --mirror http://ou63pmih66umazou.onion/meta # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 meta meta/ https://public-inbox.org/meta \ meta@public-inbox.org public-inbox-index meta Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta nntp://news.gmane.io/gmane.mail.public-inbox.general note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/public-inbox.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git