From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 60DCD1F55B for ; Sat, 16 May 2020 10:03:22 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH/RFC 0/2] recurse into message/rfc822 parts Date: Sat, 16 May 2020 10:03:20 +0000 Message-Id: <20200516100322.19793-1-e@yhbt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Multipart parts aren't the only things which nest, message/rfc822 attachments can contain any sort of full message. I noticed gmime supports this while evaluating(*) it to replace Email::MIME, and it seems needed for IMAP support. Email::MIME seemed to attempt to support descending into message/*, but didn't do it properly, so it never got triggered. There's definitely some message/rfc822 attachments in various archives out there, and it looks like message/global is becoming a thing, and some message/news for legacy stuff... gmime supports message/rfc2822, too, which doesn't seem specified anywhere... Search indexing multiple From/To/Cc/Subject/Message-ID/List-Id headers is straightforward, Date is not... Also, note the t/data/message_embed.eml example includes a circular reference :) I have no intention of doing threading for attached messages (AFAIK no mail client does this), but maybe making the contents of References / In-Reply-To is a helpful thing in general. Eric Wong (2): t/psgi_attach: assert message/* parts are downloadable descend into message/(rfc822|news|global) parts MANIFEST | 1 + lib/PublicInbox/Eml.pm | 37 ++++++-- lib/PublicInbox/MsgIter.pm | 6 +- lib/PublicInbox/SearchIdx.pm | 47 ++++++---- lib/PublicInbox/View.pm | 30 ++++++- t/data/message_embed.eml | 163 +++++++++++++++++++++++++++++++++++ t/eml.t | 28 ++++++ t/psgi_attach.t | 27 ++++++ t/search.t | 25 ++++++ 9 files changed, 336 insertions(+), 28 deletions(-) create mode 100644 t/data/message_embed.eml