From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=AWL,BAYES_00, RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from out02.mta.xmission.com (out02.mta.xmission.com [166.70.13.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 844D71F55B; Sat, 16 May 2020 10:50:49 +0000 (UTC) Received: from in01.mta.xmission.com ([166.70.13.51]) by out02.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jZuP2-0005uA-95; Sat, 16 May 2020 04:50:48 -0600 Received: from ip68-227-160-95.om.om.cox.net ([68.227.160.95] helo=x220.xmission.com) by in01.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.87) (envelope-from ) id 1jZuP1-0002hi-Gu; Sat, 16 May 2020 04:50:48 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: Eric Wong Cc: meta@public-inbox.org References: <87eeyvmx74.fsf@x220.int.ebiederm.org> <20200513193144.GA9299@dcvr> <87ftc3mrq6.fsf@x220.int.ebiederm.org> <20200513221715.GA11718@dcvr> <877dxelmsr.fsf@x220.int.ebiederm.org> <87ftc0c3r4.fsf_-_@x220.int.ebiederm.org> <87a728c3p3.fsf_-_@x220.int.ebiederm.org> <87o8qoao09.fsf@x220.int.ebiederm.org> <20200515225644.GB4131@dcvr> Date: Sat, 16 May 2020 05:47:11 -0500 In-Reply-To: <20200515225644.GB4131@dcvr> (Eric Wong's message of "Fri, 15 May 2020 22:56:44 +0000") Message-ID: <87mu6888cw.fsf@x220.int.ebiederm.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1jZuP1-0002hi-Gu;;;mid=<87mu6888cw.fsf@x220.int.ebiederm.org>;;;hst=in01.mta.xmission.com;;;ip=68.227.160.95;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX19TByPAfaBUxN0mU7ODfgo5IZ6ld8GAKls= X-SA-Exim-Connect-IP: 68.227.160.95 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: Re: [PATCH 2/2] imap_fetch: Add a command to continuously fetch from an imap mailbox X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) List-Id: Eric Wong writes: > "Eric W. Biederman" wrote: >> ebiederm@xmission.com (Eric W. Biederman) writes: >> >> > The command imap_fetch connects to the specified imap mailbox and >> > fetches any unfetch messages than waits with imap idle until there are >> > more messages to fetch. >> > >> > By default messages are placed in the specified public inbox mailbox. >> > The value of List-ID is consulted and if it is present used to >> > select an alternate public-inbox mailbox to place the messages in. >> > >> > The email messages are placed without modification into the public >> > inbox repository so minimize changes of corruption or of loosing >> > valuable information. I use the command imap_fetch for all of my >> > email and not just a mailling list mirror so I don't want automation >> > to accidentally cause something important to be lost. > > Btw, Email::MIME usage is gone from 1.5.0 due to nasty > performance problems and replaced by PublicInbox::Eml. Eml > should be completely non-destructive unless somebody sends an > abusive message which exceeds the new safety limits; in which > case it won't OOM or burn CPU like E::M did. > > That said, {-public_inbox_raw} still works and Eml looks > like a drop-in replacement as far as imap_fetch is concerned. I almost did that. But I looked and saw PublicInbox::MIME still present and a number of other references to Email::MIME so I wasn't certain exactly how that was being handled. But since Email::MIME still worked I didn't mess with that. >> > No email messages are deleted from the server instead IMAPTracker >> > is used to remember which messages were downloaded. > > Yup. I've integrated IMAPTracker into a local branch, already. > I think it could be reused for tracking NNTP fetches, too, > since UIDs and NNTP article numbers seem interchangeable. True. I guess it is possible to make the equivalent of my imap_fetch for NNTP queries as well. Does NNTP have the equivalent of IDLE where it is possible to get timely updates of new messages? I suppose I should see if the git protocol does as well. I am trying to figure out what the best way to track other public inbox git repositories. Currently I have a script that every N minutes does git remote update. Which is ok. But I think something like imap_fetch might be nicer. >> Bah. I sent this a little too soon. The patch needs this small >> incremental fix to actually work. > > No worries. > > Btw, any reason you create the SSLSocket yourself instead of > passing (Ssl => \@SSL_Socket_options) to IMAPClient->new? When I read the documentation it looked like that was the way to do things. Even now when I reread the documentation that looks like the way to go. Especially if I wanted to be certain the connection was encrypted. Eric