From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6315 166.70.0.0/16 X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from out01.mta.xmission.com (out01.mta.xmission.com [166.70.13.231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 0D9491F453; Fri, 28 Sep 2018 21:01:41 +0000 (UTC) Received: from in01.mta.xmission.com ([166.70.13.51]) by out01.mta.xmission.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1g5ztK-0003Qc-M2; Fri, 28 Sep 2018 15:01:38 -0600 Received: from [105.184.227.67] (helo=x220.xmission.com) by in01.mta.xmission.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.87) (envelope-from ) id 1g5ztJ-0003Ev-Ar; Fri, 28 Sep 2018 15:01:38 -0600 From: ebiederm@xmission.com (Eric W. Biederman) To: Johannes Berg Cc: Eric Wong , meta@public-inbox.org References: <87k1q1bky6.fsf@xmission.com> <20180712014715.dn5aouayoa3uejp4@dcvr> <87k1q07dyc.fsf@xmission.com> <20180712230946.mqv3yjw4aabf7xrf@dcvr.yhbt.net> <878t6f1ch7.fsf@xmission.com> <20180713230746.GA2680@dcvr> <1538165409.14416.92.camel@sipsolutions.net> Date: Fri, 28 Sep 2018 23:01:19 +0200 In-Reply-To: <1538165409.14416.92.camel@sipsolutions.net> (Johannes Berg's message of "Fri, 28 Sep 2018 22:10:09 +0200") Message-ID: <87zhw1cp5s.fsf@xmission.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-XM-SPF: eid=1g5ztJ-0003Ev-Ar;;;mid=<87zhw1cp5s.fsf@xmission.com>;;;hst=in01.mta.xmission.com;;;ip=105.184.227.67;;;frm=ebiederm@xmission.com;;;spf=neutral X-XM-AID: U2FsdGVkX18zVOKdGXiM5KlEmILapltG/tEhWKygBrw= X-SA-Exim-Connect-IP: 105.184.227.67 X-SA-Exim-Mail-From: ebiederm@xmission.com Subject: Re: IMAP server [was: Q: V2 format] X-SA-Exim-Version: 4.2.1 (built Thu, 05 May 2016 13:38:54 -0600) X-SA-Exim-Scanned: Yes (on in01.mta.xmission.com) List-Id: Johannes Berg writes: > Sorry to just jump into an old thread; I was wondering about IMAP server > support as well, in particular because unlike NNTP that allows pushing > the search to the server, and that would be useful for local archives. > >> Hosting an IMAP/POP3 server is way more overhead for the admin >> as it requires storing user credentials and storing per-reader >> state. So the preference is to do NNTP as well as possible and >> layer the complexity of per-user account data on top of it. > > I'm not really sure that's true; dovecot, for example, provides their > lists archives via anonymous IMAP: > https://www.dovecot.org/mailinglists.html > > They have instructions here on how to do that over dovecot: > https://wiki2.dovecot.org/HowTo/ReadOnlyArchive > > In particular: > > /var/home/anonymous/control# ls -la > drwxr-xr-x 3 root root 4096 May 25 15:43 ./ > drwxr-xr-x 3 anondove root 4096 Mar 20 14:39 .imap/ > -rw-r--r-- 1 root root 33 May 25 15:43 .subscriptions > > Create the .subscriptions file manually to contain all the mailboxes > you. Note that the control directory isn't writable by anondove, so > that the subscriptions can't be changed. > > [...] > > * INBOX must always exists even if it's empty. Make sure it's not > writable. > * Make sure the mail directory isn't writable so users can't create new > mailboxes. > * The mboxes can be placed in the directory itself, or symlinks can be > used. Above you'll see that mailman places all Dovecot archives under > /var/home/archives. Make sure none of these files are writable by > anondove. > > They also set up some read-only ACLs, I think to make the read-only > state clear to the user agent, but of course a public-inbox IMAP server > can hard-code all of this and not accept any write commands to start > with. > > Anyway, just FYI; since I don't know perl at all I don't think I'll be > doing any work on this. I have looked at gnus and there is support in there for performing searches via the old gmane web interface. Public inbox already provides an attribute that tells you what the web server is. So all it will really take is someone with a little time to wire up the search interface. Beyond that if you have the archives local (and that is easy) it is quite possible to just git grep through them and find things of interest. I should verify this but I don't think IMAP has a good version of the NNTP overview database. Which seems to make IMAP quite a bit slower for reading news. Certainly gnus+public-inbox locally is running quite a bit faster than my old gnus+cyrus-imap configuration. I tried to read through the IMAP search specification to see how it compares with what public-inbox makes available and I did not get particularly far. It was not easy to match up the various search capabilities. The biggest issue is that IMAP tends to not talk about message-ids. Where that is fundamentally one of the most important fields to index if you are dealing with threaded mail. So long story short while I am not opposed to a read-only IMAP configuration I think NNTP has much to recommend it. I do think we need little things like SSL support for NNTP. Just to prevent inappropriate access to traffic in flight. It won't be for a while yet but I have some scripts I need to push at least to the public-inbox scripts directory that simplify the process taking a single email address subscribing to email and sorting it out into different public-inbox git archives. Currently I have every mailling list I am subscribed to pushed into public-inbox. Eric