From: ebiederm@xmission.com (Eric W. Biederman)
To: Johannes Berg <johannes@sipsolutions.net>
Cc: Eric Wong <e@80x24.org>, meta@public-inbox.org
Subject: Re: IMAP server [was: Q: V2 format]
Date: Mon, 01 Oct 2018 10:51:21 +0200 [thread overview]
Message-ID: <87y3biyrqu.fsf@xmission.com> (raw)
In-Reply-To: <1538379966.3126.13.camel@sipsolutions.net> (Johannes Berg's message of "Mon, 01 Oct 2018 09:46:06 +0200")
Johannes Berg <johannes@sipsolutions.net> writes:
> On Fri, 2018-09-28 at 23:01 +0200, Eric W. Biederman wrote:
>>
>> I have looked at gnus and there is support in there for performing
>> searches via the old gmane web interface. Public inbox already provides
>> an attribute that tells you what the web server is. So all it will
>> really take is someone with a little time to wire up the search
>> interface.
>
> That's ... interesting, but of course completely out-of-band. I'm not
> sure it should or could be advocated that every email client actually
> implement that :-)
>
> But if you think broader than that, you don't even necessarily need a
> web server to run p-i.
>> Beyond that if you have the archives local (and that is easy) it is
>> quite possible to just git grep through them and find things of
>> interest.
>
> That also doesn't use the index, not sure how that's any better?
So for linux-kernel. I have 7G for the git email archive and 65G more
for the indexes. Which makes the indexes quite expensive. So for
personal use I am not certain an archive is a benefit. Especially when
the email archive fits in ram and the index does not.
I have to wonder if there is a way to make the indexes an order of
magnitude smaller.
>> I should verify this but I don't think IMAP has a good version of the
>> NNTP overview database. Which seems to make IMAP quite a bit slower for
>> reading news. Certainly gnus+public-inbox locally is running quite a
>> bit faster than my old gnus+cyrus-imap configuration.
>
> IMAP servers typically should do header/MIME parsing, so you should be
> able to query such a thing - but not as easily as XOVER, I suppose.
>
> However, I think FETCH could be made to return the data similar to
> XOVER, though it may not be backed by a pre-created database file, and
> it depends on what the client does to show the overview in the first
> place.
>> I tried to read through the IMAP search specification to see how it
>> compares with what public-inbox makes available and I did not get
>> particularly far. It was not easy to match up the various search
>> capabilities. The biggest issue is that IMAP tends to not talk
>> about message-ids. Where that is fundamentally one of the most
>> important fields to index if you are dealing with threaded mail.
>
> You can search for arbitrary headers in search by using
>
> HEADER <field-name> <string>
>
> where the string is "contains", so you can use it for both Message-Id
> and References headers.
>> So long story short while I am not opposed to a read-only IMAP
>> configuration I think NNTP has much to recommend it. I do think we need
>> little things like SSL support for NNTP. Just to prevent inappropriate
>> access to traffic in flight.
>
> Sure. I'm not saying NNTP is bad, just saying that the choice of clients
> is rather limited. Also, posting isn't supported over NNTP, so if I had
> it all in my email client I could read in the public-inbox archive, and
> respond via normal email.
The thing I can confirm and I have gotten as far as is that nntp has a
sequential message id, and IMAP has a sequential message id and
public-inbox has a sequential message id (now reliably based upon the
order of the messages in the git archive). So it is very possible to
have a read-only IMAP view.
The really noticable downside of IMAP is that it does want to keep the
status of messages you have read on the server. That makes a read-only
archive a bit of a pain.
So I am not certain the choice of clients when you restrict IMAP to what
is an advantage. Nor am I certain the general IMAP search functionality
maps well to what public-inbox indexes or people want to search for.
Which is me again saying while things can make I am not certain IMAP is
the best protocol for the job.
>> It won't be for a while yet but I have some scripts I need to push at
>> least to the public-inbox scripts directory that simplify the process
>> taking a single email address subscribing to email and sorting it out
>> into different public-inbox git archives. Currently I have every
>> mailling list I am subscribed to pushed into public-inbox.
>
> :-)
I do love that public-inbox makes it very easy to archive all my content
and still be able to take it all with me when I travel.
Eric
prev parent reply other threads:[~2018-10-01 8:51 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-11 20:01 Q: V2 format Eric W. Biederman
2018-07-11 21:18 ` Konstantin Ryabitsev
2018-07-11 21:41 ` Eric W. Biederman
2018-07-12 1:47 ` Eric Wong
2018-07-12 13:58 ` Eric W. Biederman
2018-07-12 23:09 ` Eric Wong
2018-07-13 13:39 ` Eric W. Biederman
2018-07-13 20:03 ` Eric W. Biederman
2018-07-13 22:22 ` msgmap serial number regeneration [was: Q: V2 format] Eric Wong
2018-07-14 19:01 ` Eric W. Biederman
2018-07-15 3:18 ` Eric Wong
2018-07-16 15:20 ` Eric W. Biederman
2018-07-13 22:02 ` bug: v2 deletes on incremental fetch " Eric Wong
2018-07-13 22:51 ` Eric W. Biederman
2018-07-14 0:46 ` [PATCH] v2writable: unindex deleted messages after incremental fetch Eric Wong
2018-07-13 23:07 ` IMAP server [was: Q: V2 format] Eric Wong
2018-07-13 23:12 ` Eric W. Biederman
2018-09-28 20:10 ` Johannes Berg
2018-09-28 21:01 ` Eric W. Biederman
2018-10-01 7:46 ` Johannes Berg
2018-10-01 8:51 ` Eric W. Biederman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87y3biyrqu.fsf@xmission.com \
--to=ebiederm@xmission.com \
--cc=e@80x24.org \
--cc=johannes@sipsolutions.net \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).