user/dev discussion of public-inbox itself
 help / color / Atom feed
* MUA and client-side limits w.r.t. IMAP
@ 2020-05-19  9:00 Eric Wong
  2020-06-04  8:57 ` Eric Wong
  0 siblings, 1 reply; 2+ messages in thread
From: Eric Wong @ 2020-05-19  9:00 UTC (permalink / raw)
  To: meta

TL;DR: is 50K messages a reasonable IMAP folder size?

So far, the IMAP daemon code shares quite a bit of data and
logic with the NNTP code.  NNTP article numbers are reusable as
IMAP UIDs; and for small inboxes, newsgroups can be an IMAP
folder.  But that only works for small inboxes...

One major difference between IMAP and NNTP is every NNTP client
I've looked at (even going back to the 90s) has default limits
on the number of messages it can fetch for a newsgroup.

I'm not sure if any IMAP client has NNTP-like limits by default.
Instead, they'll attempt to fetch headers for every single
message in that IMAP folder.

Bigger public-inboxes have 300-400K of messages per epoch.  I
don't think most MUAs will be happy with even that, not to
mention monster multi-epoch inboxes like LKML.

So I'm thinking 10K-60K is more reasonable these days, at least
on the hardware I have and TokyoCabinet header caching in mutt.
10K is a pretty small window for lists like LKML; and 50K only
takes a few seconds for my old Centrino laptop over IMAP.

All messages can still be exposed via extra folders with numbers
tacked on:

* default, same as newsgroup name, messages "expire"
inbox.comp.version-control.git           =>    most recent 50K messages

* messages never expire from these IMAP folders:
inbox.comp.version-control.git.400000    =>    350,001..400K
inbox.comp.version-control.git.350000    =>    300,001..350K
inbox.comp.version-control.git.300000    =>    250,001..300K
...

I don't want to expose a configuration knob, here; since
users can get thrown off and have to resync if the admin
changes the window size.

I don't know about MUAs other than mutt; but is 50K a reasonable
folder size?

Also, IMAP search may be able to cross into the older ranges,
not sure, yet...

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: MUA and client-side limits w.r.t. IMAP
  2020-05-19  9:00 MUA and client-side limits w.r.t. IMAP Eric Wong
@ 2020-06-04  8:57 ` Eric Wong
  0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2020-06-04  8:57 UTC (permalink / raw)
  To: meta

Eric Wong <e@yhbt.net> wrote:
> * default, same as newsgroup name, messages "expire"
> inbox.comp.version-control.git           =>    most recent 50K messages

I think that's best left an empty parent folder, since having
the most recent X messages means the client will have to
refetch/resync that entire 50K again when a new
numerically-suffixed mailbox is needed to hold older messages.

> * messages never expire from these IMAP folders:
> inbox.comp.version-control.git.400000    =>    350,001..400K
> inbox.comp.version-control.git.350000    =>    300,001..350K
> inbox.comp.version-control.git.300000    =>    250,001..300K
> ...

Now, I'm thinking more along the lines of:

inbox.comp.version-control.git.7    =>    350,001..400K
inbox.comp.version-control.git.6    =>    300,001..350K
inbox.comp.version-control.git.5    =>    250,001..300K
inbox.comp.version-control.git.4    =>    200,001..250K
inbox.comp.version-control.git.3    =>    150,001..200K
inbox.comp.version-control.git.2    =>    100,001..150K
inbox.comp.version-control.git.1    =>     50,001..100K
inbox.comp.version-control.git.0    =>          1..50K

Since bigger numbers are harder-to-read and memorize.
But LKML is currently up to 72...

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, back to index

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-19  9:00 MUA and client-side limits w.r.t. IMAP Eric Wong
2020-06-04  8:57 ` Eric Wong

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git