* [PATCH 40/82] imap: case-insensitive mailbox name comparisons
2020-06-10 7:03 6% [PATCH 00/82] public-inbox-imapd: read-only IMAP server Eric Wong
@ 2020-06-10 7:04 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-06-10 7:04 UTC (permalink / raw)
To: meta
IMAP RFC 3501 stipulates case-insensitive comparisons, and so
does RFC 977 (NNTP). However, INN (nnrpd) uses case-sensitive
comparisons, so we've always used case-sensitive comparisons for
NNTP to match nnrpd behavior.
Unfortunately, some IMAP clients insist on sending "INBOX" with
caps, which causes problems for us. Since NNTP group names are
typically all lowercase anyways, just force all comparisons to
lowercase for IMAP and warn admins if uppercase-containing
newsgroups won't be accessible over IMAP.
This ensures our existing -nntpd behavior remains unchanged
while being compatible with the expectations of real-world IMAP
clients.
---
lib/PublicInbox/IMAPD.pm | 7 +++++++
t/imap.t | 7 ++++++-
t/imapd.t | 2 +-
3 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/IMAPD.pm b/lib/PublicInbox/IMAPD.pm
index a3a2598661b..966879d8ea5 100644
--- a/lib/PublicInbox/IMAPD.pm
+++ b/lib/PublicInbox/IMAPD.pm
@@ -25,6 +25,13 @@ sub refresh_inboxlist ($) {
my ($self) = @_;
my @names = map { $_->{newsgroup} } @{delete $self->{grouplist}};
my %ns; # "\Noselect \HasChildren"
+
+ if (my @uc = grep(/[A-Z]/, @names)) {
+ warn "Uppercase not allowed for IMAP newsgroup(s):\n",
+ map { "\t$_\n" } @uc;
+ my %uc = map { $_ => 1 } @uc;
+ @names = grep { !$uc{$_} } @names;
+ }
for (@names) {
my $up = $_;
while ($up =~ s/\.[^\.]+\z//) {
diff --git a/t/imap.t b/t/imap.t
index 451b6596bf9..aa262a19632 100644
--- a/t/imap.t
+++ b/t/imap.t
@@ -8,9 +8,14 @@ use PublicInbox::IMAP;
use PublicInbox::IMAPD;
{ # make sure we get '%' globbing right
- my @n = map { { newsgroup => $_ } } (qw(x.y.z x.z.y));
+ my @w;
+ local $SIG{__WARN__} = sub { push @w, @_ };
+ my @n = map { { newsgroup => $_ } } (qw(x.y.z x.z.y IGNORE.THIS));
my $self = { imapd => { grouplist => \@n } };
PublicInbox::IMAPD::refresh_inboxlist($self->{imapd});
+ is(scalar(@w), 1, 'got a warning for upper-case');
+ like($w[0], qr/IGNORE\.THIS/, 'warned about upper-case');
+
my $res = PublicInbox::IMAP::cmd_list($self, 'tag', 'x', '%');
is(scalar($$res =~ tr/\n/\n/), 2, 'only one result');
like($$res, qr/ x\r\ntag OK/, 'saw expected');
diff --git a/t/imapd.t b/t/imapd.t
index 2c4315dec30..a5324f78da0 100644
--- a/t/imapd.t
+++ b/t/imapd.t
@@ -88,7 +88,7 @@ like($e, qr/\bNO\b/, 'got a NO on EXAMINE for non-existent');
ok(!$mic->select('foo') && ($e = $@), 'EXAMINE non-existent');
like($e, qr/\bNO\b/, 'got a NO on EXAMINE for non-existent');
ok($mic->select('inbox.i1'), 'SELECT succeeds');
-ok($mic->examine('inbox.i1'), 'EXAMINE succeeds');
+ok($mic->examine('INBOX.i1'), 'EXAMINE succeeds');
my @raw = $mic->status('inbox.i1', qw(Messages uidnext uidvalidity));
is(scalar(@raw), 2, 'got status response');
like($raw[0], qr/\A\*\x20STATUS\x20inbox\.i1\x20
^ permalink raw reply related [relevance 7%]
* [PATCH 00/82] public-inbox-imapd: read-only IMAP server
@ 2020-06-10 7:03 6% Eric Wong
2020-06-10 7:04 7% ` [PATCH 40/82] imap: case-insensitive mailbox name comparisons Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-06-10 7:03 UTC (permalink / raw)
To: meta
So I finally wrote my first IMAP server! And I'm actually
fairly satisfied with how it's turning out to support a bunch
of other performance + scalability work I've wanted to do.
Some previous notes here:
https://public-inbox.org/meta/20200609113442.GA16856@dcvr/
I finally seem to have gotten it to play nicely with mutt header
caching, so it's fit for public consumption :)
imaps://news.public-inbox.org/INBOX.comp.mail.public-inbox.meta.0
You can use any username+password, and AUTH=ANONYMOUS also
works if your client does that.
It doesn't support UTF-7 (mailbox names) or advertise UTF-8
in CAPABILITIES, yet; I still have RFCs to read :P
And there's a bunch of new things which could use some
testing from non-mutt/mbsync/offlineimap users.
Maybe you'll find some client-side bugs like I did :P
v1 reindexing also gets a little bit of parallelism :)
Anyways, I'll probably be porting some of the scalability
and slow-storage work to older parts of the code before
fiddling with more IMAP extensions.
Eric Wong (82):
doc: add some IMAP standards
nntpd: restrict allowed newsgroup names
preliminary imap server implementation
inboxidle: new class to detect inbox changes
imap: support IDLE
msgmap: split ->max into its own method
imap: delay InboxIdle start, support refresh
imap: implement STATUS command
imap: use Text::ParseWords::parse_line to handle quoted words
imap: support LIST command
t/imapd: support FakeInotify and KQNotify
imap: support fetch for BODYSTRUCTURE and BODY
eml: each_part: single part $idx is 1
imap: allow fetch of partial of BODY[...] and headers
imap: always include `resp-text' in responses
imap: split out unit tests and benchmarks
imap: fix multi-message partial header fetches
imap: simplify partial fetch structure
imap: support sequence number FETCH
imap: do not include ".PEEK" in responses
imap: support the CLOSE command
imap: speed up HEADER.FIELDS[.NOT] range fetches
git: async: flatten the inflight array
git: do our own read buffering for cat-file
imap: use git-cat-file asynchronously
git: idle rbuf for async
imap: support LSUB command
imap: FETCH: support comma-delimited ranges
add imapd compression test
testcommon: tcp_(server|connect): BAIL_OUT on failure
*deflate: drop invalid comment about rbuf
imap: fix pipelining with async git
git: cat_async: provide requested OID + "missing" on missing blobs
git: move async_cat reference to PublicInbox::Git
git: async: automatic retry on alternates change
imapclient: wrapper for Mail::IMAPClient
xt: add imapd-validate and imapd-mbsync-oimap
imap: support out-of-bounds ranges
xt/perf-imap-list: time refresh_inboxlist
imap: case-insensitive mailbox name comparisons
imap: break giant inboxes into sub-inboxes of 50K messages
imap: start introducing iterative config reloading
imap: require ".$UID_MIN-$UID_END" suffix
imapd: ensure LIST is sorted alphabetically, for now
imap: omit $UID_END from mailbox name, use index
t/config.t: always compare against git bool behavior
xt/*: show some tunable parameters
imap: STATUS and LIST are case-insensitive, too
imap: EXAMINE/STATUS: return correct counts
imap: avoid uninitialized warnings on incomplete commands
imap: start parsing out queries for SQLite and Xapian
imap: SEARCH: clamp results to the 50K UID range
imap: allow UID range search on timestamps
over: get_art: use dbh->prepare_cached
search: index byte size of a message for IMAP search
search: index UID for IMAP search, too
imap: remove dummies from sequence number FETCH
imap: compile UID FETCH to opcodes
imap: UID FETCH: optimize for smsg-only case
imap: UID FETCH: optimize (UID FLAGS) harder
imap: IDLE: avoid extraneous wakeups, keep-alive
imap: 30 minute auto-logout timer
imap: split ->logged_in attribute into a separate class
searchidx: v1 (re)-index uses git asynchronously
index: account for CRLF conversion when storing bytes
imap: rely on smsg->{bytes} for RFC822.SIZE
imap: UID FETCH requires at least one data item
imap: LIST shows "INBOX" in all caps
imap: support 8000 octet lines
imap: reinstate some message sequence number support
imap: cleanup ->{uid_base} usage
imap: FETCH: more granular CRLF conversion
imap: further speed up HEADER.FIELDS FETCH requests
imap: FETCH: try to make fake MSNs sequentially
imap: STATUS/EXAMINE: rely on SQLite overview
imap: UID SEARCH: support multiple ranges
imap: wire up Xapian search, msn SEARCH and multiple ranges
imap: misc cleanups and notes
imapd: don't bother sorting LIST output
imap: drop non-UID SEARCH for now
over: uid_range: remove LIMIT
imap: FETCH: proper MSN => UID mapping for requests
Documentation/public-inbox-imapd.pod | 91 ++
Documentation/standards.perl | 10 +
MANIFEST | 18 +
lib/PublicInbox/Config.pm | 18 +
lib/PublicInbox/Daemon.pm | 24 +-
lib/PublicInbox/DummyInbox.pm | 22 +
lib/PublicInbox/Eml.pm | 9 +-
lib/PublicInbox/FakeInotify.pm | 59 ++
lib/PublicInbox/Git.pm | 163 +--
lib/PublicInbox/GitAsyncCat.pm | 51 +
lib/PublicInbox/IMAP.pm | 1397 ++++++++++++++++++++++++++
lib/PublicInbox/IMAPClient.pm | 119 +++
lib/PublicInbox/IMAPD.pm | 114 +++
lib/PublicInbox/IMAPdeflate.pm | 126 +++
lib/PublicInbox/Import.pm | 2 +-
lib/PublicInbox/In2Tie.pm | 17 +
lib/PublicInbox/Inbox.pm | 33 +-
lib/PublicInbox/InboxIdle.pm | 79 ++
lib/PublicInbox/KQNotify.pm | 66 ++
lib/PublicInbox/Lock.pm | 7 +
lib/PublicInbox/MsgIter.pm | 2 +-
lib/PublicInbox/Msgmap.pm | 20 +-
lib/PublicInbox/NNTPD.pm | 12 +-
lib/PublicInbox/NNTPdeflate.pm | 1 -
lib/PublicInbox/Over.pm | 50 +-
lib/PublicInbox/Search.pm | 32 +-
lib/PublicInbox/SearchIdx.pm | 89 +-
lib/PublicInbox/SearchIdxShard.pm | 11 +-
lib/PublicInbox/Smsg.pm | 8 +-
lib/PublicInbox/TestCommon.pm | 7 +-
lib/PublicInbox/V2Writable.pm | 10 +-
script/public-inbox-imapd | 14 +
t/config.t | 15 +-
t/eml.t | 2 +-
t/git.t | 40 +-
t/imap.t | 133 +++
t/imapd-tls.t | 204 ++++
t/imapd.t | 398 ++++++++
t/import.t | 5 +-
t/inbox_idle.t | 72 ++
t/nntpd.t | 5 +-
t/over.t | 3 +
t/search.t | 19 +
xt/cmp-msgstr.t | 1 -
xt/cmp-msgview.t | 1 -
xt/eml_check_limits.t | 6 +-
xt/git_async_cmp.t | 2 +-
xt/imapd-mbsync-oimap.t | 132 +++
xt/imapd-validate.t | 177 ++++
xt/mem-msgview.t | 1 +
xt/msgtime_cmp.t | 1 -
xt/perf-msgview.t | 1 -
52 files changed, 3718 insertions(+), 181 deletions(-)
create mode 100644 Documentation/public-inbox-imapd.pod
create mode 100644 lib/PublicInbox/DummyInbox.pm
create mode 100644 lib/PublicInbox/FakeInotify.pm
create mode 100644 lib/PublicInbox/GitAsyncCat.pm
create mode 100644 lib/PublicInbox/IMAP.pm
create mode 100644 lib/PublicInbox/IMAPClient.pm
create mode 100644 lib/PublicInbox/IMAPD.pm
create mode 100644 lib/PublicInbox/IMAPdeflate.pm
create mode 100644 lib/PublicInbox/In2Tie.pm
create mode 100644 lib/PublicInbox/InboxIdle.pm
create mode 100644 lib/PublicInbox/KQNotify.pm
create mode 100644 script/public-inbox-imapd
create mode 100644 t/imap.t
create mode 100644 t/imapd-tls.t
create mode 100644 t/imapd.t
create mode 100644 t/inbox_idle.t
create mode 100644 xt/imapd-mbsync-oimap.t
create mode 100644 xt/imapd-validate.t
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-06-10 7:03 6% [PATCH 00/82] public-inbox-imapd: read-only IMAP server Eric Wong
2020-06-10 7:04 7% ` [PATCH 40/82] imap: case-insensitive mailbox name comparisons Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).