From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id C43BC1F55F; Tue, 12 Sep 2023 22:40:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1694558434; bh=rORemhD+Uy76V1s7wIl1yKYlIjKpc2ZAqpl2vYB3IPU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ofBu12CFeLtWS/jhTqetLeWcbBAH2dIsJiMvxiCosmrfylMhPapVAe7onNlYXI9+E ycHZrNYx44fTThxDemhac6u9sU5+0O3V8SiFXdZqfr1CwQ3iVEGJOiQ7MLa38Tq+Vn B8/mxSObcUYzFv5m29MttZNdIAhAsvCqm68/RL38= Date: Tue, 12 Sep 2023 22:40:34 +0000 From: Eric Wong To: Konstantin Ryabitsev Cc: meta@public-inbox.org Subject: [RFC] pop3: support `?limit=$NUM' parameter in mailbox name Message-ID: <20230912224034.M689061@dcvr> References: <20230912-impart-swinger-4c2434@meerkat> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20230912-impart-swinger-4c2434@meerkat> List-Id: Konstantin Ryabitsev wrote: > Hello: > > I've been playing around with pop3, and I'm wondering if we can improve its > usability by adding a "last NNN messages" pseudo-folder. Currently, if someone > wants to access the git mailing list archive via pop3, they have to do the > following: > > - know that the username should be $(uuidgen)@org.kernel.vger.git.1 (the > default username would access slice 0, right? Or is it the last 50,000 > messages?) The /\.[0-9]+$/ slice is actually optional for POP3. `$(uuidgen)@org.kernel.vger.git' alone will get you the latest 50k. > - wait for their client to retrieve tens of thousands of unread messages on > first access Perhaps 50K is too much? I figured clients would have a way to limit that, but I don't really pay attention to POP3 clients... Patch below adds a `?limit=$NUM' parameter, but I'm not sure if `?' or `=' are allowed in POP3 mailbox names. mpop(1) doesn't complain... Haven't looked at other POP3 clients. > - if the remote archive rolls over to the next slice, they have to edit their > account info to get new messages (unless I'm wrong about #1) Yeah, that only applies to IMAP. IMAP is a pain since connections can be long-lived and per-connection MSN <=> UID mappings can grow without bound after more messages arrive. Perhaps our -imapd can be less nice and forcibly terminate connections if the most recent window gets too big. > Perhaps the default could be slightly different: > > - $(uuidgen)@org.kernel.vger.git would start with an empty view (or something > like the last 10 messages) Small numbers would be very unuseful, too, I think... > - it would only get any new messages added to the archive > > I think this would be a friendlier experience, but not sure how difficult it > would be to implement. I'm also not 100% sure all my assumptions are correct, > so please feel free to correct me. No worries, the POP3 stuff hasn't seen much use. IMAP's been hammered relentlessly by bots on my server, at least :> Lightly-tested patch to support ?limit=$NUM -------8<-------- Subject: [PATCH] pop3: support `?limit=$NUM' parameter in mailbox name I'm not sure if `?' or `=' are allowed characters in POP3 mailbox names. In fact, I can't find any information on valid characters allowed in RFC 1081 nor RFC 1939. In any case, it seems to work fine with mpop. --- lib/PublicInbox/POP3.pm | 18 ++++++++++++------ xt/pop3d-mpop.t | 4 ++-- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/lib/PublicInbox/POP3.pm b/lib/PublicInbox/POP3.pm index d32793e4..4a21ef5e 100644 --- a/lib/PublicInbox/POP3.pm +++ b/lib/PublicInbox/POP3.pm @@ -41,6 +41,7 @@ use PublicInbox::IMAP; # for UID slice stuff use constant { LINE_MAX => 512, # XXX unsure + UID_SLICE => PublicInbox::IMAP::UID_SLICE, }; # XXX FIXME: duplicated stuff from NNTP.pm and IMAP.pm @@ -70,20 +71,25 @@ sub cmd_user ($$) { my $user = $1; $user =~ tr/-//d; # most have dashes, some (dbus-uuidgen) don't $user =~ m!\A[a-f0-9]{32}\z!i or return \"-ERR user has no UUID\r\n"; - my $slice; - $mailbox =~ s/\.([0-9]+)\z// and $slice = $1 + 0; + + my $limit = UID_SLICE; + $mailbox =~ s/\?limit=([0-9]+)\z// and + $limit = $1 > UID_SLICE ? UID_SLICE : $1; + + my $slice = $mailbox =~ s/\.([0-9]+)\z// ? $1 + 0 : undef; + my $ibx = $self->{pop3d}->{pi_cfg}->lookup_newsgroup($mailbox) // return \"-ERR $mailbox does not exist\r\n"; my $uidmax = $ibx->mm(1)->num_highwater // 0; if (defined $slice) { - my $max = int($uidmax / PublicInbox::IMAP::UID_SLICE); + my $max = int($uidmax / UID_SLICE); my $tip = "$mailbox.$max"; return \"-ERR $mailbox.$slice does not exist ($tip does)\r\n" if $slice > $max; - $self->{uid_base} = $slice * PublicInbox::IMAP::UID_SLICE; + $self->{uid_base} = ($slice * UID_SLICE) + UID_SLICE - $limit; $self->{slice} = $slice; - } else { # latest 50K messages - my $base = $uidmax - PublicInbox::IMAP::UID_SLICE; + } else { # latest $limit messages + my $base = $uidmax - $limit; $self->{uid_base} = $base < 0 ? 0 : $base; $self->{slice} = -1; } diff --git a/xt/pop3d-mpop.t b/xt/pop3d-mpop.t index fc82bc6b..9da1050c 100644 --- a/xt/pop3d-mpop.t +++ b/xt/pop3d-mpop.t @@ -53,7 +53,7 @@ delivery maildir $tmpdir/md account default host ${\$sock->sockhost} port ${\$sock->sockport} -user $uuid\@$newsgroup +user $uuid\@$newsgroup?limit=10000 auth user password anonymous received_header off @@ -65,7 +65,7 @@ EOM my $pid = spawn($cmd, undef, { 1 => 2 }); $pids{$pid} = $cmd; } - +diag "mpop is writing to $tmpdir/md ..."; while (scalar keys %pids) { my $pid = waitpid(-1, 0) or next; my $cmd = delete $pids{$pid} or next;