user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: Ricardo Ribalda <ribalda@chromium.org>
Cc: meta@public-inbox.org
Subject: [PATCH] lei: add diagnostics for IMAP writer failures
Date: Fri, 9 Sep 2022 17:44:10 +0000	[thread overview]
Message-ID: <20220909174410.M560915@dcvr> (raw)
In-Reply-To: <CANiDSCsDfutAUMBLPZbxdyka+_jnhv+4YNYdL9QPRoC=wNUGCQ@mail.gmail.com>

Ricardo Ribalda <ribalda@chromium.org> wrote:
> Hi
> 
> I am getting a lot of those messages when using lei in imap mode.
> 
> Nonetheless the mail seems to arrive fine to its destination (on the
> first run, some of the mail was lost, but for small batches of mails
> it seems to work fine).
> 
> It is a valid message or just some red-herring.
> 
> I am using debian testing/
> 
> Regards!
> 
> # https://lore.kernel.org/all/ limiting to 2022-09-07 10:03 +0000 and newer
> # /usr/local/google/home/ribalda/.local/share/lei/store 13/13
> # /usr/bin/curl -Sf -s -d ''
> https://lore.kernel.org/all/?x=m&t=1&q=((ribalda)+AND+rt%3A1660039651..)+AND+dt%3A20220907100321..
> 1285740 lei2mail 6 wq_worker: do_post_auth: Can't call method
> "uidvalidity" on an undefined value at
> /usr/share/perl5/PublicInbox/LeiToMail.pm line 313.

<snip>

> 1285735 lei2mail 1 wq_worker: write_mail: Can't use an undefined value
> as a subroutine reference at /usr/share/perl5/PublicInbox/LeiToMail.pm
> line 783.
> # https://lore.kernel.org/all/ 18/18
> # 14 written to imaps://imap.gmail.com/lei/me (31 matches)

I wonder if it's excessive parallelism for gmail's IMAP.
I haven't tested IMAP destinations, much...

Can you try the patch at the bottom?

There's also another patch coming to document the `--jobs|-j' CLI
switch for `lei up' and `lei q', but trying `-j ,1' may help you
if it's parallelism.  Note the comma before `1', it accepts
`-j $Q,$W' since $Q is the number of query processes and $W is
the number of LeiToMail writers.

-------8<-------
From: Eric Wong <e@80x24.org>
Subject: [PATCH] lei: add diagnostics for IMAP writer failures

This may help diagnose the problem with IMAP destinations
encountered at:
https://public-inbox.org/meta/CANiDSCsDfutAUMBLPZbxdyka+_jnhv+4YNYdL9QPRoC=wNUGCQ@mail.gmail.com/
---
 lib/PublicInbox/LeiToMail.pm | 4 ++--
 lib/PublicInbox/NetReader.pm | 8 +++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm
index 2aa3977e..bc00b96a 100644
--- a/lib/PublicInbox/LeiToMail.pm
+++ b/lib/PublicInbox/LeiToMail.pm
@@ -310,8 +310,8 @@ sub _imap_write_cb ($$) {
 	my $dedupe = $lei->{dedupe};
 	$dedupe->prepare_dedupe if $dedupe;
 	my $append = $lei->{net}->can('imap_append');
-	my $uri = $self->{uri};
-	my $mic = $lei->{net}->mic_get($uri);
+	my $uri = $self->{uri} // die 'BUG: no {uri}';
+	my $mic = $lei->{net}->mic_get($uri) // die 'BUG: no $mic';
 	my $folder = $uri->mailbox;
 	$uri->uidvalidity($mic->uidvalidity($folder));
 	my $lse = $lei->{lse}; # may be undef
diff --git a/lib/PublicInbox/NetReader.pm b/lib/PublicInbox/NetReader.pm
index c1af03a3..4de2583e 100644
--- a/lib/PublicInbox/NetReader.pm
+++ b/lib/PublicInbox/NetReader.pm
@@ -685,7 +685,13 @@ sub mic_get {
 	}
 	my $mic = mic_new($self, $mic_arg, $sec, $uri);
 	$cached //= {}; # invalid placeholder if no cache enabled
-	$mic && $mic->IsConnected ? ($cached->{$sec} = $mic) : undef;
+	if ($mic && $mic->IsConnected) {
+		$cached->{$sec} = $mic;
+	} else {
+		warn 'IMAP LastError: ',$mic->LastError, "\n" if $mic;
+		warn "IMAP errno: $!\n" if $!;
+		undef;
+	}
 }
 
 sub imap_each {

  reply	other threads:[~2022-09-09 17:45 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-09-09 10:09 imap: "Can't use an undefined value as a subroutine reference" Ricardo Ribalda
2022-09-09 17:44 ` Eric Wong [this message]
2022-09-09 18:00   ` [PATCH] doc: document --jobs for `lei q' and `lei up' Eric Wong
2022-09-09 20:35   ` [PATCH] lei: add diagnostics for IMAP writer failures Ricardo Ribalda
2022-09-10  1:18     ` [PATCH v2] lei: bail out earlier on " Eric Wong
2022-09-10 19:34       ` Ricardo Ribalda
2022-09-10 19:50         ` Eric Wong
2022-09-10 19:53           ` Ricardo Ribalda
2022-09-10 20:19             ` Eric Wong
2022-11-14  8:07               ` [PATCH] lei q|up: limit default write --jobs for IMAP(S) Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220909174410.M560915@dcvr \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    --cc=ribalda@chromium.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).