* [PATCH] lei_search: xsmsg_vmd: retry_reopen properly
@ 2021-09-06 7:20 5% Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2021-09-06 7:20 UTC (permalink / raw)
To: meta
The deeper eval was preventing retry_reopen from retrying
with readers and writers working in parallel:
FOO=imaps://example.com/INBOX.huge
lei lcat $FOO -f mboxcl | lei tag -F mboxcl +L:bar -
Fixes: c7bcfe6cd6648ff0 ("lei: diagnostics for /Document \d+ not found/ errors")
---
lib/PublicInbox/LeiSearch.pm | 21 +++++++++------------
1 file changed, 9 insertions(+), 12 deletions(-)
diff --git a/lib/PublicInbox/LeiSearch.pm b/lib/PublicInbox/LeiSearch.pm
index 47160ed9..a10e6e17 100644
--- a/lib/PublicInbox/LeiSearch.pm
+++ b/lib/PublicInbox/LeiSearch.pm
@@ -55,17 +55,13 @@ sub _xsmsg_vmd { # retry_reopen
$kw{flagged} = 1 if delete($smsg->{lei_q_tt_flagged});
my @num = $self->over->blob_exists($smsg->{blob});
for my $num (@num) { # there should only be one...
- eval {
- $doc = $xdb->get_document(num2docid($self, $num));
- $x = xap_terms('K', $doc);
- %kw = (%kw, %$x);
- if ($want_label) { # JSON/JMAP only
- $x = xap_terms('L', $doc);
- %L = (%L, %$x);
- }
- };
- warn "$$ $0 #$num (nshard=$self->{nshard}) $smsg->{blob}: $@"
- if $@;
+ $doc = $xdb->get_document(num2docid($self, $num));
+ $x = xap_terms('K', $doc);
+ %kw = (%kw, %$x);
+ if ($want_label) { # JSON/JMAP only
+ $x = xap_terms('L', $doc);
+ %L = (%L, %$x);
+ }
}
$smsg->{kw} = [ sort keys %kw ] if scalar(keys(%kw));
$smsg->{L} = [ sort keys %L ] if scalar(keys(%L));
@@ -75,7 +71,8 @@ sub _xsmsg_vmd { # retry_reopen
sub xsmsg_vmd {
my ($self, $smsg, $want_label) = @_;
return if $smsg->{kw}; # already set by LeiXSearch->mitem_kw
- $self->retry_reopen(\&_xsmsg_vmd, $smsg, $want_label);
+ eval { $self->retry_reopen(\&_xsmsg_vmd, $smsg, $want_label) };
+ warn "$$ $0 (nshard=$self->{nshard}) $smsg->{blob}: $@" if $@;
}
# when a message has no Message-IDs at all, this is needed for
^ permalink raw reply related [relevance 5%]
* [PATCH] lei_search: avoid unconditional warning when no exception
@ 2021-08-16 5:39 5% Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2021-08-16 5:39 UTC (permalink / raw)
To: meta
Oops, we shouldn't warn on "$@" unless "$@" is truthy.
Fixes: c7bcfe6cd6648ff0 ("lei: diagnostics for /Document \d+ not found/ errors")
---
lib/PublicInbox/LeiSearch.pm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lib/PublicInbox/LeiSearch.pm b/lib/PublicInbox/LeiSearch.pm
index f9e5c8e9..47160ed9 100644
--- a/lib/PublicInbox/LeiSearch.pm
+++ b/lib/PublicInbox/LeiSearch.pm
@@ -64,7 +64,8 @@ sub _xsmsg_vmd { # retry_reopen
%L = (%L, %$x);
}
};
- warn "$$ $0 #$num (nshard=$self->{nshard}) $smsg->{blob}: $@";
+ warn "$$ $0 #$num (nshard=$self->{nshard}) $smsg->{blob}: $@"
+ if $@;
}
$smsg->{kw} = [ sort keys %kw ] if scalar(keys(%kw));
$smsg->{L} = [ sort keys %L ] if scalar(keys(%L));
^ permalink raw reply related [relevance 5%]
* [PATCH 1/3] lei: diagnostics for /Document \d+ not found/ errors
2021-08-14 0:29 5% [PATCH 0/3] lei: hopefully kill /Document \d+ not found/ errors Eric Wong
@ 2021-08-14 0:29 7% ` Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2021-08-14 0:29 UTC (permalink / raw)
To: meta
This may help diagnose "Exception: Document \d+ not found"
errors I'm seeing from "lei up" with HTTPS endpoints.
---
lib/PublicInbox/IPC.pm | 2 +-
lib/PublicInbox/LeiSearch.pm | 17 ++++++++++-------
2 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/lib/PublicInbox/IPC.pm b/lib/PublicInbox/IPC.pm
index 497a6035..d909dc1c 100644
--- a/lib/PublicInbox/IPC.pm
+++ b/lib/PublicInbox/IPC.pm
@@ -236,7 +236,7 @@ sub recv_and_run {
undef $buf;
my $sub = shift @$args;
eval { $self->$sub(@$args) };
- warn "$$ wq_worker: $@" if $@;
+ warn "$$ $0 wq_worker: $@" if $@;
delete @$self{0..($nfd-1)};
$n;
}
diff --git a/lib/PublicInbox/LeiSearch.pm b/lib/PublicInbox/LeiSearch.pm
index 79b2fd7d..f9e5c8e9 100644
--- a/lib/PublicInbox/LeiSearch.pm
+++ b/lib/PublicInbox/LeiSearch.pm
@@ -55,13 +55,16 @@ sub _xsmsg_vmd { # retry_reopen
$kw{flagged} = 1 if delete($smsg->{lei_q_tt_flagged});
my @num = $self->over->blob_exists($smsg->{blob});
for my $num (@num) { # there should only be one...
- $doc = $xdb->get_document(num2docid($self, $num));
- $x = xap_terms('K', $doc);
- %kw = (%kw, %$x);
- if ($want_label) { # JSON/JMAP only
- $x = xap_terms('L', $doc);
- %L = (%L, %$x);
- }
+ eval {
+ $doc = $xdb->get_document(num2docid($self, $num));
+ $x = xap_terms('K', $doc);
+ %kw = (%kw, %$x);
+ if ($want_label) { # JSON/JMAP only
+ $x = xap_terms('L', $doc);
+ %L = (%L, %$x);
+ }
+ };
+ warn "$$ $0 #$num (nshard=$self->{nshard}) $smsg->{blob}: $@";
}
$smsg->{kw} = [ sort keys %kw ] if scalar(keys(%kw));
$smsg->{L} = [ sort keys %L ] if scalar(keys(%L));
^ permalink raw reply related [relevance 7%]
* [PATCH 0/3] lei: hopefully kill /Document \d+ not found/ errors
@ 2021-08-14 0:29 5% Eric Wong
2021-08-14 0:29 7% ` [PATCH 1/3] lei: diagnostics for " Eric Wong
0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2021-08-14 0:29 UTC (permalink / raw)
To: meta
2/3 is probably a fix for a long-standing problem, 3/3 was
noticed while working on it. If 2/3 doesn't fix it, then maybe
1/3 will help us narrow it down.
Eric Wong (3):
lei: diagnostics for /Document \d+ not found/ errors
lei <q|up>: wait on remote mboxrd imports synchronously
lei: hexdigest mocks account for unwanted headers
lib/PublicInbox/FakeImport.pm | 3 +++
lib/PublicInbox/IPC.pm | 2 +-
lib/PublicInbox/LEI.pm | 5 +++++
lib/PublicInbox/LeiQuery.pm | 2 +-
lib/PublicInbox/LeiRemote.pm | 9 +++++----
lib/PublicInbox/LeiSearch.pm | 17 ++++++++++-------
lib/PublicInbox/LeiStore.pm | 10 +++++++++-
lib/PublicInbox/LeiXSearch.pm | 11 +++++++----
8 files changed, 41 insertions(+), 18 deletions(-)
^ permalink raw reply [relevance 5%]
Results 1-4 of 4 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-08-14 0:29 5% [PATCH 0/3] lei: hopefully kill /Document \d+ not found/ errors Eric Wong
2021-08-14 0:29 7% ` [PATCH 1/3] lei: diagnostics for " Eric Wong
2021-08-16 5:39 5% [PATCH] lei_search: avoid unconditional warning when no exception Eric Wong
2021-09-06 7:20 5% [PATCH] lei_search: xsmsg_vmd: retry_reopen properly Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).