From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 227101FAF5 for ; Wed, 28 Feb 2018 23:42:09 +0000 (UTC) From: "Eric Wong (Contractor, The Linux Foundation)" To: meta@public-inbox.org Subject: [PATCH 18/21] searchidxskeleton: extra error checking Date: Wed, 28 Feb 2018 23:41:59 +0000 Message-Id: <20180228234202.8839-19-e@80x24.org> In-Reply-To: <20180228234202.8839-1-e@80x24.org> References: <20180228234202.8839-1-e@80x24.org> List-Id: I added these while chasing down the DatabaseCorruptError exceptions which turned out to be caused by Xapian DB modifications during iteration. --- lib/PublicInbox/SearchIdxSkeleton.pm | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/lib/PublicInbox/SearchIdxSkeleton.pm b/lib/PublicInbox/SearchIdxSkeleton.pm index 0016f89..aa2713f 100644 --- a/lib/PublicInbox/SearchIdxSkeleton.pm +++ b/lib/PublicInbox/SearchIdxSkeleton.pm @@ -42,7 +42,6 @@ sub new { sub skeleton_worker_loop { my ($self, $r) = @_; $0 = 'pi-v2-skeleton'; - my $msg; my $xdb = $self->_xdb_acquire; $xdb->begin_transaction; my $txn = 1; @@ -54,9 +53,12 @@ sub skeleton_worker_loop { $self->_xdb_release; $xdb = $txn = undef; } else { - read($r, $msg, $line) or die "read failed: $!\n"; + my $len = int($line); + my $n = read($r, my $msg, $len) or die "read: $!\n"; + $n == $len or die "short read: $n != $len\n"; $msg = thaw($msg); # should raise on error defined $msg or die "failed to thaw buffer\n"; + $xdb ||= $self->_xdb_acquire; if (!$txn) { $xdb->begin_transaction; $txn = 1; @@ -65,6 +67,8 @@ sub skeleton_worker_loop { warn "failed to index message <$msg->[-1]>: $@\n" if $@; } } + die "xdb not released\n" if $xdb; + die "in transaction\n" if $txn; } # called by a partition worker -- EW