user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
* [PATCH] searchidx: do not release Xapian lock while (only) Msgmap is indexing
@ 2016-08-14 10:22 Eric Wong
  0 siblings, 0 replies; only message in thread
From: Eric Wong @ 2016-08-14 10:22 UTC (permalink / raw)
  To: meta

SQLite might index quickly, so we hold the lock used by Xapian
for the duration.  This probably needs to be reworked entirely,
actually.
---
 lib/PublicInbox/SearchIdx.pm | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
index 0eb07a1..f8155ec 100644
--- a/lib/PublicInbox/SearchIdx.pm
+++ b/lib/PublicInbox/SearchIdx.pm
@@ -436,19 +436,24 @@ sub _index_sync {
 
 	my $mm = _msgmap_init($self);
 	my $dbh = $mm->{dbh} if $mm;
+	my $mm_only;
 	my $cb = sub {
 		my ($commit, $more) = @_;
 		if ($dbh) {
 			$mm->last_commit($commit) if $commit;
 			$dbh->commit;
 		}
-		$xdb->set_metadata($mkey, $commit) if $mkey && $commit;
-		$xdb->commit_transaction;
-		$xdb = _xdb_release($self);
+		if (!$mm_only) {
+			$xdb->set_metadata($mkey, $commit) if $mkey && $commit;
+			$xdb->commit_transaction;
+			$xdb = _xdb_release($self);
+		}
 		# let another process do some work... <
 		if ($more) {
-			$xdb = _xdb_acquire($self);
-			$xdb->begin_transaction;
+			if (!$mm_only) {
+				$xdb = _xdb_acquire($self);
+				$xdb->begin_transaction;
+			}
 			$dbh->begin_work if $dbh;
 		}
 	};
@@ -472,14 +477,13 @@ sub _index_sync {
 			my $mkey_prev = $mkey;
 			$mkey = undef; # ignore xapian, for now
 			my $mlog = _git_log($self, $r);
+			$mm_only = 1;
 			rlog($self, $mlog, *index_mm, *unindex_mm, $cb);
-			$mlog = undef;
+			$mm_only = $mlog = undef;
 
 			# now deal with Xapian
 			$mkey = $mkey_prev;
 			$dbh = undef;
-			$xdb = _xdb_acquire($self);
-			$xdb->begin_transaction;
 			rlog($self, $xlog, *index_mm2, *unindex_mm2, $cb);
 		}
 	} else {
-- 
EW


^ permalink raw reply related	[flat|nested] only message in thread

only message in thread, other threads:[~2016-08-14 10:22 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-14 10:22 [PATCH] searchidx: do not release Xapian lock while (only) Msgmap is indexing Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).