about summary refs log tree commit homepage
path: root/lib/PublicInbox/LeiXSearch.pm
diff options
context:
space:
mode:
authorEric Wong <e@80x24.org>2021-08-24 13:06:39 +0000
committerEric Wong <e@80x24.org>2021-08-24 20:11:39 +0000
commit5a27f82749d723625848c8d1cf216a0f4de73597 (patch)
tree65ad9470b8b149b860d8a7863b8c14d3067829b1 /lib/PublicInbox/LeiXSearch.pm
parent9d6b13687e0e4403523bff301adb4aae027dfdb1 (diff)
downloadpublic-inbox-5a27f82749d723625848c8d1cf216a0f4de73597.tar.gz
lei: non-blocking lei/store->done in lei-daemon
This allows client sockets to wait for "done" commits to
lei/store while the daemon reacts asynchronously.  The goal
of this change is to keep the script/lei client alive until
lei/store commits changes to the filesystem, but without
blocking the lei-daemon event loop.  It depends on Perl
refcounting to close the socket.

This change also highlighted our over-use of "done" requests to
lei/store processes, which is now corrected so we only issue it
on collective socket EOF rather than upon reaping every single
worker.

This also fixes "lei forget-mail-sync" when it is the initial
command.

This took several iterations and much debugging to arrive at the
current implementation:

1. The initial iteration of this change utilized socket passing
   from lei-daemon to lei/store, which necessitated switching
   from faster pipes to slower Unix sockets.

2. The second iteration switched to registering notification sockets
   independently of "done" requests, but that could lead to early
   wakeups when "done" was requested by other workers.  This
   appeared to work most of the time, but suffered races under
   high load which were difficult to track down.

Finally, this iteration passes the stringified socket GLOB ref
to lei/store which is echoed back to lei-daemon upon completion
of that particular "done" request.
Diffstat (limited to 'lib/PublicInbox/LeiXSearch.pm')
-rw-r--r--lib/PublicInbox/LeiXSearch.pm4
1 files changed, 2 insertions, 2 deletions
diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index 5e34d864..1f83e582 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -374,8 +374,8 @@ sub query_done { # EOF callback for main daemon
         if ($lei->{opt}->{'mail-sync'} && !$lei->{sto}) {
                 warn "BUG: {sto} missing with --mail-sync";
         }
-        my $wait = $lei->{sto} ? $lei->{sto}->ipc_do('done') : undef;
-        $wait = $lei->{v2w} ? $lei->{v2w}->ipc_do('done') : undef;
+        $lei->sto_done_request if $lei->{sto};
+        my $wait = $lei->{v2w} ? $lei->{v2w}->ipc_do('done') : undef;
         $lei->{ovv}->ovv_end($lei);
         my $start_mua;
         if ($l2m) { # close() calls LeiToMail reap_compress