user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 20/29] t/v2mirror: switch to default run_mode for speedup
Date: Fri, 15 Nov 2019 09:50:51 +0000	[thread overview]
Message-ID: <20191115095100.25633-21-e@80x24.org> (raw)
In-Reply-To: <20191115095100.25633-1-e@80x24.org>

We need to be careful and explicitly close FDs before doing
-index, since we can't rely on FD_CLOEXEC without execve(2)
syscalls.
---
 t/v2mirror.t | 27 +++++++++++++++++++--------
 1 file changed, 19 insertions(+), 8 deletions(-)

diff --git a/t/v2mirror.t b/t/v2mirror.t
index 3c238093..2c7f6a84 100644
--- a/t/v2mirror.t
+++ b/t/v2mirror.t
@@ -58,9 +58,13 @@ for my $i (1..9) {
 my $epoch_max = $v2w->{epoch_max};
 ok($epoch_max > 0, "multiple epochs");
 $v2w->done;
+$ibx->cleanup;
 
 my ($sock, $pid);
-END { kill 'TERM', $pid if defined $pid };
+
+# TODO: replace this with ->DESTROY:
+my $owner_pid = $$;
+END { kill('TERM', $pid) if defined($pid) && $owner_pid == $$ };
 
 $! = 0;
 $sock = tcp_server();
@@ -85,8 +89,9 @@ foreach my $i (0..$epoch_max) {
 
 @cmd = ("-init", '-V2', 'm', "$tmpdir/m", 'http://example.com/m',
 	'alt@example.com');
-ok(run_script(\@cmd, undef, {run_mode => 0}), 'initialized public-inbox -V2');
-ok(run_script(['-index', "$tmpdir/m"], undef, { run_mode => 0}), 'indexed');
+ok(run_script(\@cmd), 'initialized public-inbox -V2');
+
+ok(run_script(['-index', "$tmpdir/m"]), 'indexed');
 
 my $mibx = { inboxdir => "$tmpdir/m", address => 'alt@example.com' };
 $mibx = PublicInbox::Inbox->new($mibx);
@@ -98,7 +103,8 @@ for my $i (10..15) {
 	$mime->header_set('Subject', "subject = $i");
 	ok($v2w->add($mime), "add msg $i OK");
 }
-$v2w->barrier;
+$v2w->done;
+$ibx->cleanup;
 
 sub fetch_each_epoch {
 	foreach my $i (0..$epoch_max) {
@@ -112,7 +118,7 @@ fetch_each_epoch();
 
 my $mset = $mibx->search->reopen->query('m:15@example.com', {mset => 1});
 is(scalar($mset->items), 0, 'new message not found in mirror, yet');
-ok(run_script(["-index", "$tmpdir/m"], undef, {run_mode=>0}), 'index updated');
+ok(run_script(["-index", "$tmpdir/m"]), 'index updated');
 is_deeply([$mibx->mm->minmax], [$ibx->mm->minmax], 'index synched minmax');
 $mset = $mibx->search->reopen->query('m:15@example.com', {mset => 1});
 is(scalar($mset->items), 1, 'found message in mirror');
@@ -130,7 +136,7 @@ $mime->header_set('Subject', 'subject = 10');
 	is_deeply(\@subj, ["# subject = 10"], "only rewrote one");
 }
 
-$v2w->barrier;
+$v2w->done;
 
 my $msgs = $mibx->search->{over_ro}->get_thread('10@example.com');
 my $to_purge = $msgs->[0]->{blob};
@@ -140,9 +146,12 @@ is(scalar($mset->items), 0, 'purged message gone from origin');
 
 fetch_each_epoch();
 {
+	$ibx->cleanup;
+	PublicInbox::InboxWritable::cleanup($mibx);
+	$v2w->done;
 	my $cmd = [ '-index', '--prune', "$tmpdir/m" ];
 	my ($out, $err) = ('', '');
-	my $opt = { 1 => \$out, 2 => \$err, run_mode => 0 };
+	my $opt = { 1 => \$out, 2 => \$err };
 	ok(run_script($cmd, undef, $opt), '-index --prune');
 	like($err, qr/discontiguous range/, 'warned about discontiguous range');
 	unlike($err, qr/fatal/, 'no scary fatal error shown');
@@ -172,11 +181,13 @@ is($mibx->git->check($to_purge), undef, 'unindex+prune successful in mirror');
 	$mime->header_set('Subject', 'subject = 1');
 	ok($v2w->remove($mime), 'removed <1@example.com> from source');
 	$v2w->done;
+	$ibx->cleanup;
 	fetch_each_epoch();
+	PublicInbox::InboxWritable::cleanup($mibx);
 
 	my $cmd = [ "-index", "$tmpdir/m" ];
 	my ($out, $err) = ('', '');
-	my $opt = { 1 => \$out, 2 => \$err, run_mode => 0 };
+	my $opt = { 1 => \$out, 2 => \$err };
 	ok(run_script($cmd, undef, $opt), 'index ran');
 	is($err, '', 'no errors reported by index');
 	$mset = $mibx->search->reopen->query('m:1@example.com', {mset => 1});

  parent reply	other threads:[~2019-11-15  9:51 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-15  9:50 [PATCH 00/29] speed up tests by preloading Eric Wong
2019-11-15  9:50 ` [PATCH 01/29] edit: pass global variables into subs Eric Wong
2019-11-15  9:50 ` [PATCH 02/29] edit: use OO API of File::Temp to shorten lifetime Eric Wong
2019-11-15  9:50 ` [PATCH 03/29] admin: get rid of singleton $CFG var Eric Wong
2019-11-15  9:50 ` [PATCH 04/29] index: pass global variables into subs Eric Wong
2019-11-15  9:50 ` [PATCH 05/29] init: " Eric Wong
2019-11-15  9:50 ` [PATCH 06/29] mda: " Eric Wong
2019-11-15  9:50 ` [PATCH 07/29] learn: " Eric Wong
2019-11-15  9:50 ` [PATCH 08/29] inboxwritable: add ->cleanup method Eric Wong
2019-11-15  9:50 ` [PATCH 09/29] import: only pass Inbox object to SearchIdx->new Eric Wong
2019-11-15  9:50 ` [PATCH 10/29] xapcmd: do not fire END and DESTROY handlers in child Eric Wong
2019-11-15  9:50 ` [PATCH 11/29] spawn: which: allow embedded slash for relative path Eric Wong
2019-11-15  9:50 ` [PATCH 12/29] t/common: introduce run_script wrapper for t/cgi.t Eric Wong
2019-11-15  9:50 ` [PATCH 13/29] t/edit: switch to use run_script Eric Wong
2019-11-15  9:50 ` [PATCH 14/29] t/init: convert to using run_script Eric Wong
2019-11-15  9:50 ` [PATCH 15/29] t/purge: convert to run_script Eric Wong
2019-11-15  9:50 ` [PATCH 16/29] t/v2mirror: get rid of IPC::Run dependency Eric Wong
2019-11-15  9:50 ` [PATCH 17/29] t/mda: switch to run_script for testing Eric Wong
2019-11-15  9:50 ` [PATCH 18/29] t/mda_filter_rubylang: drop IPC::Run dependency Eric Wong
2019-11-15  9:50 ` [PATCH 19/29] doc: remove IPC::Run as a dev and test dependency Eric Wong
2019-11-15  9:50 ` Eric Wong [this message]
2019-11-15  9:50 ` [PATCH 21/29] t/convert-compact: convert to run_script Eric Wong
2019-11-15  9:50 ` [PATCH 22/29] t/httpd: use run_script for -init Eric Wong
2019-11-15  9:50 ` [PATCH 23/29] t/watch_maildir_v2: " Eric Wong
2019-11-15  9:50 ` [PATCH 24/29] t/nntpd: " Eric Wong
2019-11-15  9:50 ` [PATCH 25/29] t/watch_filter_rubylang: run_script for -init and -index Eric Wong
2019-11-15  9:50 ` [PATCH 26/29] t/v2mda: switch to run_script in many places Eric Wong
2019-11-15  9:50 ` [PATCH 27/29] t/indexlevels-mirror*: switch to run_script Eric Wong
2019-11-15  9:50 ` [PATCH 28/29] t/xcpdb-reshard: use run_script for -xcpdb Eric Wong
2019-11-15  9:51 ` [PATCH 29/29] t/common: start_script replaces spawn_listener Eric Wong
2019-11-16  6:52   ` Eric Wong
2019-11-16 11:43     ` Eric Wong
2019-11-24  0:22       ` [PATCH 00/17] test fixes and cleanups Eric Wong
2019-11-24  0:22         ` [PATCH 01/17] tests: disable daemon workers in a few more places Eric Wong
2019-11-24  0:22         ` [PATCH 02/17] tests: use strict everywhere Eric Wong
2019-11-24  0:22         ` [PATCH 03/17] t/v1-add-remove-add: quiet down "git init" Eric Wong
2019-11-24  0:22         ` [PATCH 04/17] t/xcpdb-reshard: test xcpdb --compact Eric Wong
2019-11-24  0:22         ` [PATCH 05/17] t/httpd-corner: wait for worker process death Eric Wong
2019-11-24  0:22         ` [PATCH 06/17] t/nntpd-tls: sometimes SSL_connect succeeds quickly Eric Wong
2019-11-24  0:22         ` [PATCH 07/17] .gitignore: ignore local prove(1) files Eric Wong
2019-11-24  0:22         ` [PATCH 08/17] daemon: use sigprocmask to block signals at startup Eric Wong
2019-11-24  0:22         ` [PATCH 09/17] daemon: use sigprocmask when respawning workers Eric Wong
2019-11-24  0:22         ` [PATCH 10/17] daemon: avoid race when quitting workers Eric Wong
2019-11-25  8:59           ` Eric Wong
2019-11-27  1:33             ` [PATCH 0/2] fix kqueue support and missed signal wakeups Eric Wong
2019-11-27  1:33               ` [PATCH 1/2] dskqxs: fix missing EV_DISPATCH define Eric Wong
2019-11-27  1:33               ` [PATCH 2/2] httpd|nntpd: avoid missed signal wakeups Eric Wong
2019-11-24  0:22         ` [PATCH 11/17] t/common: start_script replaces spawn_listener Eric Wong
2019-11-24  0:22         ` [PATCH 12/17] t/nntpd-validate: get rid of threads dependency Eric Wong
2019-11-24  0:22         ` [PATCH 13/17] xapcmd: replace Xtmpdirs with File::Temp->newdir Eric Wong
2019-11-24  0:22         ` [PATCH 14/17] tests: use File::Temp->newdir instead of tempdir() Eric Wong
2019-11-24  0:22         ` [PATCH 15/17] tests: quiet down commit graph Eric Wong
2019-11-24  0:22         ` [PATCH 16/17] t/perf-*.t: use $ENV{GIANT_INBOX_DIR} consistently Eric Wong
2019-11-24  0:22         ` [PATCH 17/17] tests: move giant inbox/git dependent tests to xt/ Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191115095100.25633-21-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).