From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 20/29] t/v2mirror: switch to default run_mode for speedup
Date: Fri, 15 Nov 2019 09:50:51 +0000 [thread overview]
Message-ID: <20191115095100.25633-21-e@80x24.org> (raw)
In-Reply-To: <20191115095100.25633-1-e@80x24.org>
We need to be careful and explicitly close FDs before doing
-index, since we can't rely on FD_CLOEXEC without execve(2)
syscalls.
---
t/v2mirror.t | 27 +++++++++++++++++++--------
1 file changed, 19 insertions(+), 8 deletions(-)
diff --git a/t/v2mirror.t b/t/v2mirror.t
index 3c238093..2c7f6a84 100644
--- a/t/v2mirror.t
+++ b/t/v2mirror.t
@@ -58,9 +58,13 @@ for my $i (1..9) {
my $epoch_max = $v2w->{epoch_max};
ok($epoch_max > 0, "multiple epochs");
$v2w->done;
+$ibx->cleanup;
my ($sock, $pid);
-END { kill 'TERM', $pid if defined $pid };
+
+# TODO: replace this with ->DESTROY:
+my $owner_pid = $$;
+END { kill('TERM', $pid) if defined($pid) && $owner_pid == $$ };
$! = 0;
$sock = tcp_server();
@@ -85,8 +89,9 @@ foreach my $i (0..$epoch_max) {
@cmd = ("-init", '-V2', 'm', "$tmpdir/m", 'http://example.com/m',
'alt@example.com');
-ok(run_script(\@cmd, undef, {run_mode => 0}), 'initialized public-inbox -V2');
-ok(run_script(['-index', "$tmpdir/m"], undef, { run_mode => 0}), 'indexed');
+ok(run_script(\@cmd), 'initialized public-inbox -V2');
+
+ok(run_script(['-index', "$tmpdir/m"]), 'indexed');
my $mibx = { inboxdir => "$tmpdir/m", address => 'alt@example.com' };
$mibx = PublicInbox::Inbox->new($mibx);
@@ -98,7 +103,8 @@ for my $i (10..15) {
$mime->header_set('Subject', "subject = $i");
ok($v2w->add($mime), "add msg $i OK");
}
-$v2w->barrier;
+$v2w->done;
+$ibx->cleanup;
sub fetch_each_epoch {
foreach my $i (0..$epoch_max) {
@@ -112,7 +118,7 @@ fetch_each_epoch();
my $mset = $mibx->search->reopen->query('m:15@example.com', {mset => 1});
is(scalar($mset->items), 0, 'new message not found in mirror, yet');
-ok(run_script(["-index", "$tmpdir/m"], undef, {run_mode=>0}), 'index updated');
+ok(run_script(["-index", "$tmpdir/m"]), 'index updated');
is_deeply([$mibx->mm->minmax], [$ibx->mm->minmax], 'index synched minmax');
$mset = $mibx->search->reopen->query('m:15@example.com', {mset => 1});
is(scalar($mset->items), 1, 'found message in mirror');
@@ -130,7 +136,7 @@ $mime->header_set('Subject', 'subject = 10');
is_deeply(\@subj, ["# subject = 10"], "only rewrote one");
}
-$v2w->barrier;
+$v2w->done;
my $msgs = $mibx->search->{over_ro}->get_thread('10@example.com');
my $to_purge = $msgs->[0]->{blob};
@@ -140,9 +146,12 @@ is(scalar($mset->items), 0, 'purged message gone from origin');
fetch_each_epoch();
{
+ $ibx->cleanup;
+ PublicInbox::InboxWritable::cleanup($mibx);
+ $v2w->done;
my $cmd = [ '-index', '--prune', "$tmpdir/m" ];
my ($out, $err) = ('', '');
- my $opt = { 1 => \$out, 2 => \$err, run_mode => 0 };
+ my $opt = { 1 => \$out, 2 => \$err };
ok(run_script($cmd, undef, $opt), '-index --prune');
like($err, qr/discontiguous range/, 'warned about discontiguous range');
unlike($err, qr/fatal/, 'no scary fatal error shown');
@@ -172,11 +181,13 @@ is($mibx->git->check($to_purge), undef, 'unindex+prune successful in mirror');
$mime->header_set('Subject', 'subject = 1');
ok($v2w->remove($mime), 'removed <1@example.com> from source');
$v2w->done;
+ $ibx->cleanup;
fetch_each_epoch();
+ PublicInbox::InboxWritable::cleanup($mibx);
my $cmd = [ "-index", "$tmpdir/m" ];
my ($out, $err) = ('', '');
- my $opt = { 1 => \$out, 2 => \$err, run_mode => 0 };
+ my $opt = { 1 => \$out, 2 => \$err };
ok(run_script($cmd, undef, $opt), 'index ran');
is($err, '', 'no errors reported by index');
$mset = $mibx->search->reopen->query('m:1@example.com', {mset => 1});
next prev parent reply other threads:[~2019-11-15 9:51 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-15 9:50 [PATCH 00/29] speed up tests by preloading Eric Wong
2019-11-15 9:50 ` [PATCH 01/29] edit: pass global variables into subs Eric Wong
2019-11-15 9:50 ` [PATCH 02/29] edit: use OO API of File::Temp to shorten lifetime Eric Wong
2019-11-15 9:50 ` [PATCH 03/29] admin: get rid of singleton $CFG var Eric Wong
2019-11-15 9:50 ` [PATCH 04/29] index: pass global variables into subs Eric Wong
2019-11-15 9:50 ` [PATCH 05/29] init: " Eric Wong
2019-11-15 9:50 ` [PATCH 06/29] mda: " Eric Wong
2019-11-15 9:50 ` [PATCH 07/29] learn: " Eric Wong
2019-11-15 9:50 ` [PATCH 08/29] inboxwritable: add ->cleanup method Eric Wong
2019-11-15 9:50 ` [PATCH 09/29] import: only pass Inbox object to SearchIdx->new Eric Wong
2019-11-15 9:50 ` [PATCH 10/29] xapcmd: do not fire END and DESTROY handlers in child Eric Wong
2019-11-15 9:50 ` [PATCH 11/29] spawn: which: allow embedded slash for relative path Eric Wong
2019-11-15 9:50 ` [PATCH 12/29] t/common: introduce run_script wrapper for t/cgi.t Eric Wong
2019-11-15 9:50 ` [PATCH 13/29] t/edit: switch to use run_script Eric Wong
2019-11-15 9:50 ` [PATCH 14/29] t/init: convert to using run_script Eric Wong
2019-11-15 9:50 ` [PATCH 15/29] t/purge: convert to run_script Eric Wong
2019-11-15 9:50 ` [PATCH 16/29] t/v2mirror: get rid of IPC::Run dependency Eric Wong
2019-11-15 9:50 ` [PATCH 17/29] t/mda: switch to run_script for testing Eric Wong
2019-11-15 9:50 ` [PATCH 18/29] t/mda_filter_rubylang: drop IPC::Run dependency Eric Wong
2019-11-15 9:50 ` [PATCH 19/29] doc: remove IPC::Run as a dev and test dependency Eric Wong
2019-11-15 9:50 ` Eric Wong [this message]
2019-11-15 9:50 ` [PATCH 21/29] t/convert-compact: convert to run_script Eric Wong
2019-11-15 9:50 ` [PATCH 22/29] t/httpd: use run_script for -init Eric Wong
2019-11-15 9:50 ` [PATCH 23/29] t/watch_maildir_v2: " Eric Wong
2019-11-15 9:50 ` [PATCH 24/29] t/nntpd: " Eric Wong
2019-11-15 9:50 ` [PATCH 25/29] t/watch_filter_rubylang: run_script for -init and -index Eric Wong
2019-11-15 9:50 ` [PATCH 26/29] t/v2mda: switch to run_script in many places Eric Wong
2019-11-15 9:50 ` [PATCH 27/29] t/indexlevels-mirror*: switch to run_script Eric Wong
2019-11-15 9:50 ` [PATCH 28/29] t/xcpdb-reshard: use run_script for -xcpdb Eric Wong
2019-11-15 9:51 ` [PATCH 29/29] t/common: start_script replaces spawn_listener Eric Wong
2019-11-16 6:52 ` Eric Wong
2019-11-16 11:43 ` Eric Wong
2019-11-24 0:22 ` [PATCH 00/17] test fixes and cleanups Eric Wong
2019-11-24 0:22 ` [PATCH 01/17] tests: disable daemon workers in a few more places Eric Wong
2019-11-24 0:22 ` [PATCH 02/17] tests: use strict everywhere Eric Wong
2019-11-24 0:22 ` [PATCH 03/17] t/v1-add-remove-add: quiet down "git init" Eric Wong
2019-11-24 0:22 ` [PATCH 04/17] t/xcpdb-reshard: test xcpdb --compact Eric Wong
2019-11-24 0:22 ` [PATCH 05/17] t/httpd-corner: wait for worker process death Eric Wong
2019-11-24 0:22 ` [PATCH 06/17] t/nntpd-tls: sometimes SSL_connect succeeds quickly Eric Wong
2019-11-24 0:22 ` [PATCH 07/17] .gitignore: ignore local prove(1) files Eric Wong
2019-11-24 0:22 ` [PATCH 08/17] daemon: use sigprocmask to block signals at startup Eric Wong
2019-11-24 0:22 ` [PATCH 09/17] daemon: use sigprocmask when respawning workers Eric Wong
2019-11-24 0:22 ` [PATCH 10/17] daemon: avoid race when quitting workers Eric Wong
2019-11-25 8:59 ` Eric Wong
2019-11-27 1:33 ` [PATCH 0/2] fix kqueue support and missed signal wakeups Eric Wong
2019-11-27 1:33 ` [PATCH 1/2] dskqxs: fix missing EV_DISPATCH define Eric Wong
2019-11-27 1:33 ` [PATCH 2/2] httpd|nntpd: avoid missed signal wakeups Eric Wong
2019-11-24 0:22 ` [PATCH 11/17] t/common: start_script replaces spawn_listener Eric Wong
2019-11-24 0:22 ` [PATCH 12/17] t/nntpd-validate: get rid of threads dependency Eric Wong
2019-11-24 0:22 ` [PATCH 13/17] xapcmd: replace Xtmpdirs with File::Temp->newdir Eric Wong
2019-11-24 0:22 ` [PATCH 14/17] tests: use File::Temp->newdir instead of tempdir() Eric Wong
2019-11-24 0:22 ` [PATCH 15/17] tests: quiet down commit graph Eric Wong
2019-11-24 0:22 ` [PATCH 16/17] t/perf-*.t: use $ENV{GIANT_INBOX_DIR} consistently Eric Wong
2019-11-24 0:22 ` [PATCH 17/17] tests: move giant inbox/git dependent tests to xt/ Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191115095100.25633-21-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).