user/dev discussion of public-inbox itself
 help / color / Atom feed
* [PATCH 0/4] test speedups for FreeBSD
@ 2020-04-19 23:19 Eric Wong
  2020-04-19 23:19 ` [PATCH 1/4] import: init_bare: allow use as method, use in tests Eric Wong
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Eric Wong @ 2020-04-19 23:19 UTC (permalink / raw)
  To: meta

There's a tiny improvement on Linux, but FreeBSD benefits
more, especially [PATCH 2/4]

Eric Wong (4):
  import: init_bare: allow use as method, use in tests
  import: init_bare: use pure Perl
  t/ds-leak: use BSD::Resource
  testcommon: spawn-aware system() and qx[] workalikes

 lib/PublicInbox/Import.pm     | 31 ++++++++++++++++++++++++-------
 lib/PublicInbox/TestCommon.pm | 26 ++++++++++++++++++++++++--
 lib/PublicInbox/V2Writable.pm |  5 ++---
 t/admin.t                     |  3 ++-
 t/altid.t                     |  2 +-
 t/cgi.t                       |  3 ++-
 t/config.t                    |  5 +++--
 t/convert-compact.t           |  3 +--
 t/ds-leak.t                   | 16 ++++++++++------
 t/edit.t                      |  2 +-
 t/feed.t                      |  2 +-
 t/git.t                       | 22 ++++++++++------------
 t/hl_mod.t                    | 20 +++++++-------------
 t/html_index.t                |  2 +-
 t/httpd-corner.t              | 30 ++++++++++++++----------------
 t/httpd.t                     |  8 ++++----
 t/import.t                    |  3 +--
 t/indexlevels-mirror.t        |  8 ++++----
 t/init.t                      |  5 +++--
 t/mda.t                       |  5 +++--
 t/mda_filter_rubylang.t       |  6 +++---
 t/multi-mid.t                 |  2 +-
 t/nntpd.t                     |  5 +++--
 t/nulsubject.t                |  5 +++--
 t/psgi_attach.t               |  2 +-
 t/psgi_mount.t                |  2 +-
 t/psgi_text.t                 |  2 +-
 t/replace.t                   |  2 +-
 t/search-thr-index.t          |  3 ++-
 t/search.t                    |  2 +-
 t/solver_git.t                |  8 +++++---
 t/v1-add-remove-add.t         |  2 +-
 t/v1reindex.t                 |  2 +-
 t/v2mda.t                     |  2 +-
 t/v2mirror.t                  |  4 ++--
 t/v2writable.t                |  2 +-
 t/watch_maildir.t             |  3 ++-
 t/watch_maildir_v2.t          |  3 ++-
 t/www_listing.t               | 24 ++++++++++++++----------
 xt/git-http-backend.t         |  6 +++---
 40 files changed, 167 insertions(+), 121 deletions(-)


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/4] import: init_bare: allow use as method, use in tests
  2020-04-19 23:19 [PATCH 0/4] test speedups for FreeBSD Eric Wong
@ 2020-04-19 23:19 ` Eric Wong
  2020-04-19 23:19 ` [PATCH 2/4] import: init_bare: use pure Perl Eric Wong
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-04-19 23:19 UTC (permalink / raw)
  To: meta

Allowing ->init_bare to be used as a method saves some
keystrokes, and we can save a little bit of time on systems with
our vfork(2)-enabled spawn().

This also sets us up for future improvements where we can
avoid spawning a process at all.
---
 lib/PublicInbox/Import.pm     | 3 ++-
 lib/PublicInbox/V2Writable.pm | 5 ++---
 t/admin.t                     | 3 ++-
 t/altid.t                     | 2 +-
 t/cgi.t                       | 3 ++-
 t/config.t                    | 3 ++-
 t/convert-compact.t           | 3 +--
 t/feed.t                      | 2 +-
 t/git.t                       | 5 +++--
 t/html_index.t                | 2 +-
 t/import.t                    | 3 +--
 t/mda.t                       | 3 ++-
 t/nulsubject.t                | 2 +-
 t/psgi_attach.t               | 2 +-
 t/psgi_mount.t                | 2 +-
 t/psgi_text.t                 | 2 +-
 t/search-thr-index.t          | 3 ++-
 t/solver_git.t                | 3 ++-
 t/v1-add-remove-add.t         | 2 +-
 t/v1reindex.t                 | 2 +-
 t/watch_maildir.t             | 3 ++-
 t/watch_maildir_v2.t          | 3 ++-
 t/www_listing.t               | 5 +++--
 23 files changed, 37 insertions(+), 29 deletions(-)

diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index c72c1e92..351bc660 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -441,7 +441,8 @@ sub run_die ($;$$) {
 }
 
 sub init_bare {
-	my ($dir) = @_;
+	my ($dir) = @_; # or self
+	$dir = $dir->{git}->{git_dir} if ref($dir);
 	my @cmd = (qw(git init --bare -q), $dir);
 	run_die(\@cmd);
 	# set a reasonable default:
diff --git a/lib/PublicInbox/V2Writable.pm b/lib/PublicInbox/V2Writable.pm
index 1c78ef24..ca5f4f1b 100644
--- a/lib/PublicInbox/V2Writable.pm
+++ b/lib/PublicInbox/V2Writable.pm
@@ -730,9 +730,8 @@ sub fill_alternates ($$) {
 sub git_init {
 	my ($self, $epoch) = @_;
 	my $git_dir = "$self->{-inbox}->{inboxdir}/git/$epoch.git";
-	my @cmd = (qw(git init --bare -q), $git_dir);
-	PublicInbox::Import::run_die(\@cmd);
-	@cmd = (qw/git config/, "--file=$git_dir/config",
+	PublicInbox::Import::init_bare($git_dir);
+	my @cmd = (qw/git config/, "--file=$git_dir/config",
 			'include.path', '../../all.git/config');
 	PublicInbox::Import::run_die(\@cmd);
 	fill_alternates($self, $epoch);
diff --git a/t/admin.t b/t/admin.t
index 41aebe46..a9d67d25 100644
--- a/t/admin.t
+++ b/t/admin.t
@@ -4,13 +4,14 @@ use strict;
 use warnings;
 use Test::More;
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 use_ok 'PublicInbox::Admin', qw(resolve_repo_dir);
 my ($tmpdir, $for_destroy) = tmpdir();
 my $git_dir = "$tmpdir/v1";
 my $v2_dir = "$tmpdir/v2";
 my ($res, $err, $v);
 
-is(0, system(qw(git init -q --bare), $git_dir), 'git init v1');
+PublicInbox::Import::init_bare($git_dir);
 
 # v1
 is(resolve_repo_dir($git_dir), $git_dir, 'top-level GIT_DIR resolved');
diff --git a/t/altid.t b/t/altid.t
index 6c34cdd6..3134e627 100644
--- a/t/altid.t
+++ b/t/altid.t
@@ -23,9 +23,9 @@ my $ibx;
 }
 
 {
-	is(system(qw(git init -q --bare), $git_dir), 0, 'git init ok');
 	my $git = PublicInbox::Git->new($git_dir);
 	my $im = PublicInbox::Import->new($git, 'testbox', 'test@example');
+	$im->init_bare;
 	$im->add(Email::MIME->create(
 		header => [
 			From => 'a@example.com',
diff --git a/t/cgi.t b/t/cgi.t
index 97bdebd9..52f80e88 100644
--- a/t/cgi.t
+++ b/t/cgi.t
@@ -7,6 +7,7 @@ use warnings;
 use Test::More;
 use Email::MIME;
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 require_mods(qw(Plack::Handler::CGI Plack::Util));
 my ($tmpdir, $for_destroy) = tmpdir();
 my $home = "$tmpdir/pi-home";
@@ -18,7 +19,7 @@ my $addr = 'test-public@example.com';
 {
 	is(1, mkdir($home, 0755), "setup ~/ for testing");
 	is(1, mkdir($pi_home, 0755), "setup ~/.public-inbox");
-	is(0, system(qw(git init -q --bare), $maindir), "git init (main)");
+	PublicInbox::Import::init_bare($maindir);
 
 	open my $fh, '>', "$maindir/description" or die "open: $!\n";
 	print $fh "test for public-inbox\n";
diff --git a/t/config.t b/t/config.t
index d08c3b9c..0db12359 100644
--- a/t/config.t
+++ b/t/config.t
@@ -5,10 +5,11 @@ use warnings;
 use Test::More;
 use PublicInbox::Config;
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 my ($tmpdir, $for_destroy) = tmpdir();
 
 {
-	is(system(qw(git init -q --bare), $tmpdir), 0, "git init successful");
+	PublicInbox::Import::init_bare($tmpdir);
 	my @cmd = ('git', "--git-dir=$tmpdir", qw(config foo.bar), "hi\nhi");
 	is(system(@cmd), 0, "set config");
 
diff --git a/t/convert-compact.t b/t/convert-compact.t
index 70609c7d..af16b701 100644
--- a/t/convert-compact.t
+++ b/t/convert-compact.t
@@ -20,8 +20,7 @@ my $ibx = {
 	-primary_address => 'test@example.com',
 };
 
-ok(PublicInbox::Import::run_die([qw(git init --bare -q), $ibx->{inboxdir}]),
-	'initialized v1 repo');
+PublicInbox::Import::init_bare($ibx->{inboxdir});
 ok(umask(077), 'set restrictive umask');
 ok(PublicInbox::Import::run_die([qw(git) , "--git-dir=$ibx->{inboxdir}",
 	qw(config core.sharedRepository 0644)]), 'set sharedRepository');
diff --git a/t/feed.t b/t/feed.t
index cfa09a7c..ffd5ca7e 100644
--- a/t/feed.t
+++ b/t/feed.t
@@ -34,7 +34,7 @@ my $git = $ibx->git;
 my $im = PublicInbox::Import->new($git, $ibx->{name}, 'test@example');
 
 {
-	is(0, system(qw(git init -q --bare), $git_dir), "git init");
+	$im->init_bare;
 	local $ENV{GIT_DIR} = $git_dir;
 
 	foreach my $i (1..6) {
diff --git a/t/git.t b/t/git.t
index 8224969d..0936ac5e 100644
--- a/t/git.t
+++ b/t/git.t
@@ -6,11 +6,12 @@ use Test::More;
 use PublicInbox::TestCommon;
 my ($dir, $for_destroy) = tmpdir();
 use PublicInbox::Spawn qw(popen_rd);
+use PublicInbox::Import;
 
 use_ok 'PublicInbox::Git';
 
 {
-	is(system(qw(git init -q --bare), $dir), 0, 'created git directory');
+	PublicInbox::Import::init_bare($dir);
 	my $fi_data = './t/git.fast-import-data';
 	ok(-r $fi_data, "fast-import data readable (or run test at top level)");
 	local $ENV{GIT_DIR} = $dir;
@@ -90,7 +91,7 @@ if (1) {
 if ('alternates reloaded') {
 	my ($alt, $alt_obj) = tmpdir();
 	my @cmd = ('git', "--git-dir=$alt", qw(hash-object -w --stdin));
-	is(system(qw(git init -q --bare), $alt), 0, 'create alt directory');
+	PublicInbox::Import::init_bare($alt);
 	open my $fh, '<', "$alt/config" or die "open failed: $!\n";
 	my $rd = popen_rd(\@cmd, {}, { 0 => $fh } );
 	close $fh or die "close failed: $!";
diff --git a/t/html_index.t b/t/html_index.t
index 158a7862..fda3962a 100644
--- a/t/html_index.t
+++ b/t/html_index.t
@@ -22,7 +22,7 @@ my $im = PublicInbox::Import->new($git, 'tester', 'test@example');
 
 # setup
 {
-	is(0, system(qw(git init -q --bare), $git_dir), "git init");
+	$im->init_bare;
 	my $prev = "";
 
 	foreach my $i (1..6) {
diff --git a/t/import.t b/t/import.t
index 703aa362..79af9846 100644
--- a/t/import.t
+++ b/t/import.t
@@ -12,10 +12,9 @@ use File::Temp qw/tempfile/;
 use PublicInbox::TestCommon;
 my ($dir, $for_destroy) = tmpdir();
 
-is(system(qw(git init -q --bare), $dir), 0, 'git init successful');
 my $git = PublicInbox::Git->new($dir);
-
 my $im = PublicInbox::Import->new($git, 'testbox', 'test@example');
+$im->init_bare;
 my $mime = PublicInbox::MIME->create(
 	header => [
 		From => 'a@example.com',
diff --git a/t/mda.t b/t/mda.t
index ec09cf69..af5e1931 100644
--- a/t/mda.t
+++ b/t/mda.t
@@ -9,6 +9,7 @@ use PublicInbox::MID qw(mid2path);
 use PublicInbox::Git;
 use PublicInbox::InboxWritable;
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 my ($tmpdir, $for_destroy) = tmpdir();
 my $home = "$tmpdir/pi-home";
 my $pi_home = "$home/.public-inbox";
@@ -48,7 +49,7 @@ my $fail_bad_header = sub ($$$) {
 		"spamc mock found (run in top of source tree");
 	is(1, mkdir($home, 0755), "setup ~/ for testing");
 	is(1, mkdir($pi_home, 0755), "setup ~/.public-inbox");
-	is(0, system(qw(git init -q --bare), $maindir), "git init (main)");
+	PublicInbox::Import::init_bare($maindir);
 
 	open my $fh, '>>', $pi_config or die;
 	print $fh <<EOF or die;
diff --git a/t/nulsubject.t b/t/nulsubject.t
index fec6c1ae..b21507c2 100644
--- a/t/nulsubject.t
+++ b/t/nulsubject.t
@@ -11,9 +11,9 @@ my ($tmpdir, $for_destroy) = tmpdir();
 my $git_dir = "$tmpdir/a.git";
 
 {
-	is(system(qw(git init -q --bare), $git_dir), 0, 'git init ok');
 	my $git = PublicInbox::Git->new($git_dir);
 	my $im = PublicInbox::Import->new($git, 'testbox', 'test@example');
+	$im->init_bare;
 	$im->add(Email::MIME->create(
 		header => [
 			From => 'a@example.com',
diff --git a/t/psgi_attach.t b/t/psgi_attach.t
index 297e92e1..a47f3754 100644
--- a/t/psgi_attach.t
+++ b/t/psgi_attach.t
@@ -21,9 +21,9 @@ my $config = PublicInbox::Config->new(\<<EOF);
 $cfgpfx.address=$addr
 $cfgpfx.inboxdir=$maindir
 EOF
-is(0, system(qw(git init -q --bare), $maindir), "git init (main)");
 my $git = PublicInbox::Git->new($maindir);
 my $im = PublicInbox::Import->new($git, 'test', $addr);
+$im->init_bare;
 
 {
 	open my $fh, '<', '/dev/urandom' or die "unable to open urandom: $!\n";
diff --git a/t/psgi_mount.t b/t/psgi_mount.t
index 8cac3502..3afb1fb5 100644
--- a/t/psgi_mount.t
+++ b/t/psgi_mount.t
@@ -21,9 +21,9 @@ my $config = PublicInbox::Config->new(\<<EOF);
 $cfgpfx.address=$addr
 $cfgpfx.inboxdir=$maindir
 EOF
-is(0, system(qw(git init -q --bare), $maindir), "git init (main)");
 my $git = PublicInbox::Git->new($maindir);
 my $im = PublicInbox::Import->new($git, 'test', $addr);
+$im->init_bare;
 {
 	my $mime = Email::MIME->new(<<EOF);
 From: Me <me\@example.com>
diff --git a/t/psgi_text.t b/t/psgi_text.t
index 6d8a518d..77fc9ee2 100644
--- a/t/psgi_text.t
+++ b/t/psgi_text.t
@@ -21,7 +21,7 @@ my $config = PublicInbox::Config->new(\<<EOF);
 $cfgpfx.address=$addr
 $cfgpfx.inboxdir=$maindir
 EOF
-is(0, system(qw(git init -q --bare), $maindir), "git init (main)");
+PublicInbox::Import::init_bare($maindir);
 my $www = PublicInbox::WWW->new($config);
 
 test_psgi(sub { $www->call(@_) }, sub {
diff --git a/t/search-thr-index.t b/t/search-thr-index.t
index f073304a..6c564b19 100644
--- a/t/search-thr-index.t
+++ b/t/search-thr-index.t
@@ -11,10 +11,11 @@ require_mods(qw(DBD::SQLite Search::Xapian));
 require PublicInbox::SearchIdx;
 require PublicInbox::Smsg;
 require PublicInbox::Inbox;
+use PublicInbox::Import;
 my ($tmpdir, $for_destroy) = tmpdir();
 my $git_dir = "$tmpdir/a.git";
 
-is(0, system(qw(git init -q --bare), $git_dir), "git init (main)");
+PublicInbox::Import::init_bare($git_dir);
 my $ibx = PublicInbox::Inbox->new({inboxdir => $git_dir});
 my $rw = PublicInbox::SearchIdx->new($ibx, 1);
 ok($rw, "search indexer created");
diff --git a/t/solver_git.t b/t/solver_git.t
index 7f0cd999..b5231d2c 100644
--- a/t/solver_git.t
+++ b/t/solver_git.t
@@ -109,7 +109,8 @@ SKIP: {
 	require_mods(@psgi, 7 + scalar(@psgi));
 	use_ok($_) for @psgi;
 	my $binfoo = "$inboxdir/binfoo.git";
-	system(qw(git init --bare -q), $binfoo) == 0 or die "git init: $?";
+	require PublicInbox::Import;
+	PublicInbox::Import::init_bare($binfoo);
 	require_ok 'PublicInbox::ViewVCS';
 	my $big_size = do {
 		no warnings 'once';
diff --git a/t/v1-add-remove-add.t b/t/v1-add-remove-add.t
index 736e414e..fdf06a96 100644
--- a/t/v1-add-remove-add.t
+++ b/t/v1-add-remove-add.t
@@ -9,7 +9,6 @@ use PublicInbox::TestCommon;
 require_mods(qw(DBD::SQLite Search::Xapian));
 require PublicInbox::SearchIdx;
 my ($inboxdir, $for_destroy) = tmpdir();
-is(system(qw(git init --bare -q), $inboxdir), 0);
 my $ibx = {
 	inboxdir => $inboxdir,
 	name => 'test-add-remove-add',
@@ -27,6 +26,7 @@ my $mime = PublicInbox::MIME->create(
 	body => "hello world\n",
 );
 my $im = PublicInbox::Import->new($ibx->git, undef, undef, $ibx);
+$im->init_bare;
 ok($im->add($mime), 'message added');
 ok($im->remove($mime), 'message removed');
 ok($im->add($mime), 'message added again');
diff --git a/t/v1reindex.t b/t/v1reindex.t
index 240e28f9..378c8efb 100644
--- a/t/v1reindex.t
+++ b/t/v1reindex.t
@@ -12,7 +12,6 @@ require_mods(qw(DBD::SQLite Search::Xapian));
 use_ok 'PublicInbox::SearchIdx';
 use_ok 'PublicInbox::Import';
 my ($inboxdir, $for_destroy) = tmpdir();
-is(system(qw(git init -q --bare), $inboxdir), 0);
 my $ibx_config = {
 	inboxdir => $inboxdir,
 	name => 'test-v1reindex',
@@ -35,6 +34,7 @@ my ($mark1, $mark2, $mark3, $mark4);
 	my %config = %$ibx_config;
 	my $ibx = PublicInbox::Inbox->new(\%config);
 	my $im = PublicInbox::Import->new($ibx->git, undef, undef, $ibx);
+	$im->init_bare;
 	foreach my $i (1..10) {
 		$mime->header_set('Message-Id', "<$i\@example.com>");
 		ok($im->add($mime), "message $i added");
diff --git a/t/watch_maildir.t b/t/watch_maildir.t
index 5fb908fb..9ed04cab 100644
--- a/t/watch_maildir.t
+++ b/t/watch_maildir.t
@@ -6,6 +6,7 @@ use Email::MIME;
 use Cwd;
 use PublicInbox::Config;
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 require_mods(qw(Filesys::Notify::Simple));
 my ($tmpdir, $for_destroy) = tmpdir();
 my $git_dir = "$tmpdir/test.git";
@@ -15,7 +16,7 @@ use_ok 'PublicInbox::WatchMaildir';
 use_ok 'PublicInbox::Emergency';
 my $cfgpfx = "publicinbox.test";
 my $addr = 'test-public@example.com';
-is(system(qw(git init -q --bare), $git_dir), 0, 'initialized git dir');
+PublicInbox::Import::init_bare($git_dir);
 
 my $msg = <<EOF;
 From: user\@example.com
diff --git a/t/watch_maildir_v2.t b/t/watch_maildir_v2.t
index 685cd6ed..dd5030ea 100644
--- a/t/watch_maildir_v2.t
+++ b/t/watch_maildir_v2.t
@@ -6,6 +6,7 @@ use PublicInbox::MIME;
 use Cwd;
 use PublicInbox::Config;
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 require_git(2.6);
 require_mods(qw(Search::Xapian DBD::SQLite Filesys::Notify::Simple));
 require PublicInbox::V2Writable;
@@ -141,7 +142,7 @@ More majordomo info at  http://vger.kernel.org/majordomo-info.html\n);
 	my $v1repo = "$tmpdir/v1";
 	my $v1pfx = "publicinbox.v1";
 	my $v1addr = 'v1-public@example.com';
-	is(system(qw(git init -q --bare), $v1repo), 0, 'v1 init OK');
+	PublicInbox::Import::init_bare($v1repo);
 	my $cfg2 = <<EOF;
 $orig$v1pfx.address=$v1addr
 $v1pfx.inboxdir=$v1repo
diff --git a/t/www_listing.t b/t/www_listing.t
index 9230329c..c132edbc 100644
--- a/t/www_listing.t
+++ b/t/www_listing.t
@@ -6,6 +6,7 @@ use warnings;
 use Test::More;
 use PublicInbox::Spawn qw(which);
 use PublicInbox::TestCommon;
+use PublicInbox::Import;
 require_mods(qw(URI::Escape Plack::Builder Digest::SHA
 		IO::Compress::Gzip IO::Uncompress::Gunzip HTTP::Tiny));
 require PublicInbox::WwwListing;
@@ -18,7 +19,7 @@ use_ok 'PublicInbox::Git';
 
 my ($tmpdir, $for_destroy) = tmpdir();
 my $bare = PublicInbox::Git->new("$tmpdir/bare.git");
-is(system(qw(git init -q --bare), $bare->{git_dir}), 0, 'git init --bare');
+PublicInbox::Import::init_bare($bare->{git_dir});
 is(PublicInbox::WwwListing::fingerprint($bare), undef,
 	'empty repo has no fingerprint');
 {
@@ -77,7 +78,7 @@ SKIP: {
 	my @clone = qw(git clone -q -s --bare);
 	is(system(@clone, $bare->{git_dir}, $alt), 0, 'clone shared repo');
 
-	system(qw(git init --bare -q), "$v2/all.git") == 0 or die;
+	PublicInbox::Import::init_bare("$v2/all.git");
 	for my $i (0..2) {
 		is(system(@clone, $alt, "$v2/git/$i.git"), 0, "clone epoch $i");
 	}

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 2/4] import: init_bare: use pure Perl
  2020-04-19 23:19 [PATCH 0/4] test speedups for FreeBSD Eric Wong
  2020-04-19 23:19 ` [PATCH 1/4] import: init_bare: allow use as method, use in tests Eric Wong
@ 2020-04-19 23:19 ` Eric Wong
  2020-04-19 23:19 ` [PATCH 3/4] t/ds-leak: use BSD::Resource Eric Wong
  2020-04-19 23:19 ` [PATCH 4/4] testcommon: spawn-aware system() and qx[] workalikes Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-04-19 23:19 UTC (permalink / raw)
  To: meta

Even on systems with Inline::C spawn(), this cuts a primed
"make check-run" time by 2-3% on Linux, and roughly 5-7% on
FreeBSD when using vfork-enabled spawn.

I doubt anybody cares: this omits the sample hooks and some
empty and useless-for-us or obsolete directories created by
git-init(1).
---
 lib/PublicInbox/Import.pm | 28 ++++++++++++++++++++++------
 1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm
index 351bc660..95d654f6 100644
--- a/lib/PublicInbox/Import.pm
+++ b/lib/PublicInbox/Import.pm
@@ -440,15 +440,31 @@ sub run_die ($;$$) {
 	$? == 0 or die join(' ', @$cmd) . " failed: $?\n";
 }
 
+my @INIT_FILES = ('HEAD' => "ref: refs/heads/master\n",
+		'description' => <<EOD,
+Unnamed repository; edit this file 'description' to name the repository.
+EOD
+		'config' => <<EOC);
+[core]
+	repositoryFormatVersion = 0
+	filemode = true
+	bare = true
+[repack]
+	writeBitmaps = true
+EOC
+
 sub init_bare {
 	my ($dir) = @_; # or self
 	$dir = $dir->{git}->{git_dir} if ref($dir);
-	my @cmd = (qw(git init --bare -q), $dir);
-	run_die(\@cmd);
-	# set a reasonable default:
-	@cmd = (qw/git config/, "--file=$dir/config",
-		'repack.writeBitmaps', 'true');
-	run_die(\@cmd);
+	require File::Path;
+	File::Path::mkpath([ map { "$dir/$_" } qw(objects/info refs/heads) ]);
+	for (my $i = 0; $i < @INIT_FILES; $i++) {
+		my $f = $dir.'/'.$INIT_FILES[$i++];
+		next if -f $f;
+		open my $fh, '>', $f or die "open $f: $!";
+		print $fh $INIT_FILES[$i] or die "print $f: $!";
+		close $fh or die "close $f: $!";
+	}
 }
 
 sub done {

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 3/4] t/ds-leak: use BSD::Resource
  2020-04-19 23:19 [PATCH 0/4] test speedups for FreeBSD Eric Wong
  2020-04-19 23:19 ` [PATCH 1/4] import: init_bare: allow use as method, use in tests Eric Wong
  2020-04-19 23:19 ` [PATCH 2/4] import: init_bare: use pure Perl Eric Wong
@ 2020-04-19 23:19 ` Eric Wong
  2020-04-19 23:19 ` [PATCH 4/4] testcommon: spawn-aware system() and qx[] workalikes Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-04-19 23:19 UTC (permalink / raw)
  To: meta

We use BSD::Resource in other places, so there's no sense
in avoiding it, here.
---
 t/ds-leak.t | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/t/ds-leak.t b/t/ds-leak.t
index ea0eeaa6..b29d814e 100644
--- a/t/ds-leak.t
+++ b/t/ds-leak.t
@@ -6,6 +6,7 @@
 use strict;
 use warnings;
 use Test::More;
+use PublicInbox::TestCommon;
 use_ok 'PublicInbox::DS';
 
 if ('close-on-exec for epoll and kqueue') {
@@ -44,8 +45,9 @@ if ('close-on-exec for epoll and kqueue') {
 }
 
 SKIP: {
-	# not bothering with BSD::Resource
-	chomp(my $n = `/bin/sh -c 'ulimit -n'`);
+	require_mods('BSD::Resource', 1);
+	my $rlim = BSD::Resource::RLIMIT_NOFILE();
+	my ($n,undef) = BSD::Resource::getrlimit($rlim);
 
 	# FreeBSD 11.2 with 2GB RAM gives RLIMIT_NOFILE=57987!
 	if ($n > 1024 && !$ENV{TEST_EXPENSIVE}) {

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 4/4] testcommon: spawn-aware system() and qx[] workalikes
  2020-04-19 23:19 [PATCH 0/4] test speedups for FreeBSD Eric Wong
                   ` (2 preceding siblings ...)
  2020-04-19 23:19 ` [PATCH 3/4] t/ds-leak: use BSD::Resource Eric Wong
@ 2020-04-19 23:19 ` Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2020-04-19 23:19 UTC (permalink / raw)
  To: meta

Barely noticeable on Linux, but this gives a 1-2% speedup
on a FreeBSD 11.3 VM and lets us use built-in redirects
rather than relying on /bin/sh.
---
 lib/PublicInbox/TestCommon.pm | 26 ++++++++++++++++++++++++--
 t/config.t                    |  2 +-
 t/ds-leak.t                   | 10 ++++++----
 t/edit.t                      |  2 +-
 t/git.t                       | 17 +++++++----------
 t/hl_mod.t                    | 20 +++++++-------------
 t/httpd-corner.t              | 30 ++++++++++++++----------------
 t/httpd.t                     |  8 ++++----
 t/indexlevels-mirror.t        |  8 ++++----
 t/init.t                      |  5 +++--
 t/mda.t                       |  2 +-
 t/mda_filter_rubylang.t       |  6 +++---
 t/multi-mid.t                 |  2 +-
 t/nntpd.t                     |  5 +++--
 t/nulsubject.t                |  3 ++-
 t/replace.t                   |  2 +-
 t/search.t                    |  2 +-
 t/solver_git.t                |  5 +++--
 t/v2mda.t                     |  2 +-
 t/v2mirror.t                  |  4 ++--
 t/v2writable.t                |  2 +-
 t/www_listing.t               | 19 +++++++++++--------
 xt/git-http-backend.t         |  6 +++---
 23 files changed, 104 insertions(+), 84 deletions(-)

diff --git a/lib/PublicInbox/TestCommon.pm b/lib/PublicInbox/TestCommon.pm
index e9efbac7..b50871e8 100644
--- a/lib/PublicInbox/TestCommon.pm
+++ b/lib/PublicInbox/TestCommon.pm
@@ -9,7 +9,7 @@ use Fcntl qw(FD_CLOEXEC F_SETFD F_GETFD :seek);
 use POSIX qw(dup2);
 use IO::Socket::INET;
 our @EXPORT = qw(tmpdir tcp_server tcp_connect require_git require_mods
-	run_script start_script key2sub);
+	run_script start_script key2sub xsys xqx);
 
 sub tmpdir (;$) {
 	my ($base) = @_;
@@ -87,7 +87,7 @@ sub require_mods {
 
 sub key2script ($) {
 	my ($key) = @_;
-	return $key if (index($key, '/') >= 0);
+	return $key if ($key eq 'git' || index($key, '/') >= 0);
 	# n.b. we may have scripts which don't start with "public-inbox" in
 	# the future:
 	$key =~ s/\A([-\.])/public-inbox$1/;
@@ -244,6 +244,28 @@ sub run_script ($;$$) {
 
 sub wait_for_tail () { sleep(2) }
 
+# like system() built-in, but uses spawn() for env/rdr + vfork
+sub xsys {
+	my ($cmd, $env, $rdr) = @_;
+	if (ref($cmd)) {
+		$rdr ||= {};
+	} else {
+		$cmd = [ @_ ];
+		$env = undef;
+		$rdr = {};
+	}
+	run_script($cmd, $env, { %$rdr, run_mode => 0 });
+	$? >> 8
+}
+
+# like `backtick` or qx{} op, but uses spawn() for env/rdr + vfork
+sub xqx {
+	my ($cmd, $env, $rdr) = @_;
+	$rdr //= {};
+	run_script($cmd, $env, { %$rdr, run_mode => 0, 1 => \(my $out) });
+	wantarray ? split(/^/m, $out) : $out;
+}
+
 sub start_script {
 	my ($cmd, $env, $opt) = @_;
 	my ($key, @argv) = @$cmd;
diff --git a/t/config.t b/t/config.t
index 0db12359..1f50bb86 100644
--- a/t/config.t
+++ b/t/config.t
@@ -11,7 +11,7 @@ my ($tmpdir, $for_destroy) = tmpdir();
 {
 	PublicInbox::Import::init_bare($tmpdir);
 	my @cmd = ('git', "--git-dir=$tmpdir", qw(config foo.bar), "hi\nhi");
-	is(system(@cmd), 0, "set config");
+	is(xsys(@cmd), 0, "set config");
 
 	my $tmp = PublicInbox::Config->new("$tmpdir/config");
 
diff --git a/t/ds-leak.t b/t/ds-leak.t
index b29d814e..72bf0379 100644
--- a/t/ds-leak.t
+++ b/t/ds-leak.t
@@ -10,7 +10,7 @@ use PublicInbox::TestCommon;
 use_ok 'PublicInbox::DS';
 
 if ('close-on-exec for epoll and kqueue') {
-	use PublicInbox::Spawn qw(spawn);
+	use PublicInbox::Spawn qw(spawn which);
 	my $pid;
 	my $evfd_re = qr/(?:kqueue|eventpoll)/i;
 
@@ -31,10 +31,12 @@ if ('close-on-exec for epoll and kqueue') {
 	my $l = <$r>;
 	is($l, undef, 'cloexec works and sleep(1) is running');
 
-	my @of = grep(/$evfd_re/, `lsof -p $pid 2>/dev/null`);
-	my $err = $?;
 	SKIP: {
-		skip "lsof missing? (\$?=$err)", 1 if $err;
+		my $lsof = which('lsof') or skip 'lsof missing', 1;
+		my $rdr = { 2 => \(my $null) };
+		my @of = grep(/$evfd_re/, xqx([$lsof, '-p', $pid], {}, $rdr));
+		my $err = $?;
+		skip "lsof broken ? (\$?=$err)", 1 if $err;
 		is_deeply(\@of, [], 'no FDs leaked to subprocess');
 	};
 	if (defined $pid) {
diff --git a/t/edit.t b/t/edit.t
index 2803dd01..d8833f9c 100644
--- a/t/edit.t
+++ b/t/edit.t
@@ -118,7 +118,7 @@ $t = 'non-interactive editor failure'; {
 
 $t = 'mailEditor set in config'; {
 	$in = $out = $err = '';
-	my $rc = system(qw(git config), "--file=$cfgfile",
+	my $rc = xsys(qw(git config), "--file=$cfgfile",
 			'publicinbox.maileditor',
 			"$^X -i -p -e 's/boolean prefix/bool pfx/'");
 	is($rc, 0, 'set publicinbox.mailEditor');
diff --git a/t/git.t b/t/git.t
index 0936ac5e..b05ac123 100644
--- a/t/git.t
+++ b/t/git.t
@@ -13,9 +13,10 @@ use_ok 'PublicInbox::Git';
 {
 	PublicInbox::Import::init_bare($dir);
 	my $fi_data = './t/git.fast-import-data';
-	ok(-r $fi_data, "fast-import data readable (or run test at top level)");
-	local $ENV{GIT_DIR} = $dir;
-	system("git fast-import --quiet <$fi_data");
+	open my $fh, '<', $fi_data or die
+		"fast-import data readable (or run test at top level: $!";
+	my $rdr = { 0 => $fh };
+	xsys([qw(git fast-import --quiet)], { GIT_DIR => $dir }, $rdr);
 	is($?, 0, 'fast-import succeeded');
 }
 
@@ -58,18 +59,14 @@ use_ok 'PublicInbox::Git';
 }
 
 if (1) {
-	my $cmd = [ 'git', "--git-dir=$dir", qw(hash-object -w --stdin) ];
-
 	# need a big file, use the AGPL-3.0 :p
 	my $big_data = './COPYING';
 	ok(-r $big_data, 'COPYING readable');
 	my $size = -s $big_data;
 	ok($size > 8192, 'file is big enough');
-
-	my $buf = do {
-		local $ENV{GIT_DIR} = $dir;
-		`git hash-object -w --stdin <$big_data`;
-	};
+	open my $fh, '<', $big_data or die;
+	my $cmd = [ 'git', "--git-dir=$dir", qw(hash-object -w --stdin) ];
+	my $buf = xqx($cmd, { GIT_DIR => $dir }, { 0 => $fh });
 	is(0, $?, 'hashed object successfully');
 	chomp $buf;
 
diff --git a/t/hl_mod.t b/t/hl_mod.t
index a4ef4a28..95057354 100644
--- a/t/hl_mod.t
+++ b/t/hl_mod.t
@@ -4,7 +4,8 @@
 use strict;
 use warnings;
 use Test::More;
-use PublicInbox::Spawn qw(which spawn);
+use PublicInbox::Spawn qw(which);
+use PublicInbox::TestCommon;
 use IO::Handle; # ->autoflush
 use Fcntl qw(:seek);
 eval { require highlight } or
@@ -29,21 +30,14 @@ my $orig = $str;
 	is($$ref, $$lref, 'do_hl_lang matches do_hl');
 
 	SKIP: {
-		which('w3m') or skip 'w3m(1) missing to check output', 1;
-		my $cmd = [ qw(w3m -T text/html -dump -config /dev/null) ];
-		open my $in, '+>', undef or die;
-		open my $out, '+>', undef or die;
-		my $rdr = { 0 => fileno($in), 1 => fileno($out) };
-		$in->autoflush(1);
-		print $in '<pre>', $$ref, '</pre>' or die;
-		$in->seek(0, SEEK_SET) or die;
-		my $pid = spawn($cmd, undef, $rdr);
-		waitpid($pid, 0);
+		my $w3m = which('w3m') or
+			skip('w3m(1) missing to check output', 1);
+		my $cmd = [ $w3m, qw(-T text/html -dump -config /dev/null) ];
+		my $in = '<pre>' . $$ref . '</pre>';
+		my $out = xqx($cmd, undef, { 0 => \$in });
 		# expand tabs and normalize whitespace,
 		# w3m doesn't preserve tabs
 		$orig =~ s/\t/        /gs;
-		$out->seek(0, SEEK_SET) or die;
-		$out = do { local $/; <$out> };
 		$out =~ s/\s*\z//sg;
 		$orig =~ s/\s*\z//sg;
 		is($out, $orig, 'w3m output matches');
diff --git a/t/httpd-corner.t b/t/httpd-corner.t
index f25a9a9c..7a6bcc66 100644
--- a/t/httpd-corner.t
+++ b/t/httpd-corner.t
@@ -6,7 +6,7 @@ use strict;
 use warnings;
 use Test::More;
 use Time::HiRes qw(gettimeofday tv_interval);
-use PublicInbox::Spawn qw(which spawn);
+use PublicInbox::Spawn qw(which spawn popen_rd);
 use PublicInbox::TestCommon;
 require_mods(qw(Plack::Util Plack::Builder HTTP::Date HTTP::Status));
 use Digest::SHA qw(sha1_hex);
@@ -26,9 +26,6 @@ my $psgi = "./t/httpd-corner.psgi";
 my $sock = tcp_server() or die;
 my @zmods = qw(PublicInbox::GzipFilter IO::Uncompress::Gunzip);
 
-# make sure stdin is not a pipe for lsof test to check for leaking pipes
-open(STDIN, '<', '/dev/null') or die 'no /dev/null: $!';
-
 # Make sure we don't clobber socket options set by systemd or similar
 # using socket activation:
 my ($defer_accept_val, $accf_arg, $TCP_DEFER_ACCEPT);
@@ -308,12 +305,12 @@ my $check_self = sub {
 };
 
 SKIP: {
-	which('curl') or skip('curl(1) missing', 4);
+	my $curl = which('curl') or skip('curl(1) missing', 4);
 	my $base = 'http://' . $sock->sockhost . ':' . $sock->sockport;
 	my $url = "$base/sha1";
 	my ($r, $w);
 	pipe($r, $w) or die "pipe: $!";
-	my $cmd = [qw(curl --tcp-nodelay --no-buffer -T- -HExpect: -sS), $url];
+	my $cmd = [$curl, qw(--tcp-nodelay -T- -HExpect: -sSN), $url];
 	open my $cout, '+>', undef or die;
 	open my $cerr, '>', undef or die;
 	my $rdr = { 0 => $r, 1 => $cout, 2 => $cerr };
@@ -330,7 +327,7 @@ SKIP: {
 	seek($cout, 0, SEEK_SET);
 	is(<$cout>, sha1_hex($str), 'read expected body');
 
-	open my $fh, '-|', qw(curl -sS), "$base/async-big" or die $!;
+	my $fh = popen_rd([$curl, '-sS', "$base/async-big"]);
 	my $n = 0;
 	my $non_zero = 0;
 	while (1) {
@@ -338,15 +335,14 @@ SKIP: {
 		$n += $r;
 		$buf =~ /\A\0+\z/ or $non_zero++;
 	}
-	close $fh or die "curl errored out \$?=$?";
+	close $fh or die "close curl pipe: $!";
+	is($?, 0, 'curl succesful');
 	is($n, 30 * 1024 * 1024, 'got expected output from curl');
 	is($non_zero, 0, 'read all zeros');
 
-	require_mods(@zmods, 1);
-	open $fh, '-|', qw(curl -sS), "$base/psgi-return-gzip" or die;
-	binmode $fh;
-	my $buf = do { local $/; <$fh> };
-	close $fh or die "curl errored out \$?=$?";
+	require_mods(@zmods, 2);
+	my $buf = xqx([$curl, '-sS', "$base/psgi-return-gzip"]);
+	is($?, 0, 'curl succesful');
 	IO::Uncompress::Gunzip::gunzip(\$buf => \(my $out));
 	is($out, "hello world\n");
 }
@@ -605,12 +601,14 @@ SKIP: {
 
 SKIP: {
 	skip 'only testing lsof(8) output on Linux', 1 if $^O ne 'linux';
-	skip 'no lsof in PATH', 1 unless which('lsof');
-	my @lsof = `lsof -p $td->{pid}`;
+	my $lsof = which('lsof') or skip 'no lsof in PATH', 1;
+	my $null_in = '';
+	my $rdr = { 2 => \(my $null_err), 0 => \$null_in };
+	my @lsof = xqx([$lsof, '-p', $td->{pid}], undef, $rdr);
 	is_deeply([grep(/\bdeleted\b/, @lsof)], [], 'no lingering deleted inputs');
 
 	# filter out pipes inherited from the parent
-	my @this = `lsof -p $$`;
+	my @this = xqx([$lsof, '-p', $$], undef, $rdr);
 	my $bad;
 	my $extract_inodes = sub {
 		map {;
diff --git a/t/httpd.t b/t/httpd.t
index 11511c73..61aec3b4 100644
--- a/t/httpd.t
+++ b/t/httpd.t
@@ -64,22 +64,22 @@ EOF
 		is($conn->read($buf, 1), 0, "EOF");
 	}
 
-	is(system(qw(git clone -q --mirror),
+	is(xsys(qw(git clone -q --mirror),
 			"http://$host:$port/$group", "$tmpdir/clone.git"),
 		0, 'smart clone successful');
 
 	# ensure dumb cloning works, too:
-	is(system('git', "--git-dir=$maindir",
+	is(xsys('git', "--git-dir=$maindir",
 		qw(config http.uploadpack false)),
 		0, 'disable http.uploadpack');
-	is(system(qw(git clone -q --mirror),
+	is(xsys(qw(git clone -q --mirror),
 			"http://$host:$port/$group", "$tmpdir/dumb.git"),
 		0, 'clone successful');
 
 	ok($td->kill, 'killed httpd');
 	$td->join;
 
-	is(system('git', "--git-dir=$tmpdir/clone.git",
+	is(xsys('git', "--git-dir=$tmpdir/clone.git",
 		  qw(fsck --no-verbose)), 0,
 		'fsck on cloned directory successful');
 }
diff --git a/t/indexlevels-mirror.t b/t/indexlevels-mirror.t
index 77c52718..f2200306 100644
--- a/t/indexlevels-mirror.t
+++ b/t/indexlevels-mirror.t
@@ -59,7 +59,7 @@ sub import_index_incremental {
 		push @cmd, "$ibx->{inboxdir}/git/0.git", "$mirror/git/0.git";
 	}
 	my $fetch_dir = $cmd[-1];
-	is(system(@cmd), 0, "v$v clone OK");
+	is(xsys(@cmd), 0, "v$v clone OK");
 
 	# inbox init
 	local $ENV{PI_CONFIG} = "$tmpdir/.picfg";
@@ -86,7 +86,7 @@ sub import_index_incremental {
 	$im->done;
 
 	# mirror updates
-	is(system('git', "--git-dir=$fetch_dir", qw(fetch -q)), 0, 'fetch OK');
+	is(xsys('git', "--git-dir=$fetch_dir", qw(fetch -q)), 0, 'fetch OK');
 	ok(run_script(['-index', $mirror]), "v$v index mirror again OK");
 	($nr, $msgs) = $ro_mirror->recent;
 	is($nr, 2, '2nd message seen in mirror');
@@ -123,7 +123,7 @@ sub import_index_incremental {
 	}
 
 	# sync the mirror
-	is(system('git', "--git-dir=$fetch_dir", qw(fetch -q)), 0, 'fetch OK');
+	is(xsys('git', "--git-dir=$fetch_dir", qw(fetch -q)), 0, 'fetch OK');
 	ok(run_script(['-index', $mirror]), "v$v index mirror again OK");
 	($nr, $msgs) = $ro_mirror->recent;
 	is($nr, 1, '2nd message gone from mirror');
@@ -148,7 +148,7 @@ sub import_index_incremental {
 		push @expect, $i;
 	}
 	$im->done;
-	is(system('git', "--git-dir=$fetch_dir", qw(fetch -q)), 0, 'fetch OK');
+	is(xsys('git', "--git-dir=$fetch_dir", qw(fetch -q)), 0, 'fetch OK');
 	ok(run_script(['-index', '--reindex', $mirror]),
 		"v$v index --reindex mirror OK");
 	@ro_nums = map { $_->{num} } @{$ro_mirror->over->query_ts(0, 0)};
diff --git a/t/init.t b/t/init.t
index a78c2fc8..94c6184e 100644
--- a/t/init.t
+++ b/t/init.t
@@ -105,7 +105,8 @@ done_testing();
 
 sub read_indexlevel {
 	my ($inbox) = @_;
-	local $ENV{GIT_CONFIG} = "$ENV{PI_DIR}/config";
-	chomp(my $lvl = `git config publicinbox.$inbox.indexlevel`);
+	my $cmd = [ qw(git config), "publicinbox.$inbox.indexlevel" ];
+	my $env = { GIT_CONFIG => "$ENV{PI_DIR}/config" };
+	chomp(my $lvl = xqx($cmd, $env));
 	$lvl;
 }
diff --git a/t/mda.t b/t/mda.t
index af5e1931..dc691616 100644
--- a/t/mda.t
+++ b/t/mda.t
@@ -298,7 +298,7 @@ Subject: this message will be trained as spam
 Date: Thu, 01 Jan 1970 00:00:00 +0000
 
 EOF
-	system(qw(git config --file), $pi_config, "$cfgpfx.listid", $list_id);
+	xsys(qw(git config --file), $pi_config, "$cfgpfx.listid", $list_id);
 	$? == 0 or die "failed to set listid $?";
 	my $in = $simple->as_string;
 	ok(run_script(['-mda'], undef, { 0 => \$in }),
diff --git a/t/mda_filter_rubylang.t b/t/mda_filter_rubylang.t
index 6f288b7e..f2cbe9d5 100644
--- a/t/mda_filter_rubylang.t
+++ b/t/mda_filter_rubylang.t
@@ -14,7 +14,7 @@ my $pi_config = "$tmpdir/pi_config";
 local $ENV{PI_CONFIG} = $pi_config;
 local $ENV{PI_EMERGENCY} = "$tmpdir/emergency";
 my @cfg = ('git', 'config', "--file=$pi_config");
-is(system(@cfg, 'publicinboxmda.spamcheck', 'none'), 0);
+is(xsys(@cfg, 'publicinboxmda.spamcheck', 'none'), 0);
 
 for my $v (qw(V1 V2)) {
 	my @warn;
@@ -26,8 +26,8 @@ for my $v (qw(V1 V2)) {
 		"http://example.com/$v", $addr ];
 	ok(run_script($cmd), 'public-inbox-init');
 	ok(run_script(['-index', $inboxdir]), 'public-inbox-index');
-	is(system(@cfg, "$cfgpfx.filter", 'PublicInbox::Filter::RubyLang'), 0);
-	is(system(@cfg, "$cfgpfx.altid",
+	is(xsys(@cfg, "$cfgpfx.filter", 'PublicInbox::Filter::RubyLang'), 0);
+	is(xsys(@cfg, "$cfgpfx.altid",
 		'serial:alerts:file=msgmap.sqlite3'), 0);
 
 	for my $i (1..2) {
diff --git a/t/multi-mid.t b/t/multi-mid.t
index 31a8fd74..5afb9693 100644
--- a/t/multi-mid.t
+++ b/t/multi-mid.t
@@ -65,7 +65,7 @@ for my $order ([$bad, $good], [$good, $bad]) {
 	my @v2 = ($ibx->over->get_art(1), $ibx->over->get_art(2));
 	is_deeply(\@v2, \@old, 'v2 conversion times match');
 
-	system(qw(git clone -sq --mirror), "$tmpdir/v2/git/0.git",
+	xsys(qw(git clone -sq --mirror), "$tmpdir/v2/git/0.git",
 		"$tmpdir/v2-clone/git/0.git") == 0 or die "clone: $?";
 	$cmd = [ '-init', '-Lbasic', '-V2', 'v2c', "$tmpdir/v2-clone",
 		'http://example.com/v2c', 'v2c@example.com' ];
diff --git a/t/nntpd.t b/t/nntpd.t
index 826e3f3d..66aa48f1 100644
--- a/t/nntpd.t
+++ b/t/nntpd.t
@@ -48,7 +48,7 @@ $ibx = PublicInbox::Inbox->new($ibx);
 	my @cmd = ('-init', $group, $inboxdir, 'http://example.com/', $addr);
 	push @cmd, "-V$version", '-Lbasic';
 	ok(run_script(\@cmd), 'init OK');
-	is(system(qw(git config), "--file=$home/.public-inbox/config",
+	is(xsys(qw(git config), "--file=$home/.public-inbox/config",
 			"publicinbox.$group.newsgroup", $group),
 		0, 'enabled newsgroup');
 	my $len;
@@ -304,7 +304,8 @@ Date: Fri, 02 Oct 1993 00:00:00 +0000
 		if ($INC{'Search/Xapian.pm'} && ($ENV{TEST_RUN_MODE}//2)) {
 			skip 'Search/Xapian.pm pre-loaded (by t/run.perl?)', 1;
 		}
-		my @of = `lsof -p $td->{pid} 2>/dev/null`;
+		my $rdr = { 2 => \(my $null) };
+		my @of = xqx(['lsof', '-p', $td->{pid}], undef, $rdr);
 		skip('lsof broken', 1) if (!scalar(@of) || $?);
 		my @xap = grep m!Search/Xapian!, @of;
 		is_deeply(\@xap, [], 'Xapian not loaded in nntpd');
diff --git a/t/nulsubject.t b/t/nulsubject.t
index b21507c2..1ded88d3 100644
--- a/t/nulsubject.t
+++ b/t/nulsubject.t
@@ -25,7 +25,8 @@ my $git_dir = "$tmpdir/a.git";
 		body => "hello world\n",
 	));
 	$im->done;
-	is(system(qw(git --git-dir), $git_dir, 'fsck', '--strict'), 0, 'git fsck ok');
+	is(xsys(qw(git --git-dir), $git_dir, 'fsck', '--strict'), 0,
+		'git fsck ok');
 }
 
 done_testing();
diff --git a/t/replace.t b/t/replace.t
index a44560fc..2efa25f1 100644
--- a/t/replace.t
+++ b/t/replace.t
@@ -95,7 +95,7 @@ EOF
 
 	for my $dir (glob("$ibx->{inboxdir}/git/*.git")) {
 		my ($bn) = ($dir =~ m!([^/]+)\z!);
-		is(system(qw(git --git-dir), $dir,
+		is(xsys(qw(git --git-dir), $dir,
 					qw(fsck --strict --no-progress)),
 			0, "git fsck is clean in epoch $bn");
 	}
diff --git a/t/search.t b/t/search.t
index 101d44e9..8508f273 100644
--- a/t/search.t
+++ b/t/search.t
@@ -14,7 +14,7 @@ my $git_dir = "$tmpdir/a.git";
 my $ibx = PublicInbox::Inbox->new({ inboxdir => $git_dir });
 my ($root_id, $last_id);
 
-is(0, system(qw(git init --shared -q --bare), $git_dir), "git init (main)")
+is(0, xsys(qw(git init --shared -q --bare), $git_dir), "git init (main)")
 	or BAIL_OUT("`git init --shared' failed, weird FS or seccomp?");
 eval { PublicInbox::Search->new($ibx)->xdb };
 ok($@, "exception raised on non-existent DB");
diff --git a/t/solver_git.t b/t/solver_git.t
index b5231d2c..7f79ff4c 100644
--- a/t/solver_git.t
+++ b/t/solver_git.t
@@ -8,8 +8,9 @@ use PublicInbox::TestCommon;
 require_git(2.6);
 use PublicInbox::Spawn qw(popen_rd);
 require_mods(qw(DBD::SQLite Search::Xapian Plack::Util));
-chomp(my $git_dir = `git rev-parse --git-dir 2>/dev/null`);
-plan skip_all => "$0 must be run from a git working tree" if $?;
+my $git_dir = xqx([qw(git rev-parse --git-dir)], undef, {2 => \(my $null)});
+$? == 0 or plan skip_all => "$0 must be run from a git working tree";
+chomp $git_dir;
 
 # needed for alternates, and --absolute-git-dir is only in git 2.13+
 $git_dir = abs_path($git_dir);
diff --git a/t/v2mda.t b/t/v2mda.t
index c2118a89..e9dcdf44 100644
--- a/t/v2mda.t
+++ b/t/v2mda.t
@@ -72,7 +72,7 @@ is($saved->{mime}->as_string, $mime->as_string, 'injected message');
 	my $config = "$ENV{PI_DIR}/config";
 	ok(-f $config, 'config exists');
 	my $k = 'publicinboxmda.spamcheck';
-	is(system('git', 'config', "--file=$config", $k, 'none'), 0,
+	is(xsys('git', 'config', "--file=$config", $k, 'none'), 0,
 		'disabled spamcheck for mda');
 
 	ok(run_script(['-mda'], undef, $rdr), 'mda did not die');
diff --git a/t/v2mirror.t b/t/v2mirror.t
index 2e23e763..406bbd4f 100644
--- a/t/v2mirror.t
+++ b/t/v2mirror.t
@@ -76,7 +76,7 @@ foreach my $i (0..$epoch_max) {
 		"http://$host:$port/v2/$i$sfx",
 		"$tmpdir/m/git/$i.git");
 
-	is(system(@cmd), 0, "cloned $i.git");
+	is(xsys(@cmd), 0, "cloned $i.git");
 	ok(-d "$tmpdir/m/git/$i.git", "mirror $i OK");
 }
 
@@ -102,7 +102,7 @@ $ibx->cleanup;
 my $fetch_each_epoch = sub {
 	foreach my $i (0..$epoch_max) {
 		my $dir = "$tmpdir/m/git/$i.git";
-		is(system('git', "--git-dir=$dir", 'fetch', '-q'), 0,
+		is(xsys('git', "--git-dir=$dir", 'fetch', '-q'), 0,
 			'fetch successful');
 	}
 };
diff --git a/t/v2writable.t b/t/v2writable.t
index 66d5663e..8897062a 100644
--- a/t/v2writable.t
+++ b/t/v2writable.t
@@ -48,7 +48,7 @@ my $git0;
 if ('ensure git configs are correct') {
 	my @cmd = (qw(git config), "--file=$inboxdir/all.git/config",
 		qw(core.sharedRepository 0644));
-	is(system(@cmd), 0, "set sharedRepository in all.git");
+	is(xsys(@cmd), 0, "set sharedRepository in all.git");
 	$git0 = PublicInbox::Git->new("$inboxdir/git/0.git");
 	chomp(my $v = $git0->qx(qw(config core.sharedRepository)));
 	is($v, '0644', 'child repo inherited core.sharedRepository');
diff --git a/t/www_listing.t b/t/www_listing.t
index c132edbc..31d76356 100644
--- a/t/www_listing.t
+++ b/t/www_listing.t
@@ -24,8 +24,10 @@ is(PublicInbox::WwwListing::fingerprint($bare), undef,
 	'empty repo has no fingerprint');
 {
 	my $fi_data = './t/git.fast-import-data';
-	local $ENV{GIT_DIR} = $bare->{git_dir};
-	is(system("git fast-import --quiet <$fi_data"), 0, 'fast-import');
+	open my $fh, '<', $fi_data or die "open $fi_data: $!";
+	my $env = { GIT_DIR => $bare->{git_dir} };
+	is(xsys([qw(git fast-import --quiet)], $env, { 0 => $fh }), 0,
+		'fast-import');
 }
 
 like(PublicInbox::WwwListing::fingerprint($bare), qr/\A[a-f0-9]{40}\z/,
@@ -76,17 +78,17 @@ SKIP: {
 	ok($sock, 'sock created');
 	my ($host, $port) = ($sock->sockhost, $sock->sockport);
 	my @clone = qw(git clone -q -s --bare);
-	is(system(@clone, $bare->{git_dir}, $alt), 0, 'clone shared repo');
+	is(xsys(@clone, $bare->{git_dir}, $alt), 0, 'clone shared repo');
 
 	PublicInbox::Import::init_bare("$v2/all.git");
 	for my $i (0..2) {
-		is(system(@clone, $alt, "$v2/git/$i.git"), 0, "clone epoch $i");
+		is(xsys(@clone, $alt, "$v2/git/$i.git"), 0, "clone epoch $i")
 	}
 	ok(open(my $fh, '>', "$v2/inbox.lock"), 'mock a v2 inbox');
 	open $fh, '>', "$alt/description" or die;
 	print $fh "we're all clones\n" or die;
 	close $fh or die;
-	is(system('git', "--git-dir=$alt", qw(config gitweb.owner lorelei)), 0,
+	is(xsys('git', "--git-dir=$alt", qw(config gitweb.owner lorelei)), 0,
 		'set gitweb user');
 	ok(unlink("$bare->{git_dir}/description"), 'removed bare/description');
 	open $fh, '>', $cfgfile or die;
@@ -114,7 +116,8 @@ SKIP: {
 
 	tiny_test($json, $host, $port);
 
-	skip 'skipping grok-pull integration test', 2 if !which('grok-pull');
+	my $grok_pull = which('grok-pull') or
+		skip('skipping grok-pull integration test', 2);
 
 	ok(mkdir("$tmpdir/mirror"), 'prepare grok mirror dest');
 	open $fh, '>', "$tmpdir/repos.conf" or die;
@@ -129,7 +132,7 @@ mymanifest = $tmpdir/local-manifest.js.gz
 
 	close $fh or die;
 
-	system(qw(grok-pull -c), "$tmpdir/repos.conf");
+	xsys($grok_pull, '-c', "$tmpdir/repos.conf");
 	is($? >> 8, 127, 'grok-pull exit code as expected');
 	for (qw(alt bare v2/git/0.git v2/git/1.git v2/git/2.git)) {
 		ok(-d "$tmpdir/mirror/$_", "grok-pull created $_");
@@ -149,7 +152,7 @@ mymanifest = $tmpdir/per-inbox-manifest.js.gz
 
 	close $fh or die;
 	ok(mkdir("$tmpdir/per-inbox"), 'prepare single-v2-inbox mirror');
-	system(qw(grok-pull -c), "$tmpdir/per-inbox.conf");
+	xsys($grok_pull, '-c', "$tmpdir/per-inbox.conf");
 	is($? >> 8, 127, 'grok-pull exit code as expected');
 	for (qw(v2/git/0.git v2/git/1.git v2/git/2.git)) {
 		ok(-d "$tmpdir/per-inbox/$_", "grok-pull created $_");
diff --git a/xt/git-http-backend.t b/xt/git-http-backend.t
index f2ae44fe..2f02725a 100644
--- a/xt/git-http-backend.t
+++ b/xt/git-http-backend.t
@@ -83,13 +83,13 @@ SKIP: { # make sure Last-Modified + If-Modified-Since works with curl
 	skip 'curl(1) not found', $nr unless $curl;
 	my $url = "http://$host:$port/description";
 	my $dst = "$tmpdir/desc";
-	is(system($curl, qw(-RsSf), '-o', $dst, $url), 0, 'curl -R');
+	is(xsys($curl, qw(-RsSf), '-o', $dst, $url), 0, 'curl -R');
 	is((stat($dst))[9], $mtime, 'curl used remote mtime');
-	is(system($curl, qw(-sSf), '-z', $dst, '-o', "$dst.2", $url), 0,
+	is(xsys($curl, qw(-sSf), '-z', $dst, '-o', "$dst.2", $url), 0,
 		'curl -z noop');
 	ok(!-e "$dst.2", 'no modification, nothing retrieved');
 	utime(0, 0, $dst) or die "utime failed: $!";
-	is(system($curl, qw(-sSfR), '-z', $dst, '-o', "$dst.2", $url), 0,
+	is(xsys($curl, qw(-sSfR), '-z', $dst, '-o', "$dst.2", $url), 0,
 		'curl -z updates');
 	ok(-e "$dst.2", 'faked modification, got new file retrieved');
 }

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-04-19 23:19 [PATCH 0/4] test speedups for FreeBSD Eric Wong
2020-04-19 23:19 ` [PATCH 1/4] import: init_bare: allow use as method, use in tests Eric Wong
2020-04-19 23:19 ` [PATCH 2/4] import: init_bare: use pure Perl Eric Wong
2020-04-19 23:19 ` [PATCH 3/4] t/ds-leak: use BSD::Resource Eric Wong
2020-04-19 23:19 ` [PATCH 4/4] testcommon: spawn-aware system() and qx[] workalikes Eric Wong

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror http://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git