user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 06/11] t/run.perl: fix for >128 tests
Date: Tue,  9 Feb 2021 07:09:32 -0100	[thread overview]
Message-ID: <20210209080937.4678-7-e@80x24.org> (raw)
In-Reply-To: <20210209080937.4678-1-e@80x24.org>

We need to explicitly close the write-end of the pipe in workers
to ensure they don't prevent each other from seeing EOF.

Also, make a note to keep using the pipe for now since
Linux <3.14 had broken read(2) semantics when file descriptions
are shared across threads/processes.
---
 t/run.perl | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/t/run.perl b/t/run.perl
index 96db3045..d0b29e68 100755
--- a/t/run.perl
+++ b/t/run.perl
@@ -127,9 +127,10 @@ my $producer = $$;
 my $eof; # we stop respawning if true
 
 my $start_worker = sub {
-	my ($i, $j, $rd, $todo) = @_;
+	my ($j, $rd, $wr, $todo) = @_;
 	my $pid = fork // DIE "fork: $!";
 	if ($pid == 0) {
+		close $wr if $wr;
 		$worker = $$;
 		while (1) {
 			my $r = sysread($rd, my $buf, UINT_SIZE);
@@ -155,15 +156,16 @@ my $start_worker = sub {
 for (my $i = $repeat; $i != 0; $i--) {
 	my @todo = $shuffle ? List::Util::shuffle(@tests) : @tests;
 
-	# single-producer, multi-consumer queue relying on POSIX semantics
+	# single-producer, multi-consumer queue relying on POSIX pipe semantics
+	# POSIX.1-2008 stipulates a regular file should work, but Linux <3.14
+	# had broken read(2) semantics according to the read(2) manpage
 	pipe(my ($rd, $wr)) or DIE "pipe: $!";
 
 	# fill the queue before forking so children can start earlier
 	my $n = (_POSIX_PIPE_BUF / UINT_SIZE);
 	if ($n >= $#todo) {
 		print $wr join('', map { pack('I', $_) } (0..$#todo)) or DIE;
-		close $wr or die;
-		$wr = undef;
+		undef $wr;
 	} else { # write what we can...
 		$wr->autoflush(1);
 		print $wr join('', map { pack('I', $_) } (0..$n)) or DIE;
@@ -186,22 +188,21 @@ for (my $i = $repeat; $i != 0; $i--) {
 			# skip_all can exit(0), respawn if needed:
 			if (!$eof) {
 				print $OLDERR "# respawning job[$j]\n";
-				$start_worker->($i, $j, $rd, \@todo);
+				$start_worker->($j, $rd, $wr, \@todo);
 			}
 		}
 	};
 
 	# start the workers to consume the queue
 	for (my $j = 0; $j < $jobs; $j++) {
-		$start_worker->($i, $j, $rd, \@todo);
+		$start_worker->($j, $rd, $wr, \@todo);
 	}
-
 	if ($wr) {
 		local $SIG{CHLD} = $sigchld;
 		# too many tests to fit in the pipe before starting workers,
 		# send the rest now the workers are running
 		print $wr join('', map { pack('I', $_) } ($n..$#todo)) or DIE;
-		close $wr or die;
+		undef $wr;
 	}
 
 	$sigchld->(0) while scalar(keys(%pids));

  parent reply	other threads:[~2021-02-09  8:09 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-09  8:09 [PATCH 00/11] Maildir code consolidation, test updates Eric Wong
2021-02-09  8:09 ` [PATCH 01/11] t/thread-index-gap.t: avoid unnecessary map Eric Wong
2021-02-09  8:09 ` [PATCH 02/11] test_common: disable fsync on the CLI where possible Eric Wong
2021-02-09  8:09 ` [PATCH 03/11] t/cgi.t: modernizations and style updates Eric Wong
2021-02-09  8:09 ` [PATCH 04/11] git: ->qx: respect caller's $/ in array context Eric Wong
2021-02-09  8:09 ` [PATCH 05/11] lei: split out MdirReader package, lazy-require earlier Eric Wong
2021-02-09  8:09 ` Eric Wong [this message]
2021-02-09  8:09 ` [PATCH 07/11] use MdirReader in -watch and InboxWritable Eric Wong
2021-02-09  8:09 ` [PATCH 08/11] lei q: prefix --alert ops with ':' instead of '-' Eric Wong
2021-02-09  8:09 ` [PATCH 09/11] t/run.perl: drop Cwd dependency Eric Wong
2021-02-09  8:09 ` [PATCH 10/11] lei: replace "I:"-prefixed info messages with "#" Eric Wong
2021-02-09  8:09 ` [PATCH 11/11] tests|lei: fixes for TEST_RUN_MODE=0 and lei oneshot Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210209080937.4678-7-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    --subject='Re: [PATCH 06/11] t/run.perl: fix for >128 tests' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V1 meta meta/ https://public-inbox.org/meta \
		meta@public-inbox.org
	public-inbox-index meta

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.mail.public-inbox.meta
	nntp://ie5yzdi7fg72h7s4sdcztq5evakq23rdt33mfyfcddc5u3ndnw24ogqd.onion/inbox.comp.mail.public-inbox.meta
	nntp://4uok3hntl7oi7b4uf4rtfwefqeexfzil2w6kgk2jn5z2f764irre7byd.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general
 note: .onion URLs require Tor: https://www.torproject.org/

code repositories for project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git