user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 12/14] treewide: use eof and close to detect readline errors
Date: Thu,  2 Nov 2023 09:35:37 +0000	[thread overview]
Message-ID: <20231102093539.2067470-13-e@80x24.org> (raw)
In-Reply-To: <20231102093539.2067470-1-e@80x24.org>

readline (<FH>) isn't wrapped by autodie, and there's no
way to know if read(2) errors truncated the readline output.
IO::Handle->error isn't reliable on Perl < v5.34.

Thus, combining the `eof' and `close' (combined with autodie) is
the only way we can detect read(2) errors (injected via strace)
when called via `readline' (aka <$fh>).  Neither using `eof'
nor `close' alone is sufficient, they must be combined to detect
errors from buffered `readline'.
---
 lib/PublicInbox/CodeSearchIdx.pm | 4 ++--
 lib/PublicInbox/Gcf2.pm          | 2 ++
 lib/PublicInbox/LeiInput.pm      | 3 ++-
 lib/PublicInbox/LeiMirror.pm     | 4 ++--
 lib/PublicInbox/Spawn.pm         | 4 ++--
 5 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/lib/PublicInbox/CodeSearchIdx.pm b/lib/PublicInbox/CodeSearchIdx.pm
index ad915fa2..0b00c303 100644
--- a/lib/PublicInbox/CodeSearchIdx.pm
+++ b/lib/PublicInbox/CodeSearchIdx.pm
@@ -57,7 +57,7 @@ use PublicInbox::Git qw(%OFMT2HEXLEN);
 use PublicInbox::Compat qw(uniqstr);
 use PublicInbox::Aspawn qw(run_await);
 use Carp ();
-use autodie qw(pipe open sysread seek sysseek send);
+use autodie qw(close pipe open sysread seek sysseek send);
 our $DO_QUIT = 15; # signal number
 our (
 	$LIVE_JOBS, # integer
@@ -628,7 +628,7 @@ sub index_repo { # run_git cb
 	my $roots_fh = delete $repo->{roots_fh} // die 'BUG: no {roots_fh}';
 	seek($roots_fh, 0, SEEK_SET);
 	chomp(my @roots = <$roots_fh>);
-	close($roots_fh);
+	$roots_fh = eof($roots_fh) | close $roots_fh; # detect readline errors
 	if (!@roots) {
 		warn("E: $git->{git_dir} has no root commits\n");
 		return index_next($self);
diff --git a/lib/PublicInbox/Gcf2.pm b/lib/PublicInbox/Gcf2.pm
index 502bf33a..6ee0d7d9 100644
--- a/lib/PublicInbox/Gcf2.pm
+++ b/lib/PublicInbox/Gcf2.pm
@@ -11,6 +11,7 @@ use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
 use IO::Handle; # autoflush
 use PublicInbox::Git;
 use PublicInbox::Lock;
+use autodie qw(close);
 
 BEGIN {
 	use autodie;
@@ -81,6 +82,7 @@ sub add_alt ($$) {
 	# See https://bugs.debian.org/975607
 	if (open(my $fh, '<', "$objdir/info/alternates")) {
 		chomp(my @abs_alt = grep(m!^/!, <$fh>));
+		$fh = eof($fh) | close $fh; # detect readline errors
 		$gcf2->add_alternate($_) for @abs_alt;
 	}
 	$gcf2->add_alternate($objdir);
diff --git a/lib/PublicInbox/LeiInput.pm b/lib/PublicInbox/LeiInput.pm
index f7c3f573..4cd18c09 100644
--- a/lib/PublicInbox/LeiInput.pm
+++ b/lib/PublicInbox/LeiInput.pm
@@ -80,7 +80,8 @@ sub input_net_cb { # imap_each, nntp_each cb
 sub input_fh {
 	my ($self, $ifmt, $fh, $name, @args) = @_;
 	if ($ifmt eq 'eml') {
-		my $buf = do { local $/; <$fh> } //
+		my $buf = do { local $/; <$fh> };
+		(defined($buf) && eof($fh) && close($fh)) or
 			return $self->{lei}->child_error(0, <<"");
 error reading $name: $!
 
diff --git a/lib/PublicInbox/LeiMirror.pm b/lib/PublicInbox/LeiMirror.pm
index fb6517bd..e4914f75 100644
--- a/lib/PublicInbox/LeiMirror.pm
+++ b/lib/PublicInbox/LeiMirror.pm
@@ -309,9 +309,9 @@ sub fgrp_update {
 	seek($srcfh, 0, SEEK_SET);
 	seek($dstfh, 0, SEEK_SET);
 	my %src = map { chomp; split(/\0/) } (<$srcfh>);
-	close $srcfh;
 	my %dst = map { chomp; split(/\0/) } (<$dstfh>);
-	close $dstfh;
+	$srcfh = eof($srcfh) | close $srcfh; # detects readline errors
+	$dstfh = eof($dstfh) | close $dstfh; # ditto
 	my $w = start_update_ref($fgrp) or return;
 	my $lei = $fgrp->{lei};
 	my $ndel;
diff --git a/lib/PublicInbox/Spawn.pm b/lib/PublicInbox/Spawn.pm
index b0edeb33..8c798b39 100644
--- a/lib/PublicInbox/Spawn.pm
+++ b/lib/PublicInbox/Spawn.pm
@@ -22,7 +22,7 @@ use Carp qw(croak);
 use PublicInbox::IO;
 our @EXPORT_OK = qw(which spawn popen_rd popen_wr run_die run_wait run_qx);
 our @RLIMITS = qw(RLIMIT_CPU RLIMIT_CORE RLIMIT_DATA);
-use autodie qw(open pipe seek sysseek truncate);
+use autodie qw(close open pipe seek sysseek truncate);
 
 BEGIN {
 	my $all_libc = <<'ALL_LIBC'; # all *nix systems we support
@@ -405,7 +405,7 @@ sub read_out_err ($) {
 		my $dst = $opt->{$fd};
 		$dst = $opt->{$fd} = $dst->[1] if ref($dst) eq 'ARRAY';
 		$$dst .= <$fh>;
-		$fh->error and croak "E: read(FD=$fd): $!";
+		$fh = eof($fh) | close $fh; # detects readline errors
 	}
 }
 

  parent reply	other threads:[~2023-11-02  9:35 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-02  9:35 [PATCH 00/14] IO/IPC-related cleanups Eric Wong
2023-11-02  9:35 ` [PATCH 01/14] xap_helper.pm: use do_fork to Reset and reseed Eric Wong
2023-11-02  9:35 ` [PATCH 02/14] ds: replace FD map hash table with array Eric Wong
2023-11-02  9:35 ` [PATCH 03/14] treewide: use ->close method rather than CORE::close Eric Wong
2023-11-02 21:35   ` [PATCH 15/14] ds: don't try ->close after ->accept_SSL failure Eric Wong
2023-11-02  9:35 ` [PATCH 04/14] cindex: drop redundant close on regular FH Eric Wong
2023-11-02  9:35 ` [PATCH 05/14] treewide: use ->close to call ProcessIO->CLOSE Eric Wong
2023-11-02  9:35 ` [PATCH 06/14] multi_git: use autodie Eric Wong
2023-11-02  9:35 ` [PATCH 07/14] git_credential: use autodie where appropriate Eric Wong
2023-11-02  9:35 ` [PATCH 08/14] replace ProcessIO with untied PublicInbox::IO Eric Wong
2023-11-02  9:35 ` [PATCH 09/14] io: introduce write_file helper sub Eric Wong
2023-11-02  9:35 ` [PATCH 10/14] spawn: support PerlIO layer in scalar redirects Eric Wong
2023-11-02  9:35 ` [PATCH 11/14] treewide: check alternates writes with eof + autodie Eric Wong
2023-11-02  9:35 ` Eric Wong [this message]
2023-11-02  9:35 ` [PATCH 13/14] move read_all, try_cat, and poll_in to PublicInbox::IO Eric Wong
2023-11-02 20:59   ` www: squash read_all usage fix Eric Wong
2023-11-02  9:35 ` [PATCH 14/14] t/cindex+extsearch: use write_file, autodie, etc Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231102093539.2067470-13-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).