* [PATCH 01/18] cindex: check `say' errors w/ close or ->flush
@ 2023-11-13 13:15 5% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2023-11-13 13:15 UTC (permalink / raw)
To: meta
We actually need to rely on autodie `close' to check for errors,
since error-checking with `say' is not useful due to perlio
write buffering. We'll also stop relying on `say ... or die'
since it's needless noise.
Fixes: 19f9089343c9 (cindex: drop redundant close on regular FH)
---
lib/PublicInbox/CodeSearchIdx.pm | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/CodeSearchIdx.pm b/lib/PublicInbox/CodeSearchIdx.pm
index 0b00c303..5b80db44 100644
--- a/lib/PublicInbox/CodeSearchIdx.pm
+++ b/lib/PublicInbox/CodeSearchIdx.pm
@@ -468,12 +468,12 @@ sub partition_refs ($$$) {
chomp $cmt;
my $n = hex(substr($cmt, 0, 8)) % scalar(@RDONLY_XDB);
if ($REINDEX && $REINDEX->set_maybe(pack('H*', $cmt), '')) {
- say { $shard_in[$n] } $cmt or die "say: $!";
+ say { $shard_in[$n] } $cmt;
++$NCHANGE;
} elsif (seen($RDONLY_XDB[$n], 'Q'.$cmt)) {
last if ++$seen > $SEEN_MAX;
} else {
- say { $shard_in[$n] } $cmt or die "say: $!";
+ say { $shard_in[$n] } $cmt;
++$NCHANGE;
$seen = 0;
}
@@ -845,7 +845,7 @@ EOM
PublicInbox::Import::init_bare($git_dir, 'cidx-all', $fmt);
open $ALT_FH{$hexlen}, '>', "$git_dir/objects/info/alternates";
}
- say { $ALT_FH{$hexlen} } $objdir or die "say: $!";
+ say { $ALT_FH{$hexlen} } $objdir;
}
sub prep_alternate_start {
@@ -969,7 +969,7 @@ sub dump_git_commits { # run_await cb
(defined($pid) && $?) and die "E: @PRUNE_BATCH: \$?=$?";
return if $DO_QUIT;
my ($hexlen) = keys(%ALT_FH) or return; # done
- delete $ALT_FH{$hexlen};
+ close(delete $ALT_FH{$hexlen}); # flushes `say' buffer
$PRUNE_BATCH[1] = "--git-dir=$TMPDIR/hexlen$hexlen.git";
run_await(\@PRUNE_BATCH, undef, $batch_opt, \&dump_git_commits);
^ permalink raw reply related [relevance 5%]
* [PATCH 04/14] cindex: drop redundant close on regular FH
2023-11-02 9:35 5% [PATCH 00/14] IO/IPC-related cleanups Eric Wong
@ 2023-11-02 9:35 7% ` Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2023-11-02 9:35 UTC (permalink / raw)
To: meta
There's no need to waste optree space on close() statements for
file handles which are (effectively) read-only on their last use
and incapable of error checking in our Perl code (since they're
only read by git).
Let Perl refcounting take care of it so we have less code to
wade through when focusing on `close' statements which actually
matter.
---
lib/PublicInbox/CodeSearchIdx.pm | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/lib/PublicInbox/CodeSearchIdx.pm b/lib/PublicInbox/CodeSearchIdx.pm
index c1a1ee90..ad915fa2 100644
--- a/lib/PublicInbox/CodeSearchIdx.pm
+++ b/lib/PublicInbox/CodeSearchIdx.pm
@@ -268,7 +268,6 @@ sub shard_index { # via wq_io_do in IDX_SHARDS
my $cmd = $git->cmd(@LOG_STDIN);
my $rd = popen_rd($cmd, undef, { 0 => $in },
\&cidx_reap_log, $cmd, $self, $op_p);
- close $in;
PublicInbox::CidxLogP->new($rd, $self, $git, $roots);
# CidxLogP->event_step will call cidx_read_log_p once there's input
}
@@ -457,7 +456,6 @@ sub partition_refs ($$$) {
my ($self, $git, $refs) = @_; # show-ref --heads --tags --hash output
sysseek($refs, 0, SEEK_SET);
my $rfh = $git->popen(qw(rev-list --stdin), undef, { 0 => $refs });
- close $refs;
my $seen = 0;
my @shard_in = map {
$_->reopen;
@@ -971,7 +969,7 @@ sub dump_git_commits { # run_await cb
(defined($pid) && $?) and die "E: @PRUNE_BATCH: \$?=$?";
return if $DO_QUIT;
my ($hexlen) = keys(%ALT_FH) or return; # done
- close(delete $ALT_FH{$hexlen});
+ delete $ALT_FH{$hexlen};
$PRUNE_BATCH[1] = "--git-dir=$TMPDIR/hexlen$hexlen.git";
run_await(\@PRUNE_BATCH, undef, $batch_opt, \&dump_git_commits);
^ permalink raw reply related [relevance 7%]
* [PATCH 00/14] IO/IPC-related cleanups
@ 2023-11-02 9:35 5% Eric Wong
2023-11-02 9:35 7% ` [PATCH 04/14] cindex: drop redundant close on regular FH Eric Wong
0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2023-11-02 9:35 UTC (permalink / raw)
To: meta
8/14 to replace ProcessIO is a major change that I've been
hammering away at for a bit, and many preceding patches to
eliminate ->close calls make it easier to review.
11-12/14 - Error checking for buffered readline is a PITA
(IO::Handle->error was insufficient until Perl 5.34), and I may
switch to sysread eventually to avoid the double-copy overhead
of buffered bulk I/O, anyways. The only place we really benefit
from userspace buffered disk reads is IdxStack, I think...
The new write_file sub in 9/14 seems long overdue.
Eric Wong (14):
xap_helper.pm: use do_fork to Reset and reseed
ds: replace FD map hash table with array
treewide: use ->close method rather than CORE::close
cindex: drop redundant close on regular FH
treewide: use ->close to call ProcessIO->CLOSE
multi_git: use autodie
git_credential: use autodie where appropriate
replace ProcessIO with untied PublicInbox::IO
io: introduce write_file helper sub
spawn: support PerlIO layer in scalar redirects
treewide: check alternates writes with eof + autodie
treewide: use eof and close to detect readline errors
move read_all, try_cat, and poll_in to PublicInbox::IO
t/cindex+extsearch: use write_file, autodie, etc.
MANIFEST | 4 +-
lib/PublicInbox/CodeSearchIdx.pm | 14 +++---
lib/PublicInbox/Config.pm | 2 +-
lib/PublicInbox/DS.pm | 22 ++++-----
lib/PublicInbox/DirIdle.pm | 2 +-
lib/PublicInbox/Gcf2.pm | 4 +-
lib/PublicInbox/Gcf2Client.pm | 7 ++-
lib/PublicInbox/Git.pm | 36 ++++----------
lib/PublicInbox/GitCredential.pm | 16 +++----
lib/PublicInbox/HTTP.pm | 4 +-
lib/PublicInbox/IO.pm | 80 ++++++++++++++++++++++++++++++++
lib/PublicInbox/IdxStack.pm | 2 +-
lib/PublicInbox/Import.pm | 18 ++++---
lib/PublicInbox/Inbox.pm | 2 +-
lib/PublicInbox/InboxWritable.pm | 2 +-
lib/PublicInbox/InputPipe.pm | 1 -
lib/PublicInbox/LEI.pm | 8 ++--
lib/PublicInbox/LeiALE.pm | 6 +--
lib/PublicInbox/LeiBlob.pm | 3 +-
lib/PublicInbox/LeiConfig.pm | 2 +-
lib/PublicInbox/LeiInput.pm | 5 +-
lib/PublicInbox/LeiMailSync.pm | 3 +-
lib/PublicInbox/LeiMirror.pm | 57 +++++++++--------------
lib/PublicInbox/LeiRediff.pm | 11 ++---
lib/PublicInbox/LeiStoreErr.pm | 2 +-
lib/PublicInbox/LeiSucks.pm | 2 +-
lib/PublicInbox/LeiToMail.pm | 34 +++++++-------
lib/PublicInbox/LeiViewText.pm | 2 +-
lib/PublicInbox/LeiXSearch.pm | 8 ++--
lib/PublicInbox/MailDiff.pm | 21 ++++-----
lib/PublicInbox/MboxReader.pm | 4 +-
lib/PublicInbox/MultiGit.pm | 13 +++---
lib/PublicInbox/ProcessIO.pm | 75 ------------------------------
lib/PublicInbox/ProcessIONBF.pm | 25 ----------
lib/PublicInbox/Qspawn.pm | 5 +-
lib/PublicInbox/SearchIdx.pm | 4 +-
lib/PublicInbox/SolverGit.pm | 38 ++++++---------
lib/PublicInbox/Spawn.pm | 42 ++++++++++-------
lib/PublicInbox/TestCommon.pm | 4 +-
lib/PublicInbox/V2Writable.pm | 2 +-
lib/PublicInbox/ViewVCS.pm | 6 +--
lib/PublicInbox/WWW.pm | 2 +-
lib/PublicInbox/XapHelper.pm | 14 ++----
lib/PublicInbox/XapHelperCxx.pm | 3 +-
script/public-inbox-convert | 6 +--
script/public-inbox-edit | 3 +-
script/public-inbox-init | 4 +-
t/cindex.t | 17 ++-----
t/clone-coderepo.t | 10 ++--
t/extsearch.t | 48 ++++++-------------
t/httpd-corner.t | 2 +-
t/init.t | 2 +-
t/io.t | 33 +++++++++++++
t/lei-mirror.t | 10 ++--
t/lei-q-kw.t | 4 +-
t/spawn.t | 36 +++++++-------
xt/check-run.t | 2 +-
xt/git_async_cmp.t | 4 +-
xt/httpd-async-stream.t | 2 +-
59 files changed, 368 insertions(+), 432 deletions(-)
create mode 100644 lib/PublicInbox/IO.pm
delete mode 100644 lib/PublicInbox/ProcessIO.pm
delete mode 100644 lib/PublicInbox/ProcessIONBF.pm
create mode 100644 t/io.t
^ permalink raw reply [relevance 5%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-11-02 9:35 5% [PATCH 00/14] IO/IPC-related cleanups Eric Wong
2023-11-02 9:35 7% ` [PATCH 04/14] cindex: drop redundant close on regular FH Eric Wong
2023-11-13 13:15 [PATCH 00/18] cindex: some --associate work Eric Wong
2023-11-13 13:15 5% ` [PATCH 01/18] cindex: check `say' errors w/ close or ->flush Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).