* [PATCH 12/14] www_coderepo: fix read buffering
2023-12-13 0:50 7% [PATCH 00/14] Alpine Linux support Eric Wong
@ 2023-12-13 0:50 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2023-12-13 0:50 UTC (permalink / raw)
To: meta
Our read buffering only worked well with the stdout buffering on
glibc and *BSD libc, but not musl. When reading the stdout of
git(1), we are likely to get smaller buffers and require more
reads on musl-based systems (tested Alpine Linux 3.19.0).
Thus we must prevent ->translate from being called with an empty
argument list (denoting EOF). We'll also avoid some local
variable assignments while at it and favor the non-OO ->zflush
dispatch inside RepoAtom and WwwCoderepo subclasses.
---
lib/PublicInbox/RepoAtom.pm | 4 ++--
lib/PublicInbox/WwwCoderepo.pm | 6 +++---
t/solver_git.t | 8 +++++---
3 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/lib/PublicInbox/RepoAtom.pm b/lib/PublicInbox/RepoAtom.pm
index c1649d0a..ab0f2fcc 100644
--- a/lib/PublicInbox/RepoAtom.pm
+++ b/lib/PublicInbox/RepoAtom.pm
@@ -43,7 +43,7 @@ sub zflush { $_[0]->SUPER::zflush('</feed>') }
# called by GzipFilter->write or GetlineResponse->getline
sub translate {
my $self = shift;
- my $rec = $_[0] // return $self->zflush; # getline
+ $_[0] // return zflush($self); # getline caller
my @out;
my $lbuf = delete($self->{lbuf}) // shift;
$lbuf .= shift while @_;
@@ -87,7 +87,7 @@ xmlns="http://www.w3.org/1999/xhtml"><pre style="white-space:pre-wrap">
}
$self->{lbuf} = $lbuf;
chomp @out;
- $self->SUPER::translate(@out);
+ @out ? $self->SUPER::translate(@out) : ''; # not EOF, yet
}
# $REPO/tags.atom endpoint
diff --git a/lib/PublicInbox/WwwCoderepo.pm b/lib/PublicInbox/WwwCoderepo.pm
index d1354af5..3814f719 100644
--- a/lib/PublicInbox/WwwCoderepo.pm
+++ b/lib/PublicInbox/WwwCoderepo.pm
@@ -273,7 +273,7 @@ sub zflush { $_[0]->SUPER::zflush('</pre>', $_[0]->_html_end) }
# called by GzipFilter->write or GetlineResponse->getline
sub translate {
my $ctx = shift;
- my $rec = $_[0] // return zflush($ctx); # getline
+ $_[0] // return zflush($ctx); # getline caller
my @out;
my $fbuf = delete($ctx->{fbuf}) // shift;
$fbuf .= shift while @_;
@@ -290,8 +290,8 @@ sub translate {
$snap_pfx, @snap_fmt);
}
}
- $ctx->{fbuf} = $fbuf;
- $ctx->SUPER::translate(@out);
+ $ctx->{fbuf} = $fbuf; # may be incomplete
+ @out ? $ctx->SUPER::translate(@out) : ''; # not EOF, yet
}
sub _refs_parse_hdr { # {parse_hdr} for Qspawn
diff --git a/t/solver_git.t b/t/solver_git.t
index ab8aba15..db672904 100644
--- a/t/solver_git.t
+++ b/t/solver_git.t
@@ -405,14 +405,16 @@ EOF
is($res->code, 200, 'Atom feed');
SKIP: {
require_mods('XML::TreePP', 1);
- my $t = XML::TreePP->new->parse($res->content);
+ my $t = eval { XML::TreePP->new->parse($res->content) }
+ or diag explain($res);
is(scalar @{$t->{feed}->{entry}}, 50,
- 'got 50 entries');
+ 'got 50 entries') or diag explain([$t, $res]);
$res = $cb->(GET('/public-inbox/atom/COPYING'));
is($res->code, 200, 'file Atom feed');
$t = XML::TreePP->new->parse($res->content);
- ok($t->{feed}->{entry}, 'got entry');
+ ok($t->{feed}->{entry}, 'got entry') or
+ diag explain([ $t, $res ]);
$res = $cb->(GET('/public-inbox/atom/README.md'));
is($res->code, 404, '404 on missing file Atom feed');
^ permalink raw reply related [relevance 7%]
* [PATCH 00/14] Alpine Linux support
@ 2023-12-13 0:50 7% Eric Wong
2023-12-13 0:50 7% ` [PATCH 12/14] www_coderepo: fix read buffering Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2023-12-13 0:50 UTC (permalink / raw)
To: meta
I haven't tested every single possible package combination, but
I think this is enough to get started.
Patch 3 was a WTF moment for me.
Some of these are relevant for other platforms, as well, and
patch 7 decoupling from Inline::C is a good step towards making
our codebase more modular.
Patch 12 is a good fix regardless.
Eric Wong (14):
t/io: strace is optional on Linux
tests: account for missing git-http-backend
t/cindex*: skip --join when join(1) is missing
tests: attempt compatibility w/ busybox lsof
lei inspect: drop unneeded strftime import
treewide: avoid strftime %k for portability
xap_helper_cxx: decouple from Inline::C
xap_helper_cxx: support clang w/o `c++' executable
install: updates for Alpine Linux and apk
test_common: extract oct_is from search.t
t/convert-compact: allow S_ISGID bit
www_coderepo: fix read buffering
gzip_filter: use OO ->zflush dispatch
t/lei-import: relax EIO regexp
install/deps.perl | 38 +++++++++++++++++++++----
install/os.perl | 12 ++++++--
lib/PublicInbox/Admin.pm | 10 ++++++-
lib/PublicInbox/ExtSearchIdx.pm | 4 +--
lib/PublicInbox/GzipFilter.pm | 2 +-
lib/PublicInbox/Hval.pm | 6 +++-
lib/PublicInbox/LeiInspect.pm | 1 -
lib/PublicInbox/LeiMirror.pm | 4 +--
lib/PublicInbox/LeiXSearch.pm | 3 +-
lib/PublicInbox/RepoAtom.pm | 4 +--
lib/PublicInbox/TestCommon.pm | 50 +++++++++++++++++++++++++++++++--
lib/PublicInbox/WwwCoderepo.pm | 6 ++--
lib/PublicInbox/WwwStatic.pm | 5 ++--
lib/PublicInbox/XapHelperCxx.pm | 26 +++++++++++------
t/cindex-join.t | 1 +
t/cindex.t | 11 ++++----
t/clone-coderepo.t | 1 +
t/convert-compact.t | 20 ++++++-------
t/ds-leak.t | 5 +---
t/httpd-corner.t | 31 ++++++++------------
t/httpd.t | 1 +
t/lei-import.t | 5 ++--
t/lei-mirror.t | 1 +
t/nntpd.t | 16 +++++------
t/search.t | 5 ----
t/solver_git.t | 8 ++++--
t/v2reindex.t | 5 ++--
t/www_listing.t | 1 +
t/xap_helper.t | 6 ++--
xt/msgtime_cmp.t | 2 +-
30 files changed, 190 insertions(+), 100 deletions(-)
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-12-13 0:50 7% [PATCH 00/14] Alpine Linux support Eric Wong
2023-12-13 0:50 7% ` [PATCH 12/14] www_coderepo: fix read buffering Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).