* [PATCH 14/14] solver: crank up max patches to 9999
2019-01-27 4:03 6% [PATCH 00/14] convert solver to use pi-httpd.async Eric Wong
@ 2019-01-27 4:03 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-01-27 4:03 UTC (permalink / raw)
To: meta
Might as well, since the only constraint is filesystem space
for temporary files for public-inbox-httpd users.
-httpd can fairly share work across clients with our use of
psgi_qx; and there's a recent patch series in git@vger with 64
patches in sequence.
---
lib/PublicInbox/SolverGit.pm | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/SolverGit.pm b/lib/PublicInbox/SolverGit.pm
index dfc28d2..39acbe4 100644
--- a/lib/PublicInbox/SolverGit.pm
+++ b/lib/PublicInbox/SolverGit.pm
@@ -24,6 +24,14 @@ use URI::Escape qw(uri_escape_utf8);
use POSIX qw(sysconf _SC_ARG_MAX);
my $ARG_SIZE_MAX = (sysconf(_SC_ARG_MAX) || 4096) - 2048;
+# By default, "git format-patch" generates filenames with a four-digit
+# prefix, so that means 9999 patch series are OK, right? :>
+# Maybe we can make this configurable, main concern is disk space overhead
+# for uncompressed patch fragments. Aside from space, public-inbox-httpd
+# is otherwise unaffected by having many patches, here, as it can share
+# work fairly. Other PSGI servers may have trouble, though.
+my $MAX_PATCH = 9999;
+
# di = diff info / a hashref with information about a diff ($di):
# {
# oid_a => abbreviated pre-image oid,
@@ -425,7 +433,7 @@ sub di_url ($$) {
sub resolve_patch ($$) {
my ($self, $want) = @_;
- if (scalar(@{$self->{patches}}) > $self->{max_patch}) {
+ if (scalar(@{$self->{patches}}) > $MAX_PATCH) {
die "Aborting, too many steps to $self->{oid_want}";
}
@@ -477,7 +485,6 @@ sub new {
bless {
gits => $ibx->{-repo_objs},
user_cb => $user_cb,
- max_patch => 100,
# TODO: config option for searching related inboxes
inboxes => [ $ibx ],
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 00/14] convert solver to use pi-httpd.async
@ 2019-01-27 4:03 6% Eric Wong
2019-01-27 4:03 7% ` [PATCH 14/14] solver: crank up max patches to 9999 Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-01-27 4:03 UTC (permalink / raw)
To: meta
Much of the groundwork for this was laid in the now-abandoned
"repobrowse" branch. The goal was to improves fairness as we no
longer wait synchronously on git (apply|update-index|ls-files)
processes and can requests for other clients.
The end result was slightly (2-3%?) slower with all the
callbacks, but reducing "git apply" invocations by relying on
pathnames (instead of stdin) made the end result ~20% faster for
a large (64) patch series.
Email::Simple (via Email::MIME/PublicInbox::MIME) remains a
performance bottleneck, as it does a lot of unnecessary header
parsing and hash-table populating we don't care about; but I'm
not sure if I'll have time to address that.
Eric Wong (14):
httpd/async: remove needless sysread wrapper
qspawn: implement psgi_return and use it for githttpbackend
qspawn|getlinebody: support streaming filters
qspawn|httpd/async: improve and fix out-of-date comments
httpd/async: stop running command if client disconnects
qspawn: implement psgi_qx
t/qspawn.t: psgi_qx stderr test
view: swap CRLF for LF in HTML output
solver: rewrite to use Qspawn->psgi_qx and pi-httpd.async
solver: hold patches in temporary directory
solver: reduce "git apply" invocations
qspawn: decode $? for user-friendliness
viewvcs: do not show final error message twice
solver: crank up max patches to 9999
lib/PublicInbox/GetlineBody.pm | 16 +-
lib/PublicInbox/Git.pm | 2 +-
lib/PublicInbox/GitHTTPBackend.pm | 64 +---
lib/PublicInbox/HTTPD/Async.pm | 27 +-
lib/PublicInbox/Qspawn.pm | 143 +++++++-
lib/PublicInbox/SolverGit.pm | 532 +++++++++++++++++-------------
lib/PublicInbox/View.pm | 4 +
lib/PublicInbox/ViewVCS.pm | 50 ++-
t/qspawn.t | 12 +-
t/solver_git.t | 22 +-
10 files changed, 543 insertions(+), 329 deletions(-)
--
EW
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-01-27 4:03 6% [PATCH 00/14] convert solver to use pi-httpd.async Eric Wong
2019-01-27 4:03 7% ` [PATCH 14/14] solver: crank up max patches to 9999 Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).