* [PATCH 03/37] t/perf-msgview: add test to check msg_html performance
2019-01-21 20:52 6% [PATCH 00/37] viewvcs: diff highlighting and more Eric Wong
@ 2019-01-21 20:52 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-01-21 20:52 UTC (permalink / raw)
To: meta
This will be necessary to ensure we maintain reasonable
performance when we add diff-highlighting support.
---
MANIFEST | 1 +
t/perf-msgview.t | 50 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 51 insertions(+)
create mode 100644 t/perf-msgview.t
diff --git a/MANIFEST b/MANIFEST
index e4f3df8..dfd9e27 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -185,6 +185,7 @@ t/nntp.t
t/nntpd.t
t/nulsubject.t
t/over.t
+t/perf-msgview.t
t/perf-nntpd.t
t/perf-threading.t
t/plack.t
diff --git a/t/perf-msgview.t b/t/perf-msgview.t
new file mode 100644
index 0000000..adeb7aa
--- /dev/null
+++ b/t/perf-msgview.t
@@ -0,0 +1,50 @@
+# Copyright (C) 2019 all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+use strict;
+use warnings;
+use Test::More;
+use Benchmark qw(:all);
+use PublicInbox::Inbox;
+use PublicInbox::View;
+require './t/common.perl';
+
+my @cat = qw(cat-file --buffer --batch-check --batch-all-objects);
+if (require_git(2.19, 1)) {
+ push @cat, '--unordered';
+} else {
+ warn
+"git <2.19, cat-file lacks --unordered, locality suffers\n";
+}
+
+my $pi_dir = $ENV{GIANT_PI_DIR};
+plan skip_all => "GIANT_PI_DIR not defined for $0" unless $pi_dir;
+
+my $ibx = PublicInbox::Inbox->new({ mainrepo => $pi_dir, name => 'name' });
+my $git = $ibx->git;
+my $fh = $git->popen(@cat);
+my $vec = '';
+vec($vec, fileno($fh), 1) = 1;
+select($vec, undef, undef, 60) or die "timed out waiting for --batch-check";
+
+my $ctx = {
+ env => { HTTP_HOST => 'example.com', 'psgi.url_scheme' => 'https' },
+ -inbox => $ibx,
+};
+my ($str, $mime, $res, $cmt, $type);
+my $n = 0;
+my $t = timeit(1, sub {
+ while (<$fh>) {
+ ($cmt, $type) = split / /;
+ next if $type ne 'blob';
+ ++$n;
+ $str = $git->cat_file($cmt);
+ $mime = PublicInbox::MIME->new($str);
+ $res = PublicInbox::View::msg_html($ctx, $mime);
+ $res = $res->[2];
+ while (defined($res->getline)) {}
+ $res->close;
+ }
+});
+diag 'msg_html took '.timestr($t)." for $n messages";
+ok 1;
+done_testing();
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 00/37] viewvcs: diff highlighting and more
@ 2019-01-21 20:52 6% Eric Wong
2019-01-21 20:52 7% ` [PATCH 03/37] t/perf-msgview: add test to check msg_html performance Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2019-01-21 20:52 UTC (permalink / raw)
To: meta
Still working on VCS integration and I'm not comfortable deploying
this on the main public-inbox.org because of performance/fairness
concerns, yet.
But, perfect is the enemy of good and I figure it's worth
publishing at the moment. It's also on a Tor mirror:
http://hjrcffqmbrq6wope.onion/meta/
http://hjrcffqmbrq6wope.onion/git/
It looks great to me in Netsurf and dillo :>
People with machines powerful enough to run Firefox
(or Tor Browser Bundle) can use "View -> Page Style" to adjust
colors.
Performance considerations:
* diff highlighting alone adds 10-20% overhead to message rendering
Maybe I can speed it up with some less-readable Perl...
* blob reconstruction is horribly unfair to other clients at the
moment. Fixing this is a priority for me.
I haven't hooked up highlight to blob viewing, yet; but that's
coming; too.
Thinking about it more, the blob lookups is so specific to git
that I'm not sure other VCSes can be supported...
The following changes since commit 55db8a2a51c13aec813ac56bbaac1505791fd262:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TODO: autolinkify that(!)
t/git.t: do not pass "-b" to git-repack(1) (2019-01-18 22:00:33 +0000)
are available in the Git repository at:
https://public-inbox.org/ viewvcs
for you to fetch changes up to c440c879d38e67f62bdbb74f616dc84d20899c33:
t/check-www-inbox: trap SIGINT for File::Temp destruction (2019-01-21 06:53:35 +0000)
----------------------------------------------------------------
Eric Wong (37):
view: disable bold in topic display
hval: force monospace for <form> elements, too
t/perf-msgview: add test to check msg_html performance
solver: initial Perl implementation
git: support multiple URL endpoints
git: add git_quote
git: check saves error on disambiguation
solver: various bugfixes and cleanups
view: wire up diff and vcs viewers with solver
git: disable abbreviations with cat-file hints
solver: operate directly on git index
view: enable naming hints for raw blob downloads
git: support 'ambiguous' result from --batch-check
solver: more verbose blob resolution
solver: break up patch application steps
solver: switch patch application to use a callback
solver: simplify control flow for initial loop
solver: break @todo loop into a callback
solver: note the synchronous nature of index preparation
solver: add a TODO note about making this fully evented
view: enforce trailing slash for /$INBOX/$OID/s/ endpoints
solver: restore diagnostics and deal with CRLF
www: admin-configurable CSS via "publicinbox.css"
$INBOX/_/text/color/ and sample user-side CSS
viewdiff: support diff-highlighting w/o coderepo
viewdiff: cleanup state transitions a bit
viewdiff: quote attributes for Atom feed
t/check-www-inbox: use xmlstarlet to validate Atom if available
viewdiff: do not link to 0{7,40} blobs (again)
viewvcs: disable white-space prewrap in blob view
solver: force quoted-printable bodies to LF
solver: remove extra "^index $OID..$OID" line
config: each_inbox iteration preserves config order
t/check-www-inbox: warn on missing Content-Type
highlight: initial wrapper and PSGI service
hval: split out escape sequences to a separate table
t/check-www-inbox: trap SIGINT for File::Temp destruction
Documentation/design_www.txt | 6 +-
MANIFEST | 15 +
Makefile.PL | 3 +
TODO | 2 -
contrib/css/216dark.css | 26 ++
contrib/css/216light.css | 25 ++
contrib/css/README | 41 +++
examples/highlight.psgi | 13 +
examples/public-inbox.psgi | 2 +-
lib/PublicInbox/Config.pm | 96 +++++-
lib/PublicInbox/Git.pm | 87 ++++-
lib/PublicInbox/HlMod.pm | 126 ++++++++
lib/PublicInbox/Hval.pm | 38 +--
lib/PublicInbox/SolverGit.pm | 454 +++++++++++++++++++++++++++
lib/PublicInbox/UserContent.pm | 78 +++++
lib/PublicInbox/View.pm | 51 ++-
lib/PublicInbox/ViewDiff.pm | 161 ++++++++++
lib/PublicInbox/ViewVCS.pm | 110 +++++++
lib/PublicInbox/WWW.pm | 152 ++++++++-
lib/PublicInbox/WwwHighlight.pm | 73 +++++
lib/PublicInbox/WwwStream.pm | 4 +-
lib/PublicInbox/WwwText.pm | 35 +++
script/public-inbox-httpd | 2 +-
t/check-www-inbox.perl | 26 +-
t/config.t | 19 ++
t/git.t | 7 +-
t/hl_mod.t | 54 ++++
t/perf-msgview.t | 50 +++
t/solve/0001-simple-mod.patch | 20 ++
t/solve/0002-rename-with-modifications.patch | 37 +++
t/solver_git.t | 91 ++++++
t/view.t | 2 +
32 files changed, 1841 insertions(+), 65 deletions(-)
create mode 100644 contrib/css/216dark.css
create mode 100644 contrib/css/216light.css
create mode 100644 contrib/css/README
create mode 100644 examples/highlight.psgi
create mode 100644 lib/PublicInbox/HlMod.pm
create mode 100644 lib/PublicInbox/SolverGit.pm
create mode 100644 lib/PublicInbox/UserContent.pm
create mode 100644 lib/PublicInbox/ViewDiff.pm
create mode 100644 lib/PublicInbox/ViewVCS.pm
create mode 100644 lib/PublicInbox/WwwHighlight.pm
create mode 100644 t/hl_mod.t
create mode 100644 t/perf-msgview.t
create mode 100644 t/solve/0001-simple-mod.patch
create mode 100644 t/solve/0002-rename-with-modifications.patch
create mode 100644 t/solver_git.t
Eric Wong (37):
view: disable bold in topic display
hval: force monospace for <form> elements, too
t/perf-msgview: add test to check msg_html performance
solver: initial Perl implementation
git: support multiple URL endpoints
git: add git_quote
git: check saves error on disambiguation
solver: various bugfixes and cleanups
view: wire up diff and vcs viewers with solver
git: disable abbreviations with cat-file hints
solver: operate directly on git index
view: enable naming hints for raw blob downloads
git: support 'ambiguous' result from --batch-check
solver: more verbose blob resolution
solver: break up patch application steps
solver: switch patch application to use a callback
solver: simplify control flow for initial loop
solver: break @todo loop into a callback
solver: note the synchronous nature of index preparation
solver: add a TODO note about making this fully evented
view: enforce trailing slash for /$INBOX/$OID/s/ endpoints
solver: restore diagnostics and deal with CRLF
www: admin-configurable CSS via "publicinbox.css"
$INBOX/_/text/color/ and sample user-side CSS
viewdiff: support diff-highlighting w/o coderepo
viewdiff: cleanup state transitions a bit
viewdiff: quote attributes for Atom feed
t/check-www-inbox: use xmlstarlet to validate Atom if available
viewdiff: do not link to 0{7,40} blobs (again)
viewvcs: disable white-space prewrap in blob view
solver: force quoted-printable bodies to LF
solver: remove extra "^index $OID..$OID" line
config: each_inbox iteration preserves config order
t/check-www-inbox: warn on missing Content-Type
highlight: initial wrapper and PSGI service
hval: split out escape sequences to a separate table
t/check-www-inbox: trap SIGINT for File::Temp destruction
Documentation/design_www.txt | 6 +-
MANIFEST | 15 +
Makefile.PL | 3 +
TODO | 2 -
contrib/css/216dark.css | 26 ++
contrib/css/216light.css | 25 +
contrib/css/README | 41 ++
examples/highlight.psgi | 13 +
examples/public-inbox.psgi | 2 +-
lib/PublicInbox/Config.pm | 96 +++-
lib/PublicInbox/Git.pm | 87 +++-
lib/PublicInbox/HlMod.pm | 126 +++++
lib/PublicInbox/Hval.pm | 38 +-
lib/PublicInbox/SolverGit.pm | 454 +++++++++++++++++++
lib/PublicInbox/UserContent.pm | 78 ++++
lib/PublicInbox/View.pm | 51 ++-
lib/PublicInbox/ViewDiff.pm | 161 +++++++
lib/PublicInbox/ViewVCS.pm | 110 +++++
lib/PublicInbox/WWW.pm | 152 ++++++-
lib/PublicInbox/WwwHighlight.pm | 73 +++
lib/PublicInbox/WwwStream.pm | 4 +-
lib/PublicInbox/WwwText.pm | 35 ++
script/public-inbox-httpd | 2 +-
t/check-www-inbox.perl | 26 +-
t/config.t | 19 +
t/git.t | 7 +-
t/hl_mod.t | 54 +++
t/perf-msgview.t | 50 ++
t/solve/0001-simple-mod.patch | 20 +
t/solve/0002-rename-with-modifications.patch | 37 ++
t/solver_git.t | 91 ++++
t/view.t | 2 +
32 files changed, 1841 insertions(+), 65 deletions(-)
create mode 100644 contrib/css/216dark.css
create mode 100644 contrib/css/216light.css
create mode 100644 contrib/css/README
create mode 100644 examples/highlight.psgi
create mode 100644 lib/PublicInbox/HlMod.pm
create mode 100644 lib/PublicInbox/SolverGit.pm
create mode 100644 lib/PublicInbox/UserContent.pm
create mode 100644 lib/PublicInbox/ViewDiff.pm
create mode 100644 lib/PublicInbox/ViewVCS.pm
create mode 100644 lib/PublicInbox/WwwHighlight.pm
create mode 100644 t/hl_mod.t
create mode 100644 t/perf-msgview.t
create mode 100644 t/solve/0001-simple-mod.patch
create mode 100644 t/solve/0002-rename-with-modifications.patch
create mode 100644 t/solver_git.t
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-01-21 20:52 6% [PATCH 00/37] viewvcs: diff highlighting and more Eric Wong
2019-01-21 20:52 7% ` [PATCH 03/37] t/perf-msgview: add test to check msg_html performance Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).