* [PATCH 26/35] t/lei_xsearch: use create_inbox
2021-03-15 11:57 7% [PATCH 00/35] memoize inbox creations Eric Wong
2021-03-15 11:57 4% ` [PATCH 02/35] t/lei_xsearch: use create_inbox Eric Wong
@ 2021-03-15 11:58 5% ` Eric Wong
1 sibling, 0 replies; 3+ results
From: Eric Wong @ 2021-03-15 11:58 UTC (permalink / raw)
To: meta
This ends up being significantly faster when confined
to a slow TMPDIR.
---
t/lei_xsearch.t | 27 ++++++---------------------
1 file changed, 6 insertions(+), 21 deletions(-)
diff --git a/t/lei_xsearch.t b/t/lei_xsearch.t
index a1ab3ec8..f626c790 100644
--- a/t/lei_xsearch.t
+++ b/t/lei_xsearch.t
@@ -6,7 +6,6 @@ use v5.10.1;
use List::Util qw(shuffle max);
use PublicInbox::TestCommon;
use PublicInbox::Eml;
-use PublicInbox::InboxWritable;
require_mods(qw(DBD::SQLite Search::Xapian));
require PublicInbox::ExtSearchIdx;
require_git 2.6;
@@ -73,26 +72,12 @@ is($lxs->over, undef, '->over fails');
{
$lxs = PublicInbox::LeiXSearch->new;
- my $v2ibx = PublicInbox::InboxWritable->new({
- inboxdir => "$home/v2full",
- name => 'v2full',
- version => 2,
- indexlevel => 'full',
- -primary_address => 'v2full@example.com',
- }, {});
- my $im = $v2ibx->importer(0);
- $im->add(eml_load('t/plack-qp.eml'));
- $im->done;
- my $v1ibx = PublicInbox::InboxWritable->new({
- inboxdir => "$home/v1medium",
- name => 'v1medium',
- version => 1,
- indexlevel => 'medium',
- -primary_address => 'v1medium@example.com',
- }, {});
- $im = $v1ibx->importer(0);
- $im->add(eml_load('t/utf8.eml'));
- $im->done;
+ my $v2ibx = create_inbox 'v2full', version => 2, sub {
+ $_[0]->add(eml_load('t/plack-qp.eml'));
+ };
+ my $v1ibx = create_inbox 'v1medium', indexlevel => 'medium', sub {
+ $_[0]->add(eml_load('t/utf8.eml'));
+ };
$lxs->prepare_external($v1ibx);
$lxs->prepare_external($v2ibx);
for my $loc ($lxs->locals) {
^ permalink raw reply related [relevance 5%]
* [PATCH 00/35] memoize inbox creations
@ 2021-03-15 11:57 7% Eric Wong
2021-03-15 11:57 4% ` [PATCH 02/35] t/lei_xsearch: use create_inbox Eric Wong
2021-03-15 11:58 5% ` [PATCH 26/35] " Eric Wong
0 siblings, 2 replies; 3+ results
From: Eric Wong @ 2021-03-15 11:57 UTC (permalink / raw)
To: meta
This speeds up a bunch of tests by a few dozen to hundreds of
milliseconds. Part of it is due to disabling fsync/fdatasync in
a few more places.
Users with slower TMPDIR will notice more, and users with TMPDIR
pointed to tmpfs will notice it least.
Overall it's not hugely significant, but the diffstat
and LoC reduction still makes it nice even if there were
no performance improvements.
Eric Wong (35):
test_common: add create_inbox helper sub
t/lei_xsearch: use create_inbox
test_common: minor simplifications to setup_public_inboxes
t/imapd-tls: switch to create_inbox
t/www_altid: use create_inbox
t/xcpdb-reshard: use create_inbox
t/v2dupindex: create_inbox
t/admin: switch to create_inbox
t/html_index: remove now-worthless test
t/plack: use create_inbox
t/psgi_attach: convert to create_inbox
t/httpd: convert to create_inbox
t/convert-compact: create_inbox
t/altid: use create_inbox
t/psgi_mount: switch to create_inbox
t/feed: switch to create_inbox
t/psgi_bad_mids: use create_inbox
t/psgi_multipart_not: use create_inbox
t/psgi_scan_all: create_inbox lots saved
t/psgi_v2: create_inbox
t/imapd: create_inbox (minor)
t/solver_git: use create_inbox
t/cgi: create_inbox
t/edit: switch to create_inbox
t/inbox_idle: switch to create_inbox
t/lei_xsearch: use create_inbox
t/indexlevels-mirror: use create_inbox
t/multi-mid: use create_inbox
t/psgi_search: use create_inbox
t/miscsearch: use create_inbox
t/nntpd-tls: use create_inbox
t/nntpd: use create_inbox
t/thread-index-gap: create_inbox
t/altid_v2: create_inbox
t/*: disable fsync on tests were create_inbox isn't worth it
MANIFEST | 2 +-
Makefile.PL | 3 +-
lib/PublicInbox/TestCommon.pm | 67 ++++++++++++++---
t/admin.t | 38 ++++------
t/altid.t | 26 ++-----
t/altid_v2.t | 48 +++++-------
t/cgi.t | 84 +++++++++------------
t/convert-compact.t | 48 +++++-------
t/data-gen/.gitignore | 2 +
t/edit.t | 34 ++++-----
t/extsearch.t | 2 +
t/feed.t | 53 ++++----------
t/html_index.t | 56 --------------
t/httpd.t | 36 +++------
t/imapd-tls.t | 38 +++-------
t/imapd.t | 66 ++++++++---------
t/inbox_idle.t | 31 ++++----
t/indexlevels-mirror.t | 24 +++---
t/lei-q-remote-import.t | 20 ++---
t/lei_xsearch.t | 56 +++++---------
t/miscsearch.t | 28 +++----
t/multi-mid.t | 34 +++------
t/nntpd-tls.t | 52 ++++---------
t/nntpd.t | 102 ++++++++++++--------------
t/plack.t | 69 +++++++----------
t/psgi_attach.t | 56 ++++++--------
t/psgi_bad_mids.t | 46 ++++--------
t/psgi_mount.t | 35 ++++-----
t/psgi_multipart_not.t | 35 +++------
t/psgi_scan_all.t | 48 ++++--------
t/psgi_search.t | 37 +++-------
t/psgi_v2.t | 134 ++++++++++++++++------------------
t/purge.t | 1 +
t/replace.t | 1 +
t/solver_git.t | 88 +++++++++++-----------
t/thread-index-gap.t | 24 +++---
t/v1reindex.t | 1 +
t/v2-add-remove-add.t | 1 +
t/v2dupindex.t | 76 +++++++++----------
t/v2mirror.t | 1 +
t/v2reindex.t | 1 +
t/v2writable.t | 1 +
t/watch_filter_rubylang.t | 2 +
t/watch_maildir_v2.t | 1 +
t/www_altid.t | 47 +++++-------
t/xcpdb-reshard.t | 54 ++++++--------
46 files changed, 692 insertions(+), 1017 deletions(-)
create mode 100644 t/data-gen/.gitignore
delete mode 100644 t/html_index.t
^ permalink raw reply [relevance 7%]
* [PATCH 02/35] t/lei_xsearch: use create_inbox
2021-03-15 11:57 7% [PATCH 00/35] memoize inbox creations Eric Wong
@ 2021-03-15 11:57 4% ` Eric Wong
2021-03-15 11:58 5% ` [PATCH 26/35] " Eric Wong
1 sibling, 0 replies; 3+ results
From: Eric Wong @ 2021-03-15 11:57 UTC (permalink / raw)
To: meta
Less code and noticeably faster when TMPDIR is on an SSD
---
t/lei_xsearch.t | 29 +++++++++++------------------
1 file changed, 11 insertions(+), 18 deletions(-)
diff --git a/t/lei_xsearch.t b/t/lei_xsearch.t
index 5bfbcfe6..a1ab3ec8 100644
--- a/t/lei_xsearch.t
+++ b/t/lei_xsearch.t
@@ -3,7 +3,6 @@
# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
use strict;
use v5.10.1;
-use Test::More;
use List::Util qw(shuffle max);
use PublicInbox::TestCommon;
use PublicInbox::Eml;
@@ -16,17 +15,11 @@ my ($home, $for_destroy) = tmpdir();
my @ibx;
for my $V (1..2) {
for my $i (3..6) {
- my $ibx = PublicInbox::InboxWritable->new({
- inboxdir => "$home/v$V-$i",
- name => "test-v$V-$i",
- version => $V,
- indexlevel => 'medium',
- -primary_address => "v$V-$i\@example.com",
- }, { nproc => int(rand(8)) + 1 });
- push @ibx, $ibx;
- my $im = $ibx->importer(0);
- for my $j (0..9) {
- my $eml = PublicInbox::Eml->new(<<EOF);
+ push @ibx, create_inbox("v$V-$i", indexlevel => 'full',
+ version => $V, sub {
+ my ($im, $ibx) = @_;
+ for my $j (0..9) {
+ my $eml = PublicInbox::Eml->new(<<EOM);
From: x\@example.com
To: $ibx->{-primary_address}
Date: Fri, 02 Oct 1993 0$V:0$i:0$j +0000
@@ -34,14 +27,14 @@ Subject: v${V}i${i}j$j
Message-ID: <v${V}i${i}j$j\@example>
${V}er ${i}on j$j
-EOF
- $im->add($eml);
- }
- $im->done;
+EOM
+ $im->add($eml) or BAIL_OUT '->add';
+ }
+ }); # create_inbox
}
}
-my $first = shift @ibx; is($first->{name}, 'test-v1-3', 'first plucked');
-my $last = pop @ibx; is($last->{name}, 'test-v2-6', 'last plucked');
+my $first = shift @ibx; is($first->{name}, 'v1-3', 'first plucked');
+my $last = pop @ibx; is($last->{name}, 'v2-6', 'last plucked');
my $eidx = PublicInbox::ExtSearchIdx->new("$home/eidx");
$eidx->attach_inbox($first);
$eidx->attach_inbox($last);
^ permalink raw reply related [relevance 4%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-03-15 11:57 7% [PATCH 00/35] memoize inbox creations Eric Wong
2021-03-15 11:57 4% ` [PATCH 02/35] t/lei_xsearch: use create_inbox Eric Wong
2021-03-15 11:58 5% ` [PATCH 26/35] " Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).