user/dev discussion of public-inbox itself
 help / color / Atom feed
From: Eric Wong <e@80x24.org>
To: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: meta@public-inbox.org
Subject: [PATCH] tests: fixup indexlevel setting in tests
Date: Thu, 19 Jul 2018 21:51:14 +0000
Message-ID: <20180719215114.zdzysw77vfzgdyfz@whir> (raw)
In-Reply-To: <20180718165325.19834-3-ebiederm@xmission.com>

"Eric W. Biederman" <ebiederm@xmission.com> wrote:
> +$ibx_config->{index_level} = 'medium';

Oops, I missed that underscore in my initial review.
(and I dislike the lack of '_' in git-config naming conventions, even).

----------8<--------
Subject: [PATCH] tests: fixup indexlevel setting in tests

The correct field is underscore-less for consistency with
git-config naming conventions.  While we're at it, beef up
the v2 tests with actual size checks, too.

I also noticed phrase searching still seems to work for
the limited test case, so I left it documented; but the
size checking verifies the space savings.
---
 t/v1reindex.t |  4 ++--
 t/v2reindex.t | 26 +++++++++++++++++++++++---
 2 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/t/v1reindex.t b/t/v1reindex.t
index ff32750..d97938d 100644
--- a/t/v1reindex.t
+++ b/t/v1reindex.t
@@ -111,7 +111,7 @@ ok(unlink "$mainrepo/public-inbox/msgmap.sqlite3", 'remove msgmap');
 remove_tree($xap);
 ok(!-d $xap, 'Xapian directories removed again');
 
-$ibx_config->{index_level} = 'medium';
+$ibx_config->{indexlevel} = 'medium';
 $ibx = PublicInbox::Inbox->new($ibx_config);
 $rw = PublicInbox::SearchIdx->new($ibx, 1);
 {
@@ -131,7 +131,7 @@ ok(unlink "$mainrepo/public-inbox/msgmap.sqlite3", 'remove msgmap');
 remove_tree($xap);
 ok(!-d $xap, 'Xapian directories removed again');
 
-$ibx_config->{index_level} = 'basic';
+$ibx_config->{indexlevel} = 'basic';
 $ibx = PublicInbox::Inbox->new($ibx_config);
 $rw = PublicInbox::SearchIdx->new($ibx, 1);
 {
diff --git a/t/v2reindex.t b/t/v2reindex.t
index 2090396..1543309 100644
--- a/t/v2reindex.t
+++ b/t/v2reindex.t
@@ -81,6 +81,7 @@ ok(!-d $xap, 'Xapian directories removed again');
 	is_deeply([ $ibx->mm->minmax ], $minmax, 'minmax unchanged');
 }
 
+my %sizes;
 ok(unlink "$mainrepo/msgmap.sqlite3", 'remove msgmap');
 remove_tree($xap);
 ok(!-d $xap, 'Xapian directories removed again');
@@ -94,13 +95,16 @@ ok(!-d $xap, 'Xapian directories removed again');
 	ok(-d $xap, 'Xapian directories recreated');
 	delete $ibx->{mm};
 	is_deeply([ $ibx->mm->minmax ], $minmax, 'minmax unchanged');
+	my $mset = $ibx->search->query('"hello world"', {mset=>1});
+	isnt(0, $mset->size, "phrase search succeeds on indexlevel=full");
+	for (<"$xap/*/*">) { $sizes{$ibx->{indexlevel}} += -s _ if -f $_ }
 }
 
 ok(unlink "$mainrepo/msgmap.sqlite3", 'remove msgmap');
 remove_tree($xap);
 ok(!-d $xap, 'Xapian directories removed again');
 
-$ibx_config->{index_level} = 'medium';
+$ibx_config->{indexlevel} = 'medium';
 $ibx = PublicInbox::Inbox->new($ibx_config);
 $im = PublicInbox::V2Writable->new($ibx);
 {
@@ -113,14 +117,26 @@ $im = PublicInbox::V2Writable->new($ibx);
 	ok(-d $xap, 'Xapian directories recreated');
 	delete $ibx->{mm};
 	is_deeply([ $ibx->mm->minmax ], $minmax, 'minmax unchanged');
-}
 
+	if (0) {
+		# not sure why, but Xapian seems to fallback to terms and
+		# phrase searches still work
+		delete $ibx->{search};
+		my $mset = $ibx->search->query('"hello world"', {mset=>1});
+		is(0, $mset->size, 'phrase search does not work on medium');
+	}
+
+	my $mset = $ibx->search->query('hello world', {mset=>1});
+	isnt(0, $mset->size, "normal search works on indexlevel=medium");
+	for (<"$xap/*/*">) { $sizes{$ibx->{indexlevel}} += -s _ if -f $_ }
+	ok($sizes{full} > $sizes{medium}, 'medium is smaller than full');
+}
 
 ok(unlink "$mainrepo/msgmap.sqlite3", 'remove msgmap');
 remove_tree($xap);
 ok(!-d $xap, 'Xapian directories removed again');
 
-$ibx_config->{index_level} = 'basic';
+$ibx_config->{indexlevel} = 'basic';
 $ibx = PublicInbox::Inbox->new($ibx_config);
 $im = PublicInbox::V2Writable->new($ibx);
 {
@@ -133,6 +149,10 @@ $im = PublicInbox::V2Writable->new($ibx);
 	ok(-d $xap, 'Xapian directories recreated');
 	delete $ibx->{mm};
 	is_deeply([ $ibx->mm->minmax ], $minmax, 'minmax unchanged');
+	my $mset = $ibx->search->query('hello', {mset=>1});
+	is(0, $mset->size, "search fails on indexlevel='basic'");
+	for (<"$xap/*/*">) { $sizes{$ibx->{indexlevel}} += -s _ if -f $_ }
+	ok($sizes{medium} > $sizes{basic}, 'basic is smaller than medium');
 }
 
 done_testing();
-- 
EW

  reply index

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-17 23:27 [PATCH 0/3] Making the search indexes optional ebiederm
2018-07-17 23:30 ` [PATCH 1/3] SearchIdx.pm: Make indexing search positions optional Eric W. Biederman
2018-07-17 23:30 ` [PATCH 2/3] SearchIdx: Add the mechanism for making all Xapian indexing optional Eric W. Biederman
2018-07-17 23:30 ` [PATCH 3/3] SearchIdx: Allow the amount of indexing be configured Eric W. Biederman
2018-07-18 10:22   ` Eric Wong
2018-07-18 16:00     ` ebiederm
2018-07-18 16:31       ` Eric Wong
2018-07-18 16:52         ` [PATCH v2 1/3] Making the search indexes optional ebiederm
2018-07-18 16:53           ` [PATCH v2 1/3] SearchIdx.pm: Make indexing search positions optional Eric W. Biederman
2018-07-18 16:53           ` [PATCH v2 2/3] SearchIdx: Add the mechanism for making all Xapian indexing optional Eric W. Biederman
2018-07-18 16:53           ` [PATCH v2 3/3] SearchIdx: Allow the amount of indexing be configured Eric W. Biederman
2018-07-19 21:51             ` Eric Wong [this message]
2018-07-18 17:32           ` [PATCH v2 3/4] public-inbox-init: Initialize indexlevel ebiederm
2018-07-19  3:52           ` [PATCH v2 1/3] Making the search indexes optional Eric Wong
2018-07-19 18:47             ` ebiederm
2018-07-20  6:58               ` [PATCH] v1: allow upgrading indexlevel=basic to 'medium' or 'full' Eric Wong
2018-07-18 10:17 ` [PATCH 0/3] Making the search indexes optional Eric Wong

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180719215114.zdzysw77vfzgdyfz@whir \
    --to=e@80x24.org \
    --cc=ebiederm@xmission.com \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.org/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox