user/dev discussion of public-inbox itself
 help / Atom feed
* [RFC] search: use boolean prefixes for git blob queries
@ 2018-07-16  4:07 Eric Wong
  2018-07-20  6:16 ` [PATCH v2] " Eric Wong
  0 siblings, 1 reply; 2+ messages in thread
From: Eric Wong @ 2018-07-16  4:07 UTC (permalink / raw)
  To: meta

I've hit some case where probabilistic searches don't seem to
work when using dfpre:/dfpost:/dfblob: search prefixes.  I'm
not sure why...

In any case, our indexing code indexes longer/unabbreviated blob
names down to its 7 character abbreviation, so there should be
no need to do wildcard searches.
---
 lib/PublicInbox/Search.pm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index 69eca9f..090d998 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -50,6 +50,9 @@ use constant {
 
 my %bool_pfx_external = (
 	mid => 'Q', # Message-ID (full/exact), this is mostly uniQue
+	dfpre => 'XDFPRE',
+	dfpost => 'XDFPOST',
+	dfblob => 'XDFPRE XDFPOST',
 );
 
 my $non_quoted_body = 'XNQ XDFN XDFA XDFB XDFHH XDFCTX XDFPRE XDFPOST';
@@ -74,9 +77,6 @@ my %prob_prefix = (
 	dfb => 'XDFB',
 	dfhh => 'XDFHH',
 	dfctx => 'XDFCTX',
-	dfpre => 'XDFPRE',
-	dfpost => 'XDFPOST',
-	dfblob => 'XDFPRE XDFPOST',
 
 	# default:
 	'' => 'XM S A XQUOT XFN ' . $non_quoted_body,
@@ -266,7 +266,7 @@ sub qp {
 		Search::Xapian::NumberValueRangeProcessor->new(DT, 'dt:'));
 
 	while (my ($name, $prefix) = each %bool_pfx_external) {
-		$qp->add_boolean_prefix($name, $prefix);
+		$qp->add_boolean_prefix($name, $_) foreach split(/ /, $prefix);
 	}
 
 	# we do not actually create AltId objects,
-- 
EW


^ permalink raw reply	[flat|nested] 2+ messages in thread

* [PATCH v2] search: use boolean prefixes for git blob queries
  2018-07-16  4:07 [RFC] search: use boolean prefixes for git blob queries Eric Wong
@ 2018-07-20  6:16 ` " Eric Wong
  0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2018-07-20  6:16 UTC (permalink / raw)
  To: meta

I've hit some case where probabilistic searches don't work when
using dfpre:/dfpost:/dfblob: search prefixes because stemming in
the query parser interferes.

In any case, our indexing code indexes longer/unabbreviated blob
names down to its 7 character abbreviation, so there should be
no need to do wildcard searches on git blob names.
---
 lib/PublicInbox/Search.pm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index 69eca9f..090d998 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -50,6 +50,9 @@ use constant {
 
 my %bool_pfx_external = (
 	mid => 'Q', # Message-ID (full/exact), this is mostly uniQue
+	dfpre => 'XDFPRE',
+	dfpost => 'XDFPOST',
+	dfblob => 'XDFPRE XDFPOST',
 );
 
 my $non_quoted_body = 'XNQ XDFN XDFA XDFB XDFHH XDFCTX XDFPRE XDFPOST';
@@ -74,9 +77,6 @@ my %prob_prefix = (
 	dfb => 'XDFB',
 	dfhh => 'XDFHH',
 	dfctx => 'XDFCTX',
-	dfpre => 'XDFPRE',
-	dfpost => 'XDFPOST',
-	dfblob => 'XDFPRE XDFPOST',
 
 	# default:
 	'' => 'XM S A XQUOT XFN ' . $non_quoted_body,
@@ -266,7 +266,7 @@ sub qp {
 		Search::Xapian::NumberValueRangeProcessor->new(DT, 'dt:'));
 
 	while (my ($name, $prefix) = each %bool_pfx_external) {
-		$qp->add_boolean_prefix($name, $prefix);
+		$qp->add_boolean_prefix($name, $_) foreach split(/ /, $prefix);
 	}
 
 	# we do not actually create AltId objects,
-- 
EW

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, back to index

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-16  4:07 [RFC] search: use boolean prefixes for git blob queries Eric Wong
2018-07-20  6:16 ` [PATCH v2] " Eric Wong

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.org/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox