user/dev discussion of public-inbox itself
 help / color / Atom feed
From: Eric Wong <e@yhbt.net>
To: meta@public-inbox.org
Subject: [PATCH 10/11] altid: warn about non-word prefixes
Date: Sat, 21 Mar 2020 02:03:53 +0000
Message-ID: <20200321020354.9056-11-e@yhbt.net> (raw)
In-Reply-To: <20200321020354.9056-1-e@yhbt.net>

We only support searching on prefixes matching /\A\w+\z/ because
Xapian requires ':' to delimit the prefix and splits on spaces
without quotes.

I've also verified Xapian supports multibyte UTF-8 characters,
underscores, and bare numbers as search prefixes, so there's
no need to restrict it beyond what Perl's UTF-8 aware \w
character class offers.
---
 lib/PublicInbox/AltId.pm  | 2 +-
 lib/PublicInbox/Search.pm | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/lib/PublicInbox/AltId.pm b/lib/PublicInbox/AltId.pm
index 8ce70e46..3be6c73c 100644
--- a/lib/PublicInbox/AltId.pm
+++ b/lib/PublicInbox/AltId.pm
@@ -22,7 +22,7 @@ sub new {
 	my ($class, $ibx, $spec, $writable) = @_;
 	my ($type, $prefix, $query) = split(/:/, $spec, 3);
 	$type eq 'serial' or die "non-serial not supported, yet\n";
-
+	$prefix =~ /\A\w+\z/ or warn "non-word prefix not searchable\n";
 	my %params = map {
 		my ($k, $v) = split(/=/, uri_unescape($_), 2);
 		$v = '' unless defined $v;
diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index 372dc5a7..00dddc6b 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -316,6 +316,8 @@ sub qp {
 		my $user_pfx = $self->{-user_pfx} = [];
 		for (@$altid) {
 			# $_ = 'serial:gmane:/path/to/gmane.msgmap.sqlite3'
+			# note: Xapian supports multibyte UTF-8, /^[0-9]+$/,
+			# and '_' with prefixes matching \w+
 			/\Aserial:(\w+):/ or next;
 			my $pfx = $1;
 			push @$user_pfx, "$pfx:", <<EOF;

  parent reply index

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-21  2:03 [PATCH 00/11] www: export SQLite altid dumps Eric Wong
2020-03-21  2:03 ` [PATCH 01/11] qspawn: reinstate filter support, add gzip filter Eric Wong
2020-03-21  2:03 ` [PATCH 02/11] gzipfilter: lazy allocate the deflate context Eric Wong
2020-03-21  2:03 ` [PATCH 03/11] wwwstream: introduce oneshot API to avoid ->getline Eric Wong
2020-03-21  2:03 ` [PATCH 04/11] extmsg: use WwwResponse::oneshot Eric Wong
2020-03-21  2:03 ` [PATCH 05/11] wwwstream: oneshot sets content-length Eric Wong
2020-03-21  2:03 ` [PATCH 06/11] mbox: need_gzip uses WwwStream::oneshot Eric Wong
2020-03-21  2:03 ` [PATCH 07/11] qspawn: handle ENOENT (and other errors on exec) Eric Wong
2020-03-21  2:03 ` [PATCH 08/11] search: clobber -user_pfx on query parser initialization Eric Wong
2020-03-21  2:03 ` [PATCH 09/11] wwwtext: show thread endpoint w/ indexlevel=basic Eric Wong
2020-03-21  2:03 ` Eric Wong [this message]
2020-03-21  2:03 ` [PATCH 11/11] www: add endpoint to retrieve altid dumps Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200321020354.9056-11-e@yhbt.net \
    --to=e@yhbt.net \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.io/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/public-inbox.git