user/dev discussion of public-inbox itself
 help / color / Atom feed
From: ebiederm@xmission.com (Eric W. Biederman)
To: Eric Wong <e@80x24.org>
Cc: meta@public-inbox.org
Subject: Re: [PATCH v2 1/3] Making the search indexes optional
Date: Thu, 19 Jul 2018 13:47:01 -0500
Message-ID: <877elrf4gq.fsf@xmission.com> (raw)
In-Reply-To: <20180719035202.qqertfmrxa4zlvmo@whir>

Eric Wong <e@80x24.org> writes:

> "Eric W. Biederman" <ebiederm@xmission.com> wrote:
>> 
>> This is my respin of these patches.  I have used the levels:
>> full, medium, basic.
>> 
>> I think basic conveys the message that it is ok to run with and you can
>> expect most things to work, better than minimal where it feels like
>> you don't know what will fail.
>
> Agreed.
>
>> I have tweaked the reindex tests to run with all 3 different levels
>> so at least these code paths get exercised.
>
> Thanks, all four pushed to 'master' on https://public-inbox.org/
> along with my flush threshold env check.
>
> I think being able to transition from an existing "basic" to
> (full|medium) can be straightforward, too, with the untested
> patch below.
>
> But most other transitions between levels will require an expensive
> --reindex.   (full|medium) => basic would be easy, too, but
> it'll be up to the user to remove the Xapian files.

I would still use "$self->{indexlevel} =~ m/(full|medium)/" instead of
ne 'basic' just in case we start recognizing another level.

Otherwise your change seems to make sense.  I have not really dug into
the reindex case where the point is to pick up things that were
previously missed.  I can definitely see that code should be guarded
with an indexlevel check.

Eric

> diff --git a/lib/PublicInbox/SearchIdx.pm b/lib/PublicInbox/SearchIdx.pm
> index bb60506..620f24d 100644
> --- a/lib/PublicInbox/SearchIdx.pm
> +++ b/lib/PublicInbox/SearchIdx.pm
> @@ -714,7 +714,7 @@ sub _index_sync {
>  			}
>  			$dbh->commit;
>  		}
> -		if ($mkey && $newest) {
> +		if ($mkey && $newest && $self->{indexlevel} ne 'basic') {
>  			my $cur = $xdb->get_metadata($mkey);
>  			if (need_update($self, $cur, $newest)) {
>  				$xdb->set_metadata($mkey, $newest);

  reply index

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-17 23:27 [PATCH 0/3] " ebiederm
2018-07-17 23:30 ` [PATCH 1/3] SearchIdx.pm: Make indexing search positions optional Eric W. Biederman
2018-07-17 23:30 ` [PATCH 2/3] SearchIdx: Add the mechanism for making all Xapian indexing optional Eric W. Biederman
2018-07-17 23:30 ` [PATCH 3/3] SearchIdx: Allow the amount of indexing be configured Eric W. Biederman
2018-07-18 10:22   ` Eric Wong
2018-07-18 16:00     ` ebiederm
2018-07-18 16:31       ` Eric Wong
2018-07-18 16:52         ` [PATCH v2 1/3] Making the search indexes optional ebiederm
2018-07-18 16:53           ` [PATCH v2 1/3] SearchIdx.pm: Make indexing search positions optional Eric W. Biederman
2018-07-18 16:53           ` [PATCH v2 2/3] SearchIdx: Add the mechanism for making all Xapian indexing optional Eric W. Biederman
2018-07-18 16:53           ` [PATCH v2 3/3] SearchIdx: Allow the amount of indexing be configured Eric W. Biederman
2018-07-19 21:51             ` [PATCH] tests: fixup indexlevel setting in tests Eric Wong
2018-07-18 17:32           ` [PATCH v2 3/4] public-inbox-init: Initialize indexlevel ebiederm
2018-07-19  3:52           ` [PATCH v2 1/3] Making the search indexes optional Eric Wong
2018-07-19 18:47             ` ebiederm [this message]
2018-07-20  6:58               ` [PATCH] v1: allow upgrading indexlevel=basic to 'medium' or 'full' Eric Wong
2018-07-18 10:17 ` [PATCH 0/3] Making the search indexes optional Eric Wong

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=877elrf4gq.fsf@xmission.com \
    --to=ebiederm@xmission.com \
    --cc=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

user/dev discussion of public-inbox itself

Archives are clonable:
	git clone --mirror https://public-inbox.org/meta
	git clone --mirror http://czquwvybam4bgbro.onion/meta
	git clone --mirror http://hjrcffqmbrq6wope.onion/meta
	git clone --mirror http://ou63pmih66umazou.onion/meta

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta
	nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta
	nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta
	nntp://news.gmane.org/gmane.mail.public-inbox.general

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox