user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Eric Wong <>
Subject: [PATCH 00/10] extindex: another round of updates
Date: Sat,  7 Nov 2020 10:56:50 +0000	[thread overview]
Message-ID: <> (raw)

A major user-visible change is renaming -eindex to -extindex,
because rhyming with "reindex" is probably confusing (and I'm
easily confused :x).

PATCH 10/10 finally starts making sense performance-wise, still
testing...  I've long thought the default 1m batch-size is too
small for 64-bit machines, so maybe that'll change, too.
But it took me way too long to figure out why indexBatchSize was
seemed to have no effect in my PI_CONFIG :<

My Internet access has been terribly unreliable, lately, too; so
relying on mosh/ssh access to work on more powerful machines
aint too pleasant.

Overall extindex it seems to be working somewhat OK for
incremental updates the past few weeks, but could still benefit
from speedups to work better on HW I have locally.

Will have to retest SQLite cache_size and mmap_size pragmas, too.

Eric Wong (10):
  extsearch: rename -eindex to -extindex
  extsearchidx: avoid needless alternates rewrite in ALL.git
  searchidxshard: reduce syscalls when writing ->eidx_key
  searchidxshard: further improve {current_info} readability
  v2writable: less expensive checkpoint for extindex
  extsearchidx: quiet warning for unindexed `d' messages
  extsearch: canonicalize topdir
  v2writable: more accurate {current_info} warnings/progress
  extindex: SIGUSR1 supports checkpoint
  extindex: fix --batch-size support

 MANIFEST                                      |  2 +-
 lib/PublicInbox/                     |  2 +-
 lib/PublicInbox/                  |  2 +
 lib/PublicInbox/               | 37 +++++++++++++------
 lib/PublicInbox/             |  9 ++---
 lib/PublicInbox/                 | 37 ++++++++++++++-----
 ...lic-inbox-eindex => public-inbox-extindex} |  8 +++-
 t/extsearch.t                                 |  6 +--
 8 files changed, 68 insertions(+), 35 deletions(-)
 rename script/{public-inbox-eindex => public-inbox-extindex} (84%)

             reply	other threads:[~2020-11-07 10:57 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-07 10:56 Eric Wong [this message]
2020-11-07 10:56 ` [PATCH 01/10] extsearch: rename -eindex to -extindex Eric Wong
2020-11-07 10:56 ` [PATCH 02/10] extsearchidx: avoid needless alternates rewrite in ALL.git Eric Wong
2020-11-07 10:56 ` [PATCH 03/10] searchidxshard: reduce syscalls when writing ->eidx_key Eric Wong
2020-11-07 10:56 ` [PATCH 04/10] searchidxshard: further improve {current_info} readability Eric Wong
2020-11-07 10:56 ` [PATCH 05/10] v2writable: less expensive checkpoint for extindex Eric Wong
2020-11-07 10:56 ` [PATCH 06/10] extsearchidx: quiet warning for unindexed `d' messages Eric Wong
2020-11-07 10:56 ` [PATCH 07/10] extsearch: canonicalize topdir Eric Wong
2020-11-07 10:56 ` [PATCH 08/10] v2writable: more accurate {current_info} warnings/progress Eric Wong
2020-11-07 10:56 ` [PATCH 09/10] extindex: SIGUSR1 supports checkpoint Eric Wong
2020-11-07 10:57 ` [PATCH 10/10] extindex: fix --batch-size support Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \
    --subject='Re: [PATCH 00/10] extindex: another round of updates' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).