From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 4D84E1F531 for ; Mon, 10 Aug 2020 02:12:05 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 00/14] more indexing related improvements Date: Mon, 10 Aug 2020 02:11:51 +0000 Message-Id: <20200810021205.18909-1-e@yhbt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: publicInbox.indexSequentialShard now works incrementally -convert also learned all the options -index learned, so it can be less painful on HDDs. Eric Wong (14): index: require --reindex when using --xapian-only index: --sequential-shard works incrementally doc: index: some more notes about latest changes doc: add some notes around -xcpdb / -edit / -purge index+xcpdb: improve SIG{INT,TERM,HUP,PIPE} behavior msgmap: tmp_clone: simplify + meaningful filename avoid File::Temp::tempfile in more places admin: use a generic veriable name index: cleanup internal variables searchidx: use singular `$opt' for consistency with v2 convert: support new -index options convert: speed up --help convert: check ARGV more correctly convert: set No_COW on copied SQLite files Documentation/public-inbox-convert.pod | 19 ++++ Documentation/public-inbox-edit.pod | 14 +++ Documentation/public-inbox-index.pod | 68 +++++++------ Documentation/public-inbox-init.pod | 2 +- Documentation/public-inbox-purge.pod | 14 +++ Documentation/public-inbox-xcpdb.pod | 15 ++- lib/PublicInbox/Admin.pm | 71 ++++++++++++-- lib/PublicInbox/Msgmap.pm | 19 ++-- lib/PublicInbox/SearchIdx.pm | 34 +++---- lib/PublicInbox/V2Writable.pm | 77 ++++++++------- lib/PublicInbox/Xapcmd.pm | 28 ++++-- script/public-inbox-convert | 131 ++++++++++++++++--------- script/public-inbox-index | 69 ++++--------- script/public-inbox-init | 17 ++-- t/import.t | 5 +- 15 files changed, 357 insertions(+), 226 deletions(-)