From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 283451F670 for ; Mon, 11 Oct 2021 08:06:20 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/6] extindex: --reindex --fast gets faster Date: Mon, 11 Oct 2021 08:06:14 +0000 Message-Id: <20211011080620.27478-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: -extindex --reindex --fast --all performance is nearly doubled. There's also a bunch of cleanups for more consistently handling with various forms of message removal from an extindex. Eric Wong (6): extindex: speed up --reindex --fast sqlite: PRAGMA optimize on close extindex: rename var: active => active_shards extindex: share unref logic in more places extindex: more consistent doc removal extindex: avoid invalid blobs after unref lib/PublicInbox/ExtSearchIdx.pm | 303 ++++++++++++++++---------------- lib/PublicInbox/LeiMailSync.pm | 3 +- lib/PublicInbox/Over.pm | 4 +- lib/PublicInbox/OverIdx.pm | 54 +----- lib/PublicInbox/V2Writable.pm | 11 ++ t/extsearch.t | 2 +- t/over.t | 5 +- 7 files changed, 173 insertions(+), 209 deletions(-)