From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 31AB11F487 for ; Sat, 28 Mar 2020 00:56:04 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/2] index: support --compact / -c Date: Sat, 28 Mar 2020 00:56:02 +0000 Message-Id: <20200328005604.17573-1-e@yhbt.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: It looks like HDDs and SSDs have gotten and will get even more expensive due to manufacturing freezes from the pandemic. Indexing (especially with --reindex to fixup old bugs) takes a large amount of space, so support running compact immediately after indexing to avoid users having to script a -compact invocation for each inbox. Compacting before indexing can be triggered by using this switch twice, to further reduce space overhead at a small time loss. Note: I only found the bug fixed in 1/2 while testing 2/2. It took me a while to fix this bug because I've probably lost 10 IQ points from the stress of recent weeks :< Eric Wong (2): searchidxshard: ensure we set indexlevel on shard[0] index: support --compact / -c on command-line Documentation/public-inbox-index.pod | 24 ++++++++++++++++++++---- lib/PublicInbox/InboxWritable.pm | 1 + lib/PublicInbox/SearchIdx.pm | 26 +++++++++++++++++--------- lib/PublicInbox/SearchIdxShard.pm | 4 +++- lib/PublicInbox/Xapcmd.pm | 4 +++- script/public-inbox-index | 20 +++++++++++++++++--- t/convert-compact.t | 13 +++++++++++++ t/init.t | 7 ++++++- 8 files changed, 80 insertions(+), 19 deletions(-)