From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 35ECC1F4B4 for ; Sat, 19 Sep 2020 09:37:14 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/7] gcf2: libgit2-based cat-file alternative Date: Sat, 19 Sep 2020 09:37:07 +0000 Message-Id: <20200919093714.21776-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: This allows a single cat-file-like process to handle multiple inboxes; instead of having a "git cat-file --batch" process for every inbox; saving pipes and process table space. The preliminary code was done months ago, but I struggled to put all the pieces together in a coherent way. My brain has been scattered :x I finally decided to make the gcf2 process a global singleton (per-worker) to avoid complexity elsewhere in the config... It doesn't detect or release unlinked packs + indices, yet, so "git gc" may not free disk space until restarted. Otherwise it does detect new epochs and seems mostly working otherwise... Eric Wong (7): gcf2: libgit2-based git cat-file alternative t/gcf2: test changes to alternates add gcf2 client and executable script gcf2: transparently retry on missing OID gcf2*: more descriptive package descriptions gcf2: require git dir with OID gcf2: wire up read-only daemons and rm -gcf2 script MANIFEST | 5 + lib/PublicInbox/Daemon.pm | 11 +++ lib/PublicInbox/Gcf2.pm | 89 ++++++++++++++++++ lib/PublicInbox/Gcf2Client.pm | 62 +++++++++++++ lib/PublicInbox/Git.pm | 41 ++++++--- lib/PublicInbox/GitAsyncCat.pm | 73 +++++++++++++-- lib/PublicInbox/IMAP.pm | 2 +- lib/PublicInbox/gcf2_libgit2.h | 142 +++++++++++++++++++++++++++++ script/public-inbox-httpd | 1 + t/gcf2.t | 162 +++++++++++++++++++++++++++++++++ t/gcf2_client.t | 90 ++++++++++++++++++ 11 files changed, 652 insertions(+), 26 deletions(-) create mode 100644 lib/PublicInbox/Gcf2.pm create mode 100644 lib/PublicInbox/Gcf2Client.pm create mode 100644 lib/PublicInbox/gcf2_libgit2.h create mode 100644 t/gcf2.t create mode 100644 t/gcf2_client.t