From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 2E78C1F424; Thu, 29 Mar 2018 10:28:29 +0000 (UTC) From: "Eric Wong (Contractor, The Linux Foundation)" To: meta@public-inbox.org Cc: "Eric Wong (Contractor, The Linux Foundation)" Subject: [PATCH 00/14] purging support, v1 conversions, cleanups + more Date: Thu, 29 Mar 2018 10:28:05 +0000 Message-Id: <20180329102819.15234-1-e@80x24.org> List-Id: There's now a public-inbox-convert tool which makes a new v2 repo from a v1 git repo via git-fast-export/import. Using those tools, purge support also works internally, but there's no command-line support for purging, yet. public-inbox-watch restart scanning can still cause removed/purged messages to reappear in the repository. This wasn't a problem with my IMAP setup since deletes happen in my Maildir/IMAP folder first (triggering inotify wakeup in -watch) but that may not always be the case for other deployments. Eric Wong (Contractor, The Linux Foundation) (14): www: remove unnecessary ghost checks v2writable: append, instead of prepending generated Message-ID lookup by Message-ID favors the "primary" one www: fix attachment downloads for conflicted Message-IDs searchmsg: document why we store To: and Cc: for NNTP public-inbox-convert: tool for converting old to new inboxes v2writable: support purging messages from git entirely search: cleanup uniqueness checking search: get rid of most lookup_* subroutines search: move find_doc_ids to searchidx v2writable: cleanup: get rid of unused fields mbox: avoid extracting Message-ID for linkification www: cleanup expensive fallback for legacy URLs view: get rid of some unnecessary imports Documentation/public-inbox-config.pod | 2 +- Documentation/public-inbox-convert.pod | 45 ++++++++++ MANIFEST | 2 + lib/PublicInbox/Import.pm | 98 +++++++++++++++++++++- lib/PublicInbox/Inbox.pm | 31 ++++--- lib/PublicInbox/Mbox.pm | 9 +- lib/PublicInbox/Search.pm | 88 +++----------------- lib/PublicInbox/SearchIdx.pm | 8 ++ lib/PublicInbox/SearchMsg.pm | 4 + lib/PublicInbox/SearchThread.pm | 14 ++-- lib/PublicInbox/SearchView.pm | 2 +- lib/PublicInbox/V2Writable.pm | 43 ++++++++-- lib/PublicInbox/View.pm | 31 +++---- lib/PublicInbox/WWW.pm | 23 ++---- script/public-inbox-convert | 109 +++++++++++++++++++++++++ t/plack.t | 18 ++++ t/psgi_v2.t | 45 +++++++++- t/search-thr-index.t | 3 +- t/search.t | 6 +- t/v2writable.t | 15 +++- 20 files changed, 443 insertions(+), 153 deletions(-) create mode 100644 Documentation/public-inbox-convert.pod create mode 100755 script/public-inbox-convert -- EW