From: "Eric Wong (Contractor, The Linux Foundation)" <e@80x24.org>
To: meta@public-inbox.org
Cc: "Eric Wong (Contractor, The Linux Foundation)" <e@80x24.org>
Subject: [PATCH 00/14] purging support, v1 conversions, cleanups + more
Date: Thu, 29 Mar 2018 10:28:05 +0000 [thread overview]
Message-ID: <20180329102819.15234-1-e@80x24.org> (raw)
There's now a public-inbox-convert tool which makes a new v2 repo
from a v1 git repo via git-fast-export/import. Using those tools,
purge support also works internally, but there's no command-line
support for purging, yet.
public-inbox-watch restart scanning can still cause
removed/purged messages to reappear in the repository. This
wasn't a problem with my IMAP setup since deletes happen in my
Maildir/IMAP folder first (triggering inotify wakeup in -watch)
but that may not always be the case for other deployments.
Eric Wong (Contractor, The Linux Foundation) (14):
www: remove unnecessary ghost checks
v2writable: append, instead of prepending generated Message-ID
lookup by Message-ID favors the "primary" one
www: fix attachment downloads for conflicted Message-IDs
searchmsg: document why we store To: and Cc: for NNTP
public-inbox-convert: tool for converting old to new inboxes
v2writable: support purging messages from git entirely
search: cleanup uniqueness checking
search: get rid of most lookup_* subroutines
search: move find_doc_ids to searchidx
v2writable: cleanup: get rid of unused fields
mbox: avoid extracting Message-ID for linkification
www: cleanup expensive fallback for legacy URLs
view: get rid of some unnecessary imports
Documentation/public-inbox-config.pod | 2 +-
Documentation/public-inbox-convert.pod | 45 ++++++++++
MANIFEST | 2 +
lib/PublicInbox/Import.pm | 98 +++++++++++++++++++++-
lib/PublicInbox/Inbox.pm | 31 ++++---
lib/PublicInbox/Mbox.pm | 9 +-
lib/PublicInbox/Search.pm | 88 +++-----------------
lib/PublicInbox/SearchIdx.pm | 8 ++
lib/PublicInbox/SearchMsg.pm | 4 +
lib/PublicInbox/SearchThread.pm | 14 ++--
lib/PublicInbox/SearchView.pm | 2 +-
lib/PublicInbox/V2Writable.pm | 43 ++++++++--
lib/PublicInbox/View.pm | 31 +++----
lib/PublicInbox/WWW.pm | 23 ++----
script/public-inbox-convert | 109 +++++++++++++++++++++++++
t/plack.t | 18 ++++
t/psgi_v2.t | 45 +++++++++-
t/search-thr-index.t | 3 +-
t/search.t | 6 +-
t/v2writable.t | 15 +++-
20 files changed, 443 insertions(+), 153 deletions(-)
create mode 100644 Documentation/public-inbox-convert.pod
create mode 100755 script/public-inbox-convert
--
EW
next reply other threads:[~2018-03-29 10:28 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-29 10:28 Eric Wong (Contractor, The Linux Foundation) [this message]
2018-03-29 10:28 ` [PATCH 01/14] www: remove unnecessary ghost checks Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 02/14] v2writable: append, instead of prepending generated Message-ID Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 03/14] lookup by Message-ID favors the "primary" one Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 04/14] www: fix attachment downloads for conflicted Message-IDs Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 05/14] searchmsg: document why we store To: and Cc: for NNTP Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 06/14] public-inbox-convert: tool for converting old to new inboxes Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 07/14] v2writable: support purging messages from git entirely Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 08/14] search: cleanup uniqueness checking Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 09/14] search: get rid of most lookup_* subroutines Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 10/14] search: move find_doc_ids to searchidx Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 11/14] v2writable: cleanup: get rid of unused fields Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 12/14] mbox: avoid extracting Message-ID for linkification Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 13/14] www: cleanup expensive fallback for legacy URLs Eric Wong (Contractor, The Linux Foundation)
2018-03-29 10:28 ` [PATCH 14/14] view: get rid of some unnecessary imports Eric Wong (Contractor, The Linux Foundation)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://public-inbox.org/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180329102819.15234-1-e@80x24.org \
--to=e@80x24.org \
--cc=meta@public-inbox.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).