Merge remote-tracking branch 'origin/edit' into next

* origin/edit:
  edit: unlink temporary file when done
  v2writable: replace: kill git processes before reindexing
  edit: drop unwanted headers before noop check
  edit|purge: improve output on rewrites
  edit: new tool to perform edits
  doc: document the --prune option for -index
  admin: expose ->config
  AdminEdit: move editability checks from -purge
  admin: beef up resolve_inboxes to handle purge options
  purge: start moving common options to AdminEdit module
  admin: remove warning arg for unconfigured inboxes
  v2writable: implement ->replace call
  import: switch to "replace_oids" interface for purge
  import: extract_author_info becomes extract_commit_info
  v2writable: consolidate overview and indexing call

4 files changed, 121 insertions, 0 deletions

diff --git a/Documentation/include.mk b/Documentation/include.mk
index 10bd29f0..8501adc8 100644
--- a/Documentation/include.mk
+++ b/Documentation/include.mk
@@ -33,6 +33,7 @@ podtext = $(PODTEXT) $(PODTEXT_OPTS)
 m1 =
 m1 += public-inbox-compact
 m1 += public-inbox-convert
+m1 += public-inbox-edit
 m1 += public-inbox-httpd
 m1 += public-inbox-index
 m1 += public-inbox-mda
diff --git a/Documentation/public-inbox-config.pod b/Documentation/public-inbox-config.pod
index db81bf1c..a86132be 100644
--- a/Documentation/public-inbox-config.pod
+++ b/Documentation/public-inbox-config.pod
@@ -234,6 +234,10 @@ C<publicinbox.cgitbin>, but may be overridden.
 Default: basename of C<publicinbox.cgitbin>, /var/www/htdocs/cgit/
 or /usr/share/cgit/
 
+=item publicinbox.mailEditor
+
+See L<public-inbox-edit(1)>
+
 =item publicinbox.wwwlisting
 
 Enable a HTML listing style when the root path of the URL '/' is accessed.
diff --git a/Documentation/public-inbox-edit.pod b/Documentation/public-inbox-edit.pod
new file mode 100644
index 00000000..97c7c92a
--- /dev/null
+++ b/Documentation/public-inbox-edit.pod
@@ -0,0 +1,109 @@
+=head1 NAME
+
+public-inbox-edit - edit messages in a public inbox
+
+=head1 SYNOPSIS
+
+        public-inbox-edit -m MESSAGE-ID --all|INBOX_DIR
+
+        public-inbox-edit -F RAW_FILE --all|INBOX_DIR [.. INBOX_DIR]
+
+=head1 DESCRIPTION
+
+public-inbox-edit allows editing messages in a given inbox
+to remove sensitive information.  It is only intended as a
+last resort, as it will cause discontiguous git history and
+draw more attention to the sensitive data in mirrors.
+
+=head1 OPTIONS
+
+=over
+
+=item --all
+
+Edit the message in all inboxes configured in ~/.public-inbox/config.
+This is an alternative to specifying individual inboxes directories
+on the command-line.
+
+=item -m MESSAGE-ID
+
+Edits the message corresponding to the given C<MESSAGE-ID>.
+If the C<MESSAGE-ID> is ambiguous, C<--force> or using the
+C<--file> of the original will be required.
+
+=item -F FILE
+
+Edits the message corresponding to the Message-ID: header
+and content given in C<FILE>.  This requires the unmodified
+raw message, and the contents of C<FILE> will not itself
+be modified.  This is useful if a Message-ID is ambiguous
+due to filtering/munging rules or other edits.
+
+=item --force
+
+Forcibly perform the edit even if Message-ID is ambiguous.
+
+=item --raw
+
+Do not perform "From " line escaping.  By default, this
+generates a mboxrd variant file to detect unpurged messages
+in the new mbox.  This makes sense if your configured
+C<publicinbox.mailEditor> is a regular editor and not
+something like C<mutt -f>
+
+=back
+
+=head1 CONFIGURATION
+
+=over 8
+
+=item publicinbox.mailEditor
+
+The command to perform the edit with.  An example of this would be
+C<mutt -f>, and the user would then use the facilities in L<mutt(1)>
+to edit the mail.  This is useful for editing attachments or
+Base64-encoded emails which are more difficult to edit with a
+normal editor (configured via C<GIT_EDITOR>, C<VISUAL> or C<EDITOR>).
+
+Default: none
+
+=back
+
+=head1 ENVIRONMENT
+
+=over 8
+
+=for comment MAIL_EDITOR is undocumented (unstable, don't want naming conflicts)
+
+=item GIT_EDITOR / VISUAL / EDITOR
+
+public-inbox-edit will fall back to using one of these variables
+(in that order) if C<publicinbox.mailEditor> is unset.
+
+=item PI_CONFIG
+
+The default config file, normally "~/.public-inbox/config".
+See L<public-inbox-config(5)>
+
+=back
+
+=head1 LIMITATIONS
+
+Only L<v2|public-inbox-v2-format(5)> repositories are supported.
+
+=head1 CONTACT
+
+Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>
+
+The mail archives are hosted at L<https://public-inbox.org/meta/>
+and L<http://hjrcffqmbrq6wope.onion/meta/>
+
+=head1 COPYRIGHT
+
+Copyright 2019 all contributors L<mailto:meta@public-inbox.org>
+
+License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>
+
+=head1 SEE ALSO
+
+L<public-inbox-purge(1)|https://public-inbox.org/public-inbox-purge.html>
diff --git a/Documentation/public-inbox-index.pod b/Documentation/public-inbox-index.pod
index c15c1cac..610dacbe 100644
--- a/Documentation/public-inbox-index.pod
+++ b/Documentation/public-inbox-index.pod
@@ -42,6 +42,13 @@ Xapian database.
 
 This does not touch the NNTP article number database.
 
+=item --prune
+
+Run L<git-gc(1)> to prune and expire reflogs if discontiguous history
+is detected.  This is intended to be used in mirrors after running
+L<public-inbox-edit(1)> or L<public-inbox-purge(1)> to ensure data
+is expunged from mirrors.
+
 =back
 
 =head1 FILES