[PATCH] doc: TODO and release notes updates ahead of 1.6

[PATCH] doc: TODO and release notes updates ahead of 1.6

[Eric Wong]

Some more things have happened...

And drop some items which are too expensive to support,
such as automatic mirroring.
---
 Documentation/RelNotes/v1.6.0.eml | 31 ++++++++++++++++++++++++++++---
 TODO                              | 20 ++++++++++----------
 2 files changed, 38 insertions(+), 13 deletions(-)

diff --git a/Documentation/RelNotes/v1.6.0.eml b/Documentation/RelNotes/v1.6.0.eml
index 4f72f352..153b1f13 100644
--- a/Documentation/RelNotes/v1.6.0.eml
+++ b/Documentation/RelNotes/v1.6.0.eml
@@ -40,10 +40,13 @@ on HDD storage, though we can't defy the laws of physics, either.
     on message delivery (or spam removal) to wake up read-only
     daemons via inotify or kqueue.
 
-* Upgrading
+  - `--help' switch supported by command-line tools
+
+* Upgrading for new features in 1.6
 
   The ordering of these steps is only necessary if you intend to
-  use some new features of this release.
+  use some new features in 1.6.  Future releases may have
+  different instructions (or be entirely transparent).
 
   0. install (use your OS package manager, or "make install")
 
@@ -129,13 +132,35 @@ on HDD storage, though we can't defy the laws of physics, either.
     retrieval
 
   - mbox search results may now include all messages pertaining to that
-    thread (requires `--reindex' mentioned in `Upgrading').
+    thread.  Needs `--reindex' mentioned above in
+    `Upgrading for new features in 1.6'.
+
+  - fix mbox.gz search results downloads for lynx users
+
+  - small navigation tweaks, more prominent mirroring instructions
 
 * public-inbox-watch
 
   - Linux::Inotify2 or IO::KQueue is used directly,
     Filesys::Notify::Simple is no longer required
 
+  - NNTP groups and IMAP mailboxes may be watched in addition
+    to Maildirs (lightly tested).
+
+* Ongoing internal changes
+
+  - reduce event loop hogging for many-inbox support
+
+  - use more Perl v5.10-isms, future-proof against Perl 8
+
+  - more consistent variable and field naming, improve internal
+    documentation and comments
+
+  - start supporting >=40 char git identifiers for SHA-256
+
+  - test -httpd-specific code paths via Plack::Test::ExternalServer
+    in addition to generic PSGI paths.
+
 Please report bugs via plain-text mail to: meta@public-inbox.org
 
 See archives at https://public-inbox.org/meta/ for all history.
diff --git a/TODO b/TODO
index 9396f661..467f047f 100644
--- a/TODO
+++ b/TODO
@@ -115,6 +115,7 @@ all need to be considered for everything we introduce)
 * extend public-inbox-watch to support IMAP, NNTP
 
 * improve performance and avoid head-of-line blocking on slow storage
+  (done for most git blob retrievals, Xapian needs work)
 
 * HTTP(S) search API (likely JMAP, but GraphQL could be an option)
   It should support git-specific prefixes (dfpre:, dfpost:, dfn:, etc)
@@ -123,6 +124,11 @@ all need to be considered for everything we introduce)
 
 * search across multiple inboxes, or admin-definable groups of inboxes
 
+  This will require a new detached Xapian index that can be used in
+  parallel with existing per-inbox indices.  Using ->add_database
+  with hundreds of shards is unusable in current Xapian as of
+  August 2020 (acknowledged by Xapian upstream).
+
 * scalability to tens/hundreds of thousands of inboxes
 
   - pagination for WwwListing
@@ -136,6 +142,8 @@ all need to be considered for everything we introduce)
 * command-line tool (similar to mairix/notmuch, but solver+git-aware)
 
 * consider removing doc_data from Xapian, redundant with over.sqlite3
+  It's no longer read as of public-inbox 1.6.0, but still written for
+  compatibility.
 
 * share "git cat-file --batch" processes across inboxes to avoid
   bumping into /proc/sys/fs/pipe-user-pages-* limits
@@ -157,15 +165,7 @@ all need to be considered for everything we introduce)
 
 * highlighting + linkification for "git format-patch --interdiff" output
 
-* highlighting + linkification for "git format-patch --range-diff" output
-  (requires mirroring of git repos)
-
-* parse and allow (semi)automatic-mirroring of "git request-pull" output
-  for coderepos
-
-* configurable diff output for solver-generated blobs
-
-* figure out how search for messages with multiple Date: headers
-  should work (some wacky examples out there...)
+* highlighting for "git format-patch --range-diff" output
+  (linkification is too expensive, as it requires mirroring)
 
 * support UUCP addresses for legacy archives

1.6 in a few hours/days?

[Eric Wong]

It seems like a good as any time to tag and release 1.6...

Detached index + libgit2 support is being worked on for 1.7;
which might be soon after 1.6.

Something's felt off for months even though I can't quite put my
finger on it...  The last round of -watch-related fixed
contention problems might've been it(*), but there's still a
feeling I just can't shake.

Anyways, I've been running the latest code off public-inbox.org
and some .onions without seeing problems, at least.

Maybe that off feeling is just the result of being cooped up
during the pandemic while dealing with heat, insects, insomnia,
aging, old injuries, etc...


(*) https://public-inbox.org/meta/20200831044140.17027-1-e@80x24.org/