about summary refs log tree commit homepage
path: root/TODO
DateCommit message (Collapse)
2019-12-18TODO: add UUCP address item
We should support historical archives from the old days, but I'm not sure how to best go about it, for now, given how tricky correct handling of modern email addresses is. We can deal with it if/when somebody decides to import some ancient archives...
2019-12-12Date::Parse is now optional
-mda should not be dealing with broken Date: headers nowadays, and deprioritize it in our documentation and internal checks.
2019-12-11TODO: update and add a few more items
SpamAssassin has used re2c (via sa-compile) for many years, now, and it seems to work fine, there. GMime also looks promising when combined with Inline::C since GMime can operate on mmap-ed regions. Given the inevitable demise of many .orgs when price rise; supporting a URL rewriter similar to .mailmap makes sense. And HTTP CONNECT seems like something our -httpd can support to let firewalled users read over NNTP.
2019-11-03TODO: update item for multiple Date: headers
That's the only head-scratcher of the bunch remaining, since that relies on ranges.
2019-10-23Merge branch 'regen'
* regen: v2writable: use msgmap as multi_mid queue v2writable: move git->cleanup to the correct place v2writable: reindex handles 3-headered monsters v2writable: improve "num_for" API and disambiguate v2writable: set unindexed article number
2019-10-21v2writable: reindex handles 3-headered monsters
And maybe 8-headered ones, too... I noticed --reindex failing on the linux-renesas-soc mirror due one 3-headed monster of a message having 3 sets of headers; while another normal message had a Message-ID that matched one of the 3 IDs of the 3-headed monster. We still try to do the majority of indexing backwards, but we defer indexing multi-Message-ID'd messages until the end to ensure we get all the "good" messages in before we process the multi-headered ones. Link: https://public-inbox.org/meta/20191016211415.GA6084@dcvr/
2019-10-21TODO: DHT (distributed hash table) for Message-IDs
This would encourage SPOF avoidance. It would also make it easier to do "Certificate Transparency"-style attestation of messages.
2019-10-17TODO: add a few more items around coderepos
Going where no email client/tool has gone before...
2019-10-15TODO: add an item for Python pygments
Pygments seems to be a popular highlighter and widely available, so we'll be providing support for that at some point... Link: https://public-inbox.org/meta/20190926131836.GB10467@chatter.i7.local/ Link: https://public-inbox.org/meta/874l0zt7sd.fsf@alyssa.is/
2019-10-14TODO: add item for config linter and grapher
It'll be useful for tools and users to test and perhaps visualize configs before reloading -httpd/-nntpd/-watch.
2019-10-01TODO: add item for searching based on git-patch-id(1)
I forgot about this feature when I was implementing blob-ID-based searches :x
2019-10-01TODO: update with "git cat-file" items
Millions of inboxes in an instance is probably not feasible, but dozens or even hundreds could happen and /proc/sys/fs/pipe-user-pages-soft is only 16384 on my system, with each "cat-file --batch" process using 16+1 pages worth of pipes.
2019-09-22doc: update HACKING and TODO with a few items
Inline::C seems alright, so we might use it more since it still allows end users to quickly make changes. Our performance on rotational disks is also terrible, and could be improved...
2019-07-13TODO: remove done items, add some more
It never ends...
2019-07-06nntp: support COMPRESS DEFLATE per RFC 8054
This is only tested so far with my patches to Net::NNTP at: https://rt.cpan.org/Ticket/Display.html?id=129967 Memory use in C10K situations is disappointing, but that's the nature of compression. gzip compression over HTTPS does have the advantage of not keeping zlib streams open when clients are idle, at the cost of worse compression.
2019-05-31TODO: add item for optional Cache::FastMmap
Taking one step out of setting up a performant deployment could make setup and administration easier (at the cost of installing an extra-but-common XS module). This can also be useful for the day NNTP servers see hug-of-death events.
2019-05-24TODO: more stuff: bundles, synonyms, dogfooding
git bundles could/should make self-hosting easier. Being able to configure synonym (and spelling) lists would make some searches more useful. Might as well dogfood kernel stuff, too, given the overlap and history between this project, git and the Linux kernel. Would be interesting to have *BSD folks throw their hat in the ring, too. Building/testing userspace stuff is often the most time-consuming, but necessary to ensure future compatibility.
2019-05-15lazy load Xapian and make it optional for v2
More tests work without Search::Xapian, now. Usability issues still need to be fixed
2019-05-05DS: workaround IO::Kqueue EINTR (mis-)handling
IO::Kqueue seems unmaintained, so workaround a long-standing bug where it falls over on signals: https://rt.cpan.org/Ticket/Display.html?id=116615
2019-05-05listener: use EPOLLEXCLUSIVE for listen sockets
Since our listen sockets are non-blocking and we may run multiple httpd|nntpd processes; we need a way to avoid thundering herds when there are multiple httpd|nntpd worker processes. EPOLLEXCLUSIVE was added just for that in Linux 4.5
2019-05-04bundle Danga::Socket and Sys::Syscall
These modules are unmaintained upstream at the moment, but I'll be able to help with the intended maintainer once/if CPAN ownership is transferred. OTOH, we've been waiting for that transfer for several years, now... Changes I intend to make: * EPOLLEXCLUSIVE for Linux * remove unused fields wasting memory * kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615 * accept4 support And some lower priority experiments: * switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes) * nginx-style buffering to tmpfile instead of string array * sendfile off tmpfile buffers * io_uring maybe?
2019-04-17TODO: add an item for hooks support
and add a note for grokmirror support/integration, too
2019-02-05TODO: add item for "scraper" importers
The git-users mailing list is on Google Groups with obfuscated addresses and censored archives. We should allow users to import them soon, as obfuscated/censored archives are better than not having archives at all when Google decides to shut down yet-another-service. There's also some mangling that (most) instances of Mailman do (e.g. cgit), but being able to follow such groups over NNTP and use our search functionality is still useful and better than what typical Mailman installations provide.
2019-02-04TODO: support migration to v2 without breaking v1 "git fetch"
I'd like to move https://public-inbox.org/git/ to v2; but cronjobs using "git fetch" to following should not break.
2019-01-31doc: remove completed TODO items
2019-01-20$INBOX/_/text/color/ and sample user-side CSS
Since we now support more CSS classes for coloring, give this feature more visibility.
2019-01-01TODO: avoid mentioning untrustworthy browser extensions
Old and new versions of Mozilla-based browsers seem to support userContent.css just fine. cf. https://www-archive.mozilla.org/unix/customizing.html#usercss http://kb.mozillazine.org/index.php?title=UserContent.css
2019-01-01TODO: support integration with cgit/gitweb/etc...
We support searching on blob identifiers for a reason :>
2018-12-30TODO: add a note for exposing a targeted reindexing API
2018-12-29TODO: add note for "IMAP IDLE"-like long-polling "git fetch"
2018-12-18TODO: add a note for davfs2 Range: support
And maybe I or somebody else interested will implement it, since fusedav is abandoned upstream and removed from Debian testing: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840388 Yes, I have fusedav patches at https://bogomips.org/fusedav.git as noted in the above bug report, but I think davfs2 has more momentum at the moment.
2018-04-23TODO: add EPOLLEXCLUSIVE item
Threads are generally discouraged in Perl5, so I won't be using a dedicated blocking accept4() thread like I would in other languages.
2018-04-19TODO: a few more updates
Mainly, v2 stuff is done
2018-01-16TODO: notes about v2 format for giant archives
Inspired by interest in LKML archival: https://public-inbox.org/meta/d5546b24-5840-4ae9-d25b-5e3e737ed73b@linuxfoundation.org
2017-02-07TODO: several updates
Always plenty to do while working on this...
2016-12-17atom: implement message threading per RFC 4685
This will allows certain feed readers to render a message thread as described in <https://www.jwz.org/doc/threading.html>. Feed readers with knowledge of of RFC 4685 are unknown to us at this time, but perhaps this will encourage future implementations. Existing feed readers I've tested (newsbeuter, feed2imap) seem to ignore these tags gracefully without degradation.
2016-12-14TODO: note IO::KQueue for the ticket
Do not require users to have network access to know what the link refers to.
2016-09-09TODO: updates for done items
The existing string -> number date range Xapian query is good enough, and having too much flexibility is probably bad for caching (as well as increasing our attack surface, because parsing queries is tricky). Tags-as-skiplists are probably not worth the effort given Xapian, and we may have to import old messages after-the-fact, anyways, and message delivery for mirrors is never orderly. Other items are all done and need to be maintained (like the search engine docs for the mairix-compatibility features that just got pushed out)
2016-08-14TODO: updates based on git@vger mirror experience
Plenty more to do!
2016-07-02TODO: clarify streaming Email::MIME replacement
I bet there's a billion other improvements to be made elsewhere.
2016-07-01TODO: update documentation for performance items
2016-06-30www: implement hybrid flat+thread conversation view
This should be more accessible to readers on narrow terminals (or giant fonts) while providing a chronological view which is also aware of message threading relationships.
2016-06-25TODO: add a few Xapian-related items
"git cat-file --batch" seems expensive for big repos and loading 70K+ tree objects in git isn't all that fast. Ideas are cheap, time, code, and testing are not :P
2016-06-17TODO: remove cookies for colors
It would be too much of a burden for caching system when user-supplied CSS is more powerful.
2016-05-23TODO: update linkification notes
Some readers will want to use "HTTPS Everywhere" conveniently; and I will support it.
2016-05-14import ssoma-replay example script I've been using
Unfortunately, most users still prefer their mail delivered over SMTP; so we'll at least document mlmmj integration for now until we can popularize pull-based reading over POP3/NNTP/ssoma.
2016-04-29TODO: add item for .mailmap support
Email addresses get out-of-date, so make sure they're mapped properly for future readers. git and linux-kernel already have an established convention for this, so we will follow it.
2016-04-25TODO: various updates
2016-01-06TODO: add a few more items
Sometimes I contribute to projects with centralization-inducing things like reply-to-list and bug trackers. I dislike those things myself, but most of the time I can deal with them... for now.
2016-01-04misc doc updates
Strongly emphasize decentralization, as that was actually the main impetus for my interest in git.