about summary refs log tree commit homepage
path: root/Documentation
DateCommit message (Collapse)
2021-05-04treewide: update to v3 Tor onions
v2 onions are insecure, deprecated and going away. v3 names are unfortunately longer and more difficult to remember, but should be more resistant to attack than v2 ones.
2021-04-30lei import: avoid IMAPTracker, use LeiMailSync more
IMAPTracker has a UNIQUE constraint on the `url' column, which may cause compatibility and/or rollback problems in attempting to deal with UIDVALIDITY changes. Having multiple sources of truth leads to confusion and bugs, so relying on LeiMailSync exclusively ought to simplify things. Furthermore, since LeiMailSync is only written to by LeiStore, it is safer in that it won't mark a UID or article as imported until git-fast-import has seen it, and the SQLite commit always happens after "done\n" is sent to fast-import. This mostly reverts recent commits to IMAPTracker to support lei, those are: 1) commit 7632d8f7590daf70c65d4270e750c36552fa9389 ("net_reader: restart on first UID when UIDVALIDITY changes") 2) commit 311a5d37ad275cd75b1e64d87827c4d13fe4bfab ("imap_tracker: prepare for use with lei"). This means public-inbox-watch will not change between 1.6 and 1.7: -watch stops synching a folder when UIDVALIDITY changes.
2021-04-28doc: lei q: split --output and --format, note "text"
I guess --format makes sense for stdout, after all; and I'm enjoying "-f text" quite a bit, so far.
2021-04-28doc: lei q: split =item aliases onto separate lines
It makes L</--augment> look nicer without resorting to L<--augment|/-a, --augment> and similarly verbose nastiness. Having each option as a separate =item (with a blank line in between each =item) seems to be the preferred style used within Perl core documentation (I used perlrun.pod as an example), so we'll follow Perl core style, here. This needs to be done for other manpages, at some point...
2021-04-28doc: lei: use /tmp for search results pathnames
This drives the point home about results being volatile and discardable.
2021-04-24doc: lei_design_notes: add a bit on WAL usage
2021-04-22lei import|convert: drop --no-kw aliases
Supporting --no-keywords and --no-flags aliases is harmful if users end up assuming "keywords:" and "flags:" are valid search prefixes (they're not).
2021-04-22lei import: --incremental default for NNTP and IMAP
No point in burning through bandwidth to import stuff we already saw. All this logic is shared with -watch but uses a different pathname for lei since it's tied to lei/store (and not a public-inbox).
2021-04-21doc: add lei_design_notes.txt and lei-store-format(5)
lei itself is a somewhat weird design, but lei/store is a fairly normal hybrid of extindex with v2-style storage.
2021-04-18doc config: mention obfuscation-related options
Obfuscation has been available since v1.0.0. Help those that want to use the feature figure out how.
2021-04-15doc: technical/whyperl: update w/ (good) news from PSC
It looks like stability and compatibility will prevail, after all.
2021-03-31doc: lei-overview: favor Maildir for mutt examples
mboxes are generally horrible for interactive read-write use due to locking. Describe our parallel behavior with mutt, since writing mail can take a long while and being able to read results as they're written is nice. We'll also use a gzipped mboxrd for the import example, since we can decompress gzipped mboxrds automatically, now.
2021-03-31doc: add lei-mail-formats(5) manpage
While plenty of online documentation exists, it's good to have a locally-available summary for users to look at offline. Fix a URL in Watch.pm while we're at it, too.
2021-03-30lei tag: rename from "lei mark"
I've decided "tag" is a better verb since it seems more widely-used term for associating metadata with data. Not only is it analogous to the "notmuch tag" command, but also makes sense when compared to tooling for manipulating metadata for non-mail data (e.g. audio metadata tags). There's even a Wikipedia entry for it: https://en.wikipedia.org/wiki/Tag_(metadata) whereas "mark" is used in the description, but has no entry of its own with regards to metadata.
2021-03-29doc: lei: update description, add warnings
Describing the design details and architecture doesn't seem appropriate for a section 1 manpage. We'll also add a warning about it being in the early stages.
2021-03-29doc: lei q: clarify default output as stdout
Seeing a plain "-" may be confusing, especially when we also support it for --stdin. Use the C<> POD directive to denote it as code, too.
2021-03-29doc: lei q: add warning for --output clobbering
The behavior matching mairix still frightens me a bit when it comes to supporting new users. On the other hand, I've rarely ever used --augment with mairix, so I still think the current (dangerous) behavior makes sense in the context of search results.
2021-03-29doc: lei q: drop NNTP from --output description
We only support NNTP as inputs for convert, import, and mark|tag. I'm not sure if supporting NNTP output is worth it, nor do we have a good way to test it.
2021-03-29doc config: don't render a to-do comment
In the public-inbox-config manpage, the match=domain item under publicinbox.wwwlisting has a to-do comment that gets rendered as "support showing cgit listing". That's potential confusing to readers, especially given that the "TODO" is dropped. Change the markup so that the comment isn't rendered.
2021-03-29doc lei overview: better explain routes into local store
2021-03-29doc lei overview: note that lei-init is usually unnecessary
cf. https://public-inbox.org/meta/20210325083207.GA30551@dcvr
2021-03-29doc lei: add manpages for new commands
2021-03-29doc lei: update manpages with new options
2021-03-29doc lei: don't render most to-do comments
The lei manpages have a number of to-dos, but with the exception of the lei-q's -tt warning, none of them seem worth displaying to the reader (and some might not be worth addressing at all).
2021-03-29doc lei: drop an unnecessary to-do comment
When a new command is implemented, it is probably clear that it should be added to lei.pod, but either way, having a to-do comment in lei.pod isn't likely to help.
2021-03-29doc lei: note --stdin shortcut in synopses
2021-03-29doc lei-q: fix typo in -tt description
2021-03-19doc: glossary: add missing over/back POD directives
Oops :x
2021-03-17lei_store: keywords => vmd (volatile metadata), prepare for labels
Since keywords and mailboxes (AKA labels) are separate things in JMAP; and only keywords can map reliably to Maildir and mbox; we'll keep them separate in our internal data representations, too. I initially wanted to call this just "meta" for "metadata", but that might be confused with our mailing list name. "metadata" is already used in Xapian's own API, to add another layer of confusion. "tags" was also considered, but probably confusing to notmuch users since our "labels" are analogous to "tags" in notmuch, and notmuch doesn't seem to cover "keywords" separately... So "vmd" it is, since we haven't used this particular three-letter-abbreviation anywhere before; and "volatile" seems like a good description of this metadata since everything else up to this point has been mostly WORM (write-once, read-many).
2021-03-11doc: update 1.7 release notes, tuning, TODO
Some stuff done, some stuff still needs doing.
2021-03-11doc: glossary: add information for dates and timestamps
These have been confusing to me in the past, too.
2021-03-10doc: start glossary for overlapping concepts
This is intended to keep track of concepts with different terms between NNTP, IMAP, config file, lei storage, and upcoming JMAP support.
2021-03-10doc: technical/data_structures: update for EOFpipe
ParentPipe no longer exists and was replaced by the more flexible EOFpipe.
2021-02-27doc: lei-overview: add performance and bash completion sections
Take care of a couple of the items mentioned at <https://public-inbox.org/meta/20210218202818.GA19443@dcvr>.
2021-02-27doc: lei-import: drop markup of "stdin"
stdin isn't placed in C<> elsewhere.
2021-02-27doc: lei: update manpages
Catch up with recent developments.
2021-02-26lei q: -tt marks direct hits as "flagged"
This can be used to quickly distinguish messages which were direct hits when doing thread expansion vs messages that were merely part of the same thread. This is NOT mairix-derived behavior, but I occasionally found it useful when looking at results in an MUA to know whether a message was a direct hit or not. This makes "-t" consistent with non-"-t" cases as far as keyword reading goes.
2021-02-26lei import: use --in-format/-F for consistency
Since we recommend $IN_FORMAT:$LOCATION, this is hopefully not intrusive (not that this is released software, yet). This is to be consistent with "lei convert" usage. We'll keep "-f" only for output formats, since that is used for "lei q" and "lei convert" for outputs
2021-02-23doc: lei: favor "-o format:$PATHNAME" over "-f"
The --format argument is redundant and may be dropped entirely. Update the lei manpages to prefer the format prefix. cf. https://public-inbox.org/meta/20210217044032.GA17934@dcvr/
2021-02-11doc: lei: update manpages
Catch up with recent developments.
2021-02-11doc: add lei-import(1)
2021-02-11doc: lei: prefer 'location' and 'dirname'
This follows the help output change in 52342875 (lei help: split out into separate file, 2021-02-06).
2021-02-11doc: lei q: use 'mfolder' as --output placeholder
'mfolder' is familiar to mairix users, and 'path' isn't a good choice because support will be added for IMAP. Link: https://public-inbox.org/meta/YCBh62OqkYnr5cqw@dcvr
2021-02-07lei: replace --thread with --threads
Nobody is expected to use long options, but for consistency with mairix(1), we'll use the pluralized option throughout (including existing PublicInbox::{Search,SearchView}). Link: https://public-inbox.org/meta/20210206090119.GA14519@dcvr/
2021-02-07lei: remove --mua-cmd alias for --mua
While "mua-cmd" may be more accurate, nobody is expected to type 4 extra characters. It's a needless ambiguity with no precedence or prior art to follow. Link: https://public-inbox.org/meta/20210206090119.GA14519@dcvr/
2021-02-03doc: lei-q: note "-a" and link to Xapian QueryParser
"-a" is supported by mairix, too. We should also note somewhere the query parsing features supported by Xapian.
2021-02-01doc: note optional BSD::Resource use
We've actually been capable of using this since 2019(*) in our spawn code for PSGI limiters. And it's been used since 2016 in our tests. It's a dependency of SpamAssassin, and Danga::Socket used it, too. (*) commit 721368cd04bfbd03c0d9173fff633ae34f16409a ("spawn: support RLIMIT_CPU, RLIMIT_DATA and RLIMIT_CORE")
2021-02-01doc: add lei-overview(7)
[ew: s/mboxrd/mboxcl2/ since that's what mutt uses]
2021-02-01doc: start manpages for lei commands
Add manpages for lei and the currently implemented subcommands. The included options and their descriptions follow to a large degree the --help output, dropping some options that are not currently wired up.
2021-01-26doc: start working on public-inbox-extindex(1) manpage
It's barely started, but I started writing this weeks ago, but I'm still unsure about some behavioral/usability things and hoping work on lei(1) can flush them out.