about summary refs log tree commit homepage
path: root/lib/PublicInbox/LeiTag.pm
DateCommit message (Collapse)
2021-10-24lei: always pass $lei to LeiAuth->op_merge
This will make future developments easier.
2021-09-21lei: various completion improvements
"lei export-kw" no longer completes for anonymous sources. More commands use "lei refresh-mail-sync" as a basis for their completion work, as well. ";AUTH=ANONYMOUS@" is stripped from completions since it was preventing bash completion from working on AUTH=ANONYMOUS IMAP URLs. I'm not sure if there's a better way, but all of our code works fine without specifying AUTH=ANONYMOUS as a command-line arg. Finally, we fallback to using more candidates if none can be found, allowing multiple URLs to be completed.
2021-09-19lei/store: use SOCK_SEQPACKET rather than pipe
This has several advantages: * no need to use ipc.lock to protect a pipe for non-atomic writes * ability to pass FDs. In another commit, this will let us simplify lei->sto_done_request and pass newly-created sockets to lei/store directly. disadvantages: - an extra pipe is required for rare messages over several hundred KB, this is probably a non-issue, though The performance delta is unknown, but I expect shards (which remain pipes) to be the primary bottleneck IPC-wise for lei/store.
2021-09-18lei: lock worker counts
It doesn't seem worthwhile to change worker counts dynamically on a per-command-basis with lei, and I don't know how such an interface would even work...
2021-09-06lei_auth: simplify users
There's no need to alias net_merge_all in each WQ class which uses LeiAuth, `$obj->$sub' works even when `$sub' is a fully-qualified subroutine name with `::' in it. perlobj(1) documents it under "Method Call Variations".
2021-06-10lei tag: less confusing warning about unimported messages
"unimported" is more meaningful than "missing", here. And instead of having every worker spew about unimported messages, we'll accumulate and only print one warning line. This necessitated alterating ->DESTROY behavior and persisting the client socket within the $lei object itself, not just the PktOp consumer object.
2021-06-09lei tag: parallelize Maildir access
Since Maildir isn't guaranteed to have any sort of order, we can parallelize inputs, here. On a 4-core system, this reduced one of my tag invocations from 5.5 to 1.4s.
2021-06-08lei: generalize auxiliary WQ handling
op_wait_event is now more lei-specific since we no longer have to care about oneshot and use a synchronous loop. {ikw} (import-keywords) started a trend, but LeiPmdir (parallel Maildir) is an upcoming WQ class that will follow this idea. Eventually, {l2m} usage may be updated to follow this, too.
2021-05-23lei tag: support tagging index-only messages
This will make some of our tests faster and allow users to try more features of lei without high storage requirements.
2021-05-03lei: simplify workers_start API
In most cases, we just name the worker process based on the command. The only change is for LeiMirror vs "lei add-external --mirror", but I doubt it matters.
2021-05-03lei_input: common net_merge_all_done for lei <import|tag>
I suspect there'll be more lei_input-only things in the future.
2021-05-01lei_auth: s/net_merge_complete/net_merge_all_done/
We use the "done" term elsewhere for similar things, and my easily-confused mind equates "complete" with shell completion.
2021-04-28lei: simple WQ workers use {wq1} field
This lets us share more code and reduces cognitive overhead when it comes to picking names (because {lsss} was ridiculous). We'll need to ensure the first error set in lei is the actual error we exit with, otherwise things can get confusing and errors may get lost.
2021-04-27lei: standardize on _lei_wq_eof callback for workers
Simplify our internals a little bit.
2021-04-23lei import: support adding keywords and labels on import
This saves some work and makes it easier to set volatile metadata on a message at import time.
2021-04-21lei: share common *done_wait callbacks
Code is the enemy, and there's no need to duplicate things, here. There may be further opportunities along these lines to further deduplicate things...
2021-04-05lei_tag: fix comments w.r.t support levels
RFC 8621 registers $flagged, $answered, $seen, $draft which map to IMAP, Maildir, and mbox Status/X-Status flags. $forwarded is noted in JMAP, but only Maildir and and the "Lemonade" IMAP profile (RFC 5550) support it
2021-04-03lei tag: note message mismatches on failure
Just exiting with a failure code and no error message is confusing :x
2021-04-03lei tag: fix tagging of IMAP inputs
We need net_merge_all and to lock the number of worker jobs. Parallel inputs are not supported, yet (is it needed?, I don't expect this to be used for multiple files very often...).
2021-03-31lei_input: reduce IPC traffic with multiple inputs
No point in sending a command for every input when a single one will do. We'll also trigger LeiStore->done sooner in the worker rather than later.
2021-03-31lei: fix IMAP auth failure handling
We must use the $ops hashref returned by lei->workers_start, since it's modified to include extra handlers for auth failures and whatnot. Fixes: 954581b8e575966a ("lei: simplify PktOp callers")
2021-03-30lei tag: rename from "lei mark"
I've decided "tag" is a better verb since it seems more widely-used term for associating metadata with data. Not only is it analogous to the "notmuch tag" command, but also makes sense when compared to tooling for manipulating metadata for non-mail data (e.g. audio metadata tags). There's even a Wikipedia entry for it: https://en.wikipedia.org/wiki/Tag_(metadata) whereas "mark" is used in the description, but has no entry of its own with regards to metadata.