about summary refs log tree commit homepage
path: root/lib/PublicInbox/LeiInspect.pm
DateCommit message (Collapse)
2021-10-19lei inspect: show ISO8601 {rt} and {dt}, too
While inspect is intended for debugging, the Unix epoch in seconds requires extra steps for human consumption; just steal what we used for "lei q -f json" output.
2021-10-19lei inspect: add atfork hook
This is necessary for in case an inspect command is run in a parallel with other commands.
2021-10-15lei + ipc: simplify process reaping
Simplify our APIs and force dwaitpid() to work in async mode for all lei workers. This avoids having lingering zombies for parallel searches if one worker finishes soon before another. The old distinction between "old" and "new" workers was needlessly complex, error-prone, and embarrasingly bad. We also never handled v2:// writers properly before on Ctrl-C/Ctrl-Z (SIGINT/SIGTSTP), so add them to @WQ_KEYS to ensure they get handled by $lei when appropropriate.
2021-10-14lei inspect: account for non-extindex inboxes
Inbox->xdb does not exist, but this code path was apparently never tested :x I noticed this on basic v2 inbox, but it could happen with any v1/v2 inbox. Move ->num2docid into Search so it's less awkward to use.
2021-10-02lei inspect: fix "mid:" prefix, expand to Xapian
This fixes inspect for uninitialized instances, and adds Xapian ("xdoc") output if available. Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org> Message-ID: <20211001204943.l4yl6xvc45c5eapz@meerkat.local>
2021-10-02lei inspect: integerize "bytes" and "lines" fields
These are always numeric, but none of the Perl code cares; we want to prevent JSON from quoting them.
2021-09-21lei: various completion improvements
"lei export-kw" no longer completes for anonymous sources. More commands use "lei refresh-mail-sync" as a basis for their completion work, as well. ";AUTH=ANONYMOUS@" is stripped from completions since it was preventing bash completion from working on AUTH=ANONYMOUS IMAP URLs. I'm not sure if there's a better way, but all of our code works fine without specifying AUTH=ANONYMOUS as a command-line arg. Finally, we fallback to using more candidates if none can be found, allowing multiple URLs to be completed.
2021-09-21lei: simplify internal arg2folder usage
We can set opt->{quiet} for (internal) 'note-event' command to quiet ->qerr, since we use ->qerr everywhere else. And we'll just die() instead of setting a ->{fail} message, since eval + die are more inline with the rest of our Perl code.
2021-09-21lei_mail_sync: account for non-unique cases
NNTP servers, IMAP servers, and various MUAs may recycle "unique" identifiers due to software bugs or careless BOFHs. Warn about them, but always be prepared to account for them.
2021-09-21lei inspect: support NNTP URLs
No reason not to support them, since there's more public-inbox-nntpd instances than -imapd instances, currently.
2021-09-21lei inspect: convert to WQ worker
Xapian and SQLite access can be slow when a DB is large and/or on high-latency storage.
2021-09-18lei_mail_sync: rely on flock(2), avoid IPC
Since 44917fdd24a8bec1 ("lei_mail_sync: do not use transactions"), relying on lei/store to serialize access was a pointless endeavor. Rely on flock(2) to serialize multiple writers since (in my experience) it's the easiest way to deal with parallel writers when using SQLite. This allows us to simplify existing callers while speeding up 'lei refresh-mail-sync --all=local' by 5% or so.
2021-09-16lei: git_oid: replace git_blob_id
We'll be using binary SHA-1 and SHA-256 in-memory since that's what mail_sync.sqlite3 stores.
2021-09-03lei inspect: support reading eml from --stdin
This can be useful inside mutt since I was diagnosing why a label ("L:$FOO") search was giving me a false-positive search result...
2021-09-03lei: use lei->lms in place of lse->lms in a few places
We can golf out some code and refcounts this way.
2021-08-18lei: add ->lms shortcut for LeiMailSync
We access this read-only in many places (and will in more), so provide a shortcut to simplify callers.
2021-07-25lei_mail_sync: locations_for API uses oidbin for comparisons
Favor oidbin use internally to reduce internal memory traffic.
2021-07-25lei_inspect: fix typo
Not sure how this wasn't caught, earlier...
2021-07-01lei inspect: support "mid:" (and "m:") prefix
Using this to track down deduplication failures in -extindex...
2021-07-01lei inspect: support automatic pager in output
All commands which output non-trivial amounts of data to the terminal should support this.
2021-06-17lei inspect: learn "num:" and "docid:" prefixes
"num:" is useful for inspecting Inbox-ish directories, while "docid:" can be used for any Xapian DB (not just stuff managed by our code).
2021-05-30lei lcat+inspect: start wiring up completion
Colons and other delimiters still cause problems for our bash completion, but some completion is better than no completion.
2021-05-28lei: handle a single IMAP message in most places
"lei import" can now import a single IMAP message via <imaps://example.com/MAILBOX/;UID=$UID> Likewise, "lei inspect" can show the blob information for UID URLs and "lei lcat" can display the blob without network access if imported. "lei lcat" also gets rid of some unused code and supports "blob:$OIDHEX" syntax as described in the comments (and used by our "text" output format). v2: enforce UID in URL, fail without v3: fix error reporting (s/fail/child_error/)
2021-05-25lei_mail_sync: args2folder: common folder lookup sub
This lets us have a more consistent UX for mapping easily-typed command-line arguments to canonical folder locations.
2021-05-24lei inspect: use LeiMailSync->match_imap_url
Move match_imap_url into LeiMailSync so it can be used in more places, such as "lei inspect". Upcoming commands such as "lei forget-mail-sync" and {add,forget,pause,resume}-watch will also support relaxed IMAP matching rules since there's no reasonable way to expect users use ";UIDVALIDITY=" on the command-line.
2021-05-06lei_mail_sync: Maildir canonicalization omits trailing slash
We use trailing slashes internally, but should not increase visual noise for users by exposing them in config files or DB storage (and shell completion/listings). This fixes a long-standing bug in $lei->rel2abs that prevented absolute paths from being canonicalized.
2021-05-01lei import: fix --mail-sync handling in LeiInput
"lei inspect" also shows "mail-sync" as a field name
2021-04-24lei import: keep sync info for Maildir and IMAP folders
We aren't using it, yet, but the plan is to be able to use this information to propagate keyword changes back to IMAP and Maildir folders using some to-be-implemented command. "lei inspect" is a half-baked new command to make testing this change easier. It will be updated to support more SQLite+Xapian introspection duties in the future, including public-inbox things independent of lei.