about summary refs log tree commit homepage
path: root/lib/PublicInbox/LeiExternal.pm
DateCommit message (Collapse)
2021-10-19lei up: support --exclude=, --no-(external|remote|local)
These can be used to temporarily disable using certain externals in case of temporary network failure or mount point unavailability.
2021-10-19lei: use die for external and query handling
This allows "lei up" to continue processing unrelated externals if on output fails.
2021-09-25lei ls-external: split into separate file
This was written before we had auto-loading and rarely used.
2021-09-25lei add-external: split into separate file
Also was written before we had auto-loading and rarely used.
2021-09-25lei forget-external: split into separate file
This was written before we had auto-loading, and forget-external should be a rarely-used command that's not worth loading at startup. Do some golfing while we're in the area, too.
2021-09-21lei: various completion improvements
"lei export-kw" no longer completes for anonymous sources. More commands use "lei refresh-mail-sync" as a basis for their completion work, as well. ";AUTH=ANONYMOUS@" is stripped from completions since it was preventing bash completion from working on AUTH=ANONYMOUS IMAP URLs. I'm not sure if there's a better way, but all of our code works fine without specifying AUTH=ANONYMOUS as a command-line arg. Finally, we fallback to using more candidates if none can be found, allowing multiple URLs to be completed.
2021-09-19lei config --edit: use controlling terminal
As with "lei edit-search", "lei config --edit" may spawn an interactive editor which works best from the terminal running script/lei. So implement LeiConfig as a superclass of LeiEditSearch so the two commands can share the same verification hooks and retry logic.
2021-04-30lei import: support shell completion of known folders
This also fixes completion of "lei up" for IMAP folders.
2021-04-27lei: add "ls-sync" command for listing sync folders
This will be useful, later.
2021-04-18lei ls-search: command to list saved searches
Going forward, we'll probably support JSON for all the "ls-*" subcommands. This also provides the basis for "lei up" shell completion.
2021-03-25lei add-external: do not initialize writable store
There's no need to create or write lei/store when adding an external, we just need to write to the config file.
2021-03-24lei: improve management around short-lived workers
Instead of creating a short-lived circular reference, ensure they don't exist in the first place. Note the following changes to hold an extra ref to $sto: - $self->_lei_store(1)->write_prepare($self); + my $sto = $self->_lei_store(1); + $sto->write_prepare($self); I'm not a perlguts expert, but I actually wanted to switch to the one-line version for LeiImport, but xt/lei-auth-fail.t was getting stuck for some reason. It seems the extra ref to the LeiStore ($sto) object is necessary.
2021-03-23lei: support -c <name>=<value> to overrides
It's a bit nasty, but seems to mostly work for debugging IMAP and NNTP commands.
2021-03-21lei: fix some warnings in tests
And then test the contents of $lei_err to ensure it doesn't happen again. We'll also make MboxLock emit nicer warnings without the line number, since the line number is irrelevant to the user fixing an mbox lock contention problem. Finally, we'll also allow showing loud warnings via TEST_LEI_ERR_LOUD=1
2021-03-21lei: tie ALE lifetime to config file
This should make a future change to "lei import" work more nicely, since we'll be needing ALE to vivify external-only messages upon explicit "lei import".
2021-03-21lei: All Local Externals: bare git dir for alternates
This will be used for keyword (and label) storage for externals. We'll be using this to ensure we don't redundantly auto-import messages into lei/store if they're already in a local external (they can still be imported explicitly via "lei import").
2021-03-19lei: disallow "\n" in local externals paths
git 2.11 and earlier could not handle git directories with newlines in them, nor does libgit2 support them. Followup-to: d87dd0e679587043 ("config: reject `\n' in `inboxdir'")
2021-02-25lei_external: don't treat IPv6 URLs as globs
IPv6 addresses are hexadecimals and colons inside brackets, so add some DWIM-ery to ensure we don't attempt to treat addresses like "http://[dead:beef]/foo/" as a glob.
2021-02-22lei: _lei_cfg: return empty hashref if unconfigured
Existing callers in LeiExternal actually depend on this, and LeiAuth shouldn't need to be creating a config file just to do a conversion against an anonymous IMAP server.
2021-02-10lei_external: fix+test handling of escaped braces
While '{' and '}' are rare in path names, somebody may still use them or deal with software which does (e.g. GNU arch).
2021-02-10lei: note some TODO items (curl, externals)
I don't know if it's worth it to use libcurl directly (nor the effort to support and maintain tests)
2021-02-10lei ls-external: support --local and --remote
Similar to "lei q", "--local" means only local and "--remote" means remote only. I can't think of a reason to have --no-* variants for these switches. There's also updates to the TestCommon for more common lei cases.
2021-02-10lei_external: remove unnecessary Exporter use
We don't need to export for methods which are only called via "->" or "->can".
2021-02-10lei *external: glob improvements, ls-external filtering
The "ls-external" now accepts the same glob patterns used by with lei q --{include,only,exclude}. If no glob is detected, it will be treated as a literal substring match (like "grep -F"). Inverting matches is also supported ("grep -v").
2021-02-07lei add-external: reject index and remote opts w/o mirror
Option combinations which make no sense should fail to prevent misunderstandings and avoid surprises.
2021-02-07lei: add-external --mirror support
This can be useful for users who want to clone and mirror an existing public-inbox. This doesn't have update support, yet, so users will need to run "git fetch && public-inbox-index" for now.
2021-02-04lei add-external: completion for existing URL basenames
Given the presence of one external on a certain host or prefix path, it's logical other inboxes would share a common prefix. For bash users, attempt to complete that using the "-o nospace" option of bash
2021-02-04lei q: -I/--exclude/--only support globs and basenames
We can do basename matching when it's unambiguous. Since '*?[]' characters are rare in URLs and pathnames, we'll do glob matching by default to support a (curl-inspired) --globoff/-g option to disable globbing. And fix --exclude while we're at it
2021-02-03lei q: support --only, --include and --exclude
-I is short for --include since it's standard for C compilers (along with Perl and Ruby). There are no single-character shortcuts for --exclude or --only, since I don't expect --exclude to be used very often and --only is already short (and will support shell completion).
2021-02-01lei: remove SIGPIPE handler
It doesn't save us any code, and the action-at-a-distance element was making it confusing to track down actual problems. Another potential problem was keeping references alive too long. So do like we would a C100K server and check every write while still ensuring lei(1) exit with a proper SIGPIPE iff needed.
2021-01-24lei add-external: don't allow non-existent directories
At least not yet, though we may support mirroring via git.
2021-01-23lei forget-external: don't show redundant "not found"
Pathname/URL canonicalization may not change the result at all, so there's no point in trying (and failing) the same form twice if pre and post-canonicalization are identical.
2021-01-23lei forget-external: just show the location
No need to show the full key name since the user mainly uses the location.
2021-01-23lei completion: handle URLs with port numbers
This improves the experience for developers running local instances of PublicInbox::WWW without permissions to bind port 80 or 443.
2021-01-22lei forget-external: bash completion support
The tricky bit was getting around word splitting bash does on URLs. This may work with other shells, too.
2021-01-22lei: forget-external support with canonicalization
For proper matching, we'll do a better job canonicalizing URLs and path names for matching. Of course, users may edit the file outside of lei, so ensure we try both the canonicalized and as-is form provided by the user. I also don't think we'll need to store externals info in MiscIdx; just the config file is fine.
2021-01-12lei query + pagination sorta working
Parallelism and interactivity with pager + SIGPIPE needs work; but results are shown and phrase search works without shell users having to apply Xapian quoting rules on top of standard shell quoting.
2021-01-01update copyrights for 2021
Using "make update-copyrights" after setting GNULIB_PATH in my config.mak
2021-01-01lei: rename "extinbox" => "external"
The words "extinbox" and "extindex" are too close and easy to confuse with the other. Rename "extinbox" to "external", since these could be IMAP, JMAP or other non-public-inbox search APIs. Link: https://public-inbox.org/meta/20201226112649.GB6226@dcvr/