| Date | Commit message (Collapse) |
|---|
|
This allows us to handle odd inboxes w/o a newsgroup configured
if they also make the strange choice of having backslashes in
their path name. Also, ensure we use case-sensitive LIKE, since
case-insensitive FSes are not worth supporting.
|
|
We aren't using it, yet, but the plan is to be able to use
this information to propagate keyword changes back to IMAP
and Maildir folders using some to-be-implemented command.
"lei inspect" is a half-baked new command to make testing this
change easier. It will be updated to support more SQLite+Xapian
introspection duties in the future, including public-inbox
things independent of lei.
|
|
We'll be using the new class to efficiently propagate keyword
changes from lei/store back to Maildir or IMAP folders.
|
|
This will allow the callback to reliably maintain OID <=> UID
mappings between lei/store and the IMAP folder.
|
|
These will be useful for keyword synchronization, and perhaps
importing a single IMAP message with ->iuid.
|
|
"lei import" behavior will may change w.r.t. keyword
handling. Use separate $HOME between different test_lei
to ensure isolation between the tests.
|
|
Followup-to: 49b036771ef3bf45 ("lei_input: support compressed mboxes")
|
|
This saves some work and makes it easier to set volatile
metadata on a message at import time.
|
|
Since we don't have *at() syscalls readily available to us,
lei-daemon may call ->poke_dst in the wrong relative directory.
Despite not having *at() syscalls, we can still capture the
"$MAILDIR/cur" directory handle at pre_augment time so we can
reliably call futimes(2) on it using the `utime' perlop.
|
|
In other words, treat the same IMAP folder with a different
UIDVALIDITY as a completely different folder. If the UIDVALIDITY
changes, we can start from UID=1 without falling behind or
losing data. If the UIDVALIDITY gets reset to a previously
known-good message, we can still resume where we left off
before the first UIDVALIDITY change.
This affects public-inbox-watch and "lei import"
One potential downside of this is for rare altid users, but
that's mainly intended for NNTP article numbers which are/were
often publicized; not IMAP UIDs which are rarely publicized.
The other potential downside is bandwidth waste in in the rare
case UIDVALIDITY changes while IMAP folder contents remain
unchanged. There's no extra storage used due to existing
(v1|v2|lei/store) deduplication mechanisms.
Before this change, we were matching offlineimap behavior and
stopped synching an IMAP folder when its UIDVALIDITY changed.
offlineimap behavior made sense for IMAP <=> Maildir
synchronization since Maildirs had no sense of UIDVALIDITY and
could only rely on name mapping.
|
|
On my default FreeBSD 11.x system, "/home" is a symlink to
"/usr/home", which causes "lei up" path resolution to fail when
I use outputs in $HOME. Fall back to a slow path of globbing
and matching pathnames based on st_ino+st_dev.
|
|
This is less surprising in case users are used to using --dedupe=
without --save.
|
|
We'll support this mode of operation for now to quiet down
testing of oneshot mode where the daemon doesn't persist.
|
|
Supporting --no-keywords and --no-flags aliases is harmful
if users end up assuming "keywords:" and "flags:" are valid
search prefixes (they're not).
|
|
No point in burning through bandwidth to import stuff we already
saw. All this logic is shared with -watch but uses a different
pathname for lei since it's tied to lei/store (and not a
public-inbox).
|
|
We'll support nodatacow as we do in other SQLite DBs
|
|
Maildir and IMAP can both handle `forwarded'. Ensure we don't
lose `forwarded' when reading from stores which do not support
it, but ensure we can set it when reading from IMAP and Maildir
stores.
|
|
Code is the enemy, and there's no need to duplicate things, here.
There may be further opportunities along these lines to further
deduplicate things...
|
|
POSIX.pm shipped with Perl 5.16.3 did not support lround,
at least. So just rely on built-in core functions.
|
|
Our use of `ref' was triggering ambiguity in older versions of
the Perl parser.
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20210420174912.h6d2yv7zu5xr4yfc@nitro.local/
|
|
This may be useful for users to tweak search parameters.
This command is also the reason lei.saved-search is a git-config
file rather than JSON.
|
|
Readers may lose interest in subscription topics. This lets
them avoid clutter by forgetting a saved search.
This does not and will not destroy the contents of an --output
mailbox. In other words, this is similar to unsubscribing
from an Atom/RSS feed or NNTP group.
I've also decided we won't support 'mv-search', since it'll
probably be rarely used and "lei convert" can be used, instead.
|
|
Users may wish to update several saved searches at once. We can
support parallel updates in lei-daemon so users won't have to do
it themselves via xargs or similar.
Supporting IMAP outputs would be significantly more involved
since we'd have to pre-authenticate for every single IMAP
output before entering the redispatch loop.
|
|
We'll support editing the saved search config file, so user
errors may happen and we need to throw sensible errors in that
case.
|
|
It's slightly less code.
|
|
We don't support changing search terms once "lei q --save" is
used.
|
|
This necessitated fixing pause_dedupe to release the handle
used by ->lock_for_scope_fast, but otherwise no changes to
the LeiToMail package.
|
|
They're more different than alike, and having two separate
methods seems less confusing to me.
|
|
I don't know if it's worth it to sub (or super)class
PublicInbox::Config into something more generic for
lei, but this change simplifies a good chunk of lei
code that reuses the public-inbox config parsing.
|
|
While perl (5.28) doesn't complain about this, it's confusing to
my easily-confused mind.
|
|
This makes "lei q --save" as safe as "lei q" to prevent against
accidental data loss when clobbering an existing output,
|
|
It's conceivable a user will want to erase all previous
results but still rerun/refresh a search to get new results.
We probably won't support prune functionality, here, and
instead require explicit removal of saved searches.
|
|
Going forward, we'll probably support JSON for all the "ls-*"
subcommands. This also provides the basis for "lei up" shell
completion.
|
|
We'll keep it in PublicInbox::Config for now, since I'm not
sure if there's a better place to put it.
|
|
Repeated vfork+execv costing us around 20ms on t/lei-q-save.t,
so just learn to quote git-config values and write directly to
the config file.
|
|
We want to be able to use "lei up ." when inside a Maildir.
We'll also relax Maildir/mbox basenames to be any non-'/'
character after converting relative paths to absolute. The
old restriction on allowed characters was unnecessary and made
it impossible to reliably map "." when used as the sole argument
for "lei up".
|
|
We always represent --output destination directories with a
trailing slash to disambiguate directories from mbox filenames.
Therefore, we must use the trailing slash when mapping the
destination beck from the lei/saved-search/* directory.
"lei up" now relies exclusively on the users --output pathname
or URL for updates. This ought to be less confusing since
pathnames in ~/.local/store/lei/saved-searches aren't ideal.
|
|
Since "lei q" may read queries from stdin, we must reconnect a
known terminal before spawning terminal MUAs. Attempt to use
stdout as stdin for this purpose, since terminal MUAs tend to
expect stdout to be a terminal.
Reported-By: Kyle Meyer <kyle@kyleam.com>
Link: https://public-inbox.org/meta/87v98klxg3.fsf@kyleam.com/
|
|
A user may wish to clobber/refine existing search parameters
by issuing "lei q --save" again. Support that by overwriting
the lei.saved-search state file entirely.
We continue to preserve over.sqlite3 for deduplication purposes.
This way, we don't get something redundant like:
[lei]
q = term1
q = term2
q = term1
q = term2
q = term3
...whenever a user wants to refine their search. Instead,
we'll just have:
[lei]
q = term1
q = term2
q = term3
On the second go.
|
|
It is redundant since we stuff everything into the lei.q.output
config key.
|
|
Specifying a directory in ~/.local/share/lei/saved-searches/
is painful, so support (and start encouraging) the use of
the output.
|
|
We don't want pathnames with "GLOB(0xADD12355)" in them.
|
|
Since --stdin could be waiting on user keyboard input or
something else slow, we handle it in the event loop. That
means other commands can change the working directory of
lei-daemon while a query is being trickled to us via stdin.
Rearranging query handling internals to delay opening the
--output destination in commit 26e0fe73de93f451 meant
another command could throw off our --output pathname if
it is relative.
Fixes: 26e0fe73de93f451 ("lei_query: rearrange internals to capture query early")
|
|
We use it in t/lei-q-save.t, and were inadvertently writing
to the worktree.
v2: fix -C $DIR with TEST_RUN_MODE=0
|
|
NetReader->add_url supports URI-like objects, now. We'll be
relying on the canonicalization for LeiSavedSearch.
|
|
We want users to be able to edit and refine the query over
time while using the same output destination.
|
|
Since saved-searches aren't a part of lei/store, nor
could it be considered cache data... (or can it? it
is discardable, after all).
|
|
Somebody may want a saved search which consistently asks for
messages within a rolling time period window. In other words,
we want to support using "lei q --save dt:last.week.." and keeps
the "dt:last.week.." relative to whenever "lei up" is run. This
ensures relative date-time specifications get used in the future
rather than converting into an absolute date-time from the
initial "lei q" invocation.
|
|
If a user specifies "d:" with a higher precision than it was
traditionally able to handle, switch transparently to "dt:".
This lowers the learning curve and improves DWIM-ness.
v2: fix "d:YYYYMMDD..$NEEDS_APPROXIDATE" case
|
|
The command isn't finalized, yet, but it's intended to update
an existing saved search.
|