| Date | Commit message (Collapse) |
|---|
|
We'll be supporting some sort of text view for pager or
piping to an $EDITOR buffer.
|
|
This will be useful, later.
|
|
And the UID one, too, as well. This simplifies future
comparison and matching code once case is no longer
taken into account.
|
|
Merely redirecting the failure message from git to our stderr is
insufficient.
|
|
This gives "lei import", "lei tag", and similar commands
the ability to use URLs recognized by our PSGI frontend
directly.
This is more convenient than an equivalent shell pipeline
since "set -o pipefail" is not portable and errors may be
lost.
|
|
Don't attempt to return HTTP 300 via Extmsg on it,
since whoever uses /raw is likely piping it to some
other command.
|
|
This allows proper error reporting on daemon failure
when using "check-run".
|
|
We aren't using it, yet, but the plan is to be able to use
this information to propagate keyword changes back to IMAP
and Maildir folders using some to-be-implemented command.
"lei inspect" is a half-baked new command to make testing this
change easier. It will be updated to support more SQLite+Xapian
introspection duties in the future, including public-inbox
things independent of lei.
|
|
We'll be using the new class to efficiently propagate keyword
changes from lei/store back to Maildir or IMAP folders.
|
|
This will allow the callback to reliably maintain OID <=> UID
mappings between lei/store and the IMAP folder.
|
|
These will be useful for keyword synchronization, and perhaps
importing a single IMAP message with ->iuid.
|
|
"lei import" behavior will may change w.r.t. keyword
handling. Use separate $HOME between different test_lei
to ensure isolation between the tests.
|
|
This saves some work and makes it easier to set volatile
metadata on a message at import time.
|
|
On my default FreeBSD 11.x system, "/home" is a symlink to
"/usr/home", which causes "lei up" path resolution to fail when
I use outputs in $HOME. Fall back to a slow path of globbing
and matching pathnames based on st_ino+st_dev.
|
|
This is less surprising in case users are used to using --dedupe=
without --save.
|
|
We'll support this mode of operation for now to quiet down
testing of oneshot mode where the daemon doesn't persist.
|
|
Supporting --no-keywords and --no-flags aliases is harmful
if users end up assuming "keywords:" and "flags:" are valid
search prefixes (they're not).
|
|
No point in burning through bandwidth to import stuff we already
saw. All this logic is shared with -watch but uses a different
pathname for lei since it's tied to lei/store (and not a
public-inbox).
|
|
Maildir and IMAP can both handle `forwarded'. Ensure we don't
lose `forwarded' when reading from stores which do not support
it, but ensure we can set it when reading from IMAP and Maildir
stores.
|
|
"chmod 0000" on a Unix socket can't stop root from connecting to it;
so just skip the test for rare cases when testing as root.
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20210420213712.qfpftr2r543cqg7l@nitro.local/
|
|
Readers may lose interest in subscription topics. This lets
them avoid clutter by forgetting a saved search.
This does not and will not destroy the contents of an --output
mailbox. In other words, this is similar to unsubscribing
from an Atom/RSS feed or NNTP group.
I've also decided we won't support 'mv-search', since it'll
probably be rarely used and "lei convert" can be used, instead.
|
|
Users may wish to update several saved searches at once. We can
support parallel updates in lei-daemon so users won't have to do
it themselves via xargs or similar.
Supporting IMAP outputs would be significantly more involved
since we'd have to pre-authenticate for every single IMAP
output before entering the redispatch loop.
|
|
We have "lei import" and better test infrastructure for lei,
now, so we can more easily test SIGPIPE without relying on
an already-configured instance.
|
|
This necessitated fixing pause_dedupe to release the handle
used by ->lock_for_scope_fast, but otherwise no changes to
the LeiToMail package.
|
|
This makes "lei q --save" as safe as "lei q" to prevent against
accidental data loss when clobbering an existing output,
|
|
It's conceivable a user will want to erase all previous
results but still rerun/refresh a search to get new results.
We probably won't support prune functionality, here, and
instead require explicit removal of saved searches.
|
|
Going forward, we'll probably support JSON for all the "ls-*"
subcommands. This also provides the basis for "lei up" shell
completion.
|
|
We'll keep it in PublicInbox::Config for now, since I'm not
sure if there's a better place to put it.
|
|
We want to be able to use "lei up ." when inside a Maildir.
We'll also relax Maildir/mbox basenames to be any non-'/'
character after converting relative paths to absolute. The
old restriction on allowed characters was unnecessary and made
it impossible to reliably map "." when used as the sole argument
for "lei up".
|
|
We always represent --output destination directories with a
trailing slash to disambiguate directories from mbox filenames.
Therefore, we must use the trailing slash when mapping the
destination beck from the lei/saved-search/* directory.
"lei up" now relies exclusively on the users --output pathname
or URL for updates. This ought to be less confusing since
pathnames in ~/.local/store/lei/saved-searches aren't ideal.
|
|
Specifying a directory in ~/.local/share/lei/saved-searches/
is painful, so support (and start encouraging) the use of
the output.
|
|
Somebody may want a saved search which consistently asks for
messages within a rolling time period window. In other words,
we want to support using "lei q --save dt:last.week.." and keeps
the "dt:last.week.." relative to whenever "lei up" is run. This
ensures relative date-time specifications get used in the future
rather than converting into an absolute date-time from the
initial "lei q" invocation.
|
|
If a user specifies "d:" with a higher precision than it was
traditionally able to handle, switch transparently to "dt:".
This lowers the learning curve and improves DWIM-ness.
v2: fix "d:YYYYMMDD..$NEEDS_APPROXIDATE" case
|
|
The command isn't finalized, yet, but it's intended to update
an existing saved search.
|
|
This will have a over.sqlite3 for content-based deduplication.
It may exhibit ibxish methods, so serving a read-only (or even
R/W) IMAP or instance or displaying HTML isn't outside the realm
of possibility.
|
|
LeiSavedSearch will use a LeiDedupe-like internal API,
so we won't have to make as many changes to callsites
between saved and unsaved searches.
|
|
As they are likely Message-IDs. If an email address ends up in
a URL, then it's likely public, so there's even less reason to
obfuscate that particular address.
[km: add xt/perf-obfuscate.t]
[ew: modernize perf test (5.10.1), use diag instead of print]
This version of the patch avoids the massive slowdown noted by Kyle in
<https://public-inbox.org/meta/87wnt9or6t.fsf@kyleam.com/>.
Performance remains roughly the same, if not slightly faster
(which may be due to me testing this on a busy server). Results
from xt/perf-obfuscate.t against 6078 messages on a local mirror
of <https://public-inbox.org/meta/>:
before: 6.67 usr + 0.04 sys = 6.71 CPU
after: 6.64 usr + 0.04 sys = 6.68 CPU
Reported-by: Kyle Meyer <kyle@kyleam.com>
Helped-by: Kyle Meyer <kyle@kyleam.com>
Link: https://public-inbox.org/meta/87a6q8p5qa.fsf@kyleam.com/
|
|
We'll eventually want lei_input users like "lei import" and
"lei tag" to support parallel reads.
|
|
It currently conflicts with the way OverIdx and SearchIdx
index messages, ultimately leading to violating a NOT NULL
constraint on id2num.id in over.sqlite3.
We may allow searching Resent-* fields separately, though I'm
not sure how useful it'll be.
|
|
We need a stable fallback time for digest2mid in the presence
of messages without Received/Date headers. Furthermore, we
must avoid using uninitialized smsg->{mid} when parsing
References for draft replies.
|
|
We need net_merge_all and to lock the number of worker jobs.
Parallel inputs are not supported, yet (is it needed?, I don't
expect this to be used for multiple files very often...).
|
|
It's a bit of an Easter egg, though it's not possible to hide those
in Free Software... Anyways, it doesn't cost us an entry in %CMD
of LEI.pm and anybody frustrated enough with lei just might type
"lei sucks" on the command-line :>
|
|
Thanks to git-describe, we can generate and update this
file to aid users in bug reporting.
|
|
Assume a user specifying --mail doesn't want to spend cycles
reconstructing a blob from a code repo. Also, don't require
users to use add-external or a previous -I or --only to ready an
external for use with ale.git.
|
|
I've decided "tag" is a better verb since it seems more
widely-used term for associating metadata with data.
Not only is it analogous to the "notmuch tag" command, but
also makes sense when compared to tooling for manipulating
metadata for non-mail data (e.g. audio metadata tags).
There's even a Wikipedia entry for it:
https://en.wikipedia.org/wiki/Tag_(metadata)
whereas "mark" is used in the description, but has no
entry of its own with regards to metadata.
|
|
Since "lei q" and "lei convert" already support writing these
compressed inboxes, it makes sense that all mbox readers support
them, as well.
Using compression is one reliable way to know an mboxrd or mboxo
hasn't been unexpectedly truncated.
|
|
".eml" is a suffix supported by (/usr/local)/etc/mime.types
on Debian and FreeBSD systems using the "mime-support" package.
".patch" is what "git format-patch" generates by default since
git v1.5.0 in 2007.
|
|
It's slightly easier to test on a machine without Test::More
and I'm hand-rolling my own is() and ok() subs.
|
|
File::Temp only requires four 'X' characters (unlike mkstemp(3),
which requires six). So only so only give it 4 to avoid an
80-column violation and maybe save metadata space on FSes.
|
|
Introduce a new LeiRemote wrapper to provide an internal API
which SolverGit expects. This lets us use HTTP/HTTPS endpoints
to reconstruct blobs off patches as we would with local
endpoints, just more slowly...
|