| Date | Commit message (Collapse) |
|---|
|
Introduce a new LeiRemote wrapper to provide an internal API
which SolverGit expects. This lets us use HTTP/HTTPS endpoints
to reconstruct blobs off patches as we would with local
endpoints, just more slowly...
|
|
This is needed for SolverGit, at least, and maybe other
bits we share with PSGI in lei.
|
|
Most of it already gets tested since most of the logic is in
SolverGit, but make sure it's all wired up properly to lei.
|
|
It's possible for a abbreviated OID to be resolved unambiguously
to an email before we attempt to look at externals via xsearch;
so provide a way for a user to force searching coderepos.
If hints (--oid-a, --path-a, --path-b) are present, we'll
assume --no-mail by default, otherwise we'll assume the
user wants to look through mail for a matching blob.
|
|
This makes it easier to manage test dependencies on systems
where optional stuff isn't installed. This fixes some lei tests
which didn't check for Plack before starting -httpd, and ensures
Parse::RecDescent is available for -imapd in case
Mail::IMAPClient stops using it.
|
|
It seems safe to use ALLCAPS labels like "INBOX" with Xapian.
We'll also allow single-character labels.
|
|
This implements blob reconstruction via SolverGit,
emulating the functionality of /$INBOX/$OID/s/ endpoint
in PublicInbox::WWW.
It uses the current working tree as a coderepo, and
accepts any number of --git-dir=$PATH args.
Remote externals are not yet supported.
v2: use absolute path for git repos
|
|
Since lei-daemon won't have the same FDs as the client, we
need to special-case thse mappings and won't be able to open
arbitrary, non-standard FDs.
We also won't attempt to support /proc/self/fd/[0-2] since
that's a Linux-ism. /dev/fd/[0-2] and /dev/std{in,out,err}
are portable to FreeBSD, at least. mawk(1) also supports
/dev/std{out,err}, as does gawk(1) (which supports everything
we can support, and arbitrary /dev/fd/$FD).
|
|
I keep copy-pasting the addresses provided,
I keep writing my plaintext reply in a file,
and I keep forgetting to add a subject
(because I am "just" writing a plaintext file)
Teach `git-send-email-reply` to append a `--subject` line.
[ew: avoid URI-encoded subject on command-line, adjust t/reply.t]
Signed-off-by: Stavros Ntentos <133706+stdedos@users.noreply.github.com>
|
|
"lei q" now displays labels in JSON output, "lei mark"
can add or remove labels for any messages.
"lei ls-label" is supported, too.
Unfortunately, "lei q" won't hande "kw:" or "L:" for
external messages, they must be imported, first.
|
|
This seems to error out while looping the test suite and
I'm not 100% sure why.
|
|
Perl can't check for interrupts when inside a blocking syscall,
as there's no self-pipe mechanism inside Perl itself. So fork
a child and have it repeated kill(2) instead of relying on alarm(3).
|
|
"lei import" should never be without a {sto}, and *_done should
not be called multiple times, so ensure we can fail if it's
missing.
Update some existing tests to complain loudly by introducing a
handy "xbail" function which wraps "explain" and BAIL_OUT.
BAIL_OUT was painful to type and concatenating the result of
"explain" doesn't work as I thought it would since "explain"
always returns an array, and BAIL_OUT only accepts a single
scalar arg (unlike "die").
|
|
We'll use a lei-daemon if it's already running and
TEST_LEI_DAEMON_PERSIST_DIR is set, but we can also start
one and manage it from t/run.perl
This drops "make check-run TEST_LEI_DAEMON_ONLY=1"
time by ~10% for me.
|
|
Outside of mirrors, we can -init with indexlevel to avoid
calling -index explicitly.
|
|
While we were exiting with a error code, showing a successful
"# mirrored $URL" message is misleading and wrong. Don't show
success until everything is complete and the config is written.
|
|
There's no need to create or write lei/store when adding
an external, we just need to write to the config file.
|
|
If a client passes us the incorrect number of FDs, we'll vivify
them into PerlIO objects so they can be auto-closed. Using
POSIX::close was considered, but it would've been more code to
handle an uncommon case.
|
|
I'm not sure exactly why this is needed with run_script
localizing %SIG and everything else, but explictly cleaning up
seems to fix the occasional test failures I see.
Followup-to: 4c6c853494b49368 ("tests: show lsof output on deleted-file-check failures")
|
|
Since lei-daemon will fchdir on every request, we must ensure
we're in the correct directory before unlink(2) is called,
since we can't use unlinkat(2) from pure Perl.
|
|
Only tested for keywords and labels with file inputs, so far;
but it seems to do what it needs to do. There's a bit more
redundant code than I'd like, and more opportunities for code
sharing in the future
"lei import" will be expanded to support +kw:$KEYWORD and
+L:$LABEL in the future.
|
|
Those headers only have meaning with for mboxes. Don't surprise
users by trying to make sense of a header that is defined for mboxes.
It's possible to send email with (Status|X-Status) headers and
have those headers show up in a recipient's IMAP mailbox.
This was bad because an IMAP user may want to import a single
message through their MUA and pipe its contents to "lei import"
without noticing a mischievious sender stuck "X-Status: F"
(flagged/important) in there.
|
|
This matches the long-standing behavior of public-inbox-mda,
public-inbox-learn and our other tools. It is useful because
mutt, "git format-patch", and likely other tools will
pipe a single message with a "From " header line, but with
no further "From " escaping or Content-Length: header.
|
|
It's a bit nasty, but seems to mostly work for debugging
IMAP and NNTP commands.
|
|
This will let us tie keywords from remote externals
to those which only exist in local externals.
|
|
Keyword storage for external-only messages was preventing
messages from being explicitly imported. Teach lei_store
to vivify keyword-only entries into fully-indexed messages
on import.
|
|
Stop showing `docid' since it's not useful with shards.
`bytes' and `lines' are probably noise, but maybe could be
visible in some "fuller" view.
v2: t/lei_xsearch: fix warnings from {docid} removal
|
|
This should make a future change to "lei import" work more
nicely, since we'll be needing ALE to vivify external-only
messages upon explicit "lei import".
|
|
These changes may make it easier to do byte-for-byte comparisons
with mail copied out of mutt, a popular MUA for our target
audience.
mutt currently outputs the 'R' (seen) flag before the 'O'
character in the Status: header. We'll assume that stays
the case (it has been for a while).
Status now comes before X-Status, also matching mutt behavior.
|
|
Don't waste precious terminal space when there are only a small
number of possible keywords supported/reserved for JMAP. In the
future, we may implement more sophisticated wrapping for labels,
but it we'll cross tha bridge when we come to it.
|
|
"lei q" now preserves changes per-message keywords across
invocations when it's --output (Maildir or mbox) is reused
(with or without --augment).
In the future, these changes will be monitored via inotify,
EVFILT_VNODE or IMAP IDLE, too.
Unfortunately, this currently prevents "lei import" from ever
importing a message that's in an external. That will be fixed
in a future change.
|
|
This will be used for keyword (and label) storage for externals.
We'll be using this to ensure we don't redundantly auto-import
messages into lei/store if they're already in a local external
(they can still be imported explicitly via "lei import").
|
|
Assume that anybody using -I/--include for external locations
will want to override --no-$FOO if they're explicitly including
a location.
With some effort, we could make it order-dependent (e.g.
"-I $LOCATION --no-$FOO" and "--no-$FOO -I $LOCATION"
behave differently). However that's not straightforward
when using Getopt::Long to parse command-line options into
a hashref.
I'm also not sure if order-dependent switches are a desirable
UI/UX quality.
|
|
git 2.11 and earlier could not handle git directories with
newlines in them, nor does libgit2 support them.
Followup-to: d87dd0e679587043 ("config: reject `\n' in `inboxdir'")
|
|
Since keywords and mailboxes (AKA labels) are separate things in
JMAP; and only keywords can map reliably to Maildir and mbox;
we'll keep them separate in our internal data representations,
too.
I initially wanted to call this just "meta" for "metadata", but
that might be confused with our mailing list name. "metadata"
is already used in Xapian's own API, to add another layer of
confusion.
"tags" was also considered, but probably confusing to notmuch
users since our "labels" are analogous to "tags" in notmuch,
and notmuch doesn't seem to cover "keywords" separately...
So "vmd" it is, since we haven't used this particular
three-letter-abbreviation anywhere before; and "volatile" seems
like a good description of this metadata since everything else
up to this point has been mostly WORM (write-once, read-many).
|
|
This may help track down some occasional test failures I'm
seeing.
|
|
Extsearch objects are duck-types of Inbox objects, and
are capable of supporting code repos all the same.
|
|
We'll try to share a bit more configuration with
extindex entries for WWW PSGI usage.
|
|
It's redundant and the same functionality is in MdirReader.
|
|
MboxReader is a more appropriate place for it than LeiStore.
|
|
Using create_inbox doesn't seem worth the trouble, here, at the
moment, but disabling fsync(2) gives a noticeable speedup on
my system even with an SSD.
|
|
Another 100ms saved
|
|
Yes, this loses some randomness w.r.t. shuffle, but I
think it's "good enough" and buys us ~100ms speedup on
an SSD TMPDIR.
|
|
|
|
Another 100ms or so saved.
|
|
|
|
|
|
|
|
This saves hundreds of milliseconds and reduces LoC.
|
|
This ends up being significantly faster when confined
to a slow TMPDIR.
|