| Date | Commit message (Collapse) |
|---|
|
git always quotes with leading zeros to ensure the octal
representation is 3 characters long. We enforce that to match
low ASCII characters (e.g. [x01-\x06]) that don't need the
range provided by 3 characters.
git_unquote now does a single pass so it won't get fooled by
decoded backslashes into parsing a digit as an octal character.
git_unquote is also capped to "\377" so we don't overflow a
byte.
|
|
We already use PublicInbox::DS in this test and I've always
found the terminology of sig* APIs confusing :x
|
|
FreeBSD (and other *BSDs) do not have stable syscall numbers, so
drop no-op checks for it and add a note to use Inline::C,
instead. Drop an implicit return for the syscall.ph loading
while we're at it, too.
On Linux, epoll_create(2) ignores the size arg since Linux
2.6.8, so just hard code it to some non-zero value.
On a side note, we can probably drop epoll_create(2) support
soon and just use epoll_create1(2) which appeared in 2.6.27+
(2008-10-09). Our userspace (Perl and git) requirements are
already further ahead.
|
|
We must not accumulate mset totals for messages which
have already been counted. Furthermore, the combined
search was being passed an extra arg and causing the
total to go missing.
|
|
We use trailing slashes internally, but should not increase
visual noise for users by exposing them in config files or
DB storage (and shell completion/listings).
This fixes a long-standing bug in $lei->rel2abs that prevented
absolute paths from being canonicalized.
|
|
We don't need Xapian positional info when searching
for blob pre/post-images. Furthermore, rediff will
usually be used for a single email or at most, one
patchset. So there's little point in parallelizing
or having multiple shards.
|
|
We can use a temporary lei/store to avoid cluttering up
future search results. This is especially useful since
we expect "lei rediff" to be useful for non-email diffs
and individual attachments, too.
|
|
Don't lose file mode information when regenerating a diff.
|
|
Everything else that's intended to be executable at some
point has the executable bit set. Remove an inaccurate
comment while we're at it.
|
|
Normal git retrieval don't work for Maildir blobs indexed using
"lei index". Fortunately, this oddness is limited to the
LeiStore class and we can override smsg_eml with a fallback
to read blobs from Maildirs.
|
|
Sometimes a mailed patch is generated with non-ideal output,
(lacking context, noisy whitespace changes, etc.), or a user
wants to use the same external diff viewer they've configured
git to use.
Since we have SolverGit to regenerate arbitrary blobs from
patches; this new command allows us to regenerate a diff with
different options using the blobs SolverGit gives us.
The amount of git-diff(1) options is mind numbing, so it's
likely I missed some favorites or botched the getopt spec
translation.
This also fixes Inbox::base_url to check psgi.url_scheme
before attempting to generate URLs and avoid uninitialized
variable warnings. Oddly, the "lei blob" tests did not
trigger these uninitialized warnings.
Note: this will automatically import+index the message(s)
it's regenerating, because solver relies on being able
to lookup pre/postimage OIDs and read blobs.
|
|
Since completely purging blobs from git is slow, users may wish
to index messages in Maildirs (and eventually other local
storage) without storing data in git.
Much code from LeiImport and LeiInput is reused, and a new dummy
FakeImport class supplies a non-storing $im->add and minimize
changes to LeiStore.
The tricky part of this command is to support "lei import"
after a message has gone through "lei index". Relying on
$smsg->{bytes} == 0 (as we do for external-only vmd storage)
does not work here, since it would break searching for "z:"
byte-ranges when not using externals.
This eventually required PublicInbox::Import::add to use a
SharedKV to keep track of imported blobs and prevent
duplication.
|
|
We should not have "SCALAR(XXXXXXX)" showing up in SQLite DBs
because we passed a SCALAR ref instead of a non-ref SCALAR.
|
|
If lei_ls_mail_sync() is given a filter without any wildcards
and --globoff is unspecified, glob2re() will return undef,
resulting in the final regular expression being undefined.
Always use a fallback value when there's no RE.
Based-on-patch-by: Kyle Meyer <kyle@kyleam.com>
Link: https://public-inbox.org/meta/20210504044559.12941-5-kyle@kyleam.com/
|
|
ls-external and ls-mail-sync accept an --invert-match option. Show it
in the --help output.
|
|
lei_ls_mail_sync() is written to accept a filter, and ls-mail-sync has
related command-line options (--globoff, --invert-match), but a
positional argument isn't actually accepted. Add it.
|
|
|
|
ls-sync was renamed to ls-mail-sync in cb0e9d42b799c748. Update a
stale reference to the old name.
|
|
This failure was also passing under FreeBSD make + /bin/sh;
so we also avoid the '&&' chain is avoided and use '>$@' as a
separate line in the Makefile.
|
|
v2 onions are insecure, deprecated and going away. v3 names are
unfortunately longer and more difficult to remember, but should
be more resistant to attack than v2 ones.
|
|
LeiToMail Maildir and IMAP write callbacks need to account for
the caller-supplied smsg. We'll also make better use of the
user-supplied smsg object by ensuring blob deduplication happens
ASAP.
Fixes: e76683309ca4f254 ("lei <q|up>: distinguish between mset and l2m counts")
|
|
This will allow keyword updates from other folders to propagate
to folders where search results may be duplicated.
|
|
Instead of "+FLAGS.SILENT" which merely adds to the keywords.
We store all keywords together, so it's unlikely we will rely
on the "+FLAGS.SILENT" or "-FLAGS.SILENT".
|
|
In most cases, we just name the worker process based
on the command. The only change is for LeiMirror
vs "lei add-external --mirror", but I doubt it matters.
|
|
I'm not sure how we'll distinguish JMAP vs read-only HTTPS,
yet; but we'll focus on currently-supported stuff, first.
|
|
I suspect there'll be more lei_input-only things in the future.
|
|
Having multiple lines of output mean they can be interleaved in
daemon mode. Put stats into one line to reduce screen
real-estate size and improve readability.
|
|
We use the "done" term elsewhere for similar things, and
my easily-confused mind equates "complete" with shell
completion.
|
|
The contents of the old lei.q.output will not be removed,
but will be converted into the new one.
|
|
"lei inspect" also shows "mail-sync" as a field name
|
|
This allows tab-completion for "ls-search" to work with fewer
characters ("ls-s<TAB>" instead of "ls-se<TAB>"), and I expect
"ls-search" to be used more frequently than "ls-mail-sync".
This also matches the --mail-sync switch of "lei import"
|
|
This was harmless, but ugly and possibly confusing to
users who run "lei edit-search".
|
|
The number of messages we write to --output is usually different
than the mset count due to deduplication from combining multiple
sources.
This change makes the stderr output of "lei up --all=local" way
more useful IMHO.
|
|
These tests require a running Tor instance (defaulting to
127.0.0.1:9050) and Internet connectivity, but otherwise
work pretty well.
|
|
This allows us to use URL-matching config in git and specify
proxies on a per-host basis. git 2.26+ users may use wildcards
to enable Tor (on 127.0.0.1:9050) for all NNTP and IMAP .onion
domains.
My ~/.config/lei/config file has the following:
[imap "imap://*.onion"]
proxy = socks5h://127.0.0.1:9050
[nntp "nntp://*.onion"]
proxy = socks5h://127.0.0.1:9050
|
|
Since Net::NNTP doesn't support Socket or RawSocket
options/accessors like Mail::IMAPClient does; we must perform
localized @ISA manipulation and massage Net::NNTP into using
IO::Socket::Socks rather than IO::Socket::IP.
This is a bit fragile, but Net::Cmd and Net::NNTP rarely change;
and I keep an eye on them, anyways.
|
|
Mail::IMAPClient provides the ability to pass a pre-connected
Socket to it. We can rely on this functionality to use
IO::Socket::Socks in place whatever socket class
Mail::IMAPClient chooses to use.
The --proxy=s is shared with curl(1), though we only support
socks5h:// at the moment. Is there any need for SOCKS4 or SOCKS5
without name resolution? Tor .onions require socks5h:// for
name resolution and to prevent data leakage.
|
|
We don't use these subs elsewhere, so stick prototypes on them
to give them a little extra checking.
|
|
This fixes error reporting for oneshot tests in xt/lei-auth-failure.t
|
|
This ensures hitting Ctrl-C on a long-running "lei convert" or
similar will stop the WQ worker, even after we've closed
the WQ socketpair in the daemon.
|
|
LD_PRELOAD sent by a client can't affect lei-daemon.
|
|
It's helpful for us to distinguish x86 kernels from x86_64
kernels when using an x86 userspace. OSes are dropping i386
support and only support i486 and newer, so "x86" is a more
appropriate description for that platform than "i386".
|
|
Prior to this change, it was possible for oneshot lei processes
to race on epoch creation/rollover. lei-daemon normally
prevents the problem by funnelling all writes to a single
socket, but oneshot lei has no such protection.
|
|
This also fixes completion of "lei up" for IMAP folders.
|
|
Specifying a UIDVALIDITY value allows the user to enforce
a strict match and force failure. This necessitated changes
to NetReader to allow die() and make error reporting more
suitable for CLI usage rather than daemonized usage of -watch.
|
|
IMAPTracker has a UNIQUE constraint on the `url' column,
which may cause compatibility and/or rollback problems
in attempting to deal with UIDVALIDITY changes.
Having multiple sources of truth leads to confusion and bugs,
so relying on LeiMailSync exclusively ought to simplify things.
Furthermore, since LeiMailSync is only written to by LeiStore,
it is safer in that it won't mark a UID or article as imported
until git-fast-import has seen it, and the SQLite commit always
happens after "done\n" is sent to fast-import.
This mostly reverts recent commits to IMAPTracker to support
lei, those are:
1) commit 7632d8f7590daf70c65d4270e750c36552fa9389
("net_reader: restart on first UID when UIDVALIDITY changes")
2) commit 311a5d37ad275cd75b1e64d87827c4d13fe4bfab
("imap_tracker: prepare for use with lei").
This means public-inbox-watch will not change between 1.6 and
1.7: -watch stops synching a folder when UIDVALIDITY changes.
|
|
This will be convenient to avoid the overhead of
PublicInbox::Eml for verifying synchronization in lei.
|
|
This seems to fix the occasional "make check-run" failures I've
been chasing.
Some parts of our code assumes we can close($lei->{1})
and similar, which causes IO::Handle::autoflush to behave
badly when STDOUT is the "select"-ed FH of the Perl process.
Since oneshot mode is (hopefully) the uncommon case, we'll
just accept the cost of extra FDs and minimize differences
between lei in oneshot vs daemon mode.
|
|
Knowing which process ran what before dying unexpectedly
helps with debugging.
|
|
This should be familiar to git users who wish to force color
when writing to pipes or disable color.
|