Date | Commit message (Collapse) |
|
->newsgroup_matches was never used, and ->shard_over_check
was dropped in 89193578d21f (extindex: --gc checkpoints, 2021-10-06).
|
|
Future-proofing in case future versions of Perl warn on this, since
2-arg forms of open may be subject to injection vulnerabilities
with non-literal args.
|
|
This is future-proofing in case we build against Xapian directly
in the future, which would require a C++ compiler.
|
|
This fixes the "Modification of a read-only value attempted at ..."
error in an initial run of t/reindex-time-range.t. It was
reproducible by running `rm -rf t/data-gen/reindex-time-range.v*'
before `make && prove -bvw t/reindex-time-range.t'. Thanks to
Jörg Rödel for providing the backtrace which helped find this.
Debugged-by: Jörg Rödel <joro@8bytes.org>
Link: https://public-inbox.org/meta/YZuZEY+WSnm4wlrS@8bytes.org/
|
|
This enables tab-completion, since I'm using --prune quite a bit
and my fingers are about to fall off :<
|
|
Following commit 57fed2e4b78ed394 (lei: normalize whitespace in
remote queries, 2021-09-11), leaving the trailing `\n' from
stdin queries to be normalized to ` ' (SP) causes it to appear
as `+' in URLs, which Xapian ignores.
|
|
I don't expect this to be hit in real-world use via normal
interactive shells. However, somebody could accidentally add
"\n" in languages (e.g. Perl, C) where it's easy to pass "\n"
in argv[].
|
|
For --stdin searches created prior to commit 666dde69a3f6 (lei
q|up: fix saved searches for single-phrase search, 2021-11-08)
we still want to be able to run "lei up" on them without
regressions. So assume nobody manages to enter "\n" as an
argv[] element and consider the presence of "\n" as a previous
--stdin use.
This fixes errors from "lei up" such as:
lei_xsearch 2 wq_worker: Exception: Key too long: length was 840 bytes,
maximum length of a key is 255 bytes at ../PublicInbox/IPC.pm line 250.
Fixes: 666dde69a3f6 ("lei q|up: fix saved searches for single-phrase search")
|
|
Hopefully problems can get diagnosed more quickly with
the sub name in the error message.
|
|
Tested manually on a newish project I'm working on.
|
|
`"' (double-quote) needs to be quoted for stdin searches.
We also need to differentiate between "lei q --stdin" usage
when calling "lei up", do it by setting an internal "rawstr"
knob to ensure we can parse the config properly regardless
of whether the initial search used --stdin or not.
|
|
While we do detailed indexing of git diffs, the header itself
was failing and queries like 'nq:diff' would not work.
Noticed-by: Rob Herring <robh@kernel.org>
|
|
Using the --proxy on the command-line affects the entire
lei invocation, and users searching HTTP(S) remotes and
writing to an IMAP folder may want more fine-grained proxy
use:
lei q -o imap://no-proxy.example/foo -O https://need-proxy.example/bar ...
|
|
The new Documentation/common.perl file will be used for
all manpages in the future.
|
|
These commands are usually run on a single message, so saving
the user the trouble of typing `-F eml' on the command-line
seems reasonable. I don't think commands like "index" and
"import" will be too useful for single messages, though.
|
|
This method replaces a common pattern of starting workers,
preparing internal auth ops, and asynchronous waiting of
command completion.
It also adds missing LeiAuth support to rediff and rm
which rarely need auth.
|
|
In retrospect, this doesn't make sense, since it needs at least
two messages to diff. So go about "normal" input rules and
require users to specify the format.
|
|
FreeBSD gzip does not support --rsyncable, though my VM
usually has pigz installed.
|
|
As stated in the previous change, conditional hash assignments
which trigger other hash assignments seem problematic, at times.
So replace:
$h->{k} //= do { $h->{x} = ...; $val };
$h->{k} // do {
$h->{x} = ...;
$hk->{k} = $val
};
"||=" is affected the same way, and some instances of "||=" are
replaced with "//=" or "// do {", now.
|
|
I've been seeing the following error on occasion during "make check-run":
$PWD/t/data-gen/reindex-time-range.v1-master index failed: Modification of a read-only value attempted at $DIR/lib/PublicInbox/SearchIdx.pm line 899, <$r> line 1.
Perhaps this fixes it. In any case, a construct of:
$h->{k} //= do { $h->{x} = ...; $val };
seems wrong and may cause Perl to error out depending on how
hashes are randomized.
|
|
Xapian boolean terms rely on upper-case prefixes, so the terms
themselves need to be all lowercase.
|
|
->DESTROY ordering via "exit()" calls is tricky, and dedupe
checks were causing problems.
AFAIK, this only affects users who manually enable WAL on
lei/store/ei*/over.sqlite3. Fortunately, there is no data
corruption as a result even though "read-only" WAL requires
write permissions.
|
|
SIGPIPE and SIGTERM are common and user-induced, so they're
not worth warning on. Add the value of "$?", though, since
it can help users notice other errors (e.g. SIGSEGV).
|
|
"text" and "reply" outputs are intended for the pager, so
parallelizing them is a waste of resources.
v2 has shards, of course, so parallelizing writes to it
is also a waste since the deduplication work is a bit
more complex.
|
|
It's possible for this to break out of the event loop if
note_sigpipe fires via PktOp in the same iteration.
|
|
We don't want to read a users'
$XDG_CACHE_HOME/lei/all_locals_ever.git during tests.
Reported-by: Thomas Weißschuh <thomas@t-8ch.de>
Tested-by: Thomas Weißschuh <thomas@t-8ch.de>
Link: https://public-inbox.org/meta/f239abac-4aee-4573-a0d6-e533c7a32662@t-8ch.de/
|
|
It's not much of a savings, right now, but maybe it can be in the
future. I wanted to eliminate the "lei convert" one, too, but
convert needs to preserve keywords which isn't possible with the
generic fallback, so new tests were written for convert, instead.
|
|
Some bugs are triggered with more CPUs, some with 1 CPU.
|
|
Use the one provided by the LeiInput parent class.
|
|
The extra dashes should help users find the correct option
more easily.
|
|
I just did a double-take and nearly thought authentication
was broken while reading LeiConvert.pm. Add a comment in
LeiConvert.pm to clarify things, too.
|
|
This prevents noisy errors in syslog when running t/lei-watch.t
|
|
We need to update the {-nr_remote_eml} counter regardless
of progress display being enabled since it's needed for
saved searches. We'll also split out the {-imported} flag
separately and only call LeiStore->done if a new message
was imported.
Note: this change is NOT expected to fix errors reported by
Thomas in <ebf92218-1470-4602-b534-6dae59639dc6@t-8ch.de>
Cc: Thomas Weißschuh <thomas@t-8ch.de>
|
|
This lets users change their global init.defaultBranch config
knob in ~/.gitconfig or similar without breaking tests.
Reported-by: Thomas Weißschuh <thomas@t-8ch.de>
Tested-by: Thomas Weißschuh <thomas@t-8ch.de>
|
|
Support --in-format like the rest of LeiInput users, and don't
default to .eml if a per-input format was specified. In any
case, I saved a bunch of messages from mutt which uses mboxcl2.
We'll also split newlines for diff, since it's a pain to read
diffs with escaped "\n" characters in them.
|
|
Mail synchronization in lei_to_mail only works for IMAP and
Maildir; so don't waste time preparing mbox* writers for it.
|
|
It's possible for ->event_step to fire twice due to ->requeue
with EPOLLET (but not EPOLLONESHOT). So account for that and
avoid causing event loop errors as a result.
|
|
These are supplied by the base LeiInput class
|
|
The LeiInput backend now allows p2q to work like any other
command which reads .eml, .patch, mbox*, Maildir, IMAP, and NNTP
input. Running "git format-patch --stdout -1 $COMMIT" remains
supported.
This is intended to allow lower memory use while parsing
"git log --pretty=mboxrd -p" output. Previously, the entire
output of "git log" would be slurped into memory at once.
The intended use is to allow easy(-ish :P) searching for
unapplied patches as documented in the new example in the
manpage.
|
|
All of these commands should support --proxy, at least, if not
other curl options.
|
|
The misnamed sub wasn't firing, but was unlikely to be
noticeable given the short lifetime of the process.
Fixes: 1f887bd51d92b0d4 ("lei inspect: add atfork hook")
|
|
FLAG_PURE_NOT is too expensive for public-facing WWW use, but
lei isn't public-facing. We'll also unconditionally enable
phrase search on old "chert" DBs since lei doesn't need to
worry about fairness across 10K users.
|
|
This easily allows us to treat "git diff" output as header-less
"messages" for commands such as "lei p2q".
|
|
This is useful for users lacking in local storage. Also,
referencing lei-add-external(1) seems to make less sense
than referencing lei-q(1).
We'll also start dropping years from the copyright statement
to reduce future churn.
|
|
As of commit 738c4a65, the code for reporting NNTP information in
_/text/mirror/ incorrectly uses ->imap_url rather than ->nntp_url.
Fixes: 738c4a65719e6278 ("www: various help text updates")
|
|
No need to go through the lei/store process when we write
mail_sync.sqlite3. This ought to reduce ENOBUFS errors (and the
sleep workaround) on RAM-starved systems.
|
|
By using the charset specified in the message, web browsers are
more likely to display the raw text properly for human readers.
Inspired by a patch by Thomas Weißschuh:
https://public-inbox.org/meta/20211024214337.161779-3-thomas@t-8ch.de/
Cc: Thomas Weißschuh <thomas@t-8ch.de>
|
|
This will let us modify the response header later to set
a proper charset for Content-Type when displaying raw
messages.
Cc: Thomas Weißschuh <thomas@t-8ch.de>
|
|
Just let Plack::Util::run_app catch the error and generate
a 500 response for it.
|
|
Creating a scalar ref directly off substr() seemed to be causing
the underlying non-ref scalar to end up in Perl's scratchpad.
Assign the substr result to a local variable seems sufficient to
prevent multi-megabyte SVs from lingering indefinitely when a
read-only daemon serves rare, oversized blobs.
|