| Date | Commit message (Collapse) |
|---|
|
`-d' seems like a non-brainer for --dir with inspect.
I find myself using `--only' a bit, too, and `-O' seems like
a reasonable shortcut for it.
|
|
This allows -fetch to work out-of-the-box on using the
grokmirror 2.x default of "_grokmirror".
|
|
|
|
|
|
This mode only checks history for missed/stale messages
and doesn't attempt to reindex messages which are already
indexed.
|
|
This lets administrators reindex specific time ranges
according to git "approxidate" formats. These arguments
are passed directly to underlying git-log(1) invocations
and may still reach into old epochs.
Since these options rely on git committer dates (which we infer
from the most recent Received: header), they are not guaranteed
to be strictly tied to git history and it's possible to
over/under-reindex some messages. It's probably not a major
problem in practice, though; reindexing a few extra messages
is generally harmless aside from some extra device wear.
Since this currently relies on git-log, these options do not
affect -extindex, yet.
|
|
In case users see "lei-daemon" in ps(1) or syslog and wonder.
Helped-by: Kyle Meyer <kyle@kyleam.com>
|
|
Virtual users will probably be used for read-write IMAP/JMAP
support. The potential for various kernel/hardware bugs and
attacks also needs to be highlighted.
|
|
|
|
More switches which can be useful for users who pipe from text
editors. --drq can be helpful while writing patch review email
replies, and perhaps --dequote-only, too.
|
|
Only the ->message_string method of Mail::IMAPClient uses it,
and we have no intention of using ->message_string outside
of tests.
|
|
-F is really only useful for distinguishing between mbox
variants and single message/rfc822 files. URLs and
directory-based formats can be auto-detected easily enough.
|
|
Something is better than nothing.
|
|
lei-index really only works for Maildir, at the moment.
|
|
Implicit stdin based on standard input being a pipe or regular
file is here to stay, so save users the trouble of typing '-'
or '--stdin'.
Inline::C is required as of commit 1d6e1f9a6a66 (lei: require
Socket::MsgHdr or Inline::C, drop oneshot, 2021-05-26); but
Socket::MsgHdr still gives a noticeable improvement in bash
completion speed.
Also, spell-out "MESSAGE-ID" since "MID" is actually not a
common abbreviation ("MSGID" is used by RFC 3977 and several
other RFCs, I recall).
|
|
Try to clarify these commands are intended to be useful for
git-using (usually software) projects (and not the bare git
repos we use internally).
We'll also document some commonly useful git-diff switches
in the lei-rediff man page to highlight the usefulness
of the command.
|
|
Partial (v2) clones should be useful addition for users wanting
to conserve storage while having fast access to recent messages.
Continuing work started in 876e74283ff3 (fetch: ignore
non-writable epoch dirs, 2021-09-17), this creates bare,
read-only epoch git repos. These git repos have the remotes
pre-configured, but does not fetch any objects.
The goal is to allow users to set the writable bit on a
previously-skipped epoch and start fetching it.
Shell completion support may not be necessary given how short
the epoch ranges are, here.
Cc: Luis Chamberlain <mcgrof@kernel.org>
Link: https://public-inbox.org/meta/20210917002204.GA13112@dcvr/T/#u
|
|
Avoid slurping gigantic (e.g. 100000) result sets into a single
response if a giant limit is specified, and instead use 10000
as a window for the mset with a given offset. We'll also warn
and hint towards about the --limit= switch when the estimated
result set is larger than the default limit.
|
|
Lots of stuff out there that becomes a pain to setup
configuration for and test...
|
|
It's still a work-in-progress, but the basic debug knob
comes in handy for new users; as does proxy support.
|
|
git 2.33+ contains important optimizations for the
thousands-of-inboxes case. And combine the Inline::C stuff
with libgit2, since our use of libgit2 requires Inline::C.
|
|
Since we can't use maxuid for remote externals, automatically
maintaining the last time we got results and appending a dt:
range to the query will prevent HTTP(S) responses from getting
too big.
We could be using "rt:", but no stable release of public-inbox
supports it, yet, so we'll use dt:, instead.
By default, there's a two day fudge factor to account for MTA
downtime and delays; which is hopefully enough. The fudge
factor may be changed per-invocation with the
--remote-fudge-factor=INTERVAL option
Since different externals can have different message transport
routes, "lastresult" entries are stored on a per-external basis.
|
|
This is another feature I've found immensely useful,
but I also wonder if I'm the only one who uses it.
|
|
It seems like a good idea to have a manpage where somebody
can quickly look up and address their concerns as to what
to put on encrypted device/filesystem.
And I probably would've designed lei around make(1) for
parallelization if I didn't have to keep credentials off
the FS :P
|
|
This will eventually be useful for maintaing partial mirrors.
Keeping inline with the original public-inbox-fetch philosophy,
there are no additional config files to manage:
the user merely needs to remove write permissions to an $N.git
directory to prevent it from being updated.
Re-enabling updates just requires restoring write permission.
|
|
It comes up often enough and we need to ensure it's not
confused with IMAP "slices".
|
|
While "eml" is not an output format, it seems worthy
to document, here, since users are likely to have experience
with *.patch files from "git format-patch".
|
|
This allows PublicInbox::WWW hosts to advertise the existence of
IMAP servers in addition to NNTP servers.
|
|
As noted in the new manpage entry, this is useful for avoiding
public-inbox-index invocations when there's nothing to update.
We use 127 to match "grok-pull", and also because it doesn't
conflict with any of the current curl(1) exit codes.
|
|
~/.netrc isn't used by default any more, and I'm not sure it's
worthwhile to document the --netrc switch since it's rare for
non-FTP clients to support.
Followup-to: 9d11ed460ce113dd ("lei: do not read ~/.netrc by default")
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Setting up and maintaining git-only mirrors of v2 inboxes is
complex since multiple commands are required to clone and fetch
into epochs.
Unlike grokmirror, these commands do not require any
configuration. Instead, they rely on existing git config files
and work like "git clone --mirror" and "git fetch",
respectively.
Like grokmirror, they use manifest.js.gz, but only on a
per-inbox basis so users won't have to clone every inbox of a
large instance nor edit config files to include/exclude inboxes
they're interested in.
|
|
When composing replies in "git format-patch" cover letters,
I'd been relying on "lei q -f text ...", but that still requires
several steps to make it suitable for composing a reply:
* s/^/> / to quote the body
* drop existing In-Reply-To+References
* s/^Message-ID:/In-Reply-To:/;
* add an attribute line
...
"lei q -f reply" takes care of most of that and users will
only have to trim "From " lines, unnecessary results and
over-quoted text (and trimming is likely less error-prone
than doing all the steps above manually).
This should also be a good replacement for
"git format-patch --in-reply-to=...", since copying long
Message-IDs can be error-prone (and this lets you include
quoted text in replies).
|
|
It's a pretty incomplete command, so it's important to document
its incompleteness.
|
|
-F/--in-format and --lock=TYPE(S) are easily supported by
all classes using LeiInput.
|
|
While I don't currently see a point in supporting MMDF, we'll
still acknowledge it since mutt actually supports it. Expand a
bit on MH while we're at it, since MH seems at least relevant.
|
|
We missed a few when new documentation came in, and there's no
going back to v2 onions.
Followup-to: 0b15dfc58ceaecdc ("treewide: update to v3 Tor onions")
|
|
Since "lei up" is expected to be a heavily-used command,
better support for IMAP seems like a reasonable idea.
This is inefficient since we waste an IMAP(S) TCP connection
since it dies when an auth-only LeiUp worker process dies, but
it's better than not working at all, right now.
|
|
Cache is expensive, so help users save on storage by documenting
this behavior.
|
|
Since "lei up" is more often useful than not and incurs neglible
overhead; enable --save by default and allow --no-save to work.
This also fixes a long-standing when overwriting --output
destinations with saved searches: dedupe data from previous
searches are reset and no longer influences the new (changed)
search, so results no longer go missing if two sequential
invocations of "lei q --save" point to the same --output.
|
|
lei actually uses implicit --stdin everywhere, but I thing
these patch-related commands are the most common use of them.
|
|
This won't blindly append identical key=values, but
allows specifying multiple, different key=value pairs
as long as the values are different.
|
|
This behaves identically the lei external "boost" parameter in
prioritizing raw messages for extindex.
Relying exclusively on the config file order doesn't work well
for mirrors since it's impossible to guarantee config file
ordering via grokmirror hooks.
Config file ordering remains the default if boost is
unconfigured, or in case of ties.
Note: I chose the name "boost" rather than "priority" or "rank"
since I always get confused by whether higher or lower numbers
take precedence when it comes to kernel scheduling. "weight" is
also a part of Xapian API terminology, which we currently do not
expose to configuration (but may in the future).
|
|
Since we require separate PublicInbox::HTTPD instances for each
listen socket address (in order to support {SERVER_<NAME|PORT>}
for PSGI env), the old cache needed to be invalidated on rare
app refreshes.
SIGHUP has always been broken in -httpd (but not -imapd or
-nntpd) due to this cache.
Update the daemon documentation and 5.10.1-ize some bits while
we're in the area.
|
|
git tends to die when mmap(2) fails on this limit, so let
users know about it. Perhaps git could gracefully fallback.
|
|
While both git and libgit2 take around 16 minutes to load 100K
alternates there's already a proposed patch to make git faster:
<https://lore.kernel.org/git/20210624005806.12079-1-e@80x24.org/>
It's also easier to patch and install git locally since the
git.git build system defaults to prefix=$HOME and dealing with
dynamic linking with libgit2 is more difficult for end users
relying on Inline::C.
libgit2 remains in use for the non-ALL.git case, but maybe it's
not necessary (libgit2 is significantly slower than git in
Debian 10 due to SHA-1 collision checking).
|
|
|
|
[ew: MANIFEST: s/lei-cat/lei-lcat/]
|
|
Link: https://public-inbox.org/meta/20210429015738.GA30172@dcvr/
|
|
|
|
|