| Date | Commit message (Collapse) |
|---|
|
While git respects a user's local timezone and returns
seconds-since-the-Epoch, we were unnecessarily and incorrectly
calling gmtime+strftime on its result. So ignore calling
gmtime+strftime when the strftime format is "%s", just feed
the output time from git directly to Xapian.
This is mainly for lei, which will likely run in a variety of
timezones. While we're at it, add a recommendation to use
TZ=UTC in public-inbox-httpd, in case there are (misguided :P)
sysadmins who set a non-UTC TZ.
|
|
Like with Maildir, IMAP folders can be deleted entirely.
Ensure they can be eliminated, but don't be fooled into
removing them if they're temporarily unreachable.
|
|
There's no point in keeping mail_sync.sqlite3 entries around
if the folder is gone. We do keep saved-search configs around,
however, since somebody may decide to blow away a search and
start over.
|
|
That option was never wired up, and probably not needed...
|
|
Those stderr messages are not useful at all, and harmful with
the noise they cause.
|
|
This can cause readers and writers to conflict since the
implicit transaction from SELECT in a LeiRefreshMailSync
worker would block the LeiStore process.
|
|
Merely pruning mail synchronization information was
insufficient for Maildir: renames are common in Maildir
and we need to detect them after-the-fact when lei-daemon
isn't running.
Running this command could make "lei index" far more
useful...
v2: close R/O mail_sync.sqlite3 dbh before fork
Keeping the DB file handle open across fork can cause bad things
to happen even if we don't use it since sqlite3 itself still knows
about it (but doesn't know Perl code doesn't know about it).
|
|
It comes up often enough and we need to ensure it's not
confused with IMAP "slices".
|
|
Since some commands access both Maildirs and IMAP/NNTP servers
at the same time, LeiPmdir may see the same lei->{auth} and
lei->{net} objects as the sibling LeiInput-based workers.
Delete those at fork and do not attempt to do authentication in
those cases, since "net_merge_continue" will not be a registered
op and cause PktOp to fail even if authentication /can/ work
from a LeiPmdir worker.
|
|
While "eml" is not an output format, it seems worthy
to document, here, since users are likely to have experience
with *.patch files from "git format-patch".
|
|
This was previously undetected since SOCKS is mainly used for
read-only (single worker) tasks, and worker[0] always loaded
the module. However, "lei refresh-mail-sync" can bounce reads
to any worker, so we need to ensure worker[1..Inf] load it, too.
|
|
We'll be using binary SHA-1 and SHA-256 in-memory since that's
what mail_sync.sqlite3 stores.
|
|
While RFC 3501 doesn't require LIST responses be sorted,
it makes reading protocol dumps easier and we memoize it
once per-refresh, so it shouldn't be too expensive even
with thousands of folders.
|
|
Otherwise, public-inbox-imapd will emit mailboxes in random
order (as IMAP servers do not need to guarantee any sort of
ordering). We'll take into account numeric slice numbers
generated by -imapd if they exist, so slice "80" doesn't show up
next to "8".
|
|
We can't easily use torsocks, here, so try to be helpful
when it comes to proxy support.
|
|
The "mirror" link may not clue users into the existence of
NNTP and IMAP servers, so add a note about them (but don't
list them, in case there are dozens of URLs :>).
|
|
This allows PublicInbox::WWW hosts to advertise the existence of
IMAP servers in addition to NNTP servers.
|
|
We no longer waste a precious hash slot for a per-Inbox
{nntpserver} if it's only configured globally for all inboxes.
|
|
Because make(1), git(1), tar(1) all support -C in this form, as
do our newer commands such as lei, public-inbox-{clone,fetch}.
|
|
The full pathname for "curl -o ..." was too noisy and confusing.
Reduce confusion by adding the ".tmp" suffix and relying on
"-C". We'll also avoid displaying "-C" in run_reap() and
rely on "--git-dir=" with "git fetch" to display progress for
users.
|
|
Since the beginning of time, I've been dropping Makefiles
in $INBOX_DIR (and above hiearchies) to organize groups
of commands.
make(1) is widely available in various flavors and a familiar
tool for our target audience. It is easy to run in the right
directory, typically has built-in shell completion, and doesn't
silently ignore errors by default like Bourne shell.
|
|
Some things (extindex) done, many more to do; and -fetch should
set us up for v1 => v2 migrations at some point...
|
|
As noted in the new manpage entry, this is useful for avoiding
public-inbox-index invocations when there's nothing to update.
We use 127 to match "grok-pull", and also because it doesn't
conflict with any of the current curl(1) exit codes.
|
|
We failed to account for IMAP mailboxes containing `/'
characters when creating saved search files for them.
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20210915123347.knr4qpaei73tjc5q@meerkat.local/
|
|
IMHO, this greatly improves code sharing and organization
between v2, extindex, and lei/store. Common git-related
logic for these is lightly-refactored and easier to reason
about.
The impetus for this big change was to ensure inboxes
created+managed by public-inbox-{clone,fetch} could have
alternates and configs setup properly without depending on
SQLite (via V2Writable). This change does that while
making old code shorter and better factored.
|
|
Again, we were failing to account for '/' use in mailbox names :x
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20210914210547.akdp4cqmwaheayp5@meerkat.local/
|
|
~/.netrc isn't used by default any more, and I'm not sure it's
worthwhile to document the --netrc switch since it's rare for
non-FTP clients to support.
Followup-to: 9d11ed460ce113dd ("lei: do not read ~/.netrc by default")
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Untested at the moment(*), but we were inadvertantly truncating
mailbox names with '/' due to our work-in-progress handling of
"/;UID=$NUM" parameter.
(*) strangely, my dovecot instance doesn't allow '/' by default,
so the change to xt/net_writer-imap.t is untested.
Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20210914175025.eq7s2shkc323itaf@meerkat.local/
|
|
I'm not sure why, but I noticed the one of my latest restarts of
public-inbox-httpd wasn't loading the Inline::C .so for Gcf2 nor
Spawn. I also can't reproduce the problem as both .so files are
loaded fine on a restart with zero config changes.
In any case, some extra, automatic diagnostics for build errors
won't hurt, as no extra noise is introduced for successful builds.
This will also make future development of C code more convenient,
hopefully.
|
|
By moving %ENV localization and fchdir into ->dispatch,
we can maintain a consistent environment across multiple
dispatches while having different clients.
|
|
We want to remove any inotify-watched files before removing
~/.local/lei/store/ipc.lock, since sto_done_request was failing
on attempts to lock a non-existent lei/store/ipc.lock file.
|
|
While persisting lei-daemon across different test cases isn't
the default anymore, we can notice problems more quickly if
the daemon PID changes since the daemon gets auto-restarted
after failures.
|
|
Failures here can cause the lei-daemon event loop to break
since PktOp doesn't guard dispatch. Add a guard here (and
not deeper in the stack) so we can use the $lei object to
report errors.
|
|
This should help us notice (and fix) bugs more easily.
|
|
Oops :x
Fixes: b584a53f053a7629 ("lei up: support --all for IMAP folders")
|
|
I've been creating and destroying lots of externals, lately...
|
|
We need to restore %ENV of script/lei in case another
lei client has a different %ENV than what daemon has.
|
|
PIPE_BUF accounts for Linux being 4096 (and presumably other
OSes differing), while _POSIX_PIPE_BUF is the minimum 512
value.
|
|
The reason(s) we had for not restoring stdout haven't been valid
since script/lei (and not lei-daemon) spawns the pager, nowadays.
|
|
t/v2mirror.t and t/lei-mirror.t are now skipped when curl
is missing (instead of failing in appropriate places).
A bunch of which() checks are updated to use require_cmd
to avoid explicitly loading Spawn.
|
|
Timestamp comparisons only have 1 second granularity, which
isn't nearly enough for our test cases, and probably not for
real world use for "git send-email" bursts and fast SMTP
servers.
We'll continue to check modification times inside the manifest,
though, in case an extremely rare SHA-1 collision is found...
|
|
It was also totally broken by the change to use manifest.js.gz
for v1 :x
Fixes: ffb7fbda6869db4b ("fetch: use manifest.js.gz for v1")
|
|
The v1 code path was totally half-baked after the change
to use manifest.js.gz :x
Fixes: ffb7fbda6869db4b ("fetch: use manifest.js.gz for v1")
|
|
Other missing dependencies are more straightforward, but
Gcf2 requires a combination of libgit2-dev, pkg-config,
Inline::C, and proper setup.
This significantly reduces noise in tests with a minimal
set of dependencies installed.
|
|
And try to improve the message about Inline::C while we're at
it, since Socket::Msghdr isn't widely-packaged, yet.
|
|
This ensures tests are skipped properly if SQLite or Xapian
are missing and don't bail out.
|
|
This is gentler to the remote HTTP server in the no-op case and
will allow client migrations to some v2-ish format without
forcing the client to redownload everything.
|
|
"Unnamed repository" for v1 inboxes was misleading, and having a
non-existent description for v2 was equally annoying, so set a
short description based on the primary address.
We remove descriptions when setting up new test inboxes to
preserve the behavior of the t/lei-mirror.t test case.
|
|
The default value is worthless to us and git functions fine
without the file. public-inbox-init will create a useful one
in the next change.
|
|
Instead of generic "Unnamed repository" or "missing" messages,
show "mirror of $URL" since it seems like a better default when
creating a mirror.
|