about summary refs log tree commit homepage
path: root/lib/PublicInbox/Watch.pm
DateCommit message (Collapse)
2021-10-22watch: remove redundant signal mask manipulation
The top-level daemon process already blocks all signals, so there's no reason to block them around fork() calls.
2021-10-22watch: check for {quit} before IDLE
This may make it less likely for watch-dependent tests to get stuck. Unfortunately, due to the synchronous API of Mail::IMAPClient, ->idle is still susceptible to missing signals.
2021-10-16dir_idle: do not add watches in ->new
There's no savings in having two ways to add watches to an inotify nor kqueue descriptor.
2021-10-01ds: simplify signalfd use
Since signalfd is often combined with our event loop, give it a convenient API and reduce the code duplication required to use it. EventLoop is replaced with ::event_loop to allow consistent parameter passing and avoid needlessly passing the package name on stack. We also avoid exporting SFD_NONBLOCK since it's the only flag we support. There's no sense in having the memory overhead of a constant function when it's in cold code.
2021-09-22treewide: fix %SIG localization, harder
This fixes the occasional t/lei-sigpipe.t infinite loop under "make check-run". Link: http://nntp.perl.org/group/perl.perl5.porters/258784 <CAHhgV8hPbcmkzWizp6Vijw921M5BOXixj4+zTh3nRS9vRBYk8w@mail.gmail.com> Followup-to: b552bb9150775fe4 ("daemon+watch: fix localization of %SIG for non-signalfd users")
2021-09-19watch: use net_reader->mic_new wrapper for SOCKS+TLS
This brings -watch up to feature parity with lei with SOCKS support.
2021-09-09net_reader: combine Net::NNTP and IMAPClient args
Since these are keyed by IMAP and NNTP URIs which can never conflict, it simplifies our internals to keep them in one big hash since we'll add POP3 and JMAP client support.
2021-09-09net_reader: imap_opt => cfg_opt
Since this our internal IMAP options are keyed by URI section, there's no need to have separate hashes for NNTP and IMAP options since they URI already distinguishes them. This will make future changes to support POP3 and JMAP and arg caching with lei/store easier.
2021-09-09net_reader: nntp_opt => cfg_opt
Since this our internal NNTP options are keyed by URI section, there's no need to have separate hashes for NNTP and IMAP options since they URI already distinguishes them. This will make future changes to support POP3 and JMAP and arg caching with lei/store easier.
2021-09-09net_reader: set IMAPClient Keepalive flag late
Since we always enable SO_KEEPALIVE unconditionally, having it in {mic_arg} leads to unnecessary IPC overhead and memory use.
2021-09-03lei: fix read/write IMAP access
xt/net_writer-imap.t was completely broken in recent months and I completely forgot this test. net->add_url still only accepts bare scalars (and not scalar refs), so we must set that up properly. Furthermore, our changes to do FLAGS-only synchronization in lei of old messages was causing us to not handle FLAGS properly for the test.
2021-04-20config: favor ->get_all when possible
It's slightly less code.
2021-03-31doc: add lei-mail-formats(5) manpage
While plenty of online documentation exists, it's good to have a locally-available summary for users to look at offline. Fix a URL in Watch.pm while we're at it, too.
2021-03-10watch: IMAP: ignore \Deleted and \Draft messages
This matches existing Maildir behavior, as trash and draft messages have little reason to be exposed publicly.
2021-02-24net_reader: trim exports and remove unused uri_new
More network things for -watch are isolated in NetReader, now, so fewer exports are necessary.
2021-02-24watch: switch IMAP and NNTP fetch loops to NetReader
NetReader::<imap|nntp>_each were based on the -watch code they now replace. v2: do not warn on EINTR if user quit to fix occasional test failure in t/imapd.t
2021-02-24lei <import|convert>: support NNTP sources
We can read NNTP in -watch and Net::NNTP is shipped with Perl5, so lei import and convert have no excuse not to support NNTP as a client. Authentication is not tested, yet; but should be close to what IMAP is like...
2021-02-21net_reader: use and accept URIimap objects in more places
This flexibility should save us some code down-the-line.
2021-02-18watch: connect to NNTP and IMAP in config order
This is hopefully less surprising to users when they're prompted for credentials.
2021-02-18watch: move imap_common_init to NetReader
We'll use this in LeiImport and likely other places.
2021-02-10net_reader: new package split from -watch
We'll be using some of this for IMAP and NNTP support in lei, too. More will need to be done to improve code sharing and reusability, soon, but this is a start.
2021-02-10use MdirReader in -watch and InboxWritable
MdirReader now handles files in "$MAILDIR/new" properly and is stricter about what it accepts. eml_from_path is also made robust against FIFOs while eliminating TOCTOU races with between stat(2) and open(2) calls.
2021-02-08ds: improve add_timer usability
Packing args into an arrayref is awkward and we may be using this API more in lei.
2021-02-05eml: handle warning ignores for lei
There's nothing we can do about bad emails in our search results, so quiet things down and don't fight the MUA for the terminal.
2021-01-26use defined-or in a few more places
Mainly around fork() calls, but some nearby places as well.
2021-01-24treewide: reseed RNG in child processes
This prevents name conflicts leading to retries and slowdowns in temporary file name generation. No actual data corruption resulted because all temporary files are opened with O_EXCL anyways. This may increase security for IMAP, NNTP, and HTTPS sessions using TLS, but it's all public data anyways.
2021-01-12ds: block signals when reaping
This lets us call dwaitpid long before a process exits and not have to wait around for it. This is advantageous for lei where we can run dwaitpid on the pager as soon as we spawn it, instead of waiting for a client socket to go away on DESTROY.
2021-01-01update copyrights for 2021
Using "make update-copyrights" after setting GNULIB_PATH in my config.mak
2020-12-26default to CORE::warn in $SIG{__WARN__} handlers
As with CORE::die and $SIG{__DIE__}, it turns out CORE::warn is safe to use inside $SIG{__WARN__} handlers without triggering infinite recursion. So fall back to reusing CORE::warn instead of creating a new sub.
2020-12-09rename {pi_config} fields to {pi_cfg}
{pi_config} may be confused with the documented `PI_CONFIG' environment variable, and we'll favor vowel-removal to be consistent with our usage of object references. The `pi_' prefix may stay in some places, for now; since a separate namespace may come into this codebase for local/private client-tooling. For InboxIdle, we'll also remove an invalid comment about holding a reference to the PublicInbox::Config object, too.
2020-09-12treewide: avoid `goto &NAME' for tail recursion
While Perl implements tail recursion via `goto' which allows avoiding warnings on deep recursion. It doesn't (as of 5.28) optimize the speed of such dispatches, though it may reduce ephemeral memory usage. Make the code less alien to hackers coming from other languages by using normal subroutine dispatch. It's actually slightly faster in micro benchmarks due to the complexity of `goto &NAME'.
2020-09-10config: flatten each_inbox and iterate_start args
In Perl, we can simplify callers by passing a single array all the way down the stack instead of a single array ref which needs to be expanded every call.
2020-09-01watch: use EOFpipe to reduce dwaitpid wakeups
It's a bit inefficient to use a pipe, here. However, using dwaitpid() on a process that's not expected to exit soon is also inefficient as it causes excessive wakeups as most of our inbox-writing code expects synchronous waitpid(). This only affects -watch instances configured for NNTP and IMAP clients.
2020-09-01watch: comments and tiny cleanups
Get rid of an unused variable, prefix a warning and try to better document control flow around various callbacks.
2020-09-01watch: block signals before fork on non-signalfd/kevent systems
In case there's non-Linux or BSD users w/o IO::KQueue, we shouldn't let signal handlers fire in the child processes. The child processes always assumed signals were blocked by the parent, so no changes were necessary, there.
2020-09-01watch: avoid unnecessary spawning on spam removals
This should further mitigate lock contention problems when -watch is configured to watch on a Maildir for spam while performing a large NNTP import. There is now a small risk a message won't get removed because if it's in the current (uncommitted) fast-import batch, but unlikely given the batch size is now only 10 messages. If a that small window is hit, flipping the \Seen flag (e.g. marking it unread, and then read again) will trigger another removal attempt via IMAP or Maildir.
2020-09-01rename WatchMaildir => Watch
This is no longer limited to Maildirs now that IMAP and NNTP support exist; so give it a shorter name.