* [PATCH 19/34] imaptracker: add {url} field to reduce args
2020-06-27 10:03 7% [PATCH 00/34] watch: add IMAP and NNTP support Eric Wong
@ 2020-06-27 10:03 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-06-27 10:03 UTC (permalink / raw)
To: meta; +Cc: Eric W . Biederman
Passing a $url parameter to every function was error-prone
and having {url} field for a short-lived object is appropriate.
This matches the version of IMAPTracker posted by
Eric W. Biederman on 2020-05-15 at:
https://public-inbox.org/meta/87ftc0c3r4.fsf_-_@x220.int.ebiederm.org/
The version I originally imported was based on the one
posted on 2019-10-09:
https://public-inbox.org/meta/874l0i9vhc.fsf_-_@x220.int.ebiederm.org/
Cc: Eric W. Biederman <ebiederm@xmission.com>
---
lib/PublicInbox/IMAPTracker.pm | 20 ++++++++++----------
lib/PublicInbox/WatchMaildir.pm | 17 ++++++++---------
2 files changed, 18 insertions(+), 19 deletions(-)
diff --git a/lib/PublicInbox/IMAPTracker.pm b/lib/PublicInbox/IMAPTracker.pm
index bb4a39cc41a..26274568b9c 100644
--- a/lib/PublicInbox/IMAPTracker.pm
+++ b/lib/PublicInbox/IMAPTracker.pm
@@ -33,29 +33,29 @@ sub dbh_new ($) {
$dbh;
}
-sub get_last ($$) {
- my ($self, $url) = @_;
+sub get_last ($) {
+ my ($self) = @_;
my $sth = $self->{dbh}->prepare_cached(<<'', undef, 1);
SELECT uid_validity, uid FROM imap_last WHERE url = ?
- $sth->execute($url);
+ $sth->execute($self->{url});
$sth->fetchrow_array;
}
-sub update_last ($$$$) {
- my ($self, $url, $validity, $last) = @_;
+sub update_last ($$$) {
+ my ($self, $validity, $last) = @_;
my $sth = $self->{dbh}->prepare_cached(<<'');
INSERT OR REPLACE INTO imap_last (url, uid_validity, uid)
VALUES (?, ?, ?)
- $sth->execute($url, $validity, $last);
+ $sth->execute($self->{url}, $validity, $last);
}
sub new {
- my ($class, $dbname) = @_;
+ my ($class, $url) = @_;
# original name for compatibility with old setups:
- $dbname //= PublicInbox::Config->config_dir() . "/imap.sqlite3";
+ my $dbname = PublicInbox::Config->config_dir() . "/imap.sqlite3";
# use the new XDG-compliant name for new setups:
if (!-f $dbname) {
@@ -65,12 +65,12 @@ sub new {
}
if (!-f $dbname) {
require File::Path;
- require File::Basename;;
+ require File::Basename;
File::Path::mkpath(File::Basename::dirname($dbname));
}
my $dbh = dbh_new($dbname);
- bless { dbname => $dbname, dbh => $dbh }, $class;
+ bless { dbname => $dbname, url => $url, dbh => $dbh }, $class;
}
1;
diff --git a/lib/PublicInbox/WatchMaildir.pm b/lib/PublicInbox/WatchMaildir.pm
index f36aa20aa33..e0caaa563b2 100644
--- a/lib/PublicInbox/WatchMaildir.pm
+++ b/lib/PublicInbox/WatchMaildir.pm
@@ -335,12 +335,12 @@ sub mic_for ($$$) { # mic = Mail::IMAPClient
$mic;
}
-sub imap_import_msg ($$$$$$) {
- my ($self, $itrk, $url, $r_uidval, $uid, $raw) = @_;
+sub imap_import_msg ($$$$$) {
+ my ($self, $itrk, $r_uidval, $uid, $raw) = @_;
# our target audience expects LF-only, save storage
$$raw =~ s/\r\n/\n/sg;
- my $inboxes = $self->{imap}->{$url};
+ my $inboxes = $self->{imap}->{$itrk->{url}};
if (ref($inboxes)) {
for my $ibx (@$inboxes) {
my $eml = PublicInbox::Eml->new($$raw);
@@ -348,12 +348,12 @@ sub imap_import_msg ($$$$$$) {
}
} elsif ($inboxes eq 'watchspam') {
my $eml = PublicInbox::Eml->new($raw);
- my $arg = [ $self, $eml, "$url UID:$uid" ];
+ my $arg = [ $self, $eml, "$itrk->{url} UID:$uid" ];
$self->{config}->each_inbox(\&remove_eml_i, $arg);
} else {
die "BUG: destination unknown $inboxes";
}
- $itrk->update_last($url, $r_uidval, $uid);
+ $itrk->update_last($r_uidval, $uid);
}
sub imap_fetch_all ($$$) {
@@ -373,8 +373,8 @@ sub imap_fetch_all ($$$) {
return "E: $url cannot get UIDVALIDITY";
$r_uidnext //= $mic->uidnext($mbx) //
return "E: $url cannot get UIDNEXT";
- my $itrk = PublicInbox::IMAPTracker->new;
- my ($l_uidval, $l_uid) = $itrk->get_last($url);
+ my $itrk = PublicInbox::IMAPTracker->new($url);
+ my ($l_uidval, $l_uid) = $itrk->get_last;
$l_uidval //= $r_uidval; # first time
$l_uid //= 1;
if ($l_uidval != $r_uidval) {
@@ -426,8 +426,7 @@ sub imap_fetch_all ($$$) {
}
# messages get deleted, so holes appear
defined(my $raw = delete $r->{$uid}->{$key}) or next;
- imap_import_msg($self, $itrk, $url, $r_uidval, $uid,
- \$raw);
+ imap_import_msg($self, $itrk, $r_uidval, $uid, \$raw);
last if $self->{quit};
}
_done_for_now($self);
^ permalink raw reply related [relevance 6%]
* [PATCH 00/34] watch: add IMAP and NNTP support
@ 2020-06-27 10:03 7% Eric Wong
2020-06-27 10:03 6% ` [PATCH 19/34] imaptracker: add {url} field to reduce args Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-06-27 10:03 UTC (permalink / raw)
To: meta
Some fairly major changes to -watch. Filesys::Notify::Simple is
no longer used, and -watch now uses inotify, signalfd or kevent
like the read-only daemons.
Credentials are handled via Net::Netrc (Perl standard library)
or "git-credential", so we do no password storage on our own.
NNTP (and non-IDLE IMAP) may allow more parallelization in the
future.
One significant project-wide change is getting rid of "use
fields". It gets in my way more than it helps, and it's
probably alien to a fair amount of Perl hackers. AFAIK, it's
never really been popular outside of Danga::Socket-based
projects.
Eric W. Biederman (1):
IMAPTracker: Add a helper to track our place in reading imap mailboxes
Eric Wong (33):
inboxwritable: ensure ssoma.lock exists on init
inbox: warn on ->on_inbox_unlock exception
imaptracker: use ~/.local/share/public-inbox/imap.sqlite3
watchmaildir: hoist out compile_watchheaders
watchmaildir: fix check for spam vs ham inbox conflicts
URI IMAP support
watch: preliminary IMAP support
kqnotify|fake_inotify: detect Maildir write ops
watch: remove Filesys::Notify::Simple dependency
watch: use signalfd for Maildir watching
ds: remove fields.pm usage
watch: wire up IMAP IDLE reapers to DS
watch: support IMAP polling
config: support ->urlmatch method for -watch
watch: stop importers before forking
watch: use UID SEARCH to avoid empty UID FETCH
ds: add_timer: allow passing arg to callback.
imaptracker: add {url} field to reduce args
imaptracker: drop {dbname} field
watch: avoid long transaction when writing to IMAPTracker
watch: support imap.fetchBatchSize parameter
watch: imap: be quieter about disconnecting on quit
watch: support multiple watch: directives per-inbox
watch: remove {mdir} array
watch: just use ->urlmatch
testcommon: $ENV{TAIL} supports non-@ARGV redirects
watch: add NNTP support
watch: show user-specified URL consistently.
watch: enable autoflush for STDOUT and STDERR
watch: use our own "git credential" wrapper
watch: support ~/.netrc via Net::Netrc
imaptracker: use flock(2) around writes
watch: simplify internal structures
Documentation/public-inbox-watch.pod | 3 +-
INSTALL | 8 -
MANIFEST | 11 +
Makefile.PL | 4 -
ci/deps.perl | 1 -
lib/PublicInbox/Config.pm | 21 +-
lib/PublicInbox/DS.pm | 29 +-
lib/PublicInbox/Daemon.pm | 19 +-
lib/PublicInbox/DirIdle.pm | 49 ++
lib/PublicInbox/FakeInotify.pm | 56 +-
lib/PublicInbox/GitAsyncCat.pm | 4 +-
lib/PublicInbox/GitCredential.pm | 55 ++
lib/PublicInbox/HTTP.pm | 23 +-
lib/PublicInbox/HTTPD/Async.pm | 22 +-
lib/PublicInbox/IMAP.pm | 19 +-
lib/PublicInbox/IMAPTracker.pm | 82 +++
lib/PublicInbox/In2Tie.pm | 13 +
lib/PublicInbox/Inbox.pm | 1 +
lib/PublicInbox/InboxIdle.pm | 20 +-
lib/PublicInbox/InboxWritable.pm | 3 +
lib/PublicInbox/KQNotify.pm | 38 +-
lib/PublicInbox/Listener.pm | 8 +-
lib/PublicInbox/NNTP.pm | 12 +-
lib/PublicInbox/NNTPdeflate.pm | 5 +-
lib/PublicInbox/ParentPipe.pm | 8 +-
lib/PublicInbox/Sigfd.pm | 21 +-
lib/PublicInbox/TestCommon.pm | 40 +-
lib/PublicInbox/URIimap.pm | 113 +++
lib/PublicInbox/WatchMaildir.pm | 998 +++++++++++++++++++++++----
script/public-inbox-watch | 33 +-
t/config.t | 18 +
t/dir_idle.t | 6 +
t/fake_inotify.t | 45 ++
t/imap_tracker.t | 54 ++
t/imapd.t | 74 ++
t/kqnotify.t | 41 ++
t/nntpd.t | 52 ++
t/uri_imap.t | 65 ++
t/watch_filter_rubylang.t | 2 +-
t/watch_imap.t | 21 +
t/watch_maildir.t | 96 ++-
t/watch_maildir_v2.t | 4 +-
t/watch_multiple_headers.t | 2 +-
t/watch_nntp.t | 17 +
xt/mem-imapd-tls.t | 18 +-
45 files changed, 1944 insertions(+), 290 deletions(-)
create mode 100644 lib/PublicInbox/DirIdle.pm
create mode 100644 lib/PublicInbox/GitCredential.pm
create mode 100644 lib/PublicInbox/IMAPTracker.pm
create mode 100644 lib/PublicInbox/URIimap.pm
create mode 100644 t/dir_idle.t
create mode 100644 t/fake_inotify.t
create mode 100644 t/imap_tracker.t
create mode 100644 t/kqnotify.t
create mode 100644 t/uri_imap.t
create mode 100644 t/watch_imap.t
create mode 100644 t/watch_nntp.t
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-06-27 10:03 7% [PATCH 00/34] watch: add IMAP and NNTP support Eric Wong
2020-06-27 10:03 6% ` [PATCH 19/34] imaptracker: add {url} field to reduce args Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).