From: ebiederm@xmission.com (Eric W. Biederman) To: Eric Wong <e@80x24.org> Cc: meta@public-inbox.org Subject: [PATCH 4/4] IMAPTracker: Add a helper to track our place in reading imap mailboxes Date: Wed, 09 Oct 2019 03:25:03 -0500 Message-ID: <874l0i9vhc.fsf_-_@x220.int.ebiederm.org> (raw) In-Reply-To: <87wode9vxw.fsf_-_@x220.int.ebiederm.org> (Eric W. Biederman's message of "Wed, 09 Oct 2019 03:15:07 -0500") Date: Fri, 27 Jul 2018 20:54:27 -0500 This removes the need to delete from an imap mailbox when downloading it's messages. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com> --- This is simple and potentially very useful. lib/PublicInbox/IMAPTracker.pm | 73 ++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 lib/PublicInbox/IMAPTracker.pm diff --git a/lib/PublicInbox/IMAPTracker.pm b/lib/PublicInbox/IMAPTracker.pm new file mode 100644 index 000000000000..65241eeebf51 --- /dev/null +++ b/lib/PublicInbox/IMAPTracker.pm @@ -0,0 +1,73 @@ +# Copyright (C) 2018 all contributors <meta@public-inbox.org> +# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt> + +package PublicInbox::IMAPTracker; +use strict; +use warnings; +use DBI; +use DBD::SQLite; +use PublicInbox::Config; + +sub create_tables ($) +{ + my ($dbh) = @_; + + $dbh->do(<<''); +CREATE TABLE IF NOT EXISTS imap_last ( + url VARCHAR PRIMARY KEY NOT NULL, + uid_validity INTEGER NOT NULL, + uid INTEGER NOT NULL, + UNIQUE (url) +) + +} + + +sub dbh_new ($) +{ + my ($dbname) = @_; + my $dbh = DBI->connect("dbi:SQLite:dbname=$dbname", '', '', { + AutoCommit => 1, + RaiseError => 1, + PrintError => 0, + ReadOnly => 0, + sqlite_use_immediate_transaction => 1, + }); + $dbh->{sqlite_unicode} = 1; + $dbh->do('PRAGMA journal_mode = TRUNCATE'); + $dbh->do('PRAGMA cache_size = 80000'); + create_tables($dbh); + $dbh; +} + +sub get_last($$) +{ + my ($self, $url) = @_; + my $dbh = $self->{dbh}; + + my $sth = $dbh->prepare_cached(<<'', undef, 1); +SELECT uid_validity, uid FROM imap_last WHERE url = ? + + $sth->execute($url); + $sth->fetchrow_array; +} + +sub update_last($$$$) +{ + my ($self, $url, $validity, $last) = @_; + my $dbh = $self->{dbh}; + my $sth = $dbh->prepare_cached(<<''); +INSERT OR REPLACE INTO imap_last (url, uid_validity, uid) +VALUES (?, ?, ?) + + $sth->execute($url, $validity, $last); +} + +sub new { + my ($class) = @_; + my $dbname = PublicInbox::Config->config_dir() . "/imap.sqlite3"; + my $dbh = dbh_new($dbname); + bless { dbname => $dbname, dbh => $dbh }, $class; +} + +1; -- 2.23.0
next prev parent reply other threads:[~2019-10-09 8:25 UTC|newest] Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-10-07 22:13 Do I need multiple publicinbox.<name>.address values? Alyssa Ross 2019-10-08 0:10 ` Eric Wong 2019-10-08 12:18 ` Eric W. Biederman 2019-10-08 12:23 ` [PATCH] Config.pm: Add support for mailing list information Eric W. Biederman 2019-10-08 22:11 ` Do I need multiple publicinbox.<name>.address values? Eric Wong 2019-10-08 22:24 ` Eric W. Biederman 2019-10-08 22:41 ` Eric Wong 2019-10-09 7:58 ` Eric W. Biederman 2019-10-09 8:15 ` [PATCH 0/4] Various bits to support import_imap_mailbox Eric W. Biederman 2019-10-09 8:16 ` [PATCH 1/4] PublicInbox::Import Smuggle a raw message into add Eric W. Biederman 2019-10-15 20:26 ` Eric Wong 2019-10-15 23:05 ` Eric W. Biederman 2019-10-09 8:17 ` [PATCH 2/4] PublicInbox::Config: Process mailboxes in sorted order Eric W. Biederman 2019-10-10 9:43 ` Eric Wong 2019-10-10 11:05 ` Eric W. Biederman 2019-10-09 8:23 ` [PATCH 3/4] Config.pm: Add support for looking up repos by their directories Eric W. Biederman 2019-10-09 8:25 ` Eric W. Biederman [this message] 2019-10-10 19:08 ` ibx->{listid} autoviv fixup [was: [PATCH 0/4] Various bits to support import_imap_mailbox] Eric Wong 2019-10-10 21:23 ` Eric W. Biederman 2019-10-10 8:31 ` Do I need multiple publicinbox.<name>.address values? Eric Wong 2019-10-10 10:56 ` Eric W. Biederman 2019-10-09 11:59 ` Alyssa Ross 2019-10-10 10:06 ` Eric Wong
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://public-inbox.org/README * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=874l0i9vhc.fsf_-_@x220.int.ebiederm.org \ --to=ebiederm@xmission.com \ --cc=e@80x24.org \ --cc=meta@public-inbox.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
user/dev discussion of public-inbox itself This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/meta git clone --mirror http://czquwvybam4bgbro.onion/meta git clone --mirror http://hjrcffqmbrq6wope.onion/meta git clone --mirror http://ou63pmih66umazou.onion/meta # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 meta meta/ https://public-inbox.org/meta \ meta@public-inbox.org public-inbox-index meta Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta nntp://news.gmane.io/gmane.mail.public-inbox.general note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/public-inbox.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git