about summary refs log tree commit homepage
path: root/t/lei-import-nntp.t
DateCommit message (Collapse)
2021-09-21lei lcat: support NNTP URLs
NNTP URLs are probably more prevalent in public message archives than IMAP URLs.
2021-09-21lei inspect: support NNTP URLs
No reason not to support them, since there's more public-inbox-nntpd instances than -imapd instances, currently.
2021-09-19net_reader: NNTP: remove article numbers from mail_sync folders
NNTP article numbers are stored separately from folder names in mail_sync.sqlite3. Recovering from this is optional, worse case is wasting bandwidth refetching some messages. To (optionally) recover from this, use: lei forget-mail-sync $URL_WITH_ARTNUMS Some articles will be refetched on the next import, but duplicate data won't be indexed in Xapian.
2021-09-19net_reader: fix single NNTP article fetch, test ranges
While NNTP ranges was already working, fetching a single message was broken. We'll also simplify the code a bit and ensure incremental synchronization is ignored when ranges are specified.
2021-06-12lei ls-mail-source: list IMAP folders and NNTP groups
While other tools can provide the same functionality, having integration with git-credential is convenient, here. Caching and completion will be implemented separately.
2021-05-04lei: fix mail_sync.sqlite3 folder names for NNTP
We should not have "SCALAR(XXXXXXX)" showing up in SQLite DBs because we passed a SCALAR ref instead of a non-ref SCALAR.
2021-04-30lei import: avoid IMAPTracker, use LeiMailSync more
IMAPTracker has a UNIQUE constraint on the `url' column, which may cause compatibility and/or rollback problems in attempting to deal with UIDVALIDITY changes. Having multiple sources of truth leads to confusion and bugs, so relying on LeiMailSync exclusively ought to simplify things. Furthermore, since LeiMailSync is only written to by LeiStore, it is safer in that it won't mark a UID or article as imported until git-fast-import has seen it, and the SQLite commit always happens after "done\n" is sent to fast-import. This mostly reverts recent commits to IMAPTracker to support lei, those are: 1) commit 7632d8f7590daf70c65d4270e750c36552fa9389 ("net_reader: restart on first UID when UIDVALIDITY changes") 2) commit 311a5d37ad275cd75b1e64d87827c4d13fe4bfab ("imap_tracker: prepare for use with lei"). This means public-inbox-watch will not change between 1.6 and 1.7: -watch stops synching a folder when UIDVALIDITY changes.
2021-04-22lei import: --incremental default for NNTP and IMAP
No point in burning through bandwidth to import stuff we already saw. All this logic is shared with -watch but uses a different pathname for lei since it's tied to lei/store (and not a public-inbox).
2021-03-05search: use "z:" instead of "bytes:" prefix
So far, searching by size has never been publicly documented, and IMHO, of questionable utility. In any case, "z:" is what mairix(1) uses, so it may be familiar to existing mairix users (I've never used this prefix myself). So far, this prefix is only used internally in tests and in auto-translated queries from IMAP; thus this incompatible change is unlikely to affect anyone.
2021-02-24lei <import|convert>: support NNTP sources
We can read NNTP in -watch and Net::NNTP is shipped with Perl5, so lei import and convert have no excuse not to support NNTP as a client. Authentication is not tested, yet; but should be close to what IMAP is like...