about summary refs log tree commit homepage
path: root/scripts/import_slrnspool
DateCommit message (Collapse)
2020-02-06treewide: run update-copyrights from gnulib for 2019
I didn't wait until September to do it, this year!
2020-01-27inbox: add ->version method
This allows us to simplify version checking by avoiding "//" or "||" operators sprinkled around.
2019-09-09run update-copyrights from gnulib for 2019
2019-06-05scripts: require ASCII digits in a few places
I haven't touched most these scripts in ages, but we might as well purge \d usage from here, as well.
2018-12-28add filter for gmane archives
Extracted from import_slrnspool, since some spools get converted to mbox or what not.
2018-05-02scripts/import_slrnspool: cleanup progress messages
Stop showing redundant slashes and stop showing progress for messages which do not exist.
2018-05-02scripts/import_slrnspool: support v2 repos
2018-02-28use PublicInbox::MIME consistently
It works around some bugs in older Email::MIME which we'll find useful.
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2016-08-14import_slrnspool: reimplement using fast-import
I needed to use this to resurrect some messages missing from my initial downloads from gmane...
2016-05-14rename most instances of "list" to "inbox"
A public-inbox is NOT necessarily a mailing list, but it could serve as an input point for zero, one, or infinite mailing lists :D
2015-09-06update copyright headers and email addresses
In the future, it should be possible to use this: git ls-files | UPDATE_COPYRIGHT_HOLDER='all contributors' \ UPDATE_COPYRIGHT_USE_INTERVALS=2 \ xargs /path/to/gnulib/build-aux/update-copyright
2015-01-12import_slrnspool: fork a process for each message
This prevents process growth when importing large messages. Memory growth could be due to the sliding sbrk window in glibc malloc or a circular reference in the Email::* Perl code somewhere.
2015-01-11import_slrnspool: load private config key
PublicInbox::Config->lookup won't return unknown keys
2015-01-11import_slrnspool: graceful exit for interruptibility
This should alleviate fears of interrupting the process.
2015-01-11import_slrnspool: make filtering optional
2015-01-11import_slrnspool: use ssoma-mda instead
Some mailing lists (e.g. git@vger.kernel.org) accept messages via Bcc: and possibly other things which get rejected by the strict PublicInbox::Filter rules. So rely on ssoma-mda instead. This prefers a recent revision of ssoma-mda (commit 7fce38e9 onwards) to display subject/author/date information in the commit message.
2015-01-11*slrnspool* old gmane archives set Original-To
Apparently it's not a problem with recent archives.
2015-01-11import_slrnspool: fix off-by-one error
We start with zero and only store the next valid ID.
2015-01-11scripts/import_slrnspool: new incremental importer
This allows incremental imports of slrn spools, ideal for tracking lists via gmane.