about summary refs log tree commit homepage
path: root/scripts/import_slrnspool
DateCommit message (Collapse)
2024-02-01scripts/import_*: update usage to include lei tips
These scripts probably don't offer anything useful now that lei has fleshed out read-only MH support and v2 outputs.
2021-01-01update copyrights for 2021
Using "make update-copyrights" after setting GNULIB_PATH in my config.mak
2020-12-09rename {pi_config} fields to {pi_cfg}
{pi_config} may be confused with the documented `PI_CONFIG' environment variable, and we'll favor vowel-removal to be consistent with our usage of object references. The `pi_' prefix may stay in some places, for now; since a separate namespace may come into this codebase for local/private client-tooling. For InboxIdle, we'll also remove an invalid comment about holding a reference to the PublicInbox::Config object, too.
2020-05-20scripts/import_*: remove PublicInbox::MIME usage
These aren't really supported and will probably be replaced with better tools, but PublicInbox::Eml should be readily available to anybody who already has our source tree.
2020-04-19favor `do {}' over `eval {}' for localized slurp
I did not know to use the return value of `do' back in the day. There's probably no practical difference in these cases, but `eval' is overkill for these uses and may hide actual errors. We can get rid of a few redundant `scalar' ops and pass scalar refs to Email::MIME->new to avoid copies in a few more places, too.
2020-02-06treewide: run update-copyrights from gnulib for 2019
I didn't wait until September to do it, this year!
2020-01-27inbox: add ->version method
This allows us to simplify version checking by avoiding "//" or "||" operators sprinkled around.
2019-09-09run update-copyrights from gnulib for 2019
2019-06-05scripts: require ASCII digits in a few places
I haven't touched most these scripts in ages, but we might as well purge \d usage from here, as well.
2018-12-28add filter for gmane archives
Extracted from import_slrnspool, since some spools get converted to mbox or what not.
2018-05-02scripts/import_slrnspool: cleanup progress messages
Stop showing redundant slashes and stop showing progress for messages which do not exist.
2018-05-02scripts/import_slrnspool: support v2 repos
2018-02-28use PublicInbox::MIME consistently
It works around some bugs in older Email::MIME which we'll find useful.
2018-02-07update copyrights for 2018
Using update-copyrights from gnulib While we're at it, use the SPDX identifier for AGPL-3.0+ to ease mechanical processing.
2016-08-14import_slrnspool: reimplement using fast-import
I needed to use this to resurrect some messages missing from my initial downloads from gmane...
2016-05-14rename most instances of "list" to "inbox"
A public-inbox is NOT necessarily a mailing list, but it could serve as an input point for zero, one, or infinite mailing lists :D
2015-09-06update copyright headers and email addresses
In the future, it should be possible to use this: git ls-files | UPDATE_COPYRIGHT_HOLDER='all contributors' \ UPDATE_COPYRIGHT_USE_INTERVALS=2 \ xargs /path/to/gnulib/build-aux/update-copyright
2015-01-12import_slrnspool: fork a process for each message
This prevents process growth when importing large messages. Memory growth could be due to the sliding sbrk window in glibc malloc or a circular reference in the Email::* Perl code somewhere.
2015-01-11import_slrnspool: load private config key
PublicInbox::Config->lookup won't return unknown keys
2015-01-11import_slrnspool: graceful exit for interruptibility
This should alleviate fears of interrupting the process.
2015-01-11import_slrnspool: make filtering optional
2015-01-11import_slrnspool: use ssoma-mda instead
Some mailing lists (e.g. git@vger.kernel.org) accept messages via Bcc: and possibly other things which get rejected by the strict PublicInbox::Filter rules. So rely on ssoma-mda instead. This prefers a recent revision of ssoma-mda (commit 7fce38e9 onwards) to display subject/author/date information in the commit message.
2015-01-11*slrnspool* old gmane archives set Original-To
Apparently it's not a problem with recent archives.
2015-01-11import_slrnspool: fix off-by-one error
We start with zero and only store the next valid ID.
2015-01-11scripts/import_slrnspool: new incremental importer
This allows incremental imports of slrn spools, ideal for tracking lists via gmane.