user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
From: Luke Kenneth Casson Leighton <lkcl@lkcl.net>
To: meta@public-inbox.org
Subject: setting up mailman-to-atom-converter then atom-to-public-inbox
Date: Tue, 4 Feb 2020 18:42:36 +0000	[thread overview]
Message-ID: <CAPweEDz9wN+OO9twe3JJFYgi2_3yD+9X4s9Z0g2Hqjyf2F8=JA@mail.gmail.com> (raw)

hi, just as the subject says, i'm currently modifying mailman_rss to
support atom and would like to set it up on libre-soc.org shortly.

firstly: very grateful that public-inbox even exists, it is kinda
important to have really, really simple offline archives of project
mailing lists.

second: i have no idea how to go about setting it up :)

third: sigh, i have two unknowns (three), because i am actually
modifying mailman_rss to support atom, *and* i would prefer not to
overload my server by splitting up the creation of atom feeds into
multiple separate processing sections (by month) *and* i have no idea
if public-inbox can support feeds-of-feeds.

to explain / unpack that: here's how i would envisage the workflow so
as to minimise the server load:

* cron job goes through the monthly mailman archives *by month*
performing a re-creation *only* of the latest month's atom feed
* same cron job adds to a "global" atom file containing "links to the
monthly atom files"
* public-inbox sees that list-of-monthly-atom-files
* public-inbox walks the "tree" of monthly atom files, grabbing each one in turn
* public-inbox loads all messages from all monthly atom files.

is this possible or does public-inbox expect one whopping monster
resource-hogging beast-of-an-atom-file potentially hundreds of
megabytes long?  (the reason i ask all this is because the server i am
running this on only has 1GB of RAM and i'm not going to be upgrading
it as it costs money).

tia,

l.

             reply	other threads:[~2020-02-04 18:43 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-04 18:42 Luke Kenneth Casson Leighton [this message]
2020-02-04 20:55 ` setting up mailman-to-atom-converter then atom-to-public-inbox Eric Wong
2020-02-04 21:49   ` Luke Kenneth Casson Leighton
2020-02-04 22:14     ` Eric Wong
     [not found]       ` <CAPweEDy1qTK93pXDKdbT-HqJV184fH7x0hqqJYDTMv_nxvoKqQ@mail.gmail.com>
2020-02-05  0:10         ` Eric Wong
     [not found]           ` <CAPweEDyYA+38B4uc+stMpZ9q6CrHaaAAkkorCuH4ONHmhBXbXg@mail.gmail.com>
2020-02-05  0:43             ` Eric Wong
2020-02-05  1:02               ` Kyle Meyer
2020-02-05  1:04                 ` Eric Wong
2020-03-10  0:07   ` setting up mailman2 and public-inbox Luke Kenneth Casson Leighton
2020-03-11 10:33     ` Eric Wong
2020-03-11 11:58       ` Luke Kenneth Casson Leighton
2020-03-11 12:47         ` Luke Kenneth Casson Leighton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAPweEDz9wN+OO9twe3JJFYgi2_3yD+9X4s9Z0g2Hqjyf2F8=JA@mail.gmail.com' \
    --to=lkcl@lkcl.net \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).