| Date | Commit message (Collapse) |
|---|
|
Unlike DBD::SQLite, the sqlite3(1) CLI does not have a default
busy timeout enabled, so it easily times out while acquiring a
SHARED lock for read-only queries. We can avoid battery-wasting
polling from the SQLite timeout handler by relying on flock(2)
as we do in our Perl code.
Furthermore, this avoids triggering some locking problems[1]
from a long "SELECT COUNT(*) ..." query and reindex.
While there may be other SQLite-related parallelism issues[1],
this works around one of them by relying on flock(2).
[1] https://public-inbox.org/meta/20200825001204.GA840@dcvr/
|
|
We've got examples for all the other daemons, too!
|
|
--sequential-shard offers better performance on HDD than -j0
since the on-disk active set can be kept small (with -j $HIGH_NUM).
--batch-size can also be helpful for systems with much RAM.
|
|
I finally noticed descriptions weren't showing up in my mirrors :x
|
|
grok-pull is still painful with serialization on an old USB 2.0
HDD, but at least it can finish with flock(1) and disabling
parallelization. While parallel "git fetch" doesn't seem so
bad, slow seeks are exacerbated by parallel reads in Xapian.
That means some updates can take days instead of hours. The
same updates take only seconds or minutes on an SSD.
|
|
Instead of gzipping some (mbox.gz, manifest.js.gz) responses and
leaving P::M::D to do the rest, we gzip everything ourselves,
now, so P::M::D is redundant.
|
|
Users are encouraged to edit this script, anyways, so make it
easy for them to swap out and use whatever URL they need.
|
|
The value of infourl parameters are shared in the config, so
include them in the mirror.
|
|
The $INBOX_URL/description endpoint is available since v1.3.0,
so use it in mirrors.
|
|
public-inbox-httpd should work with any PSGI files, so make
it more apparent to people reading .psgi examples.
|
|
It was the only file in our tree which had CRLF line endings,
so make it consistent with the rest.
|
|
I didn't wait until September to do it, this year!
|
|
Instead of providing a generic "mailto:foo+unsubscribe@example.com"
address in List-Unsubscribe which requires confirmation, replace it
with a mailto: header with a unique subject which contains the same
unique ID we put in the https:// URL.
This makes it easier for some MUAs without https:// support to
unsubscribe with a single action via the List-Unsubscribe header.
|
|
Mail to gmane is being delivered to gmane-mx.org, nowadays, and
we don't want ordinary readers to be able to trigger unconfirmed
unsubscription off any mailing lists which go through our
unsubscribe.milter.
https://lars.ingebrigtsen.no/2020/01/06/whatever-happened-to-news-gmane-org/
|
|
This is necessary for Filesys::Notify::Simple 0.13 using
Linux::Inotify2, since 0.13 started croaking on
inotify_add_watch failures.
|
|
We need to account for both the old ("mainrepo") and new
("inboxdir") names. But "dir" was just a search+replace
error and we don't use that outside of "coderepo.dir".
|
|
"mainrepo" ws a bad name and artifact from the early days when I
intended for there to be a "spamrepo" (now just the
ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be
especially confusing, since v2 needs at least two git
repositories (epoch + all.git) to function and we shouldn't
confuse users by having them point to a git repository for v2.
Much of our documentation already references "INBOX_DIR" for
command-line arguments, so use "inboxdir" as the
git-config(1)-friendly variant for that.
"mainrepo" remains supported indefinitely for compatibility.
Users may need to revert to old versions, or may be referring
to old documentation and must not be forced to change config
files to account for this change.
So if you're using "mainrepo" today, I do NOT recommend changing
it right away because other bugs can lurk.
Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
|
|
Move away from using "mainrepo" since it's confusing to
new users, especially with v2.
|
|
This requires the latest (to be in 1.2) -init changes for
synchronization and has no dependencies on GNU or bash-isms
so it should run on *BSD systems without GNU tools.
It does attempt to use curl on <$INBOX_URL/_/text/config/raw>,
but curl is fairly standard nowadays, and falls back to using
an invalid address to initialize.
|
|
NNTPS and STARTTLS seems to be working for several months
without incident on news.public-inbox.org, so consider it a
success and maybe others can try using it.
HTTPS technically works, too, but isn't documented at
the moment since I can't recommend production deployments
without varnish protecting it.
|
|
|
|
For users running multiple (-nntpd@1, -nntpd@2) instances of
either -httpd or -nntpd via systemd to implement zero-downtime
restarts; it's possible for a listen socket to become blocking
for a moment during an accept syscall and cause a daemons to
get stuck in a blocking accept() during
PublicInbox::Listener::event_step (event_read in previous
versions).
Since O_NONBLOCK is a file description flag, systemd clearing
O_NONBLOCK momentarily (before PublicInbox::Listener::new
re-enables it) creates a window for another instance of our
daemon to get stuck in accept().
cf. systemd.service(5)
|
|
The sample configuration can be used to proxy-pass requests
to public-inbox-httpd or to a standalone PSGI/Plack server.
|
|
Deflating responses may be done by the reverse proxy (e.g. varnish
or nginx), so the warning for it could be invalid.
|
|
It's been a while since I wrote this, and it needs to be kept
up-to-date with some advances in our Perl code.
|
|
I'm using this as the cgit about-filter and source-filter
in https://80x24.org/public-inbox.git
|
|
|
|
We depend on git-http-backend for smart HTTP clone support,
however; since cgit does not support smart clones natively.
WWW.pm will be able to cascade down to this as a 404 handler in
the future.
|
|
Plack::Builder allows "mounting" on with hostnames as well as
path names to enable virtual hosting. This example demonstrates
how port 80/443 for "news.example.com" can redirect browser
requests when somebody attempts to use a "nntp://" URL and
the software assumes "http://"
|
|
I'll probably expose the PSGI service for cgit;
but it could be useful to others as well.
|
|
Maybe we'll default to a dark theme to promote energy savings...
See contrib/css/README for details
|
|
|
|
Let's Encrypt is working out nicely, so we can rely on HTTPS,
now. Use 80x24.org instead of bogomips.org while we're at it,
since I don't think the latter will remain.
|
|
I guess I forgot to include this, but I've been running
public-inbox-watch as a systemd service for nearly two
years, now.
|
|
Using update-copyrights from gnulib
While we're at it, use the SPDX identifier for AGPL-3.0+ to
ease mechanical processing.
|
|
Fewer conditionals means theres fewer code paths to test
and makes things easier-to-read.
|
|
Same as nginx :>
|
|
Our nntpd and httpd are similar so configuration differences
should be minimized
|
|
Document and simplify things a bit. The major functional change
is we no longer waste space caching objects from dumb HTTP
clones.
|
|
We don't need to care about client IPs anywhere.
|
|
Well, I'm fumbling along with this config. Might as well
fumble along with it publically :)
|
|
It's browseable, too!
|
|
This means we can still show non-git users a somewhat browseable
URL with a link to the README.html file while allowing git users
to type less when cloning.
All of the following are supported:
git clone https://public-inbox.org/ public-inbox
git clone https://public-inbox.org/public-inbox
git clone https://public-inbox.org/public-inbox.git
torsocks git clone http://ou63pmih66umazou.onion/public-inbox
|
|
Might as well eat our own dogfood...
|
|
Because sometimes folks will want to download gigantic mboxes
or make large clones over Tor which are not resume-friendly.
Note: the timeout logic in nntpd is somewhat over-aggressive
and can break some large slrnpulls. This ought to be easily
recoverable on the client-side, though, since it's based on
per-message fetches.
|
|
For our daemons, killing only the master process is enough.
Killing the entire control group (as done by default in
systemd) may cause subprocesses such as git to shut down
unexpectedly.
Having systemd kill workers directly will also cause an
immediate shutdown since the master would've already signaled
the workers; and workers will die after two shutdown requests.
|
|
Since our daemons are built to take advantage of socket activation,
provide example files to allow systems administrators to hit the
ground running with systemd.
Example init files for other systems greatly appreciated.
|
|
This makes unsubscribing easier and frictionless.
|
|
We don't want people following links from archivers and
breaking archival.
|
|
* unsubscribe:
unsubscribe.milter: use default postfork dispatcher
unsubscribe: prevent decrypt from showing random crap
examples/unsubscribe-psgi@.service: disable worker processes
unsubscribe: bad URL fixup
unsubscribe: get off mah lawn^H^H^Hist
|