about summary refs log tree commit homepage
path: root/ci
DateCommit message (Collapse)
2023-09-20drop GNU nproc(1) support in favor of getconf(1)
`getconf NPROCESSORS_ONLN' will succeed on GNU/Linux systems anyways; and the non-underscore-prefixed invocation works fine on all BSD flavors tested. Thus the `nproc' and `gnproc' attempts will never be reached. The only downside is we lose the ability to account for CPU affinity, but that's probably not an issue since CPU affinity (AFAIK) isn't a commonly-used feature.
2023-09-20ci/run: favor `make check' as the default target
While parallelism isn't always a good thing, `make check' has more verbose output to help us track down occasionally failing tests.
2023-09-14install/deps: safer defaults for regular users
We'll leave forcing `--yes' to ci/run.sh and remove --purge usage with apt-get(1) entirely. Also start defining some more profiles aimed at users who want a minimal install for the subset of public-inbox they wish to use. There'll be some more built-in dependency handling to work across different distros, but the $always_deps thing is a start.
2023-09-14move deps.perl into new install/ directory
deps.perl can be useful for non-CI purposes as long as it's not blindly removing packages. Thus, a --allow-remove flag now exists for CI use and removals are disabled by default. deps.perl also gets easier-to-use in that now install/os.perl is split off from from ci/profiles.perl so OS-supplied packaged manager.
2023-09-11treewide: favor Xapian (SWIG binding) over Search::Xapian
The Xapian SWIG bindings are favored by Xapian upstream for ease-of-maintenance compared to the XS version. While Debian lags on this front, the SWIG bindings are widely available on all *BSDs.
2023-09-10ci/profiles: strip everything after the `-' in utsname.release
This fixes the script under FreeBSD (tested 13.2) FreeBSD 13.2 has `13.2-RELEASE-p3' in its uname(2) utsname.release. While the `.2' component is a welcome addition over the old script, Perl parses the `-' as a subtraction operation, which isn't what we want.
2023-09-09update CI helper scripts for NetBSD and `pkgin'
2023-09-09ci/profiles: rewrite in Perl
Reading os-release(5) is a bit more painful, now; and still requires using the shell. However, sharing code between *BSDs and being able to use v-strings for version comparisons is much easier. Test profiles for *BSDs are also trimmed down and more focused on portability stuff.
2023-09-09ci/run.sh: parameterize BUILD_JOBS TEST_JOBS and TEST_TARGET
Parallezing BUILD_JOBS is usually harmless, but TEST_JOBS can be problematic when tracking down problems on new platforms. TEST_TARGET can be `check' or `check-run' for performance.
2023-09-08ci/deps: redo and fix essential package handling
git depends on p5-TimeDate on FreeBSD, too, so ensure git doesn't get uninstalled on FreeBSD. Instead of making @precious a separate array, we can actually stuff dependencies into the $non_auto map and save us some code. We can also eliminate some duplication in $non_auto by populating the Perl standard library packages in a loop.
2023-09-08ci: updates for OpenBSD
Still a work-in-progress, but OpenBSD's pkg_add/pkg_delete seem to be working somewhat. The dependency system seems to need some extra help to ensure leaf packages with their own dependencies (e.g. `xapian-bindings-perl') get uninstalled before their dependencies (`xapian-core'). Deduplicating the command-line is also required since both pkg_add and pkg_delete will repeat the installation/removal if a package is specified multiple times in the same invocation.
2023-09-08ci/deps: drop unnecessary mappings and add Inline
The automatic mapping can work for more packages, so redundant entries in $non_auto are just clutter. Unfortunately, `Inline::C' is part of `Inline' on CentOS 7.x and OpenBSD 7.3, so we'll add $non_auto mappings for those. We'll also depend on `IO::Compress' to simplify mappings since that's the CPAN distribution which holds both IO::Compress::Gzip and IO::Compress::Gunzip and I'm not aware of any packagers who split them.
2023-09-08ci/deps: add IMAP-related optional packages
Mail::IMAPClient and Parse::RecDescent are widely available in all relevant OS package/ports systems.
2023-09-08update docs + tests for xapian-delve use
Since -cindex uses the xapian-delve(1) command for `--prune' functionality, we'll rename our `xapian-compact' dependency to the Debian package name (xapian-tools) since `xapian-delve' is in the same package.
2023-08-28Fix some typos/grammar/errors in docs and comments
2023-08-28ci/profiles.sh: fix case matching logic
'-' could never match, remove that alternative (it might have been a typo of '--', but that is already covered by '*--|--*' ('*' matches the null string)). Replace '*--*' with the equivalent '*' ('--' is always present). It would seem clearer to just replace the whole case command with something like '[ "$ID" -a "$VERSION_ID" ] && break' (or the POSIX-non-deprecated equivalent '[ "$ID" ] && [ "$VERSION_ID" ]' ); I assume a preference of using case here (e.g., to avoid syscall overhead in case [ is not implemented as a shell builtin (which seems far-fetched given the context, though)).
2021-09-23daemons: revamp periodic cleanup task
Neither Inboxes nor ExtSearch objects were retrying correctly when there are live git processes, but the inboxes were getting rescanned for search or other reasons. Ensure the scan retries eventually if there's live processes. We also need to update the cleanup task to detect Xapian shard count changes, since Xapian ->reopen is enough to detect any other Xapian changes. Otherwise, we just issue an inexpensive ->reopen call and let Xapian check whether there's anything worth reopening. This also lets us eliminate the Devel::Peek dependency.
2021-09-10INSTALL: depend on URI rather than URI::Escape
As far as I can tell, URI::Escape has always been a part of the `URI' package (aka "distribution" on CPAN) and not distributed separately (unlike URI::Escape::XS). So avoid confusing users with `URI::Escape' and just document `URI' instead. Along the same lines, we depend on the `Plack' package rather than Plack::Util or Plack::Builder, after all.
2021-01-01update copyrights for 2021
Using "make update-copyrights" after setting GNULIB_PATH in my config.mak
2020-09-15ci/deps: add Plack::Test::ExternalServer for devtest
More of our Plack tests exercise public-inbox-httpd, nowadays; and ExternalServer lets us test it easily alongside generic PSGI stuff.
2020-07-06stop auto-loading Plack::Middleware::Deflater
Instead of gzipping some (mbox.gz, manifest.js.gz) responses and leaving P::M::D to do the rest, we gzip everything ourselves, now, so P::M::D is redundant.
2020-06-28watch: remove Filesys::Notify::Simple dependency
Since we already use inotify and EVFILT_VNODE (kqueue) in -imapd, we might as well use them directly in -watch, too. This will allow public-inbox-watch to use PublicInbox::DS for timers to watch newsgroups/mailboxes and have saner signal handling in future commits.
2020-05-09remove most internal Email::MIME usage
We no longer load or use Email::MIME outside of comparison tests.
2020-04-22make zlib-related modules a hard dependency
This allows us to simplify some of our existing code and make future changes easier. I doubt anybody goes through the trouble to have a Perl installation without zlib support. The zlib source code is even bundled with Perl since 5.9.3 for systems without existing zlib development headers and libraries. Of course, zlib is also a requirement of git, too; and we're not going to stop using git :) [squashed: "wwwaltid: use gzipfilter up front"]
2020-02-07tests: switch to XML::TreePP for testing Atom feeds
XML::Feed pulls in a lot of dependencies, some of which XS. That makes testing with blead or any non-OS-supplied Perl installations more time consuming and more difficult because of the need to have development headers and libraries for libexpat1 or libxml2. Performance from libexpat1 or libxml2 for our small tests cases isn't relevant, either, and the pure Perl XML::TreePP seems up to the task. It's also available in CentOS 7.x, FreeBSD 11.x, and Debian, at least.
2020-02-06treewide: run update-copyrights from gnulib for 2019
I didn't wait until September to do it, this year!
2020-01-11make Filesys::Notify::Simple optional
It's only used by us in public-inbox-watch, and maybe not for long. It's in most installations because Plack pulls it in though, but Plack is no longer required.
2020-01-11make Plack optional for non-WWW and non-httpd users
Some users just want to run -mda, -watch, and/or -nntpd. Let them run just those without forcing them to pull in a bunch of dependencies.
2019-12-12Date::Parse is now optional
-mda should not be dealing with broken Date: headers nowadays, and deprioritize it in our documentation and internal checks.
2019-11-16doc: remove IPC::Run as a dev and test dependency
It's no longer needed and we're able to speed up some of our tests as a result.
2019-06-13doc: update dependencies for CentOS 7.x
Digest::SHA is the most notable missing package at runtime for a minimal system. Tests don't run at all without Test::Simple (or Test::More). Plack::Test is also a separate package, too... Also, the package for IO::Compress::Gzip should be IO::Compress; as perl-PerlIO-gzip is a different thing entirely which is not relevant to our needs. Test::HTTP::Server::Simple doesn't seem required at all for Plack tests. ExtUtils::MakeMaker needs to be documented as a install dependency for people installing this, too; since AFAIK public-inbox is not yet in any distros.
2019-05-22ci: support CentOS-7
Tested on an amd64 chroot built with rinse 3.4
2019-05-15lazy load Xapian and make it optional for v2
More tests work without Search::Xapian, now. Usability issues still need to be fixed
2019-05-09ci: include some scripts to make CI easier
This should make it easier to test a bunch of package installation profiles across whatever OS isolation one chooses (chroots, containers, jails, VMs).