| Date | Commit message (Collapse) |
|---|
|
For long-lived daemons across config reloads, we shouldn't keep
Xapian DBs open forever under FD pressure. So estimate the
number of FDs we need per-shard and start clearing some out
if we have too many open.
While we're at it, hoist out our ulimit_n helper and share it
across extindex and the Perl XapHelper implementation.
|
|
systemd setups may use role accounts (e.g. `news') with
XDG_CACHE_HOME unset and a non-existent HOME directory
which the user has no permission to create.
In those cases, fallback to using PERL_INLINE_DIRECTORY if
available for building the just-ahead-of-time C++ binary.
|
|
While fast build times from -O0 is critical to my sanity when
actively working on C++, the files installed via package
managers or `make install' aren't likely to change frequently.
In that case, expensive -O2 optimizations make sense since the
10-20s saved from a single large --join more than covers the
cost of waiting on g++ to optimize.
|
|
This makes the C++ build work on Alpine Linux (tested 3.19.0)
without having to install g++ to get the `c++' executable.
I've tested this change with and without g++ on Alpine so it'll
continue to work if a user decides to install g++.
This should continue to work if the Xapian package on Alpine is
changed to link against libc++ instead of libstdc++, since we
only add `-lstdc++' as a fallback. For reference, Xapian is
already linked against libc++ and not libstdc++ on FreeBSD 13.x
|
|
We don't actually need Inline::C support to build a standalone
executable implemented in C++.
|
|
While chdir simplifies path manipulation on our end, its use
falls over when PERL5LIB/@INC contains relative paths which need
to be made absolute. It's fewer lines of code to get eliminate
chdir usage than it is to keep using relative paths in most
places.
|
|
The C++ version will allow us to take full advantage of Xapian's
APIs for better queries, and the Perl bindings version can still
be advantageous in the future since we'll be able to support
timeouts effectively.
|
|
It ought to help a bit with organization since xap_helper.h
is getting somewhat large and we'll need new endpoints to
support WWW, lei, and whatever else that needs to come.
|
|
This also reduces repetition in the setup code.
|
|
No need to waste memory bandwidth when we can just rely on
the preprocessor to load the header.
|
|
Eric Wong <e@80x24.org> wrote:
> Avoid mixing autodie use in different scopes since it's likely
> to cause problems like it did in Gcf2. While none of these
> fix known problems with test cases, it's likely worthwhile to
> avoid it anyways to avoid future surprises.
> lib/PublicInbox/XapHelperCxx.pm | 18 ++++++++----------
That XapHelperCxx change was totally necessary for running the
C++ build on CentOS 7.x (but the test is auto-skipped on any
build failure), as is this one:
--------8<--------
Subject: [PATCH] xap_helper_cxx: accept leading spaces from pkg-config
pkg-config 0.27.1 and xapian14-core-devel (1.4.24-1.el7) on
CentOS 7.x will print a leading space when running
`pkg-config --libs --cflags xapian-core'. This leading
space creates an empty string when `split' with /\s+/ as
a pattern. Instead, use the documented ' ' (SP) character
to put split into "awk mode" which eats leading (and
redundant) spaces and tabs.
|
|
Avoid mixing autodie use in different scopes since it's likely
to cause problems like it did in Gcf2. While none of these
fix known problems with test cases, it's likely worthwhile to
avoid it anyways to avoid future surprises.
For Process::IO, we'll add some additional tests in t/io.t
to ensure we don't get unintended exceptions for try_cat.
|
|
No need to suffer through an extra dose of slow Perl load times
when we can drive the build in the big parent Perl process and
get the executable path name to pass to spawn directly.
|
|
-ggdb3 is already used for g++ and clang, and -pipe is supported
by clang even if it's a no-op. So just use it to speed up g++
since it saves me 30-40ms.
We'll also get rid of the explicit `-O0' since it's the default
for both clang and g++.
|
|
We need to have stable filenames and separate compilation
from the linkage stage for ccache to hit. So avoid the use
of a temporary directory and instead rely on a lock file to
guard against parallel builds.
|
|
PublicInbox::IO already gets loaded by PublicInbox::Spawn, so
there's no avoiding it even if we want fast startup time :<
But startup time for this piece will be less relevant in the
near future...
|
|
The IO package seems like a better home for I/O subs than the
Git package. We lose the 60 second read timeout for `git
cat-file --batch-*' processes since it's probably not necessary
given how reliable the code has proven and things would fall
over hard in other ways if the storage device were completely
hosed.
|
|
This saves us some code, and is a small step towards getting
ProcessIO working with stat, fcntl and other perlops that don't
work with tied handles.
|
|
This ought to lower the cognitive overhead of reading our code.
|
|
`readline' ops may not detect errors on partial reads.
This saves us some code to reduce cognitive overhead for
readers. We'll also support reusing a destination buffers so it
can work more nicely with existing code.
|
|
[ew: leave install/README unchanged pending wording changes]
|
|
For users sharing home directories (or just XDG_CACHE_HOME)
across hosts of different architectures, we must use a compiler
and architecture-specific destination directory for storing the
binary result. Even on the same OS and architecture, different
C++ compilers may have different ABIs, so we must account for
that.
|
|
As with libgit2, Xapian can be upgraded and break linkage
with the xap_helper binary. For now, save the result of
`pkg-config --modversion xapian-core' to a file and
compare it with the current output.
Perhaps portable some make(1) can be used for this...
|
|
While rpath is frowned upon by Debian and other distro packagers; it
appears embraced by in the world of NetBSD ports and packages. This is
because ldconfig(8) on NetBSD doesn't put /usr/pkg/lib in its search
path by default. This behavior differs from the ports and packaging
systems of FreeBSD and OpenBSD which do search library paths of
pkg*-installed packages (and presumably ports).
|
|
ld(1) on OpenBSD 7.3 doesn't appear to support zlib-compressed
debug sections out-of-the-box. Oh well, being able to build
this C++ bit at all is required to get acceptable performance
with -cindex --associate.
|
|
This allows us to perform the expensive "dump_ibx" operations in
native C++ code using the Xapian C++ library. This provides the
majority of the speedup with the -cindex --associate switch.
Eventually this may be expanded to cover all uses of Xapian
within the project to ensure we have access to Xapian APIs which
aren't available in XS|SWIG bindings; and also for
ease-of-installation on systems which don't provide
pre-packaged Perl Xapian bindings (e.g. OpenBSD 7.3) but
do provide Xapian development libraries.
Most of the C++ code is still C, as I'm not remotely familiar
with C++ compared to C. I suspect many users and potential
hackers being from git, Linux kernel, and glibc world are in the
same boat.
|