* [PATCH 02/14] t/cindex*: require SCM_RIGHTS for these tests
2023-11-28 14:56 5% [PATCH 00/14] IT'S ALIVE! www loads cindex join data Eric Wong
@ 2023-11-28 14:56 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2023-11-28 14:56 UTC (permalink / raw)
To: meta
Code search will require SCM_RIGHTS, and Inline::C on BSDs
probably isn't too onerous a dependency for new features as
all the ones I've tested have it packaged.
Furthermore, requiring SCM_RIGHTS isn't far off since OpenBSD's
Perl is patched to route the `syscall' perlop through libc[1],
while NetBSD[2] and FreeBSD[3] actually do strive for backwards
compatibility. We'd just need to use the numbers and not rely
on syscall.ph shipped with Perl since the macro names themselves
are unstable.
[1] https://cvsweb.openbsd.org/src/gnu/usr.bin/perl/gen_syscall_emulator.pl
[2] https://www.netbsd.org/docs/internals/en/chap-processes.html#syscall_versioning
[3] https://wiki.freebsd.org/AddingSyscalls#Backward_compatibily
---
t/cindex-join.t | 2 +-
t/cindex.t | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/t/cindex-join.t b/t/cindex-join.t
index 2836eb6c..ac90cd64 100644
--- a/t/cindex-join.t
+++ b/t/cindex-join.t
@@ -12,7 +12,7 @@ use autodie;
use File::Spec;
$ENV{TEST_REMOTE_JOIN} or plan skip_all => 'TEST_REMOTE_JOIN unset';
local $ENV{TAIL_ALL} = $ENV{TAIL_ALL} // 1; # while features are unstable
-require_mods(qw(json Xapian DBD::SQLite));
+require_mods(qw(json Xapian DBD::SQLite +SCM_RIGHTS));
my @code = qw(https://80x24.org/mwrap-perl.git
https://80x24.org/mwrap.git);
my @inboxes = qw(https://80x24.org/mwrap-public 2 inbox.comp.lang.ruby.mwrap
diff --git a/t/cindex.t b/t/cindex.t
index 1a9e564a..261945bf 100644
--- a/t/cindex.t
+++ b/t/cindex.t
@@ -6,7 +6,7 @@ use PublicInbox::TestCommon;
use Cwd qw(getcwd abs_path);
use List::Util qw(sum);
use autodie qw(close open rename);
-require_mods(qw(json Xapian));
+require_mods(qw(json Xapian +SCM_RIGHTS));
use_ok 'PublicInbox::CodeSearchIdx';
use PublicInbox::Import;
my ($tmp, $for_destroy) = tmpdir();
^ permalink raw reply related [relevance 7%]
* [PATCH 00/14] IT'S ALIVE! www loads cindex join data
@ 2023-11-28 14:56 5% Eric Wong
2023-11-28 14:56 7% ` [PATCH 02/14] t/cindex*: require SCM_RIGHTS for these tests Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2023-11-28 14:56 UTC (permalink / raw)
To: meta
8/14 is the killer one which actually makes the cindex data
useful for WWW and powering solver. Keep in mind, I've had
to cap solver at 3 coderepos as a temporary measure since
there's a lot of "weak" joins we should be weeding out.
More documentation coming, but cindex joins are very much
a fuzzy thing which will have to deal with false positives
and such. So figuring out the scoring for sanity would
make sense...
Fortunately, --join=aggressive,reset only takes ~1 hour for me,
so probably 1/3 that on modern hardware. Incremental
`-cindex --join' (no suboptions) usually takes <5 minutes if
done frequently.
New performance problem: solver could definitely be smarter
about dealing with common roots/groups. For the longest time,
I've only had 1 coderepo per-inbox, having hundreds is wacky.
Actual searching against the cindex isn't done, yet, but
that's kinda straightforward.
Eric Wong (14):
test_common: create_*: detect changes all parameters
t/cindex*: require SCM_RIGHTS for these tests
codesearch: eliminate redundant substitutions
solver: schedule cleanup after synchronous git->check
xap_helper.h: move cindex endpoints to separate file
xap_helper: implement mset endpoint for WWW, IMAP, etc...
hval: use File::Spec to make relative paths for href
www: load and use cindex join data
git: speed up ->git_path for non-worktrees
cindex: require `-g GIT_DIR' or `-r PROJECT_ROOT'
git: speed up Git->new by 5% or so
admin: resolve_git_dir respects symlinks
cindex: extra quit checks
www: start working on a repo listing
Documentation/public-inbox-cindex.pod | 2 +-
MANIFEST | 3 +
Makefile.PL | 8 +-
lib/PublicInbox/Admin.pm | 25 +-
lib/PublicInbox/CodeSearch.pm | 162 ++++++++++-
lib/PublicInbox/CodeSearchIdx.pm | 52 ++--
lib/PublicInbox/Config.pm | 39 ++-
lib/PublicInbox/Git.pm | 27 +-
lib/PublicInbox/Hval.pm | 12 +-
lib/PublicInbox/RepoList.pm | 39 +++
lib/PublicInbox/Search.pm | 42 +++
lib/PublicInbox/SearchIdx.pm | 10 +-
lib/PublicInbox/SolverGit.pm | 9 +-
lib/PublicInbox/TestCommon.pm | 35 ++-
lib/PublicInbox/View.pm | 7 +-
lib/PublicInbox/WWW.pm | 1 +
lib/PublicInbox/WwwCoderepo.pm | 44 ++-
lib/PublicInbox/WwwStream.pm | 11 +-
lib/PublicInbox/WwwText.pm | 19 +-
lib/PublicInbox/XapHelper.pm | 51 ++--
lib/PublicInbox/XapHelperCxx.pm | 14 +-
lib/PublicInbox/xap_helper.h | 379 +++++++-------------------
lib/PublicInbox/xh_cidx.h | 244 +++++++++++++++++
lib/PublicInbox/xh_mset.h | 96 +++++++
script/public-inbox-cindex | 38 ++-
t/admin.t | 12 +
t/cindex-join.t | 9 +-
t/cindex.t | 91 ++++++-
t/xap_helper.t | 53 +++-
xt/solver.t | 3 +-
30 files changed, 1111 insertions(+), 426 deletions(-)
create mode 100644 lib/PublicInbox/RepoList.pm
create mode 100644 lib/PublicInbox/xh_cidx.h
create mode 100644 lib/PublicInbox/xh_mset.h
^ permalink raw reply [relevance 5%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-11-28 14:56 5% [PATCH 00/14] IT'S ALIVE! www loads cindex join data Eric Wong
2023-11-28 14:56 7% ` [PATCH 02/14] t/cindex*: require SCM_RIGHTS for these tests Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).