* [PATCH 0/5] www: start using xap_helper process
@ 2024-04-24 6:44 7% Eric Wong
2024-04-24 6:44 6% ` [PATCH 2/5] xap_helper.h: remove _SC_NPROCESSORS_ONLN default Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2024-04-24 6:44 UTC (permalink / raw)
To: meta
Putting Xapian work into subprocesses will allow us to implement
proper timeouts and ultra-expensive queries without harming
unrelated queries.
It's on by default right now, but I think it's better to keep it
off by default to avoid tripping up existing process monitoring
tools.
Eric Wong (5):
searchview: get rid of unused adump callback arg
xap_helper.h: remove _SC_NPROCESSORS_ONLN default
xap_helper: drop terms+data from `mset' command
mbox: hoist out refill_result_ids
www: wire up search to use async xap_helper
MANIFEST | 2 +
lib/PublicInbox/Isearch.pm | 50 +++++++-----
lib/PublicInbox/Mbox.pm | 127 ++++++++++++++++-------------
lib/PublicInbox/MboxGz.pm | 4 +-
lib/PublicInbox/Search.pm | 53 +++++++++++-
lib/PublicInbox/SearchView.pm | 58 +++++++------
lib/PublicInbox/XapClient.pm | 1 +
lib/PublicInbox/XapHelper.pm | 16 +---
lib/PublicInbox/XhcMset.pm | 51 ++++++++++++
lib/PublicInbox/XhcMsetIterator.pm | 20 +++++
lib/PublicInbox/xap_helper.h | 17 ++--
lib/PublicInbox/xh_mset.h | 57 ++-----------
t/cindex.t | 23 +++---
t/xap_helper.t | 29 +++----
14 files changed, 302 insertions(+), 206 deletions(-)
create mode 100644 lib/PublicInbox/XhcMset.pm
create mode 100644 lib/PublicInbox/XhcMsetIterator.pm
^ permalink raw reply [relevance 7%]
* [PATCH 2/5] xap_helper.h: remove _SC_NPROCESSORS_ONLN default
2024-04-24 6:44 7% [PATCH 0/5] www: start using xap_helper process Eric Wong
@ 2024-04-24 6:44 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2024-04-24 6:44 UTC (permalink / raw)
To: meta
It's never straightforward to pick an ideal number of processes
for anything and Xapian helper processes are no exception since
there may be a massive disparities in CPU count and I/O
performance. So default to a single worker for now in the C++
version since that's the default is for the Perl/(XS|SWIG)
version, and also the same as for our normal public-facing
daemons.
This keeps the behavior between the Perl+(XS|SWIG) and C++
version as similar as possible.
---
lib/PublicInbox/xap_helper.h | 6 ------
1 file changed, 6 deletions(-)
diff --git a/lib/PublicInbox/xap_helper.h b/lib/PublicInbox/xap_helper.h
index 3456910b..0e6805b3 100644
--- a/lib/PublicInbox/xap_helper.h
+++ b/lib/PublicInbox/xap_helper.h
@@ -945,12 +945,6 @@ int main(int argc, char *argv[])
}
nworker = 1;
-#ifdef _SC_NPROCESSORS_ONLN
- long j = sysconf(_SC_NPROCESSORS_ONLN);
- if (j > 0)
- nworker = j > WORKER_MAX ? WORKER_MAX : j;
-#endif // _SC_NPROCESSORS_ONLN
-
// make warn/warnx/err multi-process friendly:
if (my_setlinebuf(stderr))
err(EXIT_FAILURE, "setlinebuf(stderr)");
^ permalink raw reply related [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2024-04-24 6:44 7% [PATCH 0/5] www: start using xap_helper process Eric Wong
2024-04-24 6:44 6% ` [PATCH 2/5] xap_helper.h: remove _SC_NPROCESSORS_ONLN default Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).