* [PATCH 9/9] searchidxshard: simplify newline elimination
@ 2020-12-15 2:02 5% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-12-15 2:02 UTC (permalink / raw)
To: meta
This overdue change fixes {current_info} to not inject a newline
into every warning message.
Simpler code helps us avoid bugs and the need to make
fixes like commit 44de182766037948d62bc2a8ba924de2264dd5fc
("searchidxshard: chomp $eidx_key from pipe").
---
lib/PublicInbox/SearchIdxShard.pm | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/lib/PublicInbox/SearchIdxShard.pm b/lib/PublicInbox/SearchIdxShard.pm
index ee00858b..2e654769 100644
--- a/lib/PublicInbox/SearchIdxShard.pm
+++ b/lib/PublicInbox/SearchIdxShard.pm
@@ -69,33 +69,31 @@ sub shard_worker_loop ($$$$$) {
$0 = "shard[$shard]";
$self->begin_txn_lazy;
while (my $line = readline($r)) {
+ chomp $line;
$v2w->{current_info} = "[$shard] $line";
- if ($line eq "commit\n") {
+ if ($line eq 'commit') {
$self->commit_txn_lazy;
- } elsif ($line eq "close\n") {
+ } elsif ($line eq 'close') {
$self->idx_release;
- } elsif ($line eq "barrier\n") {
+ } elsif ($line eq 'barrier') {
$self->commit_txn_lazy;
# no need to lock < 512 bytes is atomic under POSIX
print $bnote "barrier $shard\n" or
die "write failed for barrier $!\n";
- } elsif ($line =~ /\AD ([0-9]+)\n\z/s) {
+ } elsif ($line =~ /\AD ([0-9]+)\z/s) {
$self->remove_by_docid($1 + 0);
} elsif ($line =~ s/\A\+X //) {
my ($len, $docid, $eidx_key) = split(/ /, $line, 3);
- chomp $eidx_key;
$self->add_eidx_info($docid, $eidx_key, eml($r, $len));
} elsif ($line =~ s/\A-X //) {
my ($len, $docid, $eidx_key) = split(/ /, $line, 3);
- chomp $eidx_key;
$self->remove_eidx_info($docid, $eidx_key,
eml($r, $len));
- } elsif ($line =~ s/\AO ([^\n]+)\n//) {
+ } elsif ($line =~ s/\AO ([^\n]+)//) {
my $over_fn = $1;
$over_fn =~ tr/\0/\n/;
$self->over_check(PublicInbox::Over->new($over_fn));
} else {
- chomp $line;
my $eidx_key;
if ($line =~ s/\AX=(.+)\0//) {
$eidx_key = $1;
^ permalink raw reply related [relevance 5%]
* [PATCH] searchidxshard: chomp $eidx_key from pipe
@ 2020-11-28 7:06 7% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-11-28 7:06 UTC (permalink / raw)
To: meta
We were accidentally adding "\n" to terms (which Xapian happily
accepts), causing incompatibilities when enabling parallel
sharding in some invocations of -extindex but not others.
This is an extindex incompatibility and starting a new extindex
will be required to take advantage of in-development features,
so it's not urgent to start another one, either.
(other incompatible things may happen before a 1.7 release)
---
lib/PublicInbox/SearchIdxShard.pm | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/PublicInbox/SearchIdxShard.pm b/lib/PublicInbox/SearchIdxShard.pm
index 875a9ec9..dcfeb0be 100644
--- a/lib/PublicInbox/SearchIdxShard.pm
+++ b/lib/PublicInbox/SearchIdxShard.pm
@@ -83,11 +83,13 @@ sub shard_worker_loop ($$$$$) {
} elsif ($line =~ s/\A\+X //) {
my ($len, $docid, $oid, $eidx_key) =
split(/ /, $line, 4);
+ chomp $eidx_key;
$self->add_eidx_info($docid, $oid, $eidx_key,
eml($r, $len));
} elsif ($line =~ s/\A-X //) {
my ($len, $docid, $oid, $eidx_key) =
split(/ /, $line, 4);
+ chomp $eidx_key;
$self->remove_eidx_info($docid, $oid, $eidx_key,
eml($r, $len));
} else {
^ permalink raw reply related [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-11-28 7:06 7% [PATCH] searchidxshard: chomp $eidx_key from pipe Eric Wong
2020-12-15 2:02 [PATCH 0/9] extindex: --reindex support Eric Wong
2020-12-15 2:02 5% ` [PATCH 9/9] searchidxshard: simplify newline elimination Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).