* [PATCH 1/9] t/multi-mid.t: extra test for -convert highwater mark
2020-02-02 6:52 6% [PATCH 0/9] convert: more fixes and tests Eric Wong
@ 2020-02-02 6:52 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2020-02-02 6:52 UTC (permalink / raw)
To: meta
This is derived from a real-world test case where I encounterd
multiple Message-IDs in a v1 inbox causing regen problems.
Fixes: eea47b676127bcdb ("convert: preserve highwater mark from v1 msgmap")
---
MANIFEST | 1 +
t/multi-mid.t | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 62 insertions(+)
create mode 100644 t/multi-mid.t
diff --git a/MANIFEST b/MANIFEST
index 3736c777..5eb5d53a 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -250,6 +250,7 @@ t/mime.t
t/msg_iter.t
t/msgmap.t
t/msgtime.t
+t/multi-mid.t
t/nntp.t
t/nntpd-tls.t
t/nntpd.t
diff --git a/t/multi-mid.t b/t/multi-mid.t
new file mode 100644
index 00000000..0d283a0d
--- /dev/null
+++ b/t/multi-mid.t
@@ -0,0 +1,61 @@
+# Copyright (C) 2020 all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+use Test::More;
+use PublicInbox::MIME;
+use PublicInbox::TestCommon;
+use PublicInbox::InboxWritable;
+use PublicInbox::SearchIdx;
+require_git(2.6);
+require_mods(qw(DBD::SQLite));
+
+my $addr = 'test@example.com';
+my $bad = PublicInbox::MIME->new(<<EOF);
+Message-ID: <a\@example.com>
+Message-ID: <b\@example.com>
+From: a\@example.com
+To: $addr
+Date: Fri, 02 Oct 1993 00:00:00 +0000
+Subject: bad
+
+EOF
+
+my $good = PublicInbox::MIME->new(<<EOF);
+Message-ID: <b\@example.com>
+Date: Fri, 02 Oct 1993 00:00:00 +0000
+From: b\@example.com
+To: $addr
+Subject: good
+
+EOF
+
+for my $order ([$bad, $good], [$good, $bad]) {
+ my $before;
+ my ($tmpdir, $for_destroy) = tmpdir();
+ my $ibx = PublicInbox::InboxWritable->new({
+ inboxdir => "$tmpdir/v1",
+ name => 'test-v1',
+ indexlevel => 'basic',
+ -primary_address => $addr,
+ }, my $creat_opt = {});
+ if ('setup v1 inbox') {
+ my $im = $ibx->importer(0);
+ ok($im->add($_), 'added '.$_->header('Subject')) for @$order;
+ $im->done;
+ my $s = PublicInbox::SearchIdx->new($ibx, 1);
+ $s->index_sync;
+ $before = [ $ibx->mm->minmax ];
+ $ibx->cleanup;
+ }
+ my $rdr = { 1 => \(my $out = ''), 2 => \(my $err = '') };
+ my $cmd = [ '-convert', $ibx->{inboxdir}, "$tmpdir/v2" ];
+ ok(run_script($cmd, undef, $rdr),
+ 'convert to v2');
+ $err =~ s!\AW: $tmpdir/v1 not configured[^\n]+\n!!s;
+ is($err, '', 'no errors or warnings from -convert');
+ $ibx->{version} = 2;
+ $ibx->{inboxdir} = "$tmpdir/v2";
+ is_deeply([$ibx->mm->minmax], $before,
+ 'min, max article numbers unchanged');
+}
+
+done_testing();
^ permalink raw reply related [relevance 7%]
* [PATCH 0/9] convert: more fixes and tests
@ 2020-02-02 6:52 6% Eric Wong
2020-02-02 6:52 7% ` [PATCH 1/9] t/multi-mid.t: extra test for -convert highwater mark Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2020-02-02 6:52 UTC (permalink / raw)
To: meta
Nothing major, but SMP systems should be better utilized
now that -convert can take advantage of parallel shard
indexing in Xapian. The switches for convert are also
now documented in the manpage.
Eric Wong (9):
t/multi-mid.t: extra test for -convert highwater mark
v2writable: nproc_shards: subtract 1 from given value
v2writable: do not clobber {shards} or {parallel} if unset
convert: remove unused variables capturing :from
searchidxshard: rely on autoflush instead of ->flush
convert: shift @ARGV explicitly
convert: fix --no-index switch
doc: -convert: document switches
v2writable: more ways to detect online CPU count
Documentation/public-inbox-convert.pod | 41 ++++++++++++++++-
MANIFEST | 1 +
lib/PublicInbox/SearchIdxShard.pm | 4 +-
lib/PublicInbox/V2Writable.pm | 38 +++++++++++-----
script/public-inbox-convert | 16 ++-----
t/convert-compact.t | 3 ++
t/multi-mid.t | 61 ++++++++++++++++++++++++++
t/xcpdb-reshard.t | 2 +-
8 files changed, 137 insertions(+), 29 deletions(-)
create mode 100644 t/multi-mid.t
^ permalink raw reply [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2020-02-02 6:52 6% [PATCH 0/9] convert: more fixes and tests Eric Wong
2020-02-02 6:52 7% ` [PATCH 1/9] t/multi-mid.t: extra test for -convert highwater mark Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).