Date | Commit message (Collapse) |
|
Using create_inbox doesn't seem worth the trouble, here, at the
moment, but disabling fsync(2) gives a noticeable speedup on
my system even with an SSD.
|
|
Using "make update-copyrights" after setting GNULIB_PATH in my
config.mak
|
|
Nearly all of the search uses in the production code rely on
a Xapian mset iterator being returned (instead of an array
of $smsg objects). So default to returning the mset and move
the burden of smsg array conversion into the test cases.
|
|
`->connect' is confused with the perlfunc for the `connect(2)'
syscall, and also `DBI->connect'. Since SQLite doesn't use
sockets, the word "connect" needlessly confuses me. Give
it a short name to match the field name we use for it, which
also matches the variable name used by the DBI(3pm) and
DBD::SQLite(3pm) manpages.
|
|
We'll switch to using IdxStack here to ensure we get repeatable
results and ascending THREADIDs according to git chronology.
This means we'll need a two-pass reindex to index existing
messages before indexing new messages.
Since we no longer have a long-lived git-log process, we don't
have to worry about old Xapian referencing the git-log pipe
w/o FD_CLOEXEC, either.
|
|
Older versions of public-inbox < 1.3.0 had subtly
different semantics around threading in some corner
cases. This switch (when combined with --reindex)
allows us to fix them by regenerating associations.
|
|
The old name may be confused with "Content-ID" as described in
RFC 2392, so use an alternate name to avoid confusing future
readers.
|
|
We no longer load or use Email::MIME outside of comparison
tests.
|
|
Instead, favor PublicInbox::MIME->new for non-attachment emails.
We may support alternatives to Email::MIME down the line.
We'll still keep Email::MIME->create to deal with attachments,
for now, but there's also a fair amount of test duplication
we should eliminate, later.
|
|
PublicInbox::MIME only supports ->new, and is only different
from Email::MIME for old versions of Email::MIME. In the
future, PublicInbox::MIME may not be a subclass of Email::MIME
at all.
|
|
Allowing ->init_bare to be used as a method saves some
keystrokes, and we can save a little bit of time on systems with
our vfork(2)-enabled spawn().
This also sets us up for future improvements where we can
avoid spawning a process at all.
|
|
I didn't wait until September to do it, this year!
|
|
This cuts down on lines of code in individual test cases and
fixes some misnamed error messages by using "$0" consistently.
This will also provide us with a method of swapping out
dependencies which provide equivalent functionality (e.g
"Xapian" SWIG can replace "Search::Xapian" XS bindings).
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|
|
We'll also introduce a tmpdir() API to give tempdirs
consistent names.
|
|
"mainrepo" ws a bad name and artifact from the early days when I
intended for there to be a "spamrepo" (now just the
ENV{PI_EMERGENCY} Maildir). With v2, "mainrepo" can be
especially confusing, since v2 needs at least two git
repositories (epoch + all.git) to function and we shouldn't
confuse users by having them point to a git repository for v2.
Much of our documentation already references "INBOX_DIR" for
command-line arguments, so use "inboxdir" as the
git-config(1)-friendly variant for that.
"mainrepo" remains supported indefinitely for compatibility.
Users may need to revert to old versions, or may be referring
to old documentation and must not be forced to change config
files to account for this change.
So if you're using "mainrepo" today, I do NOT recommend changing
it right away because other bugs can lurk.
Link: https://public-inbox.org/meta/874l0ice8v.fsf@alyssa.is/
|
|
|
|
It did not cause a test failure because the default fallback
is `indexlevel=full'
|
|
More tests work without Search::Xapian, now.
Usability issues still need to be fixed
|
|
This allows v1 tests to continue working on git 1.8.0 for
now. This allows git 2.1.4 packaged with Debian 8 ("jessie")
to run old tests, at least.
I suppose it's safe to drop Debian 7 ("wheezy") due to our
dependency on git 1.8.0 for "merge-base --is-ancestor".
Writing V2 repositories requires git 2.6 for "get-mark"
support, so mask out tests for older gits.
|
|
Instrument the tests to verify the highwater num highwater mark is
where it is expected.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
Capture interesting commits of the test repository in mark variables.
Use those marks to build interesting scenarios where index_sync proceeds
as if those marks are the heads of the repositor. Use this capability to
test what happens when adds and deletes are mixed within a repository.
Be sad because things don't yet work as they should.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
Deeply inspect the entire message map in the reindexing tests
as the actual message order is significant and can result
in surprises.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
Place the expected value second in is and isnt tests because when
these tests fail they report the second value as the expected value.
A report saying got 0 expected 8 'no Xapian search results' can be confusing.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
While inspecting the tests I realized that because we have been
reusing variables there can be a memory between one test case and
another. Add scopes and local variables to prevent an unintended
memory between one test and another.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
For v1 repos, we don't need to write any metadata to Xapian
and changing from 'basic' to 'medium' or 'full' will work.
For v2, the metadata for indexing is stored in msgmap (because
the Xapian databases are partitioned for parallelism), so a
reindex is required.
|
|
The correct field is underscore-less for consistency with
git-config naming conventions. While we're at it, beef up
the v2 tests with actual size checks, too.
I also noticed phrase searching still seems to work for
the limited test case, so I left it documented; but the
size checking verifies the space savings.
|
|
This adds a new inbox configuration option 'indexlevel' that can take
the values 'full', 'medium', and 'basic'.
When set to 'full' everything is indexed including the positions
of all terms.
When set to 'medium' everything except the positions of terms is
indexed.
When set to 'basic' terms and positions are not indexed. Just the
Overview database for NNTP is created. Which is still quite good and
allows searching for messages by Message-ID. But there are no indexes to support
searching inside the email messages themselves.
Update the reindex tests to exercise the full medium and basic code paths
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|
|
Decrement regen_down when visiting messages that appear in %D that we
know will later be deleted. This ensures consistent message numbers are
generated no matter which commit number is on top. Allowing deletes to
propagage separately from the messages they delete without causing
problems.
The v2 trees already do this and when the indexes are deleted and
rebuilt they maintain they commit numbers.
Add a v1 version of the v2reindex test to verify that reindexing is
working properly on v1 as well as v2.
Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
|