Date | Commit message (Collapse) |
|
We moved to PublicInbox::Eml a while back and have no plans
to go back to using Email::MIME, so don't tempt users and
packagers to waste disk space on Email::MIME.
|
|
We'll rely on defined(wantarray) to implicitly skip subtests,
and memoize these to reduce syscalls, since tests should
be short-lived enough to not be affected by new installations or
removals of git/xapian-compact/curl/etc...
|
|
While SHA-256 isn't supported for inboxes, yet
xt/git-http-backend.t now runs properly against a SHA-256 code
repository
|
|
We don't need to be keeping the raw message around after it hits
git. Shard work now relies on Storable (or Sereal) and all of
the indexing code relies on the Email::MIME-like API of Eml to
access interesting parts of the message.
Similarly, smsg->{raw_bytes} is no longer carried around and we
do the CRLF adjustment when setting smsg->{bytes}.
There's also a small simplification to t/import.t while
we're in the area to use xqx instead of spawn/popen_rd.
|
|
Using "make update-copyrights" after setting GNULIB_PATH in my
config.mak
|
|
We can use open(..., undef) natively in Perl in t/import.t
In places where we need a pathname, the File::Temp OO API
gives us auto-unlinking for free.
|
|
The eval in key2sub via t/run.perl ("make check-run") won't
trigger the warning, but running "prove -bvw t/import.t"
directly, does. In any case, ensure the contents of this
variable doesn't linger across runs.
|
|
Spammers may send emails with nasty characters which can throw
off git-fast-import. Users with non-existent or weaker spam
filters may be susceptible to corruption in the fast-import
stream as a result.
This was actually quietly fixed in git on 2020-06-01 by
commit 9ab886546cc89f37819e1ef09cb49fd9325b3a41
("smsg: introduce ->populate method"), but no test case
was created.
Reported-by: Eric W. Biederman <ebiederm@xmission.com>
Link: https://public-inbox.org/meta/87imf4qn87.fsf@x220.int.ebiederm.org/
Link: https://public-inbox.org/meta/20200601100657.14700-6-e@yhbt.net/
|
|
NNTP and IMAP both require CRLF conversions on the wire.
They're also the only components which care about
$smsg->{bytes}, so store the CRLF-adjusted value in over.sqlite3
and Xapian DBs..
This will allow us to optimize RFC822.SIZE fetch item in IMAP
without triggering size mismatch errors in some clients' default
configurations (e.g. Mail::IMAPClient), but not most others.
It could also fix hypothetical problems with NNTP clients that
report discrepancies between overview and article data.
|
|
This will eventually replace the __hdr() calling methods and
eradicate {mime} usage from Smsg. For now, we can eliminate
PublicInbox::Smsg->new since most callers already rely on an
open `bless' to avoid the old {mime} arg.
|
|
We no longer load or use Email::MIME outside of comparison
tests.
|
|
PublicInbox::Eml has enough functionality to replace the
Email::MIME-based PublicInbox::MIME.
|
|
Instead, favor PublicInbox::MIME->new for non-attachment emails.
We may support alternatives to Email::MIME down the line.
We'll still keep Email::MIME->create to deal with attachments,
for now, but there's also a fair amount of test duplication
we should eliminate, later.
|
|
PublicInbox::MIME only supports ->new, and is only different
from Email::MIME for old versions of Email::MIME. In the
future, PublicInbox::MIME may not be a subclass of Email::MIME
at all.
|
|
Allowing ->init_bare to be used as a method saves some
keystrokes, and we can save a little bit of time on systems with
our vfork(2)-enabled spawn().
This also sets us up for future improvements where we can
avoid spawning a process at all.
|
|
We can finally get rid of the awkward, ad-hoc use of V2Writable,
SearchIdx, and OverIdx args for passing {cotime} and {autime}
between classes.
We'll still use those git time fields internally within
V2Writable and SearchIdx for (re)indexing, but that's not
worth avoiding as a fallback.
|
|
Some strange "From:" lines will cause Email::Address::XS to
leave '<' (and presumably '>') in the address which
git-fast-import won't accept even if quoted. Workaround this
problem by deleting '<' and '>' the same way we delete them for
the ident name.
Reported-by: Leah Neukirchen <leah@vuxu.org>
Link: https://public-inbox.org/meta/87h7zfemur.fsf@vuxu.org/
|
|
I didn't wait until September to do it, this year!
|
|
Perl 5.14+ gained the ability to autoload IO::File
(and IO::Handle) on missing methods, so relying on
this breaks under 5.10.1.
There's no reason to load IO::File or IO::Handle
when built-in perlops work fine and are even a hair
faster.
|
|
We can save callers the trouble of {-hold} and {-dev_null}
refs as well as the trouble of calling fileno().
|
|
We want to be able to use run_script with *.t files, so
t/common.perl putting subs into the top-level "main" namespace
won't work. Instead, make it a module which uses Exporter
like other libraries.
|
|
We'll also introduce a tmpdir() API to give tempdirs
consistent names.
|
|
And use it for mda, since "0" could be a usable directory
if somebody insists on using relative paths...
|
|
|
|
This allows v1 tests to continue working on git 1.8.0 for
now. This allows git 2.1.4 packaged with Debian 8 ("jessie")
to run old tests, at least.
I suppose it's safe to drop Debian 7 ("wheezy") due to our
dependency on git 1.8.0 for "merge-base --is-ancestor".
Writing V2 repositories requires git 2.6 for "get-mark"
support, so mask out tests for older gits.
|
|
It's easier to store everything in one array ref similar
to what our Git->check routine returns
|
|
Wrap "get-mark" and "checkpoint" commands for git-fast-import
while documenting/cementing parts of the API.
|
|
Check for this before doing the Xapian-based v2 importer.
|
|
Using update-copyrights from gnulib
While we're at it, use the SPDX identifier for AGPL-3.0+ to
ease mechanical processing.
|
|
This should fix problems with multipart messages where
text/plain parts lack a header.
cf. git clone --mirror https://github.com/rjbs/Email-MIME.git
refs/pull/28/head
In the future, we may still introduce as streaming
interface to reduce memory usage on large emails.
|
|
Mailing lists I watch and mirror may not have the best spam
filtering, and an extra layer should not hurt.
|
|
This is probably trivial enough to be final?
|
|
This will allow us to write fast importers for existing
archives as well as eventually removing the ssoma dependency
for performance and ease-of-installation.
|