Date | Commit message (Collapse) |
|
This prepares us for future changes to improve scalability to
many inboxes.
|
|
While Perl implements tail recursion via `goto' which allows
avoiding warnings on deep recursion. It doesn't (as of 5.28)
optimize the speed of such dispatches, though it may reduce
ephemeral memory usage.
Make the code less alien to hackers coming from other languages
by using normal subroutine dispatch. It's actually slightly
faster in micro benchmarks due to the complexity of `goto &NAME'.
|
|
This should've always been there. I'm not sure how widely
spread 1.0 and earlier releases were, but we'll keep documenting
the version requirement.
|
|
Like the rest of the WWW code, public-inbox-httpd now uses
git_async_cat to retrieve blobs without blocking the event loop.
This improves fairness when git blobs are on slow storage and
allows us to take better advantage of SMP systems.
|
|
To avoid hogging the event loop in public-inbox-httpd when
many candidate messages match, we'll separate the steps to
ensure fairness on slow storage.
|
|
With public-inbox-httpd, this mitigates the effect of slow git
blob storage with multiple coderepos configured for an inbox.
It's still synchronous for now (and may need to remain that way
for ->last_check_err), but no longer monopolizes the event loop
when checking multiple coderepos.
We don't yet support multi-inbox scanning, yet; but this also
prepares us for a future where we do.
We'll also support >=40 char blob OIDs in preparation for future
git SHA-256 support, too.
|
|
By using the just-introduced ConfigIter class.
And make ManifestJsGz a subclass of it to reduce duplication.
|
|
With many inboxes, checking multiple SQLite repos will be slow
and time-consuming, so ensure we can schedule it fairly between
multiple inboxes.
|
|
We will need to allow simultaneous iterators on the same
config object, since we'll need this for ExtMsg, NNTPD,
WwwListing, NewsWWW, and other places.
|
|
In Perl, we can simplify callers by passing a single array
all the way down the stack instead of a single array ref which
needs to be expanded every call.
|
|
It's still as slow as before with hundreds/thousands of inboxes,
but at least it's fair. Future changes will allow it to be
cached and memoized with persistent HTTP servers.
|
|
"*foo" is ambiguous in that it may refer to a bareword file handle;
so we'll use it where we can without triggering warnings.
PublicInbox::TestCommon::run_script_exit required dropping the
prototype, however. We'll also future-proof by dropping "use
warnings" in Cgit.pm and use the less-ambiguous "//=" in Inbox.pm
while we're in the area.
|
|
With Perl upstream preparing to deprecate things, we'll move
towards only enabling warnings during development via shebang
and stop enabling them via "use".
We'll also favor "use v5.10.1" over the Perl 5.6-compatible "use
5.010_001", since our code base never worked on 5.6.
Finally, were also importing SEEK_SET without using it, just use it
for readability since we can't avoid loading Fcntl in other
places and it'll get constant-folded, anyways.
|
|
Use the full URL of the inbox being mirrored to reduce ambiguity
(instead of just the inbox name).
Using asymmetric quotes (e.g `foo') improves readability for me
in that it's more obvious when a quote begins and ends. It also
lights up fewer pixels and reduces visual noise compared to
double-quotes.
We'll also reflow the `mainrepo' vs `inboxdir' comment slightly
to emphasize the word `instead'.
|
|
Save screen space and light up fewer pixels to reduce visual noise.
|
|
The `solver' feature is not very obvious, give potential
users a hint about it.
|
|
We cannot blindly use the selected newsgroup for
HEAD/ARTICLE/BODY requests using Message-ID, since
those commands look across all newsgroups; not just
the selected one (if any).
So stuff a reference to the Inbox object into $smsg.
We can reduce args passed into set_nntp_headers() and
msg_hdr_write(), too.
Fixes: 0e6ceff37fc38f28 ("nntp: support slow blob retrievals")
|
|
Oops, I wanted to stop escaping double-quotes with `qq()' but
used `q()' instead :x
Fixes: 2f61828fcb727e51 ("www: make mirror instructions more prominent")
|
|
We don't want <a> tags without href= attributes to be colored,
since the `<a id=mirror>' tag in the HTML footer is intended
as an anchor destination for `<a href=#mirror>' link at the
top.
|
|
In order to fight the misconception that public-inboxes are
centralized, anchor "#mirror" to the clone instructions and
place an emphasis on "mirror", not just cloning.
While we're at it, better describe multi-epoch -V2 inboxes,
since some users do not seem to realize epochs consist of
different data.
|
|
We'll also fix the read-only code to ensure we notice missing
Xapian shards, since gaps would throw off our expectation that
Xapian document IDs and NNTP article numbers are interchangeable.
|
|
This may be useful for keeping our heads on straight dealing
with IMAP, NNTP, JMAP, etc.
|
|
It's inlined into the main function, which we'll shorten
slightly with the defined-or (`//') operator. Also noticed
and fixed a mismatched HTML tag.
|
|
We switched to Parse::RecDescent during development and left
some dead code behind.
|
|
Only inbox accesses the read-only {over}, now, instead of going
through ->search. This simplifies our object graph and avoids
potentially redundant FDs and DB handles pointing to the same
over.sqlite3 file.
|
|
Nearly all of the search uses in the production code rely on
a Xapian mset iterator being returned (instead of an array
of $smsg objects). So default to returning the mset and move
the burden of smsg array conversion into the test cases.
|
|
The special case (if any) belongs at a higher-level,
and this is another step towards removing {over_ro}-dependence
in our Search object.
|
|
We'll use {oidx} as the common field name for the read-write
OverIdx, here, to disambiguate it from the read-only {over}
field. This hopefully makes it clearer which code paths are
read-only and which are read-write.
|
|
It'll likely be used in the future for JMAP, detached indices,
and maybe other things.
|
|
Just some golfing to reduce scrolling and hopefully readability.
|
|
For consistency with other commands, though the
protocol-specific options should refer users to
the manpage.
|
|
And while we're at it, note edit is *destructive* to encourage
reading the fine manual.
|
|
"inboxes 1 inboxes not supported by ..." was non-sensical.
Now it'll show "-V1 inbox not supported by ...", instead.
|
|
ParentPipe was a subset of EOFpipe, except EOFpipe correctly
accounts for theoretical(*) spurious wakeups on the pipe.
(*) AFAIK, spurious wakeups are/were more likely on TCP sockets
due to checksum failures, something that's not a problem on
local pipes. We're also not sharing pipes like we do with
listen sockets on accept(2), so there's no chance of another
process grabbing bytes (unless we have bugs in our code).
|
|
It doesn't seem necessary, since we won't call dwaitpid()
until we see an EOF.
|
|
It's a bit inefficient to use a pipe, here. However, using
dwaitpid() on a process that's not expected to exit soon is
also inefficient as it causes excessive wakeups as most of
our inbox-writing code expects synchronous waitpid().
This only affects -watch instances configured for NNTP and IMAP
clients.
|
|
We should not enqueue reap_pids() to run more than once per
EventLoop iteration. We'll start reformatting reap_pids
to tabs, too, since we're no longer Danga::Socket.
We should also be able to remove timer usage for reaping
down-the-line once we stop abusing dwaitpid() in -watch.
|
|
Get rid of an unused variable, prefix a warning and try to
better document control flow around various callbacks.
|
|
In case there's non-Linux or BSD users w/o IO::KQueue, we
shouldn't let signal handlers fire in the child processes.
The child processes always assumed signals were blocked by
the parent, so no changes were necessary, there.
|
|
This should further mitigate lock contention problems
when -watch is configured to watch on a Maildir for spam
while performing a large NNTP import.
There is now a small risk a message won't get removed because if
it's in the current (uncommitted) fast-import batch, but
unlikely given the batch size is now only 10 messages.
If a that small window is hit, flipping the \Seen flag
(e.g. marking it unread, and then read again) will trigger
another removal attempt via IMAP or Maildir.
|
|
This is no longer limited to Maildirs now that IMAP and NNTP
support exist; so give it a shorter name.
|
|
Declare 5.10.1 to avoid potential compatibility problems with
Perl 7/8 down the line. We'll rely on the command-line to set
or drop warnings during development, at least.
|
|
We don't want to monopolize locks because processes can easily
block each other if using `watchspam' on a Maildir while a big
NNTP or IMAP import is happening.
This can also happen if somebody configured a single inbox to
watch from several sources to merge several mailboxes into one
(e.g. both an IMAP and Maildir are watched).
|
|
Quiet down logs from -imapd when clients are blindly
sending some unsupported flag conditions (e.g. "DRAFT",
"DELETED") specified in RFC 3501.
|
|
By making it a no-op if last_uid is not defined. This isn't a
hot code path, so the extra method dispatch isn't an issue.
It'll save some indentation/wrapping in future commits.
|
|
Data needs to hit inboxes, first. Otherwise it's possible to
skip messages in case git-fast-import is killed before it sees
"done\n". Now, -watch will just waste a little bandwidth in
re-downloading a seen message if it's interrupted immediately
before updating IMAPTracker.
|
|
Being an easily confused person, I find "next" and "prev"
ambiguous as to whether messages on the next or previous page
will be newer or older than the current page. Clarify that for
the threaded /$INBOX/ view and search results.
For search results sorted by relevance, we'll use "[>= $SCORE]"
or "[<= $SCORE]" to indicate to indicate directionality.
This also fixes $INBOX/new.html for unindexed v1 inboxes.
|
|
Sometimes it's useful to quickly get to threads and messages
which are contemporaries of the current thread/message being
focused on. This hopefully improves navigation by making:
a) the top line (where $INBOX_DIR/description) is shown
a link to the latest topics in search results and
per-thread/per-message views.
b) providing a link to contemporaries ("~YYYY-MM-DD") at
around the thread overview skeleton area for per-thread
and per-message views
|
|
This matches the behavior of Maildir `watchspam' handling in not
removing unseen messages. NNTP can't match this behavior, since
NNTP servers don't store flags, clients do.
|
|
There's no need for this to be a separate sub since there's
only a single caller. This saves a few kilobytes at least
in short-lived processes.
|