Date | Commit message (Collapse) |
|
More tests work without Search::Xapian, now.
Usability issues still need to be fixed
|
|
We don't need to rely on Xapian search functionality for the
majority of the WWW code, even. subject_normalized is moved to
SearchMsg, where it (probably) makes more sense, anyways.
|
|
None of the NNTP code actually relies on Xapian, anymore.
|
|
One small step towards making installing Xapian optional for v2
and providing more WWW and NNTP functionality without it.
|
|
We were reindexing the full history every invocation of -index
when Xapian was not used because we were incorrectly relying on
'last_commit' metadata stored in Xapian.
Rewrite the indexing logic to be less confusing while we're
at it, since we rely on `git merge-base --is-ancestor' nowadays.
Furthermore, we need to handle message removals from the
overview index correctly when Xapian is not in use.
Co-authored-by: Eric W. Biederman <ebiederm@xmission.com>
|
|
Avoiding reliance on environment variables is a bit cleaner
for writing tests
|
|
Import initialization is a little strange from history, but we
also can't change it too much because it's technically a public
API which external code may rely on...
And we may need to support v1 repos indefinitely. This should
make it easier to write tests for both formats.
|
|
* origin/danga-bundle:
DS: epoll: fix misordered EPOLL_CTL_DEL call
DS: drop unused "_undef" sub
syscall: drop readahead wrapper
build: do not manify DS and Syscall pods
DS: handle EINTR in IO::Poll path, too
DS: workaround IO::Kqueue EINTR (mis-)handling
DS: drop profiling support
DS: remove unused fields and functions
listener: use EPOLLEXCLUSIVE for listen sockets
bundle Danga::Socket and Sys::Syscall
|
|
Any operations on an fd after POSIX::close() are invalid, so
epoll_ctl will fail. Worse off, in a multi-threaded Perl, the
fd may be reused by another thread and EPOLL_CTL_DEL can hit the
wrong file description as a result.
cf. https://rt.cpan.org/Ticket/Display.html?id=129487
|
|
No longer used since we removed the *_ip_string fields
|
|
No backwards compatibility to worry about for us; and fadvise
is superior anyways.
|
|
IO::Poll::_poll returns -1, which is "true" to Perl.
cf. https://rt.cpan.org/Ticket/Display.html?id=129484
|
|
We'll ignore blank lines from clients, since that's what innd
seems to do.
|
|
It's unneeded since commit e358bd7a3833f8c5 (2016-07-02)
("inbox: base_url method takes PSGI env hashref instead")
So we only depend on URI::Escape from the "URI" CPAN distribution,
at the moment.
|
|
Noticed while testing on FreeBSD 11.2 amd64 with the optional
Inline::C extension using clang 6.0.0. The end result on
FreeBSD was spawning processes failed badly and things were
immediately unusable with this enabled.
av_len is a misleading API, and I failed to read the API
comments in perl:/av.c which state:
> Note that, unlike what the name implies, it returns
> the highest index in the array, so to get the size of
> the array you need to use "av_len(av) + 1".
> This is unlike "sv_len", which returns what you would expect.
If this bug affected anybody, it would've only affected users
using both the optional Inline::C module AND set the
PERL_INLINE_DIRECTORY environment variable.
That said, I've never seen any evidence of it on Debian
GNU/Linux + gcc on any x86 variant. That includes full 64-bit
systems, a full 32-bit system, a 64-bit system with 32-bit
userspace, across multiple gcc versions since 2016.
|
|
This can help users track down the source of warnings
when presented with imperfect emails.
While we're at it, make the __WARN__ callback in t/v2writable.t
a no-op since we don't check for warnings, there.
|
|
* origin/wwwlisting:
www: support listing of inboxes
start depending on Perl 5.10.1+
|
|
IO::Kqueue seems unmaintained, so workaround a long-standing
bug where it falls over on signals:
https://rt.cpan.org/Ticket/Display.html?id=116615
|
|
There's other ways to profile and we don't need to add runtime
branches to do this.
|
|
More will likely be dropped in the future, but drop the obvious
ones we aren't using, for now; especially since some of them are
set at ->new time and unavoidable.
This saves 579 bytes per-client on my 64-bit Debian stable
system as measured by Devel::Size::total_size from
PublicInbox::HTTP::event_read. This adds up in C10K or C100K
situations.
Things we drop are:
* corked - MSG_MORE requires fewer syscalls
* read_push_back - tried to use it, ate CPU with slow clients
* IP/port fields - accept() already returns what we care about
|
|
Since our listen sockets are non-blocking and we may run
multiple httpd|nntpd processes; we need a way to avoid
thundering herds when there are multiple httpd|nntpd worker
processes.
EPOLLEXCLUSIVE was added just for that in Linux 4.5
|
|
These modules are unmaintained upstream at the moment, but I'll
be able to help with the intended maintainer once/if CPAN
ownership is transferred. OTOH, we've been waiting for that
transfer for several years, now...
Changes I intend to make:
* EPOLLEXCLUSIVE for Linux
* remove unused fields wasting memory
* kqueue bugfixes e.g. https://rt.cpan.org/Ticket/Display.html?id=116615
* accept4 support
And some lower priority experiments:
* switch to EV_ONESHOT / EPOLLONESHOT (incompatible changes)
* nginx-style buffering to tmpfile instead of string array
* sendfile off tmpfile buffers
* io_uring maybe?
|
|
It was a MITM hazard and been killed off by the DoS subsystem
last July: https://marc.info/?i=87d0vwwkbs.fsf@riseup.net
|
|
We already escape the user-provided Message-IDs (so there's no
security problem AFAIK), but the URL templates which exist in
our source code were not escaped properly.
This quiets down tidy(1).
|
|
It seems a common case for mangled patches is editors or MUAs
dropping trailing whitespace, and lines matching /^ $/ gets
the space dropped to only match /^$/.
|
|
Document `publicinbox.cgitdata' config directive, but allow it
to be unspecified and/or missing for installations which do not
wish to serve static data at all.
For users installing cgit from source to their home directory,
we can usually infer the cgit data path based on the cgit.cgi
binary path, even.
|
|
Not all inputs are highlight-able, so reuse the original
input and just linkify it if it can't be highlighted.
|
|
Otherwise, there's no reason to use this API over highlight(1).
Maybe this can be an option in the future; but I'm struggling to
find a reason to not do it by default.
|
|
Try to appear gramatically correct and state:
"only message in thread" when there's only one known (to us)
message in the thread.
|
|
We will still return a 404 by default to '/' for compatibility
with users of Plack::App::Cascade or similar. Inboxes are
sorted by modification times to help users detect activity
(similar to the /$INBOX/ topic view).
New configuration options:
* publicinbox.wwwlisting - configure the listing type
* publicinbox.<name>.hide - hide a particular inbox from the listing
See changes to public-inbox-config.pod for full descriptions
of the new options.
Requested-by: Leah Neukirchen <leah@vuxu.org>
https://public-inbox.org/meta/871sdfzy80.fsf@gmail.com/
|
|
I mainly want to start using the '//' (defined-or) operator to
simplify code, and Perl 5.10.1 is roughly a decade old at this
point.
"given/when" would've be nice, but it's future is in doubt AFAIK.
I also started using the 'parent' module in WwwHighlight, and
'autodie' in UserContent.pm, both of which were only distributed
with Perl since 5.10.1; and testing with ancient
versions/distros is time-consuming.
Anyways, I think this a small-enough jump to not break any
existing installations, given we already depend on fairly
recent versions of git and Xapian.
Maybe we can use more newish Perl features in the future...
|
|
Dangling parentheses with trailing punctuation usually means the
parentheses is not intended as part of the URL.
|
|
Empty subjects ("") and undefined Subjects: are now both
displayed as "(no subject)" for now.
|
|
The URLs at the top of WwwStream.pm weren't getting linkified
correctly.
|
|
For inboxes with SQLite enabled (all v2, and probably most v1);
we can use the overview DB to get the timestamp of the latest
message. It's faster than scanning git branches for commit times,
but not always the same.
|
|
This will be used for generating an HTML listing for v1 inboxes,
at least. The logic for this follows that of grokmirror,
and we may dynamically generate manifest.js.gz natively...
|
|
Followup-to: 6e6f7999361925e4
("cleanup: use '$ibx' consistently when referring to Inbox refs")
|
|
This is for consistency with other fields which follow
this pattern w.r.t. field-naming when referring to internal
fields.
|
|
'$inbox' is more human-readable, so that is for the more
human-readable name in most cases. Making our variable naming
more consistent should make the code easier-to-review and
harder to screw up.
|
|
We access the Git object via the Inbox object nowadays, so
there's no point in having a shortcut to it, anymore.
|
|
We parse cgitrc for "repo.path", while we use "coderepo.dir" to
mean the same thing for non-cgit users. So I ended up confusing
myself, here.
But then again, git uses "--git-dir" and "GIT_DIR", so I suspect
"dir" is the better choice than "path", here
|
|
We'll be building off of this for showing diffs in
the coderepo views.
|
|
Hopefully this gets us closer to matching cgit upstream behavior
(which also lacks tests). We'll still need to support macro
expansion at some point for compatibility...
|
|
We can reduce the configuration needed to run cgit by reusing
the static file handling logic of the dumb git HTTP protocol.
I hate logos and icons, so don't expect public-inbox.org or
80x24.org to ever have those to waste users' bandwidth with :P
But I expect other users to find this useful.
|
|
project_list support still needs to be done
And tests need to be written... :<
|
|
We need to instate our cgit handler everywhere we use NewsWWW
to catch wildcard requests which our normal endpoints do not
handle.
|
|
Our high-level config already treats single limits as a
soft==hard limit for limiters; so stop handling that redundant
in the low-level spawn() sub.
|
|
I mainly need this to enforce RLIMIT_CPU (and RLIMIT_CORE)
when requests come which generate giant, unrealistic diffs.
Per-coderepo limiters may be added in the future. But for now,
I need to prevent cgit from monopolizing resources on my dinky
server.
|
|
This allows users to configure RLIMIT_{CORE,CPU,DATA} using
our "limiter" config directive when spawning external processes.
|
|
Requests intended for cgit are unlikely to conflict with
requests to inboxes. So we can safely hand those requests
off to cgit.cgi.
|