Date | Commit message (Collapse) |
|
We already add the extra newline when we call print.
|
|
There was a spurious test failure in t/httpd-corner.t
which I have not been able to reproduce.
|
|
This fixes some layering violations and consolidates
the cleanup into the inbox object itself. Keeping in
mind weakening does not work at all without our PSGI
server.
|
|
This only affects Linux users with MSG_MORE support.
We can avoid extra TCP overhead for sub-optimal chunk sizes
by using MSG_MORE even with chunk trailers under Linux.
This breaks real-time apps which require <= 200ms latency for
streaming small packets (e.g. implementing "tail -F"), but the
public-inbox WWW code does not (and will never) do such things.
|
|
We want to maximize fairness for large responses which may
download the entire mbox.
|
|
This allows us to yield control to other clients gracefully if
getline takes too long to generate a chunk. This is more
expensive but should not cost a syscall on modern 64-bit systems.
|
|
Use the EvCleanup::asap handler to reschedule our writes
after yielding to other clients.
|
|
We cannot let a client monopolize the single-threaded server
even if it can drain the socket buffer faster than we can
emit data.
While we're at it, acknowledge the this behavior (which happens
naturally) in httpd/async.
The same idea is present in NNTP for the long_response code.
This is the HTTP followup to:
commit 0d0fde0bff97 ("nntp: introduce long response API for streaming")
commit 79d8bfedcdd2 ("nntp: avoid signals for long responses")
|
|
It's a low priority, but acknowledge it.
|
|
Lightly tested, this seems to work when mass-aborting
responses. Will still need to automate the testing...
|
|
We no longer override Danga::Socket::event_write and instead
re-enable reads by queuing up another callback in the $close
response callback. This is necessary because event_write may not be
completely done writing a response, only the existing buffered data.
Furthermore, the {closed} field can almost be set at any time when
writing, so we must check it before acting on pipelined requests as
well as during write callbacks in more().
|
|
Standardize the code we have in place to avoid creating too many
timer objects. We do not need exact timers for things that don't
need to be run ASAP, so we can play things fast and loose to avoid
wasting power with unnecessary wakeups.
We only need two classes of timers:
* asap - run this on the next loop tick, after operating on
@Danga::Socket::ToClose to close remaining sockets
* later - run at some point in the future. It could be as
soon as immediately (like "asap"), and as late as 60s into
the future.
In the future, we support an "emergency" switch to fire "later"
timers immediately.
|
|
Oops, really gotta start checking logs in tests :x
Fixes: bb38f0fcce739 ("http: chunk in the server, not middleware")
|
|
Since PSGI does not require Transfer-Encoding: chunked or
Content-Length, we cannot expect random apps we host to chunk
their responses.
Thus, to improve interoperability, chunk at the HTTP layer like
other PSGI servers do. I'm chosing a more syscall-intensive method
(via multiple send(...MSG_MORE) for now to reduce copy + packet
overhead.
|
|
Unfortunately, the original design did not work because
middleware can wrap the response body and make `async_pass'
invisible to HTTP.pm
|
|
This can avoid an expensive copy for big strings.
|
|
Otherwise, we get deep recursion as we keep calling
recursively on giant responses
|
|
This will allow us to minimize buffering after we wait
(possibly a long time) for readability. This also greatly
reduces the amount of Danga::Socket-specific knowledge we
have in our PSGI code, making it easier for others to
understand.
|
|
By switching to a "pull"-based I/O model for reading
application responses, we should be able to throttle
buffering to slow clients more effectively and avoid
wasting precious RAM.
This will also allow us to more Danga::Socket-specific
knowledge out of the PSGI application and keep it
confined to PublicInbox::HTTP.
|
|
This lets us release old git processes so unlinked packs
(leftover from repacking) can be released. This may also
be helpful for Xapian as indices get rebuilt for tuning.
For SQLite (msgmap), the there may be no benefit besides
reducing FD pressure.
Followup changes will unify the Inbox and NewsGroup
classes and allow better code-sharing between NNTP and
HTTP classes (as well as the planned POP3 class).
|
|
This empty string check is for middlewares such as Deflater
which may write empty strings, not for direct real callers of
Danga::Socket who (presumably) know what they're doing.
|
|
Unnecessary on *nix, and we won't support systems
which do insane things.
|
|
We can reduce the allocation and overhead needed for
Danga::Socket timers for immediately-executed responses by
combining identical timers and reducing anonymous sub creation.
|
|
This saves us a system call for common GET/HEAD requests
with no upload body.
|
|
We need to abort connections properly if a response is prematurely
truncated. This includes problems with serving static files, since
a clumsy admin or broken FS could return truncated responses and
inadvertently leave a client waiting (since the client saw
"Content-Length" in the header and expected a certain length).
|
|
We must use a normal write instead of send(.., MSG_MORE)
when writing responses of "Content-Length: 0" to avoid
the corking effect MSG_MORE provides. We only want to
cork headers if we will send a non-empty body.
Fixes: c3eeaf664cf0 ("http: clarify intent for persistence")
This needs a proper test.
|
|
We don't actually need to know if a response is chunked or
what the actual Content-Length is; we just need to know if
the PSGI app properly terminated the response so we can
handle persistent connections.
|
|
Oops, we need to watch out for how we handle operator
precedence and ensure responses without a Content-Length
or "Transfer-Encoding: chunked" header will always
disconnect after writing.
|
|
This allows us to reduce installation dependencies while
retaining performance as it favors HTTP::Parser::XS when
it is installed and available.
PLACK_HTTP_PARSER_PP may be set to 1 to force a pure Perl
parser for testing.
|
|
Plack::Middleware::Deflater (and perhaps other middleware)
triggers zero-byte writes which wastes syscalls when
they get passed to Danga::Socket. This may also trigger
problems when we introduce TLS support in the future.
|
|
We cannot risk using all of a users' disk space buffering
gigantic requests. Use the defaults git gives us since
we primarily host git repositories.
|
|
We cannot rely on a client socket having a PSGI env before headers
are fully-parsed as we seek to avoid storing hashes for idle
clients. Sso print errors to the psgi.errors value which belongs to
the httpd listener, instead.
|
|
HTTP::Parser::XS::PP does not reject excessively large
headers like the XS version. Ensure we reject headers
over 16K since public-inbox should never need such large
request headers.
|
|
This allows us to share more code between daemons and avoids
having to make additional syscalls for preparing REMOTE_HOST
and REMOTE_PORT in the PSGI env in -httpd.
This will also make supporting HTTP (and NNTP) over Unix sockets
easier in a future commit.
|
|
Better to throw the error back to the client ASAP if we're
out-of-descriptors. We will need to implement idle client
expiration for long-lived HTTP connections.
|
|
This makes for better compile-time checking and also helps
document which calls are private for HTTP and NNTP.
While we're at it, use IO::Handle::* functions procedurally,
too, since we know we're working with native glob handles.
|
|
Checking the time is nearly free on modern systems with
vDSO/vsyscall/similar while sprintf is always expensive.
|
|
It may not be obvious where we are when we enter the event_write
callback. Hopefully this clarifies things.
|
|
Just in case we screwed up somewhere, we need to match up
syswrite to sysseek and we also favor procedural calls for
native types.
|
|
HTTP responses may be long-running or requests may be slow or
pipelined. Ensure we don't kill them off prematurely.
|
|
No point in comparing an empty string; length() is only
potentially expensive on big strings.
|
|
Danga::Socket will die on us if we hit the base implementations.
|
|
This is meant to provide an easy starting point for server admins.
It provides a basic HTTP server for admins unfamiliar with
configuring PSGI applications as well as being an identical
interface for management as our nntpd implementation.
This HTTP server may also be a generic Plack/PSGI server for
existing Plack/PSGI applications.
|