| Date | Commit message (Collapse) |
|---|
|
Let postfix (or sendmail :P) control the concurrency limit
instead of doing it ourselves. This is necessary because SMTP
connections are completely synchronous at this point and a
slow/idle SMTP connection will monopolize the worker process.
|
|
The restart_read callback has no chance of circular reference,
and weakening $self before we create it can cause $self to
be undefined inside the callback (seen during stress testing).
Fixes: 395406118cb2 ("httpd/async: prevent circular reference")
|
|
We need to avoid circular references in the generic PSGI layer,
do it by abusing DESTROY.
|
|
Lightly tested, this seems to work when mass-aborting
responses. Will still need to automate the testing...
|
|
We must avoid circular references which can cause leaks in
long-running processes. This callback is dangerous since
it may never be called to properly terminate everything.
|
|
git has stricter requirements for ident names (no '<>')
which Email::Address allows.
Even in 1.908, Email::Address also has an incomplete fix for
CVE-2015-7686 with a DoS-able regexp for comments. Since we
don't care for or need all the RFC compliance of Email::Address,
avoiding it entirely may be preferable.
Email::Address will still be installed as a requirement for
Email::MIME, but it is only used by the
Email::MIME::header_str_set which we do not use
|
|
Having an excessive amount of git-pack-objects processes is
dangerous to the health of the server. Queue up process spawning
for long-running responses and serve them sequentially, instead.
|
|
We no longer override Danga::Socket::event_write and instead
re-enable reads by queuing up another callback in the $close
response callback. This is necessary because event_write may not be
completely done writing a response, only the existing buffered data.
Furthermore, the {closed} field can almost be set at any time when
writing, so we must check it before acting on pipelined requests as
well as during write callbacks in more().
|
|
Standardize the code we have in place to avoid creating too many
timer objects. We do not need exact timers for things that don't
need to be run ASAP, so we can play things fast and loose to avoid
wasting power with unnecessary wakeups.
We only need two classes of timers:
* asap - run this on the next loop tick, after operating on
@Danga::Socket::ToClose to close remaining sockets
* later - run at some point in the future. It could be as
soon as immediately (like "asap"), and as late as 60s into
the future.
In the future, we support an "emergency" switch to fire "later"
timers immediately.
|
|
Oops, really gotta start checking logs in tests :x
Fixes: bb38f0fcce739 ("http: chunk in the server, not middleware")
|
|
Since PSGI does not require Transfer-Encoding: chunked or
Content-Length, we cannot expect random apps we host to chunk
their responses.
Thus, to improve interoperability, chunk at the HTTP layer like
other PSGI servers do. I'm chosing a more syscall-intensive method
(via multiple send(...MSG_MORE) for now to reduce copy + packet
overhead.
|
|
We will have clients dropping connections during long clone
and fetch operations; so do not retain references holding
backend processes once we detect a client has dropped.
|
|
Only check query parameters since there's no useful body
in there.
|
|
Some readers will want to use "HTTPS Everywhere" conveniently;
and I will support it.
|
|
This bit is still being redone to support gigantic repos.
|
|
We may spawn this in a large server process, so be sure
to take advantage of the optional vfork() support when
for folks who set PERL_INLINE_DIRECTORY.
|
|
Followup-to: commit 24e0219f364ed402f9136227756e0f196dc651aa
("remove GIT_DIR env usage in favor of --git-dir")
|
|
Users may change terminal sizes if the process is connected to a
terminal, so we can't reasonably expect SIGWINCH to work as
intended.
|
|
We can't rely on absolute paths when installed on other
systems.
Unfortunately, mlmmj-* requires them, but none of the core
code will use it.
|
|
The offset argument must be an integer for Xapian,
however users (or bots) type the darndest things.
AFAIK this has no security implications besides triggering
a warning (which could lead to out-of-space-errors)
|
|
This simplifies the code somewhat; but it could probably
still be made simpler. It will need to support command
queueing for expensive commands so expensive processes
can be queued up.
|
|
Unfortunately, the original design did not work because
middleware can wrap the response body and make `async_pass'
invisible to HTTP.pm
|
|
We can rely entirely on getline + close callbacks
and be compatible with 100% of PSGI servers.
|
|
We will figure out a different way to avoid overloading...
|
|
We need to ensure $? is set properly for users.
|
|
This can avoid an expensive copy for big strings.
|
|
Otherwise, we get deep recursion as we keep calling
recursively on giant responses
|
|
Sometimes we need to read something to ensure it's a successful
response.
|
|
This will allow us to minimize buffering after we wait
(possibly a long time) for readability. This also greatly
reduces the amount of Danga::Socket-specific knowledge we
have in our PSGI code, making it easier for others to
understand.
|
|
We don't need to update-server-info (or read-tree) if fast
import was spawned for removals and no changes were made.
|
|
We shouldn't need sigprocmask unless we're running multiple
native threads or using vfork, neither of which is the case,
here.
|
|
This hopefully makes the intent of the code clearer, too.
The the HTTP use of the numeric reference for getline
caused problems in Git.pm, already.
|
|
Wow, I don't know crypto at all.
|
|
This unsubscribe PSGI endpoint should never incur enough load to
justify using multiple worker processes. If it's unstable and
crashes, systemd can automatically restart it.
|
|
Fixup a comment about s/query string/PATH_INFO/ while
we're at it, as pre-published versions of this used
query strings before I determined it could be harder
to copy+paste URLs with query parameters in them.
|
|
This allows us to easily provide gigantic inboxes
with proper backpressure handling for slow clients.
It also eliminates public-inbox-httpd and Danga::Socket-specific
knowledge from this class, making it easier to follow for
those used to generic PSGI applications.
|
|
By switching to a "pull"-based I/O model for reading
application responses, we should be able to throttle
buffering to slow clients more effectively and avoid
wasting precious RAM.
This will also allow us to more Danga::Socket-specific
knowledge out of the PSGI application and keep it
confined to PublicInbox::HTTP.
|
|
Otherwise, tempfile() will use the current working directory,
which may not be writable.
|
|
While public-inbox is intended primarily for archival,
SMTP list subscriptions are still in use in most places
and users are likely to want a good unsubscribe mechanism.
HTTP (or HTTPS) links in the List-Unsubscribe header are
often preferable since some users may use an incorrect
email address for mailto: links.
Thus, it is useful to provide an example which generates an
HTTPS link for users to click on. The default .psgi requires
a POST confirmation (as destructive actions with GET are
considered bad practice). However, the "confirm" parameter
may be disabled for a true "one-click" unsubscribe.
The generated URLs are hopefully short enough and both shell
and highlighting-friendly to reduce copy+paste errors.
|
|
Having a file start with '.' or '-' can be confusing
and for users, so do not allow it.
|
|
For attachments without a filename or description, reduce
the amount of precious screen space required to display
a link to it.
|
|
We shall ensure links continue working for this.
|
|
Email::MIME >= 1.923 and < 1.935 would drop too many newlines
in attachments. This would lead to ugly text files without
a proper trailing newline if using quoted-printable, 7bit, or
8bit. Attachments encoded with base64 were not affected.
These versions of Email::MIME are widely available in Debian 8
(Jessie) and even Ubuntu LTS distros so we will need to support
this workaround for a while.
|
|
This can be useful for lists where the convention is to
attach (rather than inline) patches into the message body.
|
|
msg_iter lets us know the index of the attachment,
allow us to make more sensible labels and in a future
commit, hyperlinks to download attachments.
|
|
Unlike Email::MIME::walk_parts, this is non-recursive and gives
depth + index offset information about the part for creating
links for later retrieval
It is intended for read-only access and changes are not
propagated to the parent; however future versions of it
may clobber bodies or the original version as it iterates
to reduce memory overhead.
It is intended for making it easy to locate attachments within a
message in the WWW view.
|
|
Or is it "encoding"? Gah, Perl character set handling
confuses me no matter how many times I RTFM :<
This contains placeholders for attachment downloading
which will be in a future commit.
|
|
Oops, but at least it was mostly harmless, just ugly.
Followup-to: 9bfe40e7a4ac 'nntp: use "newsgroup" instead of "name"''
|
|
There's no place for them in the commands and we don't take
messages; potentially printing them into a log opened in a
terminal is too dangerous.
Hoist out read_til_dot in the test while we're at it.
|
|
This can be useful for hammering a live HTTP server
with requests to ensure it does not fall over under
load.
|