| Date | Commit message (Collapse) |
|---|
|
It's already fixed memory usage problems not only in our codebase,
but also the standard `Encode' XS module and `git pack-objects'.
|
|
Basically, public-inbox-clone has become grok-pull without
config files nor absolute paths.
|
|
I'll have to downsize the server due to increased hosting costs,
so stop advertising these mirrors.
The inboxes still exist, for now; but will probably be proxied
behind an ssh tunnel via slow DSL connection, but it's not worth
increasing traffic to.
|
|
Because old versions will exist for a long time and our latest
documentation is visible on the web, we must document when a
switch appears to avoid confusing users of old versions.
|
|
I'm still trying to decide how to handle HTTP(S) remotes
properly...
Link: https://public-inbox.org/meta/20230226170931.M947721@dcvr/
|
|
When AllowEncodedSlashes is Off (the default setting), URLs containing
%2f are replied with a 404 error without calling the CGI. To (maybe)
prevent others debugging this issue add a hint with the solution.
|
|
The coderepo indexer will use similar ideas, I think...
|
|
public-inbox-{clone,fetch,netd} are all relatively new
developments which we can document, here.
We'll also update the generator Makefile snippet since there may
be more Graph::Easy-based docs coming.
|
|
-netd is strictly more powerful and a gateway drug for
imapd/nntpd/pop3d instances :>
|
|
With no remaining users, we can drop dwaitpid and switch
awaitpid to rely on waitpid(-1) to save syscalls.
|
|
Since public-inbox-clone is now useful for incremental updates
with manifest, --exit-code belongs here, too.
|
|
I forgot to document these when I implemented them :x
|
|
I'm thinking the -nntpd regression fix will push this release
out sooner rather than later...
|
|
This probably makes the most sense, if a user wants to
use an alternate path to read from, it's likely they
want to write it there, too.
|
|
Users may wish to keep objstore and manifest files at
a higher level to prevent direct access via HTTP(S),
so those relative paths probably make sense.
|
|
This can be useful for intermittent network errors,
and the required code changes makes it less dependent
on global state.
|
|
We need to be able to get rid of removed branches and tags on
the remote. --prune-tags is implied for non-objstore repos,
and incompatible with objstore repos.
|
|
This will allow us to quickly check fingerprints against
remotes with a single HTTP(S) request, saving us numerous
`git show-refs' invocations.
|
|
Allowing just `--objstore' without `=' was confusing,
since it could eat one of the required parameters (URL or
DESTINATION).
|
|
We can support absolute paths to avoid surprising behaviors,
but relative paths are preferred since the goal is to be
accessible over the "dumb" HTTP git transport (the dumb
transport is uses less memory and CPU on the server).
|
|
This is part of `lei add-external --mirror', and it makes
sense to have for development and testing. We'll also add
a fallback in case somebody tries --inbox-version and fails
due to a newer remote instances of public-inbox.
|
|
This allows avoiding 404s when trying _/text/config/raw on code
repositories.
|
|
It still makes HTTP(S) requests to retrieve the manifest or
scrape HTML, but doesn't make permanent changes to the FS
(aside from modifying {acm}time of ${TMPDIR-/tmp}).
|
|
These will be handy when someone is interested in a subset of
inboxes on a large hosting site.
|
|
Eric Wong <e@80x24.org> wrote:
> Thanks for confirming things work as intended. I think the
> default should be clamped, though... 15 seems a bit high for
> smaller IMAP servers *shrug*
--------8<-------
Subject: [PATCH] lei q|up: limit default write --jobs for IMAP(S)
IMAP(S) servers often limit per-user connections, so avoid
bumping into limits to improve the out-of-the-box experience.
4 seems like a conservative default, since we already chose
that number for remote HTTP(S) endpoints.
Link: https://public-inbox.org/meta/20220910201958.GA12212@dcvr/
|
|
Linking to the manpage probably helps clarify what `lei/store'
refers to without too much clutter in the raw POD source.
|
|
Users should know where `lei import' writes to.
|
|
Another teeny step towards v5.12.
|
|
While manpages are named `L<lei-COMMAND(1)>', `lei COMMAND'
can be worth linkifying for ease-of-navigation, too.
|
|
The `OUTPUT' use may not be immediately apparent, clarify
that it's from `lei q'.
|
|
lei-index, public-inbox-netd, and public-inbox-pop3d
were not properly linkified in our HTML documentation.
|
|
This also ensures we won't waste CPU cycles on snapshots
which aren't configured if somebody attempts them by
guessing URLs.
|
|
For backwards-compatibility, this defaults to `first'. When set
to `fallback', PublicInbox::WwwCoderepo is favored and cgit is
only used as a fallback. Eventually, `rewrite' will also be
supported to rewrite cgit URLs to WwwCoderepo ones.
Of course, WwwCoderepo is still missing search and other key
features, but that's being worked on...
|
|
Calling Compress::Raw::Zlib::deflate is fairly expensive.
Relying on the `.=' (concat) operator inside ->zadd operator is
faster, but the method dispatch overhead is noticeable compared
to the original code where we had bare `.=' littered throughout.
Fortunately, `print' and `say' with the PerlIO::scalar IO layer
appears to offer better performance without high method dispatch
overhead. This doesn't allow us to save as much memory as I
originally hoped, but does allow us to rely less on concat
operators in other places and just pass a list of args to
`print' and `say' as a appropriate.
This does reduce scratchpad use, however, allowing for large
memory savings, and we still ->deflate every single $eml.
|
|
These may be helpful for users on slow disks or have
IMAP server limitations.
|
|
|
|
Getting close to another release, I think...
|
|
|
|
"PROTO" is probably ambiguous, and we'll use the same
terminology as used in curl(1) documentation.
|
|
I think "lei reindex" and some doc cleanups will be the last
things before 1.9...
|
|
This allows memory savings by allowing multiple, completely
unrelated-PSGI apps to run within the same process as IMAP,
NNTP, and POP3.
|
|
Hopefully it makes sense to new users deploying or using POP3...
|
|
We've always shared this environment with git-http-backend(1)
(but don't (yet) support http.maxRequestBuffer anywhere)
|
|
This is documented in RFC 2595, and POP3 clients may rely on
seeing "STLS" in CAPA output to initiate TLS negotiation.
|
|
Old account expiry has not been implemented, but it seems to
work well with both mpop(1) and getmail(1). The strictness of
mpop was particularly helpful in ironing out bugs in our
implementation of (dreaded) message sequence numbers.
"EXPIRE 0" (RFC 2449) can theoretically save numerous "DELE"
commands, but that's untested by real-world clients. mpop
supports PIPELINING which is effective in hiding latency,
and the core networking functionality is already well-tested
from our NNTP and IMAP implementations.
Configuration requires "publicinbox.pop3state" to point to
a directory writable by the otherwise read-only daemon.
See public-inbox-pop3d(1) manpage for more usage details.
|
|
This was pointed out by the Debian package linter "lintian".
|
|
This was pointed out by the Debian package linter "lintian".
|
|
|
|
Link: https://public-inbox.org/meta/87levfv7hs.fsf@kyleam.com/
Noticed-by: Kyle Meyer <kyle@kyleam.com>
|
|
Since we'll be adding POP3 support as our 4th network protocol;
asking admins to run yet another daemon on top of existing
-httpd, -nntpd, -imapd is a maintenance burden and a waste of
memory.
The goal of public-inbox-netd is to be able to replace all
existing read-only daemons with a single process to save memory
and reduce administrative overhead; hopefully encouraging more
users to self-host their own mirrors.
It's barely-tested at the moment. Eventually, multiple
PI_CONFIG and HOME directories will be supported, as are
per-listener .psgi config files.
|