* [PATCH 00/11] lei q --stdin, shortcut names, etc
@ 2021-02-03 8:11 7% Eric Wong
2021-02-03 8:11 7% ` [PATCH 01/11] lei: reduce FD pressure from lei2mail worker Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2021-02-03 8:11 UTC (permalink / raw)
To: meta
Since externals tend to have common URL or pathname prefixes,
it's now possible to use -I/--only/--exclude with just the
basename of a URL or directory if that's unambiguous.
Wildcard matches are also supported with -I/--only/--exclude.
forget-external still requires the full path, but that's
rarely-used.
add-external bash completion now supports URL hostnames
and common base names.
"lei q" also supports reading queries from stdin.
FD use is slightly reduced, but still far from ideal
(it's bad when I have to bump "ulimit -n" to reattach
screen(1) while I'm running stress tests).
Eric Wong (11):
lei: reduce FD pressure from lei2mail worker
lei: further reduce lei2mail FD pressure
pkt_op: rely on DS::in_loop global
lei: err: avoid uninitialized variable warnings
lei: propagate curl errors, improve internal consistency
lei q: -I/--exclude/--only support globs and basenames
lei: complete basenames for include|exclude|only
lei: help starts pager
lei add-external: completion for existing URL basenames
lei: use sleep(1) loop for infinite sleep
lei q: support reading queries from stdin
MANIFEST | 1 +
contrib/completion/lei-completion.bash | 6 ++
lib/PublicInbox/InputPipe.pm | 37 ++++++++++++
lib/PublicInbox/LEI.pm | 37 +++++++-----
lib/PublicInbox/LeiExternal.pm | 82 +++++++++++++++++++++-----
lib/PublicInbox/LeiOverview.pm | 9 ++-
lib/PublicInbox/LeiQuery.pm | 59 ++++++++++++++----
lib/PublicInbox/LeiToMail.pm | 2 +-
lib/PublicInbox/LeiXSearch.pm | 20 +++----
lib/PublicInbox/PktOp.pm | 25 +++++---
script/lei | 2 +-
t/lei.t | 51 ++++++++++++----
12 files changed, 248 insertions(+), 83 deletions(-)
create mode 100644 lib/PublicInbox/InputPipe.pm
^ permalink raw reply [relevance 7%]
* [PATCH 01/11] lei: reduce FD pressure from lei2mail worker
2021-02-03 8:11 7% [PATCH 00/11] lei q --stdin, shortcut names, etc Eric Wong
@ 2021-02-03 8:11 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2021-02-03 8:11 UTC (permalink / raw)
To: meta
lei2mail doesn't need stdin anymore, so we can use the [0] slot
for the $not_done keepalive purposes.
---
lib/PublicInbox/LeiOverview.pm | 8 ++++----
lib/PublicInbox/LeiToMail.pm | 2 +-
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/lib/PublicInbox/LeiOverview.pm b/lib/PublicInbox/LeiOverview.pm
index 52da225d..88034ada 100644
--- a/lib/PublicInbox/LeiOverview.pm
+++ b/lib/PublicInbox/LeiOverview.pm
@@ -217,13 +217,13 @@ sub ovv_each_smsg_cb { # runs in wq worker usually
};
} elsif ($l2m && $l2m->{-wq_s1}) {
my ($lei_ipc, @io) = $lei->atfork_parent_wq($l2m);
- # $io[-1] becomes a notification pipe that triggers EOF
+ # $io[0] becomes a notification pipe that triggers EOF
# in this wq worker when all outstanding ->write_mail
# calls are complete
- pipe($l2m->{each_smsg_done}, $io[$#io + 1]) or die "pipe: $!";
- fcntl($io[-1], 1031, 4096) if $^O eq 'linux'; # F_SETPIPE_SZ
+ $io[0] = undef;
+ pipe($l2m->{each_smsg_done}, $io[0]) or die "pipe: $!";
+ fcntl($io[0], 1031, 4096) if $^O eq 'linux'; # F_SETPIPE_SZ
delete @$lei_ipc{qw(l2m opt mset_opt cmd)};
- $lei_ipc->{each_smsg_not_done} = $#io;
my $git = $ibxish->git; # (LeiXSearch|Inbox|ExtSearch)->git
$self->{git} = $git;
my $git_dir = $git->{git_dir};
diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm
index c6c5f84b..c704dc2a 100644
--- a/lib/PublicInbox/LeiToMail.pm
+++ b/lib/PublicInbox/LeiToMail.pm
@@ -464,7 +464,7 @@ sub post_augment { # fast (spawn compressor or mkdir), runs in main daemon
sub write_mail { # via ->wq_do
my ($self, $git_dir, $smsg, $lei) = @_;
- my $not_done = delete $self->{$lei->{each_smsg_not_done}};
+ my $not_done = delete $self->{0} // die 'BUG: $not_done missing';
my $wcb = $self->{wcb} //= do { # first message
$lei->atfork_child_wq($self);
$self->write_cb($lei);
^ permalink raw reply related [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-02-03 8:11 7% [PATCH 00/11] lei q --stdin, shortcut names, etc Eric Wong
2021-02-03 8:11 7% ` [PATCH 01/11] lei: reduce FD pressure from lei2mail worker Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).