* Re: [PATCH 3/3] search: rectify comment (rt: documented since 738c4a65719e)
2023-10-16 11:33 7% ` [PATCH 3/3] search: rectify comment (rt: documented since 738c4a65719e) Štěpán Němec
@ 2023-10-16 21:39 0% ` Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2023-10-16 21:39 UTC (permalink / raw)
To: Štěpán Němec; +Cc: meta
Štěpán Němec <stepnem@smrk.net> wrote:
> Fixes: 738c4a65719e ("www: various help text updates")
> ---
> On a related note, there's this comment in LeiXSearch.pm:312:
>
> # this should really be rt: (received-time), but no stable
> # public-inbox releases support it, yet.
> my $dt = 'dt:'.strftime('%Y%m%d%H%M%S', gmtime($lr)).'..';
>
> So I went all "hey, it's been 3 releases with rt:, let's do this",
> and suddenly, `lei up` stopped getting any updates :-]
> The problem is that the date syntax handling for d:, dt: and rt: is not
> the same (date_parse_prepare in Search.pm), which doesn't seem ideal to
> me, either, but maybe that's intentional/necessary?
Yeah, probably: "rt:$lr.."; is correct.
It's historical baggage since rt: wasn't really used until we
got IMAP (2020); and we didn't have user-friendly approxidate
parsing until 2021.
d: and dt: had to be at least usable before 2021, so it's more
usable in "raw" form.
It's expensive to reindex so it's not something I'm inclined to
force users into doing, either...
d: was a short-sighted mistake and redundant. I suppose we
could stop indexing d: at some point (like we do with --skip-docdata)
and translate all (not just most) d: into dt: in the parser.
> On a totally unrelated note, this patch is a nice example of how
> --diff-algorithm=histogram (which I use by default) sometimes produces
> diffs that are both longer and more confusing than with any other
> (git-provided) algorithm:
Heh, I've never touched --diff-algorithm. But I sometimes pipe
messages to `lei rediff -W' or `lei rediff -U100' to get more
context.
I hope to add support for rediff to the WWW UI once coderepo
integration is improved.
^ permalink raw reply [relevance 0%]
* [PATCH 3/3] search: rectify comment (rt: documented since 738c4a65719e)
@ 2023-10-16 11:33 7% ` Štěpán Němec
2023-10-16 21:39 0% ` Eric Wong
0 siblings, 1 reply; 4+ results
From: Štěpán Němec @ 2023-10-16 11:33 UTC (permalink / raw)
To: meta
Fixes: 738c4a65719e ("www: various help text updates")
---
On a related note, there's this comment in LeiXSearch.pm:312:
# this should really be rt: (received-time), but no stable
# public-inbox releases support it, yet.
my $dt = 'dt:'.strftime('%Y%m%d%H%M%S', gmtime($lr)).'..';
So I went all "hey, it's been 3 releases with rt:, let's do this",
and suddenly, `lei up` stopped getting any updates :-]
The problem is that the date syntax handling for d:, dt: and rt: is not
the same (date_parse_prepare in Search.pm), which doesn't seem ideal to
me, either, but maybe that's intentional/necessary?
On a totally unrelated note, this patch is a nice example of how
--diff-algorithm=histogram (which I use by default) sometimes produces
diffs that are both longer and more confusing than with any other
(git-provided) algorithm:
our $ENQ_ASCENDING = 1;
our @MAIL_VMAP = (
[ YYYYMMDD, 'd:'],
- [ DT, 'dt:' ],
- # these are undocumented for WWW, but lei and IMAP use them
- [ BYTES, 'z:' ],
[ TS, 'rt:' ],
+ # these are undocumented for WWW, but lei and IMAP use them
+ [ DT, 'dt:' ],
+ [ BYTES, 'z:' ],
[ UID, 'uid:' ]
);
our @MAIL_NRP;
lib/PublicInbox/Search.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index d7a6f9d7913c..477f77dcffeb 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -68,10 +68,10 @@ our $ENQ_DESCENDING = 0;
our $ENQ_ASCENDING = 1;
our @MAIL_VMAP = (
[ YYYYMMDD, 'd:'],
- [ DT, 'dt:' ],
+ [ TS, 'rt:' ],
# these are undocumented for WWW, but lei and IMAP use them
+ [ DT, 'dt:' ],
[ BYTES, 'z:' ],
- [ TS, 'rt:' ],
[ UID, 'uid:' ]
);
our @MAIL_NRP;
--
2.42.0
^ permalink raw reply related [relevance 7%]
* [PATCH] www: mirror: fix rendering of NNTP URLs
@ 2021-10-26 0:48 7% Kyle Meyer
0 siblings, 0 replies; 4+ results
From: Kyle Meyer @ 2021-10-26 0:48 UTC (permalink / raw)
To: meta
As of commit 738c4a65, the code for reporting NNTP information in
_/text/mirror/ incorrectly uses ->imap_url rather than ->nntp_url.
Fixes: 738c4a65719e6278 ("www: various help text updates")
---
lib/PublicInbox/WwwText.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/PublicInbox/WwwText.pm b/lib/PublicInbox/WwwText.pm
index 211ee8c9..2b4e69fe 100644
--- a/lib/PublicInbox/WwwText.pm
+++ b/lib/PublicInbox/WwwText.pm
@@ -284,7 +284,7 @@ sub _add_imap_nntp_urls ($$) {
# each subfolder (starting with `0') holds 50K messages at most
EOM
}
- $urls = $ctx->{ibx}->imap_url($ctx);
+ $urls = $ctx->{ibx}->nntp_url($ctx);
if (@$urls) {
$$txt .= "\n";
$$txt .= @$urls == 1 ? 'Newsgroup' : 'Newsgroups are';
base-commit: 9f3fb207c9daa13648f6d0f960e6fef922b55eab
--
2.33.1
^ permalink raw reply related [relevance 7%]
* [PATCH] www: various help text updates
@ 2021-10-15 7:30 3% Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2021-10-15 7:30 UTC (permalink / raw)
To: meta
`dt:' documentation is redundant with `d:' approxidate support;
so drop `dt:' since mairix uses `d:'. We'll also document
`rt:' since there are legit messages from senders with broken
clocks.
Reduce indentation level of help texts to be in 2-space
increments to using too much horizontal space.
We'll always place IMAP ahead of NNTP since it's alphabetical
and there's likely more IMAP clients out there.
Add "--ng NEWSGROUP" to -init instructions if configured.
There's also some minor wording changes throughout.
---
lib/PublicInbox/Search.pm | 14 +--
lib/PublicInbox/WwwText.pm | 228 +++++++++++++++++++------------------
2 files changed, 125 insertions(+), 117 deletions(-)
diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index d89bf545..145fb56c 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -15,7 +15,7 @@ use Carp ();
# compatibility with old indices (so don't change them it)
use constant {
TS => 0, # Received: in Unix time (IMAP INTERNALDATE, JMAP receivedAt)
- YYYYMMDD => 1, # Date: header for searching in the WWW UI
+ YYYYMMDD => 1, # redundant with DT below
DT => 2, # Date: YYYYMMDDHHMMSS (IMAP SENT*, JMAP sentAt)
# added for public-inbox 1.6.0+
@@ -154,12 +154,9 @@ my %prob_prefix = (
our @HELP = (
's:' => 'match within Subject e.g. s:"a quick brown fox"',
'd:' => <<EOF,
-date range as YYYYMMDD e.g. d:19931002..20101002
-Open-ended ranges such as d:19931002.. and d:..20101002
-are also supported
-EOF
- 'dt:' => <<EOF,
-date-time range as YYYYMMDDhhmmss (e.g. dt:19931002011000..19931002011200)
+match date range, git "approxidate" formats supported
+Open-ended ranges such as `d:last.week..' and `d:..2.days.ago'
+are supported
EOF
'b:' => 'match within message body, including text attachments',
'nq:' => 'match non-quoted text within message body',
@@ -180,6 +177,9 @@ EOF
'dfpre:' => 'match pre-image git blob ID',
'dfpost:' => 'match post-image git blob ID',
'dfblob:' => 'match either pre or post-image git blob ID',
+ 'rt:' => <<EOF,
+match received time, like `d:' unless sender's clock was broken
+EOF
);
chomp @HELP;
diff --git a/lib/PublicInbox/WwwText.pm b/lib/PublicInbox/WwwText.pm
index 8b929f74..211ee8c9 100644
--- a/lib/PublicInbox/WwwText.pm
+++ b/lib/PublicInbox/WwwText.pm
@@ -75,10 +75,10 @@ sub get_text {
}
sub _srch_prefix ($$) {
- my ($srch, $txt) = @_;
+ my ($ibx, $txt) = @_;
my $pad = 0;
my $htxt = '';
- my $help = $srch->help;
+ my $help = $ibx->isrch->help;
my $i;
for ($i = 0; $i < @$help; $i += 2) {
my $pfx = $help->[$i];
@@ -89,10 +89,9 @@ sub _srch_prefix ($$) {
$htxt .= "\f\n";
}
$pad += 2;
- my $padding = ' ' x ($pad + 8);
+ my $padding = ' ' x ($pad + 4);
$htxt =~ s/^/$padding/gms;
- $htxt =~ s/^$padding(\S+)\0/" $1".
- (' ' x ($pad - length($1)))/egms;
+ $htxt =~ s/^$padding(\S+)\0/" $1".(' ' x ($pad - length($1)))/egms;
$htxt =~ s/\f\n/\n/gs;
$$txt .= $htxt;
1;
@@ -113,7 +112,7 @@ Users of browsers such as dillo, Firefox, or some browser
extensions may start by downloading the following sample CSS file
to control the colors they see:
- ${base_url}userContent.css
+ ${base_url}userContent.css
CSS sample
----------
@@ -213,8 +212,8 @@ EOF
defined(my $v = $ibx->{$k}) or next;
$$txt .= "\t$k = $v\n";
}
- $$txt .= "\tnntpmirror = $_\n" for (@{$ibx->nntp_url($ctx)});
$$txt .= "\timapmirror = $_\n" for (@{$ibx->imap_url($ctx)});
+ $$txt .= "\tnntpmirror = $_\n" for (@{$ibx->nntp_url($ctx)});
_coderepo_config($ctx, $txt);
1;
}
@@ -273,6 +272,36 @@ EOF
@ret; # may be empty, this sub is called as an arg for join()
}
+sub _add_imap_nntp_urls ($$) {
+ my ($ctx, $txt) = @_;
+ $ctx->{ibx}->can('nntp_url') or return; # TODO extindex can have IMAP
+ my $urls = $ctx->{ibx}->imap_url($ctx);
+ if (@$urls) {
+ $$txt .= "\nIMAP subfolder(s) are available under:";
+ $$txt .= "\n " . join("\n ", @$urls);
+ $$txt .= <<EOM
+
+ # each subfolder (starting with `0') holds 50K messages at most
+EOM
+ }
+ $urls = $ctx->{ibx}->imap_url($ctx);
+ if (@$urls) {
+ $$txt .= "\n";
+ $$txt .= @$urls == 1 ? 'Newsgroup' : 'Newsgroups are';
+ $$txt .= ' available over NNTP:';
+ $$txt .= "\n " . join("\n ", @$urls) . "\n";
+ }
+}
+
+sub _add_onion_note ($) {
+ my ($txt) = @_;
+ $$txt =~ m!\b[^:]+://\w+\.onion/!i and $$txt .= <<EOM
+
+note: .onion URLs require Tor: https://www.torproject.org/
+
+EOM
+}
+
sub _mirror_help ($$) {
my ($ctx, $txt) = @_;
my $ibx = $ctx->{ibx};
@@ -301,8 +330,10 @@ sub _mirror_help ($$) {
}
my $nr = scalar(@urls);
if ($nr > 1) {
- $$txt .= "\n\t";
- $$txt .= "# this inbox consists of $nr epochs:";
+ chomp($$txt .= <<EOM);
+
+ # this inbox consists of $nr epochs: (no need to clone all of them)
+EOM
$urls[0] .= " # oldest";
$urls[-1] .= " # newest";
}
@@ -316,19 +347,21 @@ sub _mirror_help ($$) {
push @urls, $u;
}
$$txt .= "\n";
- $$txt .= join('', map { "\tgit clone --mirror $_\n" } @urls);
- if (my $addrs = $ibx->{address}) {
- $addrs = join(' ', @$addrs) if ref($addrs) eq 'ARRAY';
- my $v = defined $max ? '-V2' : '-V1';
- $$txt .= <<EOF;
-
- # If you have public-inbox 1.1+ installed, you may
- # initialize and index your mirror using the following commands:
- public-inbox-init $v $ibx->{name} $dir/ $base_url \\
- $addrs
- public-inbox-index $dir
+ $$txt .= join('', map { " git clone --mirror $_\n" } @urls);
+ my $addrs = $ibx->{address} // 'inbox@example.com';
+ my $ng = $ibx->{newsgroup} // '';
+ substr($ng, 0, 0, ' --ng ') if $ng;
+ $addrs = join(' ', @$addrs) if ref($addrs) eq 'ARRAY';
+ my $v = defined $max ? '-V2' : '-V1';
+ $$txt .= <<EOF;
+
+ # If you have public-inbox 1.1+ installed, you may
+ # initialize and index your mirror using the following commands:
+ public-inbox-init $v$ng \\
+ $ibx->{name} ./$dir $base_url \\
+ $addrs
+ public-inbox-index ./$dir
EOF
- }
} else { # PublicInbox::ExtSearch
$$txt .= <<EOM;
This is an external index which is an amalgamation of several public inboxes.
@@ -346,37 +379,13 @@ EOM
Example config snippet for mirrors: $cfg_link
EOF
- if ($ibx->can('imap_url')) {
- my $imap = $ibx->imap_url($ctx);
- if (@$imap) {
- $$txt .= "\n";
- $$txt .= 'IMAP subfolder(s) available under:';
- $$txt .= "\n\t" . join("\n\t", @$imap) . "\n";
- $$txt .= <<EOM
- # each subfolder (starting with `0') holds 50K messages at most
-EOM
- }
- }
- if ($ibx->can('nntp_url')) {
- my $nntp = $ibx->nntp_url($ctx);
- if (scalar @$nntp) {
- $$txt .= "\n";
- $$txt .= @$nntp == 1 ? 'Newsgroup' : 'Newsgroups are';
- $$txt .= ' available over NNTP:';
- $$txt .= "\n\t" . join("\n\t", @$nntp) . "\n";
- }
- }
- if ($$txt =~ m!\b[^:]+://\w+\.onion/!) {
- $$txt .= <<EOM
+ _add_imap_nntp_urls($ctx, $txt);
+ _add_onion_note($txt);
-note: .onion URLs require Tor: https://www.torproject.org/
-
-EOM
- }
my $code_url = prurl($ctx->{env}, $PublicInbox::WwwStream::CODE_URL);
$$txt .= join("\n\n",
coderepos_raw($ctx, $top_url), # may be empty
- "AGPL code for this site:\n\tgit clone $code_url");
+ "AGPL code for this site:\n git clone $code_url");
1;
}
@@ -391,128 +400,127 @@ sub _default_text ($$$$) {
inbox_config($ctx, $hdr, $txt) :
extindex_config($ctx, $hdr, $txt);
}
-
return if $key ne 'help'; # TODO more keys?
my $ibx = $ctx->{ibx};
my $base_url = $ibx->base_url($ctx->{env});
- $$txt .= "public-inbox help for $base_url\n";
$$txt .= <<EOF;
+public-inbox help for $base_url
overview
--------
- public-inbox uses Message-ID identifiers in URLs.
- One may look up messages by substituting Message-IDs
- (without the leading '<' or trailing '>') into the URL.
- Forward slash ('/') characters in the Message-IDs
- need to be escaped as "%2F" (without quotes).
+ public-inbox uses Message-ID identifiers in URLs.
+ One may look up messages by substituting Message-IDs
+ (without the leading '<' or trailing '>') into the URL.
+ Forward slash ('/') characters in the Message-IDs
+ need to be escaped as "%2F" (without quotes).
- Thus, it is possible to retrieve any message by its
- Message-ID by going to:
+ Thus, it is possible to retrieve any message by its
+ Message-ID by going to:
- $base_url<Message-ID>/
+ $base_url<Message-ID>/
+ (without the '<' or '>')
- (without the '<' or '>')
+ Message-IDs are described at:
- Message-IDs are described at:
-
- $WIKI_URL/Message-ID
+ $WIKI_URL/Message-ID
EOF
# n.b. we use the Xapian DB for any regeneratable,
# order-of-arrival-independent data.
- my $srch = $ibx->isrch;
- if ($srch) {
+ if ($ibx->isrch) {
$$txt .= <<EOF;
search
------
- This public-inbox has search functionality provided by Xapian.
+ This public-inbox has search functionality provided by Xapian.
- It supports typical AND, OR, NOT, '+', '-' queries present
- in other search engines.
+ It supports typical AND, OR, NOT, '+', '-' queries present
+ in other search engines.
- We also support search prefixes to limit the scope of the
- search to certain fields.
+ We also support search prefixes to limit the scope of the
+ search to certain fields.
- Prefixes supported in this installation include:
+ Prefixes supported in this installation include:
EOF
- _srch_prefix($srch, $txt);
-
+ _srch_prefix($ibx, $txt);
$$txt .= <<EOF;
- Most prefixes are probabilistic, meaning they support stemming
- and wildcards ('*'). Ranges (such as 'd:') and boolean prefixes
- do not support stemming or wildcards.
- The upstream Xapian query parser documentation fully explains
- the query syntax:
+ Most prefixes are probabilistic, meaning they support stemming
+ and wildcards ('*'). Ranges (such as 'd:') and boolean prefixes
+ do not support stemming or wildcards.
+ The upstream Xapian query parser documentation fully explains
+ the query syntax:
- $QP_URL
+ $QP_URL
EOF
} # $srch
- my $over = $ibx->over;
- if ($over) {
+ if ($ibx->over) {
$$txt .= <<EOF;
message threading
-----------------
- Message threading is enabled for this public-inbox,
- additional endpoints for message threads are available:
+ Message threading is enabled for this public-inbox,
+ additional endpoints for message threads are available:
- * $base_url<Message-ID>/T/#u
+ * $base_url<Message-ID>/T/#u
- Loads the thread belonging to the given <Message-ID>
- in flat chronological order. The "#u" anchor
- focuses the browser on the given <Message-ID>.
+ Loads the thread belonging to the given <Message-ID>
+ in flat chronological order. The "#u" anchor
+ focuses the browser on the given <Message-ID>.
- * $base_url<Message-ID>/t/#u
+ * $base_url<Message-ID>/t/#u
- Loads the thread belonging to the given <Message-ID>
- in threaded order with nesting. For deep threads,
- this requires a wide display or horizontal scrolling.
+ Loads the thread belonging to the given <Message-ID>
+ in threaded order with nesting. For deep threads,
+ this requires a wide display or horizontal scrolling.
- Both of these HTML endpoints are suitable for offline reading
- using the thread overview at the bottom of each page.
+ Both of these HTML endpoints are suitable for offline reading
+ using the thread overview at the bottom of each page.
- Users of feed readers may follow a particular thread using:
+ The gzipped mbox for a thread is available for downloading and
+ importing into your favorite mail client:
- * $base_url<Message-ID>/t.atom
+ * $base_url<Message-ID>/t.mbox.gz
- Which loads the thread in Atom Syndication Standard
- described at Wikipedia and RFC4287:
+ We use the mboxrd variant of the mbox format described at:
- $WIKI_URL/Atom_(standard)
- https://tools.ietf.org/html/rfc4287
+ $WIKI_URL/Mbox
- Atom Threading Extensions (RFC4685) is supported:
+ Users of feed readers may follow a particular thread using:
- https://tools.ietf.org/html/rfc4685
+ * $base_url<Message-ID>/t.atom
- Finally, the gzipped mbox for a thread is available for
- downloading and importing into your favorite mail client:
+ Which loads the thread in Atom Syndication Standard
+ described at Wikipedia and RFC4287:
- * $base_url<Message-ID>/t.mbox.gz
+ $WIKI_URL/Atom_(standard)
+ https://tools.ietf.org/html/rfc4287
- We use the mboxrd variant of the mbox format described
- at:
+ Atom Threading Extensions (RFC4685) are supported:
- $WIKI_URL/Mbox
+ https://tools.ietf.org/html/rfc4685
EOF
} # $over
+ _add_imap_nntp_urls($ctx, \(my $note = ''));
+ $note and $note =~ s/^/ /gms and $$txt .= <<EOF;
+additional protocols
+--------------------
+$note
+EOF
$$txt .= <<EOF;
contact
-------
- This help text is maintained by public-inbox developers
- reachable via plain-text email at: meta\@public-inbox.org
- Their inbox is archived at: https://public-inbox.org/meta/
-
+ This help text is maintained by public-inbox developers
+ reachable via plain-text email at: meta\@public-inbox.org
+ Their inbox is archived at: https://public-inbox.org/meta/
EOF
# TODO: support admin contact info in ~/.public-inbox/config
1;
^ permalink raw reply related [relevance 3%]
Results 1-4 of 4 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2021-10-15 7:30 3% [PATCH] www: various help text updates Eric Wong
2021-10-26 0:48 7% [PATCH] www: mirror: fix rendering of NNTP URLs Kyle Meyer
2023-10-16 11:33 [PATCH 1/3] doc: fix some typos and grammar Štěpán Němec
2023-10-16 11:33 7% ` [PATCH 3/3] search: rectify comment (rt: documented since 738c4a65719e) Štěpán Němec
2023-10-16 21:39 0% ` Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/public-inbox.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).