[RFC] searchview: don't be too verbose about bad queries

[RFC] searchview: don't be too verbose about bad queries

[Eric Wong]

Ali sent this privately to me as a potential security issue.
I am not a security expert and I certainly don't consider this
a big enough problem to discuss privately...

The potential issue is exposing path names of Xapian installs.

I figure installation paths of open source software
(particularly with FHS / LSB systems) is well-standardized to
the point that it's pointless to obscure or obfuscate anyways.

*shrug*

---------8<-----------
From: Ali Alnubani <alialnu@mellanox.com>
Date: Tue, 11 Jun 2019 10:03:17 +0000
Subject: [PATCH] searchview: don't be too verbose about bad queries

This is to omit the message "something terrible happened at .."
from the http view when searching, since it contains absolute system paths.
This is debug information and shouldn't be displayed to the user.
---
 lib/PublicInbox/SearchView.pm | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/SearchView.pm b/lib/PublicInbox/SearchView.pm
index b089de9..b7859df 100644
--- a/lib/PublicInbox/SearchView.pm
+++ b/lib/PublicInbox/SearchView.pm
@@ -15,6 +15,7 @@ use PublicInbox::MIME;
 require PublicInbox::Git;
 require PublicInbox::SearchThread;
 our $LIM = 200;
+our $ERR_TXT_VERBOSE=0;
 
 sub noop {}
 
@@ -136,8 +137,13 @@ sub err_txt {
 	my $u = $ctx->{-inbox}->base_url($ctx->{env}) . '_/text/help/';
 	$err =~ s/^\s*Exception:\s*//; # bad word to show users :P
 	$err = ascii_html($err);
-	"\nBad query: <b>$err</b>\n" .
-		qq{See <a\nhref="$u">$u</a> for help on using search};
+	my $to_print = "\nBad query";
+	if ($ERR_TXT_VERBOSE) {
+		$to_print .= ": <b>$err</b>\n";
+	} else {
+		$to_print .= ", or search returned too many results.\n";
+	}
+	$to_print . qq{See <a\nhref="$u">$u</a> for help on using search};
 }
 
 sub search_nav_top {
-- 
EW

RE: [RFC] searchview: don't be too verbose about bad queries

[Ali Alnubani]

> -----Original Message-----
> From: Eric Wong <e@80x24.org>
> Sent: Tuesday, June 11, 2019 10:38 PM
> To: meta@public-inbox.org
> Cc: Ali Alnubani <alialnu@mellanox.com>
> Subject: [RFC] searchview: don't be too verbose about bad queries
> 
> Ali sent this privately to me as a potential security issue.
> I am not a security expert and I certainly don't consider this a big enough
> problem to discuss privately...
> 
> The potential issue is exposing path names of Xapian installs.
> 
> I figure installation paths of open source software (particularly with FHS / LSB
> systems) is well-standardized to the point that it's pointless to obscure or
> obfuscate anyways.
They are standardized for system-wide installations. But having perl libs/modules/binaries
installed per user or on non-default paths could expose some private info, including usernames
for example, making those system users subject to some attacks.
> 
> *shrug*

Anyway, I also agree that it's not that critical.

Thanks,
Ali
> 
> ---------8<-----------
> From: Ali Alnubani <alialnu@mellanox.com>
> Date: Tue, 11 Jun 2019 10:03:17 +0000
> Subject: [PATCH] searchview: don't be too verbose about bad queries
> 
> This is to omit the message "something terrible happened at .."
> from the http view when searching, since it contains absolute system paths.
> This is debug information and shouldn't be displayed to the user.
> ---
>  lib/PublicInbox/SearchView.pm | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/PublicInbox/SearchView.pm b/lib/PublicInbox/SearchView.pm
> index b089de9..b7859df 100644
> --- a/lib/PublicInbox/SearchView.pm
> +++ b/lib/PublicInbox/SearchView.pm
> @@ -15,6 +15,7 @@ use PublicInbox::MIME;  require PublicInbox::Git;
> require PublicInbox::SearchThread;  our $LIM = 200;
> +our $ERR_TXT_VERBOSE=0;
> 
>  sub noop {}
> 
> @@ -136,8 +137,13 @@ sub err_txt {
>  	my $u = $ctx->{-inbox}->base_url($ctx->{env}) . '_/text/help/';
>  	$err =~ s/^\s*Exception:\s*//; # bad word to show users :P
>  	$err = ascii_html($err);
> -	"\nBad query: <b>$err</b>\n" .
> -		qq{See <a\nhref="$u">$u</a> for help on using search};
> +	my $to_print = "\nBad query";
> +	if ($ERR_TXT_VERBOSE) {
> +		$to_print .= ": <b>$err</b>\n";
> +	} else {
> +		$to_print .= ", or search returned too many results.\n";
> +	}
> +	$to_print . qq{See <a\nhref="$u">$u</a> for help on using search};
>  }
> 
>  sub search_nav_top {
> --
> EW

Re: [RFC] searchview: don't be too verbose about bad queries

[Eric Wong]

Ali Alnubani <alialnu@mellanox.com> wrote:
> > -----Original Message-----
> > From: Eric Wong <e@80x24.org>
> > Sent: Tuesday, June 11, 2019 10:38 PM
> > To: meta@public-inbox.org
> > Cc: Ali Alnubani <alialnu@mellanox.com>
> > Subject: [RFC] searchview: don't be too verbose about bad queries
> > 
> > Ali sent this privately to me as a potential security issue.
> > I am not a security expert and I certainly don't consider this a big enough
> > problem to discuss privately...
> > 
> > The potential issue is exposing path names of Xapian installs.
> > 
> > I figure installation paths of open source software (particularly with FHS / LSB
> > systems) is well-standardized to the point that it's pointless to obscure or
> > obfuscate anyways.
> They are standardized for system-wide installations. But having perl libs/modules/binaries
> installed per user or on non-default paths could expose some private info, including usernames
> for example, making those system users subject to some attacks.

Fair point.

Maybe a reverse-mapping of %INC can be used to translate the
full path to the short path name (e.g. "Xapian/Enquire.pm")

Something like the following (totally untested):

	# global
	my %rmap_inc = map { "$INC{$_}" => $_ } keys %INC;

	# in err_txt:
	$err =~ s!\b(\S+)\b!
		my $full = $1;
		if (-e $full) {
			my $short = $rmap_inc{$full};
			unless (defined $short) {
				# rebuild rmap in case new modules were loaded
				%rmap_inc = map { "$INC{$_}" => $_ } keys %INC;
			}

			# fall back to basename as last resort
			$short = $rmap_inc{$full} // ((split('/', $full))[-1];
		} else {
			$full;
		}
	!sge;

[PATCH] searchview: avoid displaying full paths on errors

[Eric Wong]

Displaying full path names of installed modules could expose
unnecessary information about user home directory names or other
potentially sensitive information.  However, displaying a module
name could still be useful for diagnosing problems, so map full
paths to the relevant part of the path name which is relevant to
the package name.

Reported-by: Ali Alnubani <alialnu@mellanox.com>
  https://public-inbox.org/meta/20190611193815.c4uovtlp574bid6x@dcvr/
---
 lib/PublicInbox/SearchView.pm | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/lib/PublicInbox/SearchView.pm b/lib/PublicInbox/SearchView.pm
index 6f07279b..a8b66dda 100644
--- a/lib/PublicInbox/SearchView.pm
+++ b/lib/PublicInbox/SearchView.pm
@@ -15,6 +15,7 @@ use PublicInbox::MIME;
 require PublicInbox::Git;
 require PublicInbox::SearchThread;
 our $LIM = 200;
+my %rmap_inc;
 
 sub noop {}
 
@@ -138,10 +139,27 @@ sub mset_summary {
 	*noop;
 }
 
+# shorten "/full/path/to/Foo/Bar.pm" to "Foo/Bar.pm" so error
+# messages don't reveal FS layout info in case people use non-standard
+# installation paths
+sub path2inc ($) {
+	my $full = $_[0];
+	if (my $short = $rmap_inc{$full}) {
+		return $short;
+	} elsif (!scalar(keys %rmap_inc) && -e $full) {
+		%rmap_inc = map {; "$INC{$_}" => $_ } keys %INC;
+		# fall back to basename as last resort
+		$rmap_inc{$full} // (split('/', $full))[-1];
+	} else {
+		$full;
+	}
+}
+
 sub err_txt {
 	my ($ctx, $err) = @_;
 	my $u = $ctx->{-inbox}->base_url($ctx->{env}) . '_/text/help/';
 	$err =~ s/^\s*Exception:\s*//; # bad word to show users :P
+	$err =~ s!(\S+)!path2inc($1)!sge;
 	$err = ascii_html($err);
 	"\nBad query: <b>$err</b>\n" .
 		qq{See <a\nhref="$u">$u</a> for help on using search};
-- 
EW

RE: [PATCH] searchview: avoid displaying full paths on errors

[Ali Alnubani]

Thanks for the patch Eric.
I apologize for taking so long to update mine.

> -----Original Message-----
> From: Eric Wong <e@80x24.org>
> Sent: Tuesday, June 25, 2019 9:37 AM
> To: Ali Alnubani <alialnu@mellanox.com>
> Cc: meta@public-inbox.org
> Subject: [PATCH] searchview: avoid displaying full paths on errors
> 
> Displaying full path names of installed modules could expose unnecessary
> information about user home directory names or other potentially sensitive
> information.  However, displaying a module name could still be useful for
> diagnosing problems, so map full paths to the relevant part of the path name
> which is relevant to the package name.
> 
> Reported-by: Ali Alnubani <alialnu@mellanox.com>
> 
<removed>
> ---
>  lib/PublicInbox/SearchView.pm | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
> 
> diff --git a/lib/PublicInbox/SearchView.pm b/lib/PublicInbox/SearchView.pm
> index 6f07279b..a8b66dda 100644
> --- a/lib/PublicInbox/SearchView.pm
> +++ b/lib/PublicInbox/SearchView.pm
> @@ -15,6 +15,7 @@ use PublicInbox::MIME;  require PublicInbox::Git;
> require PublicInbox::SearchThread;  our $LIM = 200;
> +my %rmap_inc;
> 
>  sub noop {}
> 
> @@ -138,10 +139,27 @@ sub mset_summary {
>  	*noop;
>  }
> 
> +# shorten "/full/path/to/Foo/Bar.pm" to "Foo/Bar.pm" so error #
> +messages don't reveal FS layout info in case people use non-standard #
> +installation paths sub path2inc ($) {
> +	my $full = $_[0];
> +	if (my $short = $rmap_inc{$full}) {
> +		return $short;
> +	} elsif (!scalar(keys %rmap_inc) && -e $full) {
> +		%rmap_inc = map {; "$INC{$_}" => $_ } keys %INC;
> +		# fall back to basename as last resort
> +		$rmap_inc{$full} // (split('/', $full))[-1];
> +	} else {
> +		$full;
> +	}
> +}
> +
>  sub err_txt {
>  	my ($ctx, $err) = @_;
>  	my $u = $ctx->{-inbox}->base_url($ctx->{env}) . '_/text/help/';
>  	$err =~ s/^\s*Exception:\s*//; # bad word to show users :P
> +	$err =~ s!(\S+)!path2inc($1)!sge;
>  	$err = ascii_html($err);
>  	"\nBad query: <b>$err</b>\n" .
>  		qq{See <a\nhref="$u">$u</a> for help on using search};
> --
> EW

Thanks,
Ali

Re: [PATCH] searchview: avoid displaying full paths on errors

[Eric Wong]

Ali Alnubani <alialnu@mellanox.com> wrote:
> Thanks for the patch Eric.
> I apologize for taking so long to update mine.

No worries.  Pushed as c19a4e88f49ba3496751c4b87ebcfa0f6b47f0ce

> >  sub err_txt {
> >  	my ($ctx, $err) = @_;
> >  	my $u = $ctx->{-inbox}->base_url($ctx->{env}) . '_/text/help/';
> >  	$err =~ s/^\s*Exception:\s*//; # bad word to show users :P
> > +	$err =~ s!(\S+)!path2inc($1)!sge;

One potential issue is this won't match paths if somebody has
spaces in their installation paths.  But I suppose few
distros or users building their own Perl are that strange...