user/dev discussion of public-inbox itself
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* Merge branch 'unsubscribe'
  2016-05-20 21:36  3% [PATCH] unsubscribe: get off mah lawn^H^H^Hist Eric Wong
@ 2016-06-07 13:18  7% ` Eric Wong
  0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2016-06-07 13:18 UTC (permalink / raw)
  To: meta

commit 3f779258173530ca88f31e1dc5332f951d2c44cd
Merge: e2adc94 852df98
Author: Eric Wong <e@80x24.org>
Date:   Tue Jun 7 12:57:42 2016 +0000

    Merge branch 'unsubscribe'
    
    * unsubscribe:
      unsubscribe.milter: use default postfork dispatcher
      unsubscribe: prevent decrypt from showing random crap
      examples/unsubscribe-psgi@.service: disable worker processes
      unsubscribe: bad URL fixup
      unsubscribe: get off mah lawn^H^H^Hist

^ permalink raw reply	[relevance 7%]

* [PATCH] unsubscribe: get off mah lawn^H^H^Hist
@ 2016-05-20 21:36  3% Eric Wong
  2016-06-07 13:18  7% ` Merge branch 'unsubscribe' Eric Wong
  0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2016-05-20 21:36 UTC (permalink / raw)
  To: meta

While public-inbox is intended primarily for archival,
SMTP list subscriptions are still in use in most places
and users are likely to want a good unsubscribe mechanism.

HTTP (or HTTPS) links in the List-Unsubscribe header are
often preferable since some users may use an incorrect
email address for mailto: links.

Thus, it is useful to provide an example which generates an
HTTPS link for users to click on.  The default .psgi requires
a POST confirmation (as destructive actions with GET are
considered bad practice).  However, the "confirm" parameter
may be disabled for a true "one-click" unsubscribe.

The generated URLs are hopefully short enough and both shell
and highlighting-friendly to reduce copy+paste errors.
---
 examples/README.unsubscribe          |  40 ++++++++
 examples/unsubscribe-milter.socket   |  10 ++
 examples/unsubscribe-milter@.service |  24 +++++
 examples/unsubscribe-psgi.socket     |  11 +++
 examples/unsubscribe-psgi@.service   |  20 ++++
 examples/unsubscribe.milter          | 139 +++++++++++++++++++++++++++
 examples/unsubscribe.psgi            |  68 +++++++++++++
 lib/PublicInbox/Unsubscribe.pm       | 179 +++++++++++++++++++++++++++++++++++
 8 files changed, 491 insertions(+)
 create mode 100644 examples/README.unsubscribe
 create mode 100644 examples/unsubscribe-milter.socket
 create mode 100644 examples/unsubscribe-milter@.service
 create mode 100644 examples/unsubscribe-psgi.socket
 create mode 100644 examples/unsubscribe-psgi@.service
 create mode 100644 examples/unsubscribe.milter
 create mode 100644 examples/unsubscribe.psgi
 create mode 100644 lib/PublicInbox/Unsubscribe.pm

diff --git a/examples/README.unsubscribe b/examples/README.unsubscribe
new file mode 100644
index 0000000..7c41067
--- /dev/null
+++ b/examples/README.unsubscribe
@@ -0,0 +1,40 @@
+Unsubscribe endpoints for mlmmj users (and possibly Mailman, too)
+
+* examples/unsubscribe.milter filters outgoing messages
+  and appends an HTTPS URL to the List-Unsubscribe header.
+  This List-Unsubscribe header should point to the PSGI
+  described below.
+  Currently, this is only active for a whitelist of test
+  addresses in /etc/unsubscribe-milter.whitelist
+  with one email address per line.
+
+* examples/unsubscribe.psgi is a PSGI which needs to run
+  as the mlmmj user with permission to run mlmmj-unsub.
+  This depends on the PublicInbox::Unsubscribe module
+  which may be extracted from the rest of public-inbox.
+  It is strongly recommended to NOT run the rest of the
+  public-inbox WWW code in the same process as this PSGI.
+  (The public-inbox WWW code will never need write
+   permissions to anything besides stderr).
+
+* Both the .milter and .psgi examples are bundled with
+  systemd service and socket activation examples.
+  AFAIK no other PSGI server besides public-inbox-httpd
+  supports systemd socket activation.
+
+To wire up the milter for postfix, I use the following
+in /etc/postfix/main.cf:
+
+  # Milter configuration
+  milter_default_action = accept
+  milter_protocol = 2
+
+  # other milters may be chained here (e.g. opendkim)
+  # chroot users will need to adjust this path
+  smtpd_milters = local:/var/spool/postfix/unsubscribe/unsubscribe.sock
+
+  # This is not needed for mlmmj since mlmmj uses SMTP:
+  # non_smtpd_milters = local:/var/spool/postfix/unsubscribe/unsubscribe.sock
+
+Copyright (C) 2016 all contributors <meta@public-inbox.org>
+License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
diff --git a/examples/unsubscribe-milter.socket b/examples/unsubscribe-milter.socket
new file mode 100644
index 0000000..bfaa97a
--- /dev/null
+++ b/examples/unsubscribe-milter.socket
@@ -0,0 +1,10 @@
+# ==> /etc/systemd/system/unsubscribe-milter.socket <==
+[Unit]
+Description = unsubscribe.milter socket
+
+[Socket]
+ListenStream = /var/spool/postfix/unsubscribe/unsubscribe.sock
+Service = unsubscribe-milter@1.service
+
+[Install]
+WantedBy = sockets.target
diff --git a/examples/unsubscribe-milter@.service b/examples/unsubscribe-milter@.service
new file mode 100644
index 0000000..98e3d47
--- /dev/null
+++ b/examples/unsubscribe-milter@.service
@@ -0,0 +1,24 @@
+# ==> /etc/systemd/system/unsubscribe-milter@.service <==
+# The '@' is to allow multiple simultaneous services to start
+# and share the same socket so new code can be cycled in
+# without downtime
+
+[Unit]
+Description = unsubscribe milter %i
+Wants = unsubscribe-milter.socket
+After = unsubscribe-milter.socket
+
+[Service]
+# First 8 bytes is for the key, next 8 bytes is for the IV
+# using Blowfish.  We want as short URLs as possible to avoid
+# copy+paste errors
+# umask 077 && dd if=/dev/urandom bs=16 count=1 of=.unsubscribe.key
+ExecStart = /usr/local/sbin/unsubscribe.milter /home/mlmmj/.unsubscribe.key
+Sockets = unsubscribe-milter.socket
+
+# the corresponding PSGI app needs permissions to modify the
+# mlmmj spool, so we might as well use the same user since
+User = mlmmj
+
+[Install]
+WantedBy = multi-user.target
diff --git a/examples/unsubscribe-psgi.socket b/examples/unsubscribe-psgi.socket
new file mode 100644
index 0000000..e7ab797
--- /dev/null
+++ b/examples/unsubscribe-psgi.socket
@@ -0,0 +1,11 @@
+# ==> /etc/systemd/system/unsubscribe-psgi.socket <==
+[Unit]
+Description = unsubscribe PSGI socket
+
+[Socket]
+# Forward to the PSGI using nginx or similar
+ListenStream = /run/unsubscribe-psgi.sock
+Service = unsubscribe-psgi@1.service
+
+[Install]
+WantedBy = sockets.target
diff --git a/examples/unsubscribe-psgi@.service b/examples/unsubscribe-psgi@.service
new file mode 100644
index 0000000..f588886
--- /dev/null
+++ b/examples/unsubscribe-psgi@.service
@@ -0,0 +1,20 @@
+# ==> /etc/systemd/system/unsubscribe-psgi@.service <==
+# The '@' is to allow multiple simultaneous services to start
+# and share the same socket so new code can be cycled in
+# without downtime
+
+[Unit]
+Description = unsubscribe PSGI %i
+Wants = unsubscribe-psgi.socket
+After = unsubscribe-psgi.socket
+
+[Service]
+# any PSGI server ought to work,
+# but public-inbox-httpd supports socket activation like unsubscribe.milter
+ExecStart = /usr/local/bin/public-inbox-httpd /etc/unsubscribe.psgi
+Sockets = unsubscribe-psgi.socket
+# we need to modify the mlmmj spool
+User = mlmmj
+
+[Install]
+WantedBy = multi-user.target
diff --git a/examples/unsubscribe.milter b/examples/unsubscribe.milter
new file mode 100644
index 0000000..e193638
--- /dev/null
+++ b/examples/unsubscribe.milter
@@ -0,0 +1,139 @@
+#!/usr/bin/perl -w
+# Copyright (C) 2016 all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+use strict;
+use warnings;
+use Sendmail::PMilter qw(:all);
+use IO::Socket;
+use Crypt::CBC;
+use MIME::Base64 qw(encode_base64url);
+
+my $key_file = shift @ARGV or die "Usage: $0 KEY_FILE\n";
+open my $fh, '<', $key_file or die "failed to open $key_file\n";
+my ($key, $iv);
+if (read($fh, $key, 8) != 8 || read($fh, $iv, 8) != 8 ||
+			read($fh, my $end, 8) != 0) {
+	die "KEY_FILE must be 16 bytes\n";
+}
+
+# these parameters were chosen to generate shorter parameters
+# to reduce the possibility of copy+paste errors
+my $crypt = Crypt::CBC->new(-key => $key,
+			-iv => $iv,
+			-header => 'none',
+			-cipher => 'Blowfish');
+$fh = $iv = $key = undef;
+
+my %cbs;
+$cbs{connect} = sub {
+	my ($ctx) = @_;
+	eval { $ctx->setpriv({ header => {}, envrcpt => {} }) };
+	warn $@ if $@;
+	SMFIS_CONTINUE;
+};
+
+$cbs{envrcpt} = sub {
+	my ($ctx, $addr) = @_;
+	eval {
+		$addr =~ tr!<>!!d;
+		$ctx->getpriv->{envrcpt}->{$addr} = 1;
+	};
+	warn $@ if $@;
+	SMFIS_CONTINUE;
+};
+
+$cbs{header} = sub {
+	my ($ctx, $k, $v) = @_;
+	eval {
+		my $k_ = lc $k;
+		if ($k_ eq 'list-unsubscribe') {
+			my $header = $ctx->getpriv->{header} ||= {};
+			my $ary = $header->{$k_} ||= [];
+
+			# we create placeholders in case there are
+			# multiple headers of the same name
+			my $cur = [];
+			push @$ary, $cur;
+
+			# This relies on mlmmj convention:
+			#	$LIST+unsubscribe@$DOMAIN
+			if ($v =~ /\A<mailto:([^@]+)\+unsubscribe@([^>]+)>\z/) {
+				@$cur = ($k, $v, $1, $2);
+
+			# Mailman convention:
+			#	$LIST-request@$DOMAIN?subject=unsubscribe
+			} elsif ($v =~ /\A<mailto:([^@]+)-request@
+					([^\?]+)\?subject=unsubscribe>\z/x) {
+				# @$cur = ($k, $v, $1, $2);
+			}
+		}
+	};
+	warn $@ if $@;
+	SMFIS_CONTINUE;
+};
+
+# only whitelist a few users for testing:
+my $whitelist = '/etc/unsubscribe-milter.whitelist';
+my %TEST_WHITELIST = map { $_ => 1 } eval {
+		open my $fh, '<', $whitelist or
+			die "Failed to open $whitelist: $!";
+		local $/ = "\n";
+		chomp(my @lines = (<$fh>));
+		@lines;
+	};
+die "No whitelist at $whitelist\n" unless scalar keys %TEST_WHITELIST;
+
+$cbs{eom} = sub {
+	my ($ctx) = @_;
+	eval {
+		my $priv = $ctx->getpriv;
+		$ctx->setpriv({ header => {}, envrcpt => {} });
+		my @rcpt = keys %{$priv->{envrcpt}};
+
+		# one recipient, one unique HTTP(S) URL
+		return SMFIS_CONTINUE if @rcpt != 1;
+		return SMFIS_CONTINUE unless $TEST_WHITELIST{$rcpt[0]};
+
+		my $unsub = $priv->{header}->{'list-unsubscribe'} || [];
+		my $n = 0;
+		foreach my $u (@$unsub) {
+			# Milter indices are 1-based,
+			# not 0-based like Perl arrays
+			my $index = ++$n;
+			my ($k, $v, $list, $domain) = @$u;
+
+			next unless $k && $v && $list && $domain;
+			my $u = $crypt->encrypt($rcpt[0]);
+			$u = encode_base64url($u);
+			$v .= ",\n <https://$domain/u/$u/$list>";
+
+			$ctx->chgheader($k, $index, $v);
+		}
+	};
+	warn $@ if $@;
+	SMFIS_CONTINUE;
+};
+
+my $milter = Sendmail::PMilter->new;
+
+# Try to inherit a socket from systemd or similar:
+my $fds = $ENV{LISTEN_FDS};
+if ($fds && (($ENV{LISTEN_PID} || 0) == $$)) {
+	die "$0 can only listen on one FD\n" if $fds != 1;
+	my $start_fd = 3;
+	my $s = IO::Socket->new_from_fd($start_fd, 'r') or
+		die "inherited bad FD from LISTEN_FDS: $!\n";
+	$milter->set_socket($s);
+} else {
+	# fall back to binding a socket:
+	my $sock = 'unix:/var/spool/postfix/unsubscribe/unsubscribe.sock';
+	$milter->set_listen(1024);
+	my $umask = umask 0000;
+	$milter->setconn($sock);
+	umask $umask;
+}
+
+$milter->register('unsubscribe', \%cbs, SMFI_CURR_ACTS);
+my $dispatcher = Sendmail::PMilter::prefork_dispatcher(max_children => 2);
+$milter->set_dispatcher($dispatcher);
+$milter->main();
diff --git a/examples/unsubscribe.psgi b/examples/unsubscribe.psgi
new file mode 100644
index 0000000..82e186b
--- /dev/null
+++ b/examples/unsubscribe.psgi
@@ -0,0 +1,68 @@
+#!/usr/bin/perl -w
+# Copyright (C) 2016 all contributors <meta@public-inbox.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# This should not require any other PublicInbox code, but may use
+# PublicInbox::Config if ~/.public-inbox/config exists or
+# PI_CONFIG is pointed to an appropriate location
+use strict;
+use Plack::Builder;
+use PublicInbox::Unsubscribe;
+my $app = PublicInbox::Unsubscribe->new(
+	pi_config => eval { # optional, for pointing out archives
+		require PublicInbox::Config;
+		# uses ~/.public-inbox/config by default,
+		# can override with PI_CONFIG or here since
+		# I run this .psgi as the mlmmj user while the
+		# public-inbox-mda code which actually writes to
+		# the archives runs as a different user.
+		PublicInbox::Config->new('/home/pi/.public-inbox/config')
+	},
+	code_url => 'git://80x24.org/public-inbox.git', # change if you fork
+	owner_email => 'BOFH@example.com',
+	confirm => 1,
+
+	# First 8 bytes is for the key, next 8 bytes is for the IV
+	# using Blowfish.  We want as short URLs as possible to avoid
+	# copy+paste errors
+	# umask 077 && dd if=/dev/urandom bs=16 count=1 of=.unsubscribe.key
+	key_file => '/home/mlmmj/.unsubscribe.key',
+
+	# this runs as whatever user has perms to run /usr/bin/mlmmj-unsub
+	# users of other mailing lists.  Returns '' on success.
+	unsubscribe => sub {
+		my ($user_addr, $list_addr) = @_;
+
+		# map list_addr to mlmmj spool, I use:
+		# /home/mlmmj/spool/$LIST here
+		my ($list, $domain) = split('@', $list_addr, 2);
+		my $spool = "/home/mlmmj/spool/$list";
+
+		return "Invalid list: $list" unless -d $spool;
+
+		# -c to send a confirmation email, -s is important
+		# in case a user is click-happy and clicks twice.
+		my @cmd = (qw(/usr/bin/mlmmj-unsub -c -s),
+				'-L', $spool, '-a', $user_addr);
+
+		# we don't know which version they're subscribed to,
+		# try both non-digest and digest
+		my $normal = system(@cmd);
+		my $digest = system(@cmd, '-d');
+
+		# success if either succeeds:
+		return '' if ($normal == 0 || $digest == 0);
+
+		# missing executable or FS error,
+		# otherwise -s always succeeds, right?
+		return 'Unknown error, contact admin';
+	},
+);
+
+builder {
+	mount '/u' => builder {
+		eval { enable 'Deflater' }; # optional
+		eval { enable 'ReverseProxy' }; # optional
+		enable 'Head';
+		sub { $app->call(@_) };
+	};
+};
diff --git a/lib/PublicInbox/Unsubscribe.pm b/lib/PublicInbox/Unsubscribe.pm
new file mode 100644
index 0000000..1f5ce31
--- /dev/null
+++ b/lib/PublicInbox/Unsubscribe.pm
@@ -0,0 +1,179 @@
+# Copyright (C) 2016 all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+#
+# Standalone PSGI app to handle HTTP(s) unsubscribe links generated
+# by milters like examples/unsubscribe.milter to mailing lists.
+#
+# This does not depend on any other modules in the PublicInbox::*
+# and ought to be usable with any mailing list software.
+package PublicInbox::Unsubscribe;
+use strict;
+use warnings;
+use Crypt::CBC;
+use Plack::Util;
+use MIME::Base64 qw(decode_base64url);
+my $CODE_URL = 'git://80x24.org/public-inbox.git';
+my @CT_HTML = ('Content-Type', 'text/html; charset=UTF-8');
+
+sub new {
+	my ($class, %opt) = @_;
+	my $key_file = $opt{key_file};
+	defined $key_file or die "`key_file' needed";
+	open my $fh, '<', $key_file or die
+		"failed to open key_file=$key_file: $!\n";
+	my ($key, $iv);
+	if (read($fh, $key, 8) != 8 || read($fh, $iv, 8) != 8 ||
+				read($fh, my $end, 8) != 0) {
+		die "key_file must be 16 bytes\n";
+	}
+
+	# these parameters were chosen to generate shorter parameters
+	# to reduce the possibility of copy+paste errors
+	my $cipher = Crypt::CBC->new(-key => $key,
+			-iv => $iv,
+			-header => 'none',
+			-cipher => 'Blowfish');
+
+	my $e = $opt{owner_email} or die "`owner_email' not specified\n";
+	my $unsubscribe = $opt{unsubscribe} or
+		die "`unsubscribe' callback not given\n";
+
+	bless {
+		pi_config => $opt{pi_config}, # PublicInbox::Config
+		owner_email => $opt{owner_email},
+		cipher => $cipher,
+		unsubscribe => $unsubscribe,
+		contact => qq(<a\nhref="mailto:$e">$e</a>),
+		code_url => $opt{code_url} || $CODE_URL,
+		confirm => $opt{confirm},
+	}, $class;
+}
+
+# entry point for PSGI
+sub call {
+	my ($self, $env) = @_;
+	my $m = $env->{REQUEST_METHOD};
+	if ($m eq 'GET' || $m eq 'HEAD') {
+		$self->{confirm} ? confirm_prompt($self, $env)
+				 : finalize_unsub($self, $env);
+	} elsif ($m eq 'POST') {
+		finalize_unsub($self, $env);
+	} else {
+		r($self, 405,
+			Plack::Util::encode_html($m).' method not allowed');
+	}
+}
+
+sub _user_list_addr {
+	my ($self, $env) = @_;
+	my ($blank, $u, $list) = split('/', $env->{PATH_INFO});
+
+	if (!defined $u || $u eq '') {
+		return r($self, 400, 'Bad request',
+			'Missing encrypted email address in path component');
+	}
+	if (!defined $list && $list eq '') {
+		return r($self, 400, 'Bad request',
+			'Missing mailing list name in path component');
+	}
+	my $user = eval { $self->{cipher}->decrypt(decode_base64url($u)) };
+	if (!defined $user) {
+		my $err = quotemeta($@);
+		my $errors = $env->{'psgi.errors'};
+		$errors->print("error decrypting: $u\n");
+		$errors->print("$_\n") for split("\n", $err);
+		return r($self, 400, 'Bad request', "Failed to decrypt: $u");
+	}
+
+	# The URLs are too damn long if we have the encrypted domain
+	# name in the query string
+	if (index($list, '@') < 0) {
+		my $host = (split(':', $env->{HTTP_HOST}))[0];
+		$list .= '@'.$host;
+	}
+	($user, $list);
+}
+
+sub confirm_prompt { # on GET
+	my ($self, $env) = @_;
+	my ($user_addr, $list_addr) = _user_list_addr($self, $env);
+	return $user_addr if ref $user_addr;
+
+	my $xl = Plack::Util::encode_html($list_addr);
+	my $xu = Plack::Util::encode_html($user_addr);
+	my @body = (
+		"Confirmation required to remove", '',
+		"\t$xu", '',
+		"from the mailing list at", '',
+		"\t$xl", '',
+		'You will get one last email once you hit "Confirm" below:',
+		qq(</pre><form\nmethod=post\naction="">) .
+		qq(<input\ntype=submit\nvalue="Confirm" />) .
+		'</form><pre>');
+
+	push @body, archive_info($self, $env, $list_addr);
+
+	r($self, 200, "Confirm unsubscribe for $xl", @body);
+}
+
+sub finalize_unsub { # on POST
+	my ($self, $env) = @_;
+	my ($user_addr, $list_addr) = _user_list_addr($self, $env);
+	return $user_addr if ref $user_addr;
+
+	my @archive = archive_info($self, $env, $list_addr);
+	if (my $err = $self->{unsubscribe}->($user_addr, $list_addr)) {
+		return r($self, 500, Plack::Util::encode_html($err), @archive);
+	}
+
+	my $xl = Plack::Util::encode_html($list_addr);
+	r($self, 200, "Unsubscribed from $xl",
+		'You may get one final goodbye message', @archive);
+}
+
+sub r {
+	my ($self, $code, $title, @body) = @_;
+	[ $code, [ @CT_HTML ], [
+		"<html><head><title>$title</title></head><body><pre>".
+		join("\n", "<b>$title</b>\n", @body) . '</pre><hr />'.
+		"<pre>This page is available under AGPL-3.0+\n" .
+		"git clone $self->{code_url}\n" .
+		qq(Email $self->{contact} if you have any questions).
+		'</pre></body></html>'
+	] ];
+}
+
+sub archive_info {
+	my ($self, $env, $list_addr) = @_;
+	my $archive_url = $self->{archive_urls}->{$list_addr};
+
+	unless ($archive_url) {
+		if (my $config = $self->{pi_config}) {
+			# PublicInbox::Config::lookup
+			my $inbox = $config->lookup($list_addr);
+			# PublicInbox::Inbox::base_url
+			$archive_url = $inbox->base_url if $inbox;
+		}
+	}
+
+	# protocol-relative URL:  "//example.com/" => "https://example.com/"
+	if ($archive_url =~ m!\A//!) {
+		$archive_url = "$env->{'psgi.url_scheme'}:$archive_url";
+	}
+
+	# maybe there are other places where we could map
+	# list_addr => archive_url without ~/.public-inbox/config
+	if ($archive_url) {
+		$archive_url = Plack::Util::encode_html($archive_url);
+		('',
+		'HTML and git clone-able archives are available at:',
+		qq(<a\nhref="$archive_url">$archive_url</a>))
+	} else {
+		('',
+		'There ought to be archives for this list,',
+		'but unfortunately the admin did not configure '.
+		__PACKAGE__. ' to show you the URL');
+	}
+}
+
+1;

^ permalink raw reply	[relevance 3%]

Results 1-2 of 2 | reverse | sort options + mbox downloads above
-- links below jump to the message on this page --
2016-05-20 21:36  3% [PATCH] unsubscribe: get off mah lawn^H^H^Hist Eric Wong
2016-06-07 13:18  7% ` Merge branch 'unsubscribe' Eric Wong

Code repositories for project(s) associated with this inbox:

	https://80x24.org/public-inbox.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).