search: support alt-ID for mapping legacy serial numbers

For some existing mailing list archives, messages are identified
by serial number (such as NNTP article numbers in gmane).  Those
links may become inaccessible (as is the current case for
gmane), so ensure users can still search based on old serial
numbers.

Now, I run the following periodically to get article numbers
from gmane (while news.gmane.org remains):

	NNTPSERVER=news.gmane.org
	export NNTPSERVER
	GROUP=gmane.comp.version-control.git
	perl -I lib scripts/xhdr-num2mid $GROUP --msgmap=/path/to/gmane.sqlite3

(I might integrate this further with public-inbox-* scripts one day).

My ~/.public-inbox/config as an added "altid" snippet which now
looks like this:

[publicinbox "git"]
	address = git@vger.kernel.org
	mainrepo = /path/to/git.vger.git
	newsgroup = inbox.comp.version-control.git

	; relative pathnames expand to $mainrepo/public-inbox/$file
	altid = serial:gmane:file=gmane.sqlite3

And run "public-inbox-index --reindex /path/to/git.vger.git"
periodically.

This ought to allow searching for "gmane:12345" to work for
Xapian-enabled instances.

Disclaimer: while public-inbox supports NNTP and stable article
serial numbers, use of those for public links is discouraged
since it encourages centralization.

1 files changed, 19 insertions, 1 deletions

diff --git a/lib/PublicInbox/Msgmap.pm b/lib/PublicInbox/Msgmap.pm
index 2583ff47..3fb3805f 100644
--- a/lib/PublicInbox/Msgmap.pm
+++ b/lib/PublicInbox/Msgmap.pm
@@ -20,7 +20,12 @@ sub new {
                 my $err = $!;
                 -d $d or die "$d not created: $err";
         }
-        my $f = "$d/msgmap.sqlite3";
+        new_file($class, "$d/msgmap.sqlite3", $writable);
+}
+
+sub new_file {
+        my ($class, $f, $writable) = @_;
+
         my $dbh = DBI->connect("dbi:SQLite:dbname=$f",'','', {
                 AutoCommit => 1,
                 RaiseError => 1,
@@ -40,6 +45,7 @@ sub new {
         $self;
 }
 
+# n.b. invoked directly by scripts/xhdr-num2mid
 sub meta_accessor {
         my ($self, $key, $value) = @_;
         use constant {
@@ -154,6 +160,7 @@ sub create_tables {
                         'val VARCHAR(255) NOT NULL)');
 }
 
+# used by NNTP.pm
 sub id_batch {
         my ($self, $num, $cb) = @_;
         my $dbh = $self->{dbh};
@@ -167,4 +174,15 @@ sub id_batch {
         $nr;
 }
 
+# only used for mapping external serial numbers (e.g. articles from gmane)
+# see scripts/xhdr-num2mid for usage
+sub mid_set {
+        my ($self, $num, $mid) = @_;
+        my $sth = $self->{mid_set} ||= do {
+                my $sql = 'INSERT INTO msgmap (num, mid) VALUES (?,?)';
+                $self->{dbh}->prepare($sql);
+        };
+        $sth->execute($num, $mid);
+}
+
 1;