From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00
	shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 2FD461F424;
	Thu,  5 Apr 2018 08:59:28 +0000 (UTC)
Date: Thu, 5 Apr 2018 08:59:28 +0000
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: Re: [PATCH 1/5] replace Xapian skeleton with SQLite overview DB
Message-ID: <20180405085928.GA12764@80x24.org>
References: <20180402000456.13446-1-e@80x24.org>
 <20180402000456.13446-2-e@80x24.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20180402000456.13446-2-e@80x24.org>
List-Id: <meta.public-inbox.org>

> This ought to provide better performance and scalability
> which is less dependent on inbox size.  Xapian does not
> seem optimized for some queries used by the WWW homepage,
> Atom feeds, XOVER and NEWNEWS NNTP commands.

So looking at the traffic I get on v1 public-inbox.org, less
than 1% of the requests would need to hit Xapian with this
change.  Add to that, Over (or Skeleton before it) is around 7%
the size of a compacted Xapian DB.

> This can actually make Xapian optional for NNTP usage,
> and allow more functionality to work without Xapian
> installed.

Sometime in the future it would make sense to decouple them and
allow space-constrained systems to run full NNTP and a larger
chunk of the PSGI (HTTP) code without Xapian.