From: Konstantin Ryabitsev <konstantin@linuxfoundation.org> To: Eric Wong <e@80x24.org> Cc: meta@public-inbox.org Subject: Re: brain dump detached/external index so far... Date: Mon, 14 Sep 2020 12:01:14 -0400 Message-ID: <20200914160114.kdwed3rkb52ibal4@chatter.i7.local> (raw) In-Reply-To: <20200913065550.GA2337@dcvr> On Sun, Sep 13, 2020 at 06:55:50AM +0000, Eric Wong wrote: > Currently (and since the earliest days of this project > supporting Xapian), indices were per-inbox. This allowed > inboxes to be isolated, making it easy to add and remove > inboxes. > > The detached/external indices will allows a merging of > several existing inboxes into a single (or several) > virtual inboxes. > > Why? > > We want a cross-inbox search in the WWW UI, > and perhaps an "All Mail" IMAP/JMAP inbox. FYI, this is the most often requested feature for lore.kernel.org, alongside with "give me a mbox.gz with all followups, regardless to which list they were sent." I think several virtual inboxes makes more sense than always one global search, as people may want to search something like "all Linux kernel discussions" or "all gcc/compiler discussions". There could be different frontends to indicate which search is running -- e.g. "kernel.lore.kernel.org" vs. "gcc.lore.kernel.org". > Advantages: > > Deduplication built-in. Cross-posted messages only get > expensive Xapian data indexed once (multiple List-Id can > get attached to each message). As an off-side grumbling, we found out that AWS SES (their email processing service) will force-rewrite all message IDs without any option to prevent this. So, a single message with multiple recipients will arrive with a unique Message-ID to each one of them. This is so broken, it blows my mind, but AWS doesn't care to fix it. > Disclaimer: > > My brain hasn't been working quite right... > Heat + pandemic + power outages + insects + poor air quality > have all been taking their toll :< I hope things improve on the West Coast soon. The imagery alone was frightening to watch. Best regards, -K
next prev parent reply other threads:[~2020-09-14 16:01 UTC|newest] Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-13 6:55 Eric Wong 2020-09-14 16:01 ` Konstantin Ryabitsev [this message] 2020-09-14 20:55 ` Eric Wong
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://public-inbox.org/README * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200914160114.kdwed3rkb52ibal4@chatter.i7.local \ --to=konstantin@linuxfoundation.org \ --cc=e@80x24.org \ --cc=meta@public-inbox.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
user/dev discussion of public-inbox itself This inbox may be cloned and mirrored by anyone: git clone --mirror http://public-inbox.org/meta git clone --mirror http://czquwvybam4bgbro.onion/meta git clone --mirror http://hjrcffqmbrq6wope.onion/meta git clone --mirror http://ou63pmih66umazou.onion/meta # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 meta meta/ http://public-inbox.org/meta \ meta@public-inbox.org public-inbox-index meta Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.mail.public-inbox.meta nntp://ou63pmih66umazou.onion/inbox.comp.mail.public-inbox.meta nntp://czquwvybam4bgbro.onion/inbox.comp.mail.public-inbox.meta nntp://hjrcffqmbrq6wope.onion/inbox.comp.mail.public-inbox.meta nntp://news.gmane.io/gmane.mail.public-inbox.general note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/public-inbox.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git