From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_RED shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 3DFCD1F5AE; Mon, 26 Apr 2021 18:47:17 +0000 (UTC) Date: Mon, 26 Apr 2021 18:47:17 +0000 From: Eric Wong To: meta@public-inbox.org Subject: Re: lei-managed pseudo mailing lists Message-ID: <20210426184717.GA29112@dcvr> References: <20210426164454.5zd5kgugfhfwfkpo@nitro.local> <20210426173726.GA22986@dcvr> <20210426182020.olonbxkc6a2gfzl3@nitro.local> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20210426182020.olonbxkc6a2gfzl3@nitro.local> List-Id: Konstantin Ryabitsev wrote: > On Mon, Apr 26, 2021 at 05:37:26PM +0000, Eric Wong wrote: > > > The latter is specifically something I think would be of interest to kernel > > > folks, so I envision that we'd have something like the following: > > > > > > - a maintainer publishes a configuration file we can pass to lei > > > > The command-line might be enough, the pathname of the current > > state/config file is a bit tricky and tied to its output. > > I suppose "lei import-search" can be a command, though... > > Excellent, excellent. How well does it deal with the situation when the search > parameters change? "lei edit-search" can be used to zero the maxuid parameters; and normal v2 deduplication will prevent duplicates from showing up. It's not automatic, though; though that probably seems like a good idea to keep manual, anyways, given the step 2. below. > > > - our backend lei process uses all of lore.kernel.org sources to create and > > > continuously update a new public-inbox repository with matching search > > > results > > > > There's already some accomodations for that in LeiSavedSearch > > which can present itself as a PublicInbox::Inbox-ish object to > > PublicInbox::WWW (untested). > > > > Searching an within LSS isn't implemented, yet, but I think it's > > doable w/o extra Xapian storage. > > > > However, git object storage isn't duplicated, which is nice for > > local use (instaweb-like), but supporting clone/fetch isn't as > > natural... > > I'm thinking we need the ability to make it a real clonable repository -- > perhaps without its own xapian index? Actual git repositories aren't large, > especially if they are only used for direct git operations. Disk space is > cheap, it's the IO that's expensive. :) True, though cache overheads hurt a bit. I also wonder if lei can increase traffic to public-inbox- to reduce the need/use of "git clone". > If these are real clonable repositories, then it would be easy for people to > set up replication for just the curated content people want. Understood. Using --output v2publicinbox:... w/o --shared is totally doable. > > Perhaps supporting a v2 inbox as an lei q output destination > > is in order: > > > > lei q --output v2publicinbox:/path/to/v2 --shared SEARCH_TERMS > > > > --shared would be "git clone --shared", the new v2 inbox can > > use ~/.cache/lei/all_locals_ever.git/ as an alternate and not > > duplicate space for blobs. > > Not really worried about deduping blobs, but I'm wondering how to make it work > well when search parameters change (see above). E.g.: > > 1. we create the repo with one set of parameters > 2. maintainer then broadens it up to include something else > 3. maintainer then decides that it's now *way* too much and narrows it down again > > We don't really want step 2 to lead to a permanent ballooning of the > repository, so perhaps all query changes should force-append a dt: with the > open-ended datetime of the change? Or do you already have a way to deal with > this situation? The aforementioned maxuid prevents stuff that's too old from being seen. Otherwise, there's always "public-inbox-learn rm". > > > - we set up a mlmmj list that doesn't receive any direct mail but is only fed > > > from saved search results; people can subscribe/unsubscribe as they would > > > with any other mlmmj list > > > > > > Any particular reason this wouldn't work? > > > > Nope :) As long as all the data formats can interoperate > > (mostly RFC5322/2822). "lei convert" is nice, too :) > > Great! I believe this will help untangle the current situation with "where > should I send this kernel patch". > > I want "just send it to linux-kernel@vger.kernel.org" to be a valid option > again. Participating subsystems can then define what patches they want to see > by setting up pseudo-lists and letting participating reviewers/maintainers > subscribe to them via their preferred mail delivery mechanism. Yup, that seems easiest for new contributors.