From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-3.9 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by dcvr.yhbt.net (Postfix) with ESMTP id 402701F54E for ; Sun, 14 Aug 2022 06:54:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234426AbiHNGyP (ORCPT ); Sun, 14 Aug 2022 02:54:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230062AbiHNGyO (ORCPT ); Sun, 14 Aug 2022 02:54:14 -0400 Received: from cloud.peff.net (cloud.peff.net [104.130.231.41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D83DB4333F for ; Sat, 13 Aug 2022 23:54:13 -0700 (PDT) Received: (qmail 18620 invoked by uid 109); 14 Aug 2022 06:54:13 -0000 Received: from Unknown (HELO peff.net) (10.0.1.2) by cloud.peff.net (qpsmtpd/0.94) with ESMTP; Sun, 14 Aug 2022 06:54:13 +0000 Authentication-Results: cloud.peff.net; auth=none Received: (qmail 31643 invoked by uid 111); 14 Aug 2022 06:54:13 -0000 Received: from coredump.intra.peff.net (HELO sigill.intra.peff.net) (10.0.0.2) by peff.net (qpsmtpd/0.94) with (TLS_AES_256_GCM_SHA384 encrypted) ESMTPS; Sun, 14 Aug 2022 02:54:13 -0400 Authentication-Results: peff.net; auth=none Date: Sun, 14 Aug 2022 02:54:12 -0400 From: Jeff King To: ZheNing Hu Cc: Derrick Stolee , Junio C Hamano , ZheNing Hu via GitGitGadget , Git List , Christian Couder , =?utf-8?B?w4Z2YXIgQXJuZmrDtnLDsA==?= Bjarmason , Jeff Hostetler , Derrick Stolee Subject: Re: [PATCH] [RFC] list-objects-filter: introduce new filter sparse:buffer= Message-ID: References: <46ca40a9-2d9a-3c7c-3272-938003f4967a@github.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org On Fri, Aug 12, 2022 at 11:49:06PM +0800, ZheNing Hu wrote: > > ...I agree with this. It is nice to put more power in the hands of the > > clients, but we have to balance that with other issues like server > > resource use. The approach so far has been to implement the simplest and > > most efficient operations at the client-server level, and then have the > > client build local features on top of that. So in this case, probably > > requesting that _no_ trees are sent in the initial clone, and then > > faulting them in as the client explores the tree using its own local > > sparse definition. And I think that mostly works now. > > Agree. But we have to fetch these blobs one by one after partial clone, > why not reduce some extra network overhead If we can get those blobs > that are *most* needed in the first partial clone, right? Ideally we wouldn't be getting them one-by-one, but rather batching them. I'm not sure of the current state of the code. But we should at least be able to batch by tree depth. You're right that we'll never be able to be as efficient as a server to whom we can say "I care about these paths", though. -Peff