git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: Re: [PATCH 7/9] ls-refs: ignore very long ref-prefix counts
Date: Tue, 14 Sep 2021 13:23:30 -0400	[thread overview]
Message-ID: <YUDakl7dcJpfve0T@coredump.intra.peff.net> (raw)
In-Reply-To: <YUDBokYvEBnzwsIN@coredump.intra.peff.net>

On Tue, Sep 14, 2021 at 11:37:06AM -0400, Jeff King wrote:

> The limit here is fairly arbitrary, and probably much higher than anyone
> would need in practice. It might be worth limiting it further, if only
> because we check it linearly (so with "m" local refs and "n" patterns,
> we do "m * n" string comparisons). But if we care about optimizing this,
> an even better solution may be a more advanced data structure anyway.

The limit I picked is 65536, because it seemed round and high. But note
that somebody can put up to almost-64k in a single ref-prefix line,
which means ultimately you can allocate 4GB. I do wonder if dropping
this to something like 1024 might be reasonable.

In practice I'd expect it to be a handful in most cases (refs/heads/*,
refs/tags/*, HEAD). But if you do something like:

  git fetch $remote 1 2 3 4 5 6 7 ...

then we'll prefix-expand those names with the usual lookup rules into
refs/1, refs/heads/1, refs/2, refs/heads/2, and so on.

At some point it becomes silly and works counter to the purpose of the
optimization (you send more prefix constraints than the actual ref
advertisement, not to mention that client bandwidth may not be
symmetric). I'm not sure what we want to declare as a reasonable limit.

And this is just about protecting the server; probably it makes sense
for the client to realize it's going to send a ridiculous number of
prefixes and just skip the feature entirely (since that's what actually
saves the bandwidth).

-Peff

  parent reply	other threads:[~2021-09-14 17:23 UTC|newest]

Thread overview: 77+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-14 15:29 [PATCH 0/9] reducing memory allocations for v2 servers Jeff King
2021-09-14 15:30 ` [PATCH 1/9] serve: rename is_command() to parse_command() Jeff King
2021-09-14 15:30 ` [PATCH 2/9] serve: return capability "value" from get_capability() Jeff King
2021-09-14 15:31 ` [PATCH 3/9] serve: add "receive" method for v2 capabilities table Jeff King
2021-09-14 15:31 ` [PATCH 4/9] serve: provide "receive" function for object-format capability Jeff King
2021-09-14 18:59   ` Martin Ågren
2021-09-14 15:33 ` [PATCH 5/9] serve: provide "receive" function for session-id capability Jeff King
2021-09-14 16:55   ` Taylor Blau
2021-09-14 17:06     ` Jeff King
2021-09-14 17:12       ` Taylor Blau
2021-09-14 19:02   ` Martin Ågren
2021-09-14 19:14     ` Jeff King
2021-09-14 15:33 ` [PATCH 6/9] serve: drop "keys" strvec Jeff King
2021-09-14 16:59   ` Taylor Blau
2021-09-14 17:16     ` Jeff King
2021-09-14 15:37 ` [PATCH 7/9] ls-refs: ignore very long ref-prefix counts Jeff King
2021-09-14 17:18   ` Taylor Blau
2021-09-14 17:27     ` Jeff King
2021-09-14 17:23   ` Jeff King [this message]
2021-09-14 19:06   ` Martin Ågren
2021-09-14 19:22     ` Jeff King
2021-09-14 22:09   ` Jeff King
2021-09-14 22:11     ` Taylor Blau
2021-09-14 22:15       ` Jeff King
2021-09-14 15:37 ` [PATCH 8/9] serve: reject bogus v2 "command=ls-refs=foo" Jeff King
2021-09-14 17:21   ` Taylor Blau
2021-09-14 15:37 ` [PATCH 9/9] serve: reject commands used as capabilities Jeff King
2021-09-14 17:30 ` [PATCH 0/9] reducing memory allocations for v2 servers Taylor Blau
2021-09-14 18:00 ` Junio C Hamano
2021-09-14 18:38   ` Jeff King
2021-09-14 23:51 ` [PATCH v2 0/11] limit " Jeff King
2021-09-14 23:51   ` [PATCH v2 01/11] serve: rename is_command() to parse_command() Jeff King
2021-09-14 23:51   ` [PATCH v2 02/11] serve: return capability "value" from get_capability() Jeff King
2021-09-14 23:51   ` [PATCH v2 03/11] serve: add "receive" method for v2 capabilities table Jeff King
2021-09-15  0:31     ` Ævar Arnfjörð Bjarmason
2021-09-15 16:35       ` Jeff King
2021-09-15 16:41     ` Junio C Hamano
2021-09-15 16:57       ` Jeff King
2021-09-14 23:51   ` [PATCH v2 04/11] serve: provide "receive" function for object-format capability Jeff King
2021-09-15 16:54     ` Junio C Hamano
2021-09-14 23:51   ` [PATCH v2 05/11] serve: provide "receive" function for session-id capability Jeff King
2021-09-15 16:56     ` Junio C Hamano
2021-09-14 23:51   ` [PATCH v2 06/11] serve: drop "keys" strvec Jeff King
2021-09-15 17:01     ` Junio C Hamano
2021-09-14 23:51   ` [PATCH v2 07/11] ls-refs: ignore very long ref-prefix counts Jeff King
2021-09-15  4:16     ` Taylor Blau
2021-09-15 16:39       ` Jeff King
2021-09-15  5:00     ` Eric Sunshine
2021-09-15 16:40       ` Jeff King
2021-09-14 23:52   ` [PATCH v2 08/11] docs/protocol-v2: clarify some ls-refs ref-prefix details Jeff King
2021-09-14 23:52   ` [PATCH v2 09/11] serve: reject bogus v2 "command=ls-refs=foo" Jeff King
2021-09-15  0:27     ` Ævar Arnfjörð Bjarmason
2021-09-15 16:28       ` Jeff King
2021-09-15  5:09     ` Eric Sunshine
2021-09-15 16:32       ` Jeff King
2021-09-15 17:33     ` Junio C Hamano
2021-09-15 17:39       ` Jeff King
2021-09-14 23:52   ` [PATCH v2 10/11] serve: reject commands used as capabilities Jeff King
2021-09-14 23:54   ` [PATCH v2 11/11] ls-refs: reject unknown arguments Jeff King
2021-09-15  0:09     ` Ævar Arnfjörð Bjarmason
2021-09-15 16:25       ` Jeff King
2021-09-15  4:17   ` [PATCH v2 0/11] limit memory allocations for v2 servers Taylor Blau
2021-09-15 18:33   ` Jeff King
2021-09-15 18:34     ` [PATCH v3 " Jeff King
2021-09-15 18:35       ` [PATCH v3 01/11] serve: rename is_command() to parse_command() Jeff King
2021-09-15 18:35       ` [PATCH v3 02/11] serve: return capability "value" from get_capability() Jeff King
2021-09-15 18:35       ` [PATCH v3 03/11] serve: add "receive" method for v2 capabilities table Jeff King
2021-09-15 18:35       ` [PATCH v3 04/11] serve: provide "receive" function for object-format capability Jeff King
2021-09-15 18:35       ` [PATCH v3 05/11] serve: provide "receive" function for session-id capability Jeff King
2021-09-15 18:35       ` [PATCH v3 06/11] serve: drop "keys" strvec Jeff King
2021-09-15 18:35       ` [PATCH v3 07/11] ls-refs: ignore very long ref-prefix counts Jeff King
2021-09-15 18:35       ` [PATCH v3 08/11] docs/protocol-v2: clarify some ls-refs ref-prefix details Jeff King
2021-09-15 18:36       ` [PATCH v3 09/11] serve: reject bogus v2 "command=ls-refs=foo" Jeff King
2021-09-15 18:36       ` [PATCH v3 10/11] serve: reject commands used as capabilities Jeff King
2021-09-15 18:36       ` [PATCH v3 11/11] ls-refs: reject unknown arguments Jeff King
2021-09-15  0:25 ` [PATCH 0/9] reducing memory allocations for v2 servers Ævar Arnfjörð Bjarmason
2021-09-15 16:41   ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YUDakl7dcJpfve0T@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=avarab@gmail.com \
    --cc=git@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).