git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Teng Long <dyroneteng@gmail.com>
To: peff@peff.net
Cc: avarab@gmail.com, derrickstolee@github.com, dyroneteng@gmail.com,
	git@vger.kernel.org, gitster@pobox.com, me@ttaylorr.com,
	tenglong.tl@alibaba-inc.com
Subject: Re: [PATCH v3 0/2] pack-bitmap.c: avoid exposing absolute paths
Date: Thu, 17 Nov 2022 22:19:33 +0800	[thread overview]
Message-ID: <20221117141933.73790-1-tenglong.tl@alibaba-inc.com> (raw)
In-Reply-To: <Y3K//kO3fxD7Pl3/@coredump.intra.peff.net>

> Me too. If we wanted to go further, there are two obvious next steps.
>
> One, we can break out of the bitmap loop early if we're not tracing:

> diff --git a/pack-bitmap.c b/pack-bitmap.c
> index aaa2d9a104..3b6c2f804a 100644
> --- a/pack-bitmap.c
> +++ b/pack-bitmap.c
> @@ -527,8 +527,15 @@ static int open_pack_bitmap(struct repository *r,
>  	assert(!bitmap_git->map);
>
>  	for (p = get_all_packs(r); p; p = p->next) {
> -		if (open_pack_bitmap_1(bitmap_git, p) == 0)
> +		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
>  			ret = 0;
> +			/*
> +			 * The only reason to keep looking is to report
> +			 * duplicates.
> +			 */
> +			if (!trace2_is_enabled())
> +				break;
> +		}
>  	}
>
>  	return ret;
>
> I doubt this buys us much in practice. After patch 2, looking for extra
> bitmaps is much cheaper. It's one open() call per pack (which will
> return ENOENT normally) looking for a bitmap. And while it's only 2
> lines of code, it does increase coupling of assumptions between the two
> functions. So maybe not worth doing. I dunno.

I agree and I think it's reasonable.

So If I bring it into the patch how about the commit message:

Author: Jeff King <peff@peff.net>
Date:   Thu Nov 17 19:55:23 2022 +0800

    pack-bitmap.c: break out of the bitmap loop early if not tracing

    When we successfully open a bitmap, we will continue to try to open
    other packs, and when trace2 is enabled, we will report any subsequent
    bitmap ignored information in the log. So when we find that trace2 is
    not enabled, we can actually terminate the loop early.

    Signed-off-by: Jeff King <peff@peff.net>
    Signed-off-by: Teng Long <dyroneteng@gmail.com>

> And two, we could complain when there is both a midx and a pack bitmap,
> like so:
>
> diff --git a/pack-bitmap.c b/pack-bitmap.c
> index 3b6c2f804a..44a80ed8f2 100644
> --- a/pack-bitmap.c
> +++ b/pack-bitmap.c
> @@ -524,8 +524,6 @@ static int open_pack_bitmap(struct repository *r,
>  	struct packed_git *p;
>  	int ret = -1;
>
> -	assert(!bitmap_git->map);
> -
>  	for (p = get_all_packs(r); p; p = p->next) {
>  		if (open_pack_bitmap_1(bitmap_git, p) == 0) {
>  			ret = 0;
> @@ -559,11 +557,20 @@ static int open_midx_bitmap(struct repository *r,
>  static int open_bitmap(struct repository *r,
>  		       struct bitmap_index *bitmap_git)
>  {
> +	int found = 0;
> +
>  	assert(!bitmap_git->map);
>
> -	if (!open_midx_bitmap(r, bitmap_git))
> -		return 0;
> -	return open_pack_bitmap(r, bitmap_git);
> +	found = !open_midx_bitmap(r, bitmap_git);
> +
> +	/*
> +	 * these will all be skipped if we opened a midx bitmap; but run it
> +	 * anyway if tracing is enabled to report the duplicates
> +	 */
> +	if (!found || trace2_is_enabled())
> +		found |= !open_pack_bitmap(r, bitmap_git);
> +
> +	return found ? 0 : -1;
>  }
>
>  struct bitmap_index *prepare_bitmap_git(struct repository *r)
>
> But I'm not sure if that is even something we want. I know we retained
> pack bitmaps as a quick recovery mechanism while test-deploying midx
> bitmaps. OTOH, now that the feature is stable, I doubt anybody else
> would ever do that.
>
> It also suffers from the same coupling issues as the other.
>
> So I don't know that either is worth pursuing (hence this message and
> not fully prepared patches), but these are just the two leftover things
> I noticed from the series, so I wanted to record them for posterity.

Since this is an internal mechanism, and we are doing reminders in trace2, so
the diff looks good to me. How about the commit message if I need to take it:

Author: Jeff King <peff@peff.net>
Date:   Thu Nov 17 20:25:18 2022 +0800

    pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is found

    We retained pack bitmaps as a quick recovery mechanism while
    test-deploying midx bitmaps. This is an internal mechanism, and we
    want to expose this rule externally through trace2 so that end users,
    repo-maintainers, and debuggers know what is happening in the process.

    Signed-off-by: Jeff King <peff@peff.net>
    Signed-off-by: Teng Long <dyroneteng@gmail.com>

Thank you Peff for providing such detailed suggestions for improvement.

  reply	other threads:[~2022-11-17 14:19 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-26  7:09 [PATCH 0/1] pack-bitmap.c: avoid exposing absolute paths Teng Long
2022-08-26  7:09 ` [PATCH 1/1] " Teng Long
2022-08-26 16:34 ` [PATCH 0/1] " Junio C Hamano
2022-08-29  2:48   ` Teng Long
2022-10-26 21:42     ` Taylor Blau
2022-10-26 23:19       ` Ævar Arnfjörð Bjarmason
2022-10-31 13:20         ` Teng Long
2022-10-27 20:45       ` Jeff King
2022-10-30 18:42         ` Taylor Blau
2022-10-31 12:22           ` [PATCH 0/1] pack-bitmap.c: avoid exposing absolute paths Taylor Blau <me@ttaylorr.com> writes: Teng Long
2022-11-02  5:37         ` [PATCH 0/1] pack-bitmap.c: avoid exposing absolute paths Teng Long
2022-11-02  7:54           ` Jeff King
2022-11-02 13:52             ` Teng Long
2022-10-31 13:13       ` Teng Long
2022-11-03  1:00         ` Taylor Blau
2022-11-02  9:20 ` Ævar Arnfjörð Bjarmason
2022-11-02 13:04   ` Teng Long
2022-11-02 12:56 ` [PATCH v2 " Teng Long
2022-11-02 12:56   ` [PATCH v2 1/1] " Teng Long
2022-11-03  1:16     ` Taylor Blau
2022-11-03  9:35       ` Teng Long
2022-11-05  0:35         ` Taylor Blau
2022-11-03  1:21   ` [PATCH v2 0/1] " Taylor Blau
2022-11-03  8:42     ` Teng Long
2022-11-04  3:17   ` [PATCH v3 0/2] " Teng Long
2022-11-04  3:17     ` [PATCH v3 1/2] " Teng Long
2022-11-04 22:11       ` Taylor Blau
2022-11-04  3:17     ` [PATCH v3 2/2] pack-bitmap.c: remove unnecessary "open_pack_index()" calls Teng Long
2022-11-04 22:09       ` Taylor Blau
2022-11-04 22:13     ` [PATCH v3 0/2] pack-bitmap.c: avoid exposing absolute paths Taylor Blau
2022-11-10  7:10     ` Teng Long
2022-11-10  7:10       ` [PATCH v3 1/2] pack-bitmap.c: remove unnecessary "open_pack_index()" calls Teng Long
2022-11-14 22:03         ` Jeff King
2022-11-14 22:14           ` Taylor Blau
2022-11-14 22:31             ` Jeff King
2022-11-14 22:50               ` Taylor Blau
2022-11-10  7:10       ` [PATCH v3 2/2] pack-bitmap.c: avoid exposing absolute paths Teng Long
2022-11-11 22:26       ` [PATCH v3 0/2] " Taylor Blau
2022-11-14 22:23         ` Jeff King
2022-11-17 14:19           ` Teng Long [this message]
2022-11-17 15:03             ` Jeff King
2022-11-17 21:57               ` Taylor Blau
2022-11-21  3:27                 ` Teng Long
2022-11-21 12:16     ` [PATCH v4 0/4] " Teng Long
2022-11-21 12:16       ` [PATCH v4 1/4] pack-bitmap.c: remove unnecessary "open_pack_index()" calls Teng Long
2022-11-21 12:16       ` [PATCH v4 2/4] pack-bitmap.c: avoid exposing absolute paths Teng Long
2022-11-21 12:16       ` [PATCH v4 3/4] pack-bitmap.c: break out of the bitmap loop early if not tracing Teng Long
2022-11-21 23:27         ` Junio C Hamano
2022-11-28 13:09           ` Teng Long
2022-11-21 12:16       ` [PATCH v4 4/4] pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is found Teng Long
2022-11-21 19:09         ` Jeff King
2022-11-21 23:29           ` Junio C Hamano
2022-11-28 12:29             ` Teng Long
2022-11-28 12:37           ` Teng Long
2022-11-29  1:27             ` Jeff King
2022-11-29 13:14               ` Teng Long
2022-11-21 19:04       ` [PATCH v4 0/4] pack-bitmap.c: avoid exposing absolute paths Jeff King
2022-11-28 12:48         ` Teng Long
2022-11-28 14:09       ` [PATCH v5 " Teng Long
2022-11-28 14:09         ` [PATCH v5 1/4] pack-bitmap.c: remove unnecessary "open_pack_index()" calls Teng Long
2022-11-28 14:09         ` [PATCH v5 2/4] pack-bitmap.c: avoid exposing absolute paths Teng Long
2022-11-28 14:09         ` [PATCH v5 3/4] pack-bitmap.c: break out of the bitmap loop early if not tracing Teng Long
2022-11-28 23:26           ` Taylor Blau
2022-11-29 13:17             ` Teng Long
2022-11-28 14:09         ` [PATCH v5 4/4] pack-bitmap.c: trace bitmap ignore logs when midx-bitmap is found Teng Long
2022-11-28 23:30         ` [PATCH v5 0/4] pack-bitmap.c: avoid exposing absolute paths Taylor Blau
2022-11-29 13:21           ` Teng Long

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221117141933.73790-1-tenglong.tl@alibaba-inc.com \
    --to=dyroneteng@gmail.com \
    --cc=avarab@gmail.com \
    --cc=derrickstolee@github.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=me@ttaylorr.com \
    --cc=peff@peff.net \
    --cc=tenglong.tl@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).