From: Taylor Blau <me@ttaylorr.com>
To: Teng Long <dyroneteng@gmail.com>
Cc: avarab@gmail.com, derrickstolee@github.com, git@vger.kernel.org,
gitster@pobox.com, tenglong.tl@alibaba-inc.com
Subject: Re: [PATCH v3 5/5] bitmap: add trace2 outputs during open "bitmap" file
Date: Mon, 13 Jun 2022 21:40:10 -0400 [thread overview]
Message-ID: <Yqfm+rsP5DWyj95L@nand.local> (raw)
In-Reply-To: <e118758d1dada378d65d58579cc1372fa547d720.1655018322.git.dyroneteng@gmail.com>
On Sun, Jun 12, 2022 at 03:44:20PM +0800, Teng Long wrote:
> It's supported for a repo to use bitmap in both "NORMAL" bitmap way
I'm nitpicking, but I usually say "single-pack bitmap" or "multi-pack
(MIDX) bitmap" when distinguishing between the two.
> or a MIDX (multi-pack-index) bitmap. Either of two bitmap kinds can
> exist in the repository, or both can be stored but let the config
> controls which kind of bitmap is used (like "core.multipackIndex",
> etc.). Because of this, sometimes the bitmap debug path is not
> obvious enough, for example, when executing:
>
> $git rev-list --test-bitmap HEAD
> fatal: failed to load bitmap indexes
Odd spacing, and there should be a single space character separating "$"
from "git" (like "$ git").
While I'm thinking about it: is the error message here up-to-date with
the changes made by the previous patch?
> If we see the output like this, It's not sure for us to know
> what's happened concretely, because the cause should be :
>
> 1. Neither normal nor MIDX bitmap exists.
> 2. Only MIDX bitmap exists but core.multipackIndex="false".
> 3. Config core.multipackIndex set to "true" but MIDX bitmap is
> corrupted.
> 4. Config core.multipackIndex set to "true" and no MIDX bitmap
> exists but normal bitmap file is corrupted.
> ....
>
> These are some of the scenarios I briefly tested, but maybe there are
> others (some scenarios is produced manually like "corrupted bitmap file",
> but it's not represent it's an existed bug.).
This could probably be trimmed down for brevity, but I don't feel
strongly about it. If you wanted to make your commit message a tad
shorter, perhaps something like:
When a user sees output like this, it's unclear which kind(s) of
.bitmap exist, and which were read. For example, it's possible a MIDX
bitmap exists, but was not read (e.g., because
core.multiPackIndex=false), among many other scenarios.
would suffice.
> Therefore, we added some TRACE2 code so that when we read the bitmap
s/TRACE2/trace2
> we can be more clear about the decision path, such as whether it is
> working on MIDX or NORMAL bitmap at present, or the related config is
s/NORMAL/pack
> enabled or not. This may help with logging, user troubleshooting, and
> development debugging.
>
> Here are some brief output examples on different scenarios when
> executing:
>
> $GIT_TRACE2_PERF=1 git rev-list --test-bitmap HEAD
s/$GIT/$ GIT
> Scenario 1: core.multipackIndex [false], midx bitmap exists [Y],
> normal bitmap exists [N]
The output here is quite wide, and I wonder if this whole section could
be shortened. For example, scenario 2 is arguably more interesting than
scenario 1 (I think readers would reasonably infer what happens in
scenario 1 by reading what happens in scenario 2).
> 19:21:56.580349 repo-settings.c:11 | d0 | main | data | r1 | 0.000827 | 0.000827 | config | core.multipackindex:false
> 19:21:56.580356 repo-settings.c:11 | d0 | main | data | r1 | 0.000834 | 0.000834 | config | index.sparse:false
> 19:21:56.580706 pack-bitmap.c:525 | d0 | main | region_enter | r1 | 0.001183 | | pack-bitmap | label:open_bitmap
> 19:21:56.580719 pack-bitmap.c:386 | d0 | main | data | r1 | 0.001196 | 0.000013 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap
> 19:21:56.580729 pack-bitmap.c:530 | d0 | main | region_leave | r1 | 0.001207 | 0.000024 | pack-bitmap | label:open_bitmap
> 19:21:56.580737 usage.c:60 | d0 | main | error | | | | | failed to load bitmap indexes
> fatal: failed to load bitmap indexes
> 19:21:56.580746 usage.c:74 | d0 | main | exit | | 0.001224 | | | code:128
> 19:21:56.580754 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001232 | | | code:128
>
> Scenario 2: core.multipackIndex [false], midx bitmap exists [Y],
> normal bitmap exists [Y]
>
> 19:23:44.692384 repo-settings.c:11 | d0 | main | data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false
> 19:23:44.692755 pack-bitmap.c:525 | d0 | main | region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap
> 19:23:44.692768 pack-bitmap.c:386 | d0 | main | data | r0 | 0.001149 | 0.000014 | bitmap | ..path:/home/tenglong.tl/test/dyrone_bitmap/.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap
> 19:23:44.692790 pack-bitmap.c:530 | d0 | main | region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap
> Bitmap v1 test (1 entries loaded)
> Found bitmap for d864fefa87415d6cd289c72aa9ffd45b4a8ffd84. 64 bits / 11030517 checksum
> 19:23:44.693119 progress.c:268 | d0 | main | region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries
> Verifying bitmap entries: 100% (3/3), done.
> 19:23:44.693208 progress.c:339 | d0 | main | data | r0 | 0.001589 | 0.000089 | progress | ..total_objects:3
> 19:23:44.693216 progress.c:346 | d0 | main | region_leave | r0 | 0.001597 | 0.000097 | progress | label:Verifying bitmap entries
> OK!
> 19:23:44.693234 git.c:718 | d0 | main | exit | | 0.001615 | | | code:0
> 19:23:44.693244 trace2/tr2_tgt_perf.c:215 | d0 | main | atexit | | 0.001625 | | | code:0
And scenario 2 could be cleaned up by just showing a few of the columns
from the trace2 output. Perhaps along the lines of:
> | data | r0 | 0.000765 | 0.000765 | config | core.multipackindex:false
> | region_enter | r0 | 0.001135 | | pack-bitmap | label:open_bitmap
> | data | r0 | 0.001149 | 0.000014 | bitmap | ..path:.git/objects/pack/pack-e9eb18e6a423057f4424a762069e13804a75d01e.bitmap
> | region_leave | r0 | 0.001171 | 0.000036 | pack-bitmap | label:open_bitmap
> | region_enter | r0 | 0.001500 | | progress | label:Verifying bitmap entries
Reading the below scenarios, I think just showing this example is more
than sufficient for illustrating your point.
> Signed-off-by: Teng Long <dyroneteng@gmail.com>
> ---
> pack-bitmap.c | 27 +++++++++++++++++++++------
> repo-settings.c | 1 +
> 2 files changed, 22 insertions(+), 6 deletions(-)
>
> diff --git a/pack-bitmap.c b/pack-bitmap.c
> index 5654eb7b8d..ced5993560 100644
> --- a/pack-bitmap.c
> +++ b/pack-bitmap.c
> @@ -312,9 +312,11 @@ char *pack_bitmap_filename(struct packed_git *p)
> static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,
> struct multi_pack_index *midx)
> {
> + int fd;
> struct stat st;
> char *bitmap_name = midx_bitmap_filename(midx);
> - int fd = git_open(bitmap_name);
> + trace2_data_string("midx", the_repository, "path", bitmap_name);
> + fd = git_open(bitmap_name);
>
> free(bitmap_name);
>
> @@ -343,12 +345,19 @@ static int open_midx_bitmap_1(struct bitmap_index *bitmap_git,
> MAP_PRIVATE, fd, 0);
> close(fd);
>
> - if (load_bitmap_header(bitmap_git) < 0)
> + if (load_bitmap_header(bitmap_git) < 0) {
> + trace2_data_string("midx", the_repository, "load bitmap header",
> + "failed");
I wonder, why don't we show these errors to the user? Should we
introduce a "ret" variable and set it to (for e.g.)
ret = error(_("failed to load bitmap header"));
or something?
> struct bitmap_index *prepare_bitmap_git(struct repository *r)
> diff --git a/repo-settings.c b/repo-settings.c
> index b4fbd16cdc..5bc7a97a6d 100644
> --- a/repo-settings.c
> +++ b/repo-settings.c
> @@ -8,6 +8,7 @@ static void repo_cfg_bool(struct repository *r, const char *key, int *dest,
> {
> if (repo_config_get_bool(r, key, dest))
> *dest = def;
> + trace2_data_string("config", r, key, *dest ? "true" : "false");
I'm not sure we want to dump the whole repository configuration into
trace2 output. Is there a more convenient place to log any important
value(s) _after_ they have been read?
Thanks,
Taylor
next prev parent reply other threads:[~2022-06-14 1:40 UTC|newest]
Thread overview: 128+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-24 11:43 [PATCH v1 0/3] trace2 output for bitmap decision path Teng Long
2022-03-24 11:43 ` [PATCH v1 1/3] pack-bitmap.c: use "ret" in "open_midx_bitmap()" Teng Long
2022-03-24 19:11 ` Taylor Blau
2022-03-28 7:59 ` [PATCH v1 1/3] pack-bitmap.c: use "ret" in "open_midx_bitmap() Teng Long
2022-03-30 2:39 ` Taylor Blau
2022-03-24 11:44 ` [PATCH v1 2/3] pack-bitmap.c: add "break" statement in "open_pack_bitmap()" Teng Long
2022-03-24 18:40 ` Junio C Hamano
2022-03-24 19:06 ` Taylor Blau
2022-03-24 19:03 ` Taylor Blau
2022-03-29 2:49 ` Teng Long
2022-03-30 2:55 ` Taylor Blau
2022-03-30 7:32 ` Teng Long Teng Long
2022-03-30 13:17 ` Ævar Arnfjörð Bjarmason
2022-03-24 11:44 ` [PATCH v1 3/3] bitmap: add trace outputs during open "bitmap" file Teng Long
2022-03-24 18:42 ` Junio C Hamano
2022-03-24 13:22 ` [PATCH v1 0/3] trace2 output for bitmap decision path Ævar Arnfjörð Bjarmason
2022-03-29 7:38 ` Teng Long Teng Long
2022-03-29 8:54 ` Ævar Arnfjörð Bjarmason
2022-04-21 13:26 ` [PATCH v2 0/5] trace2 output for bitmap decision path Teng Long
2022-04-21 13:26 ` [PATCH v2 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found Teng Long
2022-05-11 21:31 ` Taylor Blau
2022-04-21 13:26 ` [PATCH v2 2/5] pack-bitmap.c: rename "idx_name" to "bitmap_name" Teng Long
2022-05-11 21:31 ` Taylor Blau
2022-04-21 13:26 ` [PATCH v2 3/5] pack-bitmap.c: make warnings more detailed when opening bitmap Teng Long
2022-04-21 17:25 ` Taylor Blau
2022-05-06 9:08 ` Teng Long
2022-04-21 13:26 ` [PATCH v2 4/5] bitmap: add trace2 outputs during open "bitmap" file Teng Long
2022-04-21 15:51 ` Ævar Arnfjörð Bjarmason
2022-05-06 11:27 ` Teng Long
2022-05-06 11:53 ` Teng Long
2022-04-21 16:32 ` Jeff Hostetler
2022-05-06 12:43 ` Teng Long
2022-05-10 20:47 ` Jeff Hostetler
2022-04-21 13:26 ` [PATCH v2 5/5] pack-bitmap.c: using error() instead of silently returning -1 Teng Long
2022-04-21 15:41 ` Ævar Arnfjörð Bjarmason
2022-05-06 12:55 ` Teng Long
2022-06-12 7:44 ` [PATCH v3 0/5] Teng Long
2022-06-12 7:44 ` [PATCH v3 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found Teng Long
2022-06-12 7:44 ` [PATCH v3 2/5] pack-bitmap.c: rename "idx_name" to "bitmap_name" Teng Long
2022-06-12 7:44 ` [PATCH v3 3/5] pack-bitmap.c: make warnings support i18N when opening bitmap Teng Long
2022-06-12 7:44 ` [PATCH v3 4/5] pack-bitmap.c: using error() instead of silently returning -1 Teng Long
2022-06-14 1:15 ` Taylor Blau
2022-06-20 13:17 ` Teng Long
2022-06-12 7:44 ` [PATCH v3 5/5] bitmap: add trace2 outputs during open "bitmap" file Teng Long
2022-06-13 20:59 ` Junio C Hamano
2022-06-20 13:32 ` Teng Long
2022-06-14 1:40 ` Taylor Blau [this message]
2022-06-21 6:58 ` Teng Long
2022-06-22 12:51 ` Jeff Hostetler
2022-06-23 9:38 ` Teng Long
2022-06-23 15:14 ` Jeff Hostetler
2022-06-24 8:27 ` [PATCH v3 5/5] bitmap: add trace2 outputs during open "bitmap" Teng Long
2022-06-21 13:25 ` [PATCH v3 0/5] trace2 output for bitmap decision path Teng Long
2022-06-21 13:25 ` [PATCH v3 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found Teng Long
2022-06-21 13:25 ` [PATCH v3 2/5] pack-bitmap.c: rename "idx_name" to "bitmap_name" Teng Long
2022-06-21 13:25 ` [PATCH v3 3/5] pack-bitmap.c: make warnings support i18N when opening bitmap Teng Long
2022-06-21 13:25 ` [PATCH v3 4/5] pack-bitmap.c: using error() instead of silently returning -1 Teng Long
2022-06-21 13:25 ` [PATCH v3 5/5] bitmap: add trace2 outputs during open "bitmap" file Teng Long
2022-06-22 13:04 ` Jeff Hostetler
2022-06-22 15:12 ` Junio C Hamano
2022-06-28 8:17 ` [PATCH v5 0/5] tr2: avoid to print "interesting" config repeatedly Teng Long
2022-06-28 8:17 ` [PATCH v5 1/5] pack-bitmap.c: continue looping when first MIDX bitmap is found Teng Long
2022-06-28 8:17 ` [PATCH v5 2/5] pack-bitmap.c: rename "idx_name" to "bitmap_name" Teng Long
2022-06-28 8:17 ` [PATCH v5 3/5] pack-bitmap.c: using error() instead of silently returning -1 Teng Long
2022-06-28 18:04 ` Junio C Hamano
2022-07-05 9:04 ` Teng Long
2022-07-05 18:23 ` Junio C Hamano
2022-06-28 8:17 ` [PATCH v5 4/5] pack-bitmap.c: retrieve missing i18n translations Teng Long
2022-06-28 8:58 ` Ævar Arnfjörð Bjarmason
2022-06-28 17:28 ` Eric Sunshine
2022-07-06 14:19 ` Teng Long
2022-07-06 14:06 ` Teng Long
2022-06-28 18:07 ` Junio C Hamano
2022-07-07 11:59 ` Teng Long
2022-07-07 16:45 ` Junio C Hamano
2022-07-11 11:04 ` Teng Long
2022-06-28 8:17 ` [PATCH v5 5/5] tr2: avoid to print "interesting" config repeatedly Teng Long
2022-06-28 9:13 ` Ævar Arnfjörð Bjarmason
2022-06-28 18:12 ` Junio C Hamano
2022-07-01 14:31 ` Jeff Hostetler
2022-07-11 4:11 ` Teng Long
2022-07-11 3:51 ` Teng Long
2022-07-11 12:43 ` [PATCH v6 0/7] trace2: dump scope when print "interesting" config Teng Long
2022-07-11 12:43 ` [PATCH v6 1/7] clean: fixed issues related to text output format Teng Long
2022-07-11 21:08 ` Junio C Hamano
2022-07-13 11:44 ` Teng Long
2022-07-11 12:43 ` [PATCH v6 2/7] pack-bitmap.c: mark more strings for translations Teng Long
2022-07-11 12:43 ` [PATCH v6 3/7] pack-bitmap.c: rename "idx_name" to "bitmap_name" Teng Long
2022-07-11 12:44 ` [PATCH v6 4/7] pack-bitmap.c: don't ignore ENOENT silently Teng Long
2022-07-11 14:38 ` Ævar Arnfjörð Bjarmason
2022-07-13 14:14 ` Teng Long
2022-07-11 21:22 ` Junio C Hamano
2022-07-14 15:25 ` Teng Long
2022-07-11 12:44 ` [PATCH v6 5/7] pack-bitmap.c: using error() instead of silently returning -1 Teng Long
2022-07-11 14:53 ` Ævar Arnfjörð Bjarmason
2022-07-15 2:34 ` Teng Long
2022-07-11 12:44 ` [PATCH v6 6/7] pack-bitmap.c: continue looping when first MIDX bitmap is found Teng Long
2022-07-11 12:44 ` [PATCH v6 7/7] tr2: dump names if config exist in multiple scopes Teng Long
2022-07-11 14:40 ` Ævar Arnfjörð Bjarmason
2022-07-11 19:19 ` Jeff Hostetler
2022-07-11 14:59 ` [PATCH v6 0/7] trace2: dump scope when print "interesting" config Ævar Arnfjörð Bjarmason
2022-07-18 8:36 ` Teng Long
2022-07-18 16:45 ` [PATCH v7 " Teng Long
2022-07-18 16:46 ` [PATCH v7 1/7] pack-bitmap.c: fix formatting of error messages Teng Long
2022-07-18 16:46 ` [PATCH v7 2/7] pack-bitmap.c: mark more strings for translations Teng Long
2022-07-18 16:46 ` [PATCH v7 3/7] pack-bitmap.c: rename "idx_name" to "bitmap_name" Teng Long
2022-07-18 16:46 ` [PATCH v7 4/7] pack-bitmap.c: do not ignore error when opening a bitmap file Teng Long
2022-07-18 16:46 ` [PATCH v7 5/7] pack-bitmap.c: using error() instead of silently returning -1 Teng Long
2022-07-18 16:46 ` [PATCH v7 6/7] pack-bitmap.c: continue looping when first MIDX bitmap is found Teng Long
2022-07-18 16:46 ` [PATCH v7 7/7] tr2: dump names if config exist in multiple scopes Teng Long
2022-07-18 20:13 ` Jeff Hostetler
2022-07-19 7:40 ` tenglong.tl
2022-07-19 21:03 ` Junio C Hamano
2022-07-20 12:48 ` tenglong.tl
2022-07-18 18:57 ` [PATCH v7 0/7] trace2: dump scope when print "interesting" config Junio C Hamano
2022-07-18 19:07 ` Ævar Arnfjörð Bjarmason
2022-07-19 11:26 ` tenglong.tl
2022-07-19 11:42 ` Ævar Arnfjörð Bjarmason
2022-07-19 12:34 ` tenglong.tl
2022-07-21 9:05 ` [PATCH v8 0/6] pack-bitmap.c: optimize error messages tenglong.tl
2022-07-21 9:05 ` [PATCH v8 1/6] pack-bitmap.c: fix formatting of " tenglong.tl
2022-07-21 9:05 ` [PATCH v8 2/6] pack-bitmap.c: mark more strings for translations tenglong.tl
2022-07-21 9:05 ` [PATCH v8 3/6] pack-bitmap.c: rename "idx_name" to "bitmap_name" tenglong.tl
2022-07-21 9:05 ` [PATCH v8 4/6] pack-bitmap.c: do not ignore error when opening a bitmap file tenglong.tl
2022-07-21 9:05 ` [PATCH v8 5/6] pack-bitmap.c: using error() instead of silently returning -1 tenglong.tl
2022-07-21 9:05 ` [PATCH v8 6/6] pack-bitmap.c: continue looping when first MIDX bitmap is found tenglong.tl
2022-07-21 23:01 ` [PATCH v8 0/6] pack-bitmap.c: optimize error messages Junio C Hamano
2022-07-22 6:17 ` tenglong.tl
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Yqfm+rsP5DWyj95L@nand.local \
--to=me@ttaylorr.com \
--cc=avarab@gmail.com \
--cc=derrickstolee@github.com \
--cc=dyroneteng@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=tenglong.tl@alibaba-inc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).