From: "Ævar Arnfjörð Bjarmason" <firstname.lastname@example.org> To: Jeff King <email@example.com> Cc: "Junio C Hamano" <firstname.lastname@example.org>, "Eric Wong" <email@example.com>, firstname.lastname@example.org, "SZEDER Gábor" <email@example.com>, "Derrick Stolee" <firstname.lastname@example.org> Subject: Re: [PATCH v3] repack: enable bitmaps by default on bare repos Date: Wed, 08 May 2019 18:13:58 +0200 Message-ID: <email@example.com> (raw) In-Reply-To: <20190508071105.GA14043@sigill.intra.peff.net> On Wed, May 08 2019, Jeff King wrote: > On Tue, May 07, 2019 at 10:12:06AM +0200, Ævar Arnfjörð Bjarmason wrote: > >> > I think we'd want a way to tell the bitmap code to update our progress >> > meter as it traverses (both single objects, but also taking into account >> > when it finds a bitmap and then suddenly bumps the value by a large >> > amount). >> >> Not splitting it will fix the progress bar stalling, so it fixes the >> problem that the user is wondering if the command is entirely hanging. >> >> But I was hoping to give the user an idea of roughly where we're >> spending our time, e.g. so you can see how much the pack.useSparse >> setting is helping (or not). > > Yeah, I think that's a bigger and more complicated problem. I admit that > my main annoyance is just the stall while we fill in the bitmaps (and > it's easy because the bitmap traversal is the same unit of work as a > regular traversal). > >> So something where we report sub-progress as we go along, and perhaps >> print some brief summary at the end if it took long enough, e.g.: >> >> Enumerating Objects (X^1%) => Marking trees (Y^1%) >> Enumerating Objects (X^2%) => Calculating bitmaps (Y^2%) >> >> And at the end: >> >> Enumerating Objects (100%) in ~2m30s -- (~10s marking trees, ~2m10s bitmaps, ~10s other) >> >> I.e. bringing the whole "nested" trace2 regions full circle with the >> progress bar where we could elect to trace/show some of that info, and >> then you could turn on some trace2 mode/verbose progress to see more. > > I do wonder if this really needs to be part of the progress bar. The > goal of the progress bar is to give the user a sense that work is > happening, and (if possible, but not for "enumerating") an idea of when > it might finish. If the trace code can already do detailed timings, then > shouldn't we just be encouraging people to use that? To just show work happening we could save ourselves some horizontal space and the debates over counting v.s. enumerating with: diff --git a/progress.c b/progress.c index 0318bdd41b..83336ca391 100644 --- a/progress.c +++ b/progress.c @@ -226,3 +226,3 @@ static struct progress *start_progress_delay(const char *title, uint64_t total, struct progress *progress = xmalloc(sizeof(*progress)); - progress->title = title; + progress->title = "Reticulating splines"; progress->total = total; :) Obviously that's silly, but the point is we do show some user messaging with these now, and e.g. the other day here on-list (can't be bothered to find the msgid) someone was lamenting that the N progressbars we show on "push" were too verbose. So by coalescing some of the existing bars that do one logical operation (push) in N steps we could be less verbose without losing the "stuff's happening" part of it, and would see if something odd was going on, e.g. the "I/O write" part being proportionally slower on this box than the other, or when they upgrade bitmaps suddenly showing up as >95% of the time. The bit I find interesting about tying it into trace2 is that once you do that the trace logs can contain e.g. min/max/avg/median/percentile time for doing some operation we can break into N steps same/similar steps, which might be interesting for performance analysis.
next prev parent reply index Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-02-14 4:31 [PATCH 0/3] some prune optimizations Jeff King 2019-02-14 4:35 ` [PATCH 1/3] prune: lazily perform reachability traversal Jeff King 2019-02-14 10:54 ` Eric Sunshine 2019-02-14 11:07 ` Jeff King 2019-02-14 4:37 ` [PATCH 2/3] prune: use bitmaps for " Jeff King 2019-03-09 2:49 ` bitmaps by default? [was: prune: use bitmaps for reachability traversal] Eric Wong 2019-03-10 23:39 ` Jeff King 2019-03-12 3:13 ` [PATCH] repack: enable bitmaps by default on bare repos Eric Wong 2019-03-12 9:07 ` Ævar Arnfjörð Bjarmason 2019-03-12 10:49 ` Jeff King 2019-03-12 12:05 ` Jeff King 2019-03-13 1:51 ` Eric Wong 2019-03-13 14:54 ` Jeff King 2019-03-14 9:12 ` [PATCH v3] " Eric Wong 2019-03-14 16:02 ` Jeff King 2019-03-15 6:21 ` [PATCH 0/2] enable bitmap hash-cache by default Jeff King 2019-03-15 6:22 ` [PATCH 1/2] t5310: correctly remove bitmaps for jgit test Jeff King 2019-03-15 13:25 ` SZEDER Gábor 2019-03-15 18:36 ` Jeff King 2019-03-15 6:25 ` [PATCH 2/2] pack-objects: default to writing bitmap hash-cache Jeff King 2019-04-09 15:10 ` [PATCH v3] repack: enable bitmaps by default on bare repos Ævar Arnfjörð Bjarmason 2019-04-10 22:57 ` Jeff King 2019-04-25 7:16 ` Junio C Hamano 2019-05-04 1:37 ` Jeff King 2019-05-04 6:52 ` Ævar Arnfjörð Bjarmason 2019-05-04 13:23 ` SZEDER Gábor 2019-05-08 20:17 ` Ævar Arnfjörð Bjarmason 2019-05-09 4:24 ` Junio C Hamano 2019-05-07 7:45 ` Jeff King 2019-05-07 8:12 ` Ævar Arnfjörð Bjarmason 2019-05-08 7:11 ` Jeff King 2019-05-08 14:20 ` Derrick Stolee 2019-05-08 16:13 ` Ævar Arnfjörð Bjarmason [this message] 2019-05-08 22:25 ` Jeff King 2019-05-23 11:30 ` Jeff King 2019-05-23 12:53 ` Derrick Stolee 2019-05-24 7:24 ` Jeff King 2019-05-24 10:33 ` Derrick Stolee 2019-05-23 19:26 ` Ævar Arnfjörð Bjarmason 2019-05-24 7:27 ` Jeff King 2019-05-24 7:55 ` Ævar Arnfjörð Bjarmason 2019-05-24 8:26 ` Jeff King 2019-05-24 9:01 ` Ævar Arnfjörð Bjarmason 2019-05-24 9:29 ` SZEDER Gábor 2019-05-24 11:17 ` Ævar Arnfjörð Bjarmason 2019-05-24 11:41 ` SZEDER Gábor 2019-05-24 11:58 ` Ævar Arnfjörð Bjarmason 2019-05-24 12:34 ` SZEDER Gábor 2019-05-24 13:41 ` Ævar Arnfjörð Bjarmason 2019-05-24 11:31 ` [PATCH] pack-bitmap: look for an uninteresting bitmap Derrick Stolee 2019-04-15 15:00 ` [PATCH 2/3] prune: use bitmaps for reachability traversal Derrick Stolee 2019-04-18 19:49 ` Jeff King 2019-04-18 20:08 ` [PATCH] t5304: add a test for pruning with bitmaps Jeff King 2019-04-20 1:01 ` Derrick Stolee 2019-04-20 3:24 ` Jeff King 2019-04-20 21:01 ` Derrick Stolee 2019-02-14 4:38 ` [PATCH 3/3] prune: check SEEN flag for reachability Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
firstname.lastname@example.org list mirror (unofficial, one of many) Archives are clonable: git clone --mirror https://public-inbox.org/git git clone --mirror http://ou63pmih66umazou.onion/git git clone --mirror http://czquwvybam4bgbro.onion/git git clone --mirror http://hjrcffqmbrq6wope.onion/git Example config snippet for mirrors Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.version-control.git nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git nntp://news.gmane.io/gmane.comp.version-control.git note: .onion URLs require Tor: https://www.torproject.org/ AGPL code for this site: git clone https://public-inbox.org/public-inbox.git