From: Derrick Stolee <stolee@gmail.com> To: "Jeff King" <peff@peff.net>, "Ævar Arnfjörð Bjarmason" <avarab@gmail.com> Cc: "Junio C Hamano" <gitster@pobox.com>, "Eric Wong" <e@80x24.org>, git@vger.kernel.org, "SZEDER Gábor" <szeder.dev@gmail.com> Subject: Re: [PATCH v3] repack: enable bitmaps by default on bare repos Date: Wed, 8 May 2019 10:20:24 -0400 Message-ID: <561912f3-4761-c690-5327-501e20023e4d@gmail.com> (raw) In-Reply-To: <20190508071105.GA14043@sigill.intra.peff.net> On 5/8/2019 3:11 AM, Jeff King wrote: > On Tue, May 07, 2019 at 10:12:06AM +0200, Ævar Arnfjörð Bjarmason wrote: > >>> I think we'd want a way to tell the bitmap code to update our progress >>> meter as it traverses (both single objects, but also taking into account >>> when it finds a bitmap and then suddenly bumps the value by a large >>> amount). >> >> Not splitting it will fix the progress bar stalling, so it fixes the >> problem that the user is wondering if the command is entirely hanging. >> >> But I was hoping to give the user an idea of roughly where we're >> spending our time, e.g. so you can see how much the pack.useSparse >> setting is helping (or not). > > Yeah, I think that's a bigger and more complicated problem. I admit that > my main annoyance is just the stall while we fill in the bitmaps (and > it's easy because the bitmap traversal is the same unit of work as a > regular traversal). The pack.useSparse setting also speeds up a section that is not marked by progress: that portion usually is walking all UNINTERESTING trees and the"Enumerating Objects" progress is just for walking the INTERESTING objects. >> So something where we report sub-progress as we go along, and perhaps >> print some brief summary at the end if it took long enough, e.g.: >> >> Enumerating Objects (X^1%) => Marking trees (Y^1%) >> Enumerating Objects (X^2%) => Calculating bitmaps (Y^2%) I like this idea for splitting the "normal" mechanism, too: Enumerating Objects (X^1%) => Marking trees (Y^1%) Enumerating Objects (X^2%) => Enumerating objects to pack (Y^2%) >> I.e. bringing the whole "nested" trace2 regions full circle with the >> progress bar where we could elect to trace/show some of that info, and >> then you could turn on some trace2 mode/verbose progress to see more. > > I do wonder if this really needs to be part of the progress bar. The > goal of the progress bar is to give the user a sense that work is > happening, and (if possible, but not for "enumerating") an idea of when > it might finish. If the trace code can already do detailed timings, then > shouldn't we just be encouraging people to use that? The problem I've seen (without bitmaps) is that running `git push` can take a while before _any_ progress is listed. Good news is: `pack.useSparse` fixed our push problem in the Windows OS repo. The end-to-end time for `git push` sped up by 7.7x with the change, and this "blank" time is too fast for users to notice. Updating the progress could help in cases without pack.useSparse. Thanks, -Stolee
next prev parent reply other threads:[~2019-05-08 14:20 UTC|newest] Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-02-14 4:31 [PATCH 0/3] some prune optimizations Jeff King 2019-02-14 4:35 ` [PATCH 1/3] prune: lazily perform reachability traversal Jeff King 2019-02-14 10:54 ` Eric Sunshine 2019-02-14 11:07 ` Jeff King 2019-02-14 4:37 ` [PATCH 2/3] prune: use bitmaps for " Jeff King 2019-03-09 2:49 ` bitmaps by default? [was: prune: use bitmaps for reachability traversal] Eric Wong 2019-03-10 23:39 ` Jeff King 2019-03-12 3:13 ` [PATCH] repack: enable bitmaps by default on bare repos Eric Wong 2019-03-12 9:07 ` Ævar Arnfjörð Bjarmason 2019-03-12 10:49 ` Jeff King 2019-03-12 12:05 ` Jeff King 2019-03-13 1:51 ` Eric Wong 2019-03-13 14:54 ` Jeff King 2019-03-14 9:12 ` [PATCH v3] " Eric Wong 2019-03-14 16:02 ` Jeff King 2019-03-15 6:21 ` [PATCH 0/2] enable bitmap hash-cache by default Jeff King 2019-03-15 6:22 ` [PATCH 1/2] t5310: correctly remove bitmaps for jgit test Jeff King 2019-03-15 13:25 ` SZEDER Gábor 2019-03-15 18:36 ` Jeff King 2019-03-15 6:25 ` [PATCH 2/2] pack-objects: default to writing bitmap hash-cache Jeff King 2019-04-09 15:10 ` [PATCH v3] repack: enable bitmaps by default on bare repos Ævar Arnfjörð Bjarmason 2019-04-10 22:57 ` Jeff King 2019-04-25 7:16 ` Junio C Hamano 2019-05-04 1:37 ` Jeff King 2019-05-04 6:52 ` Ævar Arnfjörð Bjarmason 2019-05-04 13:23 ` SZEDER Gábor 2019-05-08 20:17 ` Ævar Arnfjörð Bjarmason 2019-05-09 4:24 ` Junio C Hamano 2019-05-07 7:45 ` Jeff King 2019-05-07 8:12 ` Ævar Arnfjörð Bjarmason 2019-05-08 7:11 ` Jeff King 2019-05-08 14:20 ` Derrick Stolee [this message] 2019-05-08 16:13 ` Ævar Arnfjörð Bjarmason 2019-05-08 22:25 ` Jeff King 2019-05-23 11:30 ` Jeff King 2019-05-23 12:53 ` Derrick Stolee 2019-05-24 7:24 ` Jeff King 2019-05-24 10:33 ` Derrick Stolee 2019-05-23 19:26 ` Ævar Arnfjörð Bjarmason 2019-05-24 7:27 ` Jeff King 2019-05-24 7:55 ` Ævar Arnfjörð Bjarmason 2019-05-24 8:26 ` Jeff King 2019-05-24 9:01 ` Ævar Arnfjörð Bjarmason 2019-05-24 9:29 ` SZEDER Gábor 2019-05-24 11:17 ` Ævar Arnfjörð Bjarmason 2019-05-24 11:41 ` SZEDER Gábor 2019-05-24 11:58 ` Ævar Arnfjörð Bjarmason 2019-05-24 12:34 ` SZEDER Gábor 2019-05-24 13:41 ` Ævar Arnfjörð Bjarmason 2019-05-24 11:31 ` [PATCH] pack-bitmap: look for an uninteresting bitmap Derrick Stolee 2019-04-15 15:00 ` [PATCH 2/3] prune: use bitmaps for reachability traversal Derrick Stolee 2019-04-18 19:49 ` Jeff King 2019-04-18 20:08 ` [PATCH] t5304: add a test for pruning with bitmaps Jeff King 2019-04-20 1:01 ` Derrick Stolee 2019-04-20 3:24 ` Jeff King 2019-04-20 21:01 ` Derrick Stolee 2019-02-14 4:38 ` [PATCH 3/3] prune: check SEEN flag for reachability Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=561912f3-4761-c690-5327-501e20023e4d@gmail.com \ --to=stolee@gmail.com \ --cc=avarab@gmail.com \ --cc=e@80x24.org \ --cc=git@vger.kernel.org \ --cc=gitster@pobox.com \ --cc=peff@peff.net \ --cc=szeder.dev@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
git@vger.kernel.org list mirror (unofficial, one of many) This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/git git clone --mirror http://ou63pmih66umazou.onion/git git clone --mirror http://czquwvybam4bgbro.onion/git git clone --mirror http://hjrcffqmbrq6wope.onion/git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 git git/ https://public-inbox.org/git \ git@vger.kernel.org public-inbox-index git Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.version-control.git nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git nntp://news.gmane.io/gmane.comp.version-control.git note: .onion URLs require Tor: https://www.torproject.org/ code repositories for the project(s) associated with this inbox: https://80x24.org/mirrors/git.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git