git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: David Emett <dave@sp4m.net>
Cc: git@vger.kernel.org
Subject: [PATCH v2 0/2] prune: save reachable-from-recent objects with bitmaps
Date: Wed, 28 Apr 2021 11:41:41 -0400	[thread overview]
Message-ID: <YImCNXC5DUvy5gT8@coredump.intra.peff.net> (raw)
In-Reply-To: <YIl7tUJA6sQe2K73@coredump.intra.peff.net>

On Wed, Apr 28, 2021 at 11:13:57AM -0400, Jeff King wrote:

> On Wed, Apr 28, 2021 at 01:20:03PM +0100, David Emett wrote:
> 
> > > Here's a fix. Thanks very much for reporting.
> > 
> > Thanks for the quick response! I tried the fix out on the repo I was having
> > trouble with. It's hitting a segfault in traverse_commit_list in the
> > mark_recent block. It looks like the issue is that the bitmap code leaves
> > revs->include_check set, with revs->include_check_data pointing at the stack.
> > Setting revs->include_check to NULL after the traverse_commit_list call in
> > find_objects in pack-bitmap.c fixes the segfault for me. And the original issue
> > appears to be resolved as well, so thanks!
> 
> Oof, good catch. I wondered why my test didn't see this. The answer is
> that the include_check only applies to commits, not trees. And my test
> only has a recent-but-unreachable tree. Beefing it up to include an
> actual commit shows the problem (and ASan nicely pinpoints the issue).
> 
> The solution, as you noted, is to have the bitmap code clean up after
> itself.
> 
> I'll prepare a re-roll of the series.

Here it is.

I did notice one other oddity. The bitmap code doesn't do any sort of
progress reporting. So progress is enabled, we'll _just_ count the
recent-but-unreachable entries in the progress meter.

That's not entirely wrong (it represents the actual traversal work we're
doing), but is a little funny. However, I think the best thing to do for
now is to leave it there. The right path forward is for the bitmap code
to learn about progress reporting, too (for this case, but also for
others). But that's a more involved topic, and I don't want to hold up
this fix for it.

  [1/2]: pack-bitmap: clean up include_check after use
  [2/2]: prune: save reachable-from-recent objects with bitmaps

 pack-bitmap.c              |  3 +++
 reachable.c                | 13 ++++---------
 t/t5304-prune.sh           | 16 ++++++++++++++++
 t/t6501-freshen-objects.sh | 21 +++++++++++++++------
 4 files changed, 38 insertions(+), 15 deletions(-)

-:  ---------- > 1:  43f25e88c5 pack-bitmap: clean up include_check after use
1:  a7fb2a7f5b ! 2:  0a60ea597d prune: save reachable-from-recent objects with bitmaps
    @@ t/t5304-prune.sh: test_expect_success 'trivial prune with bitmaps enabled' '
     +	to_save=$(echo bitmap-from-recent-2 | git hash-object -w --stdin) &&
     +	test-tool chmtime -86400 .git/objects/$(test_oid_to_path $to_save) &&
     +	tree=$(printf "100644 blob $to_save\tfile\n" | git mktree) &&
    ++	test-tool chmtime -86400 .git/objects/$(test_oid_to_path $tree) &&
    ++	commit=$(echo foo | git commit-tree $tree) &&
     +	git prune --expire=12.hours.ago &&
    ++	git cat-file -e $commit &&
     +	git cat-file -e $tree &&
     +	git cat-file -e $to_save &&
     +	test_must_fail git cat-file -e $to_drop

-Peff

  reply	other threads:[~2021-04-28 15:45 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-27 10:45 Two issues with mark_reachable_objects David Emett
2021-04-27 14:41 ` Jeff King
2021-04-27 15:13   ` Jeff King
2021-04-27 15:43 ` [PATCH] prune: save reachable-from-recent objects with bitmaps Jeff King
2021-04-28 12:20   ` David Emett
2021-04-28 15:13     ` Jeff King
2021-04-28 15:41       ` Jeff King [this message]
2021-04-28 15:42         ` [PATCH v2 1/2] pack-bitmap: clean up include_check after use Jeff King
2021-04-28 15:42         ` [PATCH v2 2/2] prune: save reachable-from-recent objects with bitmaps Jeff King
2021-04-29  1:37           ` Junio C Hamano

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YImCNXC5DUvy5gT8@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=dave@sp4m.net \
    --cc=git@vger.kernel.org \
    --subject='Re: [PATCH v2 0/2] prune: save reachable-from-recent objects with bitmaps' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).