From: Jeff King <peff@peff.net>
To: David Emett <dave@sp4m.net>
Cc: git@vger.kernel.org
Subject: Re: Two issues with mark_reachable_objects
Date: Tue, 27 Apr 2021 10:41:18 -0400 [thread overview]
Message-ID: <YIgijn93f639Pp7Z@coredump.intra.peff.net> (raw)
In-Reply-To: <CAJ-dYSOVx0egnyxJb6ZjgWvEDR=19QPgc70JQ7cXUjUPZ1XDiQ@mail.gmail.com>
On Tue, Apr 27, 2021 at 11:45:01AM +0100, David Emett wrote:
> I noticed that this only happened when the fetch triggered an automatic GC at
> the end. After a bit of digging I discovered two separate issues:
>
> 1) It seems that FETCH_HEAD is not considered a root by mark_reachable_objects.
Right, as you discovered, this is known and intentional. I don't have
anything to add to the thread you linked already.
> 2) If the bitmap_git branch in mark_reachable_objects is taken, the mark_recent
> argument is ignored. This doesn't _completely_ break "git prune"'s --expire
> option, as it turns out there is another explicit mtime check in
> prune_object (prune.c). If that check passes this is not propagated to
> referenced objects though. So even if a dangling commit is recent, a prune
> can discard old objects it references.
But this one is a scary bug. As you note, it's not _completely_ breaking
"--expire", but it is totally disabling the "reachable from recent"
safety added in d3038d22f9 (prune: keep objects reachable from recent
objects, 2014-10-15).
The bug here was introduced by me, and comes from a matter of timing.
Despite what you'll see in the project history, the "use bitmaps" patch
actually predates the "reachable from recent" one. I just didn't clean
it up and upstream it until 2019, and failed to notice the bad
interaction between the two.
> I assume (2) is not intentional, given that "git gc --help" explicitly says
> "Any object with modification time newer than the --prune date is kept, along
> with everything reachable from it." Is it safe to just run the mark_recent
> block after the bitmap_git block? Could add_unseen_recent_objects_to_traversal
> just be called at the start of the bitmap_git block if mark_recent?
So no, definitely not intentional.
I think we'd just want to run the whole mark_recent block after doing
the bitmap traversal.
There may be some subtlety with reusing the rev_info struct again. I
think we'd want to reset the pending objects list after calling into the
bitmap code. It _usually_ does an actual traversal that consumes the
list, but not necessarily. I think traverse_bitmap_commit_list()
probably ought to be the one to do it, so it behaves more like
traverse_commit_list(). (OTOH, I don't think it's _too_ bad if we don't;
we'd include those already-seen objects in our traversal, but they
should all by definition have the SEEN bit set, so we'd stop there).
I don't think we want to add_unseen_recent_objects_to_traversal() to
include it as part of the same traversal, for the same reason the
non-bitmap traversal does not combine them: the mark_recent traversal is
best-effort. We set revs->ignore_missing_links to be tolerant of
already-broken segments of history.
It's possible that we could do the second mark_recent traversal also
with bitmaps (but still separately). I can't offhand think of a reason
that ignore_missing_links wouldn't behave well there. But since we
expect it to be small, I'd be more comfortable just using the regular
traversal code.
-Peff
next prev parent reply other threads:[~2021-04-27 14:41 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-27 10:45 Two issues with mark_reachable_objects David Emett
2021-04-27 14:41 ` Jeff King [this message]
2021-04-27 15:13 ` Jeff King
2021-04-27 15:43 ` [PATCH] prune: save reachable-from-recent objects with bitmaps Jeff King
2021-04-28 12:20 ` David Emett
2021-04-28 15:13 ` Jeff King
2021-04-28 15:41 ` [PATCH v2 0/2] " Jeff King
2021-04-28 15:42 ` [PATCH v2 1/2] pack-bitmap: clean up include_check after use Jeff King
2021-04-28 15:42 ` [PATCH v2 2/2] prune: save reachable-from-recent objects with bitmaps Jeff King
2021-04-29 1:37 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YIgijn93f639Pp7Z@coredump.intra.peff.net \
--to=peff@peff.net \
--cc=dave@sp4m.net \
--cc=git@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).