Re: [External Mail]Re: why git is so slow for a tiny git push?

[Jeff King <peff@peff.net>]

On Tue, Oct 12, 2021 at 12:06:04PM +0200, Ævar Arnfjörð Bjarmason wrote:

> But more generally with these side-indexes it seems to me that the code
> involved might not be considering these sorts of edge cases, i.e. my
> understanding from you above is that if we have bitmaps anywhere we'll
> try to in-memory use them for all the objects in play? Or that otherwise
> having "partial" bitmaps leads to pathological behavior.

Sure, if there was an easy way to know beforehand whether the bitmap was
going to help or run into these pathological cases, it would be nice to
detect it. I don't know what that is (and I've given it quite a lot of
thought over the past 8 years).

I suspect the most direction would be to teach the bitmap code to behave
more like the regular traversal by just walking down to the
UNINTERESTING commits. Right now it gets a complete bitmap for the
commits we don't want, and then a bitmap for the ones we do want, and
takes a set difference.

It could instead walk both sides in the usual way, filling in the bitmap
for each, and then stop when it hits boundary commits. The bitmap for
the boundary commit (if we don't have a full one on-disk) is filled in
with what's in its tree. That means it's incomplete, and the result
might include some extra objects (e.g., if boundary~100 had a blob that
went away, but later came back in a descendant that isn't marked
uninteresting). That's the same tradeoff the non-bitmap traversal makes.

It would be pretty major surgery to the bitmap code. I haven't actually
tried it before.

-Peff