git@vger.kernel.org list mirror (unofficial, one of many)
 help / color / Atom feed
From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
To: SZEDER Gábor <szeder.dev@gmail.com>
Cc: Jeff King <peff@peff.net>, Derrick Stolee <dstolee@microsoft.com>,
	Junio C Hamano <gitster@pobox.com>, Eric Wong <e@80x24.org>,
	git@vger.kernel.org
Subject: Re: [PATCH v3] repack: enable bitmaps by default on bare repos
Date: Fri, 24 May 2019 15:41:40 +0200
Message-ID: <87d0k8ezqj.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <20190524123419.GI951@szeder.dev>


On Fri, May 24 2019, SZEDER Gábor wrote:

> On Fri, May 24, 2019 at 01:58:03PM +0200, Ævar Arnfjörð Bjarmason wrote:
>>
>> On Fri, May 24 2019, SZEDER Gábor wrote:
>>
>> > On Fri, May 24, 2019 at 01:17:06PM +0200, Ævar Arnfjörð Bjarmason wrote:
>> >>
>> >> On Fri, May 24 2019, SZEDER Gábor wrote:
>> >>
>> >> > On Fri, May 24, 2019 at 11:01:39AM +0200, Ævar Arnfjörð Bjarmason wrote:
>> >> >> I don't think it's a performance problem to have an old commit-graph
>> >> >> lying around. But if you turn on the commit-graph, run gc a bunch, then
>> >> >> turn it off in config we'll have it lying around forever, even if you do
>> >> >> subsequent gc's.
>> >> >>
>> >> >> So I think we should delete such things on the general principle that
>> >> >> the end-state of a gc's shouldn't be the accumulation of the values of
>> >> >> past configuration options if we can help it.
>> >> >>
>> >> >> Maybe that screws over other users who did a "commit-graph write"
>> >> >> without setting gc.writeCommitGraph, but I think the only sane thing to
>> >> >> do is to make "gc" fully 'own' such things if its turned on at all.
>> >> >
>> >> > Note that there is 'core.commitGraph' as well; as long as it's
>> >> > enabled, no commit-graph files should be deleted.
>> >>
>> >> Why? If we won't update it or write it if it's not there, why keep it
>> >> around?
>> >
>> > To read it, if 'core.commitGraph' says that is should be read.
>> >
>> >> It means the commit-graph code and anything else (like bitmaps) needs to
>> >> deal with stale data for the common and default gc --auto case.
>> >>
>> >> You also can't have e.g. a global core.commitGraph=true config along
>> >> with a per-repo gc.writeCommitGraph=true config do what you expect.
>> >>
>> >> Now just because you wanted to write it for some you'll end up keeping
>> >> it around forever because you'd also want to optimistically always use
>> >> it if it's there.
>> >
>> > This is exactly what I expect it to do.
>>
>> Do you also expect base packs with an associated bitmap to have an
>> implicit *.keep flag under gc with pack.writeBitmaps=false and
>> pack.useBitmaps=true?
>
> I don't understand what an "implicit *.keep flag" is[...]

A .keep means we keep the pack, and e.g. gc.bigPackThreshold is
effectively an implicit *.keep flag on a pack matching some critera,
which in this case caused this issue of a stale *.bitmap file (since the
pack wasn't touched, neither was the bitmap).

> [...]However, since a reachability bitmap is always associated with a
> pack, but the commit-graph is not, I don't think this is a valid
> comparison.

I don't either, I'm just trying to understand where you're coming from,
and I still don't.

That bitmaps are associated with specific packs and the commit graph
isn't is an internal implementation detail. Users who care about that
distinction either don't use "git gc" or would be willing to tweak its
settings.

For the rest I think removing existing side-indexes is a good default
for the practical reasons mentioned upthread.

>> >> Note that I'm talking about the *default* gc semantics, they don't have
>> >> to cover all advanced use-cases, just be good enough for most, and it's
>> >> also important that they're as simple as possible, and don't result in
>> >> stuff like "my performance sucks because I turned this config option on
>> >> once a year ago for 2 days".

  reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-14  4:31 [PATCH 0/3] some prune optimizations Jeff King
2019-02-14  4:35 ` [PATCH 1/3] prune: lazily perform reachability traversal Jeff King
2019-02-14 10:54   ` Eric Sunshine
2019-02-14 11:07     ` Jeff King
2019-02-14  4:37 ` [PATCH 2/3] prune: use bitmaps for " Jeff King
2019-03-09  2:49   ` bitmaps by default? [was: prune: use bitmaps for reachability traversal] Eric Wong
2019-03-10 23:39     ` Jeff King
2019-03-12  3:13       ` [PATCH] repack: enable bitmaps by default on bare repos Eric Wong
2019-03-12  9:07         ` Ævar Arnfjörð Bjarmason
2019-03-12 10:49         ` Jeff King
2019-03-12 12:05           ` Jeff King
2019-03-13  1:51           ` Eric Wong
2019-03-13 14:54             ` Jeff King
2019-03-14  9:12               ` [PATCH v3] " Eric Wong
2019-03-14 16:02                 ` Jeff King
2019-03-15  6:21                   ` [PATCH 0/2] enable bitmap hash-cache by default Jeff King
2019-03-15  6:22                     ` [PATCH 1/2] t5310: correctly remove bitmaps for jgit test Jeff King
2019-03-15 13:25                       ` SZEDER Gábor
2019-03-15 18:36                         ` Jeff King
2019-03-15  6:25                     ` [PATCH 2/2] pack-objects: default to writing bitmap hash-cache Jeff King
2019-04-09 15:10                 ` [PATCH v3] repack: enable bitmaps by default on bare repos Ævar Arnfjörð Bjarmason
2019-04-10 22:57                   ` Jeff King
2019-04-25  7:16                     ` Junio C Hamano
2019-05-04  1:37                       ` Jeff King
2019-05-04  6:52                         ` Ævar Arnfjörð Bjarmason
2019-05-04 13:23                           ` SZEDER Gábor
2019-05-08 20:17                             ` Ævar Arnfjörð Bjarmason
2019-05-09  4:24                               ` Junio C Hamano
2019-05-07  7:45                           ` Jeff King
2019-05-07  8:12                             ` Ævar Arnfjörð Bjarmason
2019-05-08  7:11                               ` Jeff King
2019-05-08 14:20                                 ` Derrick Stolee
2019-05-08 16:13                                 ` Ævar Arnfjörð Bjarmason
2019-05-08 22:25                                   ` Jeff King
2019-05-23 11:30                     ` Jeff King
2019-05-23 12:53                       ` Derrick Stolee
2019-05-24  7:24                         ` Jeff King
2019-05-24 10:33                           ` Derrick Stolee
2019-05-23 19:26                       ` Ævar Arnfjörð Bjarmason
2019-05-24  7:27                         ` Jeff King
2019-05-24  7:55                           ` Ævar Arnfjörð Bjarmason
2019-05-24  8:26                             ` Jeff King
2019-05-24  9:01                               ` Ævar Arnfjörð Bjarmason
2019-05-24  9:29                                 ` SZEDER Gábor
2019-05-24 11:17                                   ` Ævar Arnfjörð Bjarmason
2019-05-24 11:41                                     ` SZEDER Gábor
2019-05-24 11:58                                       ` Ævar Arnfjörð Bjarmason
2019-05-24 12:34                                         ` SZEDER Gábor
2019-05-24 13:41                                           ` Ævar Arnfjörð Bjarmason [this message]
2019-05-24 11:31                       ` [PATCH] pack-bitmap: look for an uninteresting bitmap Derrick Stolee
2019-04-15 15:00   ` [PATCH 2/3] prune: use bitmaps for reachability traversal Derrick Stolee
2019-04-18 19:49     ` Jeff King
2019-04-18 20:08       ` [PATCH] t5304: add a test for pruning with bitmaps Jeff King
2019-04-20  1:01         ` Derrick Stolee
2019-04-20  3:24           ` Jeff King
2019-04-20 21:01             ` Derrick Stolee
2019-02-14  4:38 ` [PATCH 3/3] prune: check SEEN flag for reachability Jeff King

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87d0k8ezqj.fsf@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=dstolee@microsoft.com \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=peff@peff.net \
    --cc=szeder.dev@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

git@vger.kernel.org list mirror (unofficial, one of many)

Archives are clonable:
	git clone --mirror https://public-inbox.org/git
	git clone --mirror http://ou63pmih66umazou.onion/git
	git clone --mirror http://czquwvybam4bgbro.onion/git
	git clone --mirror http://hjrcffqmbrq6wope.onion/git

Example config snippet for mirrors

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.version-control.git
	nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git
	nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git
	nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git
	nntp://news.gmane.org/gmane.comp.version-control.git

 note: .onion URLs require Tor: https://www.torproject.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox