From: Taylor Blau <me@ttaylorr.com>
To: "SZEDER Gábor" <szeder.dev@gmail.com>
Cc: Taylor Blau <me@ttaylorr.com>,
git@vger.kernel.org, peff@peff.net, dstolee@microsoft.com,
gitster@pobox.com
Subject: Re: [PATCH v3 14/14] builtin/commit-graph.c: introduce '--max-new-filters=<n>'
Date: Fri, 14 Aug 2020 16:20:21 -0400 [thread overview]
Message-ID: <20200814202021.GE30103@syl.lan> (raw)
In-Reply-To: <20200812114929.GA19082@szeder.dev>
On Wed, Aug 12, 2020 at 01:49:29PM +0200, SZEDER Gábor wrote:
> On Tue, Aug 11, 2020 at 04:52:14PM -0400, Taylor Blau wrote:
>
> > diff --git a/commit-graph.c b/commit-graph.c
> > index 6886f319a5..4aae5471e3 100644
> > --- a/commit-graph.c
> > +++ b/commit-graph.c
> > @@ -954,7 +954,8 @@ struct tree *get_commit_tree_in_graph(struct repository *r, const struct commit
> > }
> >
> > static int get_bloom_filter_large_in_graph(struct commit_graph *g,
> > - const struct commit *c)
> > + const struct commit *c,
> > + uint32_t max_changed_paths)
> > {
> > uint32_t graph_pos = commit_graph_position(c);
> > if (graph_pos == COMMIT_NOT_FROM_GRAPH)
> > @@ -965,6 +966,17 @@ static int get_bloom_filter_large_in_graph(struct commit_graph *g,
> >
> > if (!(g && g->bloom_large))
> > return 0;
> > + if (g->bloom_filter_settings->max_changed_paths != max_changed_paths) {
> > + /*
> > + * Force all commits which are subject to a different
> > + * 'max_changed_paths' limit to be recomputed from scratch.
> > + *
> > + * Note that this could likely be improved, but is ignored since
> > + * all real-world graphs set the maximum number of changed paths
> > + * at 512.
> > + */
> > + return 0;
> > + }
> > return bitmap_get(g->bloom_large, graph_pos - g->num_commits_in_base);
> > }
> >
> > @@ -1470,6 +1482,7 @@ static void compute_bloom_filters(struct write_commit_graph_context *ctx)
> > int i;
> > struct progress *progress = NULL;
> > int *sorted_commits;
> > + int max_new_filters;
> >
> > init_bloom_filters();
> > ctx->bloom_large = bitmap_word_alloc(ctx->commits.nr / BITS_IN_EWORD + 1);
> > @@ -1486,10 +1499,15 @@ static void compute_bloom_filters(struct write_commit_graph_context *ctx)
> > ctx->order_by_pack ? commit_pos_cmp : commit_gen_cmp,
> > &ctx->commits);
> >
> > + max_new_filters = ctx->opts->max_new_filters >= 0 ?
> > + ctx->opts->max_new_filters : ctx->commits.nr;
>
> git_test_write_commit_graph_or_die() invokes
> write_commit_graph_reachable() with opts=0x0, so 'ctx->opts' is NULL,
> and we get segfault. This breaks a lot of tests when run with
> GIT_TEST_COMMIT_GRAPH=1 GIT_TEST_COMMIT_GRAPH_CHANGED_PATHS=1.
Great catch, thanks. Fixing this is as simple as adding 'ctx->opts &&'
right before we dereference 'ctx->opts', since setting this variable
equal to 'ctx->commits.nr' is the right thing to do in that case.
Unrelated to this comment, I am hoping to send out a final version of
this series sometime next week so that we can keep moving forward with
Bloom filter improvements.
Have you had a chance to review the rest of the patches? I'll happily
wait until you have had a chance to do so before sending v5 so that we
can avoid a v6.
Thanks,
Taylor
next prev parent reply other threads:[~2020-08-14 20:20 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-03 18:57 [PATCH 00/10] more miscellaneous Bloom filter improvements Taylor Blau
2020-08-03 18:57 ` [PATCH 01/10] commit-graph: introduce 'get_bloom_filter_settings()' Taylor Blau
2020-08-04 7:24 ` Jeff King
2020-08-04 20:08 ` Taylor Blau
2020-08-03 18:57 ` [PATCH 02/10] commit-graph: pass a 'struct repository *' in more places Taylor Blau
2020-08-03 18:57 ` [PATCH 03/10] t4216: use an '&&'-chain Taylor Blau
2020-08-03 18:57 ` [PATCH 04/10] t/helper/test-read-graph.c: prepare repo settings Taylor Blau
2020-08-03 18:57 ` [PATCH 05/10] commit-graph: respect 'commitgraph.readChangedPaths' Taylor Blau
2020-08-03 18:57 ` [PATCH 06/10] commit-graph.c: sort index into commits list Taylor Blau
2020-08-04 12:31 ` Derrick Stolee
2020-08-04 20:10 ` Taylor Blau
2020-08-03 18:57 ` [PATCH 07/10] commit-graph: add large-filters bitmap chunk Taylor Blau
2020-08-03 18:59 ` Taylor Blau
2020-08-04 12:57 ` Derrick Stolee
2020-08-03 18:57 ` [PATCH 08/10] bloom: split 'get_bloom_filter()' in two Taylor Blau
2020-08-04 13:00 ` Derrick Stolee
2020-08-04 20:12 ` Taylor Blau
2020-08-03 18:57 ` [PATCH 09/10] commit-graph: rename 'split_commit_graph_opts' Taylor Blau
2020-08-03 18:57 ` [PATCH 10/10] builtin/commit-graph.c: introduce '--max-new-filters=<n>' Taylor Blau
2020-08-04 13:03 ` Derrick Stolee
2020-08-04 20:14 ` Taylor Blau
2020-08-05 17:01 ` [PATCH v2 00/14] more miscellaneous Bloom filter improvements Taylor Blau
2020-08-05 17:01 ` [PATCH v2 01/14] commit-graph: introduce 'get_bloom_filter_settings()' Taylor Blau
2020-08-05 17:02 ` [PATCH v2 02/14] t4216: use an '&&'-chain Taylor Blau
2020-08-05 17:02 ` [PATCH v2 03/14] commit-graph: pass a 'struct repository *' in more places Taylor Blau
2020-08-05 17:02 ` [PATCH v2 04/14] t/helper/test-read-graph.c: prepare repo settings Taylor Blau
2020-08-05 17:02 ` [PATCH v2 05/14] commit-graph: respect 'commitGraph.readChangedPaths' Taylor Blau
2020-08-05 17:02 ` [PATCH v2 06/14] commit-graph.c: store maximum changed paths Taylor Blau
2020-08-05 17:02 ` [PATCH v2 07/14] bloom: split 'get_bloom_filter()' in two Taylor Blau
2020-08-05 17:02 ` [PATCH v2 08/14] bloom: use provided 'struct bloom_filter_settings' Taylor Blau
2020-08-05 17:02 ` [PATCH v2 09/14] bloom/diff: properly short-circuit on max_changes Taylor Blau
2020-08-05 17:02 ` [PATCH v2 10/14] commit-graph.c: sort index into commits list Taylor Blau
2020-08-05 17:02 ` [PATCH v2 11/14] csum-file.h: introduce 'hashwrite_be64()' Taylor Blau
2020-08-05 17:02 ` [PATCH v2 12/14] commit-graph: add large-filters bitmap chunk Taylor Blau
2020-08-05 21:01 ` Junio C Hamano
2020-08-05 21:17 ` Taylor Blau
2020-08-05 22:21 ` Junio C Hamano
2020-08-05 22:25 ` Taylor Blau
2020-08-11 13:48 ` Taylor Blau
2020-08-11 18:59 ` Junio C Hamano
2020-08-05 17:03 ` [PATCH v2 13/14] commit-graph: rename 'split_commit_graph_opts' Taylor Blau
2020-08-05 17:03 ` [PATCH v2 14/14] builtin/commit-graph.c: introduce '--max-new-filters=<n>' Taylor Blau
2020-08-11 20:51 ` [PATCH v3 00/14] more miscellaneous Bloom filter improvements Taylor Blau
2020-08-11 20:51 ` [PATCH v3 01/14] commit-graph: introduce 'get_bloom_filter_settings()' Taylor Blau
2020-08-11 21:18 ` SZEDER Gábor
2020-08-11 21:21 ` Taylor Blau
2020-08-11 21:27 ` SZEDER Gábor
2020-08-11 21:34 ` Taylor Blau
2020-08-11 23:55 ` SZEDER Gábor
2020-08-12 11:48 ` Derrick Stolee
2020-08-14 20:17 ` Taylor Blau
2020-08-11 20:51 ` [PATCH v3 02/14] t4216: use an '&&'-chain Taylor Blau
2020-08-11 20:51 ` [PATCH v3 03/14] commit-graph: pass a 'struct repository *' in more places Taylor Blau
2020-08-11 20:51 ` [PATCH v3 04/14] t/helper/test-read-graph.c: prepare repo settings Taylor Blau
2020-08-11 20:51 ` [PATCH v3 05/14] commit-graph: respect 'commitGraph.readChangedPaths' Taylor Blau
2020-08-11 20:51 ` [PATCH v3 06/14] commit-graph.c: store maximum changed paths Taylor Blau
2020-08-11 20:51 ` [PATCH v3 07/14] bloom: split 'get_bloom_filter()' in two Taylor Blau
2020-08-11 20:51 ` [PATCH v3 11/14] csum-file.h: introduce 'hashwrite_be64()' Taylor Blau
2020-08-11 20:51 ` [PATCH v3 08/14] bloom: use provided 'struct bloom_filter_settings' Taylor Blau
2020-08-11 20:51 ` [PATCH v3 09/14] bloom/diff: properly short-circuit on max_changes Taylor Blau
2020-08-11 20:52 ` [PATCH v3 10/14] commit-graph.c: sort index into commits list Taylor Blau
2020-08-11 20:52 ` [PATCH v3 12/14] commit-graph: add large-filters bitmap chunk Taylor Blau
2020-08-11 21:11 ` Derrick Stolee
2020-08-11 21:18 ` Taylor Blau
2020-08-11 22:05 ` Taylor Blau
2020-08-19 13:35 ` SZEDER Gábor
2020-09-02 20:23 ` Taylor Blau
2020-09-01 14:35 ` SZEDER Gábor
2020-09-02 20:40 ` Taylor Blau
2020-08-11 20:52 ` [PATCH v3 13/14] commit-graph: rename 'split_commit_graph_opts' Taylor Blau
2020-08-19 9:56 ` SZEDER Gábor
2020-09-02 21:02 ` Taylor Blau
2020-08-11 20:52 ` [PATCH v3 14/14] builtin/commit-graph.c: introduce '--max-new-filters=<n>' Taylor Blau
2020-08-12 11:49 ` SZEDER Gábor
2020-08-14 20:20 ` Taylor Blau [this message]
2020-08-17 22:50 ` SZEDER Gábor
2020-09-02 21:03 ` Taylor Blau
2020-08-12 12:29 ` Derrick Stolee
2020-08-14 20:10 ` Taylor Blau
2020-08-18 22:23 ` SZEDER Gábor
2020-09-03 16:35 ` Taylor Blau
2020-08-19 8:20 ` SZEDER Gábor
2020-09-03 16:42 ` Taylor Blau
2020-09-04 8:50 ` SZEDER Gábor
2020-09-01 14:36 ` SZEDER Gábor
2020-09-03 18:49 ` Taylor Blau
2020-09-03 21:45 ` [PATCH v3 00/14] more miscellaneous Bloom filter improvements Junio C Hamano
2020-09-03 22:33 ` Taylor Blau
2020-09-03 22:45 ` [PATCH v4 " Taylor Blau
2020-09-03 22:46 ` [PATCH v4 01/14] commit-graph: introduce 'get_bloom_filter_settings()' Taylor Blau
2020-09-03 22:46 ` [PATCH v4 02/14] t4216: use an '&&'-chain Taylor Blau
2020-09-03 22:46 ` [PATCH v4 03/14] commit-graph: pass a 'struct repository *' in more places Taylor Blau
2020-09-03 22:46 ` [PATCH v4 04/14] t/helper/test-read-graph.c: prepare repo settings Taylor Blau
2020-09-03 22:46 ` [PATCH v4 05/14] commit-graph: respect 'commitGraph.readChangedPaths' Taylor Blau
2020-09-03 22:46 ` [PATCH v4 06/14] commit-graph.c: store maximum changed paths Taylor Blau
2020-09-03 22:46 ` [PATCH v4 07/14] bloom: split 'get_bloom_filter()' in two Taylor Blau
2020-09-05 17:22 ` Jakub Narębski
2020-09-05 17:38 ` Taylor Blau
2020-09-05 17:50 ` Jakub Narębski
2020-09-05 18:01 ` Taylor Blau
2020-09-05 18:18 ` Jakub Narębski
2020-09-05 18:38 ` Taylor Blau
2020-09-05 18:55 ` Taylor Blau
2020-09-05 19:04 ` SZEDER Gábor
2020-09-05 19:49 ` Taylor Blau
2020-09-06 21:52 ` Junio C Hamano
2020-09-03 22:46 ` [PATCH v4 08/14] bloom: use provided 'struct bloom_filter_settings' Taylor Blau
2020-09-03 22:46 ` [PATCH v4 09/14] bloom/diff: properly short-circuit on max_changes Taylor Blau
2020-09-03 22:46 ` [PATCH v4 10/14] commit-graph.c: sort index into commits list Taylor Blau
2020-09-03 22:46 ` [PATCH v4 11/14] csum-file.h: introduce 'hashwrite_be64()' Taylor Blau
2020-09-04 20:18 ` René Scharfe
2020-09-04 20:22 ` Taylor Blau
2020-09-03 22:46 ` [PATCH v4 12/14] commit-graph: add large-filters bitmap chunk Taylor Blau
2020-09-03 22:46 ` [PATCH v4 13/14] commit-graph: rename 'split_commit_graph_opts' Taylor Blau
2020-09-04 15:20 ` Taylor Blau
2020-09-03 22:47 ` [PATCH v4 14/14] builtin/commit-graph.c: introduce '--max-new-filters=<n>' Taylor Blau
2020-09-04 14:39 ` [PATCH v4 00/14] more miscellaneous Bloom filter improvements Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200814202021.GE30103@syl.lan \
--to=me@ttaylorr.com \
--cc=dstolee@microsoft.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).