From: Abhishek Kumar <abhishekkumar8222@gmail.com>
To: Derrick Stolee <stolee@gmail.com>
Cc: abhishekkumar8222@gmail.com, git@vger.kernel.org,
gitgitgadget@gmail.com, jnareb@gmail.com, me@ttaylor.com
Subject: Re: [PATCH v5 01/11] commit-graph: fix regression when computing Bloom filters
Date: Fri, 8 Jan 2021 11:15:57 +0530 [thread overview]
Message-ID: <X/fxlVc8UK7FQRpP@Abhishek-Arch> (raw)
In-Reply-To: <1694688e-0253-9d67-1982-8ce483183162@gmail.com>
On Tue, Dec 29, 2020 at 08:35:56PM -0500, Derrick Stolee wrote:
> On 12/28/2020 6:15 AM, Abhishek Kumar via GitGitGadget wrote:
> > From: Abhishek Kumar <abhishekkumar8222@gmail.com>
> >
> > Before computing Bloom fitlers, the commit-graph machinery uses
>
> s/fitlers/filters/
>
> > commit_gen_cmp to sort commits by generation order for improved diff
> > performance. 3d11275505 (commit-graph: examine commits by generation
> > number, 2020-03-30) claims that this sort can reduce the time spent to
> > compute Bloom filters by nearly half.
> >
> > But since c49c82aa4c (commit: move members graph_pos, generation to a
> > slab, 2020-06-17), this optimization is broken, since asking for a
> > 'commit_graph_generation()' directly returns GENERATION_NUMBER_INFINITY
> > while writing.
> >
> > Not all hope is lost, though: 'commit_graph_generation()' falls back to
> > comparing commits by their date when they have equal generation number,
> > and so since c49c82aa4c is purely a date comparision function. This
>
> s/comparision/comparison/
>
> > heuristic is good enough that we don't seem to loose appreciable
> > performance while computing Bloom filters. Applying this patch (compared
> > with v2.29.1) speeds up computing Bloom filters by around ~4
> > seconds.
>
> Using "~4 seconds" here is odd since there is no baseline. Which
> repository did you use?
>
I used the linux repository, will mention that.
> Previous discussion used relative terms. Something like "speeds up by
> a factor of 1.25" or something might be interesting.
>
As SZEDER Gábor found, the improvements are rather minor - ranging from
0.40% to 5.19% [1]. I want to make sure this is the correct way to word
in the commit message:
Applying this patch (compared with v2.30.0) speeds up computing Bloom
filters by factors ranging from 0.40% to 5.19% on various
repositories.
https://lore.kernel.org/git/20210105094535.GN8396@szeder.dev/
> > So, avoid the useless 'commit_graph_generation()' while writing by
> > instead accessing the slab directly. This returns the newly-computed
> > generation numbers, and allows us to avoid the heuristic by directly
> > comparing generation numbers.
>
> This introduces some timing restrictions to the ability for this
> comparison function. It would be dangerous if someone extracted
> the method for another purpose. A comment above these lines could
> warn future developers from making that mistake, but they would
> probably use the comparison functions in commit.c instead.
>
Sure, will add a comment above.
> > Signed-off-by: Abhishek Kumar <abhishekkumar8222@gmail.com>
> > ---
> > commit-graph.c | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/commit-graph.c b/commit-graph.c
> > index 06f8dc1d896..caf823295f4 100644
> > --- a/commit-graph.c
> > +++ b/commit-graph.c
> > @@ -144,8 +144,8 @@ static int commit_gen_cmp(const void *va, const void *vb)
> > const struct commit *a = *(const struct commit **)va;
> > const struct commit *b = *(const struct commit **)vb;
> >
> > - uint32_t generation_a = commit_graph_generation(a);
> > - uint32_t generation_b = commit_graph_generation(b);
> > + uint32_t generation_a = commit_graph_data_at(a)->generation;
> > + uint32_t generation_b = commit_graph_data_at(b)->generation;
> > /* lower generation commits first */
> > if (generation_a < generation_b)
> > return -1;
> >
>
next prev parent reply other threads:[~2021-01-08 5:47 UTC|newest]
Thread overview: 211+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-28 9:13 [PATCH 0/6] [GSoC] Implement Corrected Commit Date Abhishek Kumar via GitGitGadget
2020-07-28 9:13 ` [PATCH 1/6] commit-graph: fix regression when computing bloom filter Abhishek Kumar via GitGitGadget
2020-07-28 15:28 ` Taylor Blau
2020-07-30 5:24 ` Abhishek Kumar
2020-08-04 0:46 ` Jakub Narębski
2020-08-04 0:56 ` Taylor Blau
2020-08-04 10:10 ` Jakub Narębski
2020-08-04 7:55 ` Jakub Narębski
2020-07-28 9:13 ` [PATCH 2/6] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2020-07-28 13:00 ` Derrick Stolee
2020-07-28 15:30 ` Taylor Blau
2020-08-05 23:16 ` Jakub Narębski
2020-07-28 9:13 ` [PATCH 3/6] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2020-07-28 13:14 ` Derrick Stolee
2020-07-28 15:19 ` René Scharfe
2020-07-28 15:58 ` Derrick Stolee
2020-07-28 16:01 ` Taylor Blau
2020-07-30 6:07 ` Abhishek Kumar
2020-07-28 9:13 ` [PATCH 4/6] commit-graph: consolidate compare_commits_by_gen Abhishek Kumar via GitGitGadget
2020-07-28 16:03 ` Taylor Blau
2020-07-28 9:13 ` [PATCH 5/6] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2020-07-28 16:12 ` Taylor Blau
2020-07-30 6:52 ` Abhishek Kumar
2020-07-28 9:13 ` [PATCH 6/6] commit-graph: implement corrected commit date offset Abhishek Kumar via GitGitGadget
2020-07-28 15:55 ` Derrick Stolee
2020-07-28 16:23 ` Taylor Blau
2020-07-30 7:27 ` Abhishek Kumar
2020-07-28 14:54 ` [PATCH 0/6] [GSoC] Implement Corrected Commit Date Taylor Blau
2020-07-30 7:47 ` Abhishek Kumar
2020-07-28 16:35 ` Derrick Stolee
2020-08-09 2:53 ` [PATCH v2 00/10] " Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 01/10] commit-graph: fix regression when computing bloom filter Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 02/10] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 03/10] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 04/10] commit-graph: consolidate compare_commits_by_gen Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 05/10] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2020-08-10 16:28 ` Derrick Stolee
2020-08-11 11:03 ` Abhishek Kumar
2020-08-11 12:27 ` Derrick Stolee
2020-08-11 18:58 ` Taylor Blau
2020-08-09 2:53 ` [PATCH v2 06/10] commit-graph: return 64-bit generation number Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 07/10] commit-graph: implement corrected commit date Abhishek Kumar via GitGitGadget
2020-08-10 14:23 ` Derrick Stolee
2020-08-14 4:59 ` Abhishek Kumar
2020-08-14 12:24 ` Derrick Stolee
2020-08-09 2:53 ` [PATCH v2 08/10] commit-graph: handle mixed generation commit chains Abhishek Kumar via GitGitGadget
2020-08-10 16:42 ` Derrick Stolee
2020-08-11 11:36 ` Abhishek Kumar
2020-08-11 12:43 ` Derrick Stolee
2020-08-09 2:53 ` [PATCH v2 09/10] commit-reach: use corrected commit dates in paint_down_to_common() Abhishek Kumar via GitGitGadget
2020-08-09 2:53 ` [PATCH v2 10/10] doc: add corrected commit date info Abhishek Kumar via GitGitGadget
2020-08-10 16:47 ` [PATCH v2 00/10] [GSoC] Implement Corrected Commit Date Derrick Stolee
2020-08-15 16:39 ` [PATCH v3 00/11] " Abhishek Kumar via GitGitGadget
2020-08-15 16:39 ` [PATCH v3 01/11] commit-graph: fix regression when computing bloom filter Abhishek Kumar via GitGitGadget
2020-08-17 22:30 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 02/11] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2020-08-18 14:18 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 03/11] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2020-08-19 17:54 ` Jakub Narębski
2020-08-21 4:11 ` Abhishek Kumar
2020-08-25 11:11 ` Jakub Narębski
2020-09-01 11:35 ` Abhishek Kumar
2020-08-15 16:39 ` [PATCH v3 04/11] commit-graph: consolidate compare_commits_by_gen Abhishek Kumar via GitGitGadget
2020-08-17 13:22 ` Derrick Stolee
2020-08-21 11:05 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 05/11] commit-graph: return 64-bit generation number Abhishek Kumar via GitGitGadget
2020-08-21 13:14 ` Jakub Narębski
2020-08-25 5:04 ` Abhishek Kumar
2020-08-25 12:18 ` Jakub Narębski
2020-09-01 12:06 ` Abhishek Kumar
2020-09-03 13:42 ` Jakub Narębski
2020-09-05 17:21 ` Abhishek Kumar
2020-09-13 15:39 ` Jakub Narębski
2020-09-28 21:48 ` Jakub Narębski
2020-10-05 5:25 ` Abhishek Kumar
2020-08-15 16:39 ` [PATCH v3 06/11] commit-graph: add a slab to store topological levels Abhishek Kumar via GitGitGadget
2020-08-21 18:43 ` Jakub Narębski
2020-08-25 6:14 ` Abhishek Kumar
2020-08-25 7:33 ` Jakub Narębski
2020-08-25 7:56 ` Jakub Narębski
2020-09-01 10:26 ` Abhishek Kumar
2020-09-03 9:25 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 07/11] commit-graph: implement corrected commit date Abhishek Kumar via GitGitGadget
2020-08-22 0:05 ` Jakub Narębski
2020-08-25 6:49 ` Abhishek Kumar
2020-08-25 10:07 ` Jakub Narębski
2020-09-01 11:01 ` Abhishek Kumar
2020-08-15 16:39 ` [PATCH v3 08/11] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2020-08-22 13:09 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 09/11] commit-graph: use generation v2 only if entire chain does Abhishek Kumar via GitGitGadget
2020-08-22 17:14 ` Jakub Narębski
2020-08-26 7:15 ` Abhishek Kumar
2020-08-26 10:38 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 10/11] commit-reach: use corrected commit dates in paint_down_to_common() Abhishek Kumar via GitGitGadget
2020-08-22 19:09 ` Jakub Narębski
2020-09-01 10:08 ` Abhishek Kumar
2020-09-03 19:11 ` Jakub Narębski
2020-08-15 16:39 ` [PATCH v3 11/11] doc: add corrected commit date info Abhishek Kumar via GitGitGadget
2020-08-22 22:20 ` Jakub Narębski
2020-08-27 6:39 ` Abhishek Kumar
2020-08-27 12:43 ` Jakub Narębski
2020-08-27 13:15 ` Derrick Stolee
2020-09-01 13:01 ` Abhishek Kumar
2020-08-17 0:13 ` [PATCH v3 00/11] [GSoC] Implement Corrected Commit Date Jakub Narębski
[not found] ` <CANQwDwdKp7oKy9BeKdvKhwPUiq0R5MS8TCw-eWGCYCoMGv=G-g@mail.gmail.com>
2020-08-17 1:32 ` Fwd: " Taylor Blau
2020-08-17 7:56 ` Jakub Narębski
2020-08-18 6:12 ` Abhishek Kumar
2020-08-23 15:27 ` Jakub Narębski
2020-08-24 2:49 ` Abhishek Kumar
2020-10-07 14:09 ` [PATCH v4 00/10] " Abhishek Kumar via GitGitGadget
2020-10-07 14:09 ` [PATCH v4 01/10] commit-graph: fix regression when computing Bloom filters Abhishek Kumar via GitGitGadget
2020-10-24 23:16 ` Jakub Narębski
2020-10-25 20:58 ` Taylor Blau
2020-11-03 5:36 ` Abhishek Kumar
2020-10-07 14:09 ` [PATCH v4 02/10] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2020-10-24 23:41 ` Jakub Narębski
2020-10-07 14:09 ` [PATCH v4 03/10] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2020-10-25 10:52 ` Jakub Narębski
2020-10-27 6:33 ` Abhishek Kumar
2020-10-07 14:09 ` [PATCH v4 04/10] commit-graph: return 64-bit generation number Abhishek Kumar via GitGitGadget
2020-10-25 13:48 ` Jakub Narębski
2020-11-03 6:40 ` Abhishek Kumar
2020-10-07 14:09 ` [PATCH v4 05/10] commit-graph: add a slab to store topological levels Abhishek Kumar via GitGitGadget
2020-10-25 22:17 ` Jakub Narębski
2020-10-07 14:09 ` [PATCH v4 06/10] commit-graph: implement corrected commit date Abhishek Kumar via GitGitGadget
2020-10-27 18:53 ` Jakub Narębski
2020-11-03 11:44 ` Abhishek Kumar
2020-11-04 16:45 ` Jakub Narębski
2020-11-05 14:05 ` Philip Oakley
2020-11-05 18:22 ` Junio C Hamano
2020-11-06 18:26 ` Extending and updating gitglossary (was: Re: [PATCH v4 06/10] commit-graph: implement corrected commit date) Jakub Narębski
2020-11-06 19:33 ` Extending and updating gitglossary Junio C Hamano
2020-11-08 17:23 ` Extending and updating gitglossary (was: Re: [PATCH v4 06/10] commit-graph: implement corrected commit date) Philip Oakley
2020-11-10 1:35 ` Extending and updating gitglossary Jakub Narębski
2020-11-10 14:04 ` Philip Oakley
2020-11-10 23:52 ` Jakub Narębski
2020-10-07 14:09 ` [PATCH v4 07/10] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2020-10-30 12:45 ` Jakub Narębski
2020-11-06 11:25 ` Abhishek Kumar
2020-11-06 17:56 ` Jakub Narębski
2020-10-07 14:09 ` [PATCH v4 08/10] commit-graph: use generation v2 only if entire chain does Abhishek Kumar via GitGitGadget
2020-11-01 0:55 ` Jakub Narębski
2020-11-12 10:01 ` Abhishek Kumar
2020-11-13 9:59 ` Jakub Narębski
2020-10-07 14:09 ` [PATCH v4 09/10] commit-reach: use corrected commit dates in paint_down_to_common() Abhishek Kumar via GitGitGadget
2020-11-03 17:59 ` Jakub Narębski
2020-11-03 18:19 ` Junio C Hamano
2020-11-20 10:33 ` Abhishek Kumar
2020-10-07 14:09 ` [PATCH v4 10/10] doc: add corrected commit date info Abhishek Kumar via GitGitGadget
2020-11-04 1:37 ` Jakub Narębski
2020-11-21 6:30 ` Abhishek Kumar
2020-11-04 23:37 ` [PATCH v4 00/10] [GSoC] Implement Corrected Commit Date Jakub Narębski
2020-11-22 5:31 ` Abhishek Kumar
2020-12-28 11:15 ` [PATCH v5 00/11] " Abhishek Kumar via GitGitGadget
2020-12-28 11:15 ` [PATCH v5 01/11] commit-graph: fix regression when computing Bloom filters Abhishek Kumar via GitGitGadget
2020-12-30 1:35 ` Derrick Stolee
2021-01-08 5:45 ` Abhishek Kumar [this message]
2021-01-05 9:45 ` SZEDER Gábor
2021-01-05 9:47 ` SZEDER Gábor
2021-01-08 5:51 ` Abhishek Kumar
2020-12-28 11:15 ` [PATCH v5 02/11] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 03/11] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 04/11] t6600-test-reach: generalize *_three_modes Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 05/11] commit-graph: add a slab to store topological levels Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 06/11] commit-graph: return 64-bit generation number Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 07/11] commit-graph: implement corrected commit date Abhishek Kumar via GitGitGadget
2020-12-30 1:53 ` Derrick Stolee
2021-01-10 12:21 ` Abhishek Kumar
2020-12-28 11:16 ` [PATCH v5 08/11] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 09/11] commit-graph: use generation v2 only if entire chain does Abhishek Kumar via GitGitGadget
2020-12-30 3:23 ` Derrick Stolee
2021-01-10 13:13 ` Abhishek Kumar
2021-01-11 12:43 ` Derrick Stolee
2020-12-28 11:16 ` [PATCH v5 10/11] commit-reach: use corrected commit dates in paint_down_to_common() Abhishek Kumar via GitGitGadget
2020-12-28 11:16 ` [PATCH v5 11/11] doc: add corrected commit date info Abhishek Kumar via GitGitGadget
2020-12-30 4:35 ` [PATCH v5 00/11] [GSoC] Implement Corrected Commit Date Derrick Stolee
2021-01-10 14:06 ` Abhishek Kumar
2021-01-16 18:11 ` [PATCH v6 " Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 01/11] commit-graph: fix regression when computing Bloom filters Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 02/11] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 03/11] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 04/11] t6600-test-reach: generalize *_three_modes Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 05/11] commit-graph: add a slab to store topological levels Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 06/11] commit-graph: return 64-bit generation number Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 07/11] commit-graph: implement corrected commit date Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 08/11] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 09/11] commit-graph: use generation v2 only if entire chain does Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 10/11] commit-reach: use corrected commit dates in paint_down_to_common() Abhishek Kumar via GitGitGadget
2021-01-16 18:11 ` [PATCH v6 11/11] doc: add corrected commit date info Abhishek Kumar via GitGitGadget
2021-01-27 0:04 ` SZEDER Gábor
2021-01-30 5:29 ` Abhishek Kumar
2021-01-31 1:45 ` Taylor Blau
2021-01-18 21:04 ` [PATCH v6 00/11] [GSoC] Implement Corrected Commit Date Derrick Stolee
2021-01-18 22:00 ` Taylor Blau
2021-01-23 12:11 ` Abhishek Kumar
2021-01-19 0:02 ` Junio C Hamano
2021-01-23 12:07 ` Abhishek Kumar
2021-02-01 6:58 ` [PATCH v7 " Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 01/11] commit-graph: fix regression when computing Bloom filters Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 02/11] revision: parse parent in indegree_walk_step() Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 03/11] commit-graph: consolidate fill_commit_graph_info Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 04/11] t6600-test-reach: generalize *_three_modes Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 05/11] commit-graph: add a slab to store topological levels Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 06/11] commit-graph: return 64-bit generation number Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 07/11] commit-graph: document generation number v2 Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 08/11] commit-graph: implement corrected commit date Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 09/11] commit-graph: implement generation data chunk Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 10/11] commit-graph: use generation v2 only if entire chain does Abhishek Kumar via GitGitGadget
2021-02-01 6:58 ` [PATCH v7 11/11] commit-reach: use corrected commit dates in paint_down_to_common() Abhishek Kumar via GitGitGadget
2021-02-01 13:14 ` [PATCH v7 00/11] [GSoC] Implement Corrected Commit Date Derrick Stolee
2021-02-01 18:26 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=X/fxlVc8UK7FQRpP@Abhishek-Arch \
--to=abhishekkumar8222@gmail.com \
--cc=1694688e-0253-9d67-1982-8ce483183162@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitgitgadget@gmail.com \
--cc=jnareb@gmail.com \
--cc=me@ttaylor.com \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).