From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Derrick Stolee <stolee@gmail.com>
Cc: git@vger.kernel.org, gitster@pobox.com, peff@peff.net,
sbeller@google.com, szeder.dev@gmail.com,
ramsay@ramsayjones.plus.com, git@jeffhostetler.com,
Derrick Stolee <dstolee@microsoft.com>
Subject: Re: [PATCH v6 00/14] Serialized Git Commit Graph
Date: Fri, 16 Mar 2018 16:06:23 +0100 [thread overview]
Message-ID: <878tasdpqo.fsf@evledraar.gmail.com> (raw)
In-Reply-To: <20180314192736.70602-1-dstolee@microsoft.com>
On Wed, Mar 14 2018, Derrick Stolee jotted:
> Hopefully this version is ready to merge. I have several follow-up topics
> in mind to submit soon after, including:
I've been doing some preliminary testing of this internally, all good so
far, on a relatively small repo (~100k commits) I was using for testing:
- git -c core.commitGraph=true -C <repo> rev-list --all:
- /mnt/ext4_graph => min:273 mean:279 max:389 -- (273 274 275 276 277 279 282 282 345 389)
- /mnt/ext4 => min:1087 mean:1123 max:1175 -- (1087 1092 1092 1104 1117 1123 1126 1136 1143 1175)
This is on a fresh clone with one giant pack and where the commit graph
data was generated afterwards with "git commit-graph write" for the
*_graph repo, so it contains all the commits.
So less than 25% of the mean time it spent before. Nice. Those are times
in milliseconds over 10 runs, for this particular one I didn't get much
of an improvement in --graph, but still ~10%:
- git -c core.commitGraph=true -C <repo> log --oneline --graph:
- /mnt/ext4_graph => min:1420 mean:1449 max:1586 -- (1420 1423 1428 1434 1449 1449 1490 1548 1567 1586)
- /mnt/ext4 => min:1547 mean:1616 max:2136 -- (1547 1557 1581 1585 1598 1616 1621 1740 1964 2136)
I noticed that it takes a *long* time to generate the graph, on a bigger
repo I have it takes 20 minutes, and this is a repo where repack -A -d
itself takes 5-8 minutes, probably on the upper end of that with the
bitmap, but once you do that it's relatively snappy with --stdin-commits
--additive when I feed it the new commits.
I don't have any need really to make this run in 10m instead of 20m,
just something I found interesting, i.e. how it compares to the repack
itself.
next prev parent reply other threads:[~2018-03-16 15:06 UTC|newest]
Thread overview: 110+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-27 2:32 [PATCH v5 00/13] Serialized Git Commit Graph Derrick Stolee
2018-02-27 2:32 ` [PATCH v5 01/13] commit-graph: add format document Derrick Stolee
2018-02-27 2:32 ` [PATCH v5 02/13] graph: add commit graph design document Derrick Stolee
2018-02-27 2:32 ` [PATCH v5 03/13] commit-graph: create git-commit-graph builtin Derrick Stolee
2018-02-27 2:32 ` [PATCH v5 04/13] csum-file: add CSUM_KEEP_OPEN flag Derrick Stolee
2018-03-12 13:55 ` Derrick Stolee
2018-03-13 21:42 ` Junio C Hamano
2018-03-14 2:26 ` Derrick Stolee
2018-03-14 17:00 ` Junio C Hamano
2018-02-27 2:32 ` [PATCH v5 05/13] commit-graph: implement write_commit_graph() Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 06/13] commit-graph: implement 'git-commit-graph write' Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 07/13] commit-graph: implement git commit-graph read Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 08/13] commit-graph: add core.commitGraph setting Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 09/13] commit-graph: close under reachability Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 10/13] commit: integrate commit graph with commit parsing Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 11/13] commit-graph: read only from specific pack-indexes Derrick Stolee
2018-02-27 20:15 ` Stefan Beller
2018-02-27 2:33 ` [PATCH v5 12/13] commit-graph: build graph from starting commits Derrick Stolee
2018-02-27 2:33 ` [PATCH v5 13/13] commit-graph: implement "--additive" option Derrick Stolee
2018-02-27 18:50 ` [PATCH v5 00/13] Serialized Git Commit Graph Stefan Beller
2018-03-14 19:27 ` [PATCH v6 00/14] " Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 01/14] csum-file: rename hashclose() to finalize_hashfile() Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 02/14] csum-file: refactor finalize_hashfile() method Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 03/14] commit-graph: add format document Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 04/14] graph: add commit graph design document Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 05/14] commit-graph: create git-commit-graph builtin Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 06/14] commit-graph: implement write_commit_graph() Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 07/14] commit-graph: implement 'git-commit-graph write' Derrick Stolee
2018-03-18 13:25 ` Ævar Arnfjörð Bjarmason
2018-03-19 13:12 ` Derrick Stolee
2018-03-19 14:36 ` Ævar Arnfjörð Bjarmason
2018-03-19 18:27 ` Derrick Stolee
2018-03-19 18:48 ` Ævar Arnfjörð Bjarmason
2018-03-14 19:27 ` [PATCH v6 08/14] commit-graph: implement git commit-graph read Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 09/14] commit-graph: add core.commitGraph setting Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 10/14] commit-graph: close under reachability Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 11/14] commit: integrate commit graph with commit parsing Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 12/14] commit-graph: read only from specific pack-indexes Derrick Stolee
2018-03-15 22:50 ` SZEDER Gábor
2018-03-19 13:13 ` Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 13/14] commit-graph: build graph from starting commits Derrick Stolee
2018-03-14 19:27 ` [PATCH v6 14/14] commit-graph: implement "--additive" option Derrick Stolee
2018-03-14 20:10 ` [PATCH v6 00/14] Serialized Git Commit Graph Ramsay Jones
2018-03-14 20:43 ` Junio C Hamano
2018-03-15 17:23 ` Johannes Schindelin
2018-03-15 18:41 ` Junio C Hamano
2018-03-15 21:51 ` Ramsay Jones
2018-03-16 11:50 ` Johannes Schindelin
2018-03-16 17:27 ` Junio C Hamano
2018-03-19 11:41 ` Johannes Schindelin
2018-03-16 16:28 ` Lars Schneider
2018-03-19 13:10 ` Derrick Stolee
2018-03-16 15:06 ` Ævar Arnfjörð Bjarmason [this message]
2018-03-16 16:38 ` SZEDER Gábor
2018-03-16 18:33 ` Junio C Hamano
2018-03-16 19:48 ` SZEDER Gábor
2018-03-16 20:06 ` Jeff King
2018-03-16 20:19 ` Jeff King
2018-03-19 12:55 ` Derrick Stolee
2018-03-20 1:17 ` Derrick Stolee
2018-03-16 20:49 ` Jeff King
2018-04-02 20:34 ` [PATCH v7 " Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 01/14] csum-file: rename hashclose() to finalize_hashfile() Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 02/14] csum-file: refactor finalize_hashfile() method Derrick Stolee
2018-04-07 22:59 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 03/14] commit-graph: add format document Derrick Stolee
2018-04-07 23:49 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 04/14] graph: add commit graph design document Derrick Stolee
2018-04-08 11:06 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 05/14] commit-graph: create git-commit-graph builtin Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 06/14] commit-graph: implement write_commit_graph() Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 07/14] commit-graph: implement git-commit-graph write Derrick Stolee
2018-04-08 11:59 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 08/14] commit-graph: implement git commit-graph read Derrick Stolee
2018-04-02 21:33 ` Junio C Hamano
2018-04-03 11:49 ` Derrick Stolee
2018-04-08 12:59 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 09/14] commit-graph: add core.commitGraph setting Derrick Stolee
2018-04-08 13:39 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 10/14] commit-graph: close under reachability Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 11/14] commit: integrate commit graph with commit parsing Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 12/14] commit-graph: read only from specific pack-indexes Derrick Stolee
2018-04-02 20:34 ` [PATCH v7 13/14] commit-graph: build graph from starting commits Derrick Stolee
2018-04-08 13:50 ` Jakub Narebski
2018-04-02 20:34 ` [PATCH v7 14/14] commit-graph: implement "--additive" option Derrick Stolee
2018-04-05 8:27 ` SZEDER Gábor
2018-04-10 12:55 ` [PATCH v8 00/14] Serialized Git Commit Graph Derrick Stolee
2018-04-10 12:55 ` [PATCH v8 01/14] csum-file: rename hashclose() to finalize_hashfile() Derrick Stolee
2018-04-10 12:55 ` [PATCH v8 02/14] csum-file: refactor finalize_hashfile() method Derrick Stolee
2018-04-10 12:55 ` [PATCH v8 03/14] commit-graph: add format document Derrick Stolee
2018-04-10 19:10 ` Stefan Beller
2018-04-10 19:18 ` Derrick Stolee
2018-04-11 20:58 ` Jakub Narebski
2018-04-12 11:28 ` Derrick Stolee
2018-04-13 22:07 ` Jakub Narebski
2018-04-10 12:55 ` [PATCH v8 04/14] graph: add commit graph design document Derrick Stolee
2018-04-15 22:48 ` Jakub Narebski
2018-04-10 12:55 ` [PATCH v8 05/14] commit-graph: create git-commit-graph builtin Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 06/14] commit-graph: implement write_commit_graph() Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 07/14] commit-graph: implement git-commit-graph write Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 08/14] commit-graph: implement git commit-graph read Derrick Stolee
2018-04-14 22:15 ` Jakub Narebski
2018-04-15 3:26 ` Eric Sunshine
2018-04-10 12:56 ` [PATCH v8 09/14] commit-graph: add core.commitGraph setting Derrick Stolee
2018-04-14 18:33 ` Jakub Narebski
2018-04-10 12:56 ` [PATCH v8 10/14] commit-graph: close under reachability Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 11/14] commit: integrate commit graph with commit parsing Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 12/14] commit-graph: read only from specific pack-indexes Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 13/14] commit-graph: build graph from starting commits Derrick Stolee
2018-04-10 12:56 ` [PATCH v8 14/14] commit-graph: implement "--append" option Derrick Stolee
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=878tasdpqo.fsf@evledraar.gmail.com \
--to=avarab@gmail.com \
--cc=dstolee@microsoft.com \
--cc=git@jeffhostetler.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=peff@peff.net \
--cc=ramsay@ramsayjones.plus.com \
--cc=sbeller@google.com \
--cc=stolee@gmail.com \
--cc=szeder.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).