From: Derrick Stolee <dstolee@microsoft.com>
To: "git@vger.kernel.org" <git@vger.kernel.org>
Cc: "stolee@gmail.com" <stolee@gmail.com>,
"jnareb@gmail.com" <jnareb@gmail.com>,
"avarab@gmail.com" <avarab@gmail.com>,
"marten.agren@gmail.com" <marten.agren@gmail.com>,
"gitster@pobox.com" <gitster@pobox.com>,
Derrick Stolee <dstolee@microsoft.com>
Subject: [PATCH v4 20/21] gc: automatically write commit-graph files
Date: Mon, 4 Jun 2018 16:53:01 +0000 [thread overview]
Message-ID: <20180604165200.29261-21-dstolee@microsoft.com> (raw)
In-Reply-To: <20180604165200.29261-1-dstolee@microsoft.com>
The commit-graph file is a very helpful feature for speeding up git
operations. In order to make it more useful, make it possible to
write the commit-graph file during standard garbage collection
operations.
Add a 'gc.commitGraph' config setting that triggers writing a
commit-graph file after any non-trivial 'git gc' command. Defaults to
false while the commit-graph feature matures. We specifically do not
want to have this on by default until the commit-graph feature is fully
integrated with history-modifying features like shallow clones.
Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
---
Documentation/config.txt | 10 +++++++++-
Documentation/git-gc.txt | 4 ++++
builtin/gc.c | 6 ++++++
t/t5318-commit-graph.sh | 14 ++++++++++++++
4 files changed, 33 insertions(+), 1 deletion(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index 11f027194e..d2eb3c8e9b 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -900,7 +900,8 @@ the `GIT_NOTES_REF` environment variable. See linkgit:git-notes[1].
core.commitGraph::
Enable git commit graph feature. Allows reading from the
- commit-graph file.
+ commit-graph file. See `gc.commitGraph` for automatically
+ maintaining the file.
core.sparseCheckout::
Enable "sparse checkout" feature. See section "Sparse checkout" in
@@ -1553,6 +1554,13 @@ gc.autoDetach::
Make `git gc --auto` return immediately and run in background
if the system supports it. Default is true.
+gc.commitGraph::
+ If true, then gc will rewrite the commit-graph file when
+ linkgit:git-gc[1] is run. When using linkgit:git-gc[1]
+ '--auto' the commit-graph will be updated if housekeeping is
+ required. Default is false. See linkgit:git-commit-graph[1]
+ for details.
+
gc.logExpiry::
If the file gc.log exists, then `git gc --auto` won't run
unless that file is more than 'gc.logExpiry' old. Default is
diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt
index 571b5a7e3c..a6526b3592 100644
--- a/Documentation/git-gc.txt
+++ b/Documentation/git-gc.txt
@@ -119,6 +119,10 @@ The optional configuration variable `gc.packRefs` determines if
it within all non-bare repos or it can be set to a boolean value.
This defaults to true.
+The optional configuration variable `gc.commitGraph` determines if
+'git gc' should run 'git commit-graph write'. This can be set to a
+boolean value. This defaults to false.
+
The optional configuration variable `gc.aggressiveWindow` controls how
much time is spent optimizing the delta compression of the objects in
the repository when the --aggressive option is specified. The larger
diff --git a/builtin/gc.c b/builtin/gc.c
index 77fa720bd0..efd214a59f 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -20,6 +20,7 @@
#include "argv-array.h"
#include "commit.h"
#include "packfile.h"
+#include "commit-graph.h"
#define FAILED_RUN "failed to run %s"
@@ -34,6 +35,7 @@ static int aggressive_depth = 50;
static int aggressive_window = 250;
static int gc_auto_threshold = 6700;
static int gc_auto_pack_limit = 50;
+static int gc_commit_graph = 0;
static int detach_auto = 1;
static timestamp_t gc_log_expire_time;
static const char *gc_log_expire = "1.day.ago";
@@ -121,6 +123,7 @@ static void gc_config(void)
git_config_get_int("gc.aggressivedepth", &aggressive_depth);
git_config_get_int("gc.auto", &gc_auto_threshold);
git_config_get_int("gc.autopacklimit", &gc_auto_pack_limit);
+ git_config_get_bool("gc.commitgraph", &gc_commit_graph);
git_config_get_bool("gc.autodetach", &detach_auto);
git_config_get_expiry("gc.pruneexpire", &prune_expire);
git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire);
@@ -480,6 +483,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
if (pack_garbage.nr > 0)
clean_pack_garbage();
+ if (gc_commit_graph)
+ write_commit_graph_reachable(get_object_directory(), 0);
+
if (auto_gc && too_many_loose_objects())
warning(_("There are too many unreachable loose objects; "
"run 'git prune' to remove them."));
diff --git a/t/t5318-commit-graph.sh b/t/t5318-commit-graph.sh
index ffb2ed7c95..b24e8b6689 100755
--- a/t/t5318-commit-graph.sh
+++ b/t/t5318-commit-graph.sh
@@ -245,6 +245,20 @@ test_expect_success 'perform fast-forward merge in full repo' '
test_cmp expect output
'
+test_expect_success 'check that gc computes commit-graph' '
+ cd "$TRASH_DIRECTORY/full" &&
+ git commit --allow-empty -m "blank" &&
+ git commit-graph write --reachable &&
+ cp $objdir/info/commit-graph commit-graph-before-gc &&
+ git reset --hard HEAD~1 &&
+ git config gc.commitGraph true &&
+ git gc &&
+ cp $objdir/info/commit-graph commit-graph-after-gc &&
+ ! test_cmp commit-graph-before-gc commit-graph-after-gc &&
+ git commit-graph write --reachable &&
+ test_cmp commit-graph-after-gc $objdir/info/commit-graph
+'
+
# the verify tests below expect the commit-graph to contain
# exactly the commits reachable from the commits/8 branch.
# If the file changes the set of commits in the list, then the
--
2.18.0.rc1
next prev parent reply other threads:[~2018-06-04 16:53 UTC|newest]
Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-04 16:52 [PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc' Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 01/21] commit-graph: UNLEAK before die() Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 02/21] commit-graph: fix GRAPH_MIN_SIZE Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 03/21] commit-graph: parse commit from chosen graph Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 04/21] commit: force commit to parse from object database Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 05/21] commit-graph: load a root tree from specific graph Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 06/21] commit-graph: add 'verify' subcommand Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 07/21] commit-graph: verify catches corrupt signature Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 08/21] commit-graph: verify required chunks are present Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 09/21] commit-graph: verify corrupt OID fanout and lookup Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 10/21] commit-graph: verify objects exist Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 11/21] commit-graph: verify root tree OIDs Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 12/21] commit-graph: verify parent list Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 13/21] commit-graph: verify generation number Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 14/21] commit-graph: verify commit date Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 15/21] commit-graph: test for corrupted octopus edge Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 16/21] commit-graph: verify contents match checksum Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 17/21] fsck: verify commit-graph Derrick Stolee
2018-06-06 11:08 ` Ævar Arnfjörð Bjarmason
2018-06-06 11:31 ` Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 18/21] commit-graph: use string-list API for input Derrick Stolee
2018-06-04 16:52 ` [PATCH v4 19/21] commit-graph: add '--reachable' option Derrick Stolee
2018-06-04 16:53 ` Derrick Stolee [this message]
2018-06-06 11:11 ` [PATCH v4 20/21] gc: automatically write commit-graph files Ævar Arnfjörð Bjarmason
2018-06-04 16:53 ` [PATCH v4 21/21] commit-graph: update design document Derrick Stolee
2018-06-04 17:02 ` [PATCH v4 00/21] Integrate commit-graph into 'fsck' and 'gc' Derrick Stolee
2018-06-05 14:51 ` Ævar Arnfjörð Bjarmason
2018-06-05 16:37 ` Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 " Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 01/21] commit-graph: UNLEAK before die() Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 03/21] commit-graph: parse commit from chosen graph Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 02/21] commit-graph: fix GRAPH_MIN_SIZE Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 04/21] commit: force commit to parse from object database Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 05/21] commit-graph: load a root tree from specific graph Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 06/21] commit-graph: add 'verify' subcommand Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 07/21] commit-graph: verify catches corrupt signature Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 08/21] commit-graph: verify required chunks are present Derrick Stolee
2018-06-06 12:21 ` Ævar Arnfjörð Bjarmason
2018-06-06 11:36 ` [PATCH v5 09/21] commit-graph: verify corrupt OID fanout and lookup Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 10/21] commit-graph: verify objects exist Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 11/21] commit-graph: verify root tree OIDs Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 13/21] commit-graph: verify generation number Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 12/21] commit-graph: verify parent list Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 14/21] commit-graph: verify commit date Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 15/21] commit-graph: test for corrupted octopus edge Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 16/21] commit-graph: verify contents match checksum Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 18/21] commit-graph: use string-list API for input Derrick Stolee
2018-06-06 12:11 ` Ævar Arnfjörð Bjarmason
2018-06-06 12:15 ` Derrick Stolee
2018-06-06 12:26 ` Ævar Arnfjörð Bjarmason
2018-06-06 12:45 ` Derrick Stolee
2018-06-08 12:48 ` Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 17/21] fsck: verify commit-graph Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 19/21] commit-graph: add '--reachable' option Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 20/21] gc: automatically write commit-graph files Derrick Stolee
2018-06-06 11:36 ` [PATCH v5 21/21] commit-graph: update design document Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 00/21] Integrate commit-graph into 'fsck' and 'gc' Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 01/21] commit-graph: UNLEAK before die() Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 02/21] commit-graph: fix GRAPH_MIN_SIZE Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 03/21] commit-graph: parse commit from chosen graph Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 04/21] commit: force commit to parse from object database Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 05/21] commit-graph: load a root tree from specific graph Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 06/21] commit-graph: add 'verify' subcommand Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 07/21] commit-graph: verify catches corrupt signature Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 08/21] commit-graph: verify required chunks are present Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 09/21] commit-graph: verify corrupt OID fanout and lookup Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 10/21] commit-graph: verify objects exist Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 12/21] commit-graph: verify parent list Derrick Stolee
2018-06-12 16:55 ` Junio C Hamano
2018-06-08 13:56 ` [PATCH v6 11/21] commit-graph: verify root tree OIDs Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 13/21] commit-graph: verify generation number Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 14/21] commit-graph: verify commit date Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 15/21] commit-graph: test for corrupted octopus edge Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 16/21] commit-graph: verify contents match checksum Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 17/21] fsck: verify commit-graph Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 19/21] commit-graph: add '--reachable' option Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 18/21] commit-graph: use string-list API for input Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 20/21] gc: automatically write commit-graph files Derrick Stolee
2018-06-08 22:24 ` SZEDER Gábor
2018-06-25 14:48 ` Derrick Stolee
2018-06-08 13:56 ` [PATCH v6 21/21] commit-graph: update design document Derrick Stolee
2018-06-08 15:05 ` [PATCH v6 00/21] Integrate commit-graph into 'fsck' and 'gc' Jakub Narębski
2018-06-08 15:11 ` Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 00/22] " Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 01/22] t5318-commit-graph.sh: use core.commitGraph Derrick Stolee
2018-06-27 17:38 ` Junio C Hamano
2018-06-27 13:24 ` [PATCH v7 02/22] commit-graph: UNLEAK before die() Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 03/22] commit-graph: fix GRAPH_MIN_SIZE Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 04/22] commit-graph: parse commit from chosen graph Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 05/22] commit: force commit to parse from object database Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 06/22] commit-graph: load a root tree from specific graph Derrick Stolee
2018-07-11 9:38 ` SZEDER Gábor
2018-07-13 16:30 ` [PATCH] coccinelle: update commit.cocci Derrick Stolee
2018-07-13 17:45 ` Eric Sunshine
2018-06-27 13:24 ` [PATCH v7 07/22] commit-graph: add 'verify' subcommand Derrick Stolee
2018-06-27 21:59 ` Jonathan Tan
2018-06-29 12:47 ` Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 08/22] commit-graph: verify catches corrupt signature Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 09/22] commit-graph: verify required chunks are present Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 10/22] commit-graph: verify corrupt OID fanout and lookup Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 11/22] commit-graph: verify objects exist Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 12/22] commit-graph: verify root tree OIDs Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 13/22] commit-graph: verify parent list Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 14/22] commit-graph: verify generation number Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 15/22] commit-graph: verify commit date Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 16/22] commit-graph: test for corrupted octopus edge Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 17/22] commit-graph: verify contents match checksum Derrick Stolee
2018-06-27 17:51 ` Junio C Hamano
2018-06-27 13:24 ` [PATCH v7 18/22] fsck: verify commit-graph Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 19/22] commit-graph: use string-list API for input Derrick Stolee
2018-06-27 13:24 ` [PATCH v7 20/22] commit-graph: add '--reachable' option Derrick Stolee
2018-06-27 17:53 ` Junio C Hamano
2018-09-11 5:22 ` Christian Couder
2018-09-11 11:19 ` Derrick Stolee
2018-09-11 15:13 ` Christian Couder
2018-06-27 13:24 ` [PATCH v7 21/22] gc: automatically write commit-graph files Derrick Stolee
2018-06-27 18:09 ` Junio C Hamano
2018-06-27 18:24 ` Derrick Stolee
2018-08-03 9:39 ` SZEDER Gábor
2018-08-12 20:18 ` [PATCH] t5318: use 'test_cmp_bin' to compare " SZEDER Gábor
2018-08-13 11:24 ` Derrick Stolee
2018-08-13 11:41 ` SZEDER Gábor
2018-08-13 11:52 ` [PATCH v2] " SZEDER Gábor
2018-06-27 13:24 ` [PATCH v7 22/22] commit-graph: update design document Derrick Stolee
2018-06-27 18:09 ` Junio C Hamano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180604165200.29261-21-dstolee@microsoft.com \
--to=dstolee@microsoft.com \
--cc=avarab@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jnareb@gmail.com \
--cc=marten.agren@gmail.com \
--cc=stolee@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).