From: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
To: pclouds@gmail.com
Cc: git@vger.kernel.org, Eric Sunshine <sunshine@sunshineco.com>,
Jeff Hostetler <git@jeffhostetler.com>,
Junio C Hamano <gitster@pobox.com>
Subject: [PATCH v4 4/7] gc: add gc.bigPackThreshold config
Date: Sat, 24 Mar 2018 08:42:32 +0100 [thread overview]
Message-ID: <20180324074308.18934-8-pclouds@gmail.com> (raw)
In-Reply-To: <20180324074308.18934-1-pclouds@gmail.com>
The --keep-largest-pack option is not very convenient to use because
you need to tell gc to do this explicitly (and probably on just a few
large repos).
Add a config key that enables this mode when packs larger than a limit
are found. Note that there's a slight behavior difference compared to
--keep-largest-pack: all packs larger than the threshold are kept, not
just the largest one.
Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
---
Documentation/config.txt | 8 ++++++++
Documentation/git-gc.txt | 6 ++++--
builtin/gc.c | 26 ++++++++++++++++++++------
3 files changed, 32 insertions(+), 8 deletions(-)
diff --git a/Documentation/config.txt b/Documentation/config.txt
index ce9102cea8..d63db3f12c 100644
--- a/Documentation/config.txt
+++ b/Documentation/config.txt
@@ -1558,6 +1558,14 @@ gc.autoDetach::
Make `git gc --auto` return immediately and run in background
if the system supports it. Default is true.
+gc.bigPackThreshold::
+ If non-zero, all packs larger than this limit are kept when
+ `git gc` is run. This is very similar to `--keep-base-pack`
+ except that all packs that meet the threshold are kept, not
+ just the base pack. Defaults to zero. Common unit suffixes of
+ 'k', 'm', or 'g' are supported.
+
+
gc.logExpiry::
If the file gc.log exists, then `git gc --auto` won't run
unless that file is more than 'gc.logExpiry' old. Default is
diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt
index bf81b8de30..0adf381b52 100644
--- a/Documentation/git-gc.txt
+++ b/Documentation/git-gc.txt
@@ -55,7 +55,8 @@ all loose objects are combined into a single pack using
disables automatic packing of loose objects.
+
If the number of packs exceeds the value of `gc.autoPackLimit`,
-then existing packs (except those marked with a `.keep` file)
+then existing packs (except those marked with a `.keep` file
+or over `gc.bigPackThreshold` limit)
are consolidated into a single pack by using the `-A` option of
'git repack'. Setting `gc.autoPackLimit` to 0 disables
automatic consolidation of packs.
@@ -80,7 +81,8 @@ automatic consolidation of packs.
--keep-largest-pack::
All packs except the largest pack and those marked with a
- `.keep` files are consolidated into a single pack.
+ `.keep` files are consolidated into a single pack. When this
+ option is used, `gc.bigPackThreshold` is ignored.
Configuration
-------------
diff --git a/builtin/gc.c b/builtin/gc.c
index 9a09cf53b0..53a0500898 100644
--- a/builtin/gc.c
+++ b/builtin/gc.c
@@ -39,6 +39,7 @@ static timestamp_t gc_log_expire_time;
static const char *gc_log_expire = "1.day.ago";
static const char *prune_expire = "2.weeks.ago";
static const char *prune_worktrees_expire = "3.months.ago";
+static unsigned long big_pack_threshold;
static struct argv_array pack_refs_cmd = ARGV_ARRAY_INIT;
static struct argv_array reflog = ARGV_ARRAY_INIT;
@@ -126,6 +127,8 @@ static void gc_config(void)
git_config_get_expiry("gc.worktreepruneexpire", &prune_worktrees_expire);
git_config_get_expiry("gc.logexpiry", &gc_log_expire);
+ git_config_get_ulong("gc.bigpackthreshold", &big_pack_threshold);
+
git_config(git_default_config, NULL);
}
@@ -164,7 +167,7 @@ static int too_many_loose_objects(void)
return needed;
}
-static void find_base_packs(struct string_list *packs)
+static void find_base_packs(struct string_list *packs, unsigned long limit)
{
struct packed_git *p, *base = NULL;
@@ -173,7 +176,10 @@ static void find_base_packs(struct string_list *packs)
for (p = packed_git; p; p = p->next) {
if (!p->pack_local)
continue;
- if (!base || base->pack_size < p->pack_size) {
+ if (limit) {
+ if (p->pack_size >= limit)
+ string_list_append(packs, p->pack_name);
+ } else if (!base || base->pack_size < p->pack_size) {
base = p;
}
}
@@ -245,9 +251,15 @@ static int need_to_gc(void)
* we run "repack -A -d -l". Otherwise we tell the caller
* there is no need.
*/
- if (too_many_packs())
- add_repack_all_option(NULL);
- else if (too_many_loose_objects())
+ if (too_many_packs()) {
+ struct string_list keep_pack = STRING_LIST_INIT_NODUP;
+
+ if (big_pack_threshold)
+ find_base_packs(&keep_pack, big_pack_threshold);
+
+ add_repack_all_option(&keep_pack);
+ string_list_clear(&keep_pack, 0);
+ } else if (too_many_loose_objects())
add_repack_incremental_option();
else
return 0;
@@ -462,7 +474,9 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
if (keep_base_pack != -1) {
if (keep_base_pack)
- find_base_packs(&keep_pack);
+ find_base_packs(&keep_pack, 0);
+ } else if (big_pack_threshold) {
+ find_base_packs(&keep_pack, big_pack_threshold);
}
add_repack_all_option(&keep_pack);
--
2.17.0.rc0.348.gd5a49e0b6f
next prev parent reply other threads:[~2018-03-24 7:43 UTC|newest]
Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-17 7:53 [PATCH 00/36] Combine t/helper binaries into a single one Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 01/36] t/helper: add an empty test-tool program Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 02/36] t/helper: merge test-chmtime into test-tool Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 03/36] t/helper: merge test-sha1 " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 04/36] t/helper: merge test-lazy-init-name-hash " Nguyễn Thái Ngọc Duy
2018-03-18 2:11 ` Eric Sunshine
2018-03-18 8:25 ` Duy Nguyen
2018-03-18 8:47 ` Eric Sunshine
2018-03-19 9:40 ` Jeff Hostetler
2018-03-19 15:51 ` Duy Nguyen
2018-03-17 7:53 ` [PATCH 05/36] t/helper: merge test-config " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 06/36] t/helper: merge test-ctype " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 07/36] t/helper: merge test-date " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 08/36] t/helper: merge (unused) test-delta " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 09/36] t/helper: merge test-drop-caches " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 10/36] t/helper: merge test-dump-cache-tree " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 11/36] t/helper: merge test-dump-split-index " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 12/36] t/helper: merge test-example-decorate " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 13/36] t/helper: merge test-genrandom " Nguyễn Thái Ngọc Duy
2018-03-17 7:53 ` [PATCH 14/36] t/helper: merge test-hashmap " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 15/36] t/helper: merge test-index-version " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 16/36] t/helper: merge (unused) test-match-trees " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 17/36] t/helper: merge (unused) test-mergesort " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 18/36] t/helper: merge test-mktemp " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 19/36] t/helper: merge test-online-cpus " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 20/36] t/helper: merge test-path-utils " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 21/36] t/helper: merge test-prio-queue " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 22/36] t/helepr: merge test-read-cache " Nguyễn Thái Ngọc Duy
2018-03-18 2:21 ` Eric Sunshine
2018-03-17 7:54 ` [PATCH 23/36] t/helper: merge test-ref-store " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 24/36] t/helper: merge test-regex " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 25/36] t/helper: merge test-revision-walking " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 26/36] t/helper: merge test-run-command " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 27/36] t/helper: merge test-scrap-cache-tree " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 28/36] t/helper: merge test-sha1-array " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 29/36] t/helper: merge test-sigchain " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 30/36] t/helper: merge test-strcmp-offset " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 31/36] t/helper: merge test-string-list " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 32/36] t/helper: merge test-submodule-config " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 33/36] t/helper: merge test-subprocess " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 34/36] t/helper: merge test-urlmatch-.. " Nguyễn Thái Ngọc Duy
2018-03-18 2:32 ` Eric Sunshine
2018-03-17 7:54 ` [PATCH 35/36] t/helper: merge test-wildmatch " Nguyễn Thái Ngọc Duy
2018-03-17 7:54 ` [PATCH 36/36] t/helper: merge test-write-cache " Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v2 00/36] Combine t/helper binaries into a single one Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v4 1/7] t7700: have closing quote of a test at the beginning of line Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v2 01/36] t/helper: add an empty test-tool program Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v4 2/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v2 02/36] t/helper: merge test-chmtime into test-tool Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v4 3/7] gc: add --keep-largest-pack option Nguyễn Thái Ngọc Duy
2018-03-24 9:36 ` Ævar Arnfjörð Bjarmason
2018-03-24 12:13 ` Duy Nguyen
2018-03-26 2:05 ` Junio C Hamano
2018-03-24 7:42 ` [PATCH v2 03/36] t/helper: merge test-sha1 into test-tool Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` Nguyễn Thái Ngọc Duy [this message]
2018-03-24 7:42 ` [PATCH v2 04/36] t/helper: merge test-lazy-init-name-hash " Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v4 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v2 05/36] t/helper: merge test-config into test-tool Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v4 6/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v2 06/36] t/helper: merge test-ctype into test-tool Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v4 7/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-24 7:42 ` [PATCH v2 07/36] t/helper: merge test-date into test-tool Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 00/36] Combine t/helper binaries into a single one Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 01/36] t/helper: add an empty test-tool program Nguyễn Thái Ngọc Duy
2018-03-26 15:27 ` Johannes Schindelin
2018-03-26 17:09 ` Duy Nguyen
2018-03-26 22:14 ` Johannes Schindelin
2018-03-26 23:07 ` SZEDER Gábor
2018-03-27 13:57 ` Johannes Schindelin
2018-03-24 7:44 ` [PATCH v2 02/36] t/helper: merge test-chmtime into test-tool Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 03/36] t/helper: merge test-sha1 " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 04/36] t/helper: merge test-lazy-init-name-hash " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 05/36] t/helper: merge test-config " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 06/36] t/helper: merge test-ctype " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 07/36] t/helper: merge test-date " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 08/36] t/helper: merge (unused) test-delta " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 09/36] t/helper: merge test-drop-caches " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 10/36] t/helper: merge test-dump-cache-tree " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 11/36] t/helper: merge test-dump-split-index " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 12/36] t/helper: merge test-example-decorate " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 13/36] t/helper: merge test-genrandom " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 14/36] t/helper: merge test-hashmap " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 15/36] t/helper: merge test-index-version " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 16/36] t/helper: merge (unused) test-match-trees " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 17/36] t/helper: merge (unused) test-mergesort " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 18/36] t/helper: merge test-mktemp " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 19/36] t/helper: merge test-online-cpus " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 20/36] t/helper: merge test-path-utils " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 21/36] t/helper: merge test-prio-queue " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 22/36] t/helper: merge test-read-cache " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 23/36] t/helper: merge test-ref-store " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 24/36] t/helper: merge test-regex " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 25/36] t/helper: merge test-revision-walking " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 26/36] t/helper: merge test-run-command " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 27/36] t/helper: merge test-scrap-cache-tree " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 28/36] t/helper: merge test-sha1-array " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 29/36] t/helper: merge test-sigchain " Nguyễn Thái Ngọc Duy
2018-03-24 7:44 ` [PATCH v2 30/36] t/helper: merge test-strcmp-offset " Nguyễn Thái Ngọc Duy
2018-03-24 7:45 ` [PATCH v2 31/36] t/helper: merge test-string-list " Nguyễn Thái Ngọc Duy
2018-03-24 7:45 ` [PATCH v2 32/36] t/helper: merge test-submodule-config " Nguyễn Thái Ngọc Duy
2018-03-24 7:45 ` [PATCH v2 33/36] t/helper: merge test-subprocess " Nguyễn Thái Ngọc Duy
2018-03-24 7:45 ` [PATCH v2 34/36] t/helper: merge test-urlmatch-normalization " Nguyễn Thái Ngọc Duy
2018-03-24 7:45 ` [PATCH v2 35/36] t/helper: merge test-wildmatch " Nguyễn Thái Ngọc Duy
2018-03-24 7:45 ` [PATCH v2 36/36] t/helper: merge test-write-cache " Nguyễn Thái Ngọc Duy
2018-03-24 12:50 ` [PATCH v2 00/36] Combine t/helper binaries into a single one Ævar Arnfjörð Bjarmason
2018-03-24 12:57 ` Duy Nguyen
2018-03-26 2:16 ` Junio C Hamano
2018-03-27 14:00 ` Johannes Schindelin
2018-03-27 15:42 ` Junio C Hamano
2018-03-27 16:12 ` Johannes Schindelin
-- strict thread matches above, loose matches on Subject: below --
2018-03-16 19:27 [PATCH v3 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 " Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 4/7] gc: add gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180324074308.18934-8-pclouds@gmail.com \
--to=pclouds@gmail.com \
--cc=git@jeffhostetler.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=sunshine@sunshineco.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).