git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Junio C Hamano <gitster@pobox.com>
To: "Nguyễn Thái Ngọc Duy" <pclouds@gmail.com>
Cc: e@80x24.org, git@vger.kernel.org, peff@peff.net,
	"Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Subject: Re: [PATCH v2 3/5] gc --auto: exclude base pack if not enough mem to "repack -ad"
Date: Tue, 06 Mar 2018 11:19:40 -0800	[thread overview]
Message-ID: <xmqqfu5d3tbn.fsf@gitster-ct.c.googlers.com> (raw)
In-Reply-To: <20180306104158.6541-4-pclouds@gmail.com> ("Nguyễn Thái Ngọc Duy"'s message of "Tue, 6 Mar 2018 17:41:56 +0700")

Nguyễn Thái Ngọc Duy  <pclouds@gmail.com> writes:

> On linux-2.6.git, valgrind massif reports 1.6GB heap in "pack all"
> case, and 535MB [1] in "pack all except the base pack" case. We save
> roughly 1GB memory by excluding the base pack.

;-)

> gc --auto decides to do this based on an estimation of pack-objects
> memory usage, which is quite accurate at least for the heap part, and
> whether that fits in half of system memory (the assumption here is for
> desktop environment where there are many other applications running).

I was still confused by "decides to do this..." after reading the
above three times.  If this is describing the state _with_ this
patch applied, then "Teach 'gc --auto' to do this automatically..."
would make it clear that is what is going on.

> diff --git a/Documentation/git-gc.txt b/Documentation/git-gc.txt
> index 571b5a7e3c..35ad420d5c 100644
> --- a/Documentation/git-gc.txt
> +++ b/Documentation/git-gc.txt
> @@ -59,6 +59,11 @@ then existing packs (except those marked with a `.keep` file)
>  are consolidated into a single pack by using the `-A` option of
>  'git repack'. Setting `gc.autoPackLimit` to 0 disables
>  automatic consolidation of packs.
> ++
> +If the physical amount of memory is considered not enough for `git
> +repack` to run smoothly, `--keep-base-pack` is enabled. This could be
> +overridden by setting `gc.bigBasePackThreshold` which only enables
> +`--keep-base-pack` when the base pack is larger the specified limit.

I somehow find the flow of logic in these two sentences harder to
follow than necessary.  Perhaps swapping the order may make it
easier to grok?  That is:

 - When gc.bigBasePackThreshold is set, packs larger than that will
   automatically be kept (i.e. not considered for repacking);

 - When it is not set, we try to guess how memory constrained we are,
   and behave as if the threshold were set to the size of the
   largest packfile we have (i.e. that single pack is kept).

I think another and bigger reason why I found the original hard to
read is because it is described for those who already understand
what "--keep-base-pack" option does.  Rewriting it not to require
the pre-existing knowledge of that option would make it a lot easier
to grok, I would think (you may not realize it because you wrote the
feature and are very familiar with it, though).

> +--keep-base-pack::
> +	All packs except the base pack are consolidated into a single
> +	pack. The largest pack is considered the base pack.

This makes it sound as if packs with .keep are also repacked unless
they meet the threshold for "base pack".  Is that what you actually
implemented?

In order to do so, [2/5] needs to allow the "--keep-pack" option
override the on-disk .keep files.  In an earlier message, I wondered
if such an arrangement is useful in some use cases; I think it is,
and because those who do want to see the on-disk .keep files honored
can collect and include them in the set of packs to be kept via
"--keep-pack" (after all this is an option for low-level scripting
anyway).

> +Set environment variable `GIT_TRACE` in order to see the memory usage
> +estimation in `git gc --auto` that determines whether the base pack is
> +kept.

This is somewhat a puzzling use of trace.  It sounds more like a way
to find out "how" memory usage estimation is done and arriving at a
wrong value for those who want to debug the feature.

> +static unsigned long big_base_pack_threshold;
> +static unsigned long max_delta_cache_size = DEFAULT_DELTA_CACHE_SIZE;

A new symbol, which is a good addition.

> +static struct packed_git *find_the_base_pack(void)
> +{
> +	struct packed_git *p, *base = NULL;
> +
> +	prepare_packed_git();
> +
> +	for (p = packed_git; p; p = p->next) {
> +		if (p->pack_local &&
> +		    (!base || base->pack_size < p->pack_size))
> +			base = p;
> +	}
> +
> +	return base;
> +}

This is finding the largest pack.

> @@ -187,7 +211,101 @@ static int too_many_packs(void)
>  	return gc_auto_pack_limit < cnt;
>  }
>  
> -static void add_repack_all_option(void)
> +static inline unsigned long total_ram(void)

"inline"?  We'd rather have compiler figure it out, no?

> +{
> +	unsigned long default_ram = 4;

4 what?  4 bytes?  Name it perhaps "default_ram_gb" or something?

> +#ifdef HAVE_SYSINFO
> +	struct sysinfo si;
> +
> +	if (!sysinfo(&si))
> +		return si.totalram;
> +#elif defined(HAVE_BSD_SYSCTL) && defined(HW_MEMSIZE)
> +	int64_t physical_memory;
> +	int mib[2];
> +	size_t length;
> +
> +	mib[0] = CTL_HW;
> +	mib[1] = HW_MEMSIZE;
> +	length = sizeof(int64_t);
> +	if (!sysctl(mib, 2, &physical_memory, &length, NULL, 0))
> +		return physical_memory;
> +#elif defined(GIT_WINDOWS_NATIVE)
> +	MEMORYSTATUSEX memInfo;
> +
> +	memInfo.dwLength = sizeof(MEMORYSTATUSEX);
> +	if (GlobalMemoryStatusEx(&memInfo))
> +		return memInfo;ullTotalPhys;

Is this legal C in Microsoft land?

> +#else
> +	fprintf(stderr, _("unrecognized platform, assuming %lu GB RAM\n"),
> +		default_ram);
> +#endif
> +	return default_ram * 1024 * 1024 * 1024;
> +}

I wonder if the above should go somewhere under compat/ without
ifdef but split into separate files for individual archs (I do not
know the answer to this question).

> +static void add_repack_all_option(struct packed_git *keep_pack)
>  {
>  	if (prune_expire && !strcmp(prune_expire, "now"))
>  		argv_array_push(&repack, "-a");
> @@ -196,6 +314,10 @@ static void add_repack_all_option(void)
>  		if (prune_expire)
>  			argv_array_pushf(&repack, "--unpack-unreachable=%s", prune_expire);
>  	}
> +
> +	if (keep_pack)
> +		argv_array_pushf(&repack, "--keep-pack=%s",
> +				 basename(keep_pack->pack_name));
>  }
>  
>  static void add_repack_incremental_option(void)
> @@ -218,9 +340,14 @@ static int need_to_gc(void)
>  	 * we run "repack -A -d -l".  Otherwise we tell the caller
>  	 * there is no need.
>  	 */
> -	if (too_many_packs())
> -		add_repack_all_option();
> -	else if (too_many_loose_objects())
> +	if (too_many_packs()) {
> +		struct packed_git *exclude = find_the_base_pack();
> +
> +		if (!pack_objects_uses_too_much_memory(exclude))
> +			exclude = NULL;
> +
> +		add_repack_all_option(exclude);

OK, so we possibly exclude the largest one or nothing (i.e. at most
one --keep-pack is given) here.

> @@ -353,6 +480,7 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
>  	const char *name;
>  	pid_t pid;
>  	int daemonized = 0;
> +	int keep_base_pack = -1;
>  
>  	struct option builtin_gc_options[] = {
>  		OPT__QUIET(&quiet, N_("suppress progress reporting")),
> @@ -362,6 +490,8 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
>  		OPT_BOOL(0, "aggressive", &aggressive, N_("be more thorough (increased runtime)")),
>  		OPT_BOOL(0, "auto", &auto_gc, N_("enable auto-gc mode")),
>  		OPT_BOOL(0, "force", &force, N_("force running gc even if there may be another gc running")),
> +		OPT_BOOL(0, "keep-base-pack", &keep_base_pack,
> +			 N_("repack all other packs except the base pack")),
>  		OPT_END()
>  	};
>  
> @@ -427,8 +557,19 @@ int cmd_gc(int argc, const char **argv, const char *prefix)
>  			 */
>  			daemonized = !daemonize();
>  		}
> -	} else
> -		add_repack_all_option();
> +	} else {
> +		struct packed_git *base_pack = find_the_base_pack();
> +		struct packed_git *exclude = NULL;
> +
> +		if (keep_base_pack != -1) {
> +			if (keep_base_pack)
> +				exclude = base_pack;

OK, --keep-base-pack option always wins if given...

> +		} else if (base_pack && big_base_pack_threshold &&
> +			   base_pack->pack_size >= big_base_pack_threshold)
> +			exclude = base_pack;

...and then if the largest one is larger than the threshold, it (and
it alone) is kept, but otherwise nothing is kept automatically.

But to those who say "packs larger than this value is too big" via
configuration, keeping only the largest of these above-threshold
packs would look counter-intuitive, wouldn't it, I wonder?

> +		add_repack_all_option(exclude);
> +	}

> diff --git a/t/t6500-gc.sh b/t/t6500-gc.sh
> index 41b0be575d..863fdbb0fd 100755
> --- a/t/t6500-gc.sh
> +++ b/t/t6500-gc.sh
> @@ -5,6 +5,13 @@ test_description='basic git gc tests
>  
>  . ./test-lib.sh
>  
> +test_expect_success 'setup' '
> +	# do not let the amount of physical memory affects gc
> +	# behavior, make sure the pack_objects_uses_too_much_memory()
> +	# always returns false
> +	git config gc.bigBasePackThreshold 2g

Hmph, that is because the configuration wins and we know the trash
repository will never have a pack that large.  OK.

I won't comment on the style issue in the remainder, as it shares
the same as another patch.

> @@ -116,6 +123,28 @@ test_expect_success 'background auto gc respects lock for all operations' '
>  	test_path_is_file .git/refs/heads/should-be-loose
>  '
>  
> +test_expect_success 'gc --keep-base-pack' '
> +	test_create_repo keep-pack &&
> +	(
> +		cd keep-pack &&
> +		for i in 10; do
> +			test_commit $i
> +		done &&
> +		git gc &&

This, because of the set-up step, is guaranteed to pack everything
into one.

> +		( cd .git/objects/pack && ls *.pack ) >pack-list &&
> +		test_line_count = 1 pack-list &&
> +		BASE_PACK=.git/objects/pack/pack-*.pack &&

And that is what these verify.

> +		for i in 10; do
> +			test_commit more-$i
> +		done &&
> +		git gc --keep-base-pack &&

And we allow --keep-base-pack option to pick the largest pack (there
is only one pack anyway) and keep it, which ...

> +		( cd .git/objects/pack && ls *.pack ) >pack-list &&
> +		test_line_count = 2 pack-list &&

... results in two packs in total.  And

> +		test_path_is_file $BASE_PACK &&

... we make sure the first pack is left intact (unlike the earlier
'new-count' test).  This sounds sensible.

> +		git fsck
> +	)
> +'
> +
>  # DO NOT leave a detached auto gc process running near the end of the
>  # test script: it can run long enough in the background to racily
>  # interfere with the cleanup in 'test_done'.

  reply	other threads:[~2018-03-06 19:19 UTC|newest]

Thread overview: 273+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-28  9:27 Reduce pack-objects memory footprint? Duy Nguyen
2018-02-28 10:17 ` Jeff King
2018-02-28 10:58   ` Duy Nguyen
2018-02-28 11:11     ` Jeff King
2018-02-28 11:24       ` Duy Nguyen
2018-02-28 18:22 ` Eric Wong
2018-03-01  9:00   ` Duy Nguyen
2018-03-01  9:10 ` [PATCH 00/11] Reduce pack-objects memory footprint Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 01/11] pack-objects: document holes in struct object_entry.h Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-01 18:00     ` Junio C Hamano
2018-03-01  9:10   ` [PATCH 05/11] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 06/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 07/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-01 12:37     ` Ævar Arnfjörð Bjarmason
2018-03-01 14:49     ` Jeff King
2018-03-02  0:02       ` Duy Nguyen
2018-03-01 18:05     ` Junio C Hamano
2018-03-01  9:10   ` [PATCH 08/11] pack-objects: faster reverse packed_git lookup Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 09/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-01 18:08     ` Junio C Hamano
2018-03-01  9:10   ` [PATCH 10/11] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy
2018-03-01  9:10   ` [PATCH 11/11] pack-objects: increase pack file limit to 4096 Nguyễn Thái Ngọc Duy
2018-03-01 13:33   ` [PATCH 00/11] Reduce pack-objects memory footprint Ævar Arnfjörð Bjarmason
2018-03-02  0:14     ` Duy Nguyen
2018-03-02 10:57       ` Jeff King
2018-03-03  2:46   ` [PATCH/RFC v2 0/9] " Nguyễn Thái Ngọc Duy
2018-03-03  2:46     ` [PATCH/RFC v2 1/9] pack-objects: document holes in struct object_entry.h Nguyễn Thái Ngọc Duy
2018-03-03  2:46     ` [PATCH/RFC v2 2/9] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 3/9] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 4/9] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 5/9] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 6/9] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 7/9] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 8/9] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-03  2:47     ` [PATCH/RFC v2 9/9] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy
2018-03-05  9:28     ` [PATCH/RFC v2 0/9] Reduce pack-objects memory footprint Duy Nguyen
2018-03-08 11:42     ` [PATCH/RFC v3 00/12] " Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 01/12] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-09 22:34         ` Junio C Hamano
2018-03-08 11:42       ` [PATCH/RFC v3 02/12] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-09 22:54         ` Junio C Hamano
2018-03-12 17:51           ` Duy Nguyen
2018-03-08 11:42       ` [PATCH/RFC v3 03/12] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 04/12] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-09 23:07         ` Junio C Hamano
2018-03-08 11:42       ` [PATCH/RFC v3 05/12] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 06/12] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 07/12] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-09 23:21         ` Junio C Hamano
2018-03-08 11:42       ` [PATCH/RFC v3 08/12] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-14 16:18         ` Junio C Hamano
2018-03-08 11:42       ` [PATCH/RFC v3 09/12] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 10/12] pack-objects: shrink z_delta_size field in " Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 11/12] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-08 11:42       ` [PATCH/RFC v3 12/12] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-16 18:31       ` [PATCH v4 00/11] nd/pack-objects-pack-struct updates Nguyễn Thái Ngọc Duy
2018-03-16 18:31         ` [PATCH v4 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-16 20:32           ` Junio C Hamano
2018-03-17 11:59             ` Duy Nguyen
2018-03-16 18:31         ` [PATCH v4 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-16 20:49           ` Junio C Hamano
2018-03-16 18:31         ` [PATCH v4 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-16 18:31         ` [PATCH v4 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-16 18:31         ` [PATCH v4 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-16 18:31         ` [PATCH v4 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-26 20:39           ` Stefan Beller
2018-03-16 18:31         ` [PATCH v4 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-16 20:59           ` Junio C Hamano
2018-03-16 18:31         ` [PATCH v4 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-16 19:40           ` Junio C Hamano
2018-03-16 18:31         ` [PATCH v4 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-16 19:49           ` Junio C Hamano
2018-03-16 21:34             ` Junio C Hamano
2018-03-16 18:31         ` [PATCH v4 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-16 18:32         ` [PATCH v4 11/11] pack-objects.h: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-16 21:02           ` Junio C Hamano
2018-03-17 12:07             ` Duy Nguyen
2018-03-17 14:10         ` [PATCH v5 00/11] nd/pack-objects-pack-struct updates Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-17 21:26             ` Ævar Arnfjörð Bjarmason
2018-03-17 14:10           ` [PATCH v5 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-17 19:57             ` Ævar Arnfjörð Bjarmason
2018-03-18  5:09             ` Junio C Hamano
2018-03-18  8:23               ` Duy Nguyen
2018-03-17 14:10           ` [PATCH v5 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-17 14:10           ` [PATCH v5 11/11] pack-objects.h: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-17 19:53             ` Ævar Arnfjörð Bjarmason
2018-03-18  8:49               ` Duy Nguyen
2018-03-17 19:45           ` [PATCH v5 00/11] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason
2018-03-17 19:47             ` Ævar Arnfjörð Bjarmason
2018-03-18 14:25           ` [PATCH v6 " Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-18 14:49               ` Ævar Arnfjörð Bjarmason
2018-03-19 16:19               ` Junio C Hamano
2018-03-19 16:23                 ` Duy Nguyen
2018-03-19 16:43               ` Junio C Hamano
2018-03-19 16:54                 ` Duy Nguyen
2018-03-19 18:29                   ` Junio C Hamano
2018-03-19 18:45                     ` Duy Nguyen
2018-03-19 20:10                       ` Junio C Hamano
2018-03-20 18:08                         ` Duy Nguyen
2018-03-20 18:22                           ` Junio C Hamano
2018-03-21  8:03                           ` Jeff King
2018-03-21 16:12                             ` Duy Nguyen
2018-03-20 18:17                 ` Duy Nguyen
2018-03-18 14:25             ` [PATCH v6 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-18 14:25             ` [PATCH v6 11/11] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-18 14:51             ` [PATCH v6 00/11] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason
2018-03-21  8:24             ` Jeff King
2018-03-21 15:59               ` Duy Nguyen
2018-03-21 16:17                 ` Ævar Arnfjörð Bjarmason
2018-03-21 16:22                   ` Duy Nguyen
2018-03-21 16:46                 ` Duy Nguyen
2018-03-21 19:11                   ` Junio C Hamano
2018-03-22  9:32                 ` Jeff King
2018-03-22  9:46                   ` Jeff King
2018-03-22 10:57                   ` Duy Nguyen
2018-03-22 11:52                     ` Jeff King
2018-03-22 17:04                       ` Duy Nguyen
2018-03-23  1:28                   ` Ramsay Jones
2018-03-23  2:46                     ` Jeff King
2018-03-23  5:50                       ` Jeff King
2018-03-23 16:01                         ` Ramsay Jones
2018-03-24  6:40                           ` Jeff King
2018-03-23  7:05                       ` Duy Nguyen
2018-03-23 14:03                       ` Ramsay Jones
2018-03-21 16:31               ` Ævar Arnfjörð Bjarmason
2018-03-21 16:53                 ` Junio C Hamano
2018-03-21 17:00                   ` Duy Nguyen
2018-03-22  8:07                 ` Jeff King
2018-03-22  8:23                   ` Duy Nguyen
2018-03-22 10:01                     ` Jeff King
2018-03-24  6:33             ` [PATCH v7 00/13] " Nguyễn Thái Ngọc Duy
2018-03-24  6:33               ` [PATCH v7 01/13] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-24  6:33               ` [PATCH v7 02/13] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-30 20:18                 ` Jeff King
2018-03-24  6:33               ` [PATCH v7 03/13] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-30 20:23                 ` Jeff King
2018-03-24  6:33               ` [PATCH v7 04/13] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-30 20:26                 ` Jeff King
2018-03-24  6:33               ` [PATCH v7 05/13] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-30 20:30                 ` Jeff King
2018-03-24  6:33               ` [PATCH v7 06/13] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-24  9:42                 ` Ævar Arnfjörð Bjarmason
2018-03-24 12:26                   ` Duy Nguyen
2018-03-24 12:13                 ` Ævar Arnfjörð Bjarmason
2018-03-30 20:48                 ` Jeff King
2018-03-31  4:51                   ` Duy Nguyen
2018-03-31 10:20                     ` Jeff King
2018-03-31 10:45                       ` Duy Nguyen
2018-03-24  6:33               ` [PATCH v7 07/13] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-30 20:53                 ` Jeff King
2018-03-24  6:33               ` [PATCH v7 08/13] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-30 20:59                 ` Jeff King
2018-03-31  4:40                   ` Duy Nguyen
2018-03-31 10:17                     ` Jeff King
2018-03-24  6:33               ` [PATCH v7 09/13] pack-objects: don't check size when the object is bad Nguyễn Thái Ngọc Duy
2018-03-24  6:33               ` [PATCH v7 10/13] pack-objects: clarify the use of object_entry::size Nguyễn Thái Ngọc Duy
2018-03-30 21:04                 ` Jeff King
2018-03-31  4:35                   ` Duy Nguyen
2018-03-31 10:13                     ` Jeff King
2018-03-24  6:33               ` [PATCH v7 11/13] pack-objects: shrink size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-30 21:18                 ` Jeff King
2018-03-24  6:33               ` [PATCH v7 12/13] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-30 21:24                 ` Jeff King
2018-03-31  4:21                   ` Duy Nguyen
2018-03-31  9:10                   ` Duy Nguyen
2018-03-24  6:33               ` [PATCH v7 13/13] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-30 21:26                 ` Jeff King
2018-03-31  4:10                   ` Duy Nguyen
2018-03-26 15:13               ` [PATCH v7 00/13] nd/pack-objects-pack-struct updates Jeff King
2018-03-26 17:04                 ` Duy Nguyen
2018-03-27 16:53                   ` Jeff King
2018-03-31 10:02               ` [PATCH v8 00/15] " Nguyễn Thái Ngọc Duy
2018-03-31 10:02                 ` [PATCH v8 01/15] t/README: mention about running the test suite in special modes Nguyễn Thái Ngọc Duy
2018-03-31 10:02                 ` [PATCH v8 02/15] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:02                 ` [PATCH v8 03/15] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 04/15] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 05/15] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 06/15] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 07/15] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 08/15] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 09/15] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 10/15] pack-objects: don't check size when the object is bad Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 11/15] pack-objects: clarify the use of object_entry::size Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 12/15] pack-objects: shrink size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 13/15] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 14/15] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-31 10:03                 ` [PATCH v8 15/15] ci: exercise the whole test suite with uncommon code in pack-objects Nguyễn Thái Ngọc Duy
2018-03-31 11:36                 ` [PATCH v8 00/15] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason
2018-03-31 12:08                   ` Duy Nguyen
2018-03-31 15:43                     ` Ævar Arnfjörð Bjarmason
2018-04-06 21:47                 ` Jeff King
2018-03-01  9:20 ` [PATCH/RFC 0/1] Avoid expensive 'repack -ad' in gc --auto Nguyễn Thái Ngọc Duy
2018-03-01  9:20   ` [PATCH/RFC 1/1] gc --auto: exclude the largest giant pack in low-memory config Nguyễn Thái Ngọc Duy
2018-03-01 18:14     ` Junio C Hamano
2018-03-02  0:00       ` Duy Nguyen
2018-03-05 14:00     ` Ævar Arnfjörð Bjarmason
2018-03-06 10:41   ` [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto Nguyễn Thái Ngọc Duy
2018-03-06 10:41     ` [PATCH v2 1/5] fixup! Add a test showing that 'git repack' throws away grafted-away parents Nguyễn Thái Ngọc Duy
2018-03-06 18:01       ` Junio C Hamano
2018-03-06 10:41     ` [PATCH v2 2/5] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-06 18:25       ` Junio C Hamano
2018-03-07 10:19         ` Duy Nguyen
2018-03-06 10:41     ` [PATCH v2 3/5] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-06 19:19       ` Junio C Hamano [this message]
2018-03-07 10:48         ` Duy Nguyen
2018-03-07 18:38           ` Junio C Hamano
2018-03-12 18:56             ` Ævar Arnfjörð Bjarmason
2018-03-12 21:16               ` Junio C Hamano
2018-03-12 22:01                 ` Ævar Arnfjörð Bjarmason
2018-03-15 16:48               ` Duy Nguyen
2018-03-07 10:48       ` Johannes Schindelin
2018-03-07 18:40         ` Junio C Hamano
2018-03-12 19:30       ` Ævar Arnfjörð Bjarmason
2018-03-15 17:00         ` Duy Nguyen
2018-03-15 19:21           ` Ævar Arnfjörð Bjarmason
2018-03-16 17:47             ` Duy Nguyen
2018-03-06 10:41     ` [PATCH v2 4/5] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-12 18:32       ` Ævar Arnfjörð Bjarmason
2018-03-16 19:14         ` Duy Nguyen
2018-03-16 20:13           ` Duy Nguyen
2018-03-06 10:41     ` [PATCH v2 5/5] pack-objects: display progress in get_object_details() Nguyễn Thái Ngọc Duy
2018-03-06 17:49     ` [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto Junio C Hamano
2018-03-16 19:27     ` [PATCH v3 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy
2018-03-16 19:27       ` [PATCH v3 1/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-16 19:27       ` [PATCH v3 2/7] gc: add --keep-base-pack Nguyễn Thái Ngọc Duy
2018-03-16 21:05         ` Ævar Arnfjörð Bjarmason
2018-03-19 17:26           ` Duy Nguyen
2018-03-19 19:04             ` Ævar Arnfjörð Bjarmason
2018-03-16 21:25         ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27       ` [PATCH v3 3/7] gc: detect base packs based on gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
2018-03-16 21:02         ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27       ` [PATCH v3 4/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-16 21:14         ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27       ` [PATCH v3 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-03-16 21:10         ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27       ` [PATCH v3 6/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-16 19:27       ` [PATCH v3 7/7] pack-objects: display progress in get_object_details() Nguyễn Thái Ngọc Duy
2018-03-24  7:25       ` [PATCH v4 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 1/7] t7700: have closing quote of a test at the beginning of line Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 2/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 3/7] gc: add --keep-largest-pack option Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 4/7] gc: add gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 6/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-24  7:25         ` [PATCH v4 7/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-02 10:18 ` Reduce pack-objects memory footprint? Duy Nguyen
2018-03-02 10:37   ` Eric Wong
2018-03-02 10:54   ` Jeff King
2018-03-02 10:55     ` Duy Nguyen
2018-03-02 14:38     ` Duy Nguyen
2018-03-17 22:05 ` Why does pack-objects use so much memory on incremental packing? Ævar Arnfjörð Bjarmason
2018-03-18  8:37   ` Duy Nguyen
2018-03-20  5:28   ` Jeff King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=xmqqfu5d3tbn.fsf@gitster-ct.c.googlers.com \
    --to=gitster@pobox.com \
    --cc=avarab@gmail.com \
    --cc=e@80x24.org \
    --cc=git@vger.kernel.org \
    --cc=pclouds@gmail.com \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).