From: Jeff King <peff@peff.net>
To: Duy Nguyen <pclouds@gmail.com>
Cc: Git Mailing List <git@vger.kernel.org>
Subject: Re: Reduce pack-objects memory footprint?
Date: Wed, 28 Feb 2018 06:11:21 -0500 [thread overview]
Message-ID: <20180228111121.GA8925@sigill.intra.peff.net> (raw)
In-Reply-To: <CACsJy8Bg_oNj7mJ2PE5AWQ2awf434R9fHXQq-OxA3_0wJN16vw@mail.gmail.com>
On Wed, Feb 28, 2018 at 05:58:50PM +0700, Duy Nguyen wrote:
> > Yeah, the per object memory footprint is not great. Around 100 million
> > objects it becomes pretty ridiculous. I started to dig into it a year or
> > three ago when I saw such a case, but it turned out to be something that
> > we could prune.
>
> We could? What could we prune?
Sorry, I just meant that my 100 million-object case turned out not to
need all those objects, and I was able to prune it down. No code fixes
came out of it. ;)
> > The torvalds/linux fork network has ~23 million objects,
> > so it's probably 7-8 GB of book-keeping. Which is gross, but 64GB in a
> > server isn't uncommon these days.
>
> I wonder if we could just do book keeping for some but not all objects
> because all objects simply do not scale. Say we have a big pack of
> many GBs, could we keep the 80% of its bottom untouched, register the
> top 20% (mostly non-blobs, and some more blobs as delta base) for
> repack? We copy the bottom part to the new pack byte-by-byte, then
> pack-objects rebuilds the top part with objects from other sources.
Yes, though I think it would take a fair bit of surgery to do
internally. And some features (like bitmap generation) just wouldn't
work at all.
I suspect you could simulate it, though, by just packing your subset
with pack-objects (feeding it directly without using "--revs") and then
catting the resulting packfiles together with a fixed-up header.
At one point I played with a "fast pack" that would just cat packfiles
together. My goal was to make cases with 10,000 packs workable by
creating one lousy pack, and then repacking that lousy pack with a
"real" repack. In the end I abandoned it in favor of fixing the
performance problems from trying to make a real pack of 10,000 packs. :)
But I might be able to dig it up if you want to experiment in that
direction.
> They are 32 bytes per entry, so it should take less than object_entry.
> I briefly wondered if we should fall back to external rev-list too,
> just to free that memory.
>
> So about 200 MB for those objects (or maybe more for commits). Add 256
> MB delta cache on top, it's still a bit far from 1.7G. There's
> something I'm still missing.
Are you looking at RSS or heap? Keep in mind that you're mmap-ing what's
probably a 1GB packfile on disk. If you're under memory pressure that
won't all stay resident, but some of it will be counted in RSS.
> Pity we can't do the same for 'struct object'. Most of the time we
> have a giant .idx file with most hashes. We could look up in both
> places: the hash table in object.c, and the idx file, to find an
> object. Then those objects that are associated with .idx file will not
> need "oid" field (needed to as key for the hash table). But I see no
> way to make that change.
Yeah, that would be pretty invasive, I think. I also wonder if it would
perform worse due to cache effects.
-Peff
next prev parent reply other threads:[~2018-02-28 11:11 UTC|newest]
Thread overview: 273+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-28 9:27 Reduce pack-objects memory footprint? Duy Nguyen
2018-02-28 10:17 ` Jeff King
2018-02-28 10:58 ` Duy Nguyen
2018-02-28 11:11 ` Jeff King [this message]
2018-02-28 11:24 ` Duy Nguyen
2018-02-28 18:22 ` Eric Wong
2018-03-01 9:00 ` Duy Nguyen
2018-03-01 9:10 ` [PATCH 00/11] Reduce pack-objects memory footprint Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 01/11] pack-objects: document holes in struct object_entry.h Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-01 18:00 ` Junio C Hamano
2018-03-01 9:10 ` [PATCH 05/11] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 06/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 07/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-01 12:37 ` Ævar Arnfjörð Bjarmason
2018-03-01 14:49 ` Jeff King
2018-03-02 0:02 ` Duy Nguyen
2018-03-01 18:05 ` Junio C Hamano
2018-03-01 9:10 ` [PATCH 08/11] pack-objects: faster reverse packed_git lookup Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 09/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-01 18:08 ` Junio C Hamano
2018-03-01 9:10 ` [PATCH 10/11] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy
2018-03-01 9:10 ` [PATCH 11/11] pack-objects: increase pack file limit to 4096 Nguyễn Thái Ngọc Duy
2018-03-01 13:33 ` [PATCH 00/11] Reduce pack-objects memory footprint Ævar Arnfjörð Bjarmason
2018-03-02 0:14 ` Duy Nguyen
2018-03-02 10:57 ` Jeff King
2018-03-03 2:46 ` [PATCH/RFC v2 0/9] " Nguyễn Thái Ngọc Duy
2018-03-03 2:46 ` [PATCH/RFC v2 1/9] pack-objects: document holes in struct object_entry.h Nguyễn Thái Ngọc Duy
2018-03-03 2:46 ` [PATCH/RFC v2 2/9] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 3/9] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 4/9] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 5/9] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 6/9] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 7/9] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 8/9] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-03 2:47 ` [PATCH/RFC v2 9/9] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy
2018-03-05 9:28 ` [PATCH/RFC v2 0/9] Reduce pack-objects memory footprint Duy Nguyen
2018-03-08 11:42 ` [PATCH/RFC v3 00/12] " Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 01/12] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-09 22:34 ` Junio C Hamano
2018-03-08 11:42 ` [PATCH/RFC v3 02/12] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-09 22:54 ` Junio C Hamano
2018-03-12 17:51 ` Duy Nguyen
2018-03-08 11:42 ` [PATCH/RFC v3 03/12] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 04/12] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-09 23:07 ` Junio C Hamano
2018-03-08 11:42 ` [PATCH/RFC v3 05/12] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 06/12] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 07/12] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-09 23:21 ` Junio C Hamano
2018-03-08 11:42 ` [PATCH/RFC v3 08/12] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-14 16:18 ` Junio C Hamano
2018-03-08 11:42 ` [PATCH/RFC v3 09/12] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 10/12] pack-objects: shrink z_delta_size field in " Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 11/12] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-08 11:42 ` [PATCH/RFC v3 12/12] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-16 18:31 ` [PATCH v4 00/11] nd/pack-objects-pack-struct updates Nguyễn Thái Ngọc Duy
2018-03-16 18:31 ` [PATCH v4 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-16 20:32 ` Junio C Hamano
2018-03-17 11:59 ` Duy Nguyen
2018-03-16 18:31 ` [PATCH v4 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-16 20:49 ` Junio C Hamano
2018-03-16 18:31 ` [PATCH v4 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-16 18:31 ` [PATCH v4 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-16 18:31 ` [PATCH v4 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-16 18:31 ` [PATCH v4 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-26 20:39 ` Stefan Beller
2018-03-16 18:31 ` [PATCH v4 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-16 20:59 ` Junio C Hamano
2018-03-16 18:31 ` [PATCH v4 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-16 19:40 ` Junio C Hamano
2018-03-16 18:31 ` [PATCH v4 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-16 19:49 ` Junio C Hamano
2018-03-16 21:34 ` Junio C Hamano
2018-03-16 18:31 ` [PATCH v4 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-16 18:32 ` [PATCH v4 11/11] pack-objects.h: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-16 21:02 ` Junio C Hamano
2018-03-17 12:07 ` Duy Nguyen
2018-03-17 14:10 ` [PATCH v5 00/11] nd/pack-objects-pack-struct updates Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-17 21:26 ` Ævar Arnfjörð Bjarmason
2018-03-17 14:10 ` [PATCH v5 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-17 19:57 ` Ævar Arnfjörð Bjarmason
2018-03-18 5:09 ` Junio C Hamano
2018-03-18 8:23 ` Duy Nguyen
2018-03-17 14:10 ` [PATCH v5 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-17 14:10 ` [PATCH v5 11/11] pack-objects.h: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-17 19:53 ` Ævar Arnfjörð Bjarmason
2018-03-18 8:49 ` Duy Nguyen
2018-03-17 19:45 ` [PATCH v5 00/11] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason
2018-03-17 19:47 ` Ævar Arnfjörð Bjarmason
2018-03-18 14:25 ` [PATCH v6 " Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy
2018-03-18 14:49 ` Ævar Arnfjörð Bjarmason
2018-03-19 16:19 ` Junio C Hamano
2018-03-19 16:23 ` Duy Nguyen
2018-03-19 16:43 ` Junio C Hamano
2018-03-19 16:54 ` Duy Nguyen
2018-03-19 18:29 ` Junio C Hamano
2018-03-19 18:45 ` Duy Nguyen
2018-03-19 20:10 ` Junio C Hamano
2018-03-20 18:08 ` Duy Nguyen
2018-03-20 18:22 ` Junio C Hamano
2018-03-21 8:03 ` Jeff King
2018-03-21 16:12 ` Duy Nguyen
2018-03-20 18:17 ` Duy Nguyen
2018-03-18 14:25 ` [PATCH v6 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-18 14:25 ` [PATCH v6 11/11] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-18 14:51 ` [PATCH v6 00/11] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason
2018-03-21 8:24 ` Jeff King
2018-03-21 15:59 ` Duy Nguyen
2018-03-21 16:17 ` Ævar Arnfjörð Bjarmason
2018-03-21 16:22 ` Duy Nguyen
2018-03-21 16:46 ` Duy Nguyen
2018-03-21 19:11 ` Junio C Hamano
2018-03-22 9:32 ` Jeff King
2018-03-22 9:46 ` Jeff King
2018-03-22 10:57 ` Duy Nguyen
2018-03-22 11:52 ` Jeff King
2018-03-22 17:04 ` Duy Nguyen
2018-03-23 1:28 ` Ramsay Jones
2018-03-23 2:46 ` Jeff King
2018-03-23 5:50 ` Jeff King
2018-03-23 16:01 ` Ramsay Jones
2018-03-24 6:40 ` Jeff King
2018-03-23 7:05 ` Duy Nguyen
2018-03-23 14:03 ` Ramsay Jones
2018-03-21 16:31 ` Ævar Arnfjörð Bjarmason
2018-03-21 16:53 ` Junio C Hamano
2018-03-21 17:00 ` Duy Nguyen
2018-03-22 8:07 ` Jeff King
2018-03-22 8:23 ` Duy Nguyen
2018-03-22 10:01 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 00/13] " Nguyễn Thái Ngọc Duy
2018-03-24 6:33 ` [PATCH v7 01/13] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-24 6:33 ` [PATCH v7 02/13] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-30 20:18 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 03/13] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-30 20:23 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 04/13] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-30 20:26 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 05/13] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-30 20:30 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 06/13] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-24 9:42 ` Ævar Arnfjörð Bjarmason
2018-03-24 12:26 ` Duy Nguyen
2018-03-24 12:13 ` Ævar Arnfjörð Bjarmason
2018-03-30 20:48 ` Jeff King
2018-03-31 4:51 ` Duy Nguyen
2018-03-31 10:20 ` Jeff King
2018-03-31 10:45 ` Duy Nguyen
2018-03-24 6:33 ` [PATCH v7 07/13] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-30 20:53 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 08/13] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-30 20:59 ` Jeff King
2018-03-31 4:40 ` Duy Nguyen
2018-03-31 10:17 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 09/13] pack-objects: don't check size when the object is bad Nguyễn Thái Ngọc Duy
2018-03-24 6:33 ` [PATCH v7 10/13] pack-objects: clarify the use of object_entry::size Nguyễn Thái Ngọc Duy
2018-03-30 21:04 ` Jeff King
2018-03-31 4:35 ` Duy Nguyen
2018-03-31 10:13 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 11/13] pack-objects: shrink size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-30 21:18 ` Jeff King
2018-03-24 6:33 ` [PATCH v7 12/13] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-30 21:24 ` Jeff King
2018-03-31 4:21 ` Duy Nguyen
2018-03-31 9:10 ` Duy Nguyen
2018-03-24 6:33 ` [PATCH v7 13/13] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-30 21:26 ` Jeff King
2018-03-31 4:10 ` Duy Nguyen
2018-03-26 15:13 ` [PATCH v7 00/13] nd/pack-objects-pack-struct updates Jeff King
2018-03-26 17:04 ` Duy Nguyen
2018-03-27 16:53 ` Jeff King
2018-03-31 10:02 ` [PATCH v8 00/15] " Nguyễn Thái Ngọc Duy
2018-03-31 10:02 ` [PATCH v8 01/15] t/README: mention about running the test suite in special modes Nguyễn Thái Ngọc Duy
2018-03-31 10:02 ` [PATCH v8 02/15] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:02 ` [PATCH v8 03/15] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 04/15] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 05/15] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 06/15] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 07/15] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 08/15] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 09/15] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 10/15] pack-objects: don't check size when the object is bad Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 11/15] pack-objects: clarify the use of object_entry::size Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 12/15] pack-objects: shrink size field in struct object_entry Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 13/15] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 14/15] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy
2018-03-31 10:03 ` [PATCH v8 15/15] ci: exercise the whole test suite with uncommon code in pack-objects Nguyễn Thái Ngọc Duy
2018-03-31 11:36 ` [PATCH v8 00/15] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason
2018-03-31 12:08 ` Duy Nguyen
2018-03-31 15:43 ` Ævar Arnfjörð Bjarmason
2018-04-06 21:47 ` Jeff King
2018-03-01 9:20 ` [PATCH/RFC 0/1] Avoid expensive 'repack -ad' in gc --auto Nguyễn Thái Ngọc Duy
2018-03-01 9:20 ` [PATCH/RFC 1/1] gc --auto: exclude the largest giant pack in low-memory config Nguyễn Thái Ngọc Duy
2018-03-01 18:14 ` Junio C Hamano
2018-03-02 0:00 ` Duy Nguyen
2018-03-05 14:00 ` Ævar Arnfjörð Bjarmason
2018-03-06 10:41 ` [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto Nguyễn Thái Ngọc Duy
2018-03-06 10:41 ` [PATCH v2 1/5] fixup! Add a test showing that 'git repack' throws away grafted-away parents Nguyễn Thái Ngọc Duy
2018-03-06 18:01 ` Junio C Hamano
2018-03-06 10:41 ` [PATCH v2 2/5] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-06 18:25 ` Junio C Hamano
2018-03-07 10:19 ` Duy Nguyen
2018-03-06 10:41 ` [PATCH v2 3/5] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-06 19:19 ` Junio C Hamano
2018-03-07 10:48 ` Duy Nguyen
2018-03-07 18:38 ` Junio C Hamano
2018-03-12 18:56 ` Ævar Arnfjörð Bjarmason
2018-03-12 21:16 ` Junio C Hamano
2018-03-12 22:01 ` Ævar Arnfjörð Bjarmason
2018-03-15 16:48 ` Duy Nguyen
2018-03-07 10:48 ` Johannes Schindelin
2018-03-07 18:40 ` Junio C Hamano
2018-03-12 19:30 ` Ævar Arnfjörð Bjarmason
2018-03-15 17:00 ` Duy Nguyen
2018-03-15 19:21 ` Ævar Arnfjörð Bjarmason
2018-03-16 17:47 ` Duy Nguyen
2018-03-06 10:41 ` [PATCH v2 4/5] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-12 18:32 ` Ævar Arnfjörð Bjarmason
2018-03-16 19:14 ` Duy Nguyen
2018-03-16 20:13 ` Duy Nguyen
2018-03-06 10:41 ` [PATCH v2 5/5] pack-objects: display progress in get_object_details() Nguyễn Thái Ngọc Duy
2018-03-06 17:49 ` [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto Junio C Hamano
2018-03-16 19:27 ` [PATCH v3 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy
2018-03-16 19:27 ` [PATCH v3 1/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-16 19:27 ` [PATCH v3 2/7] gc: add --keep-base-pack Nguyễn Thái Ngọc Duy
2018-03-16 21:05 ` Ævar Arnfjörð Bjarmason
2018-03-19 17:26 ` Duy Nguyen
2018-03-19 19:04 ` Ævar Arnfjörð Bjarmason
2018-03-16 21:25 ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27 ` [PATCH v3 3/7] gc: detect base packs based on gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
2018-03-16 21:02 ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27 ` [PATCH v3 4/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-16 21:14 ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27 ` [PATCH v3 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-03-16 21:10 ` Ævar Arnfjörð Bjarmason
2018-03-16 19:27 ` [PATCH v3 6/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-16 19:27 ` [PATCH v3 7/7] pack-objects: display progress in get_object_details() Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 1/7] t7700: have closing quote of a test at the beginning of line Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 2/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 3/7] gc: add --keep-largest-pack option Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 4/7] gc: add gc.bigPackThreshold config Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 6/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy
2018-03-24 7:25 ` [PATCH v4 7/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy
2018-03-02 10:18 ` Reduce pack-objects memory footprint? Duy Nguyen
2018-03-02 10:37 ` Eric Wong
2018-03-02 10:54 ` Jeff King
2018-03-02 10:55 ` Duy Nguyen
2018-03-02 14:38 ` Duy Nguyen
2018-03-17 22:05 ` Why does pack-objects use so much memory on incremental packing? Ævar Arnfjörð Bjarmason
2018-03-18 8:37 ` Duy Nguyen
2018-03-20 5:28 ` Jeff King
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180228111121.GA8925@sigill.intra.peff.net \
--to=peff@peff.net \
--cc=git@vger.kernel.org \
--cc=pclouds@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).