From: Jeff King <peff@peff.net> To: Duy Nguyen <pclouds@gmail.com> Cc: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>, "Eric Wong" <e@80x24.org>, "Git Mailing List" <git@vger.kernel.org>, "Junio C Hamano" <gitster@pobox.com> Subject: Re: [PATCH v6 00/11] nd/pack-objects-pack-struct updates Date: Thu, 22 Mar 2018 05:46:49 -0400 Message-ID: <20180322094649.GA32698@sigill.intra.peff.net> (raw) In-Reply-To: <20180322093212.GB28907@sigill.intra.peff.net> On Thu, Mar 22, 2018 at 05:32:12AM -0400, Jeff King wrote: > So 27% of the total heap goes away if you switch to a separate rev-list. > Though it's mostly just going to a different process, it does help peak > because that process would have exited by the time we get to the > revindex bits. > > I suspect you could get the same effect by just teaching pack-objects to > clear obj_hash and all of the allocated object structs. I think that > should be safe to do as long as we clear _all_ of the objects, so there > are no dangling pointers. The patch below tries that. It's kind of hacky, but it drops my peak heap for packing linux.git from 1336MB to 1129MB. That's not quite as exciting as 27%, because it just moves our peak earlier, to when we do have all of the object structs in memory (so the savings are really just that we're not holding the revindex, etc at the same time as the object structs). But we also hold that peak for a lot shorter period, because we drop the memory before we do any delta compression (which itself can be memory hungry[1]), and don't hold onto it during the write phase (which can be network-limited when serving a fetch). So during that write phase we're holding only ~900MB instead of ~1250MB. -Peff [1] All of my timings are on noop repacks of a single pack, so there's no actual delta compression. On average, it will use something like "nr_threads * window * avg_blob_size". For a "normal" repo, that's only a few megabytes. But the peak will depend on the large blobs, so it could have some outsize cases. I don't know if it's worth worrying about too much for this analysis. --- Here's the patch. It's probably asking for trouble to have this kind of clearing interface, as a surprising number of things may hold onto pointers to objects (see the comment below about the bitmap code). So maybe the separate process is less insane. diff --git a/alloc.c b/alloc.c index 12afadfacd..50d444a3b0 100644 --- a/alloc.c +++ b/alloc.c @@ -30,15 +30,23 @@ struct alloc_state { int count; /* total number of nodes allocated */ int nr; /* number of nodes left in current allocation */ void *p; /* first free node in current allocation */ + + /* book-keeping for clearing */ + void *start; + struct alloc_state *prev; }; -static inline void *alloc_node(struct alloc_state *s, size_t node_size) +static inline void *alloc_node(struct alloc_state **sp, size_t node_size) { + struct alloc_state *s = *sp; void *ret; - if (!s->nr) { + if (!s || !s->nr) { + s = xmalloc(sizeof(*s)); s->nr = BLOCKING; - s->p = xmalloc(BLOCKING * node_size); + s->start = s->p = xmalloc(BLOCKING * node_size); + s->prev = *sp; + *sp = s; } s->nr--; s->count++; @@ -48,7 +56,7 @@ static inline void *alloc_node(struct alloc_state *s, size_t node_size) return ret; } -static struct alloc_state blob_state; +static struct alloc_state *blob_state; void *alloc_blob_node(void) { struct blob *b = alloc_node(&blob_state, sizeof(struct blob)); @@ -56,7 +64,7 @@ void *alloc_blob_node(void) return b; } -static struct alloc_state tree_state; +static struct alloc_state *tree_state; void *alloc_tree_node(void) { struct tree *t = alloc_node(&tree_state, sizeof(struct tree)); @@ -64,7 +72,7 @@ void *alloc_tree_node(void) return t; } -static struct alloc_state tag_state; +static struct alloc_state *tag_state; void *alloc_tag_node(void) { struct tag *t = alloc_node(&tag_state, sizeof(struct tag)); @@ -72,7 +80,7 @@ void *alloc_tag_node(void) return t; } -static struct alloc_state object_state; +static struct alloc_state *object_state; void *alloc_object_node(void) { struct object *obj = alloc_node(&object_state, sizeof(union any_object)); @@ -80,7 +88,7 @@ void *alloc_object_node(void) return obj; } -static struct alloc_state commit_state; +static struct alloc_state *commit_state; unsigned int alloc_commit_index(void) { @@ -103,7 +111,7 @@ static void report(const char *name, unsigned int count, size_t size) } #define REPORT(name, type) \ - report(#name, name##_state.count, name##_state.count * sizeof(type) >> 10) + report(#name, name##_state->count, name##_state->count * sizeof(type) >> 10) void alloc_report(void) { @@ -113,3 +121,22 @@ void alloc_report(void) REPORT(tag, struct tag); REPORT(object, union any_object); } + +static void alloc_clear(struct alloc_state **sp) +{ + while (*sp) { + struct alloc_state *s = *sp; + *sp = s->prev; + free(s->start); + free(s); + } +} + +void alloc_clear_all(void) +{ + alloc_clear(&blob_state); + alloc_clear(&tree_state); + alloc_clear(&commit_state); + alloc_clear(&tag_state); + alloc_clear(&object_state); +} diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c index 4406af640f..7ba8ab07a3 100644 --- a/builtin/pack-objects.c +++ b/builtin/pack-objects.c @@ -2959,6 +2959,13 @@ static void get_object_list(int ac, const char **av) record_recent_object, NULL); } + /* + * The bitmap code actually stores the commit pointers for future + * reference, so we can't use this memory optimization there. + */ + if (!write_bitmap_index) + free_all_objects(); + if (keep_unreachable) add_objects_in_unpacked_packs(&revs); if (pack_loose_unreachable) diff --git a/cache.h b/cache.h index b90feb3802..605bab31de 100644 --- a/cache.h +++ b/cache.h @@ -1872,6 +1872,8 @@ extern void *alloc_object_node(void); extern void alloc_report(void); extern unsigned int alloc_commit_index(void); +void alloc_clear_all(void); + /* pkt-line.c */ void packet_trace_identity(const char *prog); diff --git a/object.c b/object.c index 9e6f9ff20b..6530d6fbde 100644 --- a/object.c +++ b/object.c @@ -445,3 +445,12 @@ void clear_commit_marks_all(unsigned int flags) obj->flags &= ~flags; } } + +void free_all_objects(void) +{ + alloc_clear_all(); + free(obj_hash); + obj_hash = NULL; + obj_hash_size = 0; + nr_objs = 0; +} diff --git a/object.h b/object.h index 8ce294d6ec..3eb85215c2 100644 --- a/object.h +++ b/object.h @@ -153,4 +153,6 @@ void clear_object_flags(unsigned flags); */ extern void clear_commit_marks_all(unsigned int flags); +void free_all_objects(void); + #endif /* OBJECT_H */
next prev parent reply other threads:[~2018-03-22 9:46 UTC|newest] Thread overview: 273+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-02-28 9:27 Reduce pack-objects memory footprint? Duy Nguyen 2018-02-28 10:17 ` Jeff King 2018-02-28 10:58 ` Duy Nguyen 2018-02-28 11:11 ` Jeff King 2018-02-28 11:24 ` Duy Nguyen 2018-02-28 18:22 ` Eric Wong 2018-03-01 9:00 ` Duy Nguyen 2018-03-01 9:10 ` [PATCH 00/11] Reduce pack-objects memory footprint Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 01/11] pack-objects: document holes in struct object_entry.h Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-01 18:00 ` Junio C Hamano 2018-03-01 9:10 ` [PATCH 05/11] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 06/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 07/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-01 12:37 ` Ævar Arnfjörð Bjarmason 2018-03-01 14:49 ` Jeff King 2018-03-02 0:02 ` Duy Nguyen 2018-03-01 18:05 ` Junio C Hamano 2018-03-01 9:10 ` [PATCH 08/11] pack-objects: faster reverse packed_git lookup Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 09/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-01 18:08 ` Junio C Hamano 2018-03-01 9:10 ` [PATCH 10/11] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy 2018-03-01 9:10 ` [PATCH 11/11] pack-objects: increase pack file limit to 4096 Nguyễn Thái Ngọc Duy 2018-03-01 13:33 ` [PATCH 00/11] Reduce pack-objects memory footprint Ævar Arnfjörð Bjarmason 2018-03-02 0:14 ` Duy Nguyen 2018-03-02 10:57 ` Jeff King 2018-03-03 2:46 ` [PATCH/RFC v2 0/9] " Nguyễn Thái Ngọc Duy 2018-03-03 2:46 ` [PATCH/RFC v2 1/9] pack-objects: document holes in struct object_entry.h Nguyễn Thái Ngọc Duy 2018-03-03 2:46 ` [PATCH/RFC v2 2/9] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 3/9] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 4/9] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 5/9] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 6/9] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 7/9] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 8/9] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-03 2:47 ` [PATCH/RFC v2 9/9] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy 2018-03-05 9:28 ` [PATCH/RFC v2 0/9] Reduce pack-objects memory footprint Duy Nguyen 2018-03-08 11:42 ` [PATCH/RFC v3 00/12] " Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 01/12] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy 2018-03-09 22:34 ` Junio C Hamano 2018-03-08 11:42 ` [PATCH/RFC v3 02/12] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-09 22:54 ` Junio C Hamano 2018-03-12 17:51 ` Duy Nguyen 2018-03-08 11:42 ` [PATCH/RFC v3 03/12] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 04/12] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-09 23:07 ` Junio C Hamano 2018-03-08 11:42 ` [PATCH/RFC v3 05/12] pack-objects: note about in_pack_header_size Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 06/12] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 07/12] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-09 23:21 ` Junio C Hamano 2018-03-08 11:42 ` [PATCH/RFC v3 08/12] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-14 16:18 ` Junio C Hamano 2018-03-08 11:42 ` [PATCH/RFC v3 09/12] pack-objects: reorder 'hash' to pack struct object_entry Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 10/12] pack-objects: shrink z_delta_size field in " Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 11/12] pack-objects: shrink size " Nguyễn Thái Ngọc Duy 2018-03-08 11:42 ` [PATCH/RFC v3 12/12] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy 2018-03-16 18:31 ` [PATCH v4 00/11] nd/pack-objects-pack-struct updates Nguyễn Thái Ngọc Duy 2018-03-16 18:31 ` [PATCH v4 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy 2018-03-16 20:32 ` Junio C Hamano 2018-03-17 11:59 ` Duy Nguyen 2018-03-16 18:31 ` [PATCH v4 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-16 20:49 ` Junio C Hamano 2018-03-16 18:31 ` [PATCH v4 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-16 18:31 ` [PATCH v4 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-16 18:31 ` [PATCH v4 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-16 18:31 ` [PATCH v4 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-26 20:39 ` Stefan Beller 2018-03-16 18:31 ` [PATCH v4 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-16 20:59 ` Junio C Hamano 2018-03-16 18:31 ` [PATCH v4 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-16 19:40 ` Junio C Hamano 2018-03-16 18:31 ` [PATCH v4 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy 2018-03-16 19:49 ` Junio C Hamano 2018-03-16 21:34 ` Junio C Hamano 2018-03-16 18:31 ` [PATCH v4 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy 2018-03-16 18:32 ` [PATCH v4 11/11] pack-objects.h: reorder members to shrink " Nguyễn Thái Ngọc Duy 2018-03-16 21:02 ` Junio C Hamano 2018-03-17 12:07 ` Duy Nguyen 2018-03-17 14:10 ` [PATCH v5 00/11] nd/pack-objects-pack-struct updates Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-17 21:26 ` Ævar Arnfjörð Bjarmason 2018-03-17 14:10 ` [PATCH v5 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy 2018-03-17 19:57 ` Ævar Arnfjörð Bjarmason 2018-03-18 5:09 ` Junio C Hamano 2018-03-18 8:23 ` Duy Nguyen 2018-03-17 14:10 ` [PATCH v5 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy 2018-03-17 14:10 ` [PATCH v5 11/11] pack-objects.h: reorder members to shrink " Nguyễn Thái Ngọc Duy 2018-03-17 19:53 ` Ævar Arnfjörð Bjarmason 2018-03-18 8:49 ` Duy Nguyen 2018-03-17 19:45 ` [PATCH v5 00/11] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason 2018-03-17 19:47 ` Ævar Arnfjörð Bjarmason 2018-03-18 14:25 ` [PATCH v6 " Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 01/11] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 02/11] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 03/11] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 04/11] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 05/11] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 06/11] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 07/11] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 08/11] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 09/11] pack-objects: shrink size " Nguyễn Thái Ngọc Duy 2018-03-18 14:49 ` Ævar Arnfjörð Bjarmason 2018-03-19 16:19 ` Junio C Hamano 2018-03-19 16:23 ` Duy Nguyen 2018-03-19 16:43 ` Junio C Hamano 2018-03-19 16:54 ` Duy Nguyen 2018-03-19 18:29 ` Junio C Hamano 2018-03-19 18:45 ` Duy Nguyen 2018-03-19 20:10 ` Junio C Hamano 2018-03-20 18:08 ` Duy Nguyen 2018-03-20 18:22 ` Junio C Hamano 2018-03-21 8:03 ` Jeff King 2018-03-21 16:12 ` Duy Nguyen 2018-03-20 18:17 ` Duy Nguyen 2018-03-18 14:25 ` [PATCH v6 10/11] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy 2018-03-18 14:25 ` [PATCH v6 11/11] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy 2018-03-18 14:51 ` [PATCH v6 00/11] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason 2018-03-21 8:24 ` Jeff King 2018-03-21 15:59 ` Duy Nguyen 2018-03-21 16:17 ` Ævar Arnfjörð Bjarmason 2018-03-21 16:22 ` Duy Nguyen 2018-03-21 16:46 ` Duy Nguyen 2018-03-21 19:11 ` Junio C Hamano 2018-03-22 9:32 ` Jeff King 2018-03-22 9:46 ` Jeff King [this message] 2018-03-22 10:57 ` Duy Nguyen 2018-03-22 11:52 ` Jeff King 2018-03-22 17:04 ` Duy Nguyen 2018-03-23 1:28 ` Ramsay Jones 2018-03-23 2:46 ` Jeff King 2018-03-23 5:50 ` Jeff King 2018-03-23 16:01 ` Ramsay Jones 2018-03-24 6:40 ` Jeff King 2018-03-23 7:05 ` Duy Nguyen 2018-03-23 14:03 ` Ramsay Jones 2018-03-21 16:31 ` Ævar Arnfjörð Bjarmason 2018-03-21 16:53 ` Junio C Hamano 2018-03-21 17:00 ` Duy Nguyen 2018-03-22 8:07 ` Jeff King 2018-03-22 8:23 ` Duy Nguyen 2018-03-22 10:01 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 00/13] " Nguyễn Thái Ngọc Duy 2018-03-24 6:33 ` [PATCH v7 01/13] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy 2018-03-24 6:33 ` [PATCH v7 02/13] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-30 20:18 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 03/13] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-30 20:23 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 04/13] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-30 20:26 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 05/13] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-30 20:30 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 06/13] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-24 9:42 ` Ævar Arnfjörð Bjarmason 2018-03-24 12:26 ` Duy Nguyen 2018-03-24 12:13 ` Ævar Arnfjörð Bjarmason 2018-03-30 20:48 ` Jeff King 2018-03-31 4:51 ` Duy Nguyen 2018-03-31 10:20 ` Jeff King 2018-03-31 10:45 ` Duy Nguyen 2018-03-24 6:33 ` [PATCH v7 07/13] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-30 20:53 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 08/13] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-30 20:59 ` Jeff King 2018-03-31 4:40 ` Duy Nguyen 2018-03-31 10:17 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 09/13] pack-objects: don't check size when the object is bad Nguyễn Thái Ngọc Duy 2018-03-24 6:33 ` [PATCH v7 10/13] pack-objects: clarify the use of object_entry::size Nguyễn Thái Ngọc Duy 2018-03-30 21:04 ` Jeff King 2018-03-31 4:35 ` Duy Nguyen 2018-03-31 10:13 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 11/13] pack-objects: shrink size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-30 21:18 ` Jeff King 2018-03-24 6:33 ` [PATCH v7 12/13] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy 2018-03-30 21:24 ` Jeff King 2018-03-31 4:21 ` Duy Nguyen 2018-03-31 9:10 ` Duy Nguyen 2018-03-24 6:33 ` [PATCH v7 13/13] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy 2018-03-30 21:26 ` Jeff King 2018-03-31 4:10 ` Duy Nguyen 2018-03-26 15:13 ` [PATCH v7 00/13] nd/pack-objects-pack-struct updates Jeff King 2018-03-26 17:04 ` Duy Nguyen 2018-03-27 16:53 ` Jeff King 2018-03-31 10:02 ` [PATCH v8 00/15] " Nguyễn Thái Ngọc Duy 2018-03-31 10:02 ` [PATCH v8 01/15] t/README: mention about running the test suite in special modes Nguyễn Thái Ngọc Duy 2018-03-31 10:02 ` [PATCH v8 02/15] pack-objects: a bit of document about struct object_entry Nguyễn Thái Ngọc Duy 2018-03-31 10:02 ` [PATCH v8 03/15] pack-objects: turn type and in_pack_type to bitfields Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 04/15] pack-objects: use bitfield for object_entry::dfs_state Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 05/15] pack-objects: use bitfield for object_entry::depth Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 06/15] pack-objects: move in_pack_pos out of struct object_entry Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 07/15] pack-objects: move in_pack " Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 08/15] pack-objects: refer to delta objects by index instead of pointer Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 09/15] pack-objects: shrink z_delta_size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 10/15] pack-objects: don't check size when the object is bad Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 11/15] pack-objects: clarify the use of object_entry::size Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 12/15] pack-objects: shrink size field in struct object_entry Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 13/15] pack-objects: shrink delta_size " Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 14/15] pack-objects: reorder members to shrink " Nguyễn Thái Ngọc Duy 2018-03-31 10:03 ` [PATCH v8 15/15] ci: exercise the whole test suite with uncommon code in pack-objects Nguyễn Thái Ngọc Duy 2018-03-31 11:36 ` [PATCH v8 00/15] nd/pack-objects-pack-struct updates Ævar Arnfjörð Bjarmason 2018-03-31 12:08 ` Duy Nguyen 2018-03-31 15:43 ` Ævar Arnfjörð Bjarmason 2018-04-06 21:47 ` Jeff King 2018-03-01 9:20 ` [PATCH/RFC 0/1] Avoid expensive 'repack -ad' in gc --auto Nguyễn Thái Ngọc Duy 2018-03-01 9:20 ` [PATCH/RFC 1/1] gc --auto: exclude the largest giant pack in low-memory config Nguyễn Thái Ngọc Duy 2018-03-01 18:14 ` Junio C Hamano 2018-03-02 0:00 ` Duy Nguyen 2018-03-05 14:00 ` Ævar Arnfjörð Bjarmason 2018-03-06 10:41 ` [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto Nguyễn Thái Ngọc Duy 2018-03-06 10:41 ` [PATCH v2 1/5] fixup! Add a test showing that 'git repack' throws away grafted-away parents Nguyễn Thái Ngọc Duy 2018-03-06 18:01 ` Junio C Hamano 2018-03-06 10:41 ` [PATCH v2 2/5] repack: add --keep-pack option Nguyễn Thái Ngọc Duy 2018-03-06 18:25 ` Junio C Hamano 2018-03-07 10:19 ` Duy Nguyen 2018-03-06 10:41 ` [PATCH v2 3/5] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy 2018-03-06 19:19 ` Junio C Hamano 2018-03-07 10:48 ` Duy Nguyen 2018-03-07 18:38 ` Junio C Hamano 2018-03-12 18:56 ` Ævar Arnfjörð Bjarmason 2018-03-12 21:16 ` Junio C Hamano 2018-03-12 22:01 ` Ævar Arnfjörð Bjarmason 2018-03-15 16:48 ` Duy Nguyen 2018-03-07 10:48 ` Johannes Schindelin 2018-03-07 18:40 ` Junio C Hamano 2018-03-12 19:30 ` Ævar Arnfjörð Bjarmason 2018-03-15 17:00 ` Duy Nguyen 2018-03-15 19:21 ` Ævar Arnfjörð Bjarmason 2018-03-16 17:47 ` Duy Nguyen 2018-03-06 10:41 ` [PATCH v2 4/5] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy 2018-03-12 18:32 ` Ævar Arnfjörð Bjarmason 2018-03-16 19:14 ` Duy Nguyen 2018-03-16 20:13 ` Duy Nguyen 2018-03-06 10:41 ` [PATCH v2 5/5] pack-objects: display progress in get_object_details() Nguyễn Thái Ngọc Duy 2018-03-06 17:49 ` [PATCH v2 0/5] Avoid expensive 'repack -ad' in gc --auto Junio C Hamano 2018-03-16 19:27 ` [PATCH v3 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy 2018-03-16 19:27 ` [PATCH v3 1/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy 2018-03-16 19:27 ` [PATCH v3 2/7] gc: add --keep-base-pack Nguyễn Thái Ngọc Duy 2018-03-16 21:05 ` Ævar Arnfjörð Bjarmason 2018-03-19 17:26 ` Duy Nguyen 2018-03-19 19:04 ` Ævar Arnfjörð Bjarmason 2018-03-16 21:25 ` Ævar Arnfjörð Bjarmason 2018-03-16 19:27 ` [PATCH v3 3/7] gc: detect base packs based on gc.bigPackThreshold config Nguyễn Thái Ngọc Duy 2018-03-16 21:02 ` Ævar Arnfjörð Bjarmason 2018-03-16 19:27 ` [PATCH v3 4/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy 2018-03-16 21:14 ` Ævar Arnfjörð Bjarmason 2018-03-16 19:27 ` [PATCH v3 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy 2018-03-16 21:10 ` Ævar Arnfjörð Bjarmason 2018-03-16 19:27 ` [PATCH v3 6/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy 2018-03-16 19:27 ` [PATCH v3 7/7] pack-objects: display progress in get_object_details() Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 0/7] nd/repack-keep-pack updates Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 1/7] t7700: have closing quote of a test at the beginning of line Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 2/7] repack: add --keep-pack option Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 3/7] gc: add --keep-largest-pack option Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 4/7] gc: add gc.bigPackThreshold config Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 5/7] gc: handle a corner case in gc.bigPackThreshold Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 6/7] gc --auto: exclude base pack if not enough mem to "repack -ad" Nguyễn Thái Ngọc Duy 2018-03-24 7:25 ` [PATCH v4 7/7] pack-objects: show some progress when counting kept objects Nguyễn Thái Ngọc Duy 2018-03-02 10:18 ` Reduce pack-objects memory footprint? Duy Nguyen 2018-03-02 10:37 ` Eric Wong 2018-03-02 10:54 ` Jeff King 2018-03-02 10:55 ` Duy Nguyen 2018-03-02 14:38 ` Duy Nguyen 2018-03-17 22:05 ` Why does pack-objects use so much memory on incremental packing? Ævar Arnfjörð Bjarmason 2018-03-18 8:37 ` Duy Nguyen 2018-03-20 5:28 ` Jeff King
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: http://vger.kernel.org/majordomo-info.html * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180322094649.GA32698@sigill.intra.peff.net \ --to=peff@peff.net \ --cc=avarab@gmail.com \ --cc=e@80x24.org \ --cc=git@vger.kernel.org \ --cc=gitster@pobox.com \ --cc=pclouds@gmail.com \ --subject='Re: [PATCH v6 00/11] nd/pack-objects-pack-struct updates' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
git@vger.kernel.org list mirror (unofficial, one of many) This inbox may be cloned and mirrored by anyone: git clone --mirror https://public-inbox.org/git git clone --mirror http://ou63pmih66umazou.onion/git git clone --mirror http://czquwvybam4bgbro.onion/git git clone --mirror http://hjrcffqmbrq6wope.onion/git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V1 git git/ https://public-inbox.org/git \ git@vger.kernel.org public-inbox-index git Example config snippet for mirrors. Newsgroups are available over NNTP: nntp://news.public-inbox.org/inbox.comp.version-control.git nntp://ou63pmih66umazou.onion/inbox.comp.version-control.git nntp://czquwvybam4bgbro.onion/inbox.comp.version-control.git nntp://hjrcffqmbrq6wope.onion/inbox.comp.version-control.git nntp://news.gmane.io/gmane.comp.version-control.git note: .onion URLs require Tor: https://www.torproject.org/ code repositories for project(s) associated with this inbox: https://80x24.org/mirrors/git.git AGPL code for this site: git clone https://public-inbox.org/public-inbox.git