git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jeff King <peff@peff.net>
To: git@vger.kernel.org
Cc: Junio C Hamano <junkio@cox.net>, spearce@spearce.org
Subject: [PATCH 1/3] fast-import: grow tree storage more aggressively
Date: Sat, 10 Mar 2007 14:16:48 -0500	[thread overview]
Message-ID: <20070310191648.GA3596@coredump.intra.peff.net> (raw)
In-Reply-To: <<20070310191515.GA3416@coredump.intra.peff.net>>

When building up a tree for a commit, fast-import
dynamically allocates memory for the tree entries. When more
space is needed, the allocated memory is increased by a
constant amount. For very large trees, this means
re-allocating and memcpy()ing the memory O(n) times.

To compound this problem, releasing the previous tree
resource does not free the memory; it is kept in a pool
for future trees. This means that each of the O(n)
allocations will consume increasing amounts of memory,
giving O(n^2) memory consumption.

Signed-off-by: Jeff King <peff@peff.net>
---
 fast-import.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/fast-import.c b/fast-import.c
index fda5018..81bc6ea 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1058,7 +1058,7 @@ static void load_tree(struct tree_entry *root)
 		struct tree_entry *e = new_tree_entry();
 
 		if (t->entry_count == t->entry_capacity)
-			root->tree = t = grow_tree_content(t, 8);
+			root->tree = t = grow_tree_content(t, t->entry_count);
 		t->entries[t->entry_count++] = e;
 
 		e->tree = NULL;
@@ -1225,7 +1225,7 @@ static int tree_content_set(
 	}
 
 	if (t->entry_count == t->entry_capacity)
-		root->tree = t = grow_tree_content(t, 8);
+		root->tree = t = grow_tree_content(t, t->entry_count);
 	e = new_tree_entry();
 	e->name = to_atom(p, (unsigned short)n);
 	e->versions[0].mode = 0;
-- 
1.5.0.3.931.g55c05

       reply	other threads:[~2007-03-10 19:16 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <<20070310191515.GA3416@coredump.intra.peff.net>
2007-03-10 19:16 ` Jeff King [this message]
2007-03-10 19:21 ` [PATCH 2/3] fast-import: tree allocation cleanups Jeff King
2007-03-11  3:21   ` Shawn O. Pearce
2007-03-11 15:51     ` Jeff King
2007-03-11 15:59       ` Jeff King
2007-03-12 19:16       ` Shawn O. Pearce
2007-03-10 19:21 ` [PATCH 3/3] fast-import: improve efficiency of tree_content_set Jeff King
2007-03-10 19:23   ` Jeff King
2007-03-10 19:40     ` [PATCH] fast-import: use binary search in tree_content_remove Jeff King
2007-03-11  3:38       ` Shawn O. Pearce
2007-03-11 16:34         ` Jeff King
2007-03-11 16:54           ` Jeff King
2007-03-11 20:19             ` Shawn O. Pearce
2007-03-12 19:13           ` Shawn O. Pearce

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070310191648.GA3596@coredump.intra.peff.net \
    --to=peff@peff.net \
    --cc=git@vger.kernel.org \
    --cc=junkio@cox.net \
    --cc=spearce@spearce.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).