From: Johannes Schindelin <Johannes.Schindelin@gmx.de>
To: Linus Torvalds <torvalds@osdl.org>
Cc: Junio C Hamano <junkio@cox.net>, Sam Ravnborg <sam@ravnborg.org>,
git@vger.kernel.org
Subject: Re: dumb transports not being welcomed..
Date: Wed, 14 Sep 2005 10:38:58 +0200 (CEST) [thread overview]
Message-ID: <Pine.LNX.4.63.0509141014580.30708@wgmdd8.biozentrum.uni-wuerzburg.de> (raw)
In-Reply-To: <Pine.LNX.4.58.0509131742240.26803@g5.osdl.org>
Hi,
On Tue, 13 Sep 2005, Linus Torvalds wrote:
>
> On Wed, 14 Sep 2005, Johannes Schindelin wrote:
> >
> > IMHO the culprit is git-rev-list, which takes ages and ages for big
> > repositories (beware: this could be my Darwin client which might be
> > incapable to stop the rev enumeration in time; but if that can be done
> > unintentionally, this can be intentionally, too!).
>
> Packed too?
Yes. Almost all of it.
> git-rev-list will take a long time if the tree is unpacked and not in the
> cache. It's all disk seeks. That's _especially_ true of a full clone
> (which will walk the whole way down).
That could be the case, but my test case is a CVS project I track on one
side, and I fetch on the other side. Therefore, your diagram from your
other mail does not really apply. My history looks more or less like this:
a b c d origin
|
|
|
|
\ \ \ \ |
---------|
So, the origin is a linear CVS project. With many, many, many commits. At
one stage I broke off new git branches. I kept tracking the CVS project,
though.
What I see when fetching all heads (thanks to Junio, this is one call to
git-fetch now), where all but origin are up to date, is that it takes a
very long time. Swapping kicks in, and top tells me that 26.6% of the
memory is occupied by git-rev-list (The server has 128M, with 1G swap, and
I am unfortunately not the only user of this machine).
I fail to see why it should need those amounts of memory. (I tested this
over the ssh protocol, which should essentially do the same as git-daemon,
right?) After all, the merge point between the branches should be marked
uninteresting after one single step from each of my private branches.
> But I have tons of memory in my machines, and I haven't looked at how
> badly it does if you don't have that. I know that master.kernel.org is
> certainly not having any trouble at all with me pulling from lots of
> trees.. Maybe git-rev-list uses up lots of your memory.
That certainly is the case.
As for master.kernel.org: Unfortunately, you will not be the only puller.
And if your process needs just 5% of the RAM, then 21 pullers will be too
many.
> That said, I do think that --objects handling is _very_ CPU-hungry.
In my experience, before the swapping started, the process did not get
more than 20% CPU.
Nevertheless, I still think that it would be a good idea to reuse the
files created for the dumb transport for the intelligent transport.
Especially for a project which is more often fetched than uploaded.
I also see other strange things like packing 0 objects, and packing >0
objects after just having fetched from that repository. Hopefully I will
have time to look into that (and understand the code to begin with).
Ciao,
Dscho
next prev parent reply other threads:[~2005-09-14 8:39 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-09-13 21:07 dumb transports not being welcomed Junio C Hamano
2005-09-13 21:14 ` Sam Ravnborg
2005-09-13 21:30 ` Junio C Hamano
2005-09-13 21:42 ` Sam Ravnborg
2005-09-13 22:03 ` Horst von Brand
2005-09-13 22:23 ` Junio C Hamano
2005-09-13 22:11 ` Junio C Hamano
2005-09-13 22:21 ` Jeff Garzik
2005-09-13 22:29 ` Junio C Hamano
2005-09-14 13:21 ` Jeff Garzik
2005-09-13 22:29 ` Linus Torvalds
2005-09-13 22:37 ` Junio C Hamano
2005-09-13 22:55 ` Linus Torvalds
2005-09-13 23:02 ` Junio C Hamano
2005-09-14 2:25 ` Kay Sievers
2005-09-14 3:32 ` Junio C Hamano
2005-09-14 0:01 ` Johannes Schindelin
2005-09-14 0:57 ` Linus Torvalds
2005-09-14 1:42 ` Linus Torvalds
2005-09-14 8:38 ` Johannes Schindelin [this message]
2005-09-14 15:07 ` Linus Torvalds
2005-09-15 9:17 ` Junio C Hamano
2005-09-14 10:45 ` Sven Verdoolaege
2005-09-14 16:14 ` Junio C Hamano
2005-09-14 14:10 ` Jon Loeliger
2005-09-14 19:00 ` Junio C Hamano
2005-09-14 19:13 ` Jon Loeliger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.63.0509141014580.30708@wgmdd8.biozentrum.uni-wuerzburg.de \
--to=johannes.schindelin@gmx.de \
--cc=git@vger.kernel.org \
--cc=junkio@cox.net \
--cc=sam@ravnborg.org \
--cc=torvalds@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).