From: Linus Torvalds <torvalds@linux-foundation.org>
To: Jon Smirl <jonsmirl@gmail.com>
Cc: Nicolas Pitre <nico@cam.org>,
"Shawn O. Pearce" <spearce@spearce.org>,
Git Mailing List <git@vger.kernel.org>
Subject: Re: git-daemon on NSLU2
Date: Fri, 24 Aug 2007 16:28:58 -0700 (PDT) [thread overview]
Message-ID: <alpine.LFD.0.999.0708241616390.25853@woody.linux-foundation.org> (raw)
In-Reply-To: <9e4733910708241417l44c55306xaa322afda69c6beb@mail.gmail.com>
On Fri, 24 Aug 2007, Jon Smirl wrote:
>
> We're going something wrong in git-daemon.
Nope.
Or rather, it's mostly by design.
> I can clone the tree in five minutes using the http protocol. Using the
> git protocol would take 24hrs if I let it finish.
The http side doesn't actually do any global verification, the way
git-daemon does. So to it, everything is just temporary buffers, and you
don't need any memory at all, really.
git-daemon will create a packfile. That means that it has to generate the
*global* object reachability, and will then optimize the object packing
etc etc. That's a minimum of something like 48 bytes per object for just
the object chains, and the kernel has a *lot* of objects (over half a
million).
In addition to the object chains yourself, the native protocol will also
obviously have to actually *look* at and parse all the tree and commit
objects while it does all this, so while it doesn't necessarily keep all
of those in memory all the time, it will need to access them, and if you
don't have enough memory to cache them, that will add its own set of IO.
So I haven't checked exactly how much memory you really want to have to
serve big projects, but with some handwavy guesstimate, if you actually
want to do a good job I'd guess that you really want to have at least as
much memory as the size of largest project you are serving, and probably
add at least 10-20% on top of that.
So for the kernel, at a guess, you'd probably want to have at least 256MB
of RAM to do a half-way good job. 512MB is likely nicer and allows you to
actually cache the stuff over multiple accesses.
But I haven't actually tested. Maybe it might be bearable at 128M.
Linus
next prev parent reply other threads:[~2007-08-24 23:29 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-08-24 5:54 git-daemon on NSLU2 Jon Smirl
2007-08-24 6:21 ` Shawn O. Pearce
2007-08-24 19:38 ` Jon Smirl
2007-08-24 20:23 ` Nicolas Pitre
2007-08-24 21:17 ` Jon Smirl
2007-08-24 21:54 ` Nicolas Pitre
2007-08-24 22:06 ` Jon Smirl
2007-08-24 22:39 ` Jakub Narebski
2007-08-24 22:59 ` Junio C Hamano
2007-08-24 23:21 ` Jakub Narebski
2007-08-24 23:46 ` Jon Smirl
2007-08-25 0:04 ` Junio C Hamano
2007-08-25 7:12 ` David Kastrup
2007-08-25 17:02 ` Salikh Zakirov
2007-08-25 0:10 ` Nicolas Pitre
2007-08-24 23:28 ` Linus Torvalds [this message]
2007-08-25 15:44 ` Jon Smirl
2007-08-26 9:33 ` Jeff King
2007-08-26 16:34 ` Jon Smirl
2007-08-26 17:15 ` Linus Torvalds
2007-08-26 18:06 ` Jon Smirl
2007-08-26 18:26 ` Linus Torvalds
2007-08-26 19:00 ` Jon Smirl
2007-08-26 20:19 ` Linus Torvalds
2007-08-26 21:22 ` Junio C Hamano
2007-08-27 11:03 ` Theodore Tso
2007-08-27 16:26 ` Linus Torvalds
2007-08-26 22:24 ` Daniel Hulme
2007-08-27 0:14 ` Jakub Narebski
2007-08-24 20:27 ` Jon Smirl
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.LFD.0.999.0708241616390.25853@woody.linux-foundation.org \
--to=torvalds@linux-foundation.org \
--cc=git@vger.kernel.org \
--cc=jonsmirl@gmail.com \
--cc=nico@cam.org \
--cc=spearce@spearce.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).