From: "H. Peter Anvin" <hpa@zytor.com>
To: Linus Torvalds <torvalds@osdl.org>
Cc: Daniel Barkalow <barkalow@iabervon.org>,
Git Mailing List <git@vger.kernel.org>,
Junio C Hamano <junkio@cox.net>,
ftpadmin@kernel.org
Subject: Re: "git-send-pack"
Date: Thu, 30 Jun 2005 15:00:38 -0700 [thread overview]
Message-ID: <42C46B86.8070006@zytor.com> (raw)
In-Reply-To: <Pine.LNX.4.58.0506301432500.14331@ppc970.osdl.org>
Linus Torvalds wrote:
>
> On Thu, 30 Jun 2005, H. Peter Anvin wrote:
>
>>It does that, but it only have to do that when the actual file has
>>changed. That's acceptable, at least for the repository sizes we're
>>likely to deal with within the medium term.
>
>
> Well, realize that "incremental packs" deltify a lot worse than a "big
> pack", since pack-files don't do deltas to objects outside the pack-file.
>
> So we'd get _some_ compression, but not as much as possible. The current
> kernel compresses down to a single 63 MB pack-file (that's with the 2.6.11
> tree too, not just the HEAD history), but without deltas it weights in at
> about 177 MB.
>
> So a "sum of incremental packs" should be somewhere in between those two
> values, even today. For a single kernel archive.
>
> So repository sizes aren't exactly trivial. I don't know how expensive
> that rsync hash thing is, but one thing you lose is the ability to
> hardlink objects, so if you have a few kernel repositories at some point
> it doesn't fit in the cache any more, and then the rsync will have to read
> that much pack object stuff from disk in addition to doing the hash. Ugh.
>
The bulk of the cost in doing the hashing comes from having to read the
file.
Well, if you grow a single pack file with appending, then you can have
delta references to earlier objects within the same pack file.
At least at this point, we'd handle a few very large files a lot better
than an enormous swarm of smaller ones.
In the end, it might be that the right thing to do for git on kernel.org
is to have a single, unified object store which isn't accessible by
anything other than git-specific protocols. There would have to be some
way of dealing with, for example, conflicting tags that apply to
different repositories, though.
-hpa
next prev parent reply other threads:[~2005-06-30 21:54 UTC|newest]
Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-06-30 17:54 "git-send-pack" Linus Torvalds
2005-06-30 18:24 ` "git-send-pack" A Large Angry SCM
2005-06-30 18:27 ` "git-send-pack" A Large Angry SCM
2005-06-30 19:04 ` "git-send-pack" Linus Torvalds
2005-06-30 18:45 ` "git-send-pack" Jan Harkes
2005-06-30 19:01 ` "git-send-pack" Mike Taht
2005-06-30 19:42 ` "git-send-pack" Linus Torvalds
2005-07-01 9:50 ` "git-send-pack" Matthias Urlichs
2005-06-30 19:44 ` "git-send-pack" Linus Torvalds
2005-06-30 20:38 ` "git-send-pack" Junio C Hamano
2005-06-30 21:05 ` "git-send-pack" Daniel Barkalow
2005-06-30 21:29 ` "git-send-pack" Linus Torvalds
2005-06-30 21:55 ` "git-send-pack" H. Peter Anvin
2005-06-30 22:26 ` "git-send-pack" Linus Torvalds
2005-06-30 23:40 ` "git-send-pack" H. Peter Anvin
2005-07-01 0:02 ` "git-send-pack" Linus Torvalds
2005-07-01 1:24 ` "git-send-pack" H. Peter Anvin
2005-07-01 23:44 ` "git-send-pack" Mike Taht
2005-07-02 0:07 ` "git-send-pack" H. Peter Anvin
2005-07-02 1:56 ` "git-send-pack" Linus Torvalds
2005-07-02 4:08 ` "git-send-pack" H. Peter Anvin
2005-07-02 4:22 ` "git-send-pack" Linus Torvalds
2005-07-02 4:29 ` "git-send-pack" H. Peter Anvin
2005-07-02 17:16 ` "git-send-pack" Linus Torvalds
2005-07-02 17:37 ` "git-send-pack" H. Peter Anvin
2005-07-02 17:44 ` "git-send-pack" Tony Luck
2005-07-02 17:48 ` "git-send-pack" H. Peter Anvin
2005-07-02 18:12 ` "git-send-pack" A Large Angry SCM
2005-06-30 22:25 ` "git-send-pack" Daniel Barkalow
2005-06-30 23:56 ` "git-send-pack" Linus Torvalds
2005-07-01 5:01 ` "git-send-pack" Daniel Barkalow
2005-06-30 21:08 ` "git-send-pack" Linus Torvalds
2005-06-30 21:10 ` "git-send-pack" Dan Holmsand
2005-06-30 19:49 ` "git-send-pack" Daniel Barkalow
2005-06-30 20:12 ` "git-send-pack" Linus Torvalds
2005-06-30 20:23 ` "git-send-pack" H. Peter Anvin
2005-06-30 20:52 ` "git-send-pack" Linus Torvalds
2005-06-30 21:23 ` "git-send-pack" H. Peter Anvin
2005-06-30 21:26 ` "git-send-pack" H. Peter Anvin
2005-06-30 21:42 ` "git-send-pack" Linus Torvalds
2005-06-30 22:00 ` H. Peter Anvin [this message]
2005-07-01 10:31 ` "git-send-pack" Matthias Urlichs
2005-07-01 14:43 ` "git-send-pack" Jan Harkes
2005-07-01 13:56 ` Tags Eric W. Biederman
2005-07-01 16:37 ` Tags H. Peter Anvin
2005-07-01 22:38 ` Tags Eric W. Biederman
2005-07-01 22:44 ` Tags H. Peter Anvin
2005-07-01 23:07 ` Tags Eric W. Biederman
2005-07-01 23:22 ` Tags Daniel Barkalow
2005-07-02 0:06 ` Tags H. Peter Anvin
2005-07-02 7:00 ` Tags Eric W. Biederman
2005-07-02 17:47 ` Tags H. Peter Anvin
2005-07-02 17:54 ` Tags Eric W. Biederman
2005-07-02 17:58 ` Tags H. Peter Anvin
2005-07-02 18:31 ` Tags Eric W. Biederman
2005-07-02 19:55 ` Tags Matthias Urlichs
2005-07-02 21:16 ` Tags H. Peter Anvin
2005-07-02 21:39 ` Tags Linus Torvalds
2005-07-02 21:42 ` Tags H. Peter Anvin
2005-07-02 22:02 ` Tags A Large Angry SCM
2005-07-02 22:20 ` Tags Linus Torvalds
2005-07-02 23:49 ` Tags A Large Angry SCM
2005-07-03 0:17 ` Tags Linus Torvalds
2005-07-02 22:14 ` Tags Petr Baudis
2005-07-02 22:17 ` Tags Linus Torvalds
2005-07-03 0:04 ` Tags Dan Holmsand
2005-07-03 22:34 ` Tags Kevin Smith
2005-07-05 13:04 ` Tags Eric W. Biederman
2005-07-05 16:21 ` Tags Daniel Barkalow
2005-07-05 17:51 ` Tags Eric W. Biederman
2005-07-05 18:33 ` Tags Linus Torvalds
2005-07-05 19:22 ` Tags Junio C Hamano
2005-07-06 18:04 ` Tags Matthias Urlichs
2005-07-07 3:31 ` Tags Eric W. Biederman
2005-07-02 18:45 ` Tags Linus Torvalds
2005-07-02 20:38 ` Tags Jan Harkes
2005-07-02 22:32 ` Tags Jan Harkes
2005-07-02 16:00 ` Tags Matthias Urlichs
2005-07-01 18:09 ` Tags Petr Baudis
2005-07-01 18:37 ` Tags H. Peter Anvin
2005-07-01 21:20 ` Tags Matthias Urlichs
2005-07-01 21:42 ` Tags Petr Baudis
2005-07-01 21:52 ` Tags H. Peter Anvin
2005-07-01 22:27 ` Tags Daniel Barkalow
2005-07-01 22:59 ` Tags Petr Baudis
2005-06-30 20:49 ` "git-send-pack" Daniel Barkalow
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: http://vger.kernel.org/majordomo-info.html
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=42C46B86.8070006@zytor.com \
--to=hpa@zytor.com \
--cc=barkalow@iabervon.org \
--cc=ftpadmin@kernel.org \
--cc=git@vger.kernel.org \
--cc=junkio@cox.net \
--cc=torvalds@osdl.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).