git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: "Jon Smirl" <jonsmirl@gmail.com>
To: "Daniel Berlin" <dberlin@dberlin.org>
Cc: "Harvey Harrison" <harvey.harrison@gmail.com>,
	"David Miller" <davem@davemloft.net>,
	ismail@pardus.org.tr, gcc@gcc.gnu.org, git@vger.kernel.org
Subject: Re: Git and GCC
Date: Thu, 6 Dec 2007 01:47:54 -0500	[thread overview]
Message-ID: <9e4733910712052247x116cabb4q48ebafffb93f7e03@mail.gmail.com> (raw)
In-Reply-To: <4aca3dc20712052117j3ef5cf99y848d4962ae8ddf33@mail.gmail.com>

On 12/6/07, Daniel Berlin <dberlin@dberlin.org> wrote:
> > While you won't get the git svn metadata if you clone the infradead
> > repo, it can be recreated on the fly by git svn if you want to start
> > commiting directly to gcc svn.
> >
> I will give this a try :)

Back when I was working on the Mozilla repository we were able to
convert the full 4GB CVS repository complete with all history into a
450MB pack file. That work is where the git-fastimport tool came from.
But it took a month of messing with the import tools to achieve this
and Mozilla still chose another VCS (mainly because of poor Windows
support in git).

Like Linus says, this type of command will yield the smallest pack file:
 git repack -a -d --depth=250 --window=250

I do agree that importing multi-gigabyte repositories is not a daily
occurrence nor a turn-key operation. There are significant issues when
translating from one VCS to another. The lack of global branch
tracking in CVS causes extreme problems on import. Hand editing of CVS
files also caused endless trouble.

The key to converting repositories of this size is RAM. 4GB minimum,
more would be better. git-repack is not multi-threaded. There were a
few attempts at making it multi-threaded but none were too successful.
If I remember right, with loads of RAM, a repack on a 450MB repository
was taking about five hours on a 2.8Ghz Core2. But this is something
you only have to do once for the import. Later repacks will reuse the
original deltas.

-- 
Jon Smirl
jonsmirl@gmail.com

  reply	other threads:[~2007-12-06  6:48 UTC|newest]

Thread overview: 95+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <4aca3dc20712051108s216d3331t8061ef45b9aa324a@mail.gmail.com>
2007-12-06  2:28 ` Git and GCC David Miller
2007-12-06  2:41   ` Daniel Berlin
2007-12-06  2:52     ` David Miller
2007-12-06  3:47       ` Daniel Berlin
2007-12-06  4:20         ` David Miller
2007-12-06  4:28           ` Harvey Harrison
2007-12-06  4:32           ` Daniel Berlin
2007-12-06  4:48             ` David Miller
2007-12-06  5:11               ` Daniel Berlin
2007-12-06  5:15                 ` Harvey Harrison
2007-12-06  5:17                   ` Daniel Berlin
2007-12-06  6:47                     ` Jon Smirl [this message]
2007-12-06  7:15                       ` Jeff King
2007-12-06 14:18                         ` Nicolas Pitre
2007-12-06 17:39                           ` Jeff King
2007-12-06 18:02                             ` Nicolas Pitre
2007-12-07  6:50                               ` Jeff King
2007-12-07  7:27                                 ` Jeff King
2007-12-06 18:35                             ` Linus Torvalds
2007-12-06 18:55                               ` Jon Smirl
2007-12-06 19:08                                 ` Nicolas Pitre
2007-12-06 21:39                                   ` Jon Smirl
2007-12-06 22:08                                     ` Nicolas Pitre
2007-12-06 22:11                                       ` Jon Smirl
2007-12-06 22:22                                       ` Jon Smirl
2007-12-06 22:30                                         ` Nicolas Pitre
2007-12-06 22:44                                           ` Jon Smirl
2007-12-07  7:31                               ` Jeff King
2007-12-08  0:47                               ` Harvey Harrison
2007-12-10  9:54                                 ` Gabriel Paubert
2007-12-10 15:35                                   ` Nicolas Pitre
2007-12-07  3:31                             ` David Miller
2007-12-07  6:38                               ` Jeff King
2007-12-07  7:10                                 ` Jon Smirl
2007-12-07 12:53                                   ` David Miller
2007-12-07 17:23                                     ` Linus Torvalds
2007-12-07 20:26                                       ` Giovanni Bajo
2007-12-07 22:14                                         ` Jakub Narebski
2007-12-07 23:04                                           ` Luke Lu
2007-12-07 23:14                                           ` Giovanni Bajo
2007-12-07 23:33                                             ` Daniel Berlin
2007-12-08 12:00                                               ` Johannes Schindelin
2007-12-08  1:55                                       ` David Miller
2007-12-10  9:57                                     ` David Miller
2007-12-06  6:09                 ` Linus Torvalds
2007-12-06  7:49                   ` Harvey Harrison
2007-12-06  8:11                     ` David Brown
2007-12-06 14:01                     ` Nicolas Pitre
2007-12-06 12:03                   ` [PATCH] gc --aggressive: make it really aggressive Johannes Schindelin
2007-12-06 13:42                     ` Theodore Tso
2007-12-06 14:15                       ` Nicolas Pitre
2007-12-06 14:22                     ` Pierre Habouzit
2007-12-06 15:55                       ` Johannes Schindelin
2007-12-06 17:05                         ` David Kastrup
2007-12-06 15:30                     ` Harvey Harrison
2007-12-06 15:56                       ` Johannes Schindelin
2007-12-06 16:19                       ` Linus Torvalds
2009-03-18 16:01                     ` Johannes Schindelin
2009-03-18 16:27                       ` Teemu Likonen
2009-03-18 18:02                       ` Nicolas Pitre
2007-12-06 18:04                   ` Git and GCC Daniel Berlin
2007-12-06 18:29                     ` Linus Torvalds
2007-12-07  2:42                     ` Harvey Harrison
2007-12-07  3:01                       ` Linus Torvalds
2007-12-07  4:06                         ` Jon Smirl
2007-12-07  4:21                           ` Nicolas Pitre
2007-12-07  5:21                           ` Linus Torvalds
2007-12-07  7:08                             ` Jon Smirl
2007-12-07 19:36                               ` Nicolas Pitre
2007-12-06 18:24                   ` NightStrike
2007-12-06 18:45                     ` Linus Torvalds
2007-12-07  5:36                       ` NightStrike
2007-12-06 19:12                   ` Jon Loeliger
2007-12-06 19:39                     ` Linus Torvalds
2007-12-07  0:29                       ` Jakub Narebski
2007-12-06 20:04                     ` Junio C Hamano
2007-12-06 21:02                       ` Junio C Hamano
2007-12-06 22:26                         ` David Kastrup
2007-12-06 22:38                           ` [OT] " Randy Dunlap
2007-12-06  4:25         ` Harvey Harrison
2007-12-06  4:54           ` Linus Torvalds
2007-12-06  5:04             ` Harvey Harrison
2007-12-06 11:57       ` Johannes Schindelin
2007-12-06 12:04         ` Ismail Dönmez
     [not found] ` <2007-12-05-21-23-14+trackit+sam@rfc1149.net>
     [not found]   ` <1196891451.10408.54.camel@brick>
     [not found]     ` <jeeje0ogvk.fsf@sykes.suse.de>
     [not found]       ` <1196897840.10408.57.camel@brick>
     [not found]         ` <38a0d8450712130640p1b5d74d6nfa124ad0b0110d64@mail.gmail.com>
     [not found]           ` <1197572755.898.15.camel@brick>
2007-12-17 22:15             ` "Argument list too long" in git remote update (Was: Git and GCC) Geert Bosch
2007-12-17 22:59               ` Johannes Schindelin
2007-12-17 23:01               ` Linus Torvalds
2007-12-18  1:34                 ` Derek Fawcus
2007-12-18  1:52                   ` Shawn O. Pearce
2007-12-08  2:21 Git and GCC J.C. Pizarro
2007-12-08 12:24 ` Johannes Schindelin
2007-12-08 19:53   ` Joe Buck
2007-12-08 20:28     ` Marco Costalba
2007-12-09  1:51       ` Daniel Berlin
2007-12-15  0:18   ` Nix

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=9e4733910712052247x116cabb4q48ebafffb93f7e03@mail.gmail.com \
    --to=jonsmirl@gmail.com \
    --cc=davem@davemloft.net \
    --cc=dberlin@dberlin.org \
    --cc=gcc@gcc.gnu.org \
    --cc=git@vger.kernel.org \
    --cc=harvey.harrison@gmail.com \
    --cc=ismail@pardus.org.tr \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).