* [PATCH] resend: really plug memory leaks in git-svnimport
@ 2007-09-24 10:57 Stefan Sperling
2007-09-25 17:55 ` Junio C Hamano
0 siblings, 1 reply; 3+ messages in thread
From: Stefan Sperling @ 2007-09-24 10:57 UTC (permalink / raw)
To: git
[-- Attachment #1: Type: text/plain, Size: 4582 bytes --]
Junio asked me to resend this patch to the mailing list.
This version of the patch is adjusted to apply cleanly
to current HEAD.
@Junio: I'm not resending the multiple branch/tag dirs patch
just yet, because I want to polish it first -- I've got another
idea how to improve it.
Log message:
Fix pool handling in git-svnimport to avoid memory leaks.
- Create an explicit one-and-only root pool.
- Closely follow examples in SVN::Core man page.
Before calling a subversion function, create a subpool of our
root pool and make it the new default pool.
- Create a subpool for looping over svn revisions and clear
this subpool (i.e. it mark for reuse, don't decallocate it)
at the start of the loop instead of allocating new memory
with each iteration.
See http://marc.info/?l=git&m=118554191513822&w=2 for a detailed
explanation of the issue.
Signed-off-by: Stefan Sperling <stsp@elego.de>
diff --git a/git-svnimport.perl b/git-svnimport.perl
index aa5b3b2..ea8c1b2 100755
--- a/git-svnimport.perl
+++ b/git-svnimport.perl
@@ -54,6 +54,7 @@ my $branch_name = $opt_b || "branches";
my $project_name = $opt_P || "";
$project_name = "/" . $project_name if ($project_name);
my $repack_after = $opt_R || 1000;
+my $root_pool = SVN::Pool->new_default;
@ARGV == 1 or @ARGV == 2 or usage();
@@ -132,7 +133,7 @@ sub conn {
my $auth = SVN::Core::auth_open ([SVN::Client::get_simple_provider,
SVN::Client::get_ssl_server_trust_file_provider,
SVN::Client::get_username_provider]);
- my $s = SVN::Ra->new(url => $repo, auth => $auth);
+ my $s = SVN::Ra->new(url => $repo, auth => $auth, pool => $root_pool);
die "SVN connection to $repo: $!\n" unless defined $s;
$self->{'svn'} = $s;
$self->{'repo'} = $repo;
@@ -147,11 +148,10 @@ sub file {
print "... $rev $path ...\n" if $opt_v;
my (undef, $properties);
- my $pool = SVN::Pool->new();
$path =~ s#^/*##;
+ my $subpool = SVN::Pool::new_default_sub;
eval { (undef, $properties)
- = $self->{'svn'}->get_file($path,$rev,$fh,$pool); };
- $pool->clear;
+ = $self->{'svn'}->get_file($path,$rev,$fh); };
if($@) {
return undef if $@ =~ /Attempted to get checksum/;
die $@;
@@ -185,6 +185,7 @@ sub ignore {
print "... $rev $path ...\n" if $opt_v;
$path =~ s#^/*##;
+ my $subpool = SVN::Pool::new_default_sub;
my (undef,undef,$properties)
= $self->{'svn'}->get_dir($path,$rev,undef);
if (exists $properties->{'svn:ignore'}) {
@@ -202,6 +203,7 @@ sub ignore {
sub dir_list {
my($self,$path,$rev) = @_;
$path =~ s#^/*##;
+ my $subpool = SVN::Pool::new_default_sub;
my ($dirents,undef,$properties)
= $self->{'svn'}->get_dir($path,$rev,undef);
return $dirents;
@@ -358,10 +360,9 @@ open BRANCHES,">>", "$git_dir/svn2git";
sub node_kind($$) {
my ($svnpath, $revision) = @_;
- my $pool=SVN::Pool->new;
$svnpath =~ s#^/*##;
- my $kind = $svn->{'svn'}->check_path($svnpath,$revision,$pool);
- $pool->clear;
+ my $subpool = SVN::Pool::new_default_sub;
+ my $kind = $svn->{'svn'}->check_path($svnpath,$revision);
return $kind;
}
@@ -889,7 +890,7 @@ sub commit_all {
# Recursive use of the SVN connection does not work
local $svn = $svn2;
- my ($changed_paths, $revision, $author, $date, $message, $pool) = @_;
+ my ($changed_paths, $revision, $author, $date, $message) = @_;
my %p;
while(my($path,$action) = each %$changed_paths) {
$p{$path} = [ $action->action,$action->copyfrom_path, $action->copyfrom_rev, $path ];
@@ -925,14 +926,14 @@ print "Processing from $current_rev to $opt_l ...\n" if $opt_v;
my $from_rev;
my $to_rev = $current_rev - 1;
+my $subpool = SVN::Pool::new_default_sub;
while ($to_rev < $opt_l) {
+ $subpool->clear;
$from_rev = $to_rev + 1;
$to_rev = $from_rev + $repack_after;
$to_rev = $opt_l if $opt_l < $to_rev;
print "Fetching from $from_rev to $to_rev ...\n" if $opt_v;
- my $pool=SVN::Pool->new;
- $svn->{'svn'}->get_log("/",$from_rev,$to_rev,0,1,1,\&commit_all,$pool);
- $pool->clear;
+ $svn->{'svn'}->get_log("/",$from_rev,$to_rev,0,1,1,\&commit_all);
my $pid = fork();
die "Fork: $!\n" unless defined $pid;
unless($pid) {
--
Stefan Sperling <stsp@elego.de> Software Developer
elego Software Solutions GmbH HRB 77719
Gustav-Meyer-Allee 25, Gebaeude 12 Tel: +49 30 23 45 86 96
13355 Berlin Fax: +49 30 23 45 86 95
http://www.elego.de Geschaeftsfuehrer: Olaf Wagner
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] resend: really plug memory leaks in git-svnimport
2007-09-24 10:57 [PATCH] resend: really plug memory leaks in git-svnimport Stefan Sperling
@ 2007-09-25 17:55 ` Junio C Hamano
2007-09-25 18:37 ` Andreas Ericsson
0 siblings, 1 reply; 3+ messages in thread
From: Junio C Hamano @ 2007-09-25 17:55 UTC (permalink / raw)
To: Stefan Sperling; +Cc: git
Stefan Sperling <stsp@elego.de> writes:
> Junio asked me to resend this patch to the mailing list.
>
> This version of the patch is adjusted to apply cleanly
> to current HEAD.
>
> @Junio: I'm not resending the multiple branch/tag dirs patch
> just yet, because I want to polish it first -- I've got another
> idea how to improve it.
Ok.
People on the list who still use git-svnimport, could you help
with testing this patch? Will queue for 'pu' in the meantime.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] resend: really plug memory leaks in git-svnimport
2007-09-25 17:55 ` Junio C Hamano
@ 2007-09-25 18:37 ` Andreas Ericsson
0 siblings, 0 replies; 3+ messages in thread
From: Andreas Ericsson @ 2007-09-25 18:37 UTC (permalink / raw)
To: Junio C Hamano; +Cc: Stefan Sperling, git
Junio C Hamano wrote:
> Stefan Sperling <stsp@elego.de> writes:
>
>> Junio asked me to resend this patch to the mailing list.
>>
>> This version of the patch is adjusted to apply cleanly
>> to current HEAD.
>>
>> @Junio: I'm not resending the multiple branch/tag dirs patch
>> just yet, because I want to polish it first -- I've got another
>> idea how to improve it.
>
> Ok.
>
> People on the list who still use git-svnimport, could you help
> with testing this patch? Will queue for 'pu' in the meantime.
>
I used to use it, but I've given up on it in favour of git svn.
Partly because git-svn seems to get cases right that git-svnimport
didn't, but mostly because it remembers where I fetched from, which
is damn handy.
--
Andreas Ericsson andreas.ericsson@op5.se
OP5 AB www.op5.se
Tel: +46 8-230225 Fax: +46 8-230231
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2007-09-25 18:38 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-09-24 10:57 [PATCH] resend: really plug memory leaks in git-svnimport Stefan Sperling
2007-09-25 17:55 ` Junio C Hamano
2007-09-25 18:37 ` Andreas Ericsson
Code repositories for project(s) associated with this public inbox
https://80x24.org/mirrors/git.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).