[BUG] Out of memory in Git.pm

[BUG] Out of memory in Git.pm

[Robert Wruck]

Hi,

there seems to be a problem in Git.pm (git-1.7.4.1, sub cat_blob, line 
891) where the whole object is read into memory ($blob) before writing 
to $fh:

	my $blob;
	my $bytesRead = 0;

	while (1) {
		my $bytesLeft = $size - $bytesRead;
		last unless $bytesLeft;

		my $bytesToRead = $bytesLeft < 1024 ? $bytesLeft : 1024;
		my $read = read($in, $blob, $bytesToRead, $bytesRead);
		unless (defined($read)) {
			$self->_close_cat_blob();
			throw Error::Simple("in pipe went bad");
		}

		$bytesRead += $read;
	}

Maybe it would be better to write each chunk directly to $fh ?
If so, I could create a patch.

-Robert

Re: [BUG] Out of memory in Git.pm

[Alexander Miseler]

Robert Wruck <wruck <at> tweerlei.de> writes:
> there seems to be a problem in Git.pm (git-1.7.4.1, sub cat_blob, line 
> 891) where the whole object is read into memory ($blob) before writing 


There already seems to be a fix for this in 
git://repo.or.cz/git/mingw/4msysgit.git but I couldn't find the commit in 
git://git.kernel.org/pub/scm/git/git.git:

Author:		Gregor Uhlenheuer (kongo2002@googlemail.com)
Author date:	13 days ago (Fri Feb 18 11:42:12 2011 +0100)
Committer:	Johannes Schindelin (johannes.schindelin@gmx.de)
Commit date:	13 days ago (Fri Feb 18 17:09:08 2011 +0100)
Commit hash:	e91fdbabfe29b6e37f1bc582316a82f7784a0275


Git.pm: Use stream-like writing in cat_blob()

This commit fixes the issue with the handling of large files causing an
'Out of memory' perl exception. Instead of reading and writing the whole
blob at once now the blob is written in small pieces.

The problem was raised and discussed in this mail to the msysGit mailing
list: http://thread.gmane.org/gmane.comp.version-control.msysgit/12080

Signed-off-by: Gregor Uhlenheuer <kongo2002@googlemail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Contained in branches: devel
Contained in no tag