git@vger.kernel.org mailing list mirror (one of many)
 help / color / mirror / code / Atom feed
From: Jonathan Nieder <jrnieder@gmail.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: Brian Gernhardt <brian@gernhardtsoftware.com>,
	"git@vger.kernel.org List" <git@vger.kernel.org>,
	David Barr <david.barr@cordelta.com>
Subject: [PATCH jn/fast-import-blob-access] t9300: use perl "head -c" clone in place of "dd bs=1 count=16000" kluge
Date: Mon, 13 Dec 2010 03:28:05 -0600	[thread overview]
Message-ID: <20101213092805.GB23443@burratino> (raw)
In-Reply-To: <7vfwu25e9q.fsf@alter.siamese.dyndns.org>

It is unfortunate to have to issue thousands of one-byte read calls to
work around dd's refusal to buffer input that would fill a block after
a short read (a3a6f4, 2010-12-13).  We could do better by using
"head -c", if it were available on all platforms we cared about.
Replace it with some simple perl.

While doing so, restructure 9300.114 to use a subshell instead of a
script.  Subshells can inherit functions (like the new head_c) from
the parent shell while external scripts cannot.

Signed-off-by: Jonathan Nieder <jrnieder@gmail.com>
---
Junio C Hamano wrote:

> It is unfortunate and feels
> yucky that we have to issue 8k+ read(2) of one byte

How about this?

 t/t9300-fast-import.sh |   84 ++++++++++++++++++++++++++++--------------------
 1 files changed, 49 insertions(+), 35 deletions(-)

diff --git a/t/t9300-fast-import.sh b/t/t9300-fast-import.sh
index ed28d3c..924a833 100755
--- a/t/t9300-fast-import.sh
+++ b/t/t9300-fast-import.sh
@@ -7,6 +7,23 @@ test_description='test git fast-import utility'
 . ./test-lib.sh
 . "$TEST_DIRECTORY"/diff-lib.sh ;# test-lib chdir's into trash
 
+# Print $1 bytes from stdin to stdout.
+#
+# This could be written as "head -c $1", but IRIX "head" does not
+# support the -c option.
+head_c () {
+	perl -e '
+		my $len = $ARGV[1];
+		while ($len > 0) {
+			my $s;
+			my $nread = sysread(STDIN, $s, $len);
+			die "cannot read: $!" unless defined($nread);
+			print $s;
+			$len -= $nread;
+		}
+	' - "$1"
+}
+
 file2_data='file2
 second line of EOF'
 
@@ -1780,44 +1797,41 @@ test_expect_success PIPE 'R: copy using cat-file' '
 	rm -f blobs &&
 	cat >frontend <<-\FRONTEND_END &&
 	#!/bin/sh
-	cat <<EOF &&
-	feature cat-blob
-	blob
-	mark :1
-	data <<BLOB
-	EOF
-	cat big
-	cat <<EOF
-	BLOB
-	cat-blob :1
-	EOF
-
-	read blob_id type size <&3 &&
-	echo "$blob_id $type $size" >response &&
-	dd of=blob bs=1 count=$size <&3 &&
-	read newline <&3 &&
-
-	cat <<EOF &&
-	commit refs/heads/copied
-	committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
-	data <<COMMIT
-	copy big file as file3
-	COMMIT
-	M 644 inline file3
-	data <<BLOB
-	EOF
-	cat blob &&
-	cat <<EOF
-	BLOB
-	EOF
 	FRONTEND_END
 
 	mkfifo blobs &&
 	(
 		export GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL GIT_COMMITTER_DATE &&
-		sh frontend 3<blobs |
-		git fast-import --cat-blob-fd=3 3>blobs
-	) &&
+		cat <<-\EOF &&
+		feature cat-blob
+		blob
+		mark :1
+		data <<BLOB
+		EOF
+		cat big &&
+		cat <<-\EOF &&
+		BLOB
+		cat-blob :1
+		EOF
+
+		read blob_id type size <&3 &&
+		echo "$blob_id $type $size" >response &&
+		head_c $size >blob <&3 &&
+		read newline <&3 &&
+
+		cat <<-EOF &&
+		commit refs/heads/copied
+		committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE
+		data <<COMMIT
+		copy big file as file3
+		COMMIT
+		M 644 inline file3
+		data <<BLOB
+		EOF
+		cat blob &&
+		echo BLOB
+	) 3<blobs |
+	git fast-import --cat-blob-fd=3 3>blobs &&
 	git show copied:file3 >actual &&
 	test_cmp expect.response response &&
 	test_cmp big actual
@@ -1845,7 +1859,7 @@ test_expect_success PIPE 'R: print blob mid-commit' '
 		EOF
 
 		read blob_id type size <&3 &&
-		dd of=actual bs=1 count=$size <&3 &&
+		head_c $size >actual <&3 &&
 		read newline <&3 &&
 
 		echo
@@ -1880,7 +1894,7 @@ test_expect_success PIPE 'R: print staged blob within commit' '
 		echo "cat-blob $to_get" &&
 
 		read blob_id type size <&3 &&
-		dd of=actual bs=1 count=$size <&3 &&
+		head_c $size >actual <&3 &&
 		read newline <&3 &&
 
 		echo deleteall
-- 
1.7.2.4.568.g3733c.dirty

      reply	other threads:[~2010-12-13  9:28 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-12-12 20:49 SIGPIPE in t9300-fast-import Brian Gernhardt
2010-12-12 21:49 ` Jonathan Nieder
2010-12-12 22:04   ` Brian Gernhardt
2010-12-13  6:31     ` [PATCH jn/fast-import-blob-access] t9300: avoid short reads from dd Jonathan Nieder
2010-12-13  7:21       ` Junio C Hamano
2010-12-13  9:28         ` Jonathan Nieder [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: http://vger.kernel.org/majordomo-info.html

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20101213092805.GB23443@burratino \
    --to=jrnieder@gmail.com \
    --cc=brian@gernhardtsoftware.com \
    --cc=david.barr@cordelta.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mirrors/git.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).