[PATCH] contrib/fast-import: add perl version of simple example

[PATCH] contrib/fast-import: add perl version of simple example

[Jeff King]

This is based on the git-import.sh script, but is a little
more robust and efficient. More importantly, it should
serve as a quick template for interfacing fast-import with
perl scripts.

Signed-off-by: Jeff King <peff@peff.net>
---
 contrib/fast-import/git-import.perl |   64 +++++++++++++++++++++++++++++++++++
 1 files changed, 64 insertions(+), 0 deletions(-)
 create mode 100755 contrib/fast-import/git-import.perl

diff --git a/contrib/fast-import/git-import.perl b/contrib/fast-import/git-import.perl
new file mode 100755
index 0000000..f9fef6d
--- /dev/null
+++ b/contrib/fast-import/git-import.perl
@@ -0,0 +1,64 @@
+#!/usr/bin/perl
+#
+# Performs an initial import of a directory. This is the equivalent
+# of doing 'git init; git add .; git commit'. It's a little slower,
+# but is meant to be a simple fast-import example.
+
+use strict;
+use File::Find;
+
+my $USAGE = 'Usage: git-import branch import-message';
+my $branch = shift or die "$USAGE\n";
+my $message = shift or die "$USAGE\n";
+
+chomp(my $username = `git config user.name`);
+chomp(my $email = `git config user.email`);
+die 'You need to set user name and email'
+  unless $username && $email;
+
+system('git init');
+open(my $fi, '|-', qw(git fast-import --date-format=now))
+  or die "unable to spawn fast-import: $!";
+
+print $fi <<EOF;
+commit refs/heads/$branch
+committer $username <$email> now
+data <<MSGEOF
+$message
+MSGEOF
+
+EOF
+
+find(
+  sub {
+    if($File::Find::name eq './.git') {
+      $File::Find::prune = 1;
+      return;
+    }
+    return unless -f $_;
+
+    my $fn = $File::Find::name;
+    $fn =~ s#^.\/##;
+
+    open(my $in, '<', $_)
+      or die "unable to open $fn: $!";
+    my @st = stat($in)
+      or die "unable to stat $fn: $!";
+    my $len = $st[7];
+
+    print $fi "M 644 inline $fn\n";
+    print $fi "data $len\n";
+    while($len > 0) {
+      my $r = read($in, my $buf, $len < 4096 ? $len : 4096);
+      defined($r) or die "read error from $fn: $!";
+      $r > 0 or die "premature EOF from $fn: $!";
+      print $fi $buf;
+      $len -= $r;
+    }
+    print $fi "\n";
+
+  }, '.'
+);
+
+close($fi);
+exit $?;
-- 
1.5.3.1.967.g6bb01

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Jeff King]

On Tue, Sep 18, 2007 at 03:26:27AM -0400, Jeff King wrote:

> This is based on the git-import.sh script, but is a little

Sorry, this was meant to be 2/2 (in case you were breathless in
anticipation for the second part of the series).

-Peff

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Johannes Schindelin]

Hi,

On Tue, 18 Sep 2007, Jeff King wrote:

> This is based on the git-import.sh script, but is a little
> more robust and efficient. More importantly, it should
> serve as a quick template for interfacing fast-import with
> perl scripts.

Yes, please!  Maybe somebody will then grab the low-hanging fruit of 
writing a "git-fast-export", which can be used to dump a complete 
repository in text format?

Ciao,
Dscho

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Andreas Ericsson]

Johannes Schindelin wrote:
> Hi,
> 
> On Tue, 18 Sep 2007, Jeff King wrote:
> 
>> This is based on the git-import.sh script, but is a little
>> more robust and efficient. More importantly, it should
>> serve as a quick template for interfacing fast-import with
>> perl scripts.
> 
> Yes, please!  Maybe somebody will then grab the low-hanging fruit of 
> writing a "git-fast-export", which can be used to dump a complete 
> repository in text format?
> 

I thought that was already taken care of since format-patch handles
--root flag properly?

Otherwise, "git repack -a --window=0 --depth=0" should provide an
easily parseable dump of an entire repo.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Jeff King]

On Tue, Sep 18, 2007 at 12:28:28PM +0200, Andreas Ericsson wrote:

>> Yes, please!  Maybe somebody will then grab the low-hanging fruit of 
>> writing a "git-fast-export", which can be used to dump a complete 
>> repository in text format?
>
> I thought that was already taken care of since format-patch handles
> --root flag properly?
>
> Otherwise, "git repack -a --window=0 --depth=0" should provide an
> easily parseable dump of an entire repo.

I think he means a dump that you can meaningfully edit with sed or a
text editor. And even nicer, one that could be fed back into
git-fast-import. So you could do something like:

git-fast-export A..B >dump
vi dump
git-fast-import <dump

to rewrite history in a very flexible way.

-Peff

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Johannes Schindelin]

Hi,

On Tue, 18 Sep 2007, Andreas Ericsson wrote:

> Johannes Schindelin wrote:
>
> > Maybe somebody will then grab the low-hanging fruit of writing a 
> > "git-fast-export", which can be used to dump a complete repository in 
> > text format?
> 
> I thought that was already taken care of since format-patch handles
> --root flag properly?

Umm.  Probably you forgot about merge commits, right?  And about more than 
one branch?

> Otherwise, "git repack -a --window=0 --depth=0" should provide an easily 
> parseable dump of an entire repo.

This is not a dump.  It is a log.

Ciao,
Dscho

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Johannes Schindelin]

Hi,

On Tue, 18 Sep 2007, Jeff King wrote:

> I think he means a dump that you can meaningfully edit with sed or a 
> text editor. And even nicer, one that could be fed back into 
> git-fast-import. So you could do something like:
> 
> git-fast-export A..B >dump
> vi dump
> git-fast-import <dump
> 
> to rewrite history in a very flexible way.

Exactly what I meant.  Some people seem to have problems with 
filter-branch, but somehow no proper bug report, let alone fix, evolved 
from that.

I guess these people are more comfortable with what you just described.

Ciao,
Dscho

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Andreas Ericsson]

Johannes Schindelin wrote:
> Hi,
> 
> On Tue, 18 Sep 2007, Jeff King wrote:
> 
>> I think he means a dump that you can meaningfully edit with sed or a 
>> text editor. And even nicer, one that could be fed back into 
>> git-fast-import. So you could do something like:
>>
>> git-fast-export A..B >dump
>> vi dump
>> git-fast-import <dump
>>
>> to rewrite history in a very flexible way.
> 
> Exactly what I meant.  Some people seem to have problems with 
> filter-branch, but somehow no proper bug report, let alone fix, evolved 
> from that.
> 

The main problem is that it in my use-cases fixes a nuisance, but not a
real problem, while the man-page SYNOPSIS consists of a full 5 lines, most
of which are far from obvious at a first glance. The seeming effort involved
just doesn't seem worth bothering with.

> I guess these people are more comfortable with what you just described.
> 

I know I would be, especially since all changes would show up in an entirely
different repo. I know filter-branch is probably completely safe, but even a
0.1% risk of losing *anything* isn't worth taking to fix a small nuisance.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Sam Vilain]

Johannes Schindelin wrote:
>> I think he means a dump that you can meaningfully edit with sed or a 
>> text editor. And even nicer, one that could be fed back into 
>> git-fast-import. So you could do something like:
>>
>> git-fast-export A..B >dump
>> vi dump
>> git-fast-import <dump
>>
>> to rewrite history in a very flexible way.
>>     
>
> Exactly what I meant.  Some people seem to have problems with 
> filter-branch, but somehow no proper bug report, let alone fix, evolved 
> from that.
>
> I guess these people are more comfortable with what you just described.
>   

Guilty. my own filter-branch supports things like custom re-ordering of
commits prior to rewriting (eg, if you need to refer to one commit to
another in a commit message, you better make sure it happens in the
right order).

I personally want to be able to dump patches, including merges, to
git-format-patch format, in such a way that all other information (eg,
committer, date, etc) is preserved. And probably using something akin to
Message-Id: headers for a "patch UUID" which is what you need when
you're working with piles of patches like that.

Sam.

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Johannes Schindelin]

Hi,

On Tue, 18 Sep 2007, Sam Vilain wrote:

> I personally want to be able to dump patches, including merges, to
> git-format-patch format, in such a way that all other information (eg,
> committer, date, etc) is preserved.

We already talked about that on IRC, and you have not even _begun_ to 
think about the fundamental issues with merges-in-a-patch.  I mentioned a 
few on IRC, and am still awaiting your reply.

Unless you tackle those fundamental issues, I am afraid it is not worth 
bothering to discuss this subject any more.

Ciao,
Dscho

Re: [PATCH] contrib/fast-import: add perl version of simple example

[Sam Vilain]

Johannes Schindelin wrote:
>> I personally want to be able to dump patches, including merges, to
>> git-format-patch format, in such a way that all other information (eg,
>> committer, date, etc) is preserved.
>>     
>
> We already talked about that on IRC, and you have not even _begun_ to 
> think about the fundamental issues with merges-in-a-patch.  I mentioned a 
> few on IRC, and am still awaiting your reply.
>   

Well, you could store diffs from both parents, or a custom diff format
that marks different ancestors, etc. Sure, they wouldn't apply with
'patch', but that's the breaks.

I don't see the encoding of the information as such a fundamental and
insurmountable issue. Why do you consider it so?

Sam.