[PATCH] git-format-patch, git-send-email: generate/handle escaped >From

[PATCH] git-format-patch, git-send-email: generate/handle escaped >From

[Paolo Bonzini]

I noticed that the mbox files generated by git-format-patch (especially
with --stdout) are not proper in the sense that the lines starting with
"From " are not escaped with a > sign.  This is unlikely to cause problems
with mail clients such as mutt, but many scripts designed to work on
mbox files will fumble in this case.

This patch fixes it and dually unescapes the lines in git-send-email.

Signed-off-by: Paolo Bonzini <bonzini@gnu.org>
---
 git-send-email.perl     |    1 +
 pretty.c                |    2 ++
 t/t4014-format-patch.sh |   13 +++++++++++++
 t/t9001-send-email.sh   |   15 +++++++++++++++
 4 files changed, 31 insertions(+), 0 deletions(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 4c795a4..de57303 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -1085,6 +1085,7 @@ foreach my $t (@files) {
 	}
 	# Now parse the message body
 	while(<F>) {
+		s/^>From /From /;
 		$message .=  $_;
 		if (/^(Signed-off-by|Cc): (.*)$/i) {
 			chomp;
diff --git a/pretty.c b/pretty.c
index e5328da..d7f8228 100644
--- a/pretty.c
+++ b/pretty.c
@@ -868,6 +868,8 @@ void pp_remainder(enum cmit_fmt fmt,
 			memset(sb->buf + sb->len, ' ', indent);
 			strbuf_setlen(sb, sb->len + indent);
 		}
+		if (fmt == CMIT_FMT_EMAIL && !prefixcmp(line, "From "))
+			strbuf_addch(sb, '>');
 		strbuf_add(sb, line, linelen);
 		strbuf_addch(sb, '\n');
 	}
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 922a894..8474718 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -28,6 +28,13 @@ test_expect_success setup '
 	git update-index file &&
 	git commit -m "Side changes #3 with \\n backslash-n in it." &&
 
+	git checkout -b more &&
+	for i in A B C; do echo "$i"; done >>file &&
+	git update-index file &&
+	git commit -m "More changes
+
+From this point on..." &&
+
 	git checkout master &&
 	git diff-tree -p C2 | git apply --index &&
 	git commit -m "Master accepts moral equivalent of #2"
@@ -516,4 +523,10 @@ test_expect_success 'format-patch --signoff' '
 	grep "^Signed-off-by: $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL>"
 '
 
+test_expect_success 'format-patch escapes From' '
+	git format-patch more^..more --stdout > patch9 &&
+	grep "^>From this" patch9 &&
+	! grep "^From this" patch9
+'
+
 test_done
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 2ce24cd..5c6f0f4 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -168,6 +168,21 @@ test_expect_success 'no patch was sent' '
 	! test -e commandline1
 '
 
+test_expect_success 'fix >From lines' '
+	clean_fake_sendmail &&
+	cp $patches gt-from.patch &&
+	echo ">From abcdef" >>gt-from.patch &&
+	git send-email \
+		--from="Example <nobody@example.com>" \
+		--to=nobody@example.com \
+		--smtp-server="$(pwd)/fake.sendmail" \
+		gt-from.patch \
+		2>errors &&
+	test -e msgtxt1 &&
+	! grep "^>From abcdef" msgtxt1 &&
+	grep "^From abcdef" msgtxt1
+'
+
 test_expect_success 'Author From: in message body' '
 	clean_fake_sendmail &&
 	git send-email \
-- 
1.6.0.3

Re: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

[Jeff King]

On Thu, Jun 11, 2009 at 12:00:34PM +0200, Paolo Bonzini wrote:

> I noticed that the mbox files generated by git-format-patch (especially
> with --stdout) are not proper in the sense that the lines starting with
> "From " are not escaped with a > sign.  This is unlikely to cause problems
> with mail clients such as mutt, but many scripts designed to work on
> mbox files will fumble in this case.
> 
> This patch fixes it and dually unescapes the lines in git-send-email.

Ugh. Can we please at least make this optional?  Many modern MTAs handle
the current situation just fine by being much more strict in their
"From" matching (i.e., you would need something that really looks like a
valid "From" line to get a match), and do not do ">From" de-quoting at
all (mutt is such an example).

Some even take it a step farther and use Content-Length headers, though
I do not think that is a good idea here (we encourage people to munge
the contents while stored as an mbox).

> +++ b/pretty.c
> @@ -868,6 +868,8 @@ void pp_remainder(enum cmit_fmt fmt,
>  			memset(sb->buf + sb->len, ' ', indent);
>  			strbuf_setlen(sb, sb->len + indent);
>  		}
> +		if (fmt == CMIT_FMT_EMAIL && !prefixcmp(line, "From "))
> +			strbuf_addch(sb, '>');

This is the lossy "mboxo" conversion. A quoted From is now
indistinguishable from an original ">From". To be reversible, you need
to quote "s/^>*From />&/".

But of course, which conversion you want depends entirely on what you
are going to feed it to, and which mbox they are expecting. Which is why
it really should be configurable.

Your use case looks to be feeding it to send-email; I suspect you would
be better served by improving the 'From ' detection in send-email,
probably to something like:

  /^From \S+ \w{3} \w{3} \d+ \d\d:\d\d:\d\d \d+/

though that may be too strict. It would probably make sense to steal one
from one of the many mbox-reading CPAN modules.

-Peff

Re: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

[Paolo Bonzini]

> But of course, which conversion you want depends entirely on what you
> are going to feed it to, and which mbox they are expecting. Which is why
> it really should be configurable.
> 
> Your use case looks to be feeding it to send-email

Almost, :-) because git-send-email does not support sending multiple 
messages out of one mbox file.  I have an upcoming patch to add this 
support, and I was just "feeling the waters" before posting it.

> I suspect you would
> be better served by improving the 'From ' detection in send-email,
> probably to something like:
> 
>   /^From \S+ \w{3} \w{3} \d+ \d\d:\d\d:\d\d \d+/
> 
> though that may be too strict. It would probably make sense to steal one
> from one of the many mbox-reading CPAN modules.

Good idea.  I'll let others speak and can make this configurable (as 
well as use the s/^>*From />&/ conversion), but if no one speaks, I'll 
change my git-send-email patch to use the above regex or a similar one, 
and withdraw this patch.

Thanks for the prompt remark!

Paolo

Re: [PATCH] git-format-patch, git-send-email: generate/handle escaped >From

[Paolo Bonzini]

Paolo Bonzini wrote:
> 
>> But of course, which conversion you want depends entirely on what you
>> are going to feed it to, and which mbox they are expecting. Which is why
>> it really should be configurable.
>>
>> Your use case looks to be feeding it to send-email
> 
> Almost, :-) because git-send-email does not support sending multiple 
> messages out of one mbox file.  I have an upcoming patch to add this 
> support, and I was just "feeling the waters" before posting it.

I decided that it's easier to support sending a single mbox file with 
this alias instead:

send-mbox = "!bash -ec 'eval f=\\$$#; eval set -- `seq -f\"\\$%.0f\" 1 
$(($#-1))`; if last=`mkdir .mboxsplit && git mailsplit -d4 -o.mboxsplit 
-b -- \"$f\"`; then echo Found $last messages in \"$f\"; git send-email 
\"$@\" .mboxsplit && rm -rf .mboxsplit; else rm -rf .mboxsplit; fi' -"

So, patch withdrawn.  I don't think I'll post the patches in my 
mailsplit-with-sendemail branch.

Paolo