[PATCH 1/2] git-send-email: introduce compose-encoding

[PATCH 1/2] git-send-email: introduce compose-encoding

[Krzysztof Mazur]

The introduction email (--compose option) have encoding hardcoded to
UTF-8, but invoked editor may not use UTF-8 encoding.
The encoding used by patches can be changed by the "8bit-encoding"
option, but this option does not have effect on introduction email
and equivalent for introduction email is missing.

Added compose-encoding command line option and sendemail.composeencoding
configuration option specify encoding of introduction email.

Signed-off-by: Krzysztof Mazur <krzysiek@podlesie.net>
---
 Documentation/git-send-email.txt | 5 +++++
 git-send-email.perl              | 9 ++++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index 3241170..9f09e92 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -126,6 +126,11 @@ The --to option must be repeated for each user you want on the to list.
 +
 Note that no attempts whatsoever are made to validate the encoding.
 
+--compose-encoding=<encoding>::
+	Specify encoding of compose message. Default is the value of the
+	'sendemail.composeencoding'; if that is unspecified, UTF-8 is assumed.
++
+
 
 Sending
 ~~~~~~~
diff --git a/git-send-email.perl b/git-send-email.perl
index aea66a0..107e814 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -56,6 +56,7 @@ git send-email [options] <file | directory | rev-list options >
     --in-reply-to           <str>  * Email "In-Reply-To:"
     --annotate                     * Review each patch that will be sent in an editor.
     --compose                      * Open an editor for introduction.
+    --compose-encoding      <str>  * Encoding to assume for introduction.
     --8bit-encoding         <str>  * Encoding to assume 8bit mails if undeclared
 
   Sending:
@@ -198,6 +199,7 @@ my ($identity, $aliasfiletype, @alias_files, $smtp_domain);
 my ($validate, $confirm);
 my (@suppress_cc);
 my ($auto_8bit_encoding);
+my ($compose_encoding);
 
 my ($debug_net_smtp) = 0;		# Net::SMTP, see send_message()
 
@@ -231,6 +233,7 @@ my %config_settings = (
     "confirm"   => \$confirm,
     "from" => \$sender,
     "assume8bitencoding" => \$auto_8bit_encoding,
+    "composeencoding" => \$compose_encoding,
 );
 
 my %config_path_settings = (
@@ -315,6 +318,7 @@ my $rc = GetOptions("h" => \$help,
 		    "validate!" => \$validate,
 		    "format-patch!" => \$format_patch,
 		    "8bit-encoding=s" => \$auto_8bit_encoding,
+		    "compose-encoding=s" => \$compose_encoding,
 		    "force" => \$force,
 	 );
 
@@ -638,10 +642,13 @@ EOT
 			$summary_empty = 0 unless (/^\n$/);
 		} elsif (/^\n$/) {
 			$in_body = 1;
+			if (!defined $compose_encoding) {
+				$compose_encoding = "UTF-8";
+			}
 			if ($need_8bit_cte) {
 				print $c2 "MIME-Version: 1.0\n",
 					 "Content-Type: text/plain; ",
-					   "charset=UTF-8\n",
+					   "charset=$compose_encoding\n",
 					 "Content-Transfer-Encoding: 8bit\n";
 			}
 		} elsif (/^MIME-Version:/i) {
-- 
1.7.12.2.2.g1c3c581

[PATCH 2/2] git-send-email: use locale encoding for compose

[Krzysztof Mazur]

The introduction email (--compose option) use UTF-8 as default encoding.
The current locale encoding is much better default value.

Signed-off-by: Krzysztof Mazur <krzysiek@podlesie.net>
---
 git-send-email.perl | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 107e814..139bb35 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -590,6 +590,16 @@ sub get_patch_subject {
 	die "No subject line in $fn ?";
 }
 
+sub locale_encoding {
+	my $encoding = "UTF-8";
+	eval {
+		require I18N::Langinfo;
+		I18N::Langinfo->import(qw(langinfo CODESET));
+		$encoding = langinfo(CODESET());
+	};
+	return $encoding;
+}
+
 if ($compose) {
 	# Note that this does not need to be secure, but we will make a small
 	# effort to have it be unique
@@ -643,7 +653,7 @@ EOT
 		} elsif (/^\n$/) {
 			$in_body = 1;
 			if (!defined $compose_encoding) {
-				$compose_encoding = "UTF-8";
+				$compose_encoding = locale_encoding();
 			}
 			if ($need_8bit_cte) {
 				print $c2 "MIME-Version: 1.0\n",
-- 
1.7.12.2.2.g1c3c581

Re: [PATCH 2/2] git-send-email: use locale encoding for compose

[Junio C Hamano]

Krzysztof Mazur <krzysiek@podlesie.net> writes:

> The introduction email (--compose option) use UTF-8 as default encoding.
> The current locale encoding is much better default value.
>
> Signed-off-by: Krzysztof Mazur <krzysiek@podlesie.net>
> ---
>  git-send-email.perl | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/git-send-email.perl b/git-send-email.perl
> index 107e814..139bb35 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -590,6 +590,16 @@ sub get_patch_subject {
>  	die "No subject line in $fn ?";
>  }
>  
> +sub locale_encoding {
> +	my $encoding = "UTF-8";
> +	eval {
> +		require I18N::Langinfo;
> +		I18N::Langinfo->import(qw(langinfo CODESET));
> +		$encoding = langinfo(CODESET());
> +	};
> +	return $encoding;
> +}
> +
>  if ($compose) {
>  	# Note that this does not need to be secure, but we will make a small
>  	# effort to have it be unique
> @@ -643,7 +653,7 @@ EOT
>  		} elsif (/^\n$/) {
>  			$in_body = 1;
>  			if (!defined $compose_encoding) {
> -				$compose_encoding = "UTF-8";
> +				$compose_encoding = locale_encoding();
>  			}
>  			if ($need_8bit_cte) {
>  				print $c2 "MIME-Version: 1.0\n",

These two patches make sense in general, but t9001.62 (--compose
adds MIME for utf8 body) seems to be broken by it.  I didn't check
to see if the code is broken, or the test has expecting a wrong
behaviour.  If the latter, the test needs to be updated to match the
improved new world order.

Thanks.

Re: [PATCH 2/2] git-send-email: use locale encoding for compose

[Krzysztof Mazur]

On Tue, Oct 09, 2012 at 02:34:59PM -0700, Junio C Hamano wrote:
> Krzysztof Mazur <krzysiek@podlesie.net> writes:
> 
> > The introduction email (--compose option) use UTF-8 as default encoding.
> > The current locale encoding is much better default value.
> >
> 
> These two patches make sense in general, but t9001.62 (--compose
> adds MIME for utf8 body) seems to be broken by it.  I didn't check
> to see if the code is broken, or the test has expecting a wrong
> behaviour.  If the latter, the test needs to be updated to match the
> improved new world order.
> 
> Thanks.

The second patch was broken - for C locale the ANSI_X3.4-1968 codeset
was used, which is insane because git-send-email adds Content-Type
only when non-ASCII characters were found. I think this can be fixed
by just using UTF-8 if langinfo returns ANSI_X3.4-1968.

However I think that that patch should be dropped for now, because also
other git commands like "git commit" don't use codeset from locale.
The git commit just detects invalid UTF-8 characters and prints hint
for user to set i18n.commitencoding. If you like the idea of using codeset
from locale I can send fixed patch and also change "git commit".

For now I think it's better to just take only the first patch.

I'm resending the first patch with added tests.

Thanks,
Chris
--- 
From 0d1fccc5e70367f3eeb2372b8fc24401bf88d748 Mon Sep 17 00:00:00 2001
From: Krzysztof Mazur <krzysiek@podlesie.net>
Date: Wed, 10 Oct 2012 00:17:29 +0200
Subject: [PATCH] git-send-email: introduce compose-encoding

The introduction email (--compose option) have encoding hardcoded to
UTF-8, but invoked editor may not use UTF-8 encoding.
The encoding used by patches can be changed by the "8bit-encoding"
option, but this option does not have effect on introduction email
and equivalent for introduction email is missing.

Added compose-encoding command line option and sendemail.composeencoding
configuration option specify encoding of introduction email.

Signed-off-by: Krzysztof Mazur <krzysiek@podlesie.net>
---
 Documentation/git-send-email.txt |  5 ++++
 git-send-email.perl              |  9 ++++++-
 t/t9001-send-email.sh            | 55 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 68 insertions(+), 1 deletion(-)

diff --git a/Documentation/git-send-email.txt b/Documentation/git-send-email.txt
index 3241170..9f09e92 100644
--- a/Documentation/git-send-email.txt
+++ b/Documentation/git-send-email.txt
@@ -126,6 +126,11 @@ The --to option must be repeated for each user you want on the to list.
 +
 Note that no attempts whatsoever are made to validate the encoding.
 
+--compose-encoding=<encoding>::
+	Specify encoding of compose message. Default is the value of the
+	'sendemail.composeencoding'; if that is unspecified, UTF-8 is assumed.
++
+
 
 Sending
 ~~~~~~~
diff --git a/git-send-email.perl b/git-send-email.perl
index aea66a0..107e814 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -56,6 +56,7 @@ git send-email [options] <file | directory | rev-list options >
     --in-reply-to           <str>  * Email "In-Reply-To:"
     --annotate                     * Review each patch that will be sent in an editor.
     --compose                      * Open an editor for introduction.
+    --compose-encoding      <str>  * Encoding to assume for introduction.
     --8bit-encoding         <str>  * Encoding to assume 8bit mails if undeclared
 
   Sending:
@@ -198,6 +199,7 @@ my ($identity, $aliasfiletype, @alias_files, $smtp_domain);
 my ($validate, $confirm);
 my (@suppress_cc);
 my ($auto_8bit_encoding);
+my ($compose_encoding);
 
 my ($debug_net_smtp) = 0;		# Net::SMTP, see send_message()
 
@@ -231,6 +233,7 @@ my %config_settings = (
     "confirm"   => \$confirm,
     "from" => \$sender,
     "assume8bitencoding" => \$auto_8bit_encoding,
+    "composeencoding" => \$compose_encoding,
 );
 
 my %config_path_settings = (
@@ -315,6 +318,7 @@ my $rc = GetOptions("h" => \$help,
 		    "validate!" => \$validate,
 		    "format-patch!" => \$format_patch,
 		    "8bit-encoding=s" => \$auto_8bit_encoding,
+		    "compose-encoding=s" => \$compose_encoding,
 		    "force" => \$force,
 	 );
 
@@ -638,10 +642,13 @@ EOT
 			$summary_empty = 0 unless (/^\n$/);
 		} elsif (/^\n$/) {
 			$in_body = 1;
+			if (!defined $compose_encoding) {
+				$compose_encoding = "UTF-8";
+			}
 			if ($need_8bit_cte) {
 				print $c2 "MIME-Version: 1.0\n",
 					 "Content-Type: text/plain; ",
-					   "charset=UTF-8\n",
+					   "charset=$compose_encoding\n",
 					 "Content-Transfer-Encoding: 8bit\n";
 			}
 		} elsif (/^MIME-Version:/i) {
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 0351228..265ae04 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -854,6 +854,61 @@ test_expect_success $PREREQ 'utf8 author is correctly passed on' '
 	grep "^From: Fßùný NâmÊ <odd_?=mail@example.com>" msgtxt1
 '
 
+test_expect_success $PREREQ 'sendemail.composeencoding works' '
+	clean_fake_sendmail &&
+	git config sendemail.composeencoding iso-8859-1 &&
+	(echo "#!$SHELL_PATH" &&
+	 echo "echo utf8 body: àÊÏÜú >>\"\$1\""
+	) >fake-editor-utf8 &&
+	chmod +x fake-editor-utf8 &&
+	  GIT_EDITOR="\"$(pwd)/fake-editor-utf8\"" \
+	  git send-email \
+	  --compose --subject foo \
+	  --from="Example <nobody@example.com>" \
+	  --to=nobody@example.com \
+	  --smtp-server="$(pwd)/fake.sendmail" \
+	  $patches &&
+	grep "^utf8 body" msgtxt1 &&
+	grep "^Content-Type: text/plain; charset=iso-8859-1" msgtxt1
+'
+
+test_expect_success $PREREQ '--compose-encoding works' '
+	clean_fake_sendmail &&
+	(echo "#!$SHELL_PATH" &&
+	 echo "echo utf8 body: àÊÏÜú >>\"\$1\""
+	) >fake-editor-utf8 &&
+	chmod +x fake-editor-utf8 &&
+	  GIT_EDITOR="\"$(pwd)/fake-editor-utf8\"" \
+	  git send-email \
+	  --compose-encoding iso-8859-1 \
+	  --compose --subject foo \
+	  --from="Example <nobody@example.com>" \
+	  --to=nobody@example.com \
+	  --smtp-server="$(pwd)/fake.sendmail" \
+	  $patches &&
+	grep "^utf8 body" msgtxt1 &&
+	grep "^Content-Type: text/plain; charset=iso-8859-1" msgtxt1
+'
+
+test_expect_success $PREREQ '--compose-encoding overrides sendemail.composeencoding' '
+	clean_fake_sendmail &&
+	git config sendemail.composeencoding iso-8859-1 &&
+	(echo "#!$SHELL_PATH" &&
+	 echo "echo utf8 body: àÊÏÜú >>\"\$1\""
+	) >fake-editor-utf8 &&
+	chmod +x fake-editor-utf8 &&
+	  GIT_EDITOR="\"$(pwd)/fake-editor-utf8\"" \
+	  git send-email \
+	  --compose-encoding iso-8859-2 \
+	  --compose --subject foo \
+	  --from="Example <nobody@example.com>" \
+	  --to=nobody@example.com \
+	  --smtp-server="$(pwd)/fake.sendmail" \
+	  $patches &&
+	grep "^utf8 body" msgtxt1 &&
+	grep "^Content-Type: text/plain; charset=iso-8859-2" msgtxt1
+'
+
 test_expect_success $PREREQ 'detects ambiguous reference/file conflict' '
 	echo master > master &&
 	git add master &&
-- 
1.7.12.2.2.g1c3c581