[PATCH 0/2] format-patch handling in-body From headers

[PATCH 0/2] format-patch handling in-body From headers

[Jeff King]

As I've mentioned before on the list, I don't use git-send-email, but
rather a home-grown script that interacts more closely with my regular
MUA.  After embarrassing myself on multiple occasions by its inability
to automatically handle sending patches by other authors, I decided to
implement send-email's "stick the original author in a body header"
scheme.

However, doing it right is kind of tricky due to rfc822 quoting, rfc2047
encoding, and handling non-ascii names correctly. Instead, this patch
series takes a different approach: it teaches format-patch to do the
transformation itself, so that it can be used by my script along with
any other non-send-email workflows that exist (e.g., git-imap-send
suffers from the same problem).

  [1/2]: pretty.c: drop const-ness from pretty_print_context
  [2/2]: teach format-patch to place other authors into in-body "From"

-Peff

[PATCH 1/2] pretty.c: drop const-ness from pretty_print_context

[Jeff King]

In the current code, callers are expected to fill in the
pretty_print_context, and then the pretty.c functions simply
read from it. This leaves no room for the pretty.c functions
to communicate with each other by manipulating the context
(e.g., data seen while printing the header may impact how we
print the body).

Rather than introduce a new struct to hold modifiable data,
let's just drop the const-ness of the existing context
struct.

Signed-off-by: Jeff King <peff@peff.net>
---
 commit.h | 16 ++++++++++++----
 pretty.c | 10 +++++-----
 2 files changed, 17 insertions(+), 9 deletions(-)

diff --git a/commit.h b/commit.h
index 6e9c7cd..2057201 100644
--- a/commit.h
+++ b/commit.h
@@ -79,6 +79,9 @@ struct pretty_print_context {
 };
 
 struct pretty_print_context {
+	/*
+	 * Callers should tweak these to change the behavior of pp_* functions.
+	 */
 	enum cmit_fmt fmt;
 	int abbrev;
 	const char *subject;
@@ -92,6 +95,11 @@ struct pretty_print_context {
 	const char *output_encoding;
 	struct string_list *mailmap;
 	int color;
+
+	/*
+	 * Fields below here are manipulated internally by pp_* functions and
+	 * should not be counted on by callers.
+	 */
 };
 
 struct userformat_want {
@@ -111,20 +119,20 @@ void pp_title_line(const struct pretty_print_context *pp,
 extern void format_commit_message(const struct commit *commit,
 				  const char *format, struct strbuf *sb,
 				  const struct pretty_print_context *context);
-extern void pretty_print_commit(const struct pretty_print_context *pp,
+extern void pretty_print_commit(struct pretty_print_context *pp,
 				const struct commit *commit,
 				struct strbuf *sb);
 extern void pp_commit_easy(enum cmit_fmt fmt, const struct commit *commit,
 			   struct strbuf *sb);
-void pp_user_info(const struct pretty_print_context *pp,
+void pp_user_info(struct pretty_print_context *pp,
 		  const char *what, struct strbuf *sb,
 		  const char *line, const char *encoding);
-void pp_title_line(const struct pretty_print_context *pp,
+void pp_title_line(struct pretty_print_context *pp,
 		   const char **msg_p,
 		   struct strbuf *sb,
 		   const char *encoding,
 		   int need_8bit_cte);
-void pp_remainder(const struct pretty_print_context *pp,
+void pp_remainder(struct pretty_print_context *pp,
 		  const char **msg_p,
 		  struct strbuf *sb,
 		  int indent);
diff --git a/pretty.c b/pretty.c
index 9e43154..68cd7a0 100644
--- a/pretty.c
+++ b/pretty.c
@@ -406,7 +406,7 @@ static const char *show_ident_date(const struct ident_split *ident,
 	return show_date(date, tz, mode);
 }
 
-void pp_user_info(const struct pretty_print_context *pp,
+void pp_user_info(struct pretty_print_context *pp,
 		  const char *what, struct strbuf *sb,
 		  const char *line, const char *encoding)
 {
@@ -1514,7 +1514,7 @@ void format_commit_message(const struct commit *commit,
 	free(context.signature_check.signer);
 }
 
-static void pp_header(const struct pretty_print_context *pp,
+static void pp_header(struct pretty_print_context *pp,
 		      const char *encoding,
 		      const struct commit *commit,
 		      const char **msg_p,
@@ -1575,7 +1575,7 @@ static void pp_header(const struct pretty_print_context *pp,
 	}
 }
 
-void pp_title_line(const struct pretty_print_context *pp,
+void pp_title_line(struct pretty_print_context *pp,
 		   const char **msg_p,
 		   struct strbuf *sb,
 		   const char *encoding,
@@ -1618,7 +1618,7 @@ void pp_title_line(const struct pretty_print_context *pp,
 	strbuf_release(&title);
 }
 
-void pp_remainder(const struct pretty_print_context *pp,
+void pp_remainder(struct pretty_print_context *pp,
 		  const char **msg_p,
 		  struct strbuf *sb,
 		  int indent)
@@ -1650,7 +1650,7 @@ void pp_remainder(const struct pretty_print_context *pp,
 	}
 }
 
-void pretty_print_commit(const struct pretty_print_context *pp,
+void pretty_print_commit(struct pretty_print_context *pp,
 			 const struct commit *commit,
 			 struct strbuf *sb)
 {
-- 
1.8.3.26.g3f85fc7

[PATCH 2/2] teach format-patch to place other authors into in-body "From"

[Jeff King]

Format-patch generates emails with the "From" address set to
the author of each patch. If you are going to send the
emails, however, you would want to replace the author
identity with yours (if they are not the same), and bump the
author identity to an in-body header.

Normally this is handled by git-send-email, which does the
transformation before sending out the emails. However, some
workflows may not use send-email (e.g., imap-send, or a
custom script which feeds the mbox to a non-git MUA). They
could each implement this feature themselves, but getting it
right is non-trivial (one most canonicalize the identities
by reversing any RFC2047 encoding or RFC822 quoting of the
headers, which has caused many bugs in send-email over the
years).

This patch takes a different approach: it teaches
format-patch a "--from" option which handles the ident
check and in-body header while it is writing out the email.
It's much simpler to do at this level (because we haven't
done any quoting yet), and any workflow based on
format-patch can easily turn it on.

Signed-off-by: Jeff King <peff@peff.net>
---
 Documentation/git-format-patch.txt | 15 +++++++++++++
 builtin/log.c                      | 24 +++++++++++++++++++++
 commit.h                           |  5 +++++
 log-tree.c                         |  2 ++
 pretty.c                           | 38 +++++++++++++++++++++++++++++++++
 revision.h                         |  1 +
 t/t4014-format-patch.sh            | 43 ++++++++++++++++++++++++++++++++++++++
 7 files changed, 128 insertions(+)

diff --git a/Documentation/git-format-patch.txt b/Documentation/git-format-patch.txt
index 3911877..e394276 100644
--- a/Documentation/git-format-patch.txt
+++ b/Documentation/git-format-patch.txt
@@ -187,6 +187,21 @@ will want to ensure that threading is disabled for `git send-email`.
 	The negated form `--no-cc` discards all `Cc:` headers added so
 	far (from config or command line).
 
+--from::
+--from=<ident>::
+	Use `ident` in the `From:` header of each commit email. If the
+	author ident of the commit is not textually identical to the
+	provided `ident`, place a `From:` header in the body of the
+	message with the original author. If no `ident` is given, use
+	the committer ident.
++
+Note that this option is only useful if you are actually sending the
+emails and want to identify yourself as the sender, but retain the
+original author (and `git am` will correctly pick up the in-body
+header). Note also that `git send-email` already handles this
+transformation for you, and this option should not be used if you are
+feeding the result to `git send-email`.
+
 --add-header=<header>::
 	Add an arbitrary header to the email headers.  This is in addition
 	to any configured headers, and may be used multiple times.
diff --git a/builtin/log.c b/builtin/log.c
index 9e21232..14e60ae 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1112,6 +1112,21 @@ static int cc_callback(const struct option *opt, const char *arg, int unset)
 	return 0;
 }
 
+static int from_callback(const struct option *opt, const char *arg, int unset)
+{
+	char **from = opt->value;
+
+	free(*from);
+
+	if (unset)
+		*from = NULL;
+	else if (arg)
+		*from = xstrdup(arg);
+	else
+		*from = xstrdup(git_committer_info(IDENT_NO_DATE));
+	return 0;
+}
+
 int cmd_format_patch(int argc, const char **argv, const char *prefix)
 {
 	struct commit *commit;
@@ -1134,6 +1149,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	int quiet = 0;
 	int reroll_count = -1;
 	char *branch_name = NULL;
+	char *from = NULL;
 	const struct option builtin_format_patch_options[] = {
 		{ OPTION_CALLBACK, 'n', "numbered", &numbered, NULL,
 			    N_("use [PATCH n/m] even with a single patch"),
@@ -1177,6 +1193,9 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 			    0, to_callback },
 		{ OPTION_CALLBACK, 0, "cc", NULL, N_("email"), N_("add Cc: header"),
 			    0, cc_callback },
+		{ OPTION_CALLBACK, 0, "from", &from, N_("ident"),
+			    N_("set From address to <ident> (or committer ident if absent)"),
+			    PARSE_OPT_OPTARG, from_callback },
 		OPT_STRING(0, "in-reply-to", &in_reply_to, N_("message-id"),
 			    N_("make first mail a reply to <message-id>")),
 		{ OPTION_CALLBACK, 0, "attach", &rev, N_("boundary"),
@@ -1264,6 +1283,11 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 
 	rev.extra_headers = strbuf_detach(&buf, NULL);
 
+	if (from) {
+		if (split_ident_line(&rev.from_ident, from, strlen(from)))
+			die(_("invalid ident line: %s"), from);
+	}
+
 	if (start_number < 0)
 		start_number = 1;
 
diff --git a/commit.h b/commit.h
index 2057201..92c1d23 100644
--- a/commit.h
+++ b/commit.h
@@ -6,6 +6,7 @@
 #include "strbuf.h"
 #include "decorate.h"
 #include "gpg-interface.h"
+#include "string-list.h"
 
 struct commit_list {
 	struct commit *item;
@@ -95,11 +96,15 @@ struct pretty_print_context {
 	const char *output_encoding;
 	struct string_list *mailmap;
 	int color;
+	struct ident_split *from_ident;
 
 	/*
 	 * Fields below here are manipulated internally by pp_* functions and
 	 * should not be counted on by callers.
 	 */
+
+	/* Manipulated by the pp_* functions internally. */
+	struct string_list in_body_headers;
 };
 
 struct userformat_want {
diff --git a/log-tree.c b/log-tree.c
index 2eb69bc..67da27f 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -617,6 +617,8 @@ void show_log(struct rev_info *opt)
 	ctx.fmt = opt->commit_format;
 	ctx.mailmap = opt->mailmap;
 	ctx.color = opt->diffopt.use_color;
+	if (opt->from_ident.mail_begin && opt->from_ident.name_begin)
+		ctx.from_ident = &opt->from_ident;
 	pretty_print_commit(&ctx, commit, &msgbuf);
 
 	if (opt->add_signoff)
diff --git a/pretty.c b/pretty.c
index 68cd7a0..74563c9 100644
--- a/pretty.c
+++ b/pretty.c
@@ -432,6 +432,23 @@ void pp_user_info(struct pretty_print_context *pp,
 		map_user(pp->mailmap, &mailbuf, &maillen, &namebuf, &namelen);
 
 	if (pp->fmt == CMIT_FMT_EMAIL) {
+		if (pp->from_ident) {
+			struct strbuf buf = STRBUF_INIT;
+
+			strbuf_addstr(&buf, "From: ");
+			strbuf_add(&buf, namebuf, namelen);
+			strbuf_addstr(&buf, " <");
+			strbuf_add(&buf, mailbuf, maillen);
+			strbuf_addstr(&buf, ">\n");
+			string_list_append(&pp->in_body_headers,
+					   strbuf_detach(&buf, NULL));
+
+			mailbuf = pp->from_ident->mail_begin;
+			maillen = pp->from_ident->mail_end - mailbuf;
+			namebuf = pp->from_ident->name_begin;
+			namelen = pp->from_ident->name_end - namebuf;
+		}
+
 		strbuf_addstr(sb, "From: ");
 		if (needs_rfc2047_encoding(namebuf, namelen, RFC2047_ADDRESS)) {
 			add_rfc2047(sb, namebuf, namelen,
@@ -1602,6 +1619,16 @@ void pp_title_line(struct pretty_print_context *pp,
 	}
 	strbuf_addch(sb, '\n');
 
+	if (need_8bit_cte == 0) {
+		int i;
+		for (i = 0; i < pp->in_body_headers.nr; i++) {
+			if (has_non_ascii(pp->in_body_headers.items[i].string)) {
+				need_8bit_cte = 1;
+				break;
+			}
+		}
+	}
+
 	if (need_8bit_cte > 0) {
 		const char *header_fmt =
 			"MIME-Version: 1.0\n"
@@ -1615,6 +1642,17 @@ void pp_title_line(struct pretty_print_context *pp,
 	if (pp->fmt == CMIT_FMT_EMAIL) {
 		strbuf_addch(sb, '\n');
 	}
+
+	if (pp->in_body_headers.nr) {
+		int i;
+		for (i = 0; i < pp->in_body_headers.nr; i++) {
+			strbuf_addstr(sb, pp->in_body_headers.items[i].string);
+			free(pp->in_body_headers.items[i].string);
+		}
+		string_list_clear(&pp->in_body_headers, 0);
+		strbuf_addch(sb, '\n');
+	}
+
 	strbuf_release(&title);
 }
 
diff --git a/revision.h b/revision.h
index eeea6fb..7e3a185 100644
--- a/revision.h
+++ b/revision.h
@@ -140,6 +140,7 @@ struct rev_info {
 	int		numbered_files;
 	int		reroll_count;
 	char		*message_id;
+	struct ident_split from_ident;
 	struct string_list *ref_message_ids;
 	int		add_signoff;
 	const char	*extra_headers;
diff --git a/t/t4014-format-patch.sh b/t/t4014-format-patch.sh
index 58d4180..668933b 100755
--- a/t/t4014-format-patch.sh
+++ b/t/t4014-format-patch.sh
@@ -972,6 +972,49 @@ test_expect_success 'empty subject prefix does not have extra space' '
 	test_cmp expect actual
 '
 
+test_expect_success '--from=ident notices bogus ident' '
+	test_must_fail git format-patch -1 --stdout --from=foo >patch
+'
+
+test_expect_success '--from=ident replaces author' '
+	git format-patch -1 --stdout --from="Me <me@example.com>" >patch &&
+	cat >expect <<-\EOF &&
+	From: Me <me@example.com>
+
+	From: A U Thor <author@example.com>
+
+	EOF
+	sed -ne "/^From:/p; /^$/p; /^---$/q" <patch >patch.head &&
+	test_cmp expect patch.head
+'
+
+test_expect_success '--from uses committer ident' '
+	git format-patch -1 --stdout --from >patch &&
+	cat >expect <<-\EOF &&
+	From: C O Mitter <committer@example.com>
+
+	From: A U Thor <author@example.com>
+
+	EOF
+	sed -ne "/^From:/p; /^$/p; /^---$/q" <patch >patch.head &&
+	test_cmp expect patch.head
+'
+
+test_expect_success 'in-body headers trigger content encoding' '
+	GIT_AUTHOR_NAME="éxötìc" test_commit exotic &&
+	test_when_finished "git reset --hard HEAD^" &&
+	git format-patch -1 --stdout --from >patch &&
+	cat >expect <<-\EOF &&
+	From: C O Mitter <committer@example.com>
+	Content-Type: text/plain; charset=UTF-8
+
+	From: éxötìc <author@example.com>
+
+	EOF
+	sed -ne "/^From:/p; /^$/p; /^Content-Type/p; /^---$/q" <patch >patch.head &&
+	test_cmp expect patch.head
+'
+
 append_signoff()
 {
 	C=$(git commit-tree HEAD^^{tree} -p HEAD) &&
-- 
1.8.3.26.g3f85fc7

Re: [PATCH 2/2] teach format-patch to place other authors into in-body "From"

[Eric Sunshine]

On Wed, Jul 3, 2013 at 3:08 AM, Jeff King <peff@peff.net> wrote:
> Format-patch generates emails with the "From" address set to
> the author of each patch. If you are going to send the
> emails, however, you would want to replace the author
> identity with yours (if they are not the same), and bump the
> author identity to an in-body header.
>
> Normally this is handled by git-send-email, which does the
> transformation before sending out the emails. However, some
> workflows may not use send-email (e.g., imap-send, or a
> custom script which feeds the mbox to a non-git MUA). They
> could each implement this feature themselves, but getting it
> right is non-trivial (one most canonicalize the identities

s/most/must/

> by reversing any RFC2047 encoding or RFC822 quoting of the
> headers, which has caused many bugs in send-email over the
> years).
>
> This patch takes a different approach: it teaches
> format-patch a "--from" option which handles the ident
> check and in-body header while it is writing out the email.
> It's much simpler to do at this level (because we haven't
> done any quoting yet), and any workflow based on
> format-patch can easily turn it on.
>
> Signed-off-by: Jeff King <peff@peff.net>

Re: [PATCH 0/2] format-patch handling in-body From headers

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> However, doing it right is kind of tricky due to rfc822 quoting, rfc2047
> encoding, and handling non-ascii names correctly. Instead, this patch
> series takes a different approach: it teaches format-patch to do the
> transformation itself, so that it can be used by my script along with
> any other non-send-email workflows that exist (e.g., git-imap-send
> suffers from the same problem).

I think the original expectation when format-patch was done was to
use Sender: to identify you while keeping the author on From:, but
with the current world order to use in-body header, this addition
makes sense.

I wonder if we can lose some code from send-email then?

Re: [PATCH 0/2] format-patch handling in-body From headers

[Jeff King]

On Wed, Jul 03, 2013 at 01:58:22AM -0700, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > However, doing it right is kind of tricky due to rfc822 quoting, rfc2047
> > encoding, and handling non-ascii names correctly. Instead, this patch
> > series takes a different approach: it teaches format-patch to do the
> > transformation itself, so that it can be used by my script along with
> > any other non-send-email workflows that exist (e.g., git-imap-send
> > suffers from the same problem).
> 
> I think the original expectation when format-patch was done was to
> use Sender: to identify you while keeping the author on From:, but
> with the current world order to use in-body header, this addition
> makes sense.

Yeah, I think using "Sender" would simply be too confusing, as most MUAs
show only the "From", and authors of patches do not necessarily know or
care about the mailing of their patch. 

> I wonder if we can lose some code from send-email then?

Potentially, as long as we default to "--from" to turn this feature on
all the time (otherwise we are breaking the existing "format-patch &&
send-email" workflow).

It may also confuse people who mark up the patches on disk before
running send-email. I don't know if people actually change the From
header there or not.

-Peff