[PATCH] send-email: disable option auto-abbreviation

[PATCH] send-email: disable option auto-abbreviation

[Kyle Meyer]

send-email supports specifying format-patch options.  However, some
valid format-patch short options trigger an error because Getopt's
default auto-abbreviation is enabled.  For example, with

  git send-email -v 3 @{u}

the -v is consumed as send-email's --validate, and 3 is passed on to
the format-patch call, leading to

  fatal: ambiguous argument '3': unknown revision or path not in the
  working tree.  [...]

Disable Getopt's auto-abbreviation feature so that such options are
properly relayed to format-patch.  With this change, there is some
risk of breaking external scripts that rely on the abbreviation, but
that is hopefully unlikely given that Git does not advertise support
for auto-abbreviation and most subcommands do not support it.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 git-send-email.perl   | 2 +-
 t/t9001-send-email.sh | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 5861e99a6e..1e6d5d7677 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -24,7 +24,7 @@
 use Git;
 use Git::I18N;
 
-Getopt::Long::Configure qw/ pass_through /;
+Getopt::Long::Configure qw/ pass_through no_auto_abbrev /;
 
 package FakeTerm;
 sub new {
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 01c74b8b07..c2ebf19ec6 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -2334,6 +2334,12 @@ test_expect_success $PREREQ 'test that send-email works outside a repo' '
 		"$(pwd)/0001-add-main.patch"
 '
 
+test_expect_success $PREREQ 'send-email relays -v 4' '
+	test_when_finished "rm -f out" &&
+	git send-email --dry-run -v 4 -1 >out &&
+	grep "PATCH v4" out
+'
+
 test_expect_success $PREREQ 'test that sendmail config is rejected' '
 	test_config sendmail.program sendmail &&
 	test_must_fail git send-email \

base-commit: e7e5c6f715b2de7bea0d39c7d2ba887335b40aa0
-- 
2.38.1

Re: [PATCH] send-email: disable option auto-abbreviation

[Junio C Hamano]

Kyle Meyer <kyle@kyleam.com> writes:

> send-email supports specifying format-patch options.  However, some
> valid format-patch short options trigger an error because Getopt's
> default auto-abbreviation is enabled.  For example, with
>
>   git send-email -v 3 @{u}
>
> the -v is consumed as send-email's --validate, and 3 is passed on to
> the format-patch call, leading to
>
>   fatal: ambiguous argument '3': unknown revision or path not in the
>   working tree.  [...]
>
> Disable Getopt's auto-abbreviation feature so that such options are
> properly relayed to format-patch.  With this change, there is some
> risk of breaking external scripts that rely on the abbreviation, but
> that is hopefully unlikely given that Git does not advertise support
> for auto-abbreviation and most subcommands do not support it.

I personally have no sympathy to those who drive "format-patch" from
inside "send-email".

Having said that.

Many subcommands of "git" do take uniquely abbreviated double-dashed
option names, but it is true that we do not allow --vanything to be
given as -v even when there is no other double-dashed option that
begins with 'v', so "git send-email -v" that stands for "git
send-email --validate" indeed is an odd thing.

But robbing "git send-email --val" that expands to "--validate" from
the users is going a bit too far, I am afraid.  The right solution
for allowing "-v 3" given to "format-patch" I think is to make
send-email understand it and pass that through.  The presence of
both ("validate" => \$validate) and ("v" => \$reroll_count) in the
GetOptions() argument would prevent "-v" to be taken as "--validate"
while still allowing "--val" to be used as an abbrevatiion, no?

By the way, do we advertise support for any and all options to
format-patch when the feature to drive it from send-email is used?
Some of the options (e.g. "-o <directory>") do not make any sense in
the context I would suspect.

Re: [PATCH] send-email: disable option auto-abbreviation

[Kyle Meyer]

Junio C Hamano writes:

> Kyle Meyer <kyle@kyleam.com> writes:
[...]
>>   fatal: ambiguous argument '3': unknown revision or path not in the
>>   working tree.  [...]
>>
>> Disable Getopt's auto-abbreviation feature so that such options are
>> properly relayed to format-patch.  With this change, there is some
>> risk of breaking external scripts that rely on the abbreviation, but
>> that is hopefully unlikely given that Git does not advertise support
>> for auto-abbreviation and most subcommands do not support it.
>
> I personally have no sympathy to those who drive "format-patch" from
> inside "send-email".

I'm not one of those users myself, but I was prompted to look into this
by a report of the above error on another mailing list [*].  I do
sympathize with "skip the explicit format-patch" users that find that
error confusing.

  [*] https://yhetil.org/guix-patches/20221123190710.26517-1-paren@disroot.org

> Many subcommands of "git" do take uniquely abbreviated double-dashed
> option names, but it is true that we do not allow --vanything to be
> given as -v even when there is no other double-dashed option that
> begins with 'v', so "git send-email -v" that stands for "git
> send-email --validate" indeed is an odd thing.

Thanks for the correction. I didn't realize that many subcommands
supported abbreviated options.  I expected it to be, at most, the
remaining ones written in Perl.  When I tried out a couple of commands,
I convinced myself that auto-abbreviation wasn't generally supported:

  $ git log --onelin
  fatal: unrecognized argument: --onelin
  $ git diff --histog
  error: invalid option: --histog

But I didn't look hard enough.  Trying again, I stumbled onto a few
counterexamples (e.g., `git status --shor` works and so does `git
range-diff --le ...`).

And my claim in the commit message that "Git does not advertise support
for auto-abbreviation" is wrong.  I've now found this bit in gitcli(7):

  Abbreviating long options
  ~~~~~~~~~~~~~~~~~~~~~~~~~
  Commands that support the enhanced option parser accepts unique
  prefix of a long option as if it is fully spelled out, but use this
  with a caution.  For example, `git commit --amen` behaves as if you
  typed `git commit --amend`, but that is true only until a later version
  of Git introduces another option that shares the same prefix,
  e.g. `git commit --amenity` option.

> But robbing "git send-email --val" that expands to "--validate" from
> the users is going a bit too far, I am afraid.

Fair enough.  For the reasons above, the last sentence I wrote in the
commit message is invalid and can't justify the change.

> The right solution for allowing "-v 3" given to "format-patch" I think
> is to make send-email understand it and pass that through.  The
> presence of both ("validate" => \$validate) and ("v" =>
> \$reroll_count) in the GetOptions() argument would prevent "-v" to be
> taken as "--validate" while still allowing "--val" to be used as an
> abbrevatiion, no?

I'd think that would work, yes.  I'll look more into going this route.

With that approach, there are other cases of abbreviation intercepting
valid format patch options.  For example, send-email doesn't have the
short option -n while format-patch does, but that doesn't make it
through to format-patch:

  $ git send-email --dry-run -n @{u} | grep Subj
  Subject: [PATCH] send-email: disable option auto-abbreviation

  $ git send-email --dry-run --numbered @{u} | grep Subj
  Subject: [PATCH 1/1] send-email: disable option auto-abbreviation

> By the way, do we advertise support for any and all options to
> format-patch when the feature to drive it from send-email is used?
> Some of the options (e.g. "-o <directory>") do not make any sense in
> the context I would suspect.

Passing an -o to send-email would cause its format-patch call to fail
because send-email uses -o internally:

  $ git send-email --dry-run -o . @{u}
  fatal: two output directories?
  format-patch -o /tmp/W1ZGCr0hwv -o @{u}: command returned error: 128

In any case, here's the only relevant part I spot from
git-send-email(1):

  Patches can be specified as files, directories (which will send all
  files in the directory), or directly as a revision list.  In the last
  case, any format accepted by linkgit:git-format-patch[1] can be passed
  to git send-email, as well as options understood by
  linkgit:git-format-patch[1].

So, there's no mention that some options like -o do not make sense in
the send-email context, but perhaps that's obvious enough (at least in
my view it's much more obvious than '-v 3' and -n not being valid).

Thanks for the review.

[PATCH v2] send-email: relay '-v N' to format-patch

[Kyle Meyer]

Kyle Meyer writes:

> Junio C Hamano writes:
>
>> The right solution for allowing "-v 3" given to "format-patch" I think
>> is to make send-email understand it and pass that through.  The
>> presence of both ("validate" => \$validate) and ("v" =>
>> \$reroll_count) in the GetOptions() argument would prevent "-v" to be
>> taken as "--validate" while still allowing "--val" to be used as an
>> abbrevatiion, no?
>
> I'd think that would work, yes.  I'll look more into going this route.
>
> With that approach, there are other cases of abbreviation intercepting
> valid format patch options.  [...]

Here's a patch handling the -v case.  I don't plan on working on a more
complete fix for the other cases (as I mentioned before, I don't use
send-email to drive format-patch), but in my opinion the -v fix by
itself is still valuable.

-- >8 --
Subject: [PATCH v2] send-email: relay '-v N' to format-patch

send-email relays unrecognized arguments to its format-patch call.
Passing '-v N' leads to an error because -v is consumed as
send-email's --validate.  For example,

  git send-email -v 3 @{u}

fails with

  fatal: ambiguous argument '3': unknown revision or path not in the
  working tree.  [...]

To prevent this, add the short --reroll-count option to send-email's
main option list and explicitly provide it to the format-patch call.

There other format-patch options that send-email doesn't relay
properly, including at least -n, -N, and the diff option -D.  Punt on
these because dealing with them is more complicated:

 * they would require configuring send-email to not ignore option case

 * send-email makes three GetOptions() calls with different sets of
   options, the last being the main set of options.  Unlike -v, which
   is consumed by the last GetOptions call, the -n, -N, and -D options
   are consumed as abbreviations by the earlier calls.

Signed-off-by: Kyle Meyer <kyle@kyleam.com>
---
 git-send-email.perl   | 9 ++++++++-
 t/t9001-send-email.sh | 6 ++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/git-send-email.perl b/git-send-email.perl
index 5861e99a6e..07f2a0cbea 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -220,6 +220,10 @@ sub format_2822_time {
 my $force = 0;
 my $dump_aliases = 0;
 
+# Variables to prevent short format-patch options from being captured
+# as abbreviated send-email options
+my $reroll_count;
+
 # Handle interactive edition of files.
 my $multiedit;
 my $editor;
@@ -542,6 +546,7 @@ sub config_regexp {
 		    "batch-size=i" => \$batch_size,
 		    "relogin-delay=i" => \$relogin_delay,
 		    "git-completion-helper" => \$git_completion_helper,
+		    "v=s" => \$reroll_count,
 );
 $rc = GetOptions(%options);
 
@@ -782,7 +787,9 @@ sub is_format_patch_arg {
 	die __("Cannot run git format-patch from outside a repository\n")
 		unless $repo;
 	require File::Temp;
-	push @files, $repo->command('format-patch', '-o', File::Temp::tempdir(CLEANUP => 1), @rev_list_opts);
+	push @files, $repo->command('format-patch', '-o', File::Temp::tempdir(CLEANUP => 1),
+				    defined $reroll_count ? ('-v', $reroll_count) : (),
+				    @rev_list_opts);
 }
 
 @files = handle_backup_files(@files);
diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
index 01c74b8b07..152bd2c697 100755
--- a/t/t9001-send-email.sh
+++ b/t/t9001-send-email.sh
@@ -2334,6 +2334,12 @@ test_expect_success $PREREQ 'test that send-email works outside a repo' '
 		"$(pwd)/0001-add-main.patch"
 '
 
+test_expect_success $PREREQ 'send-email relays -v 3 to format-patch' '
+	test_when_finished "rm -f out" &&
+	git send-email --dry-run -v 3 -1 >out &&
+	grep "PATCH v3" out
+'
+
 test_expect_success $PREREQ 'test that sendmail config is rejected' '
 	test_config sendmail.program sendmail &&
 	test_must_fail git send-email \

base-commit: e7e5c6f715b2de7bea0d39c7d2ba887335b40aa0
-- 
2.38.1

Re: [PATCH v2] send-email: relay '-v N' to format-patch

[Junio C Hamano]

Kyle Meyer <kyle@kyleam.com> writes:

> Here's a patch handling the -v case.  I don't plan on working on a more
> complete fix for the other cases (as I mentioned before, I don't use
> send-email to drive format-patch), but in my opinion the -v fix by
> itself is still valuable.

Yup, I think it is a good place to stop for the first patch.  Other
people can add more when they discover the need, and anything more
complex [*] is probably not worth the effort, I would think.

    Side note: [*] we could imagine running "git format-patch -h"
    (or a new variant of it), parse its output and populate the
    %options dynamically, for example.

Will queue.  Thanks.

Re: [PATCH v2] send-email: relay '-v N' to format-patch

[Junio C Hamano]

Kyle Meyer <kyle@kyleam.com> writes:

> diff --git a/t/t9001-send-email.sh b/t/t9001-send-email.sh
> index 01c74b8b07..152bd2c697 100755
> --- a/t/t9001-send-email.sh
> +++ b/t/t9001-send-email.sh
> @@ -2334,6 +2334,12 @@ test_expect_success $PREREQ 'test that send-email works outside a repo' '
>  		"$(pwd)/0001-add-main.patch"
>  '
>  
> +test_expect_success $PREREQ 'send-email relays -v 3 to format-patch' '
> +	test_when_finished "rm -f out" &&
> +	git send-email --dry-run -v 3 -1 >out &&
> +	grep "PATCH v3" out
> +'
> +
>  test_expect_success $PREREQ 'test that sendmail config is rejected' '
>  	test_config sendmail.program sendmail &&
>  	test_must_fail git send-email \
>
> base-commit: e7e5c6f715b2de7bea0d39c7d2ba887335b40aa0

It seems that this new test, by invoking format-patch, makes a leaks
check at GitHub CI fail.

  https://github.com/git/git/actions/runs/3562362890/jobs/5984036422

Dropping PASSES_SANITIZE_LEAK from the test script would certainly
be a short-term workaround, though, but it is a rather broad
mechanism.  There should be a better way to control the leak
checker, but that is what we currently have X-<.

Re: [PATCH v2] send-email: relay '-v N' to format-patch

[Ævar Arnfjörð Bjarmason]

On Sun, Nov 27 2022, Junio C Hamano wrote:

> Kyle Meyer <kyle@kyleam.com> writes:
>
>> Here's a patch handling the -v case.  I don't plan on working on a more
>> complete fix for the other cases (as I mentioned before, I don't use
>> send-email to drive format-patch), but in my opinion the -v fix by
>> itself is still valuable.
>
> Yup, I think it is a good place to stop for the first patch.  Other
> people can add more when they discover the need, and anything more
> complex [*] is probably not worth the effort, I would think.
>
>     Side note: [*] we could imagine running "git format-patch -h"
>     (or a new variant of it), parse its output and populate the
>     %options dynamically, for example.
>
> Will queue.  Thanks.

This is just a comment on the #leftoverbits: I've looked at this option
parsing in "git-send-email" before, and IMO the right long-term fix is
to split out the *.perl code into a "git send-email--helper", and do the
option parsing in C using our parse_options().

Some of it will be a bit of a hassle, but it should be much easier after
8de2e2e41b2 (Merge branch 'ab/send-email-optim', 2021-07-22) (and the
subsquent regression fix).