[PATCH v4] send-email: --batch-size to work around some SMTP server limit

[PATCH v4] send-email: --batch-size to work around some SMTP server limit

[xiaoqiang zhao]

Some email servers (e.g. smtp.163.com) limit the number emails to be
sent per session(connection) and this will lead to a faliure when
sending many messages.

Teach send-email to disconnect after sending a number of messages
(configurable via the --batch-size=<num> option), wait for a few
seconds (configurable via the --relogin-delay=<seconds> option) and
reconnect, to work around such a limit.

Also add this two configure option for git config command.

Note:
   Re-authentication will happen every $<batch-size> messages, so it
will be much more acceptable if you use some form of credential helper
(e.g. the 'sendemail.smtppass' config option), otherwise you will have
to retype password every time when asked.

Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
---
 contrib/completion/git-completion.bash |  2 ++
 git-send-email.perl                    | 18 ++++++++++++++++++
 2 files changed, 20 insertions(+)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index af658995d..29496353a 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2608,6 +2608,8 @@ _git_config ()
 		sendemail.thread
 		sendemail.to
 		sendemail.validate
+		sendemail.smtpbatchsize
+		sendemail.smtprelogindelay
 		showbranch.default
 		status.relativePaths
 		status.showUntrackedFiles
diff --git a/git-send-email.perl b/git-send-email.perl
index eea0a517f..071d1ab9d 100755
--- a/git-send-email.perl
+++ b/git-send-email.perl
@@ -81,6 +81,10 @@ git send-email --dump-aliases
                                      This setting forces to use one of the listed mechanisms.
     --smtp-debug            <0|1>  * Disable, enable Net::SMTP debug.
 
+    --batch-size            <int>  * send max <int> message per connection.
+    --relogin-delay         <int>  * delay <int> seconds between two successive login, default to 1,
+                                     This option can only be used with --batch-size
+
   Automating:
     --identity              <str>  * Use the sendemail.<id> options.
     --to-cmd                <str>  * Email To: via `<str> \$patch_path`
@@ -153,6 +157,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };
 my $have_mail_address = eval { require Mail::Address; 1 };
 my $smtp;
 my $auth;
+my $num_sent = 0;
 
 # Regexes for RFC 2047 productions.
 my $re_token = qr/[^][()<>@,;:\\"\/?.= \000-\037\177-\377]+/;
@@ -216,6 +221,7 @@ my ($cover_cc, $cover_to);
 my ($to_cmd, $cc_cmd);
 my ($smtp_server, $smtp_server_port, @smtp_server_options);
 my ($smtp_authuser, $smtp_encryption, $smtp_ssl_cert_path);
+my ($batch_size, $relogin_delay);
 my ($identity, $aliasfiletype, @alias_files, $smtp_domain, $smtp_auth);
 my ($validate, $confirm);
 my (@suppress_cc);
@@ -247,6 +253,8 @@ my %config_settings = (
     "smtppass" => \$smtp_authpass,
     "smtpdomain" => \$smtp_domain,
     "smtpauth" => \$smtp_auth,
+    "smtpbatchsize" => \$batch_size,
+    "smtprelogindelay" => \$relogin_delay,
     "to" => \@initial_to,
     "tocmd" => \$to_cmd,
     "cc" => \@initial_cc,
@@ -358,6 +366,8 @@ $rc = GetOptions(
 		    "force" => \$force,
 		    "xmailer!" => \$use_xmailer,
 		    "no-xmailer" => sub {$use_xmailer = 0},
+		    "batch-size=i" => \$batch_size,
+		    "relogin-delay=i" => \$relogin_delay,
 	 );
 
 usage() if $help;
@@ -1664,6 +1674,14 @@ foreach my $t (@files) {
 		}
 	}
 	$message_id = undef;
+	$num_sent++;
+	if ($num_sent == $batch_size) {
+		$num_sent = 0;
+		$smtp->quit if defined $smtp;
+		undef $smtp;
+		undef $auth;
+		sleep($relogin_delay);
+	}
 }
 
 # Execute a command (e.g. $to_cmd) to get a list of email addresses
-- 
2.13.0.rc2.49.g12cd49e6e

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[Junio C Hamano]

xiaoqiang zhao <zxq_yx_007@163.com> writes:

> Some email servers (e.g. smtp.163.com) limit the number emails to be
> sent per session(connection) and this will lead to a faliure when
> sending many messages.
>
> Teach send-email to disconnect after sending a number of messages
> (configurable via the --batch-size=<num> option), wait for a few
> seconds (configurable via the --relogin-delay=<seconds> option) and
> reconnect, to work around such a limit.
>
> Also add this two configure option for git config command.

s/configure/configuration/; "for git config command" is better left
unsaid (too obvious).

> Note:
>    Re-authentication will happen every $<batch-size> messages, so it
> will be much more acceptable if you use some form of credential helper
> (e.g. the 'sendemail.smtppass' config option), otherwise you will have
> to retype password every time when asked.

I think this deserves to be in the end-user documentation (i.e. the
part of your patch that updates Documentation/git-send-email.txt).

Other than that, looking good ;-)

Thanks.

> Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
> ---
>  contrib/completion/git-completion.bash |  2 ++
>  git-send-email.perl                    | 18 ++++++++++++++++++
>  2 files changed, 20 insertions(+)

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[Jan Viktorin]

Hello,

with this patch applied to git 2.12, I could see:

Use of uninitialized value $batch_size in numeric eq (==) at /usr/lib/git-core/git-send-email line 1679

when --batch-size is NOT used. See below...

On Sat, 13 May 2017 09:57:26 +0800
xiaoqiang zhao <zxq_yx_007@163.com> wrote:

> Some email servers (e.g. smtp.163.com) limit the number emails to be
> sent per session(connection) and this will lead to a faliure when
> sending many messages.
> 
> Teach send-email to disconnect after sending a number of messages
> (configurable via the --batch-size=<num> option), wait for a few
> seconds (configurable via the --relogin-delay=<seconds> option) and
> reconnect, to work around such a limit.
> 
> Also add this two configure option for git config command.
> 
> Note:
>    Re-authentication will happen every $<batch-size> messages, so it
> will be much more acceptable if you use some form of credential helper
> (e.g. the 'sendemail.smtppass' config option), otherwise you will have
> to retype password every time when asked.
> 
> Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
> ---
>  contrib/completion/git-completion.bash |  2 ++
>  git-send-email.perl                    | 18 ++++++++++++++++++
>  2 files changed, 20 insertions(+)
> 
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index af658995d..29496353a 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -2608,6 +2608,8 @@ _git_config ()
>  		sendemail.thread
>  		sendemail.to
>  		sendemail.validate
> +		sendemail.smtpbatchsize
> +		sendemail.smtprelogindelay
>  		showbranch.default
>  		status.relativePaths
>  		status.showUntrackedFiles
> diff --git a/git-send-email.perl b/git-send-email.perl
> index eea0a517f..071d1ab9d 100755
> --- a/git-send-email.perl
> +++ b/git-send-email.perl
> @@ -81,6 +81,10 @@ git send-email --dump-aliases
>                                       This setting forces to use one of the listed mechanisms.
>      --smtp-debug            <0|1>  * Disable, enable Net::SMTP debug.
>  
> +    --batch-size            <int>  * send max <int> message per connection.
> +    --relogin-delay         <int>  * delay <int> seconds between two successive login, default to 1,
> +                                     This option can only be used with --batch-size
> +
>    Automating:
>      --identity              <str>  * Use the sendemail.<id> options.
>      --to-cmd                <str>  * Email To: via `<str> \$patch_path`
> @@ -153,6 +157,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };
>  my $have_mail_address = eval { require Mail::Address; 1 };
>  my $smtp;
>  my $auth;
> +my $num_sent = 0;
>  
>  # Regexes for RFC 2047 productions.
>  my $re_token = qr/[^][()<>@,;:\\"\/?.= \000-\037\177-\377]+/;
> @@ -216,6 +221,7 @@ my ($cover_cc, $cover_to);
>  my ($to_cmd, $cc_cmd);
>  my ($smtp_server, $smtp_server_port, @smtp_server_options);
>  my ($smtp_authuser, $smtp_encryption, $smtp_ssl_cert_path);
> +my ($batch_size, $relogin_delay);
>  my ($identity, $aliasfiletype, @alias_files, $smtp_domain, $smtp_auth);
>  my ($validate, $confirm);
>  my (@suppress_cc);
> @@ -247,6 +253,8 @@ my %config_settings = (
>      "smtppass" => \$smtp_authpass,
>      "smtpdomain" => \$smtp_domain,
>      "smtpauth" => \$smtp_auth,
> +    "smtpbatchsize" => \$batch_size,
> +    "smtprelogindelay" => \$relogin_delay,
>      "to" => \@initial_to,
>      "tocmd" => \$to_cmd,
>      "cc" => \@initial_cc,
> @@ -358,6 +366,8 @@ $rc = GetOptions(
>  		    "force" => \$force,
>  		    "xmailer!" => \$use_xmailer,
>  		    "no-xmailer" => sub {$use_xmailer = 0},
> +		    "batch-size=i" => \$batch_size,
> +		    "relogin-delay=i" => \$relogin_delay,
>  	 );
>  
>  usage() if $help;
> @@ -1664,6 +1674,14 @@ foreach my $t (@files) {
>  		}
>  	}
>  	$message_id = undef;
> +	$num_sent++;
> +	if ($num_sent == $batch_size) {

This is the line. I think, batch_size can be sometimes uninitialized
while this statement is executed.

Regards
Jan

> +		$num_sent = 0;
> +		$smtp->quit if defined $smtp;
> +		undef $smtp;
> +		undef $auth;
> +		sleep($relogin_delay);
> +	}
>  }
>  
>  # Execute a command (e.g. $to_cmd) to get a list of email addresses

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[赵小强]

> 在 2017年5月16日，20:10，Jan Viktorin <viktorin@rehivetech.com> 写道：
> 
> Hello,
> 
> with this patch applied to git 2.12, I could see:
> 
> Use of uninitialized value $batch_size in numeric eq (==) at /usr/lib/git-core/git-send-email line 1679
> 
> when --batch-size is NOT used. See below...
> 
> On Sat, 13 May 2017 09:57:26 +0800
> xiaoqiang zhao <zxq_yx_007@163.com> wrote:
> 
>> Some email servers (e.g. smtp.163.com) limit the number emails to be
>> sent per session(connection) and this will lead to a faliure when
>> sending many messages.
>> 
>> Teach send-email to disconnect after sending a number of messages
>> (configurable via the --batch-size=<num> option), wait for a few
>> seconds (configurable via the --relogin-delay=<seconds> option) and
>> reconnect, to work around such a limit.
>> 
>> Also add this two configure option for git config command.
>> 
>> Note:
>>   Re-authentication will happen every $<batch-size> messages, so it
>> will be much more acceptable if you use some form of credential helper
>> (e.g. the 'sendemail.smtppass' config option), otherwise you will have
>> to retype password every time when asked.
>> 
>> Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
>> ---
>> contrib/completion/git-completion.bash |  2 ++
>> git-send-email.perl                    | 18 ++++++++++++++++++
>> 2 files changed, 20 insertions(+)
>> 
>> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
>> index af658995d..29496353a 100644
>> --- a/contrib/completion/git-completion.bash
>> +++ b/contrib/completion/git-completion.bash
>> @@ -2608,6 +2608,8 @@ _git_config ()
>>        sendemail.thread
>>        sendemail.to
>>        sendemail.validate
>> +        sendemail.smtpbatchsize
>> +        sendemail.smtprelogindelay
>>        showbranch.default
>>        status.relativePaths
>>        status.showUntrackedFiles
>> diff --git a/git-send-email.perl b/git-send-email.perl
>> index eea0a517f..071d1ab9d 100755
>> --- a/git-send-email.perl
>> +++ b/git-send-email.perl
>> @@ -81,6 +81,10 @@ git send-email --dump-aliases
>>                                      This setting forces to use one of the listed mechanisms.
>>     --smtp-debug            <0|1>  * Disable, enable Net::SMTP debug.
>> 
>> +    --batch-size            <int>  * send max <int> message per connection.
>> +    --relogin-delay         <int>  * delay <int> seconds between two successive login, default to 1,
>> +                                     This option can only be used with --batch-size
>> +
>>   Automating:
>>     --identity              <str>  * Use the sendemail.<id> options.
>>     --to-cmd                <str>  * Email To: via `<str> \$patch_path`
>> @@ -153,6 +157,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };
>> my $have_mail_address = eval { require Mail::Address; 1 };
>> my $smtp;
>> my $auth;
>> +my $num_sent = 0;
>> 
>> # Regexes for RFC 2047 productions.
>> my $re_token = qr/[^][()<>@,;:\\"\/?.= \000-\037\177-\377]+/;
>> @@ -216,6 +221,7 @@ my ($cover_cc, $cover_to);
>> my ($to_cmd, $cc_cmd);
>> my ($smtp_server, $smtp_server_port, @smtp_server_options);
>> my ($smtp_authuser, $smtp_encryption, $smtp_ssl_cert_path);
>> +my ($batch_size, $relogin_delay);
>> my ($identity, $aliasfiletype, @alias_files, $smtp_domain, $smtp_auth);
>> my ($validate, $confirm);
>> my (@suppress_cc);
>> @@ -247,6 +253,8 @@ my %config_settings = (
>>     "smtppass" => \$smtp_authpass,
>>     "smtpdomain" => \$smtp_domain,
>>     "smtpauth" => \$smtp_auth,
>> +    "smtpbatchsize" => \$batch_size,
>> +    "smtprelogindelay" => \$relogin_delay,
>>     "to" => \@initial_to,
>>     "tocmd" => \$to_cmd,
>>     "cc" => \@initial_cc,
>> @@ -358,6 +366,8 @@ $rc = GetOptions(
>>            "force" => \$force,
>>            "xmailer!" => \$use_xmailer,
>>            "no-xmailer" => sub {$use_xmailer = 0},
>> +            "batch-size=i" => \$batch_size,
>> +            "relogin-delay=i" => \$relogin_delay,
>>     );
>> 
>> usage() if $help;
>> @@ -1664,6 +1674,14 @@ foreach my $t (@files) {
>>        }
>>    }
>>    $message_id = undef;
>> +    $num_sent++;
>> +    if ($num_sent == $batch_size) {
> 
> This is the line. I think, batch_size can be sometimes uninitialized
> while this statement is executed.
> 
> Regards
> Jan

Thank you for reporting this，I will take a look .
> 
>> +        $num_sent = 0;
>> +        $smtp->quit if defined $smtp;
>> +        undef $smtp;
>> +        undef $auth;
>> +        sleep($relogin_delay);
>> +    }
>> }
>> 
>> # Execute a command (e.g. $to_cmd) to get a list of email addresses

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[Ævar Arnfjörð Bjarmason]

On Tue, May 16, 2017 at 3:55 PM, 赵小强 <zxq_yx_007@163.com> wrote:
>
>
>> 在 2017年5月16日，20:10，Jan Viktorin <viktorin@rehivetech.com> 写道：
>>
>> Hello,
>>
>> with this patch applied to git 2.12, I could see:
>>
>> Use of uninitialized value $batch_size in numeric eq (==) at /usr/lib/git-core/git-send-email line 1679
>>
>> when --batch-size is NOT used. See below...
>>
>> On Sat, 13 May 2017 09:57:26 +0800
>> xiaoqiang zhao <zxq_yx_007@163.com> wrote:
>>
>>> Some email servers (e.g. smtp.163.com) limit the number emails to be
>>> sent per session(connection) and this will lead to a faliure when
>>> sending many messages.
>>>
>>> Teach send-email to disconnect after sending a number of messages
>>> (configurable via the --batch-size=<num> option), wait for a few
>>> seconds (configurable via the --relogin-delay=<seconds> option) and
>>> reconnect, to work around such a limit.
>>>
>>> Also add this two configure option for git config command.
>>>
>>> Note:
>>>   Re-authentication will happen every $<batch-size> messages, so it
>>> will be much more acceptable if you use some form of credential helper
>>> (e.g. the 'sendemail.smtppass' config option), otherwise you will have
>>> to retype password every time when asked.
>>>
>>> Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com>
>>> ---
>>> contrib/completion/git-completion.bash |  2 ++
>>> git-send-email.perl                    | 18 ++++++++++++++++++
>>> 2 files changed, 20 insertions(+)
>>>
>>> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
>>> index af658995d..29496353a 100644
>>> --- a/contrib/completion/git-completion.bash
>>> +++ b/contrib/completion/git-completion.bash
>>> @@ -2608,6 +2608,8 @@ _git_config ()
>>>        sendemail.thread
>>>        sendemail.to
>>>        sendemail.validate
>>> +        sendemail.smtpbatchsize
>>> +        sendemail.smtprelogindelay
>>>        showbranch.default
>>>        status.relativePaths
>>>        status.showUntrackedFiles
>>> diff --git a/git-send-email.perl b/git-send-email.perl
>>> index eea0a517f..071d1ab9d 100755
>>> --- a/git-send-email.perl
>>> +++ b/git-send-email.perl
>>> @@ -81,6 +81,10 @@ git send-email --dump-aliases
>>>                                      This setting forces to use one of the listed mechanisms.
>>>     --smtp-debug            <0|1>  * Disable, enable Net::SMTP debug.
>>>
>>> +    --batch-size            <int>  * send max <int> message per connection.
>>> +    --relogin-delay         <int>  * delay <int> seconds between two successive login, default to 1,
>>> +                                     This option can only be used with --batch-size
>>> +
>>>   Automating:
>>>     --identity              <str>  * Use the sendemail.<id> options.
>>>     --to-cmd                <str>  * Email To: via `<str> \$patch_path`
>>> @@ -153,6 +157,7 @@ my $have_email_valid = eval { require Email::Valid; 1 };
>>> my $have_mail_address = eval { require Mail::Address; 1 };
>>> my $smtp;
>>> my $auth;
>>> +my $num_sent = 0;
>>>
>>> # Regexes for RFC 2047 productions.
>>> my $re_token = qr/[^][()<>@,;:\\"\/?.= \000-\037\177-\377]+/;
>>> @@ -216,6 +221,7 @@ my ($cover_cc, $cover_to);
>>> my ($to_cmd, $cc_cmd);
>>> my ($smtp_server, $smtp_server_port, @smtp_server_options);
>>> my ($smtp_authuser, $smtp_encryption, $smtp_ssl_cert_path);
>>> +my ($batch_size, $relogin_delay);
>>> my ($identity, $aliasfiletype, @alias_files, $smtp_domain, $smtp_auth);
>>> my ($validate, $confirm);
>>> my (@suppress_cc);
>>> @@ -247,6 +253,8 @@ my %config_settings = (
>>>     "smtppass" => \$smtp_authpass,
>>>     "smtpdomain" => \$smtp_domain,
>>>     "smtpauth" => \$smtp_auth,
>>> +    "smtpbatchsize" => \$batch_size,
>>> +    "smtprelogindelay" => \$relogin_delay,
>>>     "to" => \@initial_to,
>>>     "tocmd" => \$to_cmd,
>>>     "cc" => \@initial_cc,
>>> @@ -358,6 +366,8 @@ $rc = GetOptions(
>>>            "force" => \$force,
>>>            "xmailer!" => \$use_xmailer,
>>>            "no-xmailer" => sub {$use_xmailer = 0},
>>> +            "batch-size=i" => \$batch_size,
>>> +            "relogin-delay=i" => \$relogin_delay,
>>>     );
>>>
>>> usage() if $help;
>>> @@ -1664,6 +1674,14 @@ foreach my $t (@files) {
>>>        }
>>>    }
>>>    $message_id = undef;
>>> +    $num_sent++;
>>> +    if ($num_sent == $batch_size) {
>>
>> This is the line. I think, batch_size can be sometimes uninitialized
>> while this statement is executed.
>>
>> Regards
>> Jan
>
> Thank you for reporting this，I will take a look .

You just need to initialize the variables you're using, see e.g. these
existing ones:

    my ($quiet, $dry_run) = (0, 0);

Just do the same for the ones you're adding.

>>> +        $num_sent = 0;
>>> +        $smtp->quit if defined $smtp;
>>> +        undef $smtp;
>>> +        undef $auth;
>>> +        sleep($relogin_delay);
>>> +    }
>>> }
>>>
>>> # Execute a command (e.g. $to_cmd) to get a list of email addresses
>
>

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> xiaoqiang zhao <zxq_yx_007@163.com> writes:
> ...
>> Note:
>>    Re-authentication will happen every $<batch-size> messages, so it
>> will be much more acceptable if you use some form of credential helper
>> (e.g. the 'sendemail.smtppass' config option), otherwise you will have
>> to retype password every time when asked.
>
> I think this deserves to be in the end-user documentation (i.e. the
> part of your patch that updates Documentation/git-send-email.txt).

Ah, this came out to be rather obliqu, but you do need an update to
the documentation as part of this patch.

Thanks.

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[赵小强]

> 在 2017年5月17日，07:49，Junio C Hamano <gitster@pobox.com> 写道：
> 
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> xiaoqiang zhao <zxq_yx_007@163.com> writes:
>> ...
>>> Note:
>>>   Re-authentication will happen every $<batch-size> messages, so it
>>> will be much more acceptable if you use some form of credential helper
>>> (e.g. the 'sendemail.smtppass' config option), otherwise you will have
>>> to retype password every time when asked.
>> 
>> I think this deserves to be in the end-user documentation (i.e. the
>> part of your patch that updates Documentation/git-send-email.txt).
> 
> Ah, this came out to be rather obliqu, but you do need an update to
> the documentation as part of this patch.
> 
> Thanks.

Sure！

Re: [PATCH v4] send-email: --batch-size to work around some SMTP server limit

[赵小强]

在 2017年5月17日，01:43，Ævar Arnfjörð Bjarmason <avarab@gmail.com> 写道：

>>> Regards
>>> Jan
>> 
>> Thank you for reporting this，I will take a look .
> 
> You just need to initialize the variables you're using, see e.g. these
> existing ones:
> 
>    my ($quiet, $dry_run) = (0, 0);
> 
> Just do the same for the ones you're adding.

Yes，has to be.