[PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
init_merge_options().
Consequential call of getenv() may return NULL pointer and strtol() crashes.
However the stored pointer to the obtained getenv() result may be invalidated
by some other getenv() call from another thread as getenv() is not thread-safe.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
---
 merge-recursive.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..eaac98145 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,8 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Stefan Beller]

On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options().
> Consequential call of getenv() may return NULL pointer and strtol() crashes.
> However the stored pointer to the obtained getenv() result may be invalidated
> by some other getenv() call from another thread as getenv() is not thread-safe.

But do we have other threads running at the time?
Inspecting the four callsites:
* sequencer.c:
  The prior lines to hold the index lock suggests we're not in a threaded
  environment
* builtin/merge-recursive.c:
  In cmd_merge_recursive and we're fiddling with argv/argc, which
  suggests we in a main function, not having threads around
* builtin/am.c: fall_back_threeway called by am_run.
  (am is not threaded)
* builtin/merge:
  In try_merge_strategy called from the main function, also index locking
* builtin/checkout.c:
  merge_working_tree also locks the index.

So I think this function is never called from within a threaded environment
in git.

>
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
> ---
>  merge-recursive.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1494ffdb8..eaac98145 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>
>  void init_merge_options(struct merge_options *o)
>  {
> +       const char *merge_verbosity = getenv("GIT_MERGE_VERBOSITY");

Despite not being in a threaded environment, I wonder if we want to
minimize the time between  calling getenv and the use of the result,
i.e. declare merge_verbosity here, but assign it later, just before the
condition?

(The compiler may shuffle stuff around anyway, so this is a
moot suggestion; It gears mostly towards making the code more
readable/maintainable when presenting this part of the code
to the user.)

With or without this change:
Reviewed-by: Stefan Beller <sbeller@google.com>


>         memset(o, 0, sizeof(struct merge_options));
>         o->verbosity = 2;
>         o->buffer_output = 1;
> @@ -2171,9 +2172,8 @@ void init_merge_options(struct merge_options *o)
>         o->renormalize = 0;
>         o->detect_rename = 1;
>         merge_recursive_config(o);
> -       if (getenv("GIT_MERGE_VERBOSITY"))
> -               o->verbosity =
> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
> +       if (merge_verbosity)
> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>         if (o->verbosity >= 5)
>                 o->buffer_output = 0;
>         strbuf_init(&o->obuf, 0);
> --
> 2.14.3

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Eric Sunshine]

On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
>> init_merge_options().
>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
>> However the stored pointer to the obtained getenv() result may be invalidated
>> by some other getenv() call from another thread as getenv() is not thread-safe.
>
> But do we have other threads running at the time?

I feel uncomfortable about this change, not due to lack of thread
safety, but due to the distance between the getenv() invocation and
its point of use. See below for more detail.

>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>> ---
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>  void init_merge_options(struct merge_options *o)
>>  {
>> +       const char *merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>
> Despite not being in a threaded environment, I wonder if we want to
> minimize the time between  calling getenv and the use of the result,
> i.e. declare merge_verbosity here, but assign it later, just before the
> condition?
>
> With or without this change:
> Reviewed-by: Stefan Beller <sbeller@google.com>

The distance between getenv() and the point where the value is
actually used is a big concern due to not knowing what is or might be
going on in called functions between the two points. According to [1],
the value returned by getenv() could be invalidated by another call to
getenv() (or setenv() or unsetenv() or putenv()), and we don't have
guarantee that we're safe from such invalidation considering that this
function calls out to others. For instance, after getenv() but before
the value is used, init_merge_options() calls merge_recursive_config()
which calls git_config() which calls git_xmerge_config(), and so on.

For this reason, I have difficulty endorsing this change as-is.

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/getenv.html

>>         memset(o, 0, sizeof(struct merge_options));
>>         o->verbosity = 2;
>>         o->buffer_output = 1;
>> @@ -2171,9 +2172,8 @@ void init_merge_options(struct merge_options *o)
>>         o->renormalize = 0;
>>         o->detect_rename = 1;
>>         merge_recursive_config(o);
>> -       if (getenv("GIT_MERGE_VERBOSITY"))
>> -               o->verbosity =
>> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>> +       if (merge_verbosity)
>> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>>         if (o->verbosity >= 5)
>>                 o->buffer_output = 0;
>>         strbuf_init(&o->obuf, 0);
>> --
>> 2.14.3

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Martin Ågren]

On 24 October 2017 at 18:45, Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
>> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
>>> init_merge_options().
>>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
>>> However the stored pointer to the obtained getenv() result may be invalidated
>>> by some other getenv() call from another thread as getenv() is not thread-safe.

I'm having trouble wrapping my head around this. Under which
circumstances could the second call in the current code return NULL, but
the code after your patch behave in a well-defined (and correct) way?

> The distance between getenv() and the point where the value is
> actually used is a big concern due to not knowing what is or might be
> going on in called functions between the two points. According to [1],
> the value returned by getenv() could be invalidated by another call to
> getenv() (or setenv() or unsetenv() or putenv()), and we don't have
> guarantee that we're safe from such invalidation considering that this
> function calls out to others. For instance, after getenv() but before
> the value is used, init_merge_options() calls merge_recursive_config()
> which calls git_config() which calls git_xmerge_config(), and so on.
>
> For this reason, I have difficulty endorsing this change as-is.

Yeah. The call should be immediately before `merge_verbosity` is used.
Then, if a compiler wants to move the call, it has to do the work and
prove that it's ok.

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Jeff King]

On Tue, Oct 24, 2017 at 07:11:24PM +0200, Martin Ågren wrote:

> On 24 October 2017 at 18:45, Eric Sunshine <sunshine@sunshineco.com> wrote:
> > On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
> >> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
> >>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
> >>> init_merge_options().
> >>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
> >>> However the stored pointer to the obtained getenv() result may be invalidated
> >>> by some other getenv() call from another thread as getenv() is not thread-safe.
> 
> I'm having trouble wrapping my head around this. Under which
> circumstances could the second call in the current code return NULL, but
> the code after your patch behave in a well-defined (and correct) way?

Yeah, it's not at all clear to me this is solving a real problem. I know
Andrey mentioned playing around with fault injection in an earlier
thread, so I'm wondering if there is an artificial fault being injected
into the second getenv() call. Which does not seem like something that
should be possible in the real world.

I definitely agree with the sentiment that as few things as possible
should happen between calling getenv() and using its result. I've seen
real bugs there from unexpected invalidation of the static buffer.

-Peff

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I definitely agree with the sentiment that as few things as possible
> should happen between calling getenv() and using its result. I've seen
> real bugs there from unexpected invalidation of the static buffer.

Sure.  I do not think we mind xstrdup()ing the result and freeing
after we are done, though ;-)

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Eric Sunshine]

On Tue, Oct 24, 2017 at 9:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Jeff King <peff@peff.net> writes:
>> I definitely agree with the sentiment that as few things as possible
>> should happen between calling getenv() and using its result. I've seen
>> real bugs there from unexpected invalidation of the static buffer.
>
> Sure.  I do not think we mind xstrdup()ing the result and freeing
> after we are done, though ;-)

More specifically, xstrdup_or_null(getenv(...)), if that route is chosen.

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Jeff King]

On Wed, Oct 25, 2017 at 12:07:12AM -0400, Eric Sunshine wrote:

> On Tue, Oct 24, 2017 at 9:48 PM, Junio C Hamano <gitster@pobox.com> wrote:
> > Jeff King <peff@peff.net> writes:
> >> I definitely agree with the sentiment that as few things as possible
> >> should happen between calling getenv() and using its result. I've seen
> >> real bugs there from unexpected invalidation of the static buffer.
> >
> > Sure.  I do not think we mind xstrdup()ing the result and freeing
> > after we are done, though ;-)
> 
> More specifically, xstrdup_or_null(getenv(...)), if that route is chosen.

That would be the way to do it, but I do not see thta we need to record
the string at all. The current code is calling strtol on it on it
immediately.

So the options are:

  1. Save the result of getenv() in a variable. If it is non-NULL, then
     immediately call strtol() on it.

  2. Do nothing. The double-call to getenv() is actually fine in the
     real world as it will return consistent results.

But the patch under discussion, which calls getenv() then expects it
to be correct after a call to merge_recursive_config(), introduces a
problem.

-Peff

[PATCH v2] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Check 'GIT_MERGE_VERBOSITY' environment variable only once in
init_merge_options().
Consequential call of getenv() may return NULL pointer.
However the stored pointer to the obtained getenv() result may be invalidated
by some other getenv() call as getenv() is not thread-safe.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
---
Changes since the previous patch:
* no actions are taken between the merge_verbosity assignment and check.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3

Re: [PATCH v2] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Eric Sunshine]

On Wed, Oct 25, 2017 at 7:39 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
> Check 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options().
> Consequential call of getenv() may return NULL pointer.

It would be particularly nice to have a more detailed explanation in
the commit message of the potential problem this patch is trying to
solve. Given the amount of discussion, thus far, surrounding such a
simple patch, this cryptic warning about getenv() returning NULL upon
second invocation is insufficient to explain why this patch is
desirable; it merely leads to a lot of head-scratching.

> However the stored pointer to the obtained getenv() result may be invalidated
> by some other getenv() call as getenv() is not thread-safe.

This is even more cryptic, as it appears to be arguing for or against
_something_ (it's not clear what) and it seems to be talking about a
facet of the existing code, rather than explaining why the updated
code consumes its 'merge_verbosity' value as early as possible after
being assigned. Perhaps this part could be reworded something like
this:

    Instead, call getenv() only once, storing its value and
    consulting it as many times as needed. This update takes care
    to consume the value returned by getenv() without any
    intervening calls to getenv(), setenv(), unsetenv(), or
    putenv(), any of which might invalidate the pointer returned
    by the initial call.

> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
> Reviewed-by: Stefan Beller <sbeller@google.com>

As this patch is semantically quite different from the original to
which Stefan gave his Reviewed-by: (and which other people argued
against), it might be better omit this footer and let him re-give it
if he so desires.

> ---
> Changes since the previous patch:
> * no actions are taken between the merge_verbosity assignment and check.
>  merge-recursive.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1494ffdb8..60084e3a0 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>
>  void init_merge_options(struct merge_options *o)
>  {
> +       const char *merge_verbosity;
>         memset(o, 0, sizeof(struct merge_options));
>         o->verbosity = 2;
>         o->buffer_output = 1;
> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>         o->renormalize = 0;
>         o->detect_rename = 1;
>         merge_recursive_config(o);
> -       if (getenv("GIT_MERGE_VERBOSITY"))
> -               o->verbosity =
> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
> +       merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
> +       if (merge_verbosity)
> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>         if (o->verbosity >= 5)
>                 o->buffer_output = 0;
>         strbuf_init(&o->obuf, 0);
> --
> 2.14.3

Re: [PATCH v2] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Thanks, Eric, indeed it's better to change the commit message.

25.10.2017 14:53, Eric Sunshine wrote:
> On Wed, Oct 25, 2017 at 7:39 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>> Check 'GIT_MERGE_VERBOSITY' environment variable only once in
>> init_merge_options().
>> Consequential call of getenv() may return NULL pointer.
> 
> It would be particularly nice to have a more detailed explanation in
> the commit message of the potential problem this patch is trying to
> solve. Given the amount of discussion, thus far, surrounding such a
> simple patch, this cryptic warning about getenv() returning NULL upon
> second invocation is insufficient to explain why this patch is
> desirable; it merely leads to a lot of head-scratching.
> 
>> However the stored pointer to the obtained getenv() result may be invalidated
>> by some other getenv() call as getenv() is not thread-safe.
> 
> This is even more cryptic, as it appears to be arguing for or against
> _something_ (it's not clear what) and it seems to be talking about a
> facet of the existing code, rather than explaining why the updated
> code consumes its 'merge_verbosity' value as early as possible after
> being assigned. Perhaps this part could be reworded something like
> this:
> 
>     Instead, call getenv() only once, storing its value and
>     consulting it as many times as needed. This update takes care
>     to consume the value returned by getenv() without any
>     intervening calls to getenv(), setenv(), unsetenv(), or
>     putenv(), any of which might invalidate the pointer returned
>     by the initial call.
> 
>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>> Reviewed-by: Stefan Beller <sbeller@google.com>
> 
> As this patch is semantically quite different from the original to
> which Stefan gave his Reviewed-by: (and which other people argued
> against), it might be better omit this footer and let him re-give it
> if he so desires.
> 
>> ---
>> Changes since the previous patch:
>> * no actions are taken between the merge_verbosity assignment and check.
>>  merge-recursive.c | 7 ++++---
>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> index 1494ffdb8..60084e3a0 100644
>> --- a/merge-recursive.c
>> +++ b/merge-recursive.c
>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>
>>  void init_merge_options(struct merge_options *o)
>>  {
>> +       const char *merge_verbosity;
>>         memset(o, 0, sizeof(struct merge_options));
>>         o->verbosity = 2;
>>         o->buffer_output = 1;
>> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>>         o->renormalize = 0;
>>         o->detect_rename = 1;
>>         merge_recursive_config(o);
>> -       if (getenv("GIT_MERGE_VERBOSITY"))
>> -               o->verbosity =
>> -                       strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>> +       merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>> +       if (merge_verbosity)
>> +               o->verbosity = strtol(merge_verbosity, NULL, 10);
>>         if (o->verbosity >= 5)
>>                 o->buffer_output = 0;
>>         strbuf_init(&o->obuf, 0);
>> --
>> 2.14.3
> 
> 
> 

-- 
Best regards,
Andrey Okoshkin

Re: [PATCH v3] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Get 'GIT_MERGE_VERBOSITY' environment variable only once in
init_merge_options() and store the pointer to its value for the further check.
No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done
between the initial getenv() call and the consequential result pass to strtol()
as these environment related functions could modify the string pointer returned
by the initial getenv() call.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
---
I tried to rework the commit message.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3

Re: [PATCH v3] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Stefan Beller]

On Wed, Oct 25, 2017 at 6:03 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
> Get 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options() and store the pointer to its value for the further check.
> No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done
> between the initial getenv() call and the consequential result pass to strtol()
> as these environment related functions could modify the string pointer returned
> by the initial getenv() call.
>
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>

This is
Reviewed-by: Stefan Beller <sbeller@google.com>

[PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Get 'GIT_MERGE_VERBOSITY' environment variable only once in
init_merge_options() and store the pointer to its value for the further check.
No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done
between the initial getenv() call and the consequential result pass to strtol()
as these environment related functions could modify the string pointer returned
by the initial getenv() call.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
---
Added 'reviewed-by' field.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Junio C Hamano]

Andrey Okoshkin <a.okoshkin@samsung.com> writes:

> Get 'GIT_MERGE_VERBOSITY' environment variable only once in
> init_merge_options() and store the pointer to its value for the further check.

OK, that is "what this thing does" description.

> No intervening calls to getenv(), putenv(), setenv() or unsetenv() are done
> between the initial getenv() call and the consequential result pass to strtol()
> as these environment related functions could modify the string pointer returned
> by the initial getenv() call.

That holds true for the code before or after this patch equally.  In
other words, that sounds like a justification for rejecting this
patch (i.e. explanation of why this change is not needed).

If we are worried about another thread calling these functions after
the time we call getenv() and before the time we pass the result to
strtol(), then I do not think this patch gives a better protection
against such race, so I do not think that is why you are doing this.

So... why do we want to do this change?  I am puzzled.


>
> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
> Reviewed-by: Stefan Beller <sbeller@google.com>
> ---
> Added 'reviewed-by' field.
>  merge-recursive.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/merge-recursive.c b/merge-recursive.c
> index 1494ffdb8..60084e3a0 100644
> --- a/merge-recursive.c
> +++ b/merge-recursive.c
> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>  
>  void init_merge_options(struct merge_options *o)
>  {
> +	const char *merge_verbosity;
>  	memset(o, 0, sizeof(struct merge_options));
>  	o->verbosity = 2;
>  	o->buffer_output = 1;
> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>  	o->renormalize = 0;
>  	o->detect_rename = 1;
>  	merge_recursive_config(o);
> -	if (getenv("GIT_MERGE_VERBOSITY"))
> -		o->verbosity =
> -			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
> +	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
> +	if (merge_verbosity)
> +		o->verbosity = strtol(merge_verbosity, NULL, 10);
>  	if (o->verbosity >= 5)
>  		o->buffer_output = 0;
>  	strbuf_init(&o->obuf, 0);

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> That holds true for the code before or after this patch equally.  In
> other words, that sounds like a justification for rejecting this
> patch (i.e. explanation of why this change is not needed).
>
> If we are worried about another thread calling these functions after
> the time we call getenv() and before the time we pass the result to
> strtol(), then I do not think this patch gives a better protection
> against such race, so I do not think that is why you are doing this.
>
> So... why do we want to do this change?  I am puzzled.

For the sake of fairness, I would say that the resulting code may be
easier to follow and has one less instance of a constant string that
the compiler cannot statically check if we made a typo.  That's the
only benefit in this patch as far as I can see.

The original makes a call to see if the result is NULL, and then
makes the same call, expecting that we get the same result (not just
that it is not NULL, but it is the same verbosity request the end
user made via the environment as the one we checked earlier), and I
can understand that it feels a bit redundant and ugly.


>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>> Reviewed-by: Stefan Beller <sbeller@google.com>
>> ---
>> Added 'reviewed-by' field.
>>  merge-recursive.c | 7 ++++---
>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/merge-recursive.c b/merge-recursive.c
>> index 1494ffdb8..60084e3a0 100644
>> --- a/merge-recursive.c
>> +++ b/merge-recursive.c
>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>  
>>  void init_merge_options(struct merge_options *o)
>>  {
>> +	const char *merge_verbosity;
>>  	memset(o, 0, sizeof(struct merge_options));
>>  	o->verbosity = 2;
>>  	o->buffer_output = 1;
>> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>>  	o->renormalize = 0;
>>  	o->detect_rename = 1;
>>  	merge_recursive_config(o);
>> -	if (getenv("GIT_MERGE_VERBOSITY"))
>> -		o->verbosity =
>> -			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>> +	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>> +	if (merge_verbosity)
>> +		o->verbosity = strtol(merge_verbosity, NULL, 10);
>>  	if (o->verbosity >= 5)
>>  		o->buffer_output = 0;
>>  	strbuf_init(&o->obuf, 0);

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

31.10.2017 05:26, Junio C Hamano wrote:
> Junio C Hamano <gitster@pobox.com> writes:
> 
>> That holds true for the code before or after this patch equally.  In
>> other words, that sounds like a justification for rejecting this
>> patch (i.e. explanation of why this change is not needed).
>>
>> If we are worried about another thread calling these functions after
>> the time we call getenv() and before the time we pass the result to
>> strtol(), then I do not think this patch gives a better protection
>> against such race, so I do not think that is why you are doing this.
>>
>> So... why do we want to do this change?  I am puzzled.

I think, the main benefits are:
* Code is more readable, no duplicated calls with the same constant string
argument.
* Code is potentially safer, the second getenv() call may return another
pointer value which could be NULL (and yes, this is an arguable point as it
can be done only artificially).

> For the sake of fairness, I would say that the resulting code may be
> easier to follow and has one less instance of a constant string that
> the compiler cannot statically check if we made a typo.  That's the
> only benefit in this patch as far as I can see.
> 
> The original makes a call to see if the result is NULL, and then
> makes the same call, expecting that we get the same result (not just
> that it is not NULL, but it is the same verbosity request the end
> user made via the environment as the one we checked earlier), and I
> can understand that it feels a bit redundant and ugly.

Yes, you absolutely right.
I believe this patch makes code more beautiful :-)

>>> Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
>>> Reviewed-by: Stefan Beller <sbeller@google.com>
>>> ---
>>> Added 'reviewed-by' field.
>>>  merge-recursive.c | 7 ++++---
>>>  1 file changed, 4 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/merge-recursive.c b/merge-recursive.c
>>> index 1494ffdb8..60084e3a0 100644
>>> --- a/merge-recursive.c
>>> +++ b/merge-recursive.c
>>> @@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
>>>  
>>>  void init_merge_options(struct merge_options *o)
>>>  {
>>> +	const char *merge_verbosity;
>>>  	memset(o, 0, sizeof(struct merge_options));
>>>  	o->verbosity = 2;
>>>  	o->buffer_output = 1;
>>> @@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
>>>  	o->renormalize = 0;
>>>  	o->detect_rename = 1;
>>>  	merge_recursive_config(o);
>>> -	if (getenv("GIT_MERGE_VERBOSITY"))
>>> -		o->verbosity =
>>> -			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
>>> +	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
>>> +	if (merge_verbosity)
>>> +		o->verbosity = strtol(merge_verbosity, NULL, 10);
>>>  	if (o->verbosity >= 5)
>>>  		o->buffer_output = 0;
>>>  	strbuf_init(&o->obuf, 0);
> 
> 
> 

-- 
Best regards,
Andrey Okoshkin

Re: [PATCH v4] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Junio C Hamano]

Andrey Okoshkin <a.okoshkin@samsung.com> writes:

> I think, the main benefits are:
> * Code is more readable, no duplicated calls with the same constant string
> argument.
> * Code is potentially safer, the second getenv() call may return another
> pointer value which could be NULL (and yes, this is an arguable point as it
> can be done only artificially).
>
>> For the sake of fairness, I would say that the resulting code may be
>> easier to follow and has one less instance of a constant string that
>> the compiler cannot statically check if we made a typo.  That's the
>> only benefit in this patch as far as I can see.
>> 
>> The original makes a call to see if the result is NULL, and then
>> makes the same call, expecting that we get the same result (not just
>> that it is not NULL, but it is the same verbosity request the end
>> user made via the environment as the one we checked earlier), and I
>> can understand that it feels a bit redundant and ugly.
>
> Yes, you absolutely right.

I am absolutely right when I say your "code is potentially safer" is
total BS.  The result from first getenv() call may be pointing at an
invalid piece of memory by the time it is used, if you are in a
situation in which not having the second getenv() matters
(i.e. somebody else is also mucking with getenv() at the same time).

So please update the log message so that the patch is not sold on
that basis.

Thanks.

[PATCH v5] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Get rid of the duplicated getenv('GIT_MERGE_VERBOSITY') calls with the same
constant string argument. This makes code more readable and prevents typo in
the further development.

Signed-off-by: Andrey Okoshkin <a.okoshkin@samsung.com>
Reviewed-by: Stefan Beller <sbeller@google.com>
---
Commit message is reworked according to the feedback.
 merge-recursive.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/merge-recursive.c b/merge-recursive.c
index 1494ffdb8..60084e3a0 100644
--- a/merge-recursive.c
+++ b/merge-recursive.c
@@ -2163,6 +2163,7 @@ static void merge_recursive_config(struct merge_options *o)
 
 void init_merge_options(struct merge_options *o)
 {
+	const char *merge_verbosity;
 	memset(o, 0, sizeof(struct merge_options));
 	o->verbosity = 2;
 	o->buffer_output = 1;
@@ -2171,9 +2172,9 @@ void init_merge_options(struct merge_options *o)
 	o->renormalize = 0;
 	o->detect_rename = 1;
 	merge_recursive_config(o);
-	if (getenv("GIT_MERGE_VERBOSITY"))
-		o->verbosity =
-			strtol(getenv("GIT_MERGE_VERBOSITY"), NULL, 10);
+	merge_verbosity = getenv("GIT_MERGE_VERBOSITY");
+	if (merge_verbosity)
+		o->verbosity = strtol(merge_verbosity, NULL, 10);
 	if (o->verbosity >= 5)
 		o->buffer_output = 0;
 	strbuf_init(&o->obuf, 0);
-- 
2.14.3

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

24.10.2017 22:52, Jeff King wrote:
> On Tue, Oct 24, 2017 at 07:11:24PM +0200, Martin Ågren wrote:
> 
>> On 24 October 2017 at 18:45, Eric Sunshine <sunshine@sunshineco.com> wrote:
>>> On Tue, Oct 24, 2017 at 12:28 PM, Stefan Beller <sbeller@google.com> wrote:
>>>> On Tue, Oct 24, 2017 at 8:27 AM, Andrey Okoshkin <a.okoshkin@samsung.com> wrote:
>>>>> Add check of 'GIT_MERGE_VERBOSITY' environment variable only once in
>>>>> init_merge_options().
>>>>> Consequential call of getenv() may return NULL pointer and strtol() crashes.
>>>>> However the stored pointer to the obtained getenv() result may be invalidated
>>>>> by some other getenv() call from another thread as getenv() is not thread-safe.
>>
>> I'm having trouble wrapping my head around this. Under which
>> circumstances could the second call in the current code return NULL, but
>> the code after your patch behave in a well-defined (and correct) way?
> 
> Yeah, it's not at all clear to me this is solving a real problem. I know
> Andrey mentioned playing around with fault injection in an earlier
> thread, so I'm wondering if there is an artificial fault being injected
> into the second getenv() call. Which does not seem like something that
> should be possible in the real world.
> 
> I definitely agree with the sentiment that as few things as possible
> should happen between calling getenv() and using its result. I've seen
> real bugs there from unexpected invalidation of the static buffer.
> 
> -Peff

Thanks for your comments.

Jeff is right: there were some artificial fault injections imitating valid failures
of different functions (syscalls, libc and so on).
And yes - in the real life there is no problems with current code as there are no other
threads running.
However it's not a good practice to double call getenv() with the same argument:
* Code readability.
* Still no guaranty that the second call will be valid: some linked library may be
compromised or LD_PRELOADed with the aim to create a race with getenv(). I believe
there is no profit doing it here but it's just an explanation.

In my opinion, here it's ok to save the pointer returned from the single getnev() call
doing as few actions as possible between getenv() and strtol() calls.
I'll change the patch.

-- 
Best regards,
Andrey Okoshkin

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

24.10.2017 19:45, Eric Sunshine wrote:
> I feel uncomfortable about this change, not due to lack of thread
> safety, but due to the distance between the getenv() invocation and
> its point of use. See below for more detail.

Thanks, the usage must be just after the assignment.

-- 
Best regards,
Andrey Okoshkin

Re: [PATCH] merge-recursive: check GIT_MERGE_VERBOSITY only once

[Andrey Okoshkin]

Thanks for your review.

24.10.2017 19:28, Stefan Beller wrote:
> So I think this function is never called from within a threaded environment
> in git.
You are right, it's just a hypothetic case.
 
> Despite not being in a threaded environment, I wonder if we want to
> minimize the time between  calling getenv and the use of the result,
> i.e. declare merge_verbosity here, but assign it later, just before the
> condition?
> 
> (The compiler may shuffle stuff around anyway, so this is a
> moot suggestion; It gears mostly towards making the code more
> readable/maintainable when presenting this part of the code
> to the user.)
> 
> With or without this change:
> Reviewed-by: Stefan Beller <sbeller@google.com>
Yes, in current situation it's more for readability. And I'll make the usage
of merge_verbosity just after the assignment.

-- 
Best regards,
Andrey Okoshkin