[RFC PATCH v2] Allow aliases that include other aliases

[RFC PATCH v2] Allow aliases that include other aliases

[Tim Schumacher]

Aliases can only contain non-alias git commands and their
arguments, not other user-defined aliases. Resolving further
(nested) aliases is prevented by breaking the loop after the
first alias was processed. Git then fails with a command-not-found
error.

Allow resolving nested aliases by not breaking the loop in
run_argv() after the first alias was processed. Instead, continue
incrementing `done_alias` until `handle_alias()` fails, which means that
there are no further aliases that can be processed. Prevent looping
aliases by storing substituted commands in `cmd_list` and checking if
a command has been substituted previously.
---

This is what I've come up with to prevent looping aliases. I'm not too
happy with the number of indentations needed, but this seemed to be the
easiest way to search an array for a value.

---
 git.c | 25 ++++++++++++++++++++++---
 1 file changed, 22 insertions(+), 3 deletions(-)

diff --git a/git.c b/git.c
index c27c38738..fd90a3341 100644
--- a/git.c
+++ b/git.c
@@ -674,6 +674,8 @@ static void execv_dashed_external(const char **argv)
 static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
+	const char **cmd_list = NULL;
+	int cmd_list_alloc = 0;
 
 	while (1) {
 		/*
@@ -691,17 +693,34 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);
 
+		/* Increase the array size and add the current
+		 * command to it.
+		 */
+		cmd_list_alloc += strlen(*argv[0]) + 1;
+		REALLOC_ARRAY(cmd_list, cmd_list_alloc);
+		cmd_list[done_alias] = *argv[0];
+
+		/* Search the array for occurrences of that command,
+		 * abort if something has been found.
+		 */
+		for (int i = 0; i < done_alias; i++) {
+			if (!strcmp(cmd_list[i], *argv[0])) {
+				die("loop alias: %s is called twice",
+				    cmd_list[done_alias]);
+			}
+		}
+
 		/* It could be an alias -- this works around the insanity
 		 * of overriding "git log" with "git show" by having
 		 * alias.log = show
 		 */
-		if (done_alias)
-			break;
 		if (!handle_alias(argcp, argv))
 			break;
-		done_alias = 1;
+		done_alias++;
 	}
 
+	free(cmd_list);
+
 	return done_alias;
 }
 
-- 
2.19.0.rc1.2.g8f4faccc1

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Duy Nguyen]

On Wed, Sep 5, 2018 at 10:56 AM Tim Schumacher <timschumi@gmx.de> wrote:
>
> Aliases can only contain non-alias git commands and their
> arguments, not other user-defined aliases. Resolving further
> (nested) aliases is prevented by breaking the loop after the
> first alias was processed. Git then fails with a command-not-found
> error.
>
> Allow resolving nested aliases by not breaking the loop in
> run_argv() after the first alias was processed. Instead, continue
> incrementing `done_alias` until `handle_alias()` fails, which means that
> there are no further aliases that can be processed. Prevent looping
> aliases by storing substituted commands in `cmd_list` and checking if
> a command has been substituted previously.
> ---
>
> This is what I've come up with to prevent looping aliases. I'm not too
> happy with the number of indentations needed, but this seemed to be the
> easiest way to search an array for a value.

You can just make all the new code a separate function, which reduces
indentation.

There's another thing I wanted (but probably a wrong thing to want):
if I define alias 'foo' in ~/.gitconfig, then I'd like to modify it in
some project by redefining it as alias.foo='foo --something' in
$GIT_DIR/config. This results in alias loop, but the loop is broken by
looking up 'foo' from a higher level config file instead.

This is not easy to do, and as I mentioned, I'm not even sure if it's
a sane thing to do.

> +               /* Increase the array size and add the current
> +                * command to it.
> +                */

I think this is pretty clear from the code, you don't need to add a
comment to explain how the next few lines work. Same comment for the
next comment block.

> +               cmd_list_alloc += strlen(*argv[0]) + 1;
> +               REALLOC_ARRAY(cmd_list, cmd_list_alloc);
> +               cmd_list[done_alias] = *argv[0];
> +
> +               /* Search the array for occurrences of that command,
> +                * abort if something has been found.
> +                */
> +               for (int i = 0; i < done_alias; i++) {
> +                       if (!strcmp(cmd_list[i], *argv[0])) {
> +                               die("loop alias: %s is called twice",

Please wrap the string in _() so that it can be translated in
different languages.

> +                                   cmd_list[done_alias]);
> +                       }
> +               }
> +
-- 
Duy

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Tim Schumacher]

On 05.09.18 17:48, Duy Nguyen wrote:
> On Wed, Sep 5, 2018 at 10:56 AM Tim Schumacher <timschumi@gmx.de> wrote:
>>
>> Aliases can only contain non-alias git commands and their
>> arguments, not other user-defined aliases. Resolving further
>> (nested) aliases is prevented by breaking the loop after the
>> first alias was processed. Git then fails with a command-not-found
>> error.
>>
>> Allow resolving nested aliases by not breaking the loop in
>> run_argv() after the first alias was processed. Instead, continue
>> incrementing `done_alias` until `handle_alias()` fails, which means that
>> there are no further aliases that can be processed. Prevent looping
>> aliases by storing substituted commands in `cmd_list` and checking if
>> a command has been substituted previously.
>> ---
>>
>> This is what I've come up with to prevent looping aliases. I'm not too
>> happy with the number of indentations needed, but this seemed to be the
>> easiest way to search an array for a value.
> 
> You can just make all the new code a separate function, which reduces
> indentation.

That would solve the issue, but I'm not sure if it is worth introducing
a new function exclusively for that. I didn't find anything about a
maximum indentation level in the code guidelines and since the new
parts stay within the width limit (and is imo still readable), would it
be ok to keep it like that?

> 
> There's another thing I wanted (but probably a wrong thing to want):
> if I define alias 'foo' in ~/.gitconfig, then I'd like to modify it in
> some project by redefining it as alias.foo='foo --something' in
> $GIT_DIR/config. This results in alias loop, but the loop is broken by
> looking up 'foo' from a higher level config file instead.
> 
> This is not easy to do, and as I mentioned, I'm not even sure if it's
> a sane thing to do.

The alias system is using the default functions of the config system,
I assume that adding such a functionality is not possible, at least not
without breaking compatibility.

> 
>> +               /* Increase the array size and add the current
>> +                * command to it.
>> +                */
> 
> I think this is pretty clear from the code, you don't need to add a
> comment to explain how the next few lines work. Same comment for the
> next comment block.

I'll remove them in v3.

> 
>> +               cmd_list_alloc += strlen(*argv[0]) + 1;
>> +               REALLOC_ARRAY(cmd_list, cmd_list_alloc);
>> +               cmd_list[done_alias] = *argv[0];
>> +
>> +               /* Search the array for occurrences of that command,
>> +                * abort if something has been found.
>> +                */
>> +               for (int i = 0; i < done_alias; i++) {
>> +                       if (!strcmp(cmd_list[i], *argv[0])) {
>> +                               die("loop alias: %s is called twice",
> 
> Please wrap the string in _() so that it can be translated in
> different languages.

I'll do that in v3 as well.

> 
>> +                                   cmd_list[done_alias]);
>> +                       }
>> +               }
>> +

Thanks for reviewing!

Tim

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Junio C Hamano]

Tim Schumacher <timschumi@gmx.de> writes:

> @@ -691,17 +693,34 @@ static int run_argv(int *argcp, const char ***argv)
>  		/* .. then try the external ones */
>  		execv_dashed_external(*argv);
>  
> +		/* Increase the array size and add the current
> +		 * command to it.
> +		 */
> +		cmd_list_alloc += strlen(*argv[0]) + 1;
> +		REALLOC_ARRAY(cmd_list, cmd_list_alloc);
> +		cmd_list[done_alias] = *argv[0];
> +
> +		/* Search the array for occurrences of that command,
> +		 * abort if something has been found.
> +		 */
> +		for (int i = 0; i < done_alias; i++) {
> +			if (!strcmp(cmd_list[i], *argv[0])) {
> +				die("loop alias: %s is called twice",
> +				    cmd_list[done_alias]);
> +			}
> +		}
> +

Wouldn't all of the above become three or four lines that is so
clear that there is no need for any comment if you used string-list,
perhaps?

>  		/* It could be an alias -- this works around the insanity
>  		 * of overriding "git log" with "git show" by having
>  		 * alias.log = show
>  		 */

	/*
	 * Style: our multi-line comment begins with and ends with
	 * slash-asterisk and asterisk-slash on their own lines.
	 */

> -		if (done_alias)
> -			break;
>  		if (!handle_alias(argcp, argv))
>  			break;
> -		done_alias = 1;
> +		done_alias++;
>  	}
>  
> +	free(cmd_list);
> +
>  	return done_alias;
>  }

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Tim Schumacher]

On 05.09.18 19:12, Junio C Hamano wrote:
> Tim Schumacher <timschumi@gmx.de> writes:
> 
>> @@ -691,17 +693,34 @@ static int run_argv(int *argcp, const char ***argv)
>>   		/* .. then try the external ones */
>>   		execv_dashed_external(*argv);
>>   
>> +		/* Increase the array size and add the current
>> +		 * command to it.
>> +		 */
>> +		cmd_list_alloc += strlen(*argv[0]) + 1;
>> +		REALLOC_ARRAY(cmd_list, cmd_list_alloc);
>> +		cmd_list[done_alias] = *argv[0];
>> +
>> +		/* Search the array for occurrences of that command,
>> +		 * abort if something has been found.
>> +		 */
>> +		for (int i = 0; i < done_alias; i++) {
>> +			if (!strcmp(cmd_list[i], *argv[0])) {
>> +				die("loop alias: %s is called twice",
>> +				    cmd_list[done_alias]);
>> +			}
>> +		}
>> +
> 
> Wouldn't all of the above become three or four lines that is so
> clear that there is no need for any comment if you used string-list,
> perhaps?

Whoops, I didn't know that string-list existed. I'll try reworking the
code to use that. Concerning the comments: I planned to remove them
anyways since the code should be simple enough to be understood without
them already.

> 
>>   		/* It could be an alias -- this works around the insanity
>>   		 * of overriding "git log" with "git show" by having
>>   		 * alias.log = show
>>   		 */
> 
> 	/*
> 	 * Style: our multi-line comment begins with and ends with
> 	 * slash-asterisk and asterisk-slash on their own lines.
> 	 */

I wasn't sure if I should have changed that (because I didn't introduce
that comment), but I can fix it in v3.

> 
>> -		if (done_alias)
>> -			break;
>>   		if (!handle_alias(argcp, argv))
>>   			break;
>> -		done_alias = 1;
>> +		done_alias++;
>>   	}
>>   
>> +	free(cmd_list);
>> +
>>   	return done_alias;
>>   }
> 

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Jeff King]

On Wed, Sep 05, 2018 at 10:54:27AM +0200, Tim Schumacher wrote:

> Aliases can only contain non-alias git commands and their
> arguments, not other user-defined aliases. Resolving further
> (nested) aliases is prevented by breaking the loop after the
> first alias was processed. Git then fails with a command-not-found
> error.
> 
> Allow resolving nested aliases by not breaking the loop in
> run_argv() after the first alias was processed. Instead, continue
> incrementing `done_alias` until `handle_alias()` fails, which means that
> there are no further aliases that can be processed. Prevent looping
> aliases by storing substituted commands in `cmd_list` and checking if
> a command has been substituted previously.
> ---
> 
> This is what I've come up with to prevent looping aliases. I'm not too
> happy with the number of indentations needed, but this seemed to be the
> easiest way to search an array for a value.

I think this approach is OK, though I wonder if we'd also be fine with
just:

  if (done_alias++ > 100)
	die("woah, is your alias looping?");

The point is just to prevent a runaway infinite loop, and this does that
while keeping the cost very low for the common case (not that one string
insertion is probably breaking the bank).

It could also extend to ! aliases if we wanted (i.e., my '!git foo'
example from earlier), but you'd have to carry the counter through the
environment between processes.

-Peff

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Tim Schumacher]

On 05.09.18 19:34, Jeff King wrote:
> On Wed, Sep 05, 2018 at 10:54:27AM +0200, Tim Schumacher wrote:
> 
>> Aliases can only contain non-alias git commands and their
>> arguments, not other user-defined aliases. Resolving further
>> (nested) aliases is prevented by breaking the loop after the
>> first alias was processed. Git then fails with a command-not-found
>> error.
>>
>> Allow resolving nested aliases by not breaking the loop in
>> run_argv() after the first alias was processed. Instead, continue
>> incrementing `done_alias` until `handle_alias()` fails, which means that
>> there are no further aliases that can be processed. Prevent looping
>> aliases by storing substituted commands in `cmd_list` and checking if
>> a command has been substituted previously.
>> ---
>>
>> This is what I've come up with to prevent looping aliases. I'm not too
>> happy with the number of indentations needed, but this seemed to be the
>> easiest way to search an array for a value.
> 
> I think this approach is OK, though I wonder if we'd also be fine with
> just:
> 
>    if (done_alias++ > 100)
> 	die("woah, is your alias looping?");
> 
> The point is just to prevent a runaway infinite loop, and this does that
> while keeping the cost very low for the common case (not that one string
> insertion is probably breaking the bank).

I'd opt to use the list-approach instead of aborting when the
counter reaches 100 (or any other value), because it aborts
at the earliest known looping point. I didn't run any tests
comparing both solutions, but I assume the list would perform
faster than the hard-limit, even if it requires slightly more
memory and lines of code.

I hope that I can put the string-list struct to some use,
so that the solution using lists becomes an equally good
solution code-wise.

> 
> It could also extend to ! aliases if we wanted (i.e., my '!git foo'
> example from earlier), but you'd have to carry the counter through the
> environment between processes.

That is a question about "shooting oneself in the foot" again,
but I think trying to prevent that would require more changes
than I can make, and it is definitely out-of-scope for this
patch.

> 
> -Peff
> 
Thanks for reviewing,

Tim

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Ævar Arnfjörð Bjarmason]

On Wed, Sep 05 2018, Tim Schumacher wrote:

> On 05.09.18 19:34, Jeff King wrote:
>> On Wed, Sep 05, 2018 at 10:54:27AM +0200, Tim Schumacher wrote:
>>
>>> Aliases can only contain non-alias git commands and their
>>> arguments, not other user-defined aliases. Resolving further
>>> (nested) aliases is prevented by breaking the loop after the
>>> first alias was processed. Git then fails with a command-not-found
>>> error.
>>>
>>> Allow resolving nested aliases by not breaking the loop in
>>> run_argv() after the first alias was processed. Instead, continue
>>> incrementing `done_alias` until `handle_alias()` fails, which means that
>>> there are no further aliases that can be processed. Prevent looping
>>> aliases by storing substituted commands in `cmd_list` and checking if
>>> a command has been substituted previously.
>>> ---
>>>
>>> This is what I've come up with to prevent looping aliases. I'm not too
>>> happy with the number of indentations needed, but this seemed to be the
>>> easiest way to search an array for a value.
>>
>> I think this approach is OK, though I wonder if we'd also be fine with
>> just:
>>
>>    if (done_alias++ > 100)
>> 	die("woah, is your alias looping?");
>>
>> The point is just to prevent a runaway infinite loop, and this does that
>> while keeping the cost very low for the common case (not that one string
>> insertion is probably breaking the bank).
>
> I'd opt to use the list-approach instead of aborting when the
> counter reaches 100 (or any other value), because it aborts
> at the earliest known looping point. I didn't run any tests
> comparing both solutions, but I assume the list would perform
> faster than the hard-limit, even if it requires slightly more
> memory and lines of code.

I agree that this use of a list is better for a completely different
reason (which I'll comment on in the v4 thread), but this reason doesn't
make any sense to me.

If we're looking at performance we're paying a fixed performance cost
for storing this list of strings over a counter for everything we do
with aliases.

It only helps over a counter for the case where we do have a loop, but
at that point who cares? We're going to exit with an erro anyway and the
user has to fix his config, it doesn't matter if that error happens 1
millisecond earlier.

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Ævar Arnfjörð Bjarmason]

On Wed, Sep 05 2018, Tim Schumacher wrote:

> On 05.09.18 19:34, Jeff King wrote:
>>
>> It could also extend to ! aliases if we wanted (i.e., my '!git foo'
>> example from earlier), but you'd have to carry the counter through the
>> environment between processes.
>
> That is a question about "shooting oneself in the foot" again,
> but I think trying to prevent that would require more changes
> than I can make, and it is definitely out-of-scope for this
> patch.

I agree it could be done later, but it would be great if you could
follow-up with that. Right now if you do:

    a = !git b
    b = !git a

You end up with a fork bomb, and we don't guard against this, and if you
have mixed execution / internal aliasing, e.g.:

    a = b
    b = c
    c = d
    d = !git a

The loop detection doesn't kick in.

It should be easy to add detection for this on top. See what we do with
git_config_push_parameter() in git.c already, i.e. you'd add some
simliar env variable, set items in the string list delimited by
e.g. whitespace, and then just pre-populate your string list with that
if it's set, and re-set it & carry it forward.

Then any combination of internal aliasing and custom commands will
benefit from loop detection.

[PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

Add detection for aliasing loops in cases where one of the aliases
re-invokes git as a shell command. This catches cases like:

    [alias]
    foo = !git bar
    bar = !git foo

Before this change running "git {foo,bar}" would create a
forkbomb. Now using the aliasing loop detection and call history
reporting added in 82f71d9a5a ("alias: show the call history when an
alias is looping", 2018-09-16) and c6d75bc17a ("alias: add support for
aliases of an alias", 2018-09-16) we'll instead report:

    fatal: alias loop detected: expansion of 'foo' does not terminate:
      foo <==
      bar ==>

Since the implementation carries the call history in an environment
variable, using the same sort of trick as used for -c (see
2b64fc894d ("pass "git -c foo=bar" params through environment",
2010-08-23) ). For example:

    [alias]
    one = two
    two = !git three
    three = four
    four = !git five
    five = two

Will, on "git one" report:

    fatal: alias loop detected: expansion of 'one' does not terminate:
      one
      two <==
      three
      four
      five ==>

Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
---

Implements what I suggested in
https://public-inbox.org/git/87o9dar9qc.fsf@evledraar.gmail.com/

 cache.h          |  1 +
 git.c            | 36 ++++++++++++++++++++++++++++++++++--
 t/t0001-init.sh  |  1 +
 t/t0014-alias.sh | 15 ++++++---------
 4 files changed, 42 insertions(+), 11 deletions(-)

diff --git a/cache.h b/cache.h
index d508f3d4f8..00cbd25f1c 100644
--- a/cache.h
+++ b/cache.h
@@ -478,6 +478,7 @@ static inline enum object_type object_type(unsigned int mode)
 #define TEMPLATE_DIR_ENVIRONMENT "GIT_TEMPLATE_DIR"
 #define CONFIG_ENVIRONMENT "GIT_CONFIG"
 #define CONFIG_DATA_ENVIRONMENT "GIT_CONFIG_PARAMETERS"
+#define COMMAND_HISTORY_ENVIRONMENT "GIT_COMMAND_HISTORY"
 #define EXEC_PATH_ENVIRONMENT "GIT_EXEC_PATH"
 #define CEILING_DIRECTORIES_ENVIRONMENT "GIT_CEILING_DIRECTORIES"
 #define NO_REPLACE_OBJECTS_ENVIRONMENT "GIT_NO_REPLACE_OBJECTS"
diff --git a/git.c b/git.c
index 5920f8019b..cba242836c 100644
--- a/git.c
+++ b/git.c
@@ -672,12 +672,43 @@ static void execv_dashed_external(const char **argv)
 		exit(128);
 }
 
+static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
+{
+	const char *old = getenv(COMMAND_HISTORY_ENVIRONMENT);
+	struct strbuf **cmd_history, **ptr;
+
+	if (!old || !*old)
+		return;
+
+	strbuf_addstr(env, old);
+	strbuf_rtrim(env);
+
+	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
+	for (ptr = cmd_history; *ptr; ptr++) {
+		strbuf_rtrim(*ptr);
+		string_list_append(cmd_list, (*ptr)->buf);
+	}
+	strbuf_list_free(cmd_history);
+}
+
+static void add_cmd_history(struct strbuf *env, struct string_list *cmd_list,
+			    const char *cmd)
+{
+	string_list_append(cmd_list, cmd);
+	if (env->len)
+		strbuf_addch(env, ' ');
+	strbuf_addstr(env, cmd);
+	setenv(COMMAND_HISTORY_ENVIRONMENT, env->buf, 1);
+}
+
 static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
-	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
+	struct string_list cmd_list = STRING_LIST_INIT_DUP;
 	struct string_list_item *seen;
+	struct strbuf env = STRBUF_INIT;
 
+	init_cmd_history(&env, &cmd_list);
 	while (1) {
 		/*
 		 * If we tried alias and futzed with our environment,
@@ -711,7 +742,7 @@ static int run_argv(int *argcp, const char ***argv)
 			      " not terminate:%s"), cmd_list.items[0].string, sb.buf);
 		}
 
-		string_list_append(&cmd_list, *argv[0]);
+		add_cmd_history(&env, &cmd_list, *argv[0]);
 
 		/*
 		 * It could be an alias -- this works around the insanity
@@ -724,6 +755,7 @@ static int run_argv(int *argcp, const char ***argv)
 	}
 
 	string_list_clear(&cmd_list, 0);
+	strbuf_release(&env);
 
 	return done_alias;
 }
diff --git a/t/t0001-init.sh b/t/t0001-init.sh
index 182da069f1..eb2ca8a172 100755
--- a/t/t0001-init.sh
+++ b/t/t0001-init.sh
@@ -93,6 +93,7 @@ test_expect_success 'No extra GIT_* on alias scripts' '
 		sed -n \
 			-e "/^GIT_PREFIX=/d" \
 			-e "/^GIT_TEXTDOMAINDIR=/d" \
+			-e "/^GIT_COMMAND_HISTORY=/d" \
 			-e "/^GIT_/s/=.*//p" |
 		sort
 	EOF
diff --git a/t/t0014-alias.sh b/t/t0014-alias.sh
index a070e645d7..9ed03a4a4f 100755
--- a/t/t0014-alias.sh
+++ b/t/t0014-alias.sh
@@ -27,14 +27,11 @@ test_expect_success 'looping aliases - internal execution' '
 	test_i18ngrep "^fatal: alias loop detected: expansion of" output
 '
 
-# This test is disabled until external loops are fixed, because would block
-# the test suite for a full minute.
-#
-#test_expect_failure 'looping aliases - mixed execution' '
-#	git config alias.loop-mixed-1 loop-mixed-2 &&
-#	git config alias.loop-mixed-2 "!git loop-mixed-1" &&
-#	test_must_fail git loop-mixed-1 2>output &&
-#	test_i18ngrep "^fatal: alias loop detected: expansion of" output
-#'
+test_expect_success 'looping aliases - mixed execution' '
+	git config alias.loop-mixed-1 loop-mixed-2 &&
+	git config alias.loop-mixed-2 "!git loop-mixed-1" &&
+	test_must_fail git loop-mixed-1 2>output &&
+	test_i18ngrep "^fatal: alias loop detected: expansion of" output
+'
 
 test_done
-- 
2.19.1.568.g152ad8e336

Re: [PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

On Thu, Oct 18 2018, Ævar Arnfjörð Bjarmason wrote:

> +static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
> +{
> +	const char *old = getenv(COMMAND_HISTORY_ENVIRONMENT);
> +	struct strbuf **cmd_history, **ptr;
> +
> +	if (!old || !*old)
> +		return;
> +
> +	strbuf_addstr(env, old);
> +	strbuf_rtrim(env);
> +
> +	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
> +	for (ptr = cmd_history; *ptr; ptr++) {
> +		strbuf_rtrim(*ptr);
> +		string_list_append(cmd_list, (*ptr)->buf);
> +	}
> +	strbuf_list_free(cmd_history);
> +}
> +
> +static void add_cmd_history(struct strbuf *env, struct string_list *cmd_list,
> +			    const char *cmd)
> +{
> +	string_list_append(cmd_list, cmd);
> +	if (env->len)
> +		strbuf_addch(env, ' ');
> +	strbuf_addstr(env, cmd);
> +	setenv(COMMAND_HISTORY_ENVIRONMENT, env->buf, 1);
> +}
> +
>  static int run_argv(int *argcp, const char ***argv)
>  {
>  	int done_alias = 0;
> -	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
> +	struct string_list cmd_list = STRING_LIST_INIT_DUP;
>  	struct string_list_item *seen;
> +	struct strbuf env = STRBUF_INIT;
>
> +	init_cmd_history(&env, &cmd_list);
>  	while (1) {
>  		/*
>  		 * If we tried alias and futzed with our environment,
> @@ -711,7 +742,7 @@ static int run_argv(int *argcp, const char ***argv)
>  			      " not terminate:%s"), cmd_list.items[0].string, sb.buf);
>  		}
>
> -		string_list_append(&cmd_list, *argv[0]);
> +		add_cmd_history(&env, &cmd_list, *argv[0]);
>
>  		/*
>  		 * It could be an alias -- this works around the insanity

Just to sanity check an assumption of mine: One thing I didn't do is use
sq_quote_buf() and sq_dequote_to_argv() like we do for
CONFIG_DATA_ENVIRONMENT. This is because in the case of config we need
to deal with:

    $ git config alias.cfgdump
    !env
    $ git -c x.y=z -c "foo.bar='baz'" cfgdump|grep baz
    GIT_CONFIG_PARAMETERS='x.y=z' 'foo.bar='\''baz'\'''

But in this case I don't see how a command-name would ever contain
whitespace. So we skip quoting and just delimit by space.

There's also nothing stopping you from doing e.g.:

    $ GIT_COMMAND_HISTORY='foo bar' ~/g/git/git --exec-path=$PWD one
    fatal: alias loop detected: expansion of 'foo' does not terminate:
      foo
      bar
      one
      two <==
      three
      four
      five ==>

Or even confuse the code by adding a whitespace at the beginning:

    $ GIT_COMMAND_HISTORY=' foo bar' ~/g/git/git --exec-path=$PWD one
    fatal: alias loop detected: expansion of '' does not terminate:

      foo
      bar
      one
      two <==
      three
      four
      five ==>

I thought none of this was worth dealing with. Worst case someone's
screwing with this, but I don't see how it would happen accidentally,
and even then we detect the infinite loop and just degrade to confusing
error messages because you decided to screw with git's GIT_* env vars.

Re: [PATCH] alias: detect loops in mixed execution mode

[Jeff King]

On Fri, Oct 19, 2018 at 10:28:22AM +0200, Ævar Arnfjörð Bjarmason wrote:

> > -		string_list_append(&cmd_list, *argv[0]);
> > +		add_cmd_history(&env, &cmd_list, *argv[0]);
> >
> >  		/*
> >  		 * It could be an alias -- this works around the insanity
> 
> Just to sanity check an assumption of mine: One thing I didn't do is use
> sq_quote_buf() and sq_dequote_to_argv() like we do for
> CONFIG_DATA_ENVIRONMENT. This is because in the case of config we need
> to deal with:
> 
>     $ git config alias.cfgdump
>     !env
>     $ git -c x.y=z -c "foo.bar='baz'" cfgdump|grep baz
>     GIT_CONFIG_PARAMETERS='x.y=z' 'foo.bar='\''baz'\'''
> 
> But in this case I don't see how a command-name would ever contain
> whitespace. So we skip quoting and just delimit by space.

Alias names cannot currently contain whitespace, because it's not
allowed in the key. However, we've discussed making the syntax
alias.<name>.command, which would then make it possible.

Whether anyone would use that is a different question, but hey,
apparently some people think "My Documents" is a good name for a
directory. ;)

-Peff

Re: [PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

On Fri, Oct 19 2018, Jeff King wrote:

> On Fri, Oct 19, 2018 at 10:28:22AM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> > -		string_list_append(&cmd_list, *argv[0]);
>> > +		add_cmd_history(&env, &cmd_list, *argv[0]);
>> >
>> >  		/*
>> >  		 * It could be an alias -- this works around the insanity
>>
>> Just to sanity check an assumption of mine: One thing I didn't do is use
>> sq_quote_buf() and sq_dequote_to_argv() like we do for
>> CONFIG_DATA_ENVIRONMENT. This is because in the case of config we need
>> to deal with:
>>
>>     $ git config alias.cfgdump
>>     !env
>>     $ git -c x.y=z -c "foo.bar='baz'" cfgdump|grep baz
>>     GIT_CONFIG_PARAMETERS='x.y=z' 'foo.bar='\''baz'\'''
>>
>> But in this case I don't see how a command-name would ever contain
>> whitespace. So we skip quoting and just delimit by space.
>
> Alias names cannot currently contain whitespace, because it's not
> allowed in the key. However, we've discussed making the syntax
> alias.<name>.command, which would then make it possible.
>
> Whether anyone would use that is a different question, but hey,
> apparently some people think "My Documents" is a good name for a
> directory. ;)

I'll just leave this part as it is for now. If we ever have commands
with whitespace this'll be the least of our worries.

Re: [PATCH] alias: detect loops in mixed execution mode

[Jeff King]

On Thu, Oct 18, 2018 at 10:57:39PM +0000, Ævar Arnfjörð Bjarmason wrote:

> Add detection for aliasing loops in cases where one of the aliases
> re-invokes git as a shell command. This catches cases like:
> 
>     [alias]
>     foo = !git bar
>     bar = !git foo
> 
> Before this change running "git {foo,bar}" would create a
> forkbomb. Now using the aliasing loop detection and call history
> reporting added in 82f71d9a5a ("alias: show the call history when an
> alias is looping", 2018-09-16) and c6d75bc17a ("alias: add support for
> aliases of an alias", 2018-09-16) we'll instead report:
> 
>     fatal: alias loop detected: expansion of 'foo' does not terminate:
>       foo <==
>       bar ==>

The regular alias expansion can generally assume that there's no
conditional recursion going on, because it's expanding everything
itself. But when we involve multiple processes, things get trickier.

For instance, I could do this:

  [alias]
  countdown = "!f() { echo \"$@\"; test \"$1\" -gt 0 && git countdown $(($1-1)); }; f"

which works now, but not with your patch.

Now obviously that's a silly toy example, but are there real cases which
might trigger this? Some plausible ones I can think of:

  - an alias which handles some special cases, then chains to itself for
    the simpler one (or to another alias or script, which ends up
    chaining back to the original)

  - an alias that runs a git command, which then spawns a hook or other
    user-controlled script, which incidentally uses that same alias

I'd guess this sort of thing is pretty rare. But I wonder if we're
crossing the line of trying to assume too much about what the user's
arbitrary code does.

A simple depth counter can limit the fork bomb, and with a high enough
depth would be unlikely to trigger a false positive. It could also
protect non-aliases more reasonably, too (e.g., if you have a 1000-deep
git process hierarchy, there's a good chance you've found an infinite
loop in git itself).

> +static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
> +{
> +	const char *old = getenv(COMMAND_HISTORY_ENVIRONMENT);
> +	struct strbuf **cmd_history, **ptr;
> +
> +	if (!old || !*old)
> +		return;
> +
> +	strbuf_addstr(env, old);
> +	strbuf_rtrim(env);
> +
> +	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
> +	for (ptr = cmd_history; *ptr; ptr++) {
> +		strbuf_rtrim(*ptr);
> +		string_list_append(cmd_list, (*ptr)->buf);
> +	}
> +	strbuf_list_free(cmd_history);

Maybe string_list_split() would be a little simpler?

-Peff

Re: [PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

On Fri, Oct 19 2018, Jeff King wrote:

> On Thu, Oct 18, 2018 at 10:57:39PM +0000, Ævar Arnfjörð Bjarmason wrote:
>
>> Add detection for aliasing loops in cases where one of the aliases
>> re-invokes git as a shell command. This catches cases like:
>>
>>     [alias]
>>     foo = !git bar
>>     bar = !git foo
>>
>> Before this change running "git {foo,bar}" would create a
>> forkbomb. Now using the aliasing loop detection and call history
>> reporting added in 82f71d9a5a ("alias: show the call history when an
>> alias is looping", 2018-09-16) and c6d75bc17a ("alias: add support for
>> aliases of an alias", 2018-09-16) we'll instead report:
>>
>>     fatal: alias loop detected: expansion of 'foo' does not terminate:
>>       foo <==
>>       bar ==>
>
> The regular alias expansion can generally assume that there's no
> conditional recursion going on, because it's expanding everything
> itself. But when we involve multiple processes, things get trickier.
>
> For instance, I could do this:
>
>   [alias]
>   countdown = "!f() { echo \"$@\"; test \"$1\" -gt 0 && git countdown $(($1-1)); }; f"
>
> which works now, but not with your patch.
>
> Now obviously that's a silly toy example, but are there real cases which
> might trigger this? Some plausible ones I can think of:
>
>   - an alias which handles some special cases, then chains to itself for
>     the simpler one (or to another alias or script, which ends up
>     chaining back to the original)
>
>   - an alias that runs a git command, which then spawns a hook or other
>     user-controlled script, which incidentally uses that same alias
>
> I'd guess this sort of thing is pretty rare. But I wonder if we're
> crossing the line of trying to assume too much about what the user's
> arbitrary code does.
>
> A simple depth counter can limit the fork bomb, and with a high enough
> depth would be unlikely to trigger a false positive. It could also
> protect non-aliases more reasonably, too (e.g., if you have a 1000-deep
> git process hierarchy, there's a good chance you've found an infinite
> loop in git itself).

I don't think this edge case you're describing is very plausible, and I
doubt it exists in the wild.

But going by my personal incredulity and a git release breaking code in
the wild would suck, so agree that I need to re-roll this to anticipate
that.

I don't have time to write it now, but what do you think about a version
of this where we introduce a core.recursionLimit setting, and by default
set it to "1" (for one recursion), so by default die just as we do now,
but with some advice() saying that we've bailed out early because this
looks crazy, but you can set it to e.g. "1000" if you think you know
what you're doing, or "0" for no limit.

The reason I'd like to do that is because I think it's *way* more common
to do this accidentally than intentionally, and by having a default
limit of 1000 we'd print a really long error message, or alternatively
would have to get into the mess of de-duplicating the callstack as we
print the error.

It also has the advantage that if people in the wild really use this
they'll chime in about this new annoying core.recursionLimit=1 setting,
at the cost of me having annoyed them all by breaking their working
code.

>> +static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
>> +{
>> +	const char *old = getenv(COMMAND_HISTORY_ENVIRONMENT);
>> +	struct strbuf **cmd_history, **ptr;
>> +
>> +	if (!old || !*old)
>> +		return;
>> +
>> +	strbuf_addstr(env, old);
>> +	strbuf_rtrim(env);
>> +
>> +	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
>> +	for (ptr = cmd_history; *ptr; ptr++) {
>> +		strbuf_rtrim(*ptr);
>> +		string_list_append(cmd_list, (*ptr)->buf);
>> +	}
>> +	strbuf_list_free(cmd_history);
>
> Maybe string_list_split() would be a little simpler?

Yeah looks like it. I cargo-culted this from elsewhere without looking
at that API. I'll look into it.

Re: [PATCH] alias: detect loops in mixed execution mode

[Jeff King]

On Sat, Oct 20, 2018 at 01:14:28PM +0200, Ævar Arnfjörð Bjarmason wrote:

> > I'd guess this sort of thing is pretty rare. But I wonder if we're
> > crossing the line of trying to assume too much about what the user's
> > arbitrary code does.
> >
> > A simple depth counter can limit the fork bomb, and with a high enough
> > depth would be unlikely to trigger a false positive. It could also
> > protect non-aliases more reasonably, too (e.g., if you have a 1000-deep
> > git process hierarchy, there's a good chance you've found an infinite
> > loop in git itself).
> 
> I don't think this edge case you're describing is very plausible, and I
> doubt it exists in the wild.
> 
> But going by my personal incredulity and a git release breaking code in
> the wild would suck, so agree that I need to re-roll this to anticipate
> that.

I agree it's probably quite rare, if it exists at all. But I also wonder
how important looping alias protection is. It's also rare, and the
outcome is usually "gee, I wonder why this is taking so long? ^C".

At least that's my instinct. I don't remember having run into this at
all myself (though certainly I have written my fair share of infinite
loops in other systems, like bash aliases, and that is what happened).

> I don't have time to write it now, but what do you think about a version
> of this where we introduce a core.recursionLimit setting, and by default
> set it to "1" (for one recursion), so by default die just as we do now,
> but with some advice() saying that we've bailed out early because this
> looks crazy, but you can set it to e.g. "1000" if you think you know
> what you're doing, or "0" for no limit.
> 
> The reason I'd like to do that is because I think it's *way* more common
> to do this accidentally than intentionally, and by having a default
> limit of 1000 we'd print a really long error message, or alternatively
> would have to get into the mess of de-duplicating the callstack as we
> print the error.

Would we print a long error message? I'd assume that we'd just recurse
for longer and print one error message that says:

  fatal: woah, you're 1000-levels deep in Git commands!

That doesn't help the user find the recursion, but re-running with
GIT_TRACE=1 would make it pretty clear, I'd think.

> It also has the advantage that if people in the wild really use this
> they'll chime in about this new annoying core.recursionLimit=1 setting,
> at the cost of me having annoyed them all by breaking their working
> code.

Right, I'm not too happy about that annoyance. But it seems clear that I
think the loop protection is way less important than you do, so I'm
willing to sacrifice (or more accurately, risk the possibility of
sacrificing) a lot less for it. :)

I dunno. I doubt it is likely to help or hinder that many people either
way.

> >> +	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
> >> +	for (ptr = cmd_history; *ptr; ptr++) {
> >> +		strbuf_rtrim(*ptr);
> >> +		string_list_append(cmd_list, (*ptr)->buf);
> >> +	}
> >> +	strbuf_list_free(cmd_history);
> >
> > Maybe string_list_split() would be a little simpler?
> 
> Yeah looks like it. I cargo-culted this from elsewhere without looking
> at that API. I'll look into it.

I cheated before writing that and confirmed that it does seem to work. ;)

Here's the patch in case it is useful. IMHO we should be trying to get
rid of strbuf_split, because it's a pretty crappy interface.

diff --git a/git.c b/git.c
index cba242836c..9d1b66a1fa 100644
--- a/git.c
+++ b/git.c
@@ -675,7 +675,6 @@ static void execv_dashed_external(const char **argv)
 static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
 {
 	const char *old = getenv(COMMAND_HISTORY_ENVIRONMENT);
-	struct strbuf **cmd_history, **ptr;
 
 	if (!old || !*old)
 		return;
@@ -683,12 +682,7 @@ static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
 	strbuf_addstr(env, old);
 	strbuf_rtrim(env);
 
-	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
-	for (ptr = cmd_history; *ptr; ptr++) {
-		strbuf_rtrim(*ptr);
-		string_list_append(cmd_list, (*ptr)->buf);
-	}
-	strbuf_list_free(cmd_history);
+	string_list_split(cmd_list, env->buf, ' ', -1);
 }
 
 static void add_cmd_history(struct strbuf *env, struct string_list *cmd_list,

-Peff

Re: [PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

On Sat, Oct 20 2018, Jeff King wrote:

> On Sat, Oct 20, 2018 at 01:14:28PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> > I'd guess this sort of thing is pretty rare. But I wonder if we're
>> > crossing the line of trying to assume too much about what the user's
>> > arbitrary code does.
>> >
>> > A simple depth counter can limit the fork bomb, and with a high enough
>> > depth would be unlikely to trigger a false positive. It could also
>> > protect non-aliases more reasonably, too (e.g., if you have a 1000-deep
>> > git process hierarchy, there's a good chance you've found an infinite
>> > loop in git itself).
>>
>> I don't think this edge case you're describing is very plausible, and I
>> doubt it exists in the wild.
>>
>> But going by my personal incredulity and a git release breaking code in
>> the wild would suck, so agree that I need to re-roll this to anticipate
>> that.
>
> I agree it's probably quite rare, if it exists at all. But I also wonder
> how important looping alias protection is. It's also rare, and the
> outcome is usually "gee, I wonder why this is taking so long? ^C".
>
> At least that's my instinct. I don't remember having run into this at
> all myself (though certainly I have written my fair share of infinite
> loops in other systems, like bash aliases, and that is what happened).
>
>> I don't have time to write it now, but what do you think about a version
>> of this where we introduce a core.recursionLimit setting, and by default
>> set it to "1" (for one recursion), so by default die just as we do now,
>> but with some advice() saying that we've bailed out early because this
>> looks crazy, but you can set it to e.g. "1000" if you think you know
>> what you're doing, or "0" for no limit.
>>
>> The reason I'd like to do that is because I think it's *way* more common
>> to do this accidentally than intentionally, and by having a default
>> limit of 1000 we'd print a really long error message, or alternatively
>> would have to get into the mess of de-duplicating the callstack as we
>> print the error.
>
> Would we print a long error message? I'd assume that we'd just recurse
> for longer and print one error message that says:
>
>   fatal: woah, you're 1000-levels deep in Git commands!
>
> That doesn't help the user find the recursion, but re-running with
> GIT_TRACE=1 would make it pretty clear, I'd think.

Yeah the reason I'd like the core.recursionLimit=1 setting by default is
so that we can also print the same pretty and easy to grok error message
we do now for non-! aliases by default without spewing out ~3-4k lines
of mostly duplicate output (with a default limit of 1000).

We didn't support chained aliases at all before, so I think the odds
that people will run into this now will increase as they add "!" to
existing aliases, and I'd like to have git's UI friendly enough to tell
users what went wrong by default, and not have to resort to the likes of
GIT_TRACE=1 which really should be left to powerusers.

>> It also has the advantage that if people in the wild really use this
>> they'll chime in about this new annoying core.recursionLimit=1 setting,
>> at the cost of me having annoyed them all by breaking their working
>> code.
>
> Right, I'm not too happy about that annoyance. But it seems clear that I
> think the loop protection is way less important than you do, so I'm
> willing to sacrifice (or more accurately, risk the possibility of
> sacrificing) a lot less for it. :)
>
> I dunno. I doubt it is likely to help or hinder that many people either
> way.
>
>> >> +	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
>> >> +	for (ptr = cmd_history; *ptr; ptr++) {
>> >> +		strbuf_rtrim(*ptr);
>> >> +		string_list_append(cmd_list, (*ptr)->buf);
>> >> +	}
>> >> +	strbuf_list_free(cmd_history);
>> >
>> > Maybe string_list_split() would be a little simpler?
>>
>> Yeah looks like it. I cargo-culted this from elsewhere without looking
>> at that API. I'll look into it.
>
> I cheated before writing that and confirmed that it does seem to work. ;)
>
> Here's the patch in case it is useful. IMHO we should be trying to get
> rid of strbuf_split, because it's a pretty crappy interface.
>
> diff --git a/git.c b/git.c
> index cba242836c..9d1b66a1fa 100644
> --- a/git.c
> +++ b/git.c
> @@ -675,7 +675,6 @@ static void execv_dashed_external(const char **argv)
>  static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
>  {
>  	const char *old = getenv(COMMAND_HISTORY_ENVIRONMENT);
> -	struct strbuf **cmd_history, **ptr;
>
>  	if (!old || !*old)
>  		return;
> @@ -683,12 +682,7 @@ static void init_cmd_history(struct strbuf *env, struct string_list *cmd_list)
>  	strbuf_addstr(env, old);
>  	strbuf_rtrim(env);
>
> -	cmd_history = strbuf_split_buf(old, strlen(old), ' ', 0);
> -	for (ptr = cmd_history; *ptr; ptr++) {
> -		strbuf_rtrim(*ptr);
> -		string_list_append(cmd_list, (*ptr)->buf);
> -	}
> -	strbuf_list_free(cmd_history);
> +	string_list_split(cmd_list, env->buf, ' ', -1);
>  }
>
>  static void add_cmd_history(struct strbuf *env, struct string_list *cmd_list,

Thanks! Will squash it.

Re: [PATCH] alias: detect loops in mixed execution mode

[Jeff King]

On Sat, Oct 20, 2018 at 09:18:21PM +0200, Ævar Arnfjörð Bjarmason wrote:

> We didn't support chained aliases at all before, so I think the odds
> that people will run into this now will increase as they add "!" to
> existing aliases, and I'd like to have git's UI friendly enough to tell
> users what went wrong by default, and not have to resort to the likes of
> GIT_TRACE=1 which really should be left to powerusers.

It's true that non-! aliases couldn't recurse before, but couldn't "!"
ones always do so?

-Peff

Re: [PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

On Mon, Oct 22 2018, Jeff King wrote:

> On Sat, Oct 20, 2018 at 09:18:21PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> We didn't support chained aliases at all before, so I think the odds
>> that people will run into this now will increase as they add "!" to
>> existing aliases, and I'd like to have git's UI friendly enough to tell
>> users what went wrong by default, and not have to resort to the likes of
>> GIT_TRACE=1 which really should be left to powerusers.
>
> It's true that non-! aliases couldn't recurse before, but couldn't "!"
> ones always do so?

Yes. I meant that maybe now it's a feature that works for that people
will start using it, and then convert some of that to !-aliases they
wouldn't otherwise have written. Just idle speculation...

Re: [PATCH] alias: detect loops in mixed execution mode

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> I agree it's probably quite rare, if it exists at all. But I also wonder
> how important looping alias protection is. It's also rare, and the
> outcome is usually "gee, I wonder why this is taking so long? ^C".
>
> At least that's my instinct. I don't remember having run into this at
> all myself (though certainly I have written my fair share of infinite
> loops in other systems, like bash aliases, and that is what happened).

Yup, that instict is shared with me, and I tend to prefer something
based on a simple counter for that reason.

> Would we print a long error message? I'd assume that we'd just recurse
> for longer and print one error message that says:
>
>   fatal: woah, you're 1000-levels deep in Git commands!
>
> That doesn't help the user find the recursion, but re-running with
> GIT_TRACE=1 would make it pretty clear, I'd think.

Thanks.

Re: [PATCH] alias: detect loops in mixed execution mode

[Jeff King]

On Sat, Oct 20, 2018 at 02:58:53PM -0400, Jeff King wrote:

> On Sat, Oct 20, 2018 at 01:14:28PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
> > > I'd guess this sort of thing is pretty rare. But I wonder if we're
> > > crossing the line of trying to assume too much about what the user's
> > > arbitrary code does.
> > >
> > > A simple depth counter can limit the fork bomb, and with a high enough
> > > depth would be unlikely to trigger a false positive. It could also
> > > protect non-aliases more reasonably, too (e.g., if you have a 1000-deep
> > > git process hierarchy, there's a good chance you've found an infinite
> > > loop in git itself).
> > 
> > I don't think this edge case you're describing is very plausible, and I
> > doubt it exists in the wild.
> > 
> > But going by my personal incredulity and a git release breaking code in
> > the wild would suck, so agree that I need to re-roll this to anticipate
> > that.
> 
> I agree it's probably quite rare, if it exists at all. But I also wonder
> how important looping alias protection is. It's also rare, and the
> outcome is usually "gee, I wonder why this is taking so long? ^C".

Hmph. So I was speaking before purely hypothetically, but now that your
patch is in 'next', it is part of my daily build. And indeed, I hit a
false positive within 5 minutes of building it. ;)

I have an alias like this:

  $ git help dotgit
  'dotgit' is aliased to '!git rev-parse 2>/dev/null || cd ~/compile/git; git'

The idea being that I can run "git dotgit foo" to run "git foo" in the
current directory, or if it is not a git repository, in my checkout of
git.git.

I use it in two ways:

  - some of my aliases know about it themselves. So I have an alias "ll"
    that does:

      $ git help ll
      'll' is aliased to '!git dotgit --no-pager log --no-walk=unsorted --format='%h (%s, %ad)' --date=short'

    with the idea being to produce a nice annotation for a commit id.
    Using "git dotgit" there lets me just run it from any directory,
    since 99% of the time I am working on git.git anyway.

  - I have a vim command defined:

      command! -nargs=* Git :call MaybeInlineCommand("git dotgit <args>")

    so I can do ":Git foo" inside vim and it uses either the current
    repo (e.g., if I'm writing a commit message) or git.git (e.g., if
    I'm writing an email and didn't start in the repo).

So of course the alias expansion is something like (in older versions of
Git):

  1. "git dotgit ll" runs the dotgit alias, which sees that we need to go
     to the git.git checkout

  2. that runs "git ll"

  3. that runs "git dotgit log"; this second dotgit invocation sees we're
     already in a repository and is a noop

  4. git-log runs

With your patch, step 3 complains:

  $ git dotgit ll
  fatal: alias loop detected: expansion of 'dotgit' does not terminate:
  dotgit <==
  ll ==>

So I would really prefer a depth counter that can be set sufficiently
high to make this case work. ;)


As an aside, I got to experience this error message as an unsuspecting
user would. Unfortunately the output was not super helpful for figuring
out the cause. I scratched my head for a while before remembering that
"ll" uses "dotgit" explicitly (which was quite apparent when running
GIT_TRACE=1, or "git help ll"). I think showing the alias definitions in
the loop output would have made it much more obvious (if perhaps a bit
uglier).  E.g., something like:

  fatal: alias loop...
  ==> dotgit is aliased to '!git rev-parse ...'
  <== ll is aliased to '!git dotgit ...'

-Peff

Re: [PATCH] alias: detect loops in mixed execution mode

[Ævar Arnfjörð Bjarmason]

On Fri, Oct 26 2018, Jeff King wrote:

> On Sat, Oct 20, 2018 at 02:58:53PM -0400, Jeff King wrote:
>
>> On Sat, Oct 20, 2018 at 01:14:28PM +0200, Ævar Arnfjörð Bjarmason wrote:
>>
>> > > I'd guess this sort of thing is pretty rare. But I wonder if we're
>> > > crossing the line of trying to assume too much about what the user's
>> > > arbitrary code does.
>> > >
>> > > A simple depth counter can limit the fork bomb, and with a high enough
>> > > depth would be unlikely to trigger a false positive. It could also
>> > > protect non-aliases more reasonably, too (e.g., if you have a 1000-deep
>> > > git process hierarchy, there's a good chance you've found an infinite
>> > > loop in git itself).
>> >
>> > I don't think this edge case you're describing is very plausible, and I
>> > doubt it exists in the wild.
>> >
>> > But going by my personal incredulity and a git release breaking code in
>> > the wild would suck, so agree that I need to re-roll this to anticipate
>> > that.
>>
>> I agree it's probably quite rare, if it exists at all. But I also wonder
>> how important looping alias protection is. It's also rare, and the
>> outcome is usually "gee, I wonder why this is taking so long? ^C".
>
> Hmph. So I was speaking before purely hypothetically, but now that your
> patch is in 'next', it is part of my daily build. And indeed, I hit a
> false positive within 5 minutes of building it. ;)
>
> I have an alias like this:
>
>   $ git help dotgit
>   'dotgit' is aliased to '!git rev-parse 2>/dev/null || cd ~/compile/git; git'
>
> The idea being that I can run "git dotgit foo" to run "git foo" in the
> current directory, or if it is not a git repository, in my checkout of
> git.git.
>
> I use it in two ways:
>
>   - some of my aliases know about it themselves. So I have an alias "ll"
>     that does:
>
>       $ git help ll
>       'll' is aliased to '!git dotgit --no-pager log --no-walk=unsorted --format='%h (%s, %ad)' --date=short'
>
>     with the idea being to produce a nice annotation for a commit id.
>     Using "git dotgit" there lets me just run it from any directory,
>     since 99% of the time I am working on git.git anyway.
>
>   - I have a vim command defined:
>
>       command! -nargs=* Git :call MaybeInlineCommand("git dotgit <args>")
>
>     so I can do ":Git foo" inside vim and it uses either the current
>     repo (e.g., if I'm writing a commit message) or git.git (e.g., if
>     I'm writing an email and didn't start in the repo).
>
> So of course the alias expansion is something like (in older versions of
> Git):
>
>   1. "git dotgit ll" runs the dotgit alias, which sees that we need to go
>      to the git.git checkout
>
>   2. that runs "git ll"
>
>   3. that runs "git dotgit log"; this second dotgit invocation sees we're
>      already in a repository and is a noop
>
>   4. git-log runs
>
> With your patch, step 3 complains:
>
>   $ git dotgit ll
>   fatal: alias loop detected: expansion of 'dotgit' does not terminate:
>   dotgit <==
>   ll ==>
>
> So I would really prefer a depth counter that can be set sufficiently
> high to make this case work. ;)
>
>
> As an aside, I got to experience this error message as an unsuspecting
> user would. Unfortunately the output was not super helpful for figuring
> out the cause. I scratched my head for a while before remembering that
> "ll" uses "dotgit" explicitly (which was quite apparent when running
> GIT_TRACE=1, or "git help ll"). I think showing the alias definitions in
> the loop output would have made it much more obvious (if perhaps a bit
> uglier).  E.g., something like:
>
>   fatal: alias loop...
>   ==> dotgit is aliased to '!git rev-parse ...'
>   <== ll is aliased to '!git dotgit ...'
>
> -Peff

Yikes.

Junio: After your previous "What's cooking" in
<xmqq8t2u1nkh.fsf@gitster-ct.c.googlers.com> I sent
<87ftx0dg4r.fsf@evledraar.gmail.com>, but should have just replied to
"What's cooking".

I.e. I think this topic should just be ejected, I'll try to submit a
re-roll, but don't know if I have time in the next few days.

Can you please queue a "git revert" of it (or rewind next, but not sure
if you want to do that...).

Re: [PATCH] alias: detect loops in mixed execution mode

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Hmph. So I was speaking before purely hypothetically, but now that your
> patch is in 'next', it is part of my daily build. And indeed, I hit a
> false positive within 5 minutes of building it. ;)

Sounds like somebody is having not-so-fun-a-time having "I told you
so" moment.  The 'dotgit' thing already feels bit convoluted but I
would say that it is still within the realm of reasonable workflow
elements.

> ...
> With your patch, step 3 complains:
>
>   $ git dotgit ll
>   fatal: alias loop detected: expansion of 'dotgit' does not terminate:
>   dotgit <==
>   ll ==>
>
> So I would really prefer a depth counter that can be set sufficiently
> high to make this case work. ;)

Sounds like a concrete enough case to demonstrate why one-level deep
loop detector is not sufficient X-<.

Re: [PATCH] alias: detect loops in mixed execution mode

[Jeff King]

On Mon, Oct 29, 2018 at 12:44:58PM +0900, Junio C Hamano wrote:

> Jeff King <peff@peff.net> writes:
> 
> > Hmph. So I was speaking before purely hypothetically, but now that your
> > patch is in 'next', it is part of my daily build. And indeed, I hit a
> > false positive within 5 minutes of building it. ;)
> 
> Sounds like somebody is having not-so-fun-a-time having "I told you
> so" moment.  The 'dotgit' thing already feels bit convoluted but I
> would say that it is still within the realm of reasonable workflow
> elements.

To be clear, the "dotgit" thing _is_ weird and convoluted. And I imagine
that I push Git more than 99% of our users would. But I also won't be
surprised if somebody else has something similarly disgusting in the
wild. :)

TBH, I'm still not really sold on the idea of doing loop detection at
all in this case. But I can live with it if others feel strongly. What
makes the current patch so bad is that there's no escape hatch (i.e.,
even a depth counter with a default of "1" would have given me something
I could bump).

-Peff

Re: [RFC PATCH v2] Allow aliases that include other aliases

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

>> This is what I've come up with to prevent looping aliases. I'm not too
>> happy with the number of indentations needed, but this seemed to be the
>> easiest way to search an array for a value.
>
> I think this approach is OK, though I wonder if we'd also be fine with
> just:
>
>   if (done_alias++ > 100)
> 	die("woah, is your alias looping?");
>
> The point is just to prevent a runaway infinite loop, and this does that
> while keeping the cost very low for the common case (not that one string
> insertion is probably breaking the bank).

Yeah, as a hack, I guess the simpler the solution, the better it
would be.

[PATCH v3] Allow aliases that include other aliases

[Tim Schumacher]

Aliases can only contain non-alias git commands and their
arguments, not other user-defined aliases. Resolving further
(nested) aliases is prevented by breaking the loop after the
first alias was processed. Git then fails with a command-not-found
error.

Allow resolving nested aliases by not breaking the loop in
run_argv() after the first alias was processed. Instead, continue
incrementing `done_alias` until `handle_alias()` fails, which means that
there are no further aliases that can be processed. Prevent looping
aliases by storing substituted commands in `cmd_list` and checking if
a command has been substituted previously.

While we're at it, fix a styling issue just below the added code.
---
 git.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/git.c b/git.c
index c27c38738..64f5fbd57 100644
--- a/git.c
+++ b/git.c
@@ -674,6 +674,7 @@ static void execv_dashed_external(const char **argv)
 static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
+	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
 
 	while (1) {
 		/*
@@ -691,17 +692,23 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);
 
-		/* It could be an alias -- this works around the insanity
+		if (string_list_has_string(&cmd_list, *argv[0]))
+			die(_("loop alias: %s is called twice"), *argv[0]);
+
+		string_list_append(&cmd_list, *argv[0]);
+
+		/*
+		 * It could be an alias -- this works around the insanity
 		 * of overriding "git log" with "git show" by having
 		 * alias.log = show
 		 */
-		if (done_alias)
-			break;
 		if (!handle_alias(argcp, argv))
 			break;
-		done_alias = 1;
+		done_alias++;
 	}
 
+	string_list_clear(&cmd_list, 0);
+
 	return done_alias;
 }
 
-- 
2.19.0.rc1.2.g8008c49c4.dirty

Re: [PATCH v3] Allow aliases that include other aliases

[Ævar Arnfjörð Bjarmason]

On Thu, Sep 06 2018, Tim Schumacher wrote:

> Aliases can only contain non-alias git commands and their
> arguments, not other user-defined aliases. Resolving further
> (nested) aliases is prevented by breaking the loop after the
> first alias was processed. Git then fails with a command-not-found
> error.
>
> Allow resolving nested aliases by not breaking the loop in
> run_argv() after the first alias was processed. Instead, continue
> incrementing `done_alias` until `handle_alias()` fails, which means that
> there are no further aliases that can be processed. Prevent looping
> aliases by storing substituted commands in `cmd_list` and checking if
> a command has been substituted previously.
>
> While we're at it, fix a styling issue just below the added code.
> ---
>  git.c | 15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)
>
> diff --git a/git.c b/git.c
> index c27c38738..64f5fbd57 100644
> --- a/git.c
> +++ b/git.c
> @@ -674,6 +674,7 @@ static void execv_dashed_external(const char **argv)
>  static int run_argv(int *argcp, const char ***argv)
>  {
>  	int done_alias = 0;
> +	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
>
>  	while (1) {
>  		/*
> @@ -691,17 +692,23 @@ static int run_argv(int *argcp, const char ***argv)
>  		/* .. then try the external ones */
>  		execv_dashed_external(*argv);
>
> -		/* It could be an alias -- this works around the insanity
> +		if (string_list_has_string(&cmd_list, *argv[0]))
> +			die(_("loop alias: %s is called twice"), *argv[0]);
> +
> +		string_list_append(&cmd_list, *argv[0]);
> +
> +		/*
> +		 * It could be an alias -- this works around the insanity
>  		 * of overriding "git log" with "git show" by having
>  		 * alias.log = show
>  		 */
> -		if (done_alias)
> -			break;
>  		if (!handle_alias(argcp, argv))
>  			break;
> -		done_alias = 1;
> +		done_alias++;
>  	}
>
> +	string_list_clear(&cmd_list, 0);
> +
>  	return done_alias;
>  }

[In my just-sent
https://public-inbox.org/git/87r2i6rbiy.fsf@evledraar.gmail.com/ I
should have said "the v3 thread"]

Thanks for working on this, comments:

If we don't have some test for these sort of aliasing loops that fails
now, we really should add that in a 1/2 and fix it in this patch in 2/2.

This error reporting is quite bad, consider:

    [alias]
        foo = bar
        bar = baz
        baz = foo

We then say:

    $ ./git --exec-path=$PWD foo
    fatal: loop alias: bar is called twice

That makes sense from an implementaion perspective, i.e. we lookup "bar"
twice. But let's do better. If I have aliase like:

    a = b
    b = c
    c = d
    d = e
    e = c

It should be telling me that my "e" expansion looped back to the "c = d"
expansion. Here's a patch to implement that, feel free to either squash
it in with my Signed-Off-By, or tacked onto a v4 version of this,
whichever you think makes sense:

diff --git a/git.c b/git.c
index 64f5fbd572..38f1033e52 100644
--- a/git.c
+++ b/git.c
@@ -692,8 +692,64 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);

-		if (string_list_has_string(&cmd_list, *argv[0]))
-			die(_("loop alias: %s is called twice"), *argv[0]);
+		if (string_list_has_string(&cmd_list, *argv[0])) {
+			struct strbuf sb = STRBUF_INIT;
+			int i, seen_at_idx = -1;
+
+			/*
+			 * Find the re-entry point for the alias
+			 * loop. TODO: There really should be a
+			 * "return the index of the first matching"
+			 * helper in string-list.c.
+			 */
+			for (i = 0; i < cmd_list.nr; i++) {
+				if (!strcmp(*argv[0], cmd_list.items[i].string))
+					seen_at_idx = i;
+			}
+			assert(seen_at_idx != -1);
+
+			for (i = 1; i < cmd_list.nr; i++) {
+				if (i - 1 == seen_at_idx)
+					/*
+					 * TRANSLATORS: This is a the
+					 * re-enttry point in the list
+					 * printed out by the "alias
+					 * loop" message below.
+					 */
+					strbuf_addf(&sb, _("    %d. %s = %s <== The re-entry point in the loop\n"),
+						    i,
+						    cmd_list.items[i - 1].string,
+						    cmd_list.items[i].string);
+				else
+					/*
+					 * TRANSLATORS: This is a
+					 * single item in the list
+					 * printed out by the "alias
+					 * loop" message below.
+					 */
+					strbuf_addf(&sb, _("    %d. %s = %s\n"),
+						    i,
+						    cmd_list.items[i - 1].string,
+						    cmd_list.items[i].string);
+			}
+			/*
+			 * TRANSLATORS: This is the last item in the
+			 * list printed out by the "alias loop"
+			 * message below.
+			 */
+			strbuf_addf(&sb, _("    %d. %s = %s <== This is where the loop started!"),
+				    i,
+				    cmd_list.items[i - 1].string,
+				    *argv[0]);
+			/*
+			 * TRANSLATORS: The %s here at the end is
+			 * going to be a list of aliases as formatted
+			 * by the messages whose comments mention
+			 * "alias loop" above.
+			 */
+			die(_("alias loop: When expanding the alias '%s' we ran into a loop:\n%s"),
+			    cmd_list.items[0].string, sb.buf);
+		}

 		string_list_append(&cmd_list, *argv[0]);

Now we'll print errors like:

    $ ./git --exec-path=$PWD a
    fatal: alias loop: When expanding the alias 'a' we ran into a loop:
        1. a = b
        2. b = c
        3. c = d <== The re-entry point in the loop
        4. d = e
        5. e = c <== This is where the loop started!

Or, in the much simpler case of foo = bar; bar = foo:

    $ ./git --exec-path=$PWD foo
    fatal: alias loop: When expanding the alias 'foo' we ran into a loop:
        1. foo = bar <== The re-entry point in the loop
        2. bar = foo <== This is where the loop started!

I haven't tested this much, so maybe there's some edge cases I haven't
thought of / bugs in this reporting code, but hey, that's what the tests
I suggested are for :)

It's a lot more verbose, but I think it's worth it to produce better
error messages.

Re: [PATCH v3] Allow aliases that include other aliases

[Jeff King]

On Thu, Sep 06, 2018 at 04:01:39PM +0200, Ævar Arnfjörð Bjarmason wrote:

> If we don't have some test for these sort of aliasing loops that fails
> now, we really should add that in a 1/2 and fix it in this patch in 2/2.

Yes, I'd agree that this is worth adding a test (especially if the
output routines get more complex).

> That makes sense from an implementaion perspective, i.e. we lookup "bar"
> twice. But let's do better. If I have aliase like:
> 
>     a = b
>     b = c
>     c = d
>     d = e
>     e = c
> 
> It should be telling me that my "e" expansion looped back to the "c = d"
> expansion. Here's a patch to implement that, feel free to either squash
> it in with my Signed-Off-By, or tacked onto a v4 version of this,
> whichever you think makes sense:

I don't have a strong opinion on whether this is worth it, but I think
your implementation could be a little simpler:

> diff --git a/git.c b/git.c
> index 64f5fbd572..38f1033e52 100644
> --- a/git.c
> +++ b/git.c
> @@ -692,8 +692,64 @@ static int run_argv(int *argcp, const char ***argv)
>  		/* .. then try the external ones */
>  		execv_dashed_external(*argv);
> 
> -		if (string_list_has_string(&cmd_list, *argv[0]))
> -			die(_("loop alias: %s is called twice"), *argv[0]);
> +		if (string_list_has_string(&cmd_list, *argv[0])) {
> +			struct strbuf sb = STRBUF_INIT;
> +			int i, seen_at_idx = -1;
> +
> +			/*
> +			 * Find the re-entry point for the alias
> +			 * loop. TODO: There really should be a
> +			 * "return the index of the first matching"
> +			 * helper in string-list.c.
> +			 */
> +			for (i = 0; i < cmd_list.nr; i++) {
> +				if (!strcmp(*argv[0], cmd_list.items[i].string))
> +					seen_at_idx = i;
> +			}
> +			assert(seen_at_idx != -1);

The string-list code doesn't generally deal in indices. You can use
string_list_find_insert_index(), but its return value is a little funky
for the existing case. You can also just do:

  struct string_list_item *seen;
  ...
  seen = string_list_lookup(&cmd_list, *argv[0]);
  if (seen) {
	/* we have a loop */
	int idx = seen - cmd_list.items;

That's a little intimate with the string-list implementation as an array
of string_list, but it's already pretty standard to walk over and
dereference that list (including in your patch). But also see below.

Side note: there's actually a bigger problem with the original patch:
the string list is unsorted (because it uses string_list_append(), and
which is why your linear walk works here). But string_list_has_string()
assumes it is sorted.  So I think we'd actually want to use
unsorted_string_list_has_string() or unsorted_string_list_lookup().

> +			for (i = 1; i < cmd_list.nr; i++) {
> +				if (i - 1 == seen_at_idx)
> +					/*
> +					 * TRANSLATORS: This is a the
> +					 * re-enttry point in the list
> +					 * printed out by the "alias
> +					 * loop" message below.
> +					 */
> +					strbuf_addf(&sb, _("    %d. %s = %s <== The re-entry point in the loop\n"),
> +						    i,
> +						    cmd_list.items[i - 1].string,
> +						    cmd_list.items[i].string);

This is always going to show the right-hand of the equals as the
left-hand on the next line. Would it be simpler to just show the list?
Likewise, the last item in the list is always going to be "where the
loop started". Do we need to say that?

E.g., something like:

  seen = unsorted_string_list_lookup(&cmd_list, *argv[0]);
  if (seen) {
          for (i = 0; i < cmd_list.nr; i++) {
		struct string_list *item = cmd_list.items[i];

		strbuf_addf(&sb, "  %s", item->string);
		if (item == seen)
			strbuf_add(&sb, " <==");
		strbuf_addch(&sb, '\n');
	  }
	  /* We never added this to the list, but we were about to */
	  strbuf_addch("  %s\n", seen->string);
	  die(...);
  }

I guess it's not that far off of yours. Not using words to describe the
loop entry and exit points avoids translation, which avoids notes to
translators, which is most of what makes your patch long. ;)

-Peff

Re: [PATCH v3] Allow aliases that include other aliases

[Ævar Arnfjörð Bjarmason]

On Thu, Sep 06 2018, Jeff King wrote:

> On Thu, Sep 06, 2018 at 04:01:39PM +0200, Ævar Arnfjörð Bjarmason wrote:
>
>> If we don't have some test for these sort of aliasing loops that fails
>> now, we really should add that in a 1/2 and fix it in this patch in 2/2.
>
> Yes, I'd agree that this is worth adding a test (especially if the
> output routines get more complex).
>
>> That makes sense from an implementaion perspective, i.e. we lookup "bar"
>> twice. But let's do better. If I have aliase like:
>>
>>     a = b
>>     b = c
>>     c = d
>>     d = e
>>     e = c
>>
>> It should be telling me that my "e" expansion looped back to the "c = d"
>> expansion. Here's a patch to implement that, feel free to either squash
>> it in with my Signed-Off-By, or tacked onto a v4 version of this,
>> whichever you think makes sense:
>
> I don't have a strong opinion on whether this is worth it, but I think
> your implementation could be a little simpler:
>
>> diff --git a/git.c b/git.c
>> index 64f5fbd572..38f1033e52 100644
>> --- a/git.c
>> +++ b/git.c
>> @@ -692,8 +692,64 @@ static int run_argv(int *argcp, const char ***argv)
>>  		/* .. then try the external ones */
>>  		execv_dashed_external(*argv);
>>
>> -		if (string_list_has_string(&cmd_list, *argv[0]))
>> -			die(_("loop alias: %s is called twice"), *argv[0]);
>> +		if (string_list_has_string(&cmd_list, *argv[0])) {
>> +			struct strbuf sb = STRBUF_INIT;
>> +			int i, seen_at_idx = -1;
>> +
>> +			/*
>> +			 * Find the re-entry point for the alias
>> +			 * loop. TODO: There really should be a
>> +			 * "return the index of the first matching"
>> +			 * helper in string-list.c.
>> +			 */
>> +			for (i = 0; i < cmd_list.nr; i++) {
>> +				if (!strcmp(*argv[0], cmd_list.items[i].string))
>> +					seen_at_idx = i;
>> +			}
>> +			assert(seen_at_idx != -1);
>
> The string-list code doesn't generally deal in indices. You can use
> string_list_find_insert_index(), but its return value is a little funky
> for the existing case. You can also just do:
>
>   struct string_list_item *seen;
>   ...
>   seen = string_list_lookup(&cmd_list, *argv[0]);
>   if (seen) {
> 	/* we have a loop */
> 	int idx = seen - cmd_list.items;
>
> That's a little intimate with the string-list implementation as an array
> of string_list, but it's already pretty standard to walk over and
> dereference that list (including in your patch). But also see below.
>
> Side note: there's actually a bigger problem with the original patch:
> the string list is unsorted (because it uses string_list_append(), and
> which is why your linear walk works here). But string_list_has_string()
> assumes it is sorted.  So I think we'd actually want to use
> unsorted_string_list_has_string() or unsorted_string_list_lookup().
>
>> +			for (i = 1; i < cmd_list.nr; i++) {
>> +				if (i - 1 == seen_at_idx)
>> +					/*
>> +					 * TRANSLATORS: This is a the
>> +					 * re-enttry point in the list
>> +					 * printed out by the "alias
>> +					 * loop" message below.
>> +					 */
>> +					strbuf_addf(&sb, _("    %d. %s = %s <== The re-entry point in the loop\n"),
>> +						    i,
>> +						    cmd_list.items[i - 1].string,
>> +						    cmd_list.items[i].string);
>
> This is always going to show the right-hand of the equals as the
> left-hand on the next line. Would it be simpler to just show the list?
> Likewise, the last item in the list is always going to be "where the
> loop started". Do we need to say that?

Yeah maybe that's overzealous. I figured in the spirit of clang & GCC
compiler messages these days there's no such thing as too dumbed down :)

> E.g., something like:
>
>   seen = unsorted_string_list_lookup(&cmd_list, *argv[0]);
>   if (seen) {
>           for (i = 0; i < cmd_list.nr; i++) {
> 		struct string_list *item = cmd_list.items[i];
>
> 		strbuf_addf(&sb, "  %s", item->string);
> 		if (item == seen)
> 			strbuf_add(&sb, " <==");
> 		strbuf_addch(&sb, '\n');
> 	  }
> 	  /* We never added this to the list, but we were about to */
> 	  strbuf_addch("  %s\n", seen->string);
> 	  die(...);
>   }
>
> I guess it's not that far off of yours. Not using words to describe the
> loop entry and exit points avoids translation, which avoids notes to
> translators, which is most of what makes your patch long. ;)

This still needs translation for RTL languages. I.e. they'd want to
print out the equivalent of "%s " followed by "==> %s ". We happen to
(unfortunately) not carry such a language yet, but it's worth
future-proofing output as we add it in case we get one.

Re: [PATCH v3] Allow aliases that include other aliases

[Jeff King]

On Thu, Sep 06, 2018 at 05:10:04PM +0200, Ævar Arnfjörð Bjarmason wrote:

> >   seen = unsorted_string_list_lookup(&cmd_list, *argv[0]);
> >   if (seen) {
> >           for (i = 0; i < cmd_list.nr; i++) {
> > 		struct string_list *item = cmd_list.items[i];
> >
> > 		strbuf_addf(&sb, "  %s", item->string);
> > 		if (item == seen)
> > 			strbuf_add(&sb, " <==");
> > 		strbuf_addch(&sb, '\n');
> > 	  }
> > 	  /* We never added this to the list, but we were about to */
> > 	  strbuf_addch("  %s\n", seen->string);
> > 	  die(...);
> >   }
> >
> > I guess it's not that far off of yours. Not using words to describe the
> > loop entry and exit points avoids translation, which avoids notes to
> > translators, which is most of what makes your patch long. ;)
> 
> This still needs translation for RTL languages. I.e. they'd want to
> print out the equivalent of "%s " followed by "==> %s ". We happen to
> (unfortunately) not carry such a language yet, but it's worth
> future-proofing output as we add it in case we get one.

I'd have thought even in an RTL language that something like an "I'm
pointing to this" sign wouldn't matter (i.e., an LTR language person,
I'd be fine with either "==> %s" or "%s <=="). But obviously I have no
experience in the matter, so I'd defer to people who read RTL (or at
least have handled i18n for it before).

-Peff

Re: [PATCH v3] Allow aliases that include other aliases

[Tim Schumacher]

On 06.09.18 16:57, Jeff King wrote:
> On Thu, Sep 06, 2018 at 04:01:39PM +0200, Ævar Arnfjörð Bjarmason wrote:
> 
>> If we don't have some test for these sort of aliasing loops that fails
>> now, we really should add that in a 1/2 and fix it in this patch in 2/2.
> 
> Yes, I'd agree that this is worth adding a test (especially if the
> output routines get more complex).

I'll try to come up with a few tests (or one at this point, as we only have
a solution for internal aliases so far) and put them as 1/2. However, I don't know
what file I should put those tests into. t0001-init and t1300-config both seem
to test aliases, but I'm unsure if the new tests should go into one of those
files or a completely new one that is dedicated to aliases.

> 
>> That makes sense from an implementaion perspective, i.e. we lookup "bar"
>> twice. But let's do better. If I have aliase like:
>>
>>      a = b
>>      b = c
>>      c = d
>>      d = e
>>      e = c
>>
>> It should be telling me that my "e" expansion looped back to the "c = d"
>> expansion. Here's a patch to implement that, feel free to either squash
>> it in with my Signed-Off-By, or tacked onto a v4 version of this,
>> whichever you think makes sense:
> 
> I don't have a strong opinion on whether this is worth it, but I think
> your implementation could be a little simpler:
> 
>> diff --git a/git.c b/git.c
>> index 64f5fbd572..38f1033e52 100644
>> --- a/git.c
>> +++ b/git.c
>> @@ -692,8 +692,64 @@ static int run_argv(int *argcp, const char ***argv)
>>   		/* .. then try the external ones */
>>   		execv_dashed_external(*argv);
>>
>> -		if (string_list_has_string(&cmd_list, *argv[0]))
>> -			die(_("loop alias: %s is called twice"), *argv[0]);
>> +		if (string_list_has_string(&cmd_list, *argv[0])) {
>> +			struct strbuf sb = STRBUF_INIT;
>> +			int i, seen_at_idx = -1;
>> +
>> +			/*
>> +			 * Find the re-entry point for the alias
>> +			 * loop. TODO: There really should be a
>> +			 * "return the index of the first matching"
>> +			 * helper in string-list.c.
>> +			 */
>> +			for (i = 0; i < cmd_list.nr; i++) {
>> +				if (!strcmp(*argv[0], cmd_list.items[i].string))
>> +					seen_at_idx = i;
>> +			}
>> +			assert(seen_at_idx != -1);
> 
> The string-list code doesn't generally deal in indices. You can use
> string_list_find_insert_index(), but its return value is a little funky
> for the existing case. You can also just do:
> 
>    struct string_list_item *seen;
>    ...
>    seen = string_list_lookup(&cmd_list, *argv[0]);
>    if (seen) {
> 	/* we have a loop */
> 	int idx = seen - cmd_list.items;
> 
> That's a little intimate with the string-list implementation as an array
> of string_list, but it's already pretty standard to walk over and
> dereference that list (including in your patch). But also see below.
> 
> Side note: there's actually a bigger problem with the original patch:
> the string list is unsorted (because it uses string_list_append(), and
> which is why your linear walk works here). But string_list_has_string()
> assumes it is sorted.  So I think we'd actually want to use
> unsorted_string_list_has_string() or unsorted_string_list_lookup().

I'll update v4 to use use unsorted_string_list_has_string().
> 
>> +			for (i = 1; i < cmd_list.nr; i++) {
>> +				if (i - 1 == seen_at_idx)
>> +					/*
>> +					 * TRANSLATORS: This is a the
>> +					 * re-enttry point in the list
>> +					 * printed out by the "alias
>> +					 * loop" message below.
>> +					 */
>> +					strbuf_addf(&sb, _("    %d. %s = %s <== The re-entry point in the loop\n"),
>> +						    i,
>> +						    cmd_list.items[i - 1].string,
>> +						    cmd_list.items[i].string);
> 
> This is always going to show the right-hand of the equals as the
> left-hand on the next line. Would it be simpler to just show the list?
> Likewise, the last item in the list is always going to be "where the
> loop started". Do we need to say that?
> 
> E.g., something like:
> 
>    seen = unsorted_string_list_lookup(&cmd_list, *argv[0]);
>    if (seen) {
>            for (i = 0; i < cmd_list.nr; i++) {
> 		struct string_list *item = cmd_list.items[i];
> 
> 		strbuf_addf(&sb, "  %s", item->string);
> 		if (item == seen)
> 			strbuf_add(&sb, " <==");
> 		strbuf_addch(&sb, '\n');
> 	  }
> 	  /* We never added this to the list, but we were about to */
> 	  strbuf_addch("  %s\n", seen->string);
> 	  die(...);
>    }
> 
> I guess it's not that far off of yours. Not using words to describe the
> loop entry and exit points avoids translation, which avoids notes to
> translators, which is most of what makes your patch long. ;)
> 
> -Peff
> 

I'll tinker around with both code snippets, we'll see which one is
more convenient for the user.

Thanks to all of you for the input!

Tim

Re: [PATCH v3] Allow aliases that include other aliases

[Jeff King]

On Thu, Sep 06, 2018 at 09:05:50PM +0200, Tim Schumacher wrote:

> On 06.09.18 16:57, Jeff King wrote:
> > On Thu, Sep 06, 2018 at 04:01:39PM +0200, Ævar Arnfjörð Bjarmason wrote:
> > 
> > > If we don't have some test for these sort of aliasing loops that fails
> > > now, we really should add that in a 1/2 and fix it in this patch in 2/2.
> > 
> > Yes, I'd agree that this is worth adding a test (especially if the
> > output routines get more complex).
> 
> I'll try to come up with a few tests (or one at this point, as we only have
> a solution for internal aliases so far) and put them as 1/2. However, I don't know
> what file I should put those tests into. t0001-init and t1300-config both seem
> to test aliases, but I'm unsure if the new tests should go into one of those
> files or a completely new one that is dedicated to aliases.

Yeah, I don't think there's a good place right now. It probably make
sense to start a new one (t0014-alias, maybe? This seems like a basic
functionality that should come early in the suite).

-Peff

Re: [PATCH v3] Allow aliases that include other aliases

[Jeff King]

On Thu, Sep 06, 2018 at 12:16:58PM +0200, Tim Schumacher wrote:

> @@ -691,17 +692,23 @@ static int run_argv(int *argcp, const char ***argv)
>  		/* .. then try the external ones */
>  		execv_dashed_external(*argv);
>  
> -		/* It could be an alias -- this works around the insanity
> +		if (string_list_has_string(&cmd_list, *argv[0]))
> +			die(_("loop alias: %s is called twice"), *argv[0]);

I pointed this out in my response to Ævar, but I want to make sure it
gets seen. This call assumes the list is sorted, but...

> +		string_list_append(&cmd_list, *argv[0]);

This will create an unsorted list. You'd have to use
string_list_insert() here for a sorted list, or
unsorted_string_list_has_string() in the earlier call.

It's unfortunate that string_list makes this so easy to get wrong.

> +
> +		/*
> +		 * It could be an alias -- this works around the insanity
>  		 * of overriding "git log" with "git show" by having
>  		 * alias.log = show
>  		 */
> -		if (done_alias)
> -			break;
>  		if (!handle_alias(argcp, argv))
>  			break;
> -		done_alias = 1;
> +		done_alias++;

I don't think anybody cares about done_alias being an accurate count.
Should we just leave this as-is?

-Peff

Re: [PATCH v3] Allow aliases that include other aliases

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Thu, Sep 06, 2018 at 12:16:58PM +0200, Tim Schumacher wrote:
>
>> @@ -691,17 +692,23 @@ static int run_argv(int *argcp, const char ***argv)
>>  		/* .. then try the external ones */
>>  		execv_dashed_external(*argv);
>>  
>> -		/* It could be an alias -- this works around the insanity
>> +		if (string_list_has_string(&cmd_list, *argv[0]))
>> +			die(_("loop alias: %s is called twice"), *argv[0]);
>
> I pointed this out in my response to Ævar, but I want to make sure it
> gets seen. This call assumes the list is sorted, but...
>
>> +		string_list_append(&cmd_list, *argv[0]);
>
> This will create an unsorted list. You'd have to use
> string_list_insert() here for a sorted list, or
> unsorted_string_list_has_string() in the earlier call.

Correct.

Also, normal users who have never seen this loop that implements
alias expansion would not have a clue when they see "called twice".

I actually think the caller should also pass cmd to run_argv() and
then we should use it (and not argv[]) in this die() message.  When
the original command was foo that is aliased to bar, which in turn
is aliased to baz, which in turn is aliased to bar, especially that
"git foo" invocation was in a random script written six weeks ago by
the user, it would be a lot more helpful to see 

    "alias loop detected: expansion of 'git foo' does not terminate"

than

    "loop alias: bar is called twice".

given that 'bar' is not something the user called, or written in the
script she wrote six weeks ago.

> It's unfortunate that string_list makes this so easy to get wrong.
>
>> +
>> +		/*
>> +		 * It could be an alias -- this works around the insanity
>>  		 * of overriding "git log" with "git show" by having
>>  		 * alias.log = show
>>  		 */
>> -		if (done_alias)
>> -			break;
>>  		if (!handle_alias(argcp, argv))
>>  			break;
>> -		done_alias = 1;
>> +		done_alias++;
>
> I don't think anybody cares about done_alias being an accurate count.
> Should we just leave this as-is?

Good point.  The only caller treats it as a bool (i.e. "should the
failure be reported as failure to expand an alias cmd which resulted
in (updated) argv[0] that is not a git command?").

Re: [PATCH v3] Allow aliases that include other aliases

[Jeff King]

On Thu, Sep 06, 2018 at 11:40:14AM -0700, Junio C Hamano wrote:

> Also, normal users who have never seen this loop that implements
> alias expansion would not have a clue when they see "called twice".
> 
> I actually think the caller should also pass cmd to run_argv() and
> then we should use it (and not argv[]) in this die() message.  When
> the original command was foo that is aliased to bar, which in turn
> is aliased to baz, which in turn is aliased to bar, especially that
> "git foo" invocation was in a random script written six weeks ago by
> the user, it would be a lot more helpful to see 
> 
>     "alias loop detected: expansion of 'git foo' does not terminate"
> 
> than
> 
>     "loop alias: bar is called twice".
> 
> given that 'bar' is not something the user called, or written in the
> script she wrote six weeks ago.

Good point. I think Ævar's "print the whole list" solves that, because
items[0] is that first element. But I agree the fundamental message
should be "loop in 'git foo'".

-Peff

Re: [PATCH v3] Allow aliases that include other aliases

[Tim Schumacher]

On 06.09.18 20:40, Junio C Hamano wrote:
> Jeff King <peff@peff.net> writes:
> 
>> On Thu, Sep 06, 2018 at 12:16:58PM +0200, Tim Schumacher wrote:
>>
>>> @@ -691,17 +692,23 @@ static int run_argv(int *argcp, const char ***argv)
>>>   		/* .. then try the external ones */
>>>   		execv_dashed_external(*argv);
>>>   
>>> -		/* It could be an alias -- this works around the insanity
>>> +		if (string_list_has_string(&cmd_list, *argv[0]))
>>> +			die(_("loop alias: %s is called twice"), *argv[0]);
>>
>> I pointed this out in my response to Ævar, but I want to make sure it
>> gets seen. This call assumes the list is sorted, but...
>>
>>> +		string_list_append(&cmd_list, *argv[0]);
>>
>> This will create an unsorted list. You'd have to use
>> string_list_insert() here for a sorted list, or
>> unsorted_string_list_has_string() in the earlier call.
> 
> Correct.
> 
> Also, normal users who have never seen this loop that implements
> alias expansion would not have a clue when they see "called twice".
> 
> I actually think the caller should also pass cmd to run_argv() and
> then we should use it (and not argv[]) in this die() message.

Could we just save the first element of the original argv for that
purpose? Or alternatively, use the first stored element in the
command list?

> When
> the original command was foo that is aliased to bar, which in turn
> is aliased to baz, which in turn is aliased to bar, especially that
> "git foo" invocation was in a random script written six weeks ago by
> the user, it would be a lot more helpful to see
> 
>      "alias loop detected: expansion of 'git foo' does not terminate"
> 
> than
> 
>      "loop alias: bar is called twice".
> 
> given that 'bar' is not something the user called, or written in the
> script she wrote six weeks ago.

Indeed, printing the command that the user called is a better message
than the command that is the entry-point of the loop. I'll change it
in v4.

> 
>> It's unfortunate that string_list makes this so easy to get wrong.
>>
>>> +
>>> +		/*
>>> +		 * It could be an alias -- this works around the insanity
>>>   		 * of overriding "git log" with "git show" by having
>>>   		 * alias.log = show
>>>   		 */
>>> -		if (done_alias)
>>> -			break;
>>>   		if (!handle_alias(argcp, argv))
>>>   			break;
>>> -		done_alias = 1;
>>> +		done_alias++;
>>
>> I don't think anybody cares about done_alias being an accurate count.
>> Should we just leave this as-is?
> 
> Good point.  The only caller treats it as a bool (i.e. "should the
> failure be reported as failure to expand an alias cmd which resulted
> in (updated) argv[0] that is not a git command?").
> 

As the string-list has its own counter, I guess the done_alias variable
can be reverted to a simple 0/1 value.

[RFC PATCH v4 1/3] Add support for nested aliases

[Tim Schumacher]

Aliases can only contain non-alias git commands and their
arguments, not other user-defined aliases. Resolving further
(nested) aliases is prevented by breaking the loop after the
first alias was processed. Git then fails with a command-not-found
error.

Allow resolving nested aliases by not breaking the loop in
run_argv() after the first alias was processed. Instead, continue
the loop until `handle_alias()` fails, which means that there are
no further aliases that can be processed. Prevent looping aliases
by storing substituted commands in `cmd_list` and checking if
a command has been substituted previously.

While we're at it, fix a styling issue just below the added code.

Signed-off-by: Tim Schumacher <timschumi@gmx.de>
---
Changes since v3:
 - Print the command that the user entered instead of the command
   which caused the loop (and a nicer, more explanatory error message)
 - Use unsorted_string_list_has_string() instead of the sorted version
 - Fix a code style issue just below the modified code
 - done_alias is a simple boolean again (instead of a counter)

 git.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/git.c b/git.c
index c27c38738..15727c17f 100644
--- a/git.c
+++ b/git.c
@@ -674,6 +674,7 @@ static void execv_dashed_external(const char **argv)
 static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
+	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
 
 	while (1) {
 		/*
@@ -691,17 +692,25 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);
 
-		/* It could be an alias -- this works around the insanity
+		if (unsorted_string_list_has_string(&cmd_list, *argv[0])) {
+			die(_("alias loop detected: expansion of '%s' does"
+			      " not terminate"), cmd_list.items[0].string);
+		}
+
+		string_list_append(&cmd_list, *argv[0]);
+
+		/*
+		 * It could be an alias -- this works around the insanity
 		 * of overriding "git log" with "git show" by having
 		 * alias.log = show
 		 */
-		if (done_alias)
-			break;
 		if (!handle_alias(argcp, argv))
 			break;
 		done_alias = 1;
 	}
 
+	string_list_clear(&cmd_list, 0);
+
 	return done_alias;
 }
 
-- 
2.19.0.rc2.1.g4c98b8d69.dirty

[RFC PATCH v4 2/3] Show the call history when an alias is looping

[Tim Schumacher]

Just printing the command that the user entered is not particularly
helpful when trying to find the alias that causes the loop.

Print the history of substituted commands to help the user find the
offending alias. Mark the entrypoint of the loop with "<==" and the
last command (which looped back to the entrypoint) with "==>".

Signed-off-by: Tim Schumacher <timschumi@gmx.de>
---

I now went with Peff's suggested code and I added in an arrow that points
away from the last command (which caused the loop). A "full" arrow (i.e.
starts at the last command, goes upwards and ends at the entrypoint) would
be more obvious/better, but adding much more code just for having a
vertical line wasn't worth it for me.

 git.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/git.c b/git.c
index 15727c17f..a20eb4fa1 100644
--- a/git.c
+++ b/git.c
@@ -675,6 +675,7 @@ static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
 	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
+	struct string_list_item *seen;
 
 	while (1) {
 		/*
@@ -692,9 +693,21 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);
 
-		if (unsorted_string_list_has_string(&cmd_list, *argv[0])) {
+		seen = unsorted_string_list_lookup(&cmd_list, *argv[0]);
+		if (seen) {
+			int i;
+			struct strbuf sb = STRBUF_INIT;
+			for (i = 0; i < cmd_list.nr; i++) {
+				struct string_list_item *item = &cmd_list.items[i];
+
+				strbuf_addf(&sb, "\n  %s", item->string);
+				if (item == seen)
+					strbuf_addstr(&sb, " <==");
+				else if (i == cmd_list.nr - 1)
+					strbuf_addstr(&sb, " ==>");
+			}
 			die(_("alias loop detected: expansion of '%s' does"
-			      " not terminate"), cmd_list.items[0].string);
+			      " not terminate:%s"), cmd_list.items[0].string, sb.buf);
 		}
 
 		string_list_append(&cmd_list, *argv[0]);
-- 
2.19.0.rc2.1.g4c98b8d69.dirty

Re: [RFC PATCH v4 2/3] Show the call history when an alias is looping

[Duy Nguyen]

On Sat, Sep 8, 2018 at 12:44 AM Tim Schumacher <timschumi@gmx.de> wrote:
>
> Just printing the command that the user entered is not particularly
> helpful when trying to find the alias that causes the loop.
>
> Print the history of substituted commands to help the user find the
> offending alias. Mark the entrypoint of the loop with "<==" and the
> last command (which looped back to the entrypoint) with "==>".

An even simpler way to give this information is simply suggest the
user tries again with GIT_TRACE=1. All alias expansion is shown there
and we teach the user about GIT_TRACE. But your approach is probably
more user friendly.
-- 
Duy

Re: [RFC PATCH v4 2/3] Show the call history when an alias is looping

[Jeff King]

On Sat, Sep 08, 2018 at 03:34:34PM +0200, Duy Nguyen wrote:

> On Sat, Sep 8, 2018 at 12:44 AM Tim Schumacher <timschumi@gmx.de> wrote:
> >
> > Just printing the command that the user entered is not particularly
> > helpful when trying to find the alias that causes the loop.
> >
> > Print the history of substituted commands to help the user find the
> > offending alias. Mark the entrypoint of the loop with "<==" and the
> > last command (which looped back to the entrypoint) with "==>".
> 
> An even simpler way to give this information is simply suggest the
> user tries again with GIT_TRACE=1. All alias expansion is shown there
> and we teach the user about GIT_TRACE. But your approach is probably
> more user friendly.

Good point. I'm OK with the amount of code here for the nicer message
(but would be happy either way).

If we were going to track cross-process loops like Ævar suggested, I
think I'd rather go with a simple counter and just ask the user to run
with GIT_TRACE when it exceeds some maximum sanity value. For two
reasons:

  1. Passing a counter through the environment is way simpler than
     an arbitrarily-sized list.

  2. When you get into multiple processes, there's potentially more
     going on than just Git commands. You might have a git command which
     runs a hook which runs a third party script which runs a git
     command, which runs a hook, and so on. That full dump is going to
     be more useful.

-Peff

[RFC PATCH v4 3/3] t0014: Introduce alias testing suite

[Tim Schumacher]

Introduce a testing suite that is dedicated to aliases.
For now, check only if nested aliases work and if looping
aliases are detected successfully.

The looping aliases check for mixed execution is there but
expected to fail because there is no check in place yet.

Signed-off-by: Tim Schumacher <timschumi@gmx.de>
---

Those are the tests that I've come up with. It consists of tests
for nested aliases and looping aliases, both with internal calls
and external calls.

Unfortunately I don't have a fix for the last one yet, so I
marked it as expect_failure. The problem is that the test suite
is waiting a full minute until it aborts the running command
(which I guess should not take that long, as it blocks the whole
test suite for that span of time).

Should I try to decrease the timeout or should I remove that
test completely until I manage to get external calls fixed?

As a last thing, is there any better way to use single quotes
than to write '"'"'? It isn't that bad, but it is hard to read,
especially for bash newcomers.

 t/t0014-alias.sh | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
 create mode 100755 t/t0014-alias.sh

diff --git a/t/t0014-alias.sh b/t/t0014-alias.sh
new file mode 100755
index 000000000..6c1e34694
--- /dev/null
+++ b/t/t0014-alias.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+test_description='git command aliasing'
+
+. ./test-lib.sh
+
+test_expect_success 'setup environment' '
+	git init
+'
+
+test_expect_success 'nested aliases - internal execution' '
+	git config alias.nested-internal-1 nested-internal-2 &&
+	git config alias.nested-internal-2 status
+'
+
+test_expect_success 'nested aliases - mixed execution' '
+	git config alias.nested-external-1 "!git nested-external-2" &&
+	git config alias.nested-external-2 status
+'
+
+test_expect_success 'looping aliases - internal execution' '
+	git config alias.loop-internal-1 loop-internal-2 &&
+	git config alias.loop-internal-2 loop-internal-3 &&
+	git config alias.loop-internal-3 loop-internal-2 &&
+	test_must_fail git loop-internal-1 2>output &&
+	grep -q "fatal: alias loop detected: expansion of '"'"'loop-internal-1'"'"' does not terminate" output &&
+	rm output
+'
+
+test_expect_failure 'looping aliases - mixed execution' '
+	git config alias.loop-mixed-1 loop-mixed-2 &&
+	git config alias.loop-mixed-2 "!git loop-mixed-1" &&
+	test_must_fail git loop-mixed-1 2>output &&
+	grep -q "fatal: alias loop detected: expansion of '"'"'loop-mixed-1'"'"' does not terminate" output &&
+	rm output
+'
+
+test_done
-- 
2.19.0.rc2.1.g4c98b8d69.dirty

Re: [RFC PATCH v4 3/3] t0014: Introduce alias testing suite

[Eric Sunshine]

On Fri, Sep 7, 2018 at 6:44 PM Tim Schumacher <timschumi@gmx.de> wrote:
> Introduce a testing suite that is dedicated to aliases.
> For now, check only if nested aliases work and if looping
> aliases are detected successfully.
>
> The looping aliases check for mixed execution is there but
> expected to fail because there is no check in place yet.
>
> Signed-off-by: Tim Schumacher <timschumi@gmx.de>
> ---
> Unfortunately I don't have a fix for the last one yet, so I
> marked it as expect_failure. The problem is that the test suite
> is waiting a full minute until it aborts the running command
> (which I guess should not take that long, as it blocks the whole
> test suite for that span of time).
>
> Should I try to decrease the timeout or should I remove that
> test completely until I manage to get external calls fixed?

Perhaps just comment out that test for now and add a comment above it
explaining why it's commented out.

> As a last thing, is there any better way to use single quotes
> than to write '"'"'? It isn't that bad, but it is hard to read,
> especially for bash newcomers.

You should backslash-escape the quotes ("foo \'bar\' baz"), however,
in this case, it would make sense to use regex's with 'grep' to check
that you got the expected error message rather than reproducing the
message literally here in the script.

More below.

> diff --git a/t/t0014-alias.sh b/t/t0014-alias.sh
> @@ -0,0 +1,38 @@
> +#!/bin/sh
> +
> +test_description='git command aliasing'
> +
> +. ./test-lib.sh
> +
> +test_expect_success 'setup environment' '
> +       git init
> +'

"git init" is invoked automatically by the test framework, so no need
for this test. You can drop it.

> +test_expect_success 'nested aliases - internal execution' '
> +       git config alias.nested-internal-1 nested-internal-2 &&
> +       git config alias.nested-internal-2 status
> +'

This isn't actually testing anything, is it? It's setting up the
aliases but never actually invoking them. I would have expected the
next line to actually run a command ("git nested-internal-1") and the
line after that to check that you got the expected output (whatever
"git status" would emit). Output from "git status" isn't necessarily
the easiest to test, though, so perhaps pick a different Git command
for testing (something for which the result can be very easily checked
-- maybe "git rm" or such).

> +test_expect_success 'nested aliases - mixed execution' '
> +       git config alias.nested-external-1 "!git nested-external-2" &&
> +       git config alias.nested-external-2 status
> +'

Same observation.

> +test_expect_success 'looping aliases - internal execution' '
> +       git config alias.loop-internal-1 loop-internal-2 &&
> +       git config alias.loop-internal-2 loop-internal-3 &&
> +       git config alias.loop-internal-3 loop-internal-2 &&
> +       test_must_fail git loop-internal-1 2>output &&
> +       grep -q "fatal: alias loop detected: expansion of '"'"'loop-internal-1'"'"' does not terminate" output &&

Don't bother using -q with 'grep'. Output is hidden already by the
test framework in normal mode, and not hidden when running in verbose
mode. And, the output of 'grep' might be helpful when debugging the
test if something goes wrong.

As noted above, you can use regex to match the expected error rather
than exactly duplicating the text of the message.

Finally, use 'test_i18ngrep' instead of 'grep' in order to play nice
with localization.

> +       rm output

Tests don't normally bother cleaning up their output files like this
since such output can be helpful when debugging the test if something
goes wrong. (You'd want to use test_when_finished to cleanup anyhow,
but you don't need it in this case.)

> +'

Re: [RFC PATCH v4 3/3] t0014: Introduce alias testing suite

[Tim Schumacher]

On 08.09.18 01:38, Eric Sunshine wrote:
> On Fri, Sep 7, 2018 at 6:44 PM Tim Schumacher <timschumi@gmx.de> wrote:
>> Introduce a testing suite that is dedicated to aliases.
>> For now, check only if nested aliases work and if looping
>> aliases are detected successfully.
>>
>> The looping aliases check for mixed execution is there but
>> expected to fail because there is no check in place yet.
>>
>> Signed-off-by: Tim Schumacher <timschumi@gmx.de>
>> ---
>> Unfortunately I don't have a fix for the last one yet, so I
>> marked it as expect_failure. The problem is that the test suite
>> is waiting a full minute until it aborts the running command
>> (which I guess should not take that long, as it blocks the whole
>> test suite for that span of time).
>>
>> Should I try to decrease the timeout or should I remove that
>> test completely until I manage to get external calls fixed?
> 
> Perhaps just comment out that test for now and add a comment above it
> explaining why it's commented out.

That will probably be the easiest thing to do. I commented it out for
now, added a short information about that to the code itself and a longer
explanation to the commit message.

> 
>> As a last thing, is there any better way to use single quotes
>> than to write '"'"'? It isn't that bad, but it is hard to read,
>> especially for bash newcomers.
> 
> You should backslash-escape the quotes ("foo \'bar\' baz"), however,
> in this case, it would make sense to use regex's with 'grep' to check
> that you got the expected error message rather than reproducing the
> message literally here in the script.

Backslash-escaping didn't work, that resulted in some parsing error.
I'm using i18ngrep now to search for the part of a message, which
eliminates the need for quotes completely.

> 
> More below.
> 
>> diff --git a/t/t0014-alias.sh b/t/t0014-alias.sh
>> @@ -0,0 +1,38 @@
>> +#!/bin/sh
>> +
>> +test_description='git command aliasing'
>> +
>> +. ./test-lib.sh
>> +
>> +test_expect_success 'setup environment' '
>> +       git init
>> +'
> 
> "git init" is invoked automatically by the test framework, so no need
> for this test. You can drop it.
> 
>> +test_expect_success 'nested aliases - internal execution' '
>> +       git config alias.nested-internal-1 nested-internal-2 &&
>> +       git config alias.nested-internal-2 status
>> +'
> 
> This isn't actually testing anything, is it? It's setting up the
> aliases but never actually invoking them. I would have expected the
> next line to actually run a command ("git nested-internal-1") and the
> line after that to check that you got the expected output (whatever
> "git status" would emit). Output from "git status" isn't necessarily
> the easiest to test, though, so perhaps pick a different Git command
> for testing (something for which the result can be very easily checked
> -- maybe "git rm" or such).

Whoops, I didn't know when that went missing. I added it into a new version
of this patch.

Also, I decided to keep `git status`, because it seemed to be the only
command which doesn't need any files to produce some checkable output.
Checking the "On branch" message should be enough to confirm that the
command works as intended.

> 
>> +test_expect_success 'nested aliases - mixed execution' '
>> +       git config alias.nested-external-1 "!git nested-external-2" &&
>> +       git config alias.nested-external-2 status
>> +'
> 
> Same observation.
> 
>> +test_expect_success 'looping aliases - internal execution' '
>> +       git config alias.loop-internal-1 loop-internal-2 &&
>> +       git config alias.loop-internal-2 loop-internal-3 &&
>> +       git config alias.loop-internal-3 loop-internal-2 &&
>> +       test_must_fail git loop-internal-1 2>output &&
>> +       grep -q "fatal: alias loop detected: expansion of '"'"'loop-internal-1'"'"' does not terminate" output &&
> 
> Don't bother using -q with 'grep'. Output is hidden already by the
> test framework in normal mode, and not hidden when running in verbose
> mode. And, the output of 'grep' might be helpful when debugging the
> test if something goes wrong.
> 
> As noted above, you can use regex to match the expected error rather
> than exactly duplicating the text of the message.
> 
> Finally, use 'test_i18ngrep' instead of 'grep' in order to play nice
> with localization.
> 
>> +       rm output
> 
> Tests don't normally bother cleaning up their output files like this
> since such output can be helpful when debugging the test if something
> goes wrong. (You'd want to use test_when_finished to cleanup anyhow,
> but you don't need it in this case.)

I incorporated both of these suggestions.

> 
>> +'
> 

This is the first multi-patch series that I submitted, so I'm unsure if I
should send the updated patch only or if I should send the complete series
again as v5. Any pointers to what the correct procedure for this case is would
be appreciated.

Thanks for looking at this.

Tim

Re: [RFC PATCH v4 3/3] t0014: Introduce alias testing suite

[Eric Sunshine]

On Fri, Sep 14, 2018 at 7:12 PM Tim Schumacher <timschumi@gmx.de> wrote:
> This is the first multi-patch series that I submitted, so I'm unsure if I
> should send the updated patch only or if I should send the complete series
> again as v5. Any pointers to what the correct procedure for this case is would
> be appreciated.

Re-send the entire series as v5. That makes it easier on reviewers
(who don't need to go searching through the mailing list archive to
get a full picture) and reduces Junio's workload since it's usually
easier for him to re-queue a series wholesale than having to
slice-and-dice some replacement patches into what was already queued.

Re: [RFC PATCH v4 1/3] Add support for nested aliases

[Duy Nguyen]

On Sat, Sep 8, 2018 at 12:44 AM Tim Schumacher <timschumi@gmx.de> wrote:
> +               /*
> +                * It could be an alias -- this works around the insanity
>                  * of overriding "git log" with "git show" by having
>                  * alias.log = show
>                  */

I think this comment block is about the next two lines you just
deleted. So delete it to instead of fixing style.

> -               if (done_alias)
> -                       break;
>                 if (!handle_alias(argcp, argv))
>                         break;
>                 done_alias = 1;
>         }
-- 
Duy

Re: [RFC PATCH v4 1/3] Add support for nested aliases

[Tim Schumacher]

On 08.09.18 15:28, Duy Nguyen wrote:
> On Sat, Sep 8, 2018 at 12:44 AM Tim Schumacher <timschumi@gmx.de> wrote:
>> +               /*
>> +                * It could be an alias -- this works around the insanity
>>                   * of overriding "git log" with "git show" by having
>>                   * alias.log = show
>>                   */
> 
> I think this comment block is about the next two lines you just
> deleted. So delete it to instead of fixing style.

I think that comment is talking about the code that is handing the alias,
so it still would be valid.
The check might have peen placed in between to keep it logically grouped.

> 
>> -               if (done_alias)
>> -                       break;
>>                  if (!handle_alias(argcp, argv))
>>                          break;
>>                  done_alias = 1;
>>          }

Re: [RFC PATCH v4 1/3] Add support for nested aliases

[Junio C Hamano]

Tim Schumacher <timschumi@gmx.de> writes:

> On 08.09.18 15:28, Duy Nguyen wrote:
>> On Sat, Sep 8, 2018 at 12:44 AM Tim Schumacher <timschumi@gmx.de> wrote:
>>> +               /*
>>> +                * It could be an alias -- this works around the insanity
>>>                   * of overriding "git log" with "git show" by having
>>>                   * alias.log = show
>>>                   */
>>
>> I think this comment block is about the next two lines you just
>> deleted. So delete it to instead of fixing style.
>
> I think that comment is talking about the code that is handing the alias,
> so it still would be valid.

"this" in "this works around" refers to the fact that we first check
the builtins and on-GIT_EXEC_PATH commands before trying an alias,
which is an effective way to forbid an alias from taking over
existing command names.  So it is not about a particular code but is
about how the two sections of code are laid out.

It probably will make it clear if we reworded and made it a comment
about the whole while() loop may make sense, i.e.

	/*
	 * Check if av[0] is a command before seeing if it is an
	 * alias to avoid the insanity of overriding ...
	 */
	while (1) {
		...

but that can be done after the dust settles as a clean-up, I would
think.

Re: [RFC PATCH v4 1/3] Add support for nested aliases

[Tim Schumacher]

On 17.09.18 17:37, Junio C Hamano wrote:
> Tim Schumacher <timschumi@gmx.de> writes:
> 
>> On 08.09.18 15:28, Duy Nguyen wrote:
>>> On Sat, Sep 8, 2018 at 12:44 AM Tim Schumacher <timschumi@gmx.de> wrote:
>>>> +               /*
>>>> +                * It could be an alias -- this works around the insanity
>>>>                    * of overriding "git log" with "git show" by having
>>>>                    * alias.log = show
>>>>                    */
>>>
>>> I think this comment block is about the next two lines you just
>>> deleted. So delete it to instead of fixing style.
>>
>> I think that comment is talking about the code that is handing the alias,
>> so it still would be valid.
> 
> "this" in "this works around" refers to the fact that we first check
> the builtins and on-GIT_EXEC_PATH commands before trying an alias,
> which is an effective way to forbid an alias from taking over
> existing command names.  So it is not about a particular code but is
> about how the two sections of code are laid out.
> 
> It probably will make it clear if we reworded and made it a comment
> about the whole while() loop may make sense, i.e.
> 
> 	/*
> 	 * Check if av[0] is a command before seeing if it is an
> 	 * alias to avoid the insanity of overriding ...
> 	 */
> 	while (1) {
> 		...
> 

Imho, the "insanity" part makes the intention of that comment unclear, even if
it is located at the top of the while() loop. Giving an example is nice, but wouldn't
it be better to say something like the following?

	/*
	 * Check if av[0] is a command before seeing if it is an
	 * alias to avoid taking over existing commands
	 */

> but that can be done after the dust settles as a clean-up, I would
> think.
> 

I'll keep the changed comment in my local repository for now and publish it together
with other changes in v6, but I assume there won't be much additional feedback.

Re: [RFC PATCH v4 1/3] Add support for nested aliases

[Junio C Hamano]

Tim Schumacher <timschumi@gmx.de> writes:

> it is located at the top of the while() loop. Giving an example is nice, but wouldn't
> it be better to say something like the following?
>
> 	/*
> 	 * Check if av[0] is a command before seeing if it is an
> 	 * alias to avoid taking over existing commands
> 	 */

If we have more concrete and constructive things to explain why we
choose to forbid it, that may be worth saying, but I agree that it
does not add much value to this comment to declare that an attempt
to take over existing commands is "insane".

Thanks.

[PATCH v5 1/3] Add support for nested aliases

[Tim Schumacher]

Aliases can only contain non-alias git commands and their
arguments, not other user-defined aliases. Resolving further
(nested) aliases is prevented by breaking the loop after the
first alias was processed. Git then fails with a command-not-found
error.

Allow resolving nested aliases by not breaking the loop in
run_argv() after the first alias was processed. Instead, continue
the loop until `handle_alias()` fails, which means that there are
no further aliases that can be processed. Prevent looping aliases
by storing substituted commands in `cmd_list` and checking if
a command has been substituted previously.

While we're at it, fix a styling issue just below the added code.

Signed-off-by: Tim Schumacher <timschumi@gmx.de>
---
Changes since v3:
 - Print the command that the user entered instead of the command
   which caused the loop (and a nicer, more explanatory error message)
 - Use unsorted_string_list_has_string() instead of the sorted version
 - Fix a code style issue just below the modified code
 - done_alias is a simple boolean again (instead of a counter)

Changes since v4: None.

 git.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/git.c b/git.c
index c27c38738..15727c17f 100644
--- a/git.c
+++ b/git.c
@@ -674,6 +674,7 @@ static void execv_dashed_external(const char **argv)
 static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
+	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
 
 	while (1) {
 		/*
@@ -691,17 +692,25 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);
 
-		/* It could be an alias -- this works around the insanity
+		if (unsorted_string_list_has_string(&cmd_list, *argv[0])) {
+			die(_("alias loop detected: expansion of '%s' does"
+			      " not terminate"), cmd_list.items[0].string);
+		}
+
+		string_list_append(&cmd_list, *argv[0]);
+
+		/*
+		 * It could be an alias -- this works around the insanity
 		 * of overriding "git log" with "git show" by having
 		 * alias.log = show
 		 */
-		if (done_alias)
-			break;
 		if (!handle_alias(argcp, argv))
 			break;
 		done_alias = 1;
 	}
 
+	string_list_clear(&cmd_list, 0);
+
 	return done_alias;
 }
 
-- 
2.19.0.rc2.1.g4c98b8d69.dirty

[PATCH v5 2/3] Show the call history when an alias is looping

[Tim Schumacher]

Just printing the command that the user entered is not particularly
helpful when trying to find the alias that causes the loop.

Print the history of substituted commands to help the user find the
offending alias. Mark the entrypoint of the loop with "<==" and the
last command (which looped back to the entrypoint) with "==>".

Signed-off-by: Tim Schumacher <timschumi@gmx.de>
---
No changes since v4.

 git.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/git.c b/git.c
index 15727c17f..a20eb4fa1 100644
--- a/git.c
+++ b/git.c
@@ -675,6 +675,7 @@ static int run_argv(int *argcp, const char ***argv)
 {
 	int done_alias = 0;
 	struct string_list cmd_list = STRING_LIST_INIT_NODUP;
+	struct string_list_item *seen;
 
 	while (1) {
 		/*
@@ -692,9 +693,21 @@ static int run_argv(int *argcp, const char ***argv)
 		/* .. then try the external ones */
 		execv_dashed_external(*argv);
 
-		if (unsorted_string_list_has_string(&cmd_list, *argv[0])) {
+		seen = unsorted_string_list_lookup(&cmd_list, *argv[0]);
+		if (seen) {
+			int i;
+			struct strbuf sb = STRBUF_INIT;
+			for (i = 0; i < cmd_list.nr; i++) {
+				struct string_list_item *item = &cmd_list.items[i];
+
+				strbuf_addf(&sb, "\n  %s", item->string);
+				if (item == seen)
+					strbuf_addstr(&sb, " <==");
+				else if (i == cmd_list.nr - 1)
+					strbuf_addstr(&sb, " ==>");
+			}
 			die(_("alias loop detected: expansion of '%s' does"
-			      " not terminate"), cmd_list.items[0].string);
+			      " not terminate:%s"), cmd_list.items[0].string, sb.buf);
 		}
 
 		string_list_append(&cmd_list, *argv[0]);
-- 
2.19.0.rc2.1.g4c98b8d69.dirty

[PATCH v5 3/3] t0014: Introduce an alias testing suite

[Tim Schumacher]

Introduce a testing suite that is dedicated to aliases.
For now, check only if nested aliases work and if looping
aliases are detected successfully.

The looping aliases check for mixed execution is there but
disabled, because it is blocking the test suite for a full
minute. As soon as there is a solution for loops using
external commands, it should be enabled.

Signed-off-by: Tim Schumacher <timschumi@gmx.de>
---
Changes since v4:
 - Actually execute a command in the first two cases
 - Remove the "setup code"
 - Use i18ngrep to match the part of a message
 - Comment out the last test

 t/t0014-alias.sh | 40 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
 create mode 100755 t/t0014-alias.sh

diff --git a/t/t0014-alias.sh b/t/t0014-alias.sh
new file mode 100755
index 000000000..a070e645d
--- /dev/null
+++ b/t/t0014-alias.sh
@@ -0,0 +1,40 @@
+#!/bin/sh
+
+test_description='git command aliasing'
+
+. ./test-lib.sh
+
+test_expect_success 'nested aliases - internal execution' '
+	git config alias.nested-internal-1 nested-internal-2 &&
+	git config alias.nested-internal-2 status &&
+	git nested-internal-1 >output &&
+	test_i18ngrep "^On branch " output
+'
+
+test_expect_success 'nested aliases - mixed execution' '
+	git config alias.nested-external-1 nested-external-2 &&
+	git config alias.nested-external-2 "!git nested-external-3" &&
+	git config alias.nested-external-3 status &&
+	git nested-external-1 >output &&
+	test_i18ngrep "^On branch " output
+'
+
+test_expect_success 'looping aliases - internal execution' '
+	git config alias.loop-internal-1 loop-internal-2 &&
+	git config alias.loop-internal-2 loop-internal-3 &&
+	git config alias.loop-internal-3 loop-internal-2 &&
+	test_must_fail git loop-internal-1 2>output &&
+	test_i18ngrep "^fatal: alias loop detected: expansion of" output
+'
+
+# This test is disabled until external loops are fixed, because would block
+# the test suite for a full minute.
+#
+#test_expect_failure 'looping aliases - mixed execution' '
+#	git config alias.loop-mixed-1 loop-mixed-2 &&
+#	git config alias.loop-mixed-2 "!git loop-mixed-1" &&
+#	test_must_fail git loop-mixed-1 2>output &&
+#	test_i18ngrep "^fatal: alias loop detected: expansion of" output
+#'
+
+test_done
-- 
2.19.0.rc2.1.g4c98b8d69.dirty