[PATCH] completion: clear cached --options when sourcing the completion script

[PATCH] completion: clear cached --options when sourcing the completion script

[SZEDER Gábor]

The established way to update the completion script in an already
running shell is to simply source it again: this brings in any new
--options and features, and clears caching variables.  E.g. it clears
the variables caching the list of (all|porcelain) git commands, so
when they are later lazy-initialized again, then they will list and
cache any newly installed commmands as well.

Unfortunately, since d401f3debc (git-completion.bash: introduce
__gitcomp_builtin, 2018-02-09) and subsequent patches this doesn't
work for a lot of git commands' options.  To eliminate a lot of
hard-to-maintain hard-coded lists of options, those commits changed
the completion script to use a bunch of programmatically created and
lazy-initialized variables to cache the options of those builtin
porcelain commands that use parse-options.  These variables are not
cleared upon sourcing the completion script, therefore they continue
caching the old lists of options, even when some commands recently
learned new options or when deprecated options were removed.

Always 'unset' these variables caching the options of builtin commands
when sourcing the completion script.

Redirect 'unset's stderr to /dev/null, because ZSH's 'unset' complains
if it's invoked without any arguments, i.e. no variables caching
builtin's options are set.  This can happen, if someone were to source
the completion script twice without completing any --options in
between.  Bash stays silent in this case.

Add tests to ensure that these variables are indeed cleared when the
completion script is sourced; not just the variables caching options,
but all other caching variables, i.e. the variables caching commands,
porcelain commands and merge strategies as well.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---

Fixes a recent regression introduced in 'nd/parseopt-completion'.

 contrib/completion/git-completion.bash |  4 ++++
 t/t9902-completion.sh                  | 31 ++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 7c84eb1912..602352f952 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -280,6 +280,10 @@ __gitcomp ()
 	esac
 }
 
+# Clear the variables caching builtins' options when (re-)sourcing
+# the completion script.
+unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
+
 # This function is equivalent to
 #
 #    __gitcomp "$(git xxx --git-completion-helper) ..."
diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index e6485feb0a..4c86adadf2 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1497,4 +1497,35 @@ do
 	'
 done
 
+test_expect_success 'sourcing the completion script clears cached commands' '
+	__git_compute_all_commands &&
+	verbose test -n "$__git_all_commands" &&
+	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
+	verbose test -z "$__git_all_commands"
+'
+
+test_expect_success 'sourcing the completion script clears cached porcelain commands' '
+	__git_compute_porcelain_commands &&
+	verbose test -n "$__git_porcelain_commands" &&
+	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
+	verbose test -z "$__git_porcelain_commands"
+'
+
+test_expect_success 'sourcing the completion script clears cached merge strategies' '
+	__git_compute_merge_strategies &&
+	verbose test -n "$__git_merge_strategies" &&
+	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
+	verbose test -z "$__git_merge_strategies"
+'
+
+test_expect_success 'sourcing the completion script clears cached --options' '
+	__gitcomp_builtin checkout &&
+	verbose test -n "$__gitcomp_builtin_checkout" &&
+	__gitcomp_builtin notes_edit &&
+	verbose test -n "$__gitcomp_builtin_notes_edit" &&
+	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
+	verbose test -z "$__gitcomp_builtin_checkout" &&
+	verbose test -z "$__gitcomp_builtin_notes_edit"
+'
+
 test_done
-- 
2.17.0.rc0.103.gbdc5836ed3

Re: [PATCH] completion: clear cached --options when sourcing the completion script

[Duy Nguyen]

On Thu, Mar 22, 2018 at 3:16 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:
> The established way to update the completion script in an already
> running shell is to simply source it again: this brings in any new
> --options and features, and clears caching variables.  E.g. it clears
> the variables caching the list of (all|porcelain) git commands, so
> when they are later lazy-initialized again, then they will list and
> cache any newly installed commmands as well.
>
> Unfortunately, since d401f3debc (git-completion.bash: introduce
> __gitcomp_builtin, 2018-02-09) and subsequent patches this doesn't
> work for a lot of git commands' options.  To eliminate a lot of
> hard-to-maintain hard-coded lists of options, those commits changed
> the completion script to use a bunch of programmatically created and
> lazy-initialized variables to cache the options of those builtin
> porcelain commands that use parse-options.  These variables are not
> cleared upon sourcing the completion script, therefore they continue
> caching the old lists of options, even when some commands recently
> learned new options or when deprecated options were removed.
>
> Always 'unset' these variables caching the options of builtin commands
> when sourcing the completion script.

And here I have been happily unsetting these manually when I re-source
to test stuff, not thinking it as a bug. Thanks!

> Redirect 'unset's stderr to /dev/null, because ZSH's 'unset' complains
> if it's invoked without any arguments, i.e. no variables caching
> builtin's options are set.  This can happen, if someone were to source
> the completion script twice without completing any --options in
> between.  Bash stays silent in this case.
>
> Add tests to ensure that these variables are indeed cleared when the
> completion script is sourced; not just the variables caching options,
> but all other caching variables, i.e. the variables caching commands,
> porcelain commands and merge strategies as well.
>
> Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
> ---
>
> Fixes a recent regression introduced in 'nd/parseopt-completion'.
>
>  contrib/completion/git-completion.bash |  4 ++++
>  t/t9902-completion.sh                  | 31 ++++++++++++++++++++++++++
>  2 files changed, 35 insertions(+)
>
> diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
> index 7c84eb1912..602352f952 100644
> --- a/contrib/completion/git-completion.bash
> +++ b/contrib/completion/git-completion.bash
> @@ -280,6 +280,10 @@ __gitcomp ()
>         esac
>  }
>
> +# Clear the variables caching builtins' options when (re-)sourcing
> +# the completion script.
> +unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
> +
>  # This function is equivalent to
>  #
>  #    __gitcomp "$(git xxx --git-completion-helper) ..."
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index e6485feb0a..4c86adadf2 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1497,4 +1497,35 @@ do
>         '
>  done
>
> +test_expect_success 'sourcing the completion script clears cached commands' '
> +       __git_compute_all_commands &&
> +       verbose test -n "$__git_all_commands" &&
> +       . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
> +       verbose test -z "$__git_all_commands"
> +'
> +
> +test_expect_success 'sourcing the completion script clears cached porcelain commands' '
> +       __git_compute_porcelain_commands &&
> +       verbose test -n "$__git_porcelain_commands" &&
> +       . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
> +       verbose test -z "$__git_porcelain_commands"
> +'
> +
> +test_expect_success 'sourcing the completion script clears cached merge strategies' '
> +       __git_compute_merge_strategies &&
> +       verbose test -n "$__git_merge_strategies" &&
> +       . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
> +       verbose test -z "$__git_merge_strategies"
> +'
> +
> +test_expect_success 'sourcing the completion script clears cached --options' '
> +       __gitcomp_builtin checkout &&
> +       verbose test -n "$__gitcomp_builtin_checkout" &&
> +       __gitcomp_builtin notes_edit &&
> +       verbose test -n "$__gitcomp_builtin_notes_edit" &&
> +       . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
> +       verbose test -z "$__gitcomp_builtin_checkout" &&
> +       verbose test -z "$__gitcomp_builtin_notes_edit"
> +'
> +
>  test_done
> --
> 2.17.0.rc0.103.gbdc5836ed3
>



-- 
Duy

Re: [PATCH] completion: clear cached --options when sourcing the completion script

[SZEDER Gábor]

On Thu, Mar 22, 2018 at 3:16 PM, SZEDER Gábor <szeder.dev@gmail.com> wrote:

> Add tests to ensure that these variables are indeed cleared when the
> completion script is sourced; not just the variables caching options,
> but all other caching variables, i.e. the variables caching commands,
> porcelain commands and merge strategies as well.

> +test_expect_success 'sourcing the completion script clears cached merge strategies' '
> +       __git_compute_merge_strategies &&
> +       verbose test -n "$__git_merge_strategies" &&
> +       . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&
> +       verbose test -z "$__git_merge_strategies"
> +'

Hang on, this test fails in the GETTEXT_POISON build.

The thing is, we get the merge strategies with this piece of code in
__git_list_merge_strategies() in master:

    LANG=C LC_ALL=C git merge -s help 2>&1 |
    sed -n -e '/[Aa]vailable strategies are: /,/^$/{
        # a couple of s/// commands
    }'


and that '/[Aa]vailable strategies are: /' won't match in a
GETTEXT_POISON-ed output, because that string is translated.

I think for now (-rc phase) we should just drop this test, and in the
future we should consider adding a 'git merge --list-strategies' option.

Re: [PATCH] completion: clear cached --options when sourcing the completion script

[Junio C Hamano]

SZEDER Gábor <szeder.dev@gmail.com> writes:

> Hang on, this test fails in the GETTEXT_POISON build.

Thanks.

> The thing is, we get the merge strategies with this piece of code in
> __git_list_merge_strategies() in master:
>
>     LANG=C LC_ALL=C git merge -s help 2>&1 |
>     sed -n -e '/[Aa]vailable strategies are: /,/^$/{
>         # a couple of s/// commands
>     }'
>
>
> and that '/[Aa]vailable strategies are: /' won't match in a
> GETTEXT_POISON-ed output, because that string is translated.
>
> I think for now (-rc phase) we should just drop this test, and in the
> future we should consider adding a 'git merge --list-strategies' option.

I'd say we should just add !GETTEXT_POISON prereq to the problematic
tests.  

"git merge -s help" output under forced C locale is dependable in
the real world.  It is GETTEXT_POISON that does not get this fact
right.  There is no need for '--list-strat' option to make this test
pass.

Re: [PATCH] completion: clear cached --options when sourcing the completion script

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> SZEDER Gábor <szeder.dev@gmail.com> writes:
>
>> Hang on, this test fails in the GETTEXT_POISON build.
>
> Thanks.
>
>> The thing is, we get the merge strategies with this piece of code in
>> __git_list_merge_strategies() in master:
>>
>>     LANG=C LC_ALL=C git merge -s help 2>&1 |
>>     sed -n -e '/[Aa]vailable strategies are: /,/^$/{
>>         # a couple of s/// commands
>>     }'
>>
>>
>> and that '/[Aa]vailable strategies are: /' won't match in a
>> GETTEXT_POISON-ed output, because that string is translated.
>>
>> I think for now (-rc phase) we should just drop this test, and in the
>> future we should consider adding a 'git merge --list-strategies' option.
>
> I'd say we should just add !GETTEXT_POISON prereq to the problematic
> tests.  
>
> "git merge -s help" output under forced C locale is dependable in
> the real world.  It is GETTEXT_POISON that does not get this fact
> right.  There is no need for '--list-strat' option to make this test
> pass.

IOW, this is the minumum required.

By the way, shouldn't we be running the body of these new tests
inside a subshell?  Otherwise a dot-sourcing by an earlier test of
these new ones _will_ affect all the subsequent tests.

 t/t9902-completion.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
index 4c86adadf2..b7f5b1e632 100755
--- a/t/t9902-completion.sh
+++ b/t/t9902-completion.sh
@@ -1511,7 +1511,7 @@ test_expect_success 'sourcing the completion script clears cached porcelain comm
 	verbose test -z "$__git_porcelain_commands"
 '
 
-test_expect_success 'sourcing the completion script clears cached merge strategies' '
+test_expect_success !GETTEXT_POISON 'sourcing the completion script clears cached merge strategies' '
 	__git_compute_merge_strategies &&
 	verbose test -n "$__git_merge_strategies" &&
 	. "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&

Re: [PATCH] completion: clear cached --options when sourcing the completion script

[SZEDER Gábor]

On Fri, Mar 23, 2018 at 6:35 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Junio C Hamano <gitster@pobox.com> writes:

>> I'd say we should just add !GETTEXT_POISON prereq to the problematic
>> tests.

> IOW, this is the minumum required.

Thanks for already committing the fix, I couldn't get around to it.

> By the way, shouldn't we be running the body of these new tests
> inside a subshell?  Otherwise a dot-sourcing by an earlier test of
> these new ones _will_ affect all the subsequent tests.

I don't think it matters.  All new tests first initialize the variable
they are interested in before sourcing the completion script, to avoid
false successes caused by the variable being empty to begin with.

And it shouldn't matter, either, because in the end the users will
source the completion script into their main shell process, and it
should just work no matter how many times it gets sourced.


>  t/t9902-completion.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index 4c86adadf2..b7f5b1e632 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1511,7 +1511,7 @@ test_expect_success 'sourcing the completion script clears cached porcelain comm
>         verbose test -z "$__git_porcelain_commands"
>  '
>
> -test_expect_success 'sourcing the completion script clears cached merge strategies' '
> +test_expect_success !GETTEXT_POISON 'sourcing the completion script clears cached merge strategies' '
>         __git_compute_merge_strategies &&
>         verbose test -n "$__git_merge_strategies" &&
>         . "$GIT_BUILD_DIR/contrib/completion/git-completion.bash" &&