[PATCH 0/3] completion: updates to ctags symbol names

[PATCH 0/3] completion: updates to ctags symbol names

[SZEDER Gábor]

This short series 

  - speeds up the completion of ctags symbol names by using some of
    the trick from the refs completion speedup series, and

  - offers symbol names for 'git log -S', '-G' and '-L:', which I find
    really convenient when digging history.

It should go on top of 'sg/completion-refs-speedup'.


SZEDER Gábor (3):
  completion: put matching ctags symbol names directly into COMPREPLY
  completion: extract completing ctags symbol names into helper function
  completion: offer ctags symbol names for 'git log -S', '-G' and '-L:'

 contrib/completion/git-completion.bash | 72 +++++++++++++++++++++++++++++++---
 1 file changed, 66 insertions(+), 6 deletions(-)

-- 
2.12.1.498.gded7ead47

[PATCH 1/3] completion: put matching ctags symbol names directly into COMPREPLY

[SZEDER Gábor]

The one-liner awk script in __git_match_ctag() listing ctags symbol
names for 'git grep <TAB>' is already smart enough to list only symbol
names matching the current word to be completed.

Extend this helper function to accept prefix and suffix parameters to
be prepended and appended, respectively, to each listed symbol name in
the awk script, so its output won't require any additional processing
or filtering in the completion script before being handed over to
Bash.  Use the faster __gitcomp_direct() helper instead of
__gitcomp_nl() to fill the fully processed matching symbol names into
Bash's COMPREPLY array.

Right after 'git grep <TAB>' in current git.git with 14k+ symbol names
in the tag file, best of five:

  Before:

    $ time __gitcomp_nl "$(__git_match_ctag "" tags)"

    real    0m0.178s
    user    0m0.176s
    sys     0m0.000s

  After:

    $ time __gitcomp_direct "$(__git_match_ctag "" tags "" " ")"

    real    0m0.058s
    user    0m0.048s
    sys     0m0.008s

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---

Notes:
    It's still just a simple linear search through the tags file, so there
    are no miracles: it's still hopelessly, unusably slow e.g. in the
    Linux repository.

 contrib/completion/git-completion.bash | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 69eabc2bc..5cacb0e2e 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1518,8 +1518,15 @@ _git_gitk ()
 	_gitk
 }
 
-__git_match_ctag() {
-	awk "/^${1//\//\\/}/ { print \$1 }" "$2"
+# Lists matching symbol names from a tag (as in ctags) file.
+# 1: List symbol names matching this word.
+# 2: The tag file to list symbol names from.
+# 3: A prefix to be added to each listed symbol name (optional).
+# 4: A suffix to be appended to each listed symbol name (optional).
+__git_match_ctag () {
+	awk -v pfx="${3-}" -v sfx="${4-}" "
+		/^${1//\//\\/}/ { print pfx \$1 sfx }
+		" "$2"
 }
 
 _git_grep ()
@@ -1548,7 +1555,7 @@ _git_grep ()
 	case "$cword,$prev" in
 	2,*|*,-*)
 		if test -r tags; then
-			__gitcomp_nl "$(__git_match_ctag "$cur" tags)"
+			__gitcomp_direct "$(__git_match_ctag "$cur" tags "" " ")"
 			return
 		fi
 		;;
-- 
2.12.1.498.gded7ead47

Re: [PATCH 1/3] completion: put matching ctags symbol names directly into COMPREPLY

[Jeff King]

On Thu, Mar 23, 2017 at 04:38:37PM +0100, SZEDER Gábor wrote:

> The one-liner awk script in __git_match_ctag() listing ctags symbol
> names for 'git grep <TAB>' is already smart enough to list only symbol
> names matching the current word to be completed.
> 
> Extend this helper function to accept prefix and suffix parameters to
> be prepended and appended, respectively, to each listed symbol name in
> the awk script, so its output won't require any additional processing
> or filtering in the completion script before being handed over to
> Bash.  Use the faster __gitcomp_direct() helper instead of
> __gitcomp_nl() to fill the fully processed matching symbol names into
> Bash's COMPREPLY array.

Seems like an easy win (well, neglecting the other 14 patches that let
to having gitcomp_direct).

I never really noticed it as particularly slow in git.git, but faster is
always better if it comes cheaply.

> Notes:
>     It's still just a simple linear search through the tags file, so there
>     are no miracles: it's still hopelessly, unusably slow e.g. in the
>     Linux repository.

I just tried "git grep foo<Tab>" in the kernel and it was pretty fast.
But then I tried "git grep <Tab>". Ouch. So it seems like it's not the
awk matching that's slow, but everything that comes after (and is
dependent on how much output it produces).

-Peff

[PATCH 2/3] completion: extract completing ctags symbol names into helper function

[SZEDER Gábor]

The previous commit doubled the number of __git_match_ctag()'s
positional parameters, and, to keep the position of existing
parameters for the sake of backwards compatibility, the prefix,
current word and suffix parameters ended up in different order than in
other functions accepting the same parameters.  Then there is a
condition checking the existence of the tag file before invoking this
function.

We could still live with this if there were only a single callsite,
but the next commit will add a few more, so it's worth providing a
cleaner interface.

Add the wrapper function __git_complete_symbol(), which encompasses
the condition for checking the presence of the tag file and filling
COMPREPLY, and accepts '--opt=val'-style options with default values
that keep callsites simpler.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 contrib/completion/git-completion.bash | 33 +++++++++++++++++++++++++++++----
 1 file changed, 29 insertions(+), 4 deletions(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 5cacb0e2e..e38178022 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1529,6 +1529,34 @@ __git_match_ctag () {
 		" "$2"
 }
 
+# Complete symbol names from a tag file.
+# Usage: __git_complete_symbol [<option>]...
+# --tags=<file>: The tag file to list symbol names from instead of the
+#                default "tags".
+# --pfx=<prefix>: A prefix to be added to each symbol name.
+# --cur=<word>: The current symbol name to be completed.  Defaults to
+#               the current word to be completed.
+# --sfx=<suffix>: A suffix to be appended to each symbol name instead
+#                 of the default space.
+__git_complete_symbol () {
+	local tags=tags pfx="" cur_="${cur-}" sfx=" "
+
+	while test $# != 0; do
+		case "$1" in
+		--tags=*)	tags="${1##--tags=}" ;;
+		--pfx=*)	pfx="${1##--pfx=}" ;;
+		--cur=*)	cur_="${1##--cur=}" ;;
+		--sfx=*)	sfx="${1##--sfx=}" ;;
+		*)		return 1 ;;
+		esac
+		shift
+	done
+
+	if test -r "$tags"; then
+		__gitcomp_direct "$(__git_match_ctag "$cur_" "$tags" "$pfx" "$sfx")"
+	fi
+}
+
 _git_grep ()
 {
 	__git_has_doubledash && return
@@ -1554,10 +1582,7 @@ _git_grep ()
 
 	case "$cword,$prev" in
 	2,*|*,-*)
-		if test -r tags; then
-			__gitcomp_direct "$(__git_match_ctag "$cur" tags "" " ")"
-			return
-		fi
+		__git_complete_symbol && return
 		;;
 	esac
 
-- 
2.12.1.498.gded7ead47

Re: [PATCH 2/3] completion: extract completing ctags symbol names into helper function

[Jeff King]

On Thu, Mar 23, 2017 at 04:38:38PM +0100, SZEDER Gábor wrote:

> The previous commit doubled the number of __git_match_ctag()'s
> positional parameters, and, to keep the position of existing
> parameters for the sake of backwards compatibility, the prefix,
> current word and suffix parameters ended up in different order than in
> other functions accepting the same parameters.  Then there is a
> condition checking the existence of the tag file before invoking this
> function.
> 
> We could still live with this if there were only a single callsite,
> but the next commit will add a few more, so it's worth providing a
> cleaner interface.
> 
> Add the wrapper function __git_complete_symbol(), which encompasses
> the condition for checking the presence of the tag file and filling
> COMPREPLY, and accepts '--opt=val'-style options with default values
> that keep callsites simpler.

Nice cleanup. The resulting code in _git_grep() is much more readable.

I use the same completion for "vim -t". Now I can cheat and switch it to
__git_complete_symbol. :)

-Peff

[PATCH 3/3] completion: offer ctags symbol names for 'git log -S', '-G' and '-L:'

[SZEDER Gábor]

Just like in the case of search patterns for 'git grep', see 29eec71f2
(completion: match ctags symbol names in grep patterns, 2011-10-21)),
a common thing to look for using 'git log -S', '-G' and '-L:' is the
name of a symbol.

Teach the completion for 'git log' to offer ctags symbol names after
these options, both in stuck and in unstuck forms.

Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com>
---
 contrib/completion/git-completion.bash | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index e38178022..48690ad1a 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1687,6 +1687,19 @@ _git_log ()
 	if [ -f "$__git_repo_path/MERGE_HEAD" ]; then
 		merge="--merge"
 	fi
+	case "$prev,$cur" in
+	-L,:*:*)
+		return	# fall back to Bash filename completion
+		;;
+	-L,:*)
+		__git_complete_symbol --cur="${cur#:}" --sfx=":"
+		return
+		;;
+	-G,*|-S,*)
+		__git_complete_symbol
+		return
+		;;
+	esac
 	case "$cur" in
 	--pretty=*|--format=*)
 		__gitcomp "$__git_log_pretty_formats $(__git_pretty_aliases)
@@ -1732,6 +1745,21 @@ _git_log ()
 			"
 		return
 		;;
+	-L:*:*)
+		return	# fall back to Bash filename completion
+		;;
+	-L:*)
+		__git_complete_symbol --cur="${cur#-L:}" --sfx=":"
+		return
+		;;
+	-G*)
+		__git_complete_symbol --pfx="-G" --cur="${cur#-G}"
+		return
+		;;
+	-S*)
+		__git_complete_symbol --pfx="-S" --cur="${cur#-S}"
+		return
+		;;
 	esac
 	__git_complete_revlist
 }
-- 
2.12.1.498.gded7ead47

Re: [PATCH 3/3] completion: offer ctags symbol names for 'git log -S', '-G' and '-L:'

[Jeff King]

On Thu, Mar 23, 2017 at 04:38:39PM +0100, SZEDER Gábor wrote:

> Just like in the case of search patterns for 'git grep', see 29eec71f2
> (completion: match ctags symbol names in grep patterns, 2011-10-21)),
> a common thing to look for using 'git log -S', '-G' and '-L:' is the
> name of a symbol.
> 
> Teach the completion for 'git log' to offer ctags symbol names after
> these options, both in stuck and in unstuck forms.

I think this makes sense and is an improvement over the status quo.

There are two gotchas with completing "-L" like this:

  1. You still have to come up with the filename yourself for "-L".

  2. The function name is actually a regex, so you can get bit when your
     function name is a subset of another.

I have a script (below) which makes this easier (and I complete its
argument using the tags file).  It's probably too gross to even go into
contrib, but I thought I'd share.

"log -S" sometimes benefits from limiting by filename, too, but it
depends what you're doing. I don't have a gross script for that. :)

-- >8 --
#!/usr/bin/env perl
if (!@ARGV) {
  print STDERR "usage: git flog [options] <function>\n";
  exit 1;
}

my $func = pop @ARGV;
my $file = get_file_from_tags($func);
my $regex = '[^A-Za-z_]' . $func . '[^A-Za-z0-9_]';
exec qw(git log), "-L:$regex:$file", @ARGV;
exit 1;

sub get_file_from_tags {
  my $token = shift;

  open(my $fh, '<', 'tags')
    or die "unable to open tags: $!\n";
  while (<$fh>) {
    chomp;

    # this isn't exactly right, as the Ex command may contain
    # embedded tabs, but it's accurate for the token and filename,
    # which come before, and probably good enough to match extension fields
    # which come after
    my @fields = split /\t/;

    next unless $fields[0] eq $token;

    # only look for functions; assumes your ctags uses the "kind"
    # extension field. Note also that some implementations write the "kind:"
    # header and some do not. This handles both.
    next unless grep { /^(kind:\s*)?f$/ } @fields;

    # there may be more, but we don't have any way of disambiguating,
    # so just return the first match
    return $fields[1];
  }

  die "unknown token: $token\n";
}
__END__

Re: [PATCH 3/3] completion: offer ctags symbol names for 'git log -S', '-G' and '-L:'

[SZEDER Gábor]

On Fri, Mar 24, 2017 at 1:52 AM, Jeff King <peff@peff.net> wrote:
> On Thu, Mar 23, 2017 at 04:38:39PM +0100, SZEDER Gábor wrote:
> 
> > Just like in the case of search patterns for 'git grep', see 29eec71f2
> > (completion: match ctags symbol names in grep patterns, 2011-10-21)),
> > a common thing to look for using 'git log -S', '-G' and '-L:' is the
> > name of a symbol.
> > 
> > Teach the completion for 'git log' to offer ctags symbol names after
> > these options, both in stuck and in unstuck forms.
> 
> I think this makes sense and is an improvement over the status quo.
> 
> There are two gotchas with completing "-L" like this:

Reordering your gotchas for convenience...

>   2. The function name is actually a regex, so you can get bit when your
>      function name is a subset of another.

... and when the longer name comes first in the given file.  Yeah, I
was bitten by that a few of times, too :)

However, I found that completing symbol names can be quite helpful in
this case, because now I see that there are longer symbol names
starting with what I'm looking for while typing/completing the
command, and then I can just delete the trailing ':' and add a '\(:'
or '\ :' depending on language and coding style right away.  Of course
it doesn't help, if the function name in question is the suffix or a
substring in the middle of another one, but that seems to be even
rarer, or at least I've yet to come across such a case in practice.

>   1. You still have to come up with the filename yourself for "-L".

I was already quite satisfied that both the symbol name and the 
filename can be TAB completed...  but right, in most cases the
function name uniquely determines the filename, and even when it
doesn't, it still significantly limits the possibilities.  Hmhm.

Listing the funcname:filename together is not a good idea in my
opinion.  A small change to __git_match_ctag()'s awk script can show
why:

-               /^${1//\//\\/}/ { print pfx \$1 sfx }
+               /^${1//\//\\/}/ { print pfx \$1 \":\" \$2 sfx }

The amount of text flooding my terminal on completion is just
overwhelming (and I haven't even tried it in a Java project :), and it
breaks the "helpful" use-case with longer matching symbol names I
described above (it's much harder to see that there are longer
matching symbol names and I would have to backtrack the whole filename
to modify the funcname).  It also breaks symbol name completion for
'-G' and '-S', of course, so further changes would be necessary to
deal with that.

OTOH, the proof-of-concept patch at the bottom shows how we could
start doing filename completion based on the ctags file, and I think
it's really convenient to use.  Alas, it doesn't work when the
funcname is not on its own, e.g. ends with that disambiguating '\(:'
from above, and then Bash falls back to its own filename completion.
However, if I may extrapolate from my ~/.bash_history, this would
still help the great majority of the cases.

> I have a script (below) which makes this easier (and I complete its
> argument using the tags file).  It's probably too gross to even go into
> contrib, but I thought I'd share.

Perhaps 'git log -L' could be enhanced to just DWIM when it gets only
'-L:func:', and show the log of that function, wherever it is defined.
So instead of complaining about the missing filename, it could run
'grep <func>' with a bit of magic to find the filename where the
matching function is declared, and search in the history of that file.

But then again, 'git log -L' could be enhanced in so many ways...


Gábor


 -- >8 --

Subject: [PATCH] [PoC] completion: list only relevant filenames after 'git log
 -L:funcname:'

---
 contrib/completion/git-completion.bash | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index d6f25a7e0..8b58096e4 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -1678,6 +1678,13 @@ __git_log_shortlog_options="
 __git_log_pretty_formats="oneline short medium full fuller email raw format:"
 __git_log_date_formats="relative iso8601 rfc2822 short local default raw"
 
+__git_filename_for_symbol ()
+{
+	local symbol="$1" tags="$2"
+
+	awk "/^${symbol//\//\\}/ { print \$2 }" "$tags"
+}
+
 _git_log ()
 {
 	__git_has_doubledash && return
@@ -1689,7 +1696,11 @@ _git_log ()
 	fi
 	case "$prev,$cur" in
 	-L,:*:*)
-		return	# fall back to Bash filename completion
+		local symbol="${cur#:}"
+		symbol="${symbol%%:*}"
+		__gitcomp_nl "$(__git_filename_for_symbol "$symbol" tags)" \
+			"" "${cur#:$symbol:}"
+		return
 		;;
 	-L,:*)
 		__git_complete_symbol --cur="${cur#:}" --sfx=":"
@@ -1746,7 +1757,11 @@ _git_log ()
 		return
 		;;
 	-L:*:*)
-		return	# fall back to Bash filename completion
+		local symbol="${cur#-L:}"
+		symbol="${symbol%%:*}"
+		__gitcomp_nl "$(__git_filename_for_symbol "$symbol" tags)" \
+			"" "${cur#-L:$symbol:}"
+		return
 		;;
 	-L:*)
 		__git_complete_symbol --cur="${cur#-L:}" --sfx=":"
-- 
2.12.2.453.g223d29900

Re: [PATCH 3/3] completion: offer ctags symbol names for 'git log -S', '-G' and '-L:'

[Jeff King]

On Thu, Mar 30, 2017 at 12:06:56PM +0200, SZEDER Gábor wrote:

> >   1. You still have to come up with the filename yourself for "-L".
> 
> I was already quite satisfied that both the symbol name and the 
> filename can be TAB completed...  but right, in most cases the
> function name uniquely determines the filename, and even when it
> doesn't, it still significantly limits the possibilities.  Hmhm.

I find that I often forget which file a function is defined in,
especially in Git's code base (where it sometimes feels somewhat
random :) ).

> OTOH, the proof-of-concept patch at the bottom shows how we could
> start doing filename completion based on the ctags file, and I think
> it's really convenient to use.  Alas, it doesn't work when the
> funcname is not on its own, e.g. ends with that disambiguating '\(:'
> from above, and then Bash falls back to its own filename completion.
> However, if I may extrapolate from my ~/.bash_history, this would
> still help the great majority of the cases.

Yeah, I think that would go a long way to solving my problem.

> > I have a script (below) which makes this easier (and I complete its
> > argument using the tags file).  It's probably too gross to even go into
> > contrib, but I thought I'd share.
> 
> Perhaps 'git log -L' could be enhanced to just DWIM when it gets only
> '-L:func:', and show the log of that function, wherever it is defined.
> So instead of complaining about the missing filename, it could run
> 'grep <func>' with a bit of magic to find the filename where the
> matching function is declared, and search in the history of that file.
> 
> But then again, 'git log -L' could be enhanced in so many ways...

Yes, that sounds even nicer.

-Peff