completion: updates to ctags symbol names

This short series

- speeds up the completion of ctags symbol names by using some of the trick from the refs completion speedup series, and

- offers symbol names for 'git log -S', '-G' and '-L:', which I find really convenient when digging history.

It should go on top of 'sg/completion-refs-speedup'.

SZEDER Gábor (3):
completion: put matching ctags symbol names directly into COMPREPLY
completion: extract completing ctags symbol names into helper function
completion: offer ctags symbol names for 'git log -S', '-G' and '-L:'

 contrib/completion/git-completion.bash | 72 +++++++++++++++++++++++++++++++---
 1 file changed, 66 insertions(+), 6 deletions(-)

2.12.1.498.gded7ead47

The one-liner awk script in __git_match_ctag() listing ctags symbol names for 'git grep <TAB>' is already smart enough to list only symbol names matching the current word to be completed.

Extend this helper function to accept prefix and suffix parameters to be prepended and appended, respectively, to each listed symbol name in the awk script, so its output won't require any additional processing or filtering in the completion script before being handed over to Bash. Use the faster __gitcomp_direct() helper instead of __gitcomp_nl() to fill the fully processed matching symbol names into Bash's COMPREPLY array.

Right after 'git grep <TAB>' in current git.git with 14k+ symbol names in the tag file, best of five:

Before:

    $ time __gitcomp_nl "$(__git_match_ctag "" tags)"

    real    0m0.178s
    user    0m0.176s
    sys     0m0.000s

After:

    $ time __gitcomp_direct "$(__git_match_ctag "" tags "" " ")"

    real    0m0.058s
    user    0m0.048s
    sys     0m0.008s

---

Notes:
It's still just a simple linear search through the tags file, so there are no miracles: it's still hopelessly, unusably slow e.g. in the Linux repository.

 contrib/completion/git-completion.bash | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

index 69eabc2bc..5cacb0e2e 100644
@@ -1518,8 +1518,15 @@ _git_gitk ()
 	_gitk
 }
 
-__git_match_ctag() {
-	awk "/^${1//\//\\/}/ { print \$1 }" "$2"
+# Lists matching symbol names from a tag (as in ctags) file.
+# 1: List symbol names matching this word.
+# 2: The tag file to list symbol names from.
+# 3: A prefix to be added to each listed symbol name (optional).
+# 4: A suffix to be appended to each listed symbol name (optional).
+__git_match_ctag () {
+	awk -v pfx="${3-}" -v sfx="${4-}" "
+		/^${1//\//\\/}/ { print pfx \$1 sfx }
+		" "$2"
 }
 
 _git_grep ()

@@ -1548,7 +1555,7 @@ _git_grep ()
 	case "$cword,$prev" in
 	2,*|*,-*)
 		if test -r tags; then
-			__gitcomp_nl "$(__git_match_ctag "$cur" tags)"
+			__gitcomp_direct "$(__git_match_ctag "$cur" tags "" " ")"
 			return
 		fi
 		;;

2.12.1.498.gded7ead47

Seems like an easy win (well, neglecting the other 14 patches that let to having gitcomp_direct).

I never really noticed it as particularly slow in git.git, but faster is always better if it comes cheaply.

I just tried "git grep foo<Tab>" in the kernel and it was pretty fast. But then I tried "git grep <Tab>". Ouch. So it seems like it's not the awk matching that's slow, but everything that comes after (and is dependent on how much output it produces).

-Peff

The previous commit doubled the number of __git_match_ctag()'s positional parameters, and, to keep the position of existing parameters for the sake of backwards compatibility, the prefix, current word and suffix parameters ended up in different order than in other functions accepting the same parameters. Then there is a condition checking the existence of the tag file before invoking this function.

We could still live with this if there were only a single callsite, but the next commit will add a few more, so it's worth providing a cleaner interface.

Add the wrapper function __git_complete_symbol(), which encompasses the condition for checking the presence of the tag file and filling COMPREPLY, and accepts '--opt=val'-style options with default values that keep callsites simpler.

---
 contrib/completion/git-completion.bash | 33 +++++++++++++++++++++++++++++----
 1 file changed, 29 insertions(+), 4 deletions(-)

index 5cacb0e2e..e38178022 100644
@@ -1529,6 +1529,34 @@ __git_match_ctag () {
 		" "$2"
 }
 
+# Complete symbol names from a tag file.
+# Usage: __git_complete_symbol [<option>]...
+# --tags=<file>: The tag file to list symbol names from instead of the
+#                default "tags".
+# --pfx=<prefix>: A prefix to be added to each symbol name.
+# --cur=<word>: The current symbol name to be completed.  Defaults to
+#               the current word to be completed.
+# --sfx=<suffix>: A suffix to be appended to each symbol name instead
+#                 of the default space.
+__git_complete_symbol () {
+	local tags=tags pfx="" cur_="${cur-}" sfx=" "
+
+	while test $# != 0; do
+		case "$1" in
+		--tags=*)	tags="${1##--tags=}" ;;
+		--pfx=*)	pfx="${1##--pfx=}" ;;
+		--cur=*)	cur_="${1##--cur=}" ;;
+		--sfx=*)	sfx="${1##--sfx=}" ;;
+		*)		return 1 ;;
+		esac
+		shift
+	done
+
+	if test -r "$tags"; then
+		__gitcomp_direct "$(__git_match_ctag "$cur_" "$tags" "$pfx" "$sfx")"
+	fi
+}
+
 _git_grep ()
 {
 	__git_has_doubledash && return

@@ -1554,10 +1582,7 @@ _git_grep ()
 
 	case "$cword,$prev" in
 	2,*|*,-*)
-		if test -r tags; then
-			__gitcomp_direct "$(__git_match_ctag "$cur" tags "" " ")"
-			return
-		fi
+		__git_complete_symbol && return
 		;;
 	esac
 

2.12.1.498.gded7ead47

Nice cleanup. The resulting code in _git_grep() is much more readable.

I use the same completion for "vim -t". Now I can cheat and switch it to __git_complete_symbol. :)

-Peff

Just like in the case of search patterns for 'git grep', see 29eec71f2 (completion: match ctags symbol names in grep patterns, 2011-10-21)), a common thing to look for using 'git log -S', '-G' and '-L:' is the name of a symbol.

Teach the completion for 'git log' to offer ctags symbol names after these options, both in stuck and in unstuck forms.

---
 contrib/completion/git-completion.bash | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

index e38178022..48690ad1a 100644
@@ -1687,6 +1687,19 @@ _git_log ()
 	if [ -f "$__git_repo_path/MERGE_HEAD" ]; then
 		merge="--merge"
 	fi
+	case "$prev,$cur" in
+	-L,:*:*)
+		return	# fall back to Bash filename completion
+		;;
+	-L,:*)
+		__git_complete_symbol --cur="${cur#:}" --sfx=":"
+		return
+		;;
+	-G,*|-S,*)
+		__git_complete_symbol
+		return
+		;;
+	esac
 	case "$cur" in
 	--pretty=*|--format=*)
 		__gitcomp "$__git_log_pretty_formats $(__git_pretty_aliases)

@@ -1732,6 +1745,21 @@ _git_log ()
 			"
 		return
 		;;
+	-L:*:*)
+		return	# fall back to Bash filename completion
+		;;
+	-L:*)
+		__git_complete_symbol --cur="${cur#-L:}" --sfx=":"
+		return
+		;;
+	-G*)
+		__git_complete_symbol --pfx="-G" --cur="${cur#-G}"
+		return
+		;;
+	-S*)
+		__git_complete_symbol --pfx="-S" --cur="${cur#-S}"
+		return
+		;;
 	esac
 	__git_complete_revlist
 }

2.12.1.498.gded7ead47

I think this makes sense and is an improvement over the status quo.

There are two gotchas with completing "-L" like this:

1. You still have to come up with the filename yourself for "-L".

2. The function name is actually a regex, so you can get bit when your function name is a subset of another.

I have a script (below) which makes this easier (and I complete its argument using the tags file). It's probably too gross to even go into contrib, but I thought I'd share.

"log -S" sometimes benefits from limiting by filename, too, but it depends what you're doing. I don't have a gross script for that. :)

-- >8 --
#!/usr/bin/env perl
if (!@ARGV) {
print STDERR "usage: git flog [options] <function>\n"; exit 1;
}

my $func = pop @ARGV;
my $file = get_file_from_tags($func);
my $regex = '[^A-Za-z_]' . $func . '[^A-Za-z0-9_]'; exec qw(git log), "-L:$regex:$file", @ARGV; exit 1;

sub get_file_from_tags {
my $token = shift;

open(my $fh, '<', 'tags')

    or die "unable to open tags: $!\n";

while (<$fh>) {
chomp;

# this isn't exactly right, as the Ex command may contain
# embedded tabs, but it's accurate for the token and filename, # which come before, and probably good enough to match extension fields # which come after
my @fields = split /\t/;

next unless $fields[0] eq $token;

# only look for functions; assumes your ctags uses the "kind" # extension field. Note also that some implementations write the "kind:" # header and some do not. This handles both.
next unless grep { /^(kind:\s*)?f$/ } @fields;

# there may be more, but we don't have any way of disambiguating, # so just return the first match
return $fields[1];
}

die "unknown token: $token\n";
}
__END__

Reordering your gotchas for convenience...

... and when the longer name comes first in the given file. Yeah, I was bitten by that a few of times, too :)

However, I found that completing symbol names can be quite helpful in this case, because now I see that there are longer symbol names starting with what I'm looking for while typing/completing the command, and then I can just delete the trailing ':' and add a '\(:' or '\ :' depending on language and coding style right away. Of course it doesn't help, if the function name in question is the suffix or a substring in the middle of another one, but that seems to be even rarer, or at least I've yet to come across such a case in practice.

I was already quite satisfied that both the symbol name and the filename can be TAB completed... but right, in most cases the function name uniquely determines the filename, and even when it doesn't, it still significantly limits the possibilities. Hmhm.

Listing the funcname:filename together is not a good idea in my opinion. A small change to __git_match_ctag()'s awk script can show why:

- /^${1//\//\\/}/ { print pfx \$1 sfx }
+ /^${1//\//\\/}/ { print pfx \$1 \":\" \$2 sfx }

The amount of text flooding my terminal on completion is just overwhelming (and I haven't even tried it in a Java project :), and it breaks the "helpful" use-case with longer matching symbol names I described above (it's much harder to see that there are longer matching symbol names and I would have to backtrack the whole filename to modify the funcname). It also breaks symbol name completion for '-G' and '-S', of course, so further changes would be necessary to deal with that.

OTOH, the proof-of-concept patch at the bottom shows how we could start doing filename completion based on the ctags file, and I think it's really convenient to use. Alas, it doesn't work when the funcname is not on its own, e.g. ends with that disambiguating '\(:' from above, and then Bash falls back to its own filename completion. However, if I may extrapolate from my ~/.bash_history, this would still help the great majority of the cases.

Perhaps 'git log -L' could be enhanced to just DWIM when it gets only '-L:func:', and show the log of that function, wherever it is defined. So instead of complaining about the missing filename, it could run 'grep <func>' with a bit of magic to find the filename where the matching function is declared, and search in the history of that file.

But then again, 'git log -L' could be enhanced in so many ways...

Gábor

-- >8 --

Subject: [PATCH] [PoC] completion: list only relevant filenames after 'git log
-L:funcname:'

---
 contrib/completion/git-completion.bash | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

index d6f25a7e0..8b58096e4 100644
@@ -1678,6 +1678,13 @@ __git_log_shortlog_options="
 __git_log_pretty_formats="oneline short medium full fuller email raw format:"
 __git_log_date_formats="relative iso8601 rfc2822 short local default raw"
 
+__git_filename_for_symbol ()
+{
+	local symbol="$1" tags="$2"
+
+	awk "/^${symbol//\//\\}/ { print \$2 }" "$tags"
+}
+
 _git_log ()
 {
 	__git_has_doubledash && return

@@ -1689,7 +1696,11 @@ _git_log ()
 	fi
 	case "$prev,$cur" in
 	-L,:*:*)
-		return	# fall back to Bash filename completion
+		local symbol="${cur#:}"
+		symbol="${symbol%%:*}"
+		__gitcomp_nl "$(__git_filename_for_symbol "$symbol" tags)" \
+			"" "${cur#:$symbol:}"
+		return
 		;;
 	-L,:*)
 		__git_complete_symbol --cur="${cur#:}" --sfx=":"

@@ -1746,7 +1757,11 @@ _git_log ()
 		return
 		;;
 	-L:*:*)
-		return	# fall back to Bash filename completion
+		local symbol="${cur#-L:}"
+		symbol="${symbol%%:*}"
+		__gitcomp_nl "$(__git_filename_for_symbol "$symbol" tags)" \
+			"" "${cur#-L:$symbol:}"
+		return
 		;;
 	-L:*)
 		__git_complete_symbol --cur="${cur#-L:}" --sfx=":"

2.12.2.453.g223d29900

I find that I often forget which file a function is defined in, especially in Git's code base (where it sometimes feels somewhat random :) ).

Yeah, I think that would go a long way to solving my problem.

Yes, that sounds even nicer.

-Peff