completion: dynamic completion loading

Adding external subcommands to Git is as easy as to put an executable file git-foo into PATH. Packaging such subcommands for a Linux distribution can be achieved by unpacking the executable into /usr/bin of the user's system. Adding system-wide completion scripts for new subcommands, however, can be a bit tricky.

Since bash-completion started to use dynamical loading of completion scripts since v1.90 (preview of v2.0),

it is no longer sufficient to
drop a completion script of a subcommand into the standard completions path, /usr/share/bash-completion/completions, since this script will not be loaded if called as a git subcommand.

For example, look at https://bugs.gentoo.org/544722. To give a short
summary: The popular git-flow subcommand provides a completion script, which gets installed as /usr/share/bash-completion/completions/git-flow.

If you now type into a Bash shell:

git flow <TAB>

You will not get any completions, because bash-completion only loads completions for git and git has no idea that git-flow is defined in another file. You have to load this script manually or trigger the dynamic loader with:

git-flow <TAB> # Please notice the dash instead of whitespace

This will not complete anything either, because it only defines a Bash function, without generating completions. But now the correct completion script has been loaded and the first command can use the completions.

So, the goal is now to teach the git completion script to consider the possibility of external completion scripts for subcommands, but of course without breaking current workflows.

I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion.

It will take care of
loading the correct script if present. Afterwards, the git completion script behaves as usual.

This way we can leverage bash-completion's dynamic loading for git subcommands and make it easier for developers to distribute custom completion scripts.

Signed-off-by Florian Gamböck <[email protected]>

---
 contrib/completion/git-completion.bash | 10 ++++++++++
 1 file changed, 10 insertions(+)

contrib/completion/git-completion.bash +10 -1

index b09c8a236..09a820990 100644
@@ -3096,12 +3096,22 @@ __git_main ()
        fi
 
        local completion_func="_git_${command//-/_}"
+       if ! declare -f $completion_func >/dev/null 2>/dev/null &&
+               declare -f __load_completion >/dev/null 2>/dev/null
+       then
+               __load_completion "git-$command"
+       fi
        declare -f $completion_func >/dev/null 2>/dev/null && $completion_func && return
 
        local expansion=$(__git_aliased_command "$command")
        if [ -n "$expansion" ]; then
                words[1]=$expansion
                completion_func="_git_${expansion//-/_}"
+               if ! declare -f $completion_func >/dev/null 2>/dev/null &&
+                       declare -f __load_completion >/dev/null 2>/dev/null
+               then
+                       __load_completion "git-$expansion"
+               fi
                declare -f $completion_func >/dev/null 2>/dev/null && $completion_func
        fi
 }
--

2.16.1

Quoted 3 lines · Florian Gamböck

Adding external subcommands to Git is as easy as to put an executable file git-foo into PATH. Packaging such subcommands for a Linux distribution can be achieved by unpacking the executable into /usr/bin of the user's system. Adding system-wide completion scripts for new subcommands, however, can be a bit tricky.

Since bash-completion started to use dynamical loading of completion scripts since v1.90 (preview of v2.0),

I believe the main bash-completion repository can be found at:

https://github.com/scop/bash-completion.git

This repository still contains the branch 'dynamic-loading'; for the record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2.

Two commits on that branch are worth mentioning:

20c05b43 (Load completions in separate files dynamically, get rid of

            have()., 2011-10-12)
  5baebf81 (Add _xfunc for loading and calling functions on demand,
            use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13)

Quoted 3 lines

(...)

I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion.

This is wrong, __load_completion() was introduced in cad3abfc (__load_completion: New function, use in _completion_loader and _xfunc, 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03.

The release tags '1.90' and '2.0' are from 2011-11-03 and 2012-06-17, respectively. This leaves a couple of years long hole where completions were already loaded dynamically but there was no __load_completion() function.

Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function.

Quoted 20 lines · SZEDER Gábor

Quoted 9 lines · Florian Gamböck

Adding external subcommands to Git is as easy as to put an executable file git-foo into PATH. Packaging such subcommands for a Linux distribution can be achieved by unpacking the executable into /usr/bin of the user's system. Adding system-wide completion scripts for new subcommands, however, can be a bit tricky.

Since bash-completion started to use dynamical loading of completion scripts since v1.90 (preview of v2.0),

I believe the main bash-completion repository can be found at:

https://github.com/scop/bash-completion.git

This repository still contains the branch 'dynamic-loading'; for the record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2.

Quoted 6 lines

Two commits on that branch are worth mentioning:

20c05b43 (Load completions in separate files dynamically, get rid of

            have()., 2011-10-12)
  5baebf81 (Add _xfunc for loading and calling functions on demand,
            use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13)

Nice, thanks for the pointers!

Quoted 9 lines

Quoted 4 lines

(...)

I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion.

This is wrong, __load_completion() was introduced in cad3abfc (__load_completion: New function, use in _completion_loader and _xfunc, 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03.

Dang, I thought it was introduced at the same time. Sorry for that. I guess, 2016 is a bit too young to take it for granted then?

Quoted 5 lines

The release tags '1.90' and '2.0' are from 2011-11-03 and 2012-06-17, respectively. This leaves a couple of years long hole where completions were already loaded dynamically but there was no __load_completion() function.

Quoted 3 lines

Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function.

But isn't this exactly what we want?

Lucky us, we can replace the whole
if-fi block with a simpler:

   _xfunc git-$command $completion_func 2>/dev/null && return

If _xfunc is not defined -- as in, bashcomp is not installed / loaded -- then the return will not get called and the original completion will continue:

   declare -f $completion_func >/dev/null 2>/dev/null &&
       $completion_func && return

Since this would be redundant, we could define a fall-back for _xfunc like so:

   declare -f _xfunc || _xfunc() {
       declare -f $completion_func >/dev/null 2>/dev/null &&
           $completion_func && return
   }

This way, we retain the "old" behavior and get dynamic loading if bashcomp is available. The actual call to get the completions would just be _xfunc like in my first example above.

What do you think?

--
Regards

Florian

Quoted 4 lines · Florian Gamböck

Quoted 1 line · SZEDER Gábor

Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function.

But isn't this exactly what we want?

No, that's definitely not what we want.

The bash-completion project can get away with it, because they only use their _xfunc() to source a file they themselves ship and to call a completion function they know that that file contains.

We, however, would like to load a file that might not exist and to call a function that might not be defined. Git has a lot of plumbing commands with neither a corresponding _git_plumbing_cmd() completion function in our completion script nor a corresponding 'git-plumbing-cmd' file that could be sourced dynamically to provide that function. The same applies to any 'git-foo' command in the user's $PATH (the user's own scripts or various git-related tools, e.g. Git LFS).

So if someone tries e.g. 'git diff-index <TAB>' to complete files in the current directory, then it would result in an error message like

_git_diff_index: command not found

Furthermore, earlier versions of _xfunc(), I think until the introduction of __load_completion(), tried to source the file given as parameter without checking its existence beforehand, so on whatever LTS I happen to be currently using I would also get an error like

bash: /usr/share/bash-completion/completions/git-diff-index: No such file or directory

Finally, since all this is running in the user's interactive shell, Bash will even run the 'command_not_found_handle', if it's set (and most Linux distros do set it in their default configuration (OK, maybe not most, but at least some of the more popular do)), which may or may not have any suggestions, but at the very least it takes quite a while to scan through its database.

Quoted 21 lines · Florian Gamböck

Quoted 20 lines · SZEDER Gábor

I believe the main bash-completion repository can be found at:

https://github.com/scop/bash-completion.git

This repository still contains the branch 'dynamic-loading'; for the record it points to 3b029892f6f9db3b7210a7f66d636be3e5ec5fa2.

Two commits on that branch are worth mentioning:

20c05b43 (Load completions in separate files dynamically, get rid of

            have()., 2011-10-12)
  5baebf81 (Add _xfunc for loading and calling functions on demand,
            use it in apt-get, cvsps, rsync, and sshfs., 2011-10-13)

Quoted 2 lines

I think the easiest method is to use a function that is defined by bash-completion v2.0+, namely __load_completion.

This is wrong, __load_completion() was introduced in cad3abfc (__load_completion: New function, use in _completion_loader and _xfunc, 2015-07-15), and the first release tag containg it is '2.2' from 2016-03-03.

Quoted 5 lines

Quoted 3 lines

Would it be possible to use _xfunc() instead to plug that hole? It seems the be tricky, because that function not only sources but also _calls_ the completion function.

But isn't this exactly what we want?

No, that's definitely not what we want.

In my previous emails I overlooked the _completion_loader() helper function.

It seems that this function does almost exactly what we want. It was introduced along with dynamic completion loading back in 20c05b43, so it's available for us even in older LTS/Enterprise releases. Since cad3abfc it's a wrapper around __load_completion() and thus benefits from all the improvements, notably searching for completion scripts in a user-specified directory ($BASH_COMPLETION_USER_DIR) or in the user's home directory ($XDG_DATA_HOME or ~/.local/...) as well. It loads the matching completion script, but does not call the completion function unconditionally.

The "almost" refers to he case when _completion_loader() can't find a completion script with a matching name to load, and then registers the _minimal() completion function for the given command to do basic path completion as fallback. I don't think this matters in practice, because in this case the given command is a git command in its dashed form, e.g. 'git-diff-index', and those have been deprecated for a long time.

I think all you need to do is run a s/__load_completion/_completion_loader/ on your patch and update the commit message with relevant bits from the above discussion.

Quoted 4 lines · SZEDER Gábor

In my previous emails I overlooked the _completion_loader() helper function.

Quoted 8 lines

It seems that this function does almost exactly what we want. It was introduced along with dynamic completion loading back in 20c05b43, so it's available for us even in older LTS/Enterprise releases. Since cad3abfc it's a wrapper around __load_completion() and thus benefits from all the improvements, notably searching for completion scripts in a user-specified directory ($BASH_COMPLETION_USER_DIR) or in the user's home directory ($XDG_DATA_HOME or ~/.local/...) as well. It loads the matching completion script, but does not call the completion function unconditionally.

Sounds good so far.

Quoted 6 lines

The "almost" refers to he case when _completion_loader() can't find a completion script with a matching name to load, and then registers the _minimal() completion function for the given command to do basic path completion as fallback. I don't think this matters in practice, because in this case the given command is a git command in its dashed form, e.g. 'git-diff-index', and those have been deprecated for a long time.

I sense a problem here. If I have a directory with a file xyzfoobar in it, and I type git xyz, with no defined subcommand that starts with these letters, then minimal bashcomp would give me git xyzfoobar, which can of course not execute. This can be unintuitive for users, as in: "If it can't be executed correctly, then why does it even suggest such a completion?"

Quoted 3 lines

I think all you need to do is run a s/__load_completion/_completion_loader/ on your patch and update the commit message with relevant bits from the above discussion.

I can do that, no problem. But prior to that I want to be sure that you are okay with the above mentioned drawback. Will the behavior be acceptable in this case? Or should we try to somehow "undo" the minimal completion afterwards?

Quoted 1 line · Florian Gamböck

I sense a problem here. If I have a directory with a file xyzfoobar in it, and I type git xyz, with no defined subcommand that starts with these letters, then minimal bashcomp would give me git xyzfoobar, which can of course not execute. This can be unintuitive for users, as in: "If it can't be executed correctly, then why does it even suggest such a completion?"

I'm not sure I understand the problem. After 'git xyz<TAB>' (note there is no space between 'xyz' and <TAB>) we try to complete the name of a git command, not options of a git command. This means:

- At this point we don't look for a _git_xyz() function, so we'll return from __git_main() before even reaching the piece of code your patch modifies.

- There are (presumably) no commands starting with 'xyz', so we don't list any commands. Bash will then fall back to its own filename completion, and that's where that 'xyzfoobar' will come from. It has been behaving like this basically since forever.

And after 'git xyz <TAB>' (this time with space) we already complete the next word, not 'xyz'.

Quoted 3 lines

Quoted 1 line

I think all you need to do is run a s/__load_completion/_completion_loader/ on your patch and update the commit message with relevant bits from the above discussion.

I can do that, no problem. But prior to that I want to be sure that you are okay with the above mentioned drawback. Will the behavior be acceptable in this case? Or should we try to somehow "undo" the minimal completion afterwards?

As explained above, I don't think there is any drawback here. Or at least not any new drawback that your patch is introducing. Or I'm completely missing your point; certainly a possibility, it's early Sunday afternoon, after all :)