specify encoding for sed command
PATCH 1/1 specify encoding for sed command
Fixes issue with seeing sed: RE error: illegal byte sequence when running git-completion.bash
--- contrib/completion/git-completion.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
contrib/completion/git-completion.bash +1 -1
@@ -282,7 +282,7 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +unset $(set |LANG=C sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null # This function is equivalent to #
Quoted 10 lines · Stephon Harris
Fixes issue with seeing
sed: RE error: illegal byte sequencewhen running git-completion.bash--- contrib/completion/git-completion.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)contrib/completion/git-completion.bash +0
-0
contrib/completion/git-completion.bash __gitcomp () · line 285
Quoted 5 lines
contrib/completion/git-completion.bash +1
-1@@ -282,7 +282,7 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +unset $(set |LANG=C sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
This is getting closer to the issue than your previous patch, but there's still some open questions:
1) What platform OS / version / sed version is this on?
2) What's the output from "set" that's causing this error? Do we have an isolated test case for that?
3) There's other invocations of "sed" in the file, aren't those affected as well?
4) Any reason we wouldn't just set LC_AlL=C for the whole file? I see we already do it for our invocation to "git merge".
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line · Stephon Harris
Fixes issue with seeing
sed: RE error: illegal byte sequencewhen running git-completion.bashThis is getting closer to the issue than your previous patch, but there's still some open questions:
1) What platform OS / version / sed version is this on?
Quoted 8 lines
2) What's the output from "set" that's causing this error? Do we have an isolated test case for that?
3) There's other invocations of "sed" in the file, aren't those affected as well?
4) Any reason we wouldn't just set LC_AlL=C for the whole file? I see we already do it for our invocation to "git merge".
Also, missing Signed-off-by: (see Documentation/SubmittingPatches).
Thanks.
Quoted 4 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line · Stephon Harris
Fixes issue with seeing
sed: RE error: illegal byte sequencewhen running git-completion.bash
Please:
- prefix the subject line with "completion:".
- nit: replace "running" with "sourcing".
- wrap the body of the commit message around 70 characters.
- sign off your patch; see Documentation/SubmittingPatches and 'git
commit -s'.
Quoted 10 lines
Quoted 10 lines
--- contrib/completion/git-completion.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)contrib/completion/git-completion.bash +0
-0@@ -282,7 +282,7 @@ __gitcomp ()
Quoted 4 lines
contrib/completion/git-completion.bash __gitcomp () · line 285
Quoted 4 lines
contrib/completion/git-completion.bash +1
-1# Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +unset $(set |LANG=C sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
I was wondering how could you see an error message from 'sed', when the stderr of the whole thing is redirected to /dev/null. It turns out that such a redirection doesn't affect the stderr of the commands executed inside the command substitution:
$ echo $(echo foo ; echo >&2 error) 2>/dev/null error foo
Interesting, I didn't know that.
Quoted 7 lines
This is getting closer to the issue than your previous patch, but there's still some open questions:
1) What platform OS / version / sed version is this on?
2) What's the output from "set" that's causing this error? Do we have an isolated test case for that?
A quick internet search suggests that this error message tends to appear on Macs, when its 'sed' encounters an invalid UTF-8 character in its input.
Quoted 2 lines
3) There's other invocations of "sed" in the file, aren't those affected as well?
The two 'sed' invocations in _git_stash() are suspect, as they process the output of 'git stash list', which includes the stashes' descriptions, which can contain whatever the users wished to store there (though they would probably get a "Warning: commit message did not conform to UTF-8" message from 9e83266525 (commit-tree: encourage UTF-8 commit messages., 2006-12-22) when doing so).
The 'sed' invocation in __git_complete_revlist_file() is suspect as well, as it processes the output of 'git ls-tree'. Pathnames can contain anything, and while 'git ls-tree' quotes pathnames with "unusual" characters, I don't know whether it quotes all characters that can possibly upset Stephon's 'sed'.
What about 'awk' on Stephon's platform? We already have one 'awk' invocation in __git_match_ctag(), which we use for 'git grep <TAB>' and 'git log -L:<TAB>'. That 'awk' script uses a regexp to match the symbol name in a ctags file; does any programming language allow such unusual characters in symbol names?
What about 'sed' and 'awk' scripts that don't use regexps at all? (I intend to add such an 'awk' script in a day or two...)
Quoted 2 lines
4) Any reason we wouldn't just set LC_AlL=C for the whole file? I see we already do it for our invocation to "git merge".
That would perhaps be the safest thing to do...
But how can we set it for a whole file, when the file is not executed as a script but sourced into the user's shell environment?
Quoted 1 line
1) What platform OS / version / sed version is this on?
I'm experiencing this on macOS Sierra (10.12.6). The issue occurs with the OS's native sed, which is FreeBSD sed so the version number is kind of ambiguous.
The error goes away if I set LANG=C or LC_ALL=C or change it to use GNU sed (installed via homebrew as gsed).
Quoted 2 lines
2) What's the output from "set" that's causing this error? Do we have an isolated test case for that?
On my system, it's caused by whatever powerline is doing to Bash's PS1 variable:
$ set | grep ^PS1= | sed 's/foo/bar/' sed: RE error: illegal byte sequence
I'm not sure exactly which character is bad here:
$ xxd <<<$PS1 00000000: 5c5b 1b5b 303b 3338 3b32 3b32 3535 3b32 \[.[0;38;2;255;2 00000010: 3535 3b32 3535 3b34 383b 323b 303b 3133 55;255;48;2;0;13 00000020: 353b 3137 353b 316d 5c5d c2a0 6d61 7474 5;175;1m\]..matt 00000030: c2a0 5c5b 1b5b 303b 3338 3b32 3b30 3b31 ..\[.[0;38;2;0;1 00000040: 3335 3b31 3735 3b34 383b 323b 3838 3b38 35;175;48;2;88;8 00000050: 383b 3838 3b32 326d 5c5d ee82 b0c2 a05c 8;88;22m\].....\ 00000060: 5b1b 5b30 3b33 383b 323b 3230 383b 3230 [.[0;38;2;208;20 00000070: 383b 3230 383b 3438 3b32 3b38 383b 3838 8;208;48;2;88;88 00000080: 3b38 383b 316d 5c5d 7ec2 a05c 5b1b 5b30 ;88;1m\]~..\[.[0 00000090: 3b33 383b 323b 3133 383b 3133 383b 3133 ;38;2;138;138;13 000000a0: 383b 3438 3b32 3b38 383b 3838 3b38 383b 8;48;2;88;88;88; 000000b0: 3232 6d5c 5dee 82b1 c2a0 5c5b 1b5b 303b 22m\].....\[.[0; 000000c0: 3338 3b32 3b32 3038 3b32 3038 3b32 3038 38;2;208;208;208 000000d0: 3b34 383b 323b 3838 3b38 383b 3838 3b31 ;48;2;88;88;88;1 000000e0: 6d5c 5dc2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 m\]............. 000000f0: a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 ................ 00000100: a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 ................ 00000110: a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 ................ 00000120: a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 ................ 00000130: a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a0c2 a05c ...............\ 00000140: 5b1b 5b30 3b33 383b 323b 3438 3b34 383b [.[0;38;2;48;48; 00000150: 3438 3b34 383b 323b 3838 3b38 383b 3838 48;48;2;88;88;88 00000160: 3b32 326d 5c5d c2a0 ee82 b25c 5b1b 5b30 ;22m\].....\[.[0 00000170: 3b33 383b 323b 3135 383b 3135 383b 3135 ;38;2;158;158;15 00000180: 383b 3438 3b32 3b34 383b 3438 3b34 386d 8;48;2;48;48;48m 00000190: 5c5d c2a0 3230 3138 2d30 342d 3130 5c5b \]..2018-04-10\[ 000001a0: 1b5b 303b 3338 3b32 3b39 383b 3938 3b39 .[0;38;2;98;98;9 000001b0: 383b 3438 3b32 3b34 383b 3438 3b34 383b 8;48;2;48;48;48; 000001c0: 3232 6d5c 5dc2 a0ee 82b3 5c5b 1b5b 303b 22m\].....\[.[0; 000001d0: 3338 3b32 3b32 3038 3b32 3038 3b32 3038 38;2;208;208;208 000001e0: 3b34 383b 323b 3438 3b34 383b 3438 3b31 ;48;2;48;48;48;1 000001f0: 6d5c 5dc2 a030 333a 3136 c2a0 5c5b 1b5b m\]..03:16..\[.[ 00000200: 306d 5c5d 205c 5b1b 5b30 3b33 383b 323b 0m\] \[.[0;38;2; 00000210: 3135 383b 3135 383b 3135 383b 3438 3b32 158;158;158;48;2 00000220: 3b34 383b 3438 3b34 386d 5c5d c2a0 c2a0 ;48;48;48m\].... 00000230: 5c5b 1b5b 303b 3338 3b32 3b34 383b 3438 \[.[0;38;2;48;48 00000240: 3b34 383b 3439 3b32 326d 5c5d ee82 b0c2 ;48;49;22m\].... 00000250: a05c 5b1b 5b30 6d5c 5d0a .\[.[0m\].
I found another (possibly better) way to fix this:
Quoted 6 lines · Matt Coleman
Quoted 2 lines
1) What platform OS / version / sed version is this on?
I'm experiencing this on macOS Sierra (10.12.6). The issue occurs with the OS's native sed, which is FreeBSD sed so the version number is kind of ambiguous.
The error goes away if I set LANG=C or LC_ALL=C or change it to use GNU sed (installed via homebrew as gsed).
If I change it to use awk instead of sed, it works with mawk, gawk, and macOS awk:
unset $(set | awk -F '=' '/^__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*=/ {print $1}') 2>/dev/null
I compared sed vs. awk on Linux and Mac and they all take about the same amount of time to run (within 0.002ms).
I did a little more digging into this issue today.
Quoted 7 lines · Matt Coleman
I found another (possibly better) way to fix this:
Quoted 3 lines · Matt Coleman
Quoted 2 lines
1) What platform OS / version / sed version is this on?
Quoted 8 lines
Quoted 3 lines
I'm experiencing this on macOS Sierra (10.12.6). The issue occurs with the OS's native sed, which is FreeBSD sed so the version number is kind of ambiguous.
The error goes away if I set LANG=C or LC_ALL=C or change it to use GNU sed (installed via homebrew as gsed).
If I change it to use awk instead of sed, it works with mawk, gawk, and macOS awk:
unset $(set | awk -F '=' '/^__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*=/ {print $1}') 2>/dev/nullI compared sed vs. awk on Linux and Mac and they all take about the same amount of time to run (within 0.002ms).
The issue isn't actually caused by sed: it's caused by the way that the set builtin outputs characters in the Unicode Private Use Area (PUA) in the build of Bash 3.2.57 that macOS Sierra ships with.
Powerline uses several PUA code points to make some of its pretty text UI elements:
Code point (hex value): description U+E0A0 (0xEE82A0): Version control branch U+E0A1 (0xEE82A1): LN (line) symbol U+E0A2 (0xEE82A2): Closed padlock U+E0B0 (0xEE82B0): Rightwards black arrowhead U+E0B1 (0xEE82B1): Rightwards arrowhead U+E0B2 (0xEE82B2): Leftwards black arrowhead U+E0B3 (0xEE82B3): Leftwards arrowhead
macOS Bash 3.2.57's set builtin has garbled output where Powerline's special symbols should be in the PS1 variable, but Bash 4.4.19 (installed on macOS via homebrew) and Bash 4.3.38 (Ubuntu 16.04) both display it correctly in the output of set. echo $PS1 does display the symbols correctly on these versions of Bash.
So...
Quoted 1 line
3) There's other invocations of "sed" in the file, aren't those affected as well?
The short answer: no. Slightly longer: not by the same thing that's affecting the line in the patch, at least.
Long: As described above, the problem isn't actually sed: it's the set builtin in macOS's build of Bash. The other invocations of sed should be safe, because sed properly handles the PUA code points on its own:
$ echo $'\xee\x82\xb0' | sed 's/./@/' @
The way that set is displaying the PS1 variable seems to be sending them to sed individually or somehow split up:
$ for character in $'\xee' $'\x82' $'\xb0' $'\xee\x82' $'\x82\xb0'; do echo $character | sed 's/./@/'; done sed: RE error: illegal byte sequence sed: RE error: illegal byte sequence sed: RE error: illegal byte sequence sed: RE error: illegal byte sequence sed: RE error: illegal byte sequence
Interestingly, Bash 3.2.25's set builtin on CentOS 5 correctly displays the octal values for the symbols (prompt edited to keep this email ASCII):
$ PS1=$'\xee\x82\xb0 '
Quoted 1 line
set | grep PS1
PS1=$'\356\202\260 '
I haven't started digging through Bash's codebase, but it could either be a bug that was introduced between 3.2.25 and 3.2.57 or Apple used some silly flags when compiling Bash. Ideally, this should be fixed in Bash, but since Apple's using such an old version of Bash for license reasons, I think it's unlikely that they'll fix the issue.
I think the best way to move forward with this is a new patch that uses awk instead of sed: I tested several awk variants and the command was portable without requiring any changes to LANG or LC_ALL.
Does that sound like a good plan?
Quoted 19 lines
I did a little more digging into this issue today.
The issue isn't actually caused by
sed: it's caused by the way that thesetbuiltin outputs characters in the Unicode Private Use Area (PUA) in the build of Bash 3.2.57 that macOS Sierra ships with.Powerline uses several PUA code points to make some of its pretty text UI elements:
Code point (hex value): description
U+E0A0 (0xEE82A0): Version control branch
U+E0A1 (0xEE82A1): LN (line) symbol
U+E0A2 (0xEE82A2): Closed padlock
U+E0B0 (0xEE82B0): Rightwards black arrowhead
U+E0B1 (0xEE82B1): Rightwards arrowhead
U+E0B2 (0xEE82B2): Leftwards black arrowhead
U+E0B3 (0xEE82B3): Leftwards arrowhead
Quoted 5 lines
macOS Bash 3.2.57's
setbuiltin has garbled output where Powerline's special symbols should be in the PS1 variable, but Bash 4.4.19 (installed on macOS via homebrew) and Bash 4.3.38 (Ubuntu 16.04) both display it correctly in the output ofset.echo $PS1does display the symbols correctly on these versions of Bash.
Thanks for digging.
Quoted 5 lines
I think the best way to move forward with this is a new patch that uses
awkinstead ofsed: I tested severalawkvariants and the command was portable without requiring any changes to LANG orLC_ALL.Does that sound like a good plan?
No ;)
Could you please give the patch below a try?
I intended it as a fix for a minor performance regression introduced in the same commit that triggered this issue, but if it can work around this issue, too, all the better.
I'm just not sure whether we should burden this otherwise nice and short commit message with the details of this Powerline-macOS-Bash-sed issue...
-- >8 --
Subject: [PATCH] completion: reduce overhead of clearing cached --options
To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them, 8b0eaa41f2 (completion:
clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessasry overhead with the extra processes.
In Bash we can do better: run the 'compgen -v __gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.
TODO: as an unexpected bonus, this might work around the issue with 'sed' and invalid UTF-8 characters in the environment as well, at least in Bash.
--- contrib/completion/git-completion.bash | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
contrib/completion/git-completion.bash +5 -1
@@ -282,7 +282,11 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +if [[ -n ${ZSH_VERSION-} ]]; then + unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +else + unset $(compgen -v __gitcomp_builtin_) +fi # This function is equivalent to #
Signature · 1 line
2.17.0.366.gbe216a3084
Quoted 8 lines
Quoted 5 lines
I think the best way to move forward with this is a new patch that uses
awkinstead ofsed: I tested severalawkvariants and the command was portable without requiring any changes to LANG orLC_ALL.Does that sound like a good plan?
No ;)
Could you please give the patch below a try?
Your patch works great and is clearly a way better approach than parsing the output of set.
Quoted 2 lines
I'm just not sure whether we should burden this otherwise nice and short commit message with the details of this Powerline-macOS-Bash-sed issue...
It might be worth mentioning it briefly, in order to reduce the chances of a regression in the future. Maybe something like...
"In Bash we can do better: run the 'compgen -v __gitcomp_builtin_' builtin command, which lists the same variables, but without the risk of errors caused by poor Unicode handling in some shells' 'set' builtin command or the overhead of executing 'sed'."
PATCH 1/1 completion: reduce overhead of clearing cached --options
To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them, 8b0eaa41f2 (completion:
clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessasry overhead with the extra processes.
In Bash we can do better: run the 'compgen -v __gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.
This change also happens to work around an issue reported by users of the Powerline shell prompt on macOS, which was triggered by the same commit 8b0eaa41f2 as well. Powerline uses several Unicode Private Use Area code points to represent some of its pretty text UI elements (arrows and what not), and these are stored in the $PS1 variable. Apparently the 'set' builtin command of the default Bash version shipped in macOS (3.2.57) has issues with these code points, and produces garbled output where Powerline's special symbols should be in the $PS1 variable. This, in turn, triggers the following error message in the downstream 'sed' process:
sed: RE error: illegal byte sequence
Other Bash versions, notably 4.4.19 on macOS (via homebrew) and 3.2.25 on CentOS don't seem to be affected.
With this patch neither the 'set' builtin is invoked to print garbage, nor 'sed' to choke on it.
--- contrib/completion/git-completion.bash | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
contrib/completion/git-completion.bash +5 -1
@@ -282,7 +282,11 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +if [[ -n ${ZSH_VERSION-} ]]; then + unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +else + unset $(compgen -v __gitcomp_builtin_) +fi # This function is equivalent to #
Signature · 1 line
2.17.0.366.gbe216a3084
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 6 lines · SZEDER Gábor
To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them,
8b0eaa41f2(completion:
clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessasry overhead with the extra processes.
Typo: necessasry -> necessary
Quoted 4 lines
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.
What about ZSH?
Signature · 1 line
Jakub Narębski
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 6 lines · Jakub Narebski
Quoted 3 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.What about ZSH?
Nothing, ZSH is unaffected by this patch.
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 10 lines · SZEDER Gábor
Quoted 7 lines · Jakub Narebski
Quoted 4 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.What about ZSH?
Nothing, ZSH is unaffected by this patch.
All right, so for ZSH we would need LC_ALL=C trick, or come with some equivalent of 'compgen -v __gitcomp_builtin_' for this shell.
Good patch, though it does not solve whole of the problem.
Best,
Signature · 1 line
Jakub Narębski
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 12 lines · Jakub Narebski
Quoted 10 lines · SZEDER Gábor
Quoted 7 lines · Jakub Narebski
Quoted 4 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.What about ZSH?
Nothing, ZSH is unaffected by this patch.
Quoted 8 lines
All right, so for ZSH we would need LC_ALL=C trick, or come with some equivalent of 'compgen -v
__gitcomp_builtin_' for this shell.Good patch, though it does not solve whole of the problem.
Best,
--
Jakub Narębski
Is ZSH actually affected by the broken set behavior, though?
Thanks,
Jake
Re: [PATCH] completion: reduce overhead of clearing cached --options
Disclaimer: I'm not a zsh user, so please correct anything I might have gotten wrong.
I created a .zshrc with the following contents:
autoload -Uz compinit
compinit
source /usr/local/lib/python3.6/site-packages/powerline/bindings/zsh/powerline.zsh
zsh doesn't have broken Unicode output in its set builtin, so it was not affected by the original issue. Applying the patch does not cause any change in behavior.
Since the commit only changes a file with "bash" in its name, but the conditional references zsh variables, I think it's worth mentioning something about it in the commit message.
I think the bash side of things is all set for this commit, but can a similar improvement be made (using a builtin instead of parsing set|sed) for zsh? Again, I'm not a zsh user, so some input from someone who's written zsh completion rules would be very helpful. Can any of the builtins mentioned in this zsh documentation be used instead of set|sed?
http://zsh.sourceforge.net/Doc/Release/Zsh-Modules.html#The-zsh_002fcomputil-Module
Quoted 28 lines · Jacob Keller
Quoted 12 lines · Jakub Narebski
Quoted 10 lines · SZEDER Gábor
Quoted 7 lines · Jakub Narebski
Quoted 4 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.What about ZSH?
Nothing, ZSH is unaffected by this patch.
Quoted 8 lines
All right, so for ZSH we would need LC_ALL=C trick, or come with some equivalent of 'compgen -v
__gitcomp_builtin_' for this shell.Good patch, though it does not solve whole of the problem.
Best,
--
Jakub NarębskiIs ZSH actually affected by the broken set behavior, though?
Thanks,
Jake
Re: [PATCH] completion: reduce overhead of clearing cached --options
In reply to SZEDER GáborQuoted 9 lines · SZEDER Gábor
Quoted 6 lines · Jakub Narebski
Quoted 3 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.What about ZSH?
Nothing, ZSH is unaffected by this patch.
OK, do we want to follow it up with [PATCH 2/1] to add the LC_ALL=C thing to the ZSH side to help that "sed", then?
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · SZEDER Gábor
Quoted 5 lines · Jakub Narebski
Quoted 3 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
What about ZSH?
Nothing, ZSH is unaffected by this patch.
OK, do we want to follow it up with [PATCH 2/1] to add the LC_ALL=C thing to the ZSH side to help that "sed", then?
No. 'sed' would only need need help when its input comes from a buggy 'set' builtin of a particular version of Bash from a particular vendor.
As far as I can test this in Travis CI's OSX builds, ZSH doesn't seem to be affected, neither the version Apple ships by default nor the version installed via homebrew.
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 16 lines · SZEDER Gábor
Quoted 8 lines · Junio C Hamano
Quoted 7 lines · SZEDER Gábor
Quoted 6 lines · Jakub Narebski
Quoted 4 lines · SZEDER Gábor
In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead.Quoted 6 lines
Quoted 3 lines
Quoted 1 line
What about ZSH?
Nothing, ZSH is unaffected by this patch.
OK, do we want to follow it up with [PATCH 2/1] to add the LC_ALL=C thing to the ZSH side to help that "sed", then?
Quoted 6 lines
No. 'sed' would only need need help when its input comes from a buggy 'set' builtin of a particular version of Bash from a particular vendor.
As far as I can test this in Travis CI's OSX builds, ZSH doesn't seem to be affected, neither the version Apple ships by default nor the version installed via homebrew.
That's nice - this means that the patch fixes all of the issue. The above information should be, in my opinion, included in the commit message, though.
Best,
Signature · 1 line
Jakub Narębski
Re: [PATCH] completion: reduce overhead of clearing cached --options
Quoted 11 lines · Jakub Narębski
Quoted 6 lines · SZEDER Gábor
No. 'sed' would only need need help when its input comes from a buggy 'set' builtin of a particular version of Bash from a particular vendor.
As far as I can test this in Travis CI's OSX builds, ZSH doesn't seem to be affected, neither the version Apple ships by default nor the version installed via homebrew.
That's nice - this means that the patch fixes all of the issue. The above information should be, in my opinion, included in the commit message, though.
Yeah, I tend to agree. Thanks.
PATCH v2 1/1 completion: reduce overhead of clearing cached --options
To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them, 8b0eaa41f2 (completion:
clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessary overhead with the extra processes.
In Bash we can do better: run the 'compgen -v __gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead. ZSH will still continue to run that pipeline.
This change also happens to work around an issue in the default Bash version shipped in macOS (3.2.57), reported by users of the Powerline shell prompt, which was triggered by the same commit 8b0eaa41f2 as well. Powerline uses several Unicode Private Use Area code points to represent some of its pretty text UI elements (arrows and what not), and these are stored in the $PS1 variable. Apparently the 'set' builtin of said Bash version on macOS has issues with these code points, and produces garbled output where Powerline's special symbols should be in the $PS1 variable. This, in turn, triggers the following error message in the downstream 'sed' process:
sed: RE error: illegal byte sequence
Other Bash versions, notably 4.4.19 on macOS via homebrew (i.e. a newer version on the same platform) and 3.2.25 on CentOS (i.e. a slightly earlier version, though on a different platform) are not affected. ZSH in macOS (the versions shipped by default or installed via homebrew) or on other platforms isn't affected either.
With this patch neither the 'set' builtin is invoked to print garbage, nor 'sed' to choke on it.
---
Updated the commit message to explicitly mention that ZSH is unaffected. The patch is the same.
contrib/completion/git-completion.bash | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
contrib/completion/git-completion.bash +5 -1
@@ -282,7 +282,11 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +if [[ -n ${ZSH_VERSION-} ]]; then + unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +else + unset $(compgen -v __gitcomp_builtin_) +fi # This function is equivalent to #
Signature · 1 line
2.17.0.366.gbe216a3084
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Quoted 12 lines · SZEDER Gábor
To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them,
8b0eaa41f2(completion:
clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessary overhead with the extra processes.In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead. ZSH will still continue to run that pipeline.
Quoted 4 lines
This change also happens to work around an issue in the default Bash
...
Updated the commit message to explicitly mention that ZSH is unaffected. The patch is the same.
Thanks.
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
I haven't seen any discussion about this recently. What are the chances of getting it merged? I'd like to see this included in 2.18.
Quoted 18 lines
Quoted 12 lines
To get the names of all '$__git_builtin_*' variables caching --options of builtin commands in order to unset them,
8b0eaa41f2(completion:
clear cached --options when sourcing the completion script, 2018-03-22) runs a 'set |sed s///' pipeline. This works both in Bash and in ZSH, but has a higher than necessary overhead with the extra processes.In Bash we can do better: run the 'compgen -v
__gitcomp_builtin_' builtin command, which lists the same variables, but without a pipeline and 'sed' it can do so with lower overhead. ZSH will still continue to run that pipeline.Quoted 4 lines
This change also happens to work around an issue in the default Bash
...
Updated the commit message to explicitly mention that ZSH is unaffected. The patch is the same.Thanks.
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Hi Matthew,
Quoted 3 lines · Matthew Coleman
I haven't seen any discussion about this recently. What are the chances of getting it merged? I'd like to see this included in 2.18.
Junio's last few "What's cooking" updates have mentioned it:
Quoted 9 lines
* sg/completion-clear-cached (2018-04-18) 1 commit (merged to 'next' on 2018-04-25 at
9178da6c3d)
+ completion: reduce overhead of clearing cached --optionsThe completion script (in contrib/) learned to clear cached list of command line options upon dot-sourcing it again in a more efficient way.
Will merge to 'master'.
I imagine it will show up on master sometime soon, in time for 2.18.0.
Signature · 5 lines
Todd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I expected times like this -- but never thought they'd be so bad, so
long, and so frequent.
-- Demotivators (www.despair.com)Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Quoted 13 lines · Todd Zullinger
Hi Matthew,
Quoted 3 lines · Matthew Coleman
I haven't seen any discussion about this recently. What are the chances of getting it merged? I'd like to see this included in 2.18.
Junio's last few "What's cooking" updates have mentioned it:
Quoted 4 lines
* sg/completion-clear-cached (2018-04-18) 1 commit (merged to 'next' on 2018-04-25 at
9178da6c3d)
+ completion: reduce overhead of clearing cached --options
Quoted 8 lines
Quoted 5 lines
The completion script (in contrib/) learned to clear cached list of command line options upon dot-sourcing it again in a more efficient way.
Will merge to 'master'.
I imagine it will show up on master sometime soon, in time for 2.18.0.
Yup, I do not foresee at this moment why that shouldn't be the case.
Thanks.
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
In reply to SZEDER GáborHi,
Quoted 5 lines · SZEDER Gábor
Other Bash versions, notably 4.4.19 on macOS via homebrew (i.e. a newer version on the same platform) and 3.2.25 on CentOS (i.e. a slightly earlier version, though on a different platform) are not affected. ZSH in macOS (the versions shipped by default or installed via homebrew) or on other platforms isn't affected either.
[...]
contrib/completion/git-completion.bash __gitcomp () · line 289
Quoted 12 lines
contrib/completion/git-completion.bash +5
-1@@ -282,7 +282,11 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +if [[ -n ${ZSH_VERSION-} ]]; then + unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +else + unset $(compgen -v __gitcomp_builtin_) +fi
As discussed at [1], this fails catastrophically when sourced by git-completion.zsh, which explicitly clears ZSH_VERSION. By catastrophically, I mean that Rick van Hattem and Dave Borowitz report that it segfaults.
I can't reproduce it since I am having trouble following the instructions at the top of git-completion.zsh to get the recommended zsh completion script loading mechanism working at all. (I've succeeded in using git's bash completion on zsh before, but that was many years ago.) First attempt:
1. rm -fr ~/.zsh ~/.zshrc
# you can move them out of the way instead for a less destructive
# reproduction recipe
2. zsh
3. hit "q"
4. zstyle ':completion:*:*:git:*' script \
$(pwd)/contrib/completion/git-completion.zsh
5. git checkout <TAB>Expected result: segfault
Actual result: succeeds, using zsh's standard completion script (not git's).
I also tried
1. rm -fr ~/.zsh ~/.zshrc
# you can move them out of the way instead for a less destructive
# reproduction recipe
2. zsh
3. hit "2"
4. mkdir ~/.zsh; cp contrib/completion/git-completion.zsh ~/.zsh/_git
5. echo 'fpath=(~/.zsh $fpath)' >>~/.zshrc
6. ^D; zsh
7. git checkout <TAB>and a similar process with fpath earlier in the zshrc file. Whatever I try, I end up with one of two results: either it uses zsh's standard completion, or it spews a stream of errors about missing functions like compgen. What am I doing wrong?
Related question: what would it take to add a zsh completion sanity check to t/t9902-completion.sh?
When running with "set -x", here is what Dave Borowitz gets before the segfault. I don't have a stacktrace because, as mentioned above, I haven't been able to reproduce it.
str=git [[ -n git ]] service=git i= _compskip=default eval '' ret=0 [[ default = *patterns* ]] [[ default = all ]] str=/usr/bin/git [[ -n /usr/bin/git ]] service=_dispatch:70: bad math expression: operand expected at `/usr/bin/g...' zsh: segmentation fault zsh
Here's a minimal fix, untested. As a followup, as Gábor suggests at [2], it would presumably make sense to stop overriding ZSH_VERSION, using this GIT_ scoped variable everywhere instead.
Thoughts?
[1] https://public-inbox.org/git/01020163c683e753-04629405-15f8-4a30-9dc3-e4e3f2a5aa26-000000@eu-west-1.amazonses.com/
[2] https://public-inbox.org/git/[email protected]/
contrib/completion/git-completion.bash +1 -1
@@ -348,7 +348,7 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} || -n ${GIT_SOURCING_ZSH_COMPLETION-} ]]; then unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null else unset $(compgen -v __gitcomp_builtin_)
contrib/completion/git-completion.zsh +1 -1
@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=1 ZSH_VERSION='' . "$script" __gitcomp () {
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Quoted 4 lines · Jonathan Nieder
Whatever
I try, I end up with one of two results: either it uses zsh's standard completion, or it spews a stream of errors about missing functions like compgen. What am I doing wrong?
Try adding to the top of your .zshrc:
autoload -U compinit; compinit
autoload -U bashcompinit; bashcompinit
Those are both in my .zshrc, and I think they are sufficient magic to define compgen.
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
In reply to Jonathan NiederThe zsh script expects the bash completion script to be available so that might be the issue here.
To reproduce this is what I do. First, a sparse checkout:
# mkdir ~/git # cd ~/git # git init # git remote add origin [email protected]:git/git.git # git config core.sparseCheckout true # mkdir .git/info # echo contrib/completion >> .git/info/sparse-checkout # git pull origin master --depth 1
After that I simply link the zsh script to _git:
# cd git/contrib/completion # ln git-completion.zsh _git
And add the following to a new .zshrc file:
autoload -U compinit; compinit
autoload -U bashcompinit; bashcompinit
fpath=(~/git/contrib/completion $fpath)
And that appears to be enough to reproduce:
# git<tab>
git/contrib/completion/git-completion.bash:354: read-only variable: QISUFFIX zsh:12: command not found: ___main
zsh:15: _default: function definition file not found _dispatch:70: bad math expression: operand expected at `/usr/bin/g...'
zsh: segmentation fault zsh
~rick
Quoted 126 lines · Jonathan Nieder
Hi,
Quoted 5 lines · SZEDER Gábor
Other Bash versions, notably 4.4.19 on macOS via homebrew (i.e. a newer version on the same platform) and 3.2.25 on CentOS (i.e. a slightly earlier version, though on a different platform) are not affected. ZSH in macOS (the versions shipped by default or installed via homebrew) or on other platforms isn't affected either.
[...]
contrib/completion/git-completion.bash __gitcomp () · line 289
Quoted 12 lines
contrib/completion/git-completion.bash +5
-1@@ -282,7 +282,11 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +if [[ -n ${ZSH_VERSION-} ]]; then + unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +else + unset $(compgen -v __gitcomp_builtin_) +fiAs discussed at [1], this fails catastrophically when sourced by git-completion.zsh, which explicitly clears
ZSH_VERSION. By catastrophically, I mean that Rick van Hattem and Dave Borowitz report that it segfaults.I can't reproduce it since I am having trouble following the instructions at the top of git-completion.zsh to get the recommended zsh completion script loading mechanism working at all. (I've succeeded in using git's bash completion on zsh before, but that was many years ago.) First attempt:
1. rm -fr ~/.zsh ~/.zshrc
# you can move them out of the way instead for a less destructive # reproduction recipe 2. zsh 3. hit "q" 4. zstyle ':completion:*:*:git:*' script \ $(pwd)/contrib/completion/git-completion.zsh 5. git checkout <TAB>Expected result: segfault
Actual result: succeeds, using zsh's standard completion script (not git's).
I also tried
1. rm -fr ~/.zsh ~/.zshrc
# you can move them out of the way instead for a less destructive # reproduction recipe 2. zsh 3. hit "2" 4. mkdir ~/.zsh; cp contrib/completion/git-completion.zsh ~/.zsh/_git 5. echo 'fpath=(~/.zsh $fpath)' >>~/.zshrc 6. ^D; zsh 7. git checkout <TAB>and a similar process with fpath earlier in the zshrc file. Whatever I try, I end up with one of two results: either it uses zsh's standard completion, or it spews a stream of errors about missing functions like compgen. What am I doing wrong?
Related question: what would it take to add a zsh completion sanity check to
t/t9902-completion.sh?When running with "set -x", here is what Dave Borowitz gets before the segfault. I don't have a stacktrace because, as mentioned above, I haven't been able to reproduce it.
str=git [[ -n git ]] service=git i= _compskip=default eval '' ret=0 [[ default = *patterns* ]] [[ default = all ]] str=/usr/bin/git [[ -n /usr/bin/git ]] service=_dispatch:70: bad math expression: operand expected at `/usr/bin/g...' zsh: segmentation fault zshHere's a minimal fix, untested. As a followup, as Gábor suggests at [2], it would presumably make sense to stop overriding
ZSH_VERSION, using thisGIT_scoped variable everywhere instead.Thoughts?
Reported-by Rick van Hattem <[email protected]>Reported-by Dave Borowitz <[email protected]>Signed-off-by Jonathan Nieder <[email protected]>[1] https://public-inbox.org/git/01020163c683e753-04629405-15f8-4a30-9dc3-e4e3f2a5aa26-000000@eu-west-1.amazonses.com/
[2] https://public-inbox.org/git/[email protected]/contrib/completion/git-completion.bash +1
-1@@ -348,7 +348,7 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} || -n ${GIT_SOURCING_ZSH_COMPLETION-} ]]; then unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null else unset $(compgen -v __gitcomp_builtin_)contrib/completion/git-completion.zsh +1
-1@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=1 ZSH_VERSION='' . "$script" __gitcomp () {
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
In reply to Jonathan NiederQuoted 3 lines
Related question: what would it take to add a zsh completion sanity check to
t/t9902-completion.sh?
I don't know. What I do know is that we can't just run our tests with ZSH, e.g. running 'zsh ./t0000-basic.sh' shows mostly failures. So it won't be as simple as modifying 't/lib-bash.sh' to somehow support ZSH as well.
Quoted 32 lines
Here's a minimal fix, untested. As a followup, as G�bor suggests at [2], it would presumably make sense to stop overriding
ZSH_VERSION, using thisGIT_scoped variable everywhere instead.Thoughts?
Reported-by Rick van Hattem <[email protected]>Reported-by Dave Borowitz <[email protected]>Signed-off-by Jonathan Nieder <[email protected]>[1] https://public-inbox.org/git/01020163c683e753-04629405-15f8-4a30-9dc3-e4e3f2a5aa26-000000@eu-west-1.amazonses.com/
[2] https://public-inbox.org/git/[email protected]/contrib/completion/git-completion.bash +1
-1@@ -348,7 +348,7 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} || -n ${GIT_SOURCING_ZSH_COMPLETION-} ]]; then unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null else unset $(compgen -v __gitcomp_builtin_)contrib/completion/git-completion.zsh +0
-0
contrib/completion/git-completion.zsh if [ -z "$script" ]; then · line 45
Quoted 8 lines
contrib/completion/git-completion.zsh +1
-1@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=1 ZSH_VERSION='' . "$script" __gitcomp () {
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
contrib/completion/git-completion.bash +4 -1
@@ -3244,7 +3244,10 @@ __gitk_main () __git_complete_revlist } -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z "${GIT_SOURCING_ZSH_COMPLETION}" ]] ; then echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinit
contrib/completion/git-completion.zsh +1 -1
@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script" __gitcomp () {
Jonathan Nieder
Reviewed-byRe: [PATCH v2] completion: reduce overhead of clearing cached --options
Hi,
Quoted 9 lines · SZEDER Gábor
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
contrib/completion/git-completion.bash +0
-0
contrib/completion/git-completion.bash __gitk_main () · line 3250
Quoted 8 lines
contrib/completion/git-completion.bash +4
-1@@ -3244,7 +3244,10 @@ __gitk_main () __git_complete_revlist } -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z "${GIT_SOURCING_ZSH_COMPLETION}" ]] ; then
Needs a - before the } to avoid errors in a shell where the user has chosen to use "set -u". See v1.7.4-rc0~159 (completion: fix zsh check under bash with 'set -u', 2010-10-27) for more details.
Quoted 8 lines
contrib/completion/git-completion.bash +0
-0echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinitcontrib/completion/git-completion.zsh +0
-0
contrib/completion/git-completion.zsh if [ -z "$script" ]; then · line 45
Quoted 8 lines
contrib/completion/git-completion.zsh +1
-1@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script" __gitcomp () {
Except for that tweak,
Thanks.
Now it just needs a commit message. :)
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Quoted 16 lines · Jonathan Nieder
Hi,
Quoted 12 lines · SZEDER Gábor
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
contrib/completion/git-completion.bash +0
-0@@ -3244,7 +3244,10 @@ __gitk_main () __git_complete_revlist }
Quoted 9 lines
contrib/completion/git-completion.bash __gitk_main () · line 3250
Quoted 5 lines
contrib/completion/git-completion.bash +4
-1-if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z "${GIT_SOURCING_ZSH_COMPLETION}" ]] ; thenNeeds a - before the } to avoid errors in a shell where the user has chosen to use "set -u". See v1.7.4-rc0~159 (completion: fix zsh check under bash with 'set -u', 2010-10-27) for more details.
Right... I did remember that, but by the time I finished typing out that long variable name I forgot about it... :)
However, I'm not sure it's worth caring about, because the bash-competion scripts don't work with 'set -u' anyway...
Quoted 22 lines
Quoted 8 lines
contrib/completion/git-completion.bash +0
-0echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinitcontrib/completion/git-completion.zsh +0
-0contrib/completion/git-completion.zsh if [ -z "$script" ]; then · line 45
Quoted 8 lines
contrib/completion/git-completion.zsh +1
-1@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script" __gitcomp () {Except for that tweak,
Reviewed-by Jonathan Nieder <[email protected]>Thanks.
Now it just needs a commit message. :)
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Quoted 8 lines · SZEDER Gábor
Quoted 5 lines · Jonathan Nieder
Quoted 1 line
+ [[ -z "${GIT_SOURCING_ZSH_COMPLETION}" ]] ; then
Needs a - before the } to avoid errors in a shell where the user has chosen to use "set -u". See v1.7.4-rc0~159 (completion: fix zsh check under bash with 'set -u', 2010-10-27) for more details.
Quoted 5 lines
Right... I did remember that, but by the time I finished typing out that long variable name I forgot about it... :)
However, I'm not sure it's worth caring about, because the bash-competion scripts don't work with 'set -u' anyway...
I'd be happy to see these '-'s go away (it would make the code both easier to read and easier to maintain), but as a post-2.18 topic, please.
Alternatively, if there's a real need for them, I'd be happy to see the completion tests start using 'set -u'.
Thanks,
Jonathan
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
Quoted 12 lines · Jonathan Nieder
Quoted 11 lines · SZEDER Gábor
Quoted 5 lines · Jonathan Nieder
Quoted 1 line
+ [[ -z "${GIT_SOURCING_ZSH_COMPLETION}" ]] ; then
Needs a - before the } to avoid errors in a shell where the user has chosen to use "set -u". See v1.7.4-rc0~159 (completion: fix zsh check under bash with 'set -u', 2010-10-27) for more details.
Right... I did remember that, but by the time I finished typing out that long variable name I forgot about it... :)
Quoted 6 lines
Quoted 2 lines
However, I'm not sure it's worth caring about, because the bash-competion scripts don't work with 'set -u' anyway...
I'd be happy to see these '-'s go away (it would make the code both easier to read and easier to maintain), but as a post-2.18 topic, please.
Of course.
Quoted 2 lines
Alternatively, if there's a real need for them, I'd be happy to see the completion tests start using 'set -u'.
Our test harness doesn't work with 'set -u' at all...
Re: [PATCH v2] completion: reduce overhead of clearing cached --options
In reply to Jonathan NiederQuoted 40 lines · Jonathan Nieder
Hi,
Quoted 9 lines · SZEDER Gábor
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
contrib/completion/git-completion.bash +0
-0contrib/completion/git-completion.bash __gitk_main () · line 3250
Quoted 8 lines
contrib/completion/git-completion.bash +4
-1@@ -3244,7 +3244,10 @@ __gitk_main () __git_complete_revlist } -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z "${GIT_SOURCING_ZSH_COMPLETION}" ]] ; thenNeeds a - before the } to avoid errors in a shell where the user has chosen to use "set -u". See v1.7.4-rc0~159 (completion: fix zsh check under bash with 'set -u', 2010-10-27) for more details.
Quoted 14 lines
contrib/completion/git-completion.bash +0
-0echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinitcontrib/completion/git-completion.zsh +1
-1@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script"
Quoted 8 lines
contrib/completion/git-completion.zsh if [ -z "$script" ]; then · line 45
Quoted 2 lines
contrib/completion/git-completion.zsh +0
-0__gitcomp () {Except for that tweak,
Reviewed-by Jonathan Nieder <[email protected]>Thanks.
Now it just needs a commit message. :)
OK, can we see the final version of this patch so that we can work it around before 2.18.0-rc2? I am assuming that we are treating this as a Git 2.18-rc regression that needs to be fixed before the final?
Thanks.
PATCH 1/1 completion: correct zsh detection when run from git-completion.zsh (Re: [PATCH v2] completion: reduce overhead of clearing cached --options)
In reply to SZEDER GáborFrom: SZEDER Gábor <[email protected]>
Subject: completion: correct zsh detection when run from git-completion.zsh
v2.18.0-rc0~90^2 (completion: reduce overhead of clearing cached --options, 2018-04-18) worked around a bug in bash's "set" builtin on MacOS by using compgen instead. It was careful to avoid breaking zsh by guarding this workaround with
if [[ -n ${ZSH_VERSION-}} ]]Alas, this interacts poorly with git-completion.zsh's bash emulation:
ZSH_VERSION='' . "$script"
Correct it by instead using a new GIT_SOURCING_ZSH_COMPLETION shell variable to detect whether git-completion.bash is being sourced from git-completion.zsh. This way, the zsh variant is used both when run from zsh directly and when run via git-completion.zsh.
Reproduction recipe:
1. cd git/contrib/completion && cp git-completion.zsh _git
2. Put the following in a new ~/.zshrc file:
autoload -U compinit; compinit autoload -U bashcompinit; bashcompinit fpath=(~/src/git/contrib/completion $fpath)
3. Open zsh and "git <TAB>".
With this patch:
Triggers nice git-completion.bash based tab completion
Without:
contrib/completion/git-completion.bash:354: read-only variable: QISUFFIX zsh:12: command not found: ___main
zsh:15: _default: function definition file not found _dispatch:70: bad math expression: operand expected at `/usr/bin/g...' Segmentation fault
---
Quoted 3 lines · SZEDER Gábor
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
Thanks again. May we have your sign-off?
contrib/completion/git-completion.bash | 5 ++++- contrib/completion/git-completion.zsh | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)
contrib/completion/git-completion.bash +4 -1
@@ -3223,7 +3223,10 @@ __gitk_main () __git_complete_revlist } -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z ${GIT_SOURCING_ZSH_COMPLETION-} ]]; then echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinit
contrib/completion/git-completion.zsh +1 -1
@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script" __gitcomp () {
Signature · 1 line
2.18.0.rc1.242.g61856ae69a
Re: [PATCH] completion: correct zsh detection when run from git-completion.zsh (Re: [PATCH v2] completion: reduce overhead of clearing cached --options)
Quoted 43 lines · Jonathan Nieder
From: SZEDER Gábor <[email protected]>
Subject: completion: correct zsh detection when run from git-completion.zshv2.18.0-rc0~90^2 (completion: reduce overhead of clearing cached --options, 2018-04-18) worked around a bug in bash's "set" builtin on MacOS by using compgen instead. It was careful to avoid breaking zsh by guarding this workaround with
if [[ -n ${ZSH_VERSION-}} ]]Alas, this interacts poorly with git-completion.zsh's bash emulation:
ZSH_VERSION='' . "$script"Correct it by instead using a new
GIT_SOURCING_ZSH_COMPLETIONshell variable to detect whether git-completion.bash is being sourced from git-completion.zsh. This way, the zsh variant is used both when run from zsh directly and when run via git-completion.zsh.Reproduction recipe:
1. cd git/contrib/completion && cp git-completion.zsh
_git
2. Put the following in a new ~/.zshrc file:autoload -U compinit; compinit autoload -U bashcompinit; bashcompinit fpath=(~/src/git/contrib/completion $fpath)
3. Open zsh and "git <TAB>".
With this patch:
Triggers nice git-completion.bash based tab completionWithout:
contrib/completion/git-completion.bash:354: read-only variable: QISUFFIX zsh:12: command not found:___main
zsh:15:_default: function definition file not found _dispatch:70: bad math expression: operand expected at `/usr/bin/g...' Segmentation faultReported-by Rick van Hattem <[email protected]>Reported-by Dave Borowitz <[email protected]>Signed-off-by Jonathan Nieder <[email protected]>
Quoted 8 lines
---
Quoted 3 lines · SZEDER Gábor
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
Thanks again. May we have your sign-off?
Sure:
Thank you for the commit message; had a family visit the last couple of days and could not get around to it.
Quoted 36 lines
contrib/completion/git-completion.bash | 5 ++++- contrib/completion/git-completion.zsh | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)contrib/completion/git-completion.bash +4
-1@@ -3223,7 +3223,10 @@ __gitk_main () __git_complete_revlist } -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z ${GIT_SOURCING_ZSH_COMPLETION-} ]]; then echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinitcontrib/completion/git-completion.zsh +1
-2@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script" __gitcomp () { --2.18.0.rc1.242.g61856ae69a
Re: [PATCH] completion: correct zsh detection when run from git-completion.zsh (Re: [PATCH v2] completion: reduce overhead of clearing cached --options)
In reply to Jonathan NiederQuoted 33 lines · Jonathan Nieder
From: SZEDER Gábor <[email protected]>
Subject: completion: correct zsh detection when run from git-completion.zshv2.18.0-rc0~90^2 (completion: reduce overhead of clearing cached --options, 2018-04-18) worked around a bug in bash's "set" builtin on MacOS by using compgen instead. It was careful to avoid breaking zsh by guarding this workaround with
if [[ -n ${ZSH_VERSION-}} ]]Alas, this interacts poorly with git-completion.zsh's bash emulation:
ZSH_VERSION='' . "$script"Correct it by instead using a new
GIT_SOURCING_ZSH_COMPLETIONshell variable to detect whether git-completion.bash is being sourced from git-completion.zsh. This way, the zsh variant is used both when run from zsh directly and when run via git-completion.zsh.Reproduction recipe:
1. cd git/contrib/completion && cp git-completion.zsh
_git
2. Put the following in a new ~/.zshrc file:autoload -U compinit; compinit autoload -U bashcompinit; bashcompinit fpath=(~/src/git/contrib/completion $fpath)
3. Open zsh and "git <TAB>".
With this patch:
Triggers nice git-completion.bash based tab completion
Quoted 6 lines
Without:
contrib/completion/git-completion.bash:354: read-only variable: QISUFFIX zsh:12: command not found:___main
zsh:15:_default: function definition file not found _dispatch:70: bad math expression: operand expected at `/usr/bin/g...' Segmentation fault
Btw, even if ZSH were not to segfault, the resulting shell would be next to useless. The thing is that 'compgen -v foo' is supposed to list only variable names starting with 'foo', at least that's what it does under Bash. In ZSH's Bash emulation, however, it lists _all_ variable names, including such fundamental env vars like PATH, HOME, and IFS. So when ZSH took the wrong if branch and run that
unset $(compgen -v __gitcomp_builtin_)
then it would unset PATH and everything else as well.
Re: [PATCH] completion: correct zsh detection when run from git-completion.zsh (Re: [PATCH v2] completion: reduce overhead of clearing cached --options)
In reply to Jonathan NiederQuoted 42 lines · Jonathan Nieder
Quoted 3 lines · SZEDER Gábor
Being in RC phase, I'm all for aiming for a minimal solution. However, I don't think that the better fix would be erm.. any "less minimal":
Thanks again. May we have your sign-off?
contrib/completion/git-completion.bash | 5 ++++- contrib/completion/git-completion.zsh | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-)contrib/completion/git-completion.bash +4
-1@@ -3223,7 +3223,10 @@ __gitk_main () __git_complete_revlist } -if [[ -n ${ZSH_VERSION-} ]]; then +if [[ -n ${ZSH_VERSION-} ]] && + # Don't define these functions when sourced from 'git-completion.zsh', + # it has its own implementations. + [[ -z ${GIT_SOURCING_ZSH_COMPLETION-} ]]; then echo "WARNING: this script is deprecated, please see git-completion.zsh" 1>&2 autoload -U +X compinit && compinitcontrib/completion/git-completion.zsh +1
-2@@ -39,7 +39,7 @@ if [ -z "$script" ]; then test -f $e && script="$e" && break done fi -ZSH_VERSION='' . "$script" +GIT_SOURCING_ZSH_COMPLETION=y . "$script" __gitcomp () { --
Quoted 1 line
2.18.0.rc1.242.g61856ae69a
The change looks good to me :)
Quoted 10 lines · Stephon Harris
Fixes issue with seeing
sed: RE error: illegal byte sequencewhen running git-completion.bash--- contrib/completion/git-completion.bash | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)contrib/completion/git-completion.bash +0
-0
contrib/completion/git-completion.bash __gitcomp () · line 285
Quoted 5 lines
contrib/completion/git-completion.bash +1
-1@@ -282,7 +282,7 @@ __gitcomp () # Clear the variables caching builtins' options when (re-)sourcing # the completion script. -unset $(set |sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null +unset $(set |LANG=C sed -ne 's/^\(__gitcomp_builtin_[a-zA-Z0-9_][a-zA-Z0-9_]*\)=.*/\1/p') 2>/dev/null
Shouldn't LC_ALL and LANG both set and exported to C, as
(1) ancient systems understand only LANG but not LC_*; and
(2) modern ones that understand both give precedence to LC_ALL over LANG?
If we were to set only one, it is probably more sensible to set LC_ALL, I suspect, but it may be better to set both, which sends a sign to the readers that we know what we are doing ;-)