gtk: Hardcode LC_CTYPE as C.utf-8
PATCH 1/1 gtk: Hardcode LC_CTYPE as C.utf-8
Commit 2cb5d2a4 removed setlocale() for everything except LC_MESSAGES in order to avoid unwanted side effects such as using the wrong decimal separator in generated JSON objects. However, the problem that unsetting LC_CTYPE caused is that non-ASCII characters are considered non-printable now and therefore the GTK menus display question marks for accented letters, Chinese characters etc.
A first attempt to fix this [1] was rejected because even just setting LC_CTYPE to the user's locale (and thereby modifying the semantics of the ctype.h functions) could have unwanted effects that we're not aware of yet.
Recently, however, glibc introduced a new locale "C.utf-8" that just uses UTF-8 as its charset, but otherwise leaves the semantics alone. Just setting the right character set is enough for our use case, so we can just hardcode this one without having to be afraid of nasty side effects.
Older systems that don't have the new locale will continue displaying question marks, but this should fix the problem for most users.
[1] https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03591.html
('Re: gtk: use setlocale() for LC_MESSAGES only')
---
I happened to talk to Mike Fabian of our Internationalization team at devconf.cz and mentioned our problem, and this is the solution that he suggested. I hope we can finally get things back into a non-broken state with this. :-)
ui/gtk.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
ui/gtk.c +5 -1
@@ -2205,8 +2205,12 @@ void gtk_display_init(DisplayState *ds, bool full_screen, bool grab_on_hover) s->free_scale = FALSE; -/* LC_MESSAGES only. See early_gtk_display_init() for details */ +/* Mostly LC_MESSAGES only. See early_gtk_display_init() for details. For +* LC_CTYPE, we need to make sure that non-ASCII characters are considered +* printable, but without changing any of the character classes to make +* sure that we don't accidentally break implicit assumptions. */ setlocale (LC_MESSAGES ,"" ); +setlocale (LC_CTYPE ,"C.utf-8" );bindtextdomain ("qemu" ,CONFIG_QEMU_LOCALEDIR );textdomain ("qemu" );
Signature · 1 line
2.9.3
Quoted 9 lines · Kevin Wolf
Recently, however, glibc introduced a new locale "C.utf-8" that just uses UTF-8 as its charset, but otherwise leaves the semantics alone. Just setting the right character set is enough for our use case, so we can just hardcode this one without having to be afraid of nasty side effects.
setlocale(LC_MESSAGES, ""); + setlocale(LC_CTYPE, "C.utf-8"); bindtextdomain("qemu", CONFIG_QEMU_LOCALEDIR);
A couple of quick questions:
- Is it C.utf-8 or C.UTF-8 ? 'locale -a' shows only the latter in my system.
- When was this added? This bug seems to be still open:
https://sourceware.org/bugzilla/show_bug.cgi?id=17318
Berto
Alberto Garcia <[email protected]> さんはかきました:
Quoted 8 lines · Kevin Wolf
Quoted 6 lines
Recently, however, glibc introduced a new locale "C.utf-8" that just uses UTF-8 as its charset, but otherwise leaves the semantics alone. Just setting the right character set is enough for our use case, so we can just hardcode this one without having to be afraid of nasty side effects.
Quoted 8 lines
Quoted 3 lines
setlocale(LC_MESSAGES, ""); + setlocale(LC_CTYPE, "C.utf-8"); bindtextdomain("qemu", CONFIG_QEMU_LOCALEDIR);A couple of quick questions:
- Is it C.utf-8 or C.UTF-8 ? 'locale -a' shows only the latter in my system.
Both work:
mfabian@taka:~
$ LC_ALL=C.utf-8 strace -eopen ls 2>&1 | grep LC_CTYPE
open("/usr/lib/locale/C.utf-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/C.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = 3
mfabian@taka:~
$ LC_ALL=C.UTF-8 strace -eopen ls 2>&1 | grep LC_CTYPE
open("/usr/lib/locale/C.UTF-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/C.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = 3
mfabian@taka:~
$ LC_ALL=C.UTF8 strace -eopen ls 2>&1 | grep LC_CTYPE
open("/usr/lib/locale/C.UTF8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/locale/C.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = 3
mfabian@taka:~
$ LC_ALL=C.utf8 strace -eopen ls 2>&1 | grep LC_CTYPE
open("/usr/lib/locale/C.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = 3
mfabian@taka:~
$ I like C.UTF-8 because “UTF-8” is the official spelling of that encoding:
https://en.wikipedia.org/wiki/UTF-8#Official_name_and_variants
Using “C.utf8” uses one stat less though because it is the last fallback, as you can see in the strace.
Quoted 2 lines
- When was this added? This bug seems to be still open:
https://sourceware.org/bugzilla/show_bug.cgi?id=17318
Fedora has it since Fedora 24 (spring 2016), Debian for a while longer.
I’ll ping again to get it included upstream.
It needs 1.5MB at runtime only because of
https://sourceware.org/bugzilla/show_bug.cgi?id=18978
as soon as that sorting bug is fixed, the C.UTF-8 locale will need less than 200k.
Quoted 1 line
Berto
Signature · 2 lines
Mike FABIAN <[email protected]> 睡眠不足はいい仕事の敵だ。
Quoted 4 lines · Mike FABIAN
Quoted 2 lines
- Is it C.utf-8 or C.UTF-8 ? 'locale -a' shows only the latter in my system.
Both work:
Hmmm... apparently not in my system:
$ LC_CTYPE=C.utf-8 locale locale: Cannot set LC_CTYPE to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory LANG=en_US.utf8 LANGUAGE= LC_CTYPE=C.utf-8 [...]
$ LC_CTYPE=C.UTF-8 locale LANG=en_US.utf8 LANGUAGE= LC_CTYPE=C.UTF-8
I have glibc 2.24-8 (Debian).
Quoted 7 lines
Quoted 2 lines
- When was this added? This bug seems to be still open:
https://sourceware.org/bugzilla/show_bug.cgi?id=17318Fedora has it since Fedora 24 (spring 2016), Debian for a while longer.
I’ll ping again to get it included upstream.
Ah ok, so it's a distro-specific change at the moment.
Thanks,
Berto
Quoted 8 lines · Alberto Garcia
Quoted 5 lines · Kevin Wolf
Recently, however, glibc introduced a new locale "C.utf-8" that just uses UTF-8 as its charset, but otherwise leaves the semantics alone. Just setting the right character set is enough for our use case, so we can just hardcode this one without having to be afraid of nasty side effects.
Quoted 8 lines
Quoted 3 lines
setlocale(LC_MESSAGES, ""); + setlocale(LC_CTYPE, "C.utf-8"); bindtextdomain("qemu", CONFIG_QEMU_LOCALEDIR);A couple of quick questions:
- Is it C.utf-8 or C.UTF-8 ? 'locale -a' shows only the latter in my system.
At least Cygwin has C.UTF-8, but not C.utf-8. Furthermore, since my system defaults to en_US.UTF-8, I would expect the upper-case variant for the character set name across all locales.
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Eric Blake <[email protected]> さんはかきました:
Quoted 13 lines · Alberto Garcia
Quoted 12 lines · Kevin Wolf
Quoted 10 lines
Recently, however, glibc introduced a new locale "C.utf-8" that just uses UTF-8 as its charset, but otherwise leaves the semantics alone. Just setting the right character set is enough for our use case, so we can just hardcode this one without having to be afraid of nasty side effects.
setlocale(LC_MESSAGES, ""); + setlocale(LC_CTYPE, "C.utf-8"); bindtextdomain("qemu", CONFIG_QEMU_LOCALEDIR);
Quoted 8 lines
Quoted 4 lines
A couple of quick questions:
- Is it C.utf-8 or C.UTF-8 ? 'locale -a' shows only the latter in my system.
At least Cygwin has C.UTF-8, but not C.utf-8. Furthermore, since my system defaults to en_US.UTF-8, I would expect the upper-case variant for the character set name across all locales.
Yes, I think it is more compatible to use the C.UTF-8 spelling because it uses the “official” spelling of “UTF-8”.
Signature · 2 lines
Mike FABIAN <[email protected]> 睡眠不足はいい仕事の敵だ。
Quoted 5 lines
Recently, however, glibc introduced a new locale "C.utf-8" that just uses UTF-8 as its charset, but otherwise leaves the semantics alone. Just setting the right character set is enough for our use case, so we can just hardcode this one without having to be afraid of nasty side effects.
Cool. Added to ui queue.
thanks,
Gerd.