UTF-8 fixes in comments
Quoted 2 lines · Willy Tarreau
3) if I enter Alt-196, I get a "Ä". Flushing the buffer shows that od got two bytes: c3 84.
Confirmed.
Try init=/bin/stty -a, that will show
-iutf8
So there is little wonder that canonical mode does not work as expected.
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
Samuel
Quoted 3 lines · Samuel Thibault
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
You may also need to select a UTF-8 locale (e.g. LANG="en_US.UTF-8") for programs like bash to handle this correctly.
Signature · 3 lines
Chris Adams <[email protected]> Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble.
Quoted 7 lines · Chris Adams
Quoted 3 lines · Samuel Thibault
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
You may also need to select a UTF-8 locale (e.g. LANG="en_US.UTF-8") for programs like bash to handle this correctly.
Yes of course, but here the purpose was _not_ programs like bash, but the canonical mode (i.e. programs like cat etc.), for which the LANG variable has no effect, only iutf8 has.
Samuel
Quoted 6 lines · Samuel Thibault
Quoted 5 lines · Chris Adams
Quoted 3 lines · Samuel Thibault
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
Quoted 6 lines
Quoted 2 lines
You may also need to select a UTF-8 locale (e.g. LANG="en_US.UTF-8") for programs like bash to handle this correctly.
Yes of course, but here the purpose was _not_ programs like bash, but the canonical mode (i.e. programs like cat etc.), for which the LANG variable has no effect, only iutf8 has.
exactly, thanks for understanding my problem Samuel :-)
Willy
Quoted 8 lines · Samuel Thibault
Quoted 2 lines · Willy Tarreau
3) if I enter Alt-196, I get a "Ä". Flushing the buffer shows that od got two bytes: c3 84.
Confirmed.
Try init=/bin/stty -a, that will show
Quoted 7 lines
-iutf8
So there is little wonder that canonical mode does not work as expected.
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
Will try that on a more recent install. Mine's stty does not support this option. Your analysis makes quite a lot of sense, and such a fix would wipe part of my annoyances/anger with this recent change.
Thanks,
Willy
Hello,
Quoted 13 lines · Willy Tarreau, le Wed 30 Apr 2008 21:49:20 +0200, a écrit :
Quoted 12 lines · Samuel Thibault
Quoted 2 lines · Willy Tarreau
3) if I enter Alt-196, I get a "Ä". Flushing the buffer shows that od got two bytes: c3 84.
Confirmed.
Try init=/bin/stty -a, that will show
-iutf8
So there is little wonder that canonical mode does not work as expected.
Quoted 7 lines
Quoted 3 lines
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
Will try that on a more recent install. Mine's stty does not support this option. Your analysis makes quite a lot of sense, and such a fix would wipe part of my annoyances/anger with this recent change.
Can you give the patch below a try?
Dynamic per-VT utf-8 switch should also work, provided that you reopen the VT (i.e. log out).
Samuel
Set IUTF8 as appropriate on VT tty open.
drivers/char/vt.c renamed from drivers/char/vt.c.orig +6 -0
@@ -2723,6 +2723,10 @@ static int con_open(struct tty_struct *t tty->winsize.ws_row = vc_cons[currcons].d->vc_rows; tty->winsize.ws_col = vc_cons[currcons].d->vc_cols; } +if (vc->vc_utf) + tty->termios->c_iflag |=IUTF8 ; +else + tty->termios->c_iflag &= ~IUTF8 ;release_console_sem ();vcs_make_sysfs (tty);return ret;
@@ -2899,6 +2903,8 @@ int __init vty_init(void) console_driver->minor_start =1 ; console_driver->type =TTY_DRIVER_TYPE_CONSOLE ; console_driver->init_termios = tty_std_termios; +if (default_utf8) + console_driver->init_termios.c_iflag |=IUTF8 ; console_driver->flags =TTY_DRIVER_REAL_RAW |TTY_DRIVER_RESET_TERMIOS ;tty_set_operations (console_driver, &con_ops);if (tty_register_driver (console_driver))
Hi Samuel,
Quoted 3 lines · Samuel Thibault
Can you give the patch below a try?
Dynamic per-VT utf-8 switch should also work, provided that you reopen the VT (i.e. log out).
I confirm that your patch works perfectly for me. Now backspace correctly removes multi-byte characters. My bash is still fooled though but as Alan explained it, it's readline which has to be upgraded now.
Thanks!
Willy
Fix VT canonical input in UTF-8 mode [Was: UTF-8 fixes in comments]
Quoted 25 lines · Samuel Thibault, le Sun 04 May 2008 00:50:27 +0100, a écrit :
Quoted 13 lines · Willy Tarreau, le Wed 30 Apr 2008 21:49:20 +0200, a écrit :
Quoted 12 lines · Samuel Thibault
Quoted 2 lines · Willy Tarreau
3) if I enter Alt-196, I get a "Ä". Flushing the buffer shows that od got two bytes: c3 84.
Confirmed.
Try init=/bin/stty -a, that will show
-iutf8
So there is little wonder that canonical mode does not work as expected.
Quoted 7 lines
Quoted 3 lines
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
Will try that on a more recent install. Mine's stty does not support this option. Your analysis makes quite a lot of sense, and such a fix would wipe part of my annoyances/anger with this recent change.
Can you give the patch below a try?
Dynamic per-VT utf-8 switch should also work, provided that you reopen the VT (i.e. log out).
Quoted 3 lines · Willy Tarreau, le Sun 04 May 2008 10:55:14 +0200, a écrit :
I confirm that your patch works perfectly for me. Now backspace correctly removes multi-byte characters. My bash is still fooled though but as Alan explained it, it's readline which has to be upgraded now.
I guess this is suitable for the stable trees of 2.6.24 and 2.6.25 (where UTF-8 is by default now).
Set IUTF8 as appropriate on VT tty open.
drivers/char/vt.c renamed from drivers/char/vt.c.orig +6 -0
@@ -2723,6 +2723,10 @@ static int con_open(struct tty_struct *t tty->winsize.ws_row = vc_cons[currcons].d->vc_rows; tty->winsize.ws_col = vc_cons[currcons].d->vc_cols; } +if (vc->vc_utf) + tty->termios->c_iflag |=IUTF8 ; +else + tty->termios->c_iflag &= ~IUTF8 ;release_console_sem ();vcs_make_sysfs (tty);return ret;
@@ -2899,6 +2903,8 @@ int __init vty_init(void) console_driver->minor_start =1 ; console_driver->type =TTY_DRIVER_TYPE_CONSOLE ; console_driver->init_termios = tty_std_termios; +if (default_utf8) + console_driver->init_termios.c_iflag |=IUTF8 ; console_driver->flags =TTY_DRIVER_REAL_RAW |TTY_DRIVER_RESET_TERMIOS ;tty_set_operations (console_driver, &con_ops);if (tty_register_driver (console_driver))
Re: Fix VT canonical input in UTF-8 mode [Was: UTF-8 fixes in comments]
Quoted 7 lines · Samuel Thibault
Quoted 3 lines · Willy Tarreau, le Sun 04 May 2008 10:55:14 +0200, a écrit :
I confirm that your patch works perfectly for me. Now backspace correctly removes multi-byte characters. My bash is still fooled though but as Alan explained it, it's readline which has to be upgraded now.
I guess this is suitable for the stable trees of 2.6.24 and 2.6.25 (where UTF-8 is by default now).
agreed.
Quoted 3 lines
Set IUTF8 as appropriate on VT tty open.
Signed-off-by Samuel Thibault <[email protected]>
You should have added: CC: [email protected] here so that the stable team automatically gets notified when it's merged into mainline.
Thanks!
Willy
Re: Fix VT canonical input in UTF-8 mode [Was: UTF-8 fixes in comments]
In reply to Samuel ThibaultQuoted 38 lines · On Sun, 4 May 2008 11:25:54 +0100 Samuel Thibault…
Quoted 25 lines · Samuel Thibault, le Sun 04 May 2008 00:50:27 +0100, a écrit :
Quoted 13 lines · Willy Tarreau, le Wed 30 Apr 2008 21:49:20 +0200, a écrit :
Quoted 12 lines · Samuel Thibault
Quoted 2 lines · Willy Tarreau
3) if I enter Alt-196, I get a "Ä". Flushing the buffer shows that od got two bytes: c3 84.
Confirmed.
Try init=/bin/stty -a, that will show
-iutf8
So there is little wonder that canonical mode does not work as expected.
Quoted 7 lines
Quoted 3 lines
Try init=/bin/sh, from that shell run stty iutf8. Then things will work fine. The fix is thus just to make the VT's tty initial iutf8 setup follow vt.default_utf8.
Will try that on a more recent install. Mine's stty does not support this option. Your analysis makes quite a lot of sense, and such a fix would wipe part of my annoyances/anger with this recent change.
Can you give the patch below a try?
Dynamic per-VT utf-8 switch should also work, provided that you reopen the VT (i.e. log out).Quoted 3 lines · Willy Tarreau, le Sun 04 May 2008 10:55:14 +0200, a écrit :
I confirm that your patch works perfectly for me. Now backspace correctly removes multi-byte characters. My bash is still fooled though but as Alan explained it, it's readline which has to be upgraded now.
I guess this is suitable for the stable trees of 2.6.24 and 2.6.25 (where UTF-8 is by default now).
Quoted 3 lines
Set IUTF8 as appropriate on VT tty open.
Signed-off-by Samuel Thibault <[email protected]>
That changelog is pretty darn terse :( I'll often go through the email ladder and try to extract the missing information but this time I don't really see it there.
Things like: what is the kernel's current behaviour, why does it behave that way, how does the patch fix it?
Thanks.
Re: Fix VT canonical input in UTF-8 mode [Was: UTF-8 fixes in comments]
Quoted 5 lines · Andrew Morton, le Mon 05 May 2008 16:00:44 -0700, a écrit :
Quoted 3 lines
Set IUTF8 as appropriate on VT tty open.
Signed-off-by Samuel Thibault <[email protected]>That changelog is pretty darn terse :(
Erf, sorry.
Quoted 6 lines
I'll often go through
the email ladder and try to extract the missing information but this time I don't really see it there.Things like: what is the kernel's current behaviour, why does it behave that way, how does the patch fix it?
Well, it's more an implementation than a fix. Let's try again:
For e.g. proper TTY canonical support, IUTF8 termios flag has to be set as appropriate. Linux used to not care about setting that flag for VT TTYs.
This patch fixes that by activating it according to the current mode of the VT, and sets the default value according to the vt.default_utf8 parameter.
Samuel