Skip to contentSkip to conversation

UTF-8 practically vs. theoretically in the VFS API

DISCUSSION Norman Diamond 2 participants 2 messages Feb 23, 2004 1 message not in the archive

lore
Share & tools
lore
#0 lore raw
Quoted 2 lines · Eric W. Biederman

First it is worth noting that the existing practice is that ttys always use the character set encoding of the user.

Each tty uses the character set encoding of that tty's user. There were times when I needed to have some tty windows open using EUC (ordinary work on that Linux machine) and some tty windows open using SJIS (editing files which would be sent to cellular telephones), in the same X session. They worked.

Quoted 1 line

Even X cut and paste frequently abuses the iso8859-1 range,

I'll take your word for it. I've copied and pasted EUC strings, I've copied and pasted SJIS strings, I don't know if X copy and paste abused EUC or SJIS ranges, but it worked.

One thing I never thought of trying to test is to copy and paste between one tty using EUC and one tty using SJIS.

Quoted 7 lines

Now the work is how to get multiple locales to play nicely with each other. utf-8 and unicode are convenient for that as they preserve the existing assumptions that terminals, filenames, and text files are all using the same character set encoding, even when multiple locales are involved.

So within one machine utf-8 solves the multiple locale problem.

That preserves a nice fiction. If you depend on assuming that fiction, you'll get useless results.

Quoted 4 lines

The rule ``All data that passing through a pseudo-tty is in the character set encoding specified by the locale of the owner of the tty'' seems both reasonable and no significant change from the current status quo.

Yes, that is a return to usability.

Quoted 2 lines

On the wire between two machines I recommend passing unicode characters.

Why should the wire get a different encoding than the user set in the pseudo-tty? Consider TeraTerm. The user tells TeraTerm what character set is in use on the wire, which is the same as the character set in use on the remote side (where sshd or whatever server provides the pseudo-tty). TeraTerm converts between that and the local character set (where the TeraTerm program and window and user get the character set decided for them by someone in Sasazuka or Redmond).

Quoted 1 line

By convention glibc stores unicode values in wchar_t.

That is hard to believe. glibc existed before Unicode did and wchar_t existed before Unicode did. I sure thought that glibc existed in Japan at the time, but I could be wrong, I didn't say this is impossible but merely hard to believe. In commercial Unix systems, wchar_t held either EUC or SJIS depending on the vendor.

As usual I do not even have time to keep up with this thread, so if you have questions then please CC me personally, though I don't know if I'll have time to investigate anything that needs it.

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"ND" == Norman Diamond <[email protected]> writes:

Quoted 1 line · ND> Eric W. Biederman wrote:
Quoted 1 line

Even X cut and paste frequently abuses the iso8859-1 range,

ND> I'll take your word for it. I've copied and pasted EUC ND> strings, I've copied and pasted SJIS strings, I don't know ND> if X copy and paste abused EUC or SJIS ranges, but it ND> worked.

I do not know what Eric means by "abusing the iso8859-1 rnge", but passing X selection between traditional X clients IIRC uses compound text, which is an encoding vaguely similar to ISO-2022, so clients like kterm can convert it back and forth with EUC or SJIS as needed.