Skip to contentSkip to conversation

String conversions

DISCUSSION Steve French 2 participants 2 messages Apr 28, 2009 – Apr 29, 2009 1 message not in the archive VFS

lore
Share & tools
lore
#0 lore raw

In looking at various patches for more accurately sizing the required buffer needed for conversions to UTF-8, the following question came up more than once.

Functions which do string copying often null terminate the target string (single byte of \0), and size strings one byte larger than their string name (for UCS-2 this does not work since the null
termination is two bytes). Are there local nls codepages in Linux
kernel which require double null termination (e.g. DBCS asian code pages), and if so how do you tell which ones require "double null termination?"

Signature · 3 lines
Thanks,

Steve

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 6 lines

Steve French <[email protected]> wrote:

In looking at various patches for more accurately sizing the required buffer needed for conversions to UTF-8, the following question came up more than once.

Quoted 7 lines

Functions which do string copying often null terminate the target string (single byte of \0), and size strings one byte larger than their string name (for UCS-2 this does not work since the null termination is two bytes). � Are there local nls codepages in Linux kernel which require double null termination (e.g. DBCS asian code pages), and if so how do you tell which ones require "double null termination?"

A look at fs/nls and supported charsets suggests that the linux kernel does not support a pure double-byte charsets. Some of the supported east asian charsets seem to be a superset of ASCII (for e.g TIS 620, Big5) and only non-ASCII characters are expressed in 2 bytes.

Thanks,

Signature · 1 line
Suresh Jayaraman