Skip to contentSkip to conversation

The kernel's ctype

DISCUSSION Rasmus Villemoes 2 participants 2 messages Feb 4, 2015 – Feb 12, 2015

lore
Share & tools
lore
#0 lore raw

Hi,

The kernel's ctype is almost, but not quite, equivalent to latin1. Apart from whether one wants to include the C1 control chars (0x80-0x9f), there are a few other differences. For example, 0xb5 (MICRO SIGN) is, at least according to glibc, both alpha and lower, while the kernel classifies it as punct. A slightly surprising quirk of the kernel's ctype implementation is that toupper() is not idempotent: Both 0xdf (LATIN SMALL LETTER SHARP S) and 0xff (LATIN SMALL LETTER Y WITH DIAERESIS) are correctly classified as lower, but since neither character's uppercase version is representable in latin1, correct toupper() behaviour would be to return the character itself. Instead, we have toupper(0xff) == 0xdf and toupper(0xdf) == 0xbf.

Digging in pre-git history, I see that ctype.c was originally ASCII-only, which I think is the only sane choice. It was changed around 1996, but the commit log that I've found just says "Import 2.0.1", so it's hard to tell what the intention was.

What would break if ctype.c was changed back to ASCII?

Rasmus

#1 lore raw
Quoted 14 lines · Rasmus Villemoes

Hi,

The kernel's ctype is almost, but not quite, equivalent to latin1. Apart from whether one wants to include the C1 control chars (0x80-0x9f), there are a few other differences. For example, 0xb5 (MICRO SIGN) is, at least according to glibc, both alpha and lower, while the kernel classifies it as punct. A slightly surprising quirk of the kernel's ctype implementation is that toupper() is not idempotent: Both 0xdf (LATIN SMALL LETTER SHARP S) and 0xff (LATIN SMALL LETTER Y WITH DIAERESIS) are correctly classified as lower, but since neither character's uppercase version is representable in latin1, correct toupper() behaviour would be to return the character itself. Instead, we have toupper(0xff) == 0xdf and toupper(0xdf) == 0xbf

Quoted 6 lines

Digging in pre-git history, I see that ctype.c was originally ASCII-only, which I think is the only sane choice. It was changed around 1996, but the commit log that I've found just says "Import 2.0.1", so it's hard to tell what the intention was.

What would break if ctype.c was changed back to ASCII?

The implementation of toupper() and tolower() still seems to be assuming that we're dealing with ascii only, so in that regard I don't think that much would break, as it should be broken already.

Shouldn't coccinelle be able to detect ctype usage and use of non-ascii or userland input values?

Frans