isalpha.3: behavior is undefined if c is out-of-range
PATCH 1/1 isalpha.3: behavior is undefined if c is out-of-range
Clarify that the behavior of these functions is undefined if c is neither in the unsigned char range nor EOF.
I copied the added text from toupper.3.
In practice, calling them on out-of-range values - tested with recent glibc/gcc - is simply reading from random process memory - meaning, you either get some garbage, if the memory was readable, or a segmentation fault. See also:
https://stackoverflow.com/questions/65514890/glibcs-isalpha-function-and-the-en-us-utf-8-locale
--- man3/isalpha.3 | 9 +++++++++ 1 file changed, 9 insertions(+)
man3/isalpha.3 +9 -0
@@ -145,6 +145,15 @@ is the special locale object .BR duplocale (3)) or is not a valid locale object handle. .PP +If +.I c +is neither an +.I "unsigned char" +value nor +.BR EOF , +the behavior of these functions +is undefined. +.PP The list below explains the operation of the functions without the "_l" suffix; the functions with the "_l" suffix differ only in using the locale object
--
2.31.1
Best regards,
--
Didi
Hi Yedidyah,
Quoted 5 lines · Yedidyah Bar David
Clarify that the behavior of these functions is undefined if c is neither in the unsigned char range nor EOF.
I copied the added text from toupper.3.
Quoted 8 lines
In practice, calling them on out-of-range values - tested with recent glibc/gcc - is simply reading from random process memory - meaning, you either get some garbage, if the memory was readable, or a segmentation fault. See also:
https://stackoverflow.com/questions/65514890/glibcs-isalpha-function-and-the-en-us-utf-8-locale
Signed-off-by Yedidyah Bar David <[email protected]>
This is already covered by the NOTES section, isn't it? BTW, I'll rename that section to CAVEATS.
Thanks,
Alex
Quoted 31 lines
--- man3/isalpha.3 | 9 +++++++++ 1 file changed, 9 insertions(+)man3/isalpha.3 +9
-1@@ -145,6 +145,15 @@ is the special locale object .BR duplocale (3)) or is not a valid locale object handle. .PP +If +.I c +is neither an +.I "unsigned char" +value nor +.BR EOF , +the behavior of these functions +is undefined. +.PP The list below explains the operation of the functions without the "_l" suffix; the functions with the "_l" suffix differ only in using the locale object --2.31.1
Best regards,
--
Didi
Signature · 2 lines
<http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5
PATCH 1/1 isalpha.3: Move NOTES to CAVEATS, and clarify some Undefined Behavior
---
I suggest this instead.
man3/isalpha.3 | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
man3/isalpha.3 +4 -2
@@ -345,13 +345,15 @@ .SH HISTORY .TP .BR isascii_l () glibc 2.3. -.SH NOTES +.SH CAVEATS The standards require that the argument .I c for these functions is either .B EOF or a value that is representable in the type -.IR "unsigned char" . +.IR "unsigned char" ; +otherwise, +the behavior is undefined. If the argument .I c is of type
Signature · 1 line
2.40.1
Re: [PATCH v1b] isalpha.3: Move NOTES to CAVEATS, and clarify some Undefined Behavior
Quoted 5 lines · Alejandro Colomar
Reported-by Yedidyah Bar David <[email protected]>Signed-off-by Alejandro Colomar <[email protected]>---
I suggest this instead.
lgtm & thanks!
This is something that's often got wrong by folks.
Quoted 4 lines
man3/isalpha.3 | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
Re: [PATCH v1b] isalpha.3: Move NOTES to CAVEATS, and clarify some Undefined Behavior
In reply to Alejandro ColomarQuoted 6 lines · Alejandro Colomar
Reported-by Yedidyah Bar David <[email protected]>Signed-off-by Alejandro Colomar <[email protected]>---
I suggest this instead.
OK for me as well.
Best regards,
Signature · 1 line
Didi
Alejandro Colomar
maintainerPATCH 1/1 isalpha.3: Move NOTES to CAVEATS, and clarify some Undefined Behavior
In reply to Alejandro Colomar---
I'm just resending (now with Acked-by from Sam) to test a change in my git-send-email(1) configuration.
Cheers,
Alex
man3/isalpha.3 | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
man3/isalpha.3 +4 -2
@@ -345,13 +345,15 @@ .SH HISTORY .TP .BR isascii_l () glibc 2.3. -.SH NOTES +.SH CAVEATS The standards require that the argument .I c for these functions is either .B EOF or a value that is representable in the type -.IR "unsigned char" . +.IR "unsigned char" ; +otherwise, +the behavior is undefined. If the argument .I c is of type
Signature · 1 line
2.40.1
Hi,
Quoted 14 lines · Alejandro Colomar
Hi Yedidyah,
Quoted 10 lines · Yedidyah Bar David
Clarify that the behavior of these functions is undefined if c is neither in the unsigned char range nor EOF.
I copied the added text from toupper.3.
In practice, calling them on out-of-range values - tested with recent glibc/gcc - is simply reading from random process memory - meaning, you either get some garbage, if the memory was readable, or a segmentation fault. See also:
Quoted 5 lines
Quoted 3 lines
https://stackoverflow.com/questions/65514890/glibcs-isalpha-function-and-the-en-us-utf-8-locale
Signed-off-by Yedidyah Bar David <[email protected]>This is already covered by the NOTES section, isn't it?
It's _mentioned_ there, correct - but not sure it's covered.
It's also mentioned in toupper.3's NOTES.
I think it's helpful to explicitly say that behavior is undefined in this case. If you feel like doing this inside NOTES, one way or another, ok for me.
Right now, NOTES says what you must do, but not what happens if you don't do that.
It also says that for the common case of using them on signed char, you should explicitly cast to unsigned char, first. It also tries to explain why this is necessary. The explanation explains why it's necessary for compliance with the standard, but not why it's a good thing more generally - latter is not explained,
and AFAICT from reading glibc sources, is not necessary - see e.g. this comment from ctype.h:
These point into arrays of 384, so they can be indexed by any unsigned char' value [0,255]; by EOF (-1); or by any signed char' value [-128,-1). ISO C requires that the ctype functions work for unsigned char' values and for EOF; we also support negative signed char' values for broken old programs.
The real reason why you should not call them on negative values other than EOF - casted to unsigned char or not - is simply that most likely this isn't what you meant to do. But that's not about compliance with the standard...
In my patch, I felt like "something should be done", but when deciding what actually should be done, I decided to simply go with what's in toupper.3, which seems good enough for me.
So, what do you think?
Quoted 2 lines
BTW, I'll
rename that section to CAVEATS.
Not sure all of its current content is a caveat.
Best regards,
Quoted 39 lines
Thanks,
AlexQuoted 27 lines
--- man3/isalpha.3 | 9 +++++++++ 1 file changed, 9 insertions(+)man3/isalpha.3 +9
-1@@ -145,6 +145,15 @@ is the special locale object .BR duplocale (3)) or is not a valid locale object handle. .PP +If +.I c +is neither an +.I "unsigned char" +value nor +.BR EOF , +the behavior of these functions +is undefined. +.PP The list below explains the operation of the functions without the "_l" suffix; the functions with the "_l" suffix differ only in using the locale object --2.31.1
Quoted 4 lines
Best regards,
--
Didi--
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5
Signature · 1 line
Didi
Alejandro Colomar
maintainerHi,
Quoted 17 lines · Yedidyah Bar David
Hi,
Quoted 14 lines · Alejandro Colomar
Hi Yedidyah,
Quoted 10 lines · Yedidyah Bar David
Clarify that the behavior of these functions is undefined if c is neither in the unsigned char range nor EOF.
I copied the added text from toupper.3.
In practice, calling them on out-of-range values - tested with recent glibc/gcc - is simply reading from random process memory - meaning, you either get some garbage, if the memory was readable, or a segmentation fault. See also:
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
https://stackoverflow.com/questions/65514890/glibcs-isalpha-function-and-the-en-us-utf-8-locale
Signed-off-by Yedidyah Bar David <[email protected]>This is already covered by the NOTES section, isn't it?
It's _mentioned_ there, correct - but not sure it's covered.
You're right. That's why I've sent the patch mentioning UB. What do you think about that one? (I now see that you like it).
Quoted 2 lines
It's also mentioned in toupper.3's NOTES.
I'll check that page to see if it needs some simplifying.
Quoted 2 lines
I think it's helpful to explicitly say that behavior is undefined in this case.
Yep.
Quoted 13 lines
If you feel like doing this inside NOTES, one way or another, ok for me.
Right now, NOTES says what you must do, but not what happens if you don't do that.
It also says that for the common case of using them on signed char, you should explicitly cast to unsigned char, first. It also tries to explain why this is necessary. The explanation explains why it's necessary for compliance with the standard, but not why it's a good thing more generally - latter is not explained,
and AFAICT from reading glibc sources, is not necessary - see e.g. this comment fromctype.h:
Quoted 5 lines
These point into arrays of 384, so they can be indexed by any
unsigned char' value [0,255]; by EOF (-1); or by anysigned char' value [-128,-1). ISO C requires that the ctype functions work forunsigned char' values and for EOF; we also support negativesigned char' values for broken old programs.
Consider what happens with character 0xFF. If char is signed, it will be interpreted as -1 (i.e., EOF). We're lucky, because 0xFF is not a meaningful char, so probably all isXXX() functions return false for it, but it's slightly different from EOF semantically. If no locales give a meaning for 0xFF, maybe the cast can be removed from ISO C. I do something different: use -funsigned-char when compiling, so char is effectively unsigned char (except that pointers do not convert automatically).
Quoted 4 lines
The real reason why you should not call them on negative values other than EOF - casted to unsigned char or not - is simply that most likely this isn't what you meant to do. But that's not about compliance with the standard...
I guess the standard was cautious to not make 0xFF a useless char. If that's not an issue, I agree, and these functions could do the conversion internally.
Cheers,
Alex
Quoted 56 lines
In my patch, I felt like "something should be done", but when deciding what actually should be done, I decided to simply go with what's in toupper.3, which seems good enough for me.
So, what do you think?
Quoted 2 lines
BTW, I'll
rename that section to CAVEATS.Not sure all of its current content is a caveat.
Best regards,
Quoted 39 lines
Thanks,
AlexQuoted 27 lines
--- man3/isalpha.3 | 9 +++++++++ 1 file changed, 9 insertions(+)man3/isalpha.3 +9
-1@@ -145,6 +145,15 @@ is the special locale object .BR duplocale (3)) or is not a valid locale object handle. .PP +If +.I c +is neither an +.I "unsigned char" +value nor +.BR EOF , +the behavior of these functions +is undefined. +.PP The list below explains the operation of the functions without the "_l" suffix; the functions with the "_l" suffix differ only in using the locale object --2.31.1
Quoted 4 lines
Best regards,
--
Didi--
<http://www.alejandro-colomar.es/>
GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5
Signature · 2 lines
<http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5
Alejandro Colomar
maintainerQuoted 69 lines · Alejandro Colomar
Hi,
Quoted 17 lines · Yedidyah Bar David
Hi,
Quoted 14 lines · Alejandro Colomar
Hi Yedidyah,
Quoted 10 lines · Yedidyah Bar David
Clarify that the behavior of these functions is undefined if c is neither in the unsigned char range nor EOF.
I copied the added text from toupper.3.
In practice, calling them on out-of-range values - tested with recent glibc/gcc - is simply reading from random process memory - meaning, you either get some garbage, if the memory was readable, or a segmentation fault. See also:
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
https://stackoverflow.com/questions/65514890/glibcs-isalpha-function-and-the-en-us-utf-8-locale
Signed-off-by Yedidyah Bar David <[email protected]>This is already covered by the NOTES section, isn't it?
It's _mentioned_ there, correct - but not sure it's covered.
You're right. That's why I've sent the patch mentioning UB. What do you think about that one? (I now see that you like it).
Quoted 2 lines
It's also mentioned in toupper.3's NOTES.
I'll check that page to see if it needs some simplifying.
Quoted 2 lines
I think it's helpful to explicitly say that behavior is undefined in this case.
Yep.
Quoted 13 lines
If you feel like doing this inside NOTES, one way or another, ok for me.
Right now, NOTES says what you must do, but not what happens if you don't do that.
It also says that for the common case of using them on signed char, you should explicitly cast to unsigned char, first. It also tries to explain why this is necessary. The explanation explains why it's necessary for compliance with the standard, but not why it's a good thing more generally - latter is not explained,
and AFAICT from reading glibc sources, is not necessary - see e.g. this comment fromctype.h:Quoted 5 lines
These point into arrays of 384, so they can be indexed by any
unsigned char' value [0,255]; by EOF (-1); or by anysigned char' value [-128,-1). ISO C requires that the ctype functions work forunsigned char' values and for EOF; we also support negativesigned char' values for broken old programs.Consider what happens with character
0xFF. If char is signed, it will be interpreted as -1 (i.e., EOF). We're lucky, because0xFFis not a meaningful char, so probably allisXXX()functions return false for it, but it's slightly different from EOF semantically. If no locales give a meaning for0xFF, maybe the cast can be removed from ISO C. I do something different: use -funsigned-char when compiling, so char is effectively unsigned char (except that pointers do not convert automatically).Quoted 1 line
Quoted 6 lines
Quoted 3 lines
The real reason why you should not call them on negative values other than EOF - casted to unsigned char or not - is simply that most likely this isn't what you meant to do. But that's not about compliance with the standard...
I guess the standard was cautious to not make
0xFFa useless char. If that's not an issue, I agree, and these functions could do the conversion internally.
To be clear, I'm talking about this:
$ cat iscntrl.c #include <ctype.h> #include <stdbool.h> #include <stdio.h> #include <stdlib.h>
static const char str_bool[2][8] = { "false", "true" };
static inline const char *
strbool(bool x)
{
return str_bool[!!x];
}int
main(void)
{
signed char s = 0xFF;
unsigned char u = 0xFF;
printf("iscntrl(-1): %s\n", strbool(iscntrl(s)));
printf("iscntrl(255): %s\n", strbool(iscntrl(u)));
exit(EXIT_SUCCESS);
}$ ./a.out iscntrl(-1): false iscntrl(255): false
Quoted 3 lines
Cheers,
Alex
Signature · 2 lines
<http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5
Quoted 23 lines · Alejandro Colomar
Hi,
Quoted 19 lines · Yedidyah Bar David
Hi,
Quoted 16 lines · Alejandro Colomar
Hi Yedidyah,
Quoted 12 lines · Yedidyah Bar David
Clarify that the behavior of these functions is undefined if c is neither in the unsigned char range nor EOF.
I copied the added text from toupper.3.
In practice, calling them on out-of-range values - tested with recent glibc/gcc - is simply reading from random process memory - meaning, you either get some garbage, if the memory was readable, or a segmentation fault. See also:
https://stackoverflow.com/questions/65514890/glibcs-isalpha-function-and-the-en-us-utf-8-locale
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Yedidyah Bar David <[email protected]>This is already covered by the NOTES section, isn't it?
It's _mentioned_ there, correct - but not sure it's covered.
You're right. That's why I've sent the patch mentioning UB. What do you think about that one? (I now see that you like it).
Yes, I like it. Thanks!
Thanks also for the explanations/example.
Best regards,
Signature · 1 line
Didi
Hi,
Any update on this? Thanks!
Best regards,
Signature · 1 line
Didi
Alejandro Colomar
maintainerHi Yedidyah,
Quoted 3 lines · Yedidyah Bar David
Hi,
Any update on this? Thanks!
Thanks for the ping. I've applied it now.
Cheers,
Alex
Quoted 2 lines
Best regards,
Signature · 2 lines
<http://www.alejandro-colomar.es/> GPG key fingerprint: A9348594CE31283A826FBDD8D57633D441E25BB5