lib: vsprintf: scanf: Negative number must have field width > 1
Subsystems (3)
PATCH v5 1/4 lib: vsprintf: scanf: Negative number must have field width > 1
If a signed number field starts with a '-' the field width must be > 1, or unlimited, to allow at least one digit after the '-'.
This patch adds a check for this. If a signed field starts with '-' and field_width == 1 the scanf will quit.
It is ok for a signed number field to have a field width of 1 if it starts with a digit. In that case the single digit can be converted.
--- lib/vsprintf.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
lib/vsprintf.c +5 -1
@@ -3434,8 +3434,12 @@ int vsscanf(const char *buf, const char *fmt, va_list args) str =skip_spaces (str); digit = *str; -if (is_sign && digit =='-' ) +if (is_sign && digit =='-' ) { +if (field_width ==1 ) +break ; + digit = *(str +1 ); + }if (!digit || (base == 16 && !isxdigit(digit))
Signature · 1 line
2.20.1
PATCH v5 2/4 lib: vsprintf: Fix handling of number field widths in vsscanf
The existing code attempted to handle numbers by doing a strto[u]l(), ignoring the field width, and then repeatedly dividing to extract the field out of the full converted value. If the string contains a run of valid digits longer than will fit in a long or long long, this would overflow and no amount of dividing can recover the correct value.
This patch fixes vsscanf() to obey number field widths when parsing the number.
A new _parse_integer_limit() is added that takes a limit for the number of characters to parse. The number field conversion in vsscanf is changed to use this new function.
If a number starts with a radix prefix, the field width must be long enough for at last one digit after the prefix. If not, it will be handled like this:
sscanf("0x4", "%1i", &i): i=0, scanning continues with the 'x' sscanf("0x4", "%2i", &i): i=0, scanning continues with the '4'
This is consistent with the observed behaviour of userland sscanf.
Note that this patch does NOT fix the problem of a single field value overflowing the target type. So for example:
sscanf("123456789abcdef", "%x", &i);Will not produce the correct result because the value obviously overflows INT_MAX. But sscanf will report a successful conversion.
---
Changed since v3:
- Consistently use SIZE_MAX as the "infinity" value when passing to size_t arguments.
- Use while-loop instead of for-loop in _parse_integer_limit().
- Keep the existing arguments for _parse_integer() on their original line. And the corresponding arguments to _parse_integer_limit() formatted/wrapped the same way as _parse_integer().
- Remove redundant check for (max_chars == 0) in simple_strntoull().
- Fixed "vsscanf" -> "vsscanf()" in commit message.
--- lib/kstrtox.c | 13 ++++++-- lib/kstrtox.h | 2 ++ lib/vsprintf.c | 82 +++++++++++++++++++++++++++++--------------------- 3 files changed, 59 insertions(+), 38 deletions(-)
lib/kstrtox.c +10 -3
@@ -39,20 +39,22 @@ const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)/* * Convert non-negative integer string representation in explicitly given radix -* to an integer. +* to an integer. A maximum of max_chars characters will be converted. +* * Return number of characters consumed maybe or-ed with overflow bit. * If overflow occurs, result integer (incorrect) is still returned. * * Don't you dare use this function. */ -unsigned int _parse_integer (const char *s,unsigned int base,unsigned long long *p) +unsigned int _parse_integer_limit (const char *s,unsigned int base,unsigned long long *p, +size_t max_chars) {unsigned long long res;unsigned int rv; res =0 ; rv =0 ; -while (1 ) { +while (max_chars--) {unsigned int c = *s;unsigned int lc = c |0x20 ;/* don't tolower() this line */ unsigned int val;
@@ -82,6 +84,11 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long longreturn rv; } +unsigned int _parse_integer (const char *s,unsigned int base,unsigned long long *p) +{ +return _parse_integer_limit (s, base, p,SIZE_MAX ); +} +static int _kstrtoull (const char *s,unsigned int base,unsigned long long *res) {unsigned long long _res;
lib/kstrtox.h +2 -0
@@ -4,6 +4,8 @@#define KSTRTOX_OVERFLOW (1U << 31)const char *_parse_integer_fixup_radix (const char *s,unsigned int *base); +unsigned int _parse_integer_limit (const char *s,unsigned int base,unsigned long long *res, +size_t max_chars);unsigned int _parse_integer (const char *s,unsigned int base,unsigned long long *res);#endif
lib/vsprintf.c +47 -35
124 lines not on this page · raw message
Signature · 1 line
2.20.1
Andy Shevchenko
reviewerRe: [PATCH v5 2/4] lib: vsprintf: Fix handling of number field widths in vsscanf
Quoted 22 lines · Richard Fitzgerald
The existing code attempted to handle numbers by doing a strto[u]l(), ignoring the field width, and then repeatedly dividing to extract the field out of the full converted value. If the string contains a run of valid digits longer than will fit in a long or long long, this would overflow and no amount of dividing can recover the correct value.
This patch fixes
vsscanf()to obey number field widths when parsing the number.A new
_parse_integer_limit()is added that takes a limit for the number of characters to parse. The number field conversion in vsscanf is changed to use this new function.If a number starts with a radix prefix, the field width must be long enough for at last one digit after the prefix. If not, it will be handled like this:
sscanf("0x4", "%1i", &i): i=0, scanning continues with the 'x'sscanf("0x4", "%2i", &i): i=0, scanning continues with the '4'This is consistent with the observed behaviour of userland sscanf.
Quoted 7 lines
Note that this patch does NOT fix the problem of a single field value overflowing the target type. So for example:
sscanf("123456789abcdef", "%x", &i);Will not produce the correct result because the value obviously overflows
INT_MAX. But sscanf will report a successful conversion.
I have a few nit-picks, but it's up to you and maintainers how to proceed.
...
Quoted 18 lines
lib/vsprintf.c +11
-4-unsigned long long simple_strtoull (const char *cp,char **endp,unsigned int base) +static unsigned long long simple_strntoull (const char *startp,size_t max_chars, +char **endp,unsigned int base) { -unsigned long long result; +const char *cp; +unsigned long long result =0ULL ;unsigned int rv; - cp =_parse_integer_fixup_radix (cp, &base); - rv =_parse_integer (cp, base, &result); + cp =_parse_integer_fixup_radix (startp, &base); +if ((cp - startp) >= max_chars) { + cp = startp + max_chars; + goto out; + } + + max_chars -= (cp - startp);
lib/vsprintf.c line 79
Quoted 10 lines
lib/vsprintf.c +2
-0+ rv =_parse_integer_limit (cp, base, &result, max_chars);/* FIXME */ cp += (rv & ~KSTRTOX_OVERFLOW ); +out:if (endp) *endp = (char *)cp;return result; }
A nit-pick: What if we rewrite above as
static unsigned long long simple_strntoull(const char *cp, size_t max_chars,
char **endp, unsigned int base)
{
unsigned long long result = 0ULL;
const char *startp = cp;
unsigned int rv;
size_t chars;
cp = _parse_integer_fixup_radix(cp, &base);
chars = cp - startp;
if (chars >= max_chars) {
/* We hit the limit */
cp = startp + max_chars;
} else {
rv = _parse_integer_limit(cp, base, &result, max_chars - chars);
/* FIXME */
cp += (rv & ~KSTRTOX_OVERFLOW);
}
if (endp)
*endp = (char *)cp;
return result;
}...
lib/vsprintf.c long simple_strtol(const char *cp, char **endp, unsigned int base) · line 131
Quoted 6 lines
lib/vsprintf.c +6
-0+static long long simple_strntoll (const char *cp,size_t max_chars,char **endp, +unsigned int base) +{ +/* +* simple_strntoull safely handles receiving max_chars==0 in the +* case we start with max_chars==1 and find a '-' prefix.
A nit-pick: Spaces surrounding '=='? simple_strntoull -> simple_strntoull()?
Review
Quoted 1 line
(quoted diff) +1
-0+ */
Above misses to add something like:
"Otherwise we hit the '-' as an illegal number in the following simple_strntoull() call."
Quoted 7 lines
lib/vsprintf.c +5
-0+if (*cp =='-' && max_chars >0 ) +return -simple_strntoull (cp +1 , max_chars -1 , endp, base); + +return simple_strntoull (cp, max_chars, endp, base); +}
...
lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3478
Quoted 3 lines
lib/vsprintf.c +3
-0+ val.s =simple_strntoll (str, + field_width >0 ? field_width :SIZE_MAX , + &next, base);
A nit-pick: Wouldn't be negative field_width "big enough" to just being used as is? Also, is field_width == 0 should be treated as "parse to the MAX"?
...
lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3482
Quoted 3 lines
lib/vsprintf.c +3
-0+ val.u =simple_strntoull (str, + field_width >0 ? field_width :SIZE_MAX , + &next, base);
Ditto.
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [PATCH v5 2/4] lib: vsprintf: Fix handling of number field widths in vsscanf
Quoted 92 lines · Andy Shevchenko
Quoted 22 lines · Richard Fitzgerald
The existing code attempted to handle numbers by doing a strto[u]l(), ignoring the field width, and then repeatedly dividing to extract the field out of the full converted value. If the string contains a run of valid digits longer than will fit in a long or long long, this would overflow and no amount of dividing can recover the correct value.
This patch fixes
vsscanf()to obey number field widths when parsing the number.A new
_parse_integer_limit()is added that takes a limit for the number of characters to parse. The number field conversion in vsscanf is changed to use this new function.If a number starts with a radix prefix, the field width must be long enough for at last one digit after the prefix. If not, it will be handled like this:
sscanf("0x4", "%1i", &i): i=0, scanning continues with the 'x'sscanf("0x4", "%2i", &i): i=0, scanning continues with the '4'This is consistent with the observed behaviour of userland sscanf.
Quoted 7 lines
Note that this patch does NOT fix the problem of a single field value overflowing the target type. So for example:
sscanf("123456789abcdef", "%x", &i);Will not produce the correct result because the value obviously overflows
INT_MAX. But sscanf will report a successful conversion.I have a few nit-picks, but it's up to you and maintainers how to proceed.
...
Quoted 18 lines
lib/vsprintf.c +11
-4-unsigned long long simple_strtoull (const char *cp,char **endp,unsigned int base) +static unsigned long long simple_strntoull (const char *startp,size_t max_chars, +char **endp,unsigned int base) { -unsigned long long result; +const char *cp; +unsigned long long result =0ULL ;unsigned int rv; - cp =_parse_integer_fixup_radix (cp, &base); - rv =_parse_integer (cp, base, &result); + cp =_parse_integer_fixup_radix (startp, &base); +if ((cp - startp) >= max_chars) { + cp = startp + max_chars; + goto out; + } + + max_chars -= (cp - startp);lib/vsprintf.c line 79
Quoted 10 lines
lib/vsprintf.c +2
-0+ rv =_parse_integer_limit (cp, base, &result, max_chars);/* FIXME */ cp += (rv & ~KSTRTOX_OVERFLOW ); +out:if (endp) *endp = (char *)cp;return result; }A nit-pick: What if we rewrite above as
static unsigned long long simple_strntoull(const char *cp,
size_tmax_chars,
char **endp, unsigned int base){ unsigned long long result = 0ULL; const char *startp = cp; unsigned int rv; size_t chars; cp = _parse_integer_fixup_radix(cp, &base); chars = cp - startp; if (chars >= max_chars) { /* We hit the limit */ cp = startp + max_chars; } else { rv = _parse_integer_limit(cp, base, &result, max_chars - chars); /* FIXME */ cp += (rv & ~KSTRTOX_OVERFLOW); } if (endp) *endp = (char *)cp; return result; }
Quoted 1 line
...
I don't mind rewriting that code if you prefer that way.
I am used to working on other kernel subsytems where the preference is to bail out on the error case so that the "normal" case flows without nesting.
Quoted 25 lines
lib/vsprintf.c long simple_strtol(const char *cp, char **endp, unsigned int base) · line 131
Quoted 7 lines
lib/vsprintf.c +6
-0+static long long simple_strntoll (const char *cp,size_t max_chars,char **endp, +unsigned int base) +{ +/* +* simple_strntoull safely handles receiving max_chars==0 in the +* case we start with max_chars==1 and find a '-' prefix. A nit-pick: Spaces surrounding '=='?
simple_strntoull->simple_strntoull()?Review
Quoted 1 line
(quoted diff) +1
-0+ */Above misses to add something like:
"Otherwise we hit the '-' as an illegal number in the following
simple_strntoull()call."lib/vsprintf.c long simple_strtol(const char *cp, char **endp, unsigned int base) · line 137
Quoted 7 lines
lib/vsprintf.c +5
-0+if (*cp =='-' && max_chars >0 ) +return -simple_strntoull (cp +1 , max_chars -1 , endp, base); + +return simple_strntoull (cp, max_chars, endp, base); +}
Quoted 7 lines
...
lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3478
Quoted 3 lines
lib/vsprintf.c +3
-0+ val.s =simple_strntoll (str, + field_width >0 ? field_width :SIZE_MAX , + &next, base);A nit-pick: Wouldn't be negative
field_width"big enough" to just being used as
field_width is s16 so really should be sign-extended to make it "very big". I think this would be less readable what the intention is and what assumptions it is based on. There's a risk someone would look at
(size_t)(long)field_width
and think the (long) is redundant.
Perhaps change field_width to int? There I ask myself "if it can be an int, why is it declared s16?" and worry there is something subtle in the code.
My personal preference is to avoid using tricks in code that isn't time critical.
Quoted 3 lines
is? Also, is
field_width== 0 should be treated as "parse to the MAX"?...
Earlier code terminates scanning if the width parsed from the format string is <= 0. So field_width can only be -1 or > 0 here. But now you point it out, that test would be better as field_width >= 0 ... so it deals with 0 if it ever happened to sneak through to here somehow.
Quoted 7 lines
lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3482
Quoted 4 lines
lib/vsprintf.c +3
-0+ val.u =simple_strntoull (str, + field_width >0 ? field_width :SIZE_MAX , + &next, base);Ditto.
Petr Mladek
maintainerRe: [PATCH v5 2/4] lib: vsprintf: Fix handling of number field widths in vsscanf
Quoted 61 lines · Richard Fitzgerald
Quoted 60 lines · Andy Shevchenko
Quoted 21 lines · Richard Fitzgerald
The existing code attempted to handle numbers by doing a strto[u]l(), ignoring the field width, and then repeatedly dividing to extract the field out of the full converted value. If the string contains a run of valid digits longer than will fit in a long or long long, this would overflow and no amount of dividing can recover the correct value.
lib/vsprintf.c +9
-4-unsigned long long simple_strtoull (const char *cp,char **endp,unsigned int base) +static unsigned long long simple_strntoull (const char *startp,size_t max_chars, +char **endp,unsigned int base) { -unsigned long long result; +const char *cp; +unsigned long long result =0ULL ;unsigned int rv; - cp =_parse_integer_fixup_radix (cp, &base); - rv =_parse_integer (cp, base, &result); + cp =_parse_integer_fixup_radix (startp, &base); +if ((cp - startp) >= max_chars) { + cp = startp + max_chars; + goto out; + }lib/vsprintf.c line 79
Quoted 10 lines
lib/vsprintf.c +4
-0+ + max_chars -= (cp - startp); + rv =_parse_integer_limit (cp, base, &result, max_chars);/* FIXME */ cp += (rv & ~KSTRTOX_OVERFLOW ); +out:if (endp) *endp = (char *)cp;return result; }A nit-pick: What if we rewrite above as
static unsigned long long simple_strntoull(const char *cp,
size_tmax_chars,
char **endp, unsigned int base){ unsigned long long result = 0ULL; const char *startp = cp; unsigned int rv; size_t chars; cp = _parse_integer_fixup_radix(cp, &base); chars = cp - startp; if (chars >= max_chars) { /* We hit the limit */ cp = startp + max_chars; } else { rv = _parse_integer_limit(cp, base, &result, max_chars - chars); /* FIXME */ cp += (rv & ~KSTRTOX_OVERFLOW); } if (endp) *endp = (char *)cp; return result; }
Quoted 7 lines
Quoted 1 line
...
I don't mind rewriting that code if you prefer that way.
I am used to working on other kernel subsytems where the preference is to bail out on the error case so that the "normal" case flows without nesting.
Yeah. But in this case Andy's variant looks slightly better redable to me.
...
Quoted 9 lines
Quoted 6 lines
lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3478
Quoted 4 lines
lib/vsprintf.c +3
-0+ val.s =simple_strntoll (str, + field_width >0 ? field_width :SIZE_MAX , + &next, base);A nit-pick: Wouldn't be negative
field_width"big enough" to just being used as
field_widthis s16 so really should be sign-extended
I guess that Andy just missed that it was a signed type. And it has to be because -1 means SIZE_MAX.
Quoted 6 lines
to make it "very
big". I think this would be less readable what the intention is and what assumptions it is based on. There's a risk someone would look at(size_t)(long)field_width
Quoted 7 lines
and think the (long) is redundant.
Perhaps changefield_widthto int? There I ask myself "if it can be an int, why is it declared s16?" and worry there is something subtle in the code.My personal preference is to avoid using tricks in code that isn't time critical.
I agree. Let's keep the check with signed type.
Quoted 1 line
Quoted 1 line
is? Also, is
field_width== 0 should be treated as "parse to the MAX"?
filed_width == 0 actually means that no characters are read. I should return zero value.
Quoted 4 lines
Quoted 1 line
...
Earlier code terminates scanning if the width parsed from the format string is <= 0.
To make it clear what earlier code means. vsscanf() bail out earlier when field_width == 0. It is handled by this code:
/* get field width */
field_width = -1;
if (isdigit(*fmt)) {
field_width = skip_atoi(&fmt);
if (field_width <= 0)
break;
}Quoted 4 lines
So
field_widthcan only be -1 or > 0 here. But now you point it out, that test would be better asfield_width>= 0 ... so it deals with 0 if it ever happened to sneak through to here somehow.
It might make sense to be proactive and change it to >= 0. But I would do it in a separate patch. The "< 0" condition matches the original code.
Best Regards,
Petr
Petr Mladek
maintainerRe: [PATCH v5 2/4] lib: vsprintf: Fix handling of number field widths in vsscanf
Quoted 53 lines · Petr Mladek
Quoted 29 lines · Richard Fitzgerald
Quoted 28 lines · Andy Shevchenko
On Mon, Feb 08, 2021 at 02:01:52PM +0000, Richard Fitzgerald wrote:
A nit-pick: What if we rewrite above asstatic unsigned long long simple_strntoull(const char *cp,
size_tmax_chars,
char **endp, unsigned int base){ unsigned long long result = 0ULL; const char *startp = cp; unsigned int rv; size_t chars; cp = _parse_integer_fixup_radix(cp, &base); chars = cp - startp; if (chars >= max_chars) { /* We hit the limit */ cp = startp + max_chars; } else { rv = _parse_integer_limit(cp, base, &result, max_chars - chars); /* FIXME */ cp += (rv & ~KSTRTOX_OVERFLOW); } if (endp) *endp = (char *)cp; return result; }Quoted 7 lines
Quoted 1 line
...
I don't mind rewriting that code if you prefer that way.
I am used to working on other kernel subsytems where the preference is to bail out on the error case so that the "normal" case flows without nesting.Yeah. But in this case Andy's variant looks slightly better redable to me.
...
Quoted 9 lines
Quoted 6 lines
lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3478
Quoted 4 lines
lib/vsprintf.c +3
-0+ val.s =simple_strntoll (str, + field_width >0 ? field_width :SIZE_MAX , + &next, base);is? Also, is
field_width== 0 should be treated as "parse to the MAX"?Earlier code terminates scanning if the width parsed from the format string is <= 0.
Quoted 8 lines
Quoted 4 lines
So
field_widthcan only be -1 or > 0 here. But now you point it out, that test would be better asfield_width>= 0 ... so it deals with 0 if it ever happened to sneak through to here somehow.It might make sense to be proactive and change it to >= 0. But I would do it in a separate patch. The "< 0" condition matches the original code.
Ah, I have missed that you have already sent v6 where you did this change in the same patch. There is no need to resend it just because of this. I am going to look at v6.
Best Regards,
Petr
PATCH v5 3/4 lib: test_scanf: Add tests for sscanf number conversion
In reply to Richard FitzgeraldAdds test_sscanf to test various number conversion cases, as number conversion was previously broken.
This also tests the simple_strtoxxx() functions exported from vsprintf.c.
--- MAINTAINERS | 1 + lib/Kconfig.debug | 3 + lib/Makefile | 1 + lib/test_scanf.c | 752 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 757 insertions(+) create mode 100644 lib/test_scanf.c
MAINTAINERS +1 -0
@@ -19090,6 +19090,7 @@ S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk.git F: Documentation/core-api/printk-formats.rst F: lib/test_printf.c +F: lib/test_scanf.c F: lib/vsprintf.c VT1211 HARDWARE MONITOR DRIVER
lib/Kconfig.debug +3 -0
@@ -2077,6 +2077,9 @@ config TEST_KSTRTOX config TEST_PRINTF tristate "Test printf() family of functions at runtime" +config TEST_SCANF + tristate "Test scanf() family of functions at runtime" + config TEST_BITMAP tristate "Test bitmap_*() family of functions at runtime" help
lib/Makefile +1 -0
@@ -85,6 +85,7 @@ obj-$(CONFIG_TEST_USER_COPY) += test_user_copy.o obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_keys.o obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_key_base.o obj-$(CONFIG_TEST_PRINTF) += test_printf.o +obj-$(CONFIG_TEST_SCANF) += test_scanf.o obj-$(CONFIG_TEST_BITMAP) += test_bitmap.o obj-$(CONFIG_TEST_STRSCPY) += test_strscpy.o obj-$(CONFIG_TEST_UUID) += test_uuid.o
lib/test_scanf.c new file +752 -0
752 lines not on this page · raw message
Signature · 1 line
2.20.1
PATCH v5 4/4 selftests: lib: Add wrapper script for test_scanf
In reply to Richard FitzgeraldAdds a wrapper shell script for the test_scanf module.
--- tools/testing/selftests/lib/Makefile | 2 +- tools/testing/selftests/lib/config | 1 + tools/testing/selftests/lib/scanf.sh | 4 ++++ 3 files changed, 6 insertions(+), 1 deletion(-) create mode 100755 tools/testing/selftests/lib/scanf.sh
tools/testing/selftests/lib/Makefile +1 -1
@@ -4,6 +4,6 @@ # No binaries, but make sure arg-less "make" doesn't trigger "run_tests" all: -TEST_PROGS := printf.sh bitmap.sh prime_numbers.sh strscpy.sh +TEST_PROGS := printf.sh bitmap.sh prime_numbers.sh scanf.sh strscpy.sh include ../lib.mk
tools/testing/selftests/lib/config +1 -0
@@ -1,4 +1,5 @@ CONFIG_TEST_PRINTF=m +CONFIG_TEST_SCANTF=m CONFIG_TEST_BITMAP=m CONFIG_PRIME_NUMBERS=m CONFIG_TEST_STRSCPY=m
tools/testing/selftests/lib/scanf.sh new file +4 -0
@@ -0,0 +1,4 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Tests the scanf infrastructure using test_scanf kernel module. +$(dirname $0)/../kselftest/module.sh "scanf" test_scanf
Signature · 1 line
2.20.1