Skip to contentSkip to conversation

lib: vsprintf: Fix handling of number field widths in vsscanf

PATCH In review Richard Fitzgerald 4 participants 11 messages Nov 30, 2020 – Dec 16, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/4 lib: vsprintf: Fix handling of number field widths in vsscanf

The existing code attempted to handle numbers by doing a strto[u]l(), ignoring the field width, and then bitshifting the field out of the converted value. If the string contains a run of valid digits longer than will fit in a long or long long, this would overflow and no amount of bitshifting can recover the correct value.

This patch fixes vsscanf to obey number field widths when parsing the number.

A new _parse_integer_limit() is added that takes a limit for the number of characters to parse. The number field conversion in vsscanf is changed to use this new function.

The cases of a base prefix or leading '-' that is >= the maximum field width is handled such that the result of a sccanf is consistent with the observed behaviour of userland sscanf.

---
 lib/kstrtox.c  | 13 +++++--
 lib/kstrtox.h  |  2 ++
 lib/vsprintf.c | 93 ++++++++++++++++++++++++++++++--------------------
 3 files changed, 68 insertions(+), 40 deletions(-)
lib/kstrtox.c +10 -3
index a14ccf905055..0ac2ee8bd9d0 100644
@@ -39,20 +39,22 @@ const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
 
 /*
  * Convert non-negative integer string representation in explicitly given radix
- * to an integer.
+ * to an integer. A maximum of max_chars characters will be converted.
+ *
  * Return number of characters consumed maybe or-ed with overflow bit.
  * If overflow occurs, result integer (incorrect) is still returned.
  *
  * Don't you dare use this function.
  */
-unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *p)
+unsigned int _parse_integer_limit(const char *s, unsigned int base,
+				  unsigned long long *p, size_t max_chars)
 {
 	unsigned long long res;
 	unsigned int rv;
 
 	res = 0;
 	rv = 0;
-	while (1) {
+	for (; max_chars > 0; max_chars--) {
 		unsigned int c = *s;
 		unsigned int lc = c | 0x20; /* don't tolower() this line */
 		unsigned int val;
@@ -82,6 +84,11 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long
 	return rv;
 }
 
+unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *p)
+{
+	return _parse_integer_limit(s, base, p, INT_MAX);
+}
+
 static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 {
 	unsigned long long _res;
lib/kstrtox.h +2 -0
index 3b4637bcd254..4c6536f85cac 100644
@@ -4,6 +4,8 @@
 
 #define KSTRTOX_OVERFLOW	(1U << 31)
 const char *_parse_integer_fixup_radix(const char *s, unsigned int *base);
+unsigned int _parse_integer_limit(const char *s, unsigned int base,
+				  unsigned long long *res, size_t max_chars);
 unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *res);
 
 #endif
lib/vsprintf.c +56 -37

141 lines not on this page · raw message

Signature · 1 line
2.20.1
#5 lore raw

PATCH v2 3/4 selftests: lib: Add wrapper script for test_scanf

Adds a wrapper shell script for the test_scanf module.

---
 tools/testing/selftests/lib/Makefile | 2 +-
 tools/testing/selftests/lib/config   | 1 +
 tools/testing/selftests/lib/scanf.sh | 4 ++++
 3 files changed, 6 insertions(+), 1 deletion(-)
 create mode 100755 tools/testing/selftests/lib/scanf.sh
tools/testing/selftests/lib/Makefile +1 -1
index a105f094676e..ee71fc99d5b5 100644
@@ -4,6 +4,6 @@
 # No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
 all:
 
-TEST_PROGS := printf.sh bitmap.sh prime_numbers.sh strscpy.sh
+TEST_PROGS := printf.sh bitmap.sh prime_numbers.sh scanf.sh strscpy.sh
 
 include ../lib.mk
tools/testing/selftests/lib/config +1 -0
index b80ee3f6e265..776c8c42e78d 100644
@@ -1,4 +1,5 @@
 CONFIG_TEST_PRINTF=m
+CONFIG_TEST_SCANTF=m
 CONFIG_TEST_BITMAP=m
 CONFIG_PRIME_NUMBERS=m
 CONFIG_TEST_STRSCPY=m
tools/testing/selftests/lib/scanf.sh new file +4 -0
new file mode 100755
index 000000000000..b59b8ba561c3
@@ -0,0 +1,4 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Tests the scanf infrastructure using test_scanf kernel module.
+$(dirname $0)/../kselftest/module.sh "scanf" test_scanf
Signature · 1 line
2.20.1
#1 lore raw

PATCH v2 2/4 lib: test_scanf: Add tests for sscanf number conversion

In reply to Richard Fitzgerald

Adds test_sscanf to test various number conversion cases, as number conversion was previously broken.

This also tests the simple_strtoxxx() functions exported from vsprintf.c.

---
 lib/Kconfig.debug |   3 +
 lib/Makefile      |   1 +
 lib/test_scanf.c  | 747 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 751 insertions(+)
 create mode 100644 lib/test_scanf.c
lib/Kconfig.debug +3 -0
index c789b39ed527..b934dca35a0c 100644
@@ -2054,6 +2054,9 @@ config TEST_KSTRTOX
 config TEST_PRINTF
 	tristate "Test printf() family of functions at runtime"
 
+config TEST_SCANF
+	tristate "Test scanf() family of functions at runtime"
+
 config TEST_BITMAP
 	tristate "Test bitmap_*() family of functions at runtime"
 	help
lib/Makefile +1 -0
index ce45af50983a..091463b8986a 100644
@@ -85,6 +85,7 @@ obj-$(CONFIG_TEST_USER_COPY) += test_user_copy.o
 obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_keys.o
 obj-$(CONFIG_TEST_STATIC_KEYS) += test_static_key_base.o
 obj-$(CONFIG_TEST_PRINTF) += test_printf.o
+obj-$(CONFIG_TEST_SCANF) += test_scanf.o
 obj-$(CONFIG_TEST_BITMAP) += test_bitmap.o
 obj-$(CONFIG_TEST_STRSCPY) += test_strscpy.o
 obj-$(CONFIG_TEST_UUID) += test_uuid.o
lib/test_scanf.c new file +747 -0

747 lines not on this page · raw message

Signature · 1 line
2.20.1
#7 lore raw

PATCH v2 4/4 MAINTAINERS: Add lib/test_scanf.c to VSPRINTF

In reply to Richard Fitzgerald

Adds the new scanf test to the VSPRINTF group.

---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index 7887d2161be4..d2bf38bd3d0c 100644
@@ -18782,6 +18782,7 @@ S:	Maintained
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk.git
 F:	Documentation/core-api/printk-formats.rst
 F:	lib/test_printf.c
+F:	lib/test_scanf.c
 F:	lib/vsprintf.c
 
 VT1211 HARDWARE MONITOR DRIVER
Signature · 1 line
2.20.1
#10 lore raw
In reply to Richard Fitzgerald
Quoted 9 lines · Richard Fitzgerald

The existing code attempted to handle numbers by doing a strto[u]l(), ignoring the field width, and then bitshifting the field out of the converted value. If the string contains a run of valid digits longer than will fit in a long or long long, this would overflow and no amount of bitshifting can recover the correct value.

This patch fixes vsscanf to obey number field widths when parsing the number.

Quoted 6 lines

A new _parse_integer_limit() is added that takes a limit for the number of characters to parse. The number field conversion in vsscanf is changed to use this new function.

The cases of a base prefix or leading '-' that is >= the maximum field

I have troubles to parse this sentence. It might be because I am not a native speaker.

Quoted 2 lines

width is handled such that the result of a sccanf is consistent with the observed behaviour of userland sscanf.

Anyway, it would be great to explain this on few examples that describe the corner cases. See also below.

Quoted 79 lines
lib/kstrtox.c +5 -3
@@ -39,20 +39,22 @@ const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
 
 /*
  * Convert non-negative integer string representation in explicitly given radix
- * to an integer.
+ * to an integer. A maximum of max_chars characters will be converted.
+ *
  * Return number of characters consumed maybe or-ed with overflow bit.
  * If overflow occurs, result integer (incorrect) is still returned.
  *
  * Don't you dare use this function.
  */
-unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long *p)
+unsigned int _parse_integer_limit(const char *s, unsigned int base,
+				  unsigned long long *p, size_t max_chars)
 {
 	unsigned long long res;
 	unsigned int rv;
 
 	res = 0;
 	rv = 0;
-	while (1) {
+	for (; max_chars > 0; max_chars--) {
 		unsigned int c = *s;
 		unsigned int lc = c | 0x20; /* don't tolower() this line */
 		unsigned int val;
lib/vsprintf.c +20 -13
index 14c9a6af1b23..21145da468e0 100644
@@ -53,29 +53,47 @@
 #include <linux/string_helpers.h>
 #include "kstrtox.h"
 
-/**
- * simple_strtoull - convert a string to an unsigned long long
- * @cp: The start of the string
- * @endp: A pointer to the end of the parsed string will be placed here
- * @base: The number base to use
- *
- * This function has caveats. Please use kstrtoull instead.
- */
-unsigned long long simple_strtoull(const char *cp, char **endp, unsigned int base)
+static unsigned long long simple_strntoull(const char *startp, size_t max_chars,
+					   char **endp, unsigned int base)
 {
-	unsigned long long result;
+	const char *cp;
+	unsigned long long result = 0ULL;
 	unsigned int rv;
 
-	cp = _parse_integer_fixup_radix(cp, &base);
-	rv = _parse_integer(cp, base, &result);
+	if (max_chars == 0) {
+		cp = startp;
+		goto out;
+	}
+
+	cp = _parse_integer_fixup_radix(startp, &base);
+	if ((cp - startp) >= max_chars) {
+		cp = startp + max_chars;
+		goto out;
+	}
+
+	max_chars -= (cp - startp);
+	rv = _parse_integer_limit(cp, base, &result, max_chars);
 	/* FIXME */
 	cp += (rv & ~KSTRTOX_OVERFLOW);
-
+out:
 	if (endp)
 		*endp = (char *)cp;
 
 	return result;
 }
+
+/**

lib/vsprintf.c line 95

Quoted 10 lines
lib/vsprintf.c +10 -0
+ * simple_strtoull - convert a string to an unsigned long long
+ * @cp: The start of the string
+ * @endp: A pointer to the end of the parsed string will be placed here
+ * @base: The number base to use
+ *
+ * This function has caveats. Please use kstrtoull instead.
+ */
+unsigned long long simple_strtoull(const char *cp, char **endp, unsigned int base)
+{
+	return simple_strntoull(cp, UINT_MAX, endp, base);

Please, use INT_MAX everywhere. It is an arbitrary big-enough number. And INT_MAX is already used in vscnprintf().

lib/vsprintf.c EXPORT_SYMBOL(simple_strtoull); · line 109

Quoted 10 lines
lib/vsprintf.c +2 -1
+}
 EXPORT_SYMBOL(simple_strtoull);
 
 /**
@@ -88,7 +106,7 @@ EXPORT_SYMBOL(simple_strtoull);
  */
 unsigned long simple_strtoul(const char *cp, char **endp, unsigned int base)
 {
-	return simple_strtoull(cp, endp, base);
+	return simple_strntoull(cp, UINT_MAX, endp, base);

Same here.

Quoted 24 lines
lib/vsprintf.c +13 -0
 }
 EXPORT_SYMBOL(simple_strtoul);
 
@@ -109,6 +127,19 @@ long simple_strtol(const char *cp, char **endp, unsigned int base)
 }
 EXPORT_SYMBOL(simple_strtol);
 
+static long long simple_strntoll(const char *cp, size_t max_chars, char **endp,
+				 unsigned int base)
+{
+	/*
+	 * simple_strntoull safely handles receiving max_chars==0 in the
+	 * case we start with max_chars==1 and find a '-' prefix.
+	 */
+	if (*cp == '-' && max_chars > 0)
+		return -simple_strntoull(cp + 1, max_chars - 1, endp, base);
+
+	return simple_strntoull(cp, max_chars, endp, base);
+}
+
 /**
  * simple_strtoll - convert a string to a signed long long
  * @cp: The start of the string

lib/vsprintf.c EXPORT_SYMBOL(simple_strtol); · line 153

Quoted 8 lines
lib/vsprintf.c +1 -4
@@ -119,10 +150,7 @@ EXPORT_SYMBOL(simple_strtol);
  */
 long long simple_strtoll(const char *cp, char **endp, unsigned int base)
 {
-	if (*cp == '-')
-		return -simple_strtoull(cp + 1, endp, base);
-
-	return simple_strtoull(cp, endp, base);
+	return simple_strntoll(cp, UINT_MAX, endp, base);

And here.

lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3466

Quoted 11 lines
lib/vsprintf.c +3 -1
 }
 EXPORT_SYMBOL(simple_strtoll);
 
@@ -3433,8 +3461,11 @@ int vsscanf(const char *buf, const char *fmt, va_list args)
 		str = skip_spaces(str);
 
 		digit = *str;
-		if (is_sign && digit == '-')
+		if (is_sign && digit == '-') {
+			if (field_width == 1)
+				break;

This should be handled in a separate patch. It is a subtle change that is hidden deep inside a big diff.

1. It is quite hard to notice because the above simple_strntoll() was implemented to return 0 in this case.

2. The commit message describes the situation when all read numbers overflow unsigned long long. And this is another corner that was not clearly mentioned.

lib/vsprintf.c int vsscanf(const char *buf, const char *fmt, va_list args) · line 3471

Quoted 5 lines
lib/vsprintf.c +1 -0
 			digit = *(str + 1);
+		}
 
 		if (!digit
 		    || (base == 16 && !isxdigit(digit))

Otherwise, I really like patch. Thanks a lot for the effort.

I am sorry that it took me so long to look at it. I was pretty busy last week. I am going to look at the huge 2nd patch tomorrow.

Best Regards,
Petr

maintainer
#2 lore raw

Re: [PATCH v2 2/4] lib: test_scanf: Add tests for sscanf number conversion

In reply to Richard Fitzgerald
Quoted 5 lines · Richard Fitzgerald

Adds test_sscanf to test various number conversion cases, as number conversion was previously broken.

This also tests the simple_strtoxxx() functions exported from vsprintf.c.

It is impressive.

Honestly, I do not feel to be expert on testing and mathematics. I am not sure how comprehensive the test is. Also I am not sure what experts would say about the tricks with random numbers.

Anyway, this is much more than what I have expected. And it checks great number of variants and corner cases.

I suggest only one small change, see below.

Quoted 60 lines
lib/test_scanf.c new file +57 -0
@@ -0,0 +1,747 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test cases for sscanf facility.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitops.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/printk.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include "../tools/testing/selftests/kselftest_module.h"
+
+#define BUF_SIZE 1024
+
+static unsigned total_tests __initdata;
+static unsigned failed_tests __initdata;
+static char *test_buffer __initdata;
+static char *fmt_buffer __initdata;
+static struct rnd_state rnd_state __initdata;
+
+typedef int (*check_fn)(const void *check_data, const char *string,
+			const char *fmt, int n_args, va_list ap);
+
+static void __scanf(4, 6) __init
+_test(check_fn fn, const void *check_data, const char *string, const char *fmt,
+	int n_args, ...)
+{
+	va_list ap, ap_copy;
+	int ret;
+
+	total_tests++;
+
+	va_start(ap, n_args);
+	va_copy(ap_copy, ap);
+	ret = vsscanf(string, fmt, ap_copy);
+	va_end(ap_copy);
+
+	if (ret != n_args) {
+		pr_warn("vsscanf(\"%s\", \"%s\", ...) returned %d expected %d\n",
+			string, fmt, ret, n_args);
+		goto fail;
+	}
+
+	ret = (*fn)(check_data, string, fmt, n_args, ap);
+	if (ret)
+		goto fail;
+
+	va_end(ap);
+
+	return;

lib/test_scanf.c line 67

Quoted 10 lines
lib/test_scanf.c new file +10 -0
+
+fail:
+	failed_tests++;
+	va_end(ap);
+}
+
+#define test_one_number(T, gen_fmt, scan_fmt, val, fn)			\
+do {									\
+	const T expect_val = (T)(val);					\
+	T result = ~expect_val; /* should be overwritten */		\

If I get it correctly, this is supposed to initialize the temporary variable with a value that is different from the expected value. It will cause test failure when it is not updated by vsscanf().

It does not work for zero value. A better solution might be to add a constant, for example:

	T result = expect_val + 3; /* do not match when not overwritten */ \

I did not use "+ 1" intentionally because it might hide some overflow issues.

lib/test_scanf.c line 201

Quoted 4 lines
lib/test_scanf.c new file +4 -0
+									\
+	snprintf(test_buffer, BUF_SIZE, gen_fmt, expect_val);		\
+	_test(fn, &expect_val, test_buffer, "%" scan_fmt, 1, &result);	\
+} while (0)

Otherwise, it looks good to me.

Best Regards,
Petr

#3 lore raw

Re: [PATCH v2 2/4] lib: test_scanf: Add tests for sscanf number conversion

In reply to Petr Mladek
Quoted 94 lines · Petr Mladek
Quoted 5 lines · Richard Fitzgerald

Adds test_sscanf to test various number conversion cases, as number conversion was previously broken.

This also tests the simple_strtoxxx() functions exported from vsprintf.c.

It is impressive.

Honestly, I do not feel to be expert on testing and mathematics. I am not sure how comprehensive the test is. Also I am not sure what experts would say about the tricks with random numbers.

Anyway, this is much more than what I have expected. And it checks great number of variants and corner cases.

I suggest only one small change, see below.

Quoted 60 lines
lib/test_scanf.c new file +57 -0
@@ -0,0 +1,747 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Test cases for sscanf facility.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/bitops.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/overflow.h>
+#include <linux/printk.h>
+#include <linux/random.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+
+#include "../tools/testing/selftests/kselftest_module.h"
+
+#define BUF_SIZE 1024
+
+static unsigned total_tests __initdata;
+static unsigned failed_tests __initdata;
+static char *test_buffer __initdata;
+static char *fmt_buffer __initdata;
+static struct rnd_state rnd_state __initdata;
+
+typedef int (*check_fn)(const void *check_data, const char *string,
+			const char *fmt, int n_args, va_list ap);
+
+static void __scanf(4, 6) __init
+_test(check_fn fn, const void *check_data, const char *string, const char *fmt,
+	int n_args, ...)
+{
+	va_list ap, ap_copy;
+	int ret;
+
+	total_tests++;
+
+	va_start(ap, n_args);
+	va_copy(ap_copy, ap);
+	ret = vsscanf(string, fmt, ap_copy);
+	va_end(ap_copy);
+
+	if (ret != n_args) {
+		pr_warn("vsscanf(\"%s\", \"%s\", ...) returned %d expected %d\n",
+			string, fmt, ret, n_args);
+		goto fail;
+	}
+
+	ret = (*fn)(check_data, string, fmt, n_args, ap);
+	if (ret)
+		goto fail;
+
+	va_end(ap);
+
+	return;

lib/test_scanf.c line 67

Quoted 10 lines
lib/test_scanf.c new file +10 -0
+
+fail:
+	failed_tests++;
+	va_end(ap);
+}
+
+#define test_one_number(T, gen_fmt, scan_fmt, val, fn)			\
+do {									\
+	const T expect_val = (T)(val);					\
+	T result = ~expect_val; /* should be overwritten */		\

If I get it correctly, this is supposed to initialize the temporary variable with a value that is different from the expected value. It will cause test failure when it is not updated by vsscanf().

Quoted 1 line

It does not work for zero value. A better solution might be to add

That's a ~, not a -
~0 = 0xFFFFFFFF
~-1 = 0

Quoted 17 lines

a constant, for example:

	T result = expect_val + 3; /* do not match when not overwritten */ \

I did not use "+ 1" intentionally because it might hide some overflow issues.

lib/test_scanf.c line 201

Quoted 4 lines
lib/test_scanf.c new file +4 -0
+									\
+	snprintf(test_buffer, BUF_SIZE, gen_fmt, expect_val);		\
+	_test(fn, &expect_val, test_buffer, "%" scan_fmt, 1, &result);	\
+} while (0)

Otherwise, it looks good to me.

Best Regards,
Petr

maintainer
#9 lore raw

Re: [PATCH v2 4/4] MAINTAINERS: Add lib/test_scanf.c to VSPRINTF

In reply to Petr Mladek
Quoted 10 lines · Petr Mladek
Quoted 3 lines · Richard Fitzgerald

Adds the new scanf test to the VSPRINTF group.

I would prefer to squash this into the 2nd patch that adds the file. But anyway:

Agree with Petr. Squashing this with patch 2 is preferred.

thanks,
-- Shuah

maintainer
#4 lore raw

Re: [PATCH v2 2/4] lib: test_scanf: Add tests for sscanf number conversion

In reply to Richard Fitzgerald
Quoted 33 lines · Richard Fitzgerald
Quoted 30 lines · Petr Mladek
Quoted 5 lines · Richard Fitzgerald

Adds test_sscanf to test various number conversion cases, as number conversion was previously broken.

This also tests the simple_strtoxxx() functions exported from vsprintf.c.

It is impressive.

Honestly, I do not feel to be expert on testing and mathematics. I am not sure how comprehensive the test is. Also I am not sure what experts would say about the tricks with random numbers.

Anyway, this is much more than what I have expected. And it checks great number of variants and corner cases.

I suggest only one small change, see below.

lib/test_scanf.c line 197

Quoted 6 lines
lib/test_scanf.c new file +4 -0
+#define test_one_number(T, gen_fmt, scan_fmt, val, fn)			\
+do {									\
+	const T expect_val = (T)(val);					\
+	T result = ~expect_val; /* should be overwritten */		\

If I get it correctly, this is supposed to initialize the temporary variable with a value that is different from the expected value. It will cause test failure when it is not updated by vsscanf().

Quoted 5 lines
Quoted 1 line

It does not work for zero value. A better solution might be to add

That's a ~, not a -
~0 = 0xFFFFFFFF
~-1 = 0

I see. This works well.

I am sorry for the noise. Sigh, I think that I need stronger glasses or use a monitor. I am not able to distinguish the two characeters without staring closely on the laptop screen. I wish I was able to work from the office again.

Best Regards,
Petr