Skip to contentSkip to conversation

lib/kstrtox: introduce kstrtoull_suffix() helper

PATCH Superseded Qu Wenruo 4 participants 12 messages Dec 15, 2023 – Dec 20, 2023

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

cover lib/kstrtox: introduce kstrtoull_suffix() helper

Recently David Sterba <[email protected]> exposed some weird behavior of btrfs sysfs interface, which is utilize memparse().

One example is using "echo 25e >
/sys/fs/btrfs/<uuid>/devinfo/scrub_speed_max".

The result value is not 0x25e, because there is no "0x" prefix provided. Nor (25 << 60), as that would overflow.

All these are the caveats of memparse(), which lacks:

- Overflow check
- Reasonable suffix selection
I know there may be some niche cases for memory layout, but for most callers, they just want a kstrtoull() with reasonable suffix selection.
And I don't think exabytes is a reasonable suffix.

So here we introduce a new helper, "kstrtoull_suffix", which has some the following two abilities added:

- Allow caller to select the suffix they want to enable The default list is "KkMmGgTtPp", no unreasonable "Ee" ones.

- Allow suffix parsing with overflow detection.
The int part detection is already done by _kstrtoull(), and with the extra left shift, do the check again before returning the value.

Unfortunately this new helper is still not a drop-in replacement for memparse():

- No @retptr support
It's possible to add, but I'm not sure how many call sites need that for extra separators.

- Need to check the failure properly
Which memparse() callers never seems to check anyway.

Thus the existing memparse() callers need to opt-in. For now, btrfs usage of memparse() can be easily converted to kstrtoull_suffix(), in the 2nd patch.

Qu Wenruo (2):
lib/strtox: introduce kstrtoull_suffix() helper
btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

 fs/btrfs/sysfs.c        |  20 +++----
 include/linux/kstrtox.h |   7 +++
 lib/kstrtox.c           | 113 ++++++++++++++++++++++++++++++++++++++--
 3 files changed, 123 insertions(+), 17 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH 1/2 lib/strtox: introduce kstrtoull_suffix() helper

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

So here we introduce a new helper to address the problem, kstrtoull_suffix():

- Enhance _kstrtoull()
This allow _kstrtoull() to return even if it hits an invalid char, as long as the optional parameter @retptr is provided.

If @retptr is provided, _kstrtoull() would try its best to parse the valid part, and leave the remaining to be handled by the caller.

If @retptr is not provided, the behavior is not altered.

- New kstrtoull_suffix() helper
This new helper utilize the new @retptr capability of _kstrtoull(), and provides 2 new ability:

* Allow certain suffixes to be chosen
The recommended suffix list is "KkMmGgTtPp", excluding the overflow prone "Ee". Undermost cases there is really no need to use "E" suffix anyway.
And for those who really need that exabytes suffix, they can enable that suffix pretty easily.

* Add overflow checks for the suffixes
If the original number string is fine, but with the extra left shift overflow happens, then -EOVERFLOW is returned.

Cc Andrew Morton <[email protected]>
Cc Christophe JAILLET <[email protected]>
Cc Andy Shevchenko <[email protected]>
---
 include/linux/kstrtox.h |   7 +++
 lib/kstrtox.c           | 113 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 115 insertions(+), 5 deletions(-)
include/linux/kstrtox.h +7 -0
index 7fcf29a4e0de..12c754152c15 100644
@@ -9,6 +9,13 @@
 int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res);
 int __must_check _kstrtol(const char *s, unsigned int base, long *res);
 
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */
+#define KSTRTOULL_SUFFIX_DEFAULT		("KkMmGgTtPp")
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     const char *suffixes);
 int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res);
 int __must_check kstrtoll(const char *s, unsigned int base, long long *res);
 
lib/kstrtox.c +108 -5

143 lines not on this page · raw message

Signature · 1 line
2.43.0
author
#9 lore raw

PATCH 2/2 btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

In reply to Qu Wenruo

Since memparse() itself can not handle overflow at all, use memparse_ull() to be extra safe.

Now overflow values can be properly detected.

---
 fs/btrfs/sysfs.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)
fs/btrfs/sysfs.c +8 -12
index 84c05246ffd8..089c3fc123fe 100644
@@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 {
 	struct btrfs_space_info *space_info = to_space_info(kobj);
 	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
-	char *retptr;
+	int ret;
 	u64 val;
 
 	if (!capable(CAP_SYS_ADMIN))
@@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
 		return -EPERM;
 
-	val = memparse(buf, &retptr);
-	/* There could be trailing '\n', also catch any typos after the value */
-	retptr = skip_spaces(retptr);
-	if (*retptr != 0 || val == 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 
 	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
 
@@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
 {
 	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
 						   devid_kobj);
-	char *endptr;
 	unsigned long long limit;
+	int ret;
 
-	limit = memparse(buf, &endptr);
-	/* There could be trailing '\n', also catch any typos after the value. */
-	endptr = skip_spaces(endptr);
-	if (*endptr != 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 	WRITE_ONCE(device->scrub_speed_max, limit);
 	return len;
 }
Signature · 1 line
2.43.0
#10 lore raw

Re: [PATCH 2/2] btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

Hi Qu,

Quoted 2 lines · Qu Wenruo

Since memparse() itself can not handle overflow at all, use memparse_ull() to be extra safe.

s/memparse_ull/kstrtoull_suffix/

Quoted 1 line

Now overflow values can be properly detected.

Please document how the sysfs API changes with this, in addition to overflow handling:
- support for 'E' / 'e' suffixes dropped
- only one trailing '\n' accepted, instead of many isspace()

The latter might break a few scripts.

Cheers, David

Quoted 53 lines
---
 fs/btrfs/sysfs.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)
fs/btrfs/sysfs.c +8 -12
index 84c05246ffd8..089c3fc123fe 100644
@@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 {
 	struct btrfs_space_info *space_info = to_space_info(kobj);
 	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
-	char *retptr;
+	int ret;
 	u64 val;
 
 	if (!capable(CAP_SYS_ADMIN))
@@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
 		return -EPERM;
 
-	val = memparse(buf, &retptr);
-	/* There could be trailing '\n', also catch any typos after the value */
-	retptr = skip_spaces(retptr);
-	if (*retptr != 0 || val == 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 
 	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
 
@@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
 {
 	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
 						   devid_kobj);
-	char *endptr;
 	unsigned long long limit;
+	int ret;
 
-	limit = memparse(buf, &endptr);
-	/* There could be trailing '\n', also catch any typos after the value. */
-	endptr = skip_spaces(endptr);
-	if (*endptr != 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 	WRITE_ONCE(device->scrub_speed_max, limit);
 	return len;
 }
author
#11 lore raw

Re: [PATCH 2/2] btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

Quoted 7 lines · David Disseldorp

Hi Qu,

Quoted 2 lines · Qu Wenruo

Since memparse() itself can not handle overflow at all, use memparse_ull() to be extra safe.

Quoted 8 lines

s/memparse_ull/kstrtoull_suffix/

Quoted 1 line

Now overflow values can be properly detected.

Please document how the sysfs API changes with this, in addition to overflow handling:
- support for 'E' / 'e' suffixes dropped
- only one trailing '\n' accepted, instead of many isspace()

Well, multiple spaces are already an abuse, and I don't believe sane scripts should go multiple spaces/newlines.

As all the other call sites are going kstrtox, which only accept one newline.

Although the change is indeed worthy a document update.

Thanks,
Qu

Quoted 62 lines

The latter might break a few scripts.

Cheers, David

Quoted 53 lines
---
  fs/btrfs/sysfs.c | 20 ++++++++------------
  1 file changed, 8 insertions(+), 12 deletions(-)
fs/btrfs/sysfs.c +8 -12
index 84c05246ffd8..089c3fc123fe 100644
@@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
  {
  	struct btrfs_space_info *space_info = to_space_info(kobj);
  	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
-	char *retptr;
+	int ret;
  	u64 val;
 
  	if (!capable(CAP_SYS_ADMIN))
@@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
  	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
  		return -EPERM;
 
-	val = memparse(buf, &retptr);
-	/* There could be trailing '\n', also catch any typos after the value */
-	retptr = skip_spaces(retptr);
-	if (*retptr != 0 || val == 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 
  	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
 
@@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
  {
  	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
  						   devid_kobj);
-	char *endptr;
  	unsigned long long limit;
+	int ret;
 
-	limit = memparse(buf, &endptr);
-	/* There could be trailing '\n', also catch any typos after the value. */
-	endptr = skip_spaces(endptr);
-	if (*endptr != 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
  	WRITE_ONCE(device->scrub_speed_max, limit);
  	return len;
  }
#2 lore raw

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

In reply to Qu Wenruo

Hi Qu,

Quoted 118 lines · Qu Wenruo

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

So here we introduce a new helper to address the problem, kstrtoull_suffix():

- Enhance _kstrtoull()
This allow _kstrtoull() to return even if it hits an invalid char, as long as the optional parameter @retptr is provided.

If @retptr is provided, _kstrtoull() would try its best to parse the valid part, and leave the remaining to be handled by the caller.

If @retptr is not provided, the behavior is not altered.

- New kstrtoull_suffix() helper
This new helper utilize the new @retptr capability of _kstrtoull(), and provides 2 new ability:

* Allow certain suffixes to be chosen
The recommended suffix list is "KkMmGgTtPp", excluding the overflow prone "Ee". Undermost cases there is really no need to use "E" suffix anyway.
And for those who really need that exabytes suffix, they can enable that suffix pretty easily.

* Add overflow checks for the suffixes
If the original number string is fine, but with the extra left shift overflow happens, then -EOVERFLOW is returned.

Cc Andrew Morton <[email protected]>
Cc Christophe JAILLET <[email protected]>
Cc Andy Shevchenko <[email protected]>
---
 include/linux/kstrtox.h |   7 +++
 lib/kstrtox.c           | 113 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 115 insertions(+), 5 deletions(-)
include/linux/kstrtox.h +7 -0
index 7fcf29a4e0de..12c754152c15 100644
@@ -9,6 +9,13 @@
 int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res);
 int __must_check _kstrtol(const char *s, unsigned int base, long *res);
 
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */
+#define KSTRTOULL_SUFFIX_DEFAULT		("KkMmGgTtPp")
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     const char *suffixes);
 int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res);
 int __must_check kstrtoll(const char *s, unsigned int base, long long *res);
 
lib/kstrtox.c +15 -4
index d586e6af5e5a..63831207dfdd 100644
@@ -93,7 +93,8 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long
 	return _parse_integer_limit(s, base, p, INT_MAX);
 }
 
-static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
+static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res,
+		      char **retptr)
 {
 	unsigned long long _res;
 	unsigned int rv;
@@ -105,11 +106,19 @@ static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 	if (rv == 0)
 		return -EINVAL;
 	s += rv;
-	if (*s == '\n')
+
+	/*
+	 * If @retptr is provided, caller is responsible to detect
+	 * the extra chars, otherwise we can skip one newline.
+	 */
+	if (!retptr && *s == '\n')
 		s++;
-	if (*s)
+	if (!retptr && *s)
 		return -EINVAL;
+
 	*res = _res;
+	if (retptr)
+		*retptr = (char *)s;
 	return 0;
 }
 
@@ -133,10 +142,104 @@ int kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 {
 	if (s[0] == '+')
 		s++;
-	return _kstrtoull(s, base, res);
+	return _kstrtoull(s, base, res, NULL);
 }
 EXPORT_SYMBOL(kstrtoull);
 
+/**
+ * kstrtoull_suffix - convert a string to ull with suffixes support

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 160

Quoted 10 lines
lib/kstrtox.c +10 -0
+ * @s: The start of the string. The string must be null-terminated, and may also
+ *  include a single newline before its terminating null.
+ * @base: The number base to use. The maximum supported base is 16. If base is
+ *  given as 0, then the base of the string is automatically detected with the
+ *  conventional semantics - If it begins with 0x the number will be parsed as a
+ *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
+ *  parsed as an octal number. Otherwise it will be parsed as a decimal.
+ * @res: Where to write the result of the conversion on success.
+ * @suffixes: A string of acceptable suffixes, must be provided. Or caller
+ *  should use kstrtoull() directly.

The suffixes parameter seems a bit cumbersome; callers need to provide both upper and lower cases, and unsupported characters aren't checked for. However, I can't think of any better suggestions at this stage.

Quoted 59 lines
lib/kstrtox.c +59 -0
+ *
+ *
+ * Return 0 on success.
+ *
+ * Return -ERANGE on overflow or -EINVAL if invalid chars found.
+ * Return value must be checked.
+ */
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     const char *suffixes)
+{
+	unsigned long long init_value;
+	unsigned long long final_value;
+	char *endptr;
+	int ret;
+
+	ret = _kstrtoull(s, base, &init_value, &endptr);
+	/* Either already overflow or no number string at all. */
+	if (ret < 0)
+		return ret;
+	final_value = init_value;
+	/* No suffixes. */
+	if (!*endptr)
+		goto done;
+
+	switch (*endptr) {
+	case 'K':
+	case 'k':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 10;
+		endptr++;
+		break;
+	case 'M':
+	case 'm':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 20;
+		endptr++;
+		break;
+	case 'G':
+	case 'g':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 30;
+		endptr++;
+		break;
+	case 'T':
+	case 't':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 40;
+		endptr++;
+		break;
+	case 'P':
+	case 'p':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 50;
+		endptr++;

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 229

Quoted 10 lines
lib/kstrtox.c +10 -0
+		break;
+	case 'E':
+	case 'e':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 60;
+		endptr++;
+		break;
+	}
+	if (*endptr == '\n')

Nit: the per-case logic could be simplified to a single "shift_val = X" if you initialise and handle !shift_val.

Quoted 25 lines
lib/kstrtox.c +14 -1
+		endptr++;
+	if (*endptr)
+		return -EINVAL;
+
+	/* Overflow check. */
+	if (final_value < init_value)
+		return -EOVERFLOW;
+done:
+	*res = final_value;
+	return 0;
+}
+EXPORT_SYMBOL(kstrtoull_suffix);
+
 /**
  * kstrtoll - convert a string to a long long
  * @s: The start of the string. The string must be null-terminated, and may also
@@ -159,7 +262,7 @@ int kstrtoll(const char *s, unsigned int base, long long *res)
 	int rv;
 
 	if (s[0] == '-') {
-		rv = _kstrtoull(s + 1, base, &tmp);
+		rv = _kstrtoull(s + 1, base, &tmp, NULL);
 		if (rv < 0)
 			return rv;
 		if ((long long)-tmp > 0)
#3 lore raw

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

Quoted 4 lines · David Disseldorp

Hi Qu,

On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:

[...]

Quoted 13 lines
Quoted 5 lines

+/**
+ * kstrtoull_suffix - convert a string to ull with suffixes support
+ * @s: The start of the string. The string must be null-terminated, and may also
+ * include a single newline before its terminating null.
+ * @base: The number base to use. The maximum supported base is 16. If base is

Quoted 7 lines
+ *  given as 0, then the base of the string is automatically detected with the
+ *  conventional semantics - If it begins with 0x the number will be parsed as a
+ *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
+ *  parsed as an octal number. Otherwise it will be parsed as a decimal.

+ * @res: Where to write the result of the conversion on success.
+ * @suffixes: A string of acceptable suffixes, must be provided. Or caller
+ * should use kstrtoull() directly.

Quoted 4 lines

The suffixes parameter seems a bit cumbersome; callers need to provide both upper and lower cases, and unsupported characters aren't checked for. However, I can't think of any better suggestions at this stage.

Initially I went bitmap for the prefixes, but it's not any better.

Firstly where the bitmap should start. If we go bit 0 for "K", then the code would introduce some difference between the bit number and left shift (bit 0, left shift 10), which may be a little confusing.

If we go bit 1 for "K", the bit and left shift it much better, but bit 0 behavior would be left untouched.

Finally the bitmap itself is not that straightforward.

The limitation of providing both upper and lower case is due to the fact that we don't have a case insensitive version of strchr(). But I think it's not that to fix, just convert them all to lower or upper case, then do the strchr().

Would accepting both cases for the suffixes be good enough?

Quoted 64 lines
Quoted 64 lines

+ *
+ *
+ * Return 0 on success.
+ *
+ * Return -ERANGE on overflow or -EINVAL if invalid chars found.
+ * Return value must be checked.
+ */

lib/kstrtox.c +57 -0
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     const char *suffixes)
+{
+	unsigned long long init_value;
+	unsigned long long final_value;
+	char *endptr;
+	int ret;
+
+	ret = _kstrtoull(s, base, &init_value, &endptr);
+	/* Either already overflow or no number string at all. */
+	if (ret < 0)
+		return ret;
+	final_value = init_value;
+	/* No suffixes. */
+	if (!*endptr)
+		goto done;
+
+	switch (*endptr) {
+	case 'K':
+	case 'k':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 10;
+		endptr++;
+		break;
+	case 'M':
+	case 'm':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 20;
+		endptr++;
+		break;
+	case 'G':
+	case 'g':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 30;
+		endptr++;
+		break;
+	case 'T':
+	case 't':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 40;
+		endptr++;
+		break;
+	case 'P':
+	case 'p':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
+		final_value <<= 50;
+		endptr++;
+		break;
+	case 'E':
+	case 'e':
+		if (!strchr(suffixes, *endptr))
+			return -EINVAL;
Quoted 8 lines

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 229

Quoted 5 lines
lib/kstrtox.c +5 -0
+		final_value <<= 60;
+		endptr++;
+		break;
+	}
+	if (*endptr == '\n')

Nit: the per-case logic could be simplified to a single "shift_val = X" if you initialise and handle !shift_val.

Indeed, thanks for the hint!

Thanks,
Qu

Quoted 29 lines
Quoted 26 lines
lib/kstrtox.c +14 -1
+		endptr++;
+	if (*endptr)
+		return -EINVAL;
+
+	/* Overflow check. */
+	if (final_value < init_value)
+		return -EOVERFLOW;
+done:
+	*res = final_value;
+	return 0;
+}
+EXPORT_SYMBOL(kstrtoull_suffix);
+
  /**
   * kstrtoll - convert a string to a long long
   * @s: The start of the string. The string must be null-terminated, and may also
@@ -159,7 +262,7 @@ int kstrtoll(const char *s, unsigned int base, long long *res)
  	int rv;
 
  	if (s[0] == '-') {
-		rv = _kstrtoull(s + 1, base, &tmp);
+		rv = _kstrtoull(s + 1, base, &tmp, NULL);
  		if (rv < 0)
  			return rv;
  		if ((long long)-tmp > 0)
#4 lore raw

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

Quoted 26 lines · Qu Wenruo
Quoted 4 lines · David Disseldorp

Hi Qu,

On Fri, 15 Dec 2023 19:09:23 +1030, Qu Wenruo wrote:

[...]

Quoted 13 lines
Quoted 5 lines

+/**
+ * kstrtoull_suffix - convert a string to ull with suffixes support
+ * @s: The start of the string. The string must be null-terminated, and may also
+ * include a single newline before its terminating null.
+ * @base: The number base to use. The maximum supported base is 16. If base is

Quoted 7 lines
+ *  given as 0, then the base of the string is automatically detected with the
+ *  conventional semantics - If it begins with 0x the number will be parsed as a
+ *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
+ *  parsed as an octal number. Otherwise it will be parsed as a decimal.

+ * @res: Where to write the result of the conversion on success.
+ * @suffixes: A string of acceptable suffixes, must be provided. Or caller
+ * should use kstrtoull() directly.

Quoted 4 lines

The suffixes parameter seems a bit cumbersome; callers need to provide both upper and lower cases, and unsupported characters aren't checked for. However, I can't think of any better suggestions at this stage.

Initially I went bitmap for the prefixes, but it's not any better.

Quoted 8 lines

Firstly where the bitmap should start. If we go bit 0 for "K", then the code would introduce some difference between the bit number and left shift (bit 0, left shift 10), which may be a little confusing.

If we go bit 1 for "K", the bit and left shift it much better, but bit 0 behavior would be left untouched.

Finally the bitmap itself is not that straightforward.

One benefit from a bitmap would be that unsupported @suffixes are easier to detect (instead of ignored), but I think if you rename the function kstrtoull_unit_suffix() then it should be pretty clear what's supported.

Quoted 6 lines

The limitation of providing both upper and lower case is due to the fact that we don't have a case insensitive version of strchr(). But I think it's not that to fix, just convert them all to lower or upper case, then do the strchr().

Would accepting both cases for the suffixes be good enough?

I think so.

Cheers, David

#5 lore raw

RE: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

In reply to David Disseldorp

From: David Disseldorp

Quoted 19 lines

Sent: 18 December 2023 13:00

Quoted 15 lines · Qu Wenruo

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

Some more bikeshed paint :-)
...

Quoted 8 lines

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 183

Quoted 8 lines
lib/kstrtox.c +8 -0
+	ret = _kstrtoull(s, base, &init_value, &endptr);
+	/* Either already overflow or no number string at all. */
+	if (ret < 0)
+		return ret;
+	final_value = init_value;
+	/* No suffixes. */
+	if (!*endptr)
+		goto done;

How about:

	suffix = *endptr;
	if (!strchr(suffixes, suffix))
		return -ENIVAL;
	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;
	if (shift > 50)
		return -EINVAL;
	if (value >> (64 - shift))
		return -EOVERFLOW;
	value <<= shift;

Although purists might want to multiply by 1000 not 1024. And SI multipliers are all upper-case - except k.

...

Quoted 3 lines

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 236

Quoted 3 lines
lib/kstrtox.c +3 -0
+	/* Overflow check. */
+	if (final_value < init_value)
+		return -EOVERFLOW;

That is just plain wrong.

David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)

#6 lore raw

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

Quoted 33 lines · David Laight

From: David Disseldorp

Quoted 19 lines

Sent: 18 December 2023 13:00

Quoted 15 lines · Qu Wenruo

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

Some more bikeshed paint :-)
...

Quoted 8 lines

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 183

Quoted 8 lines
lib/kstrtox.c +8 -0
+	ret = _kstrtoull(s, base, &init_value, &endptr);
+	/* Either already overflow or no number string at all. */
+	if (ret < 0)
+		return ret;
+	final_value = init_value;
+	/* No suffixes. */
+	if (!*endptr)
+		goto done;

How about:

Quoted 4 lines
	suffix = *endptr;
	if (!strchr(suffixes, suffix))
		return -ENIVAL;
	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;

This means the caller has to provide the suffix string in this particular order.
For default suffix list it's not that hard as it's already defined as a macro.

But for those call sites which needs "E", wrongly located "Ee" can screw up the whole process.

Quoted 9 lines
	if (shift > 50)
		return -EINVAL;
	if (value >> (64 - shift))
		return -EOVERFLOW;
	value <<= shift;

Although purists might want to multiply by 1000 not 1024. And SI multipliers are all upper-case - except k.

Quoted 6 lines

...

Quoted 3 lines

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 236

Quoted 3 lines
lib/kstrtox.c +3 -0
+	/* Overflow check. */
+	if (final_value < init_value)
+		return -EOVERFLOW;

That is just plain wrong.

Indeed, I just found a very simple example to prove it wrong, 4 bit binary 0110, left shift 2, result is 1000, still larger than the original one.

Thanks,
Qu

Quoted 8 lines

David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)

#7 lore raw

RE: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

From: Qu Wenruo

Quoted 1 line

Sent: 19 December 2023 21:18

...

Quoted 8 lines
Quoted 5 lines

How about:

	suffix = *endptr;
	if (!strchr(suffixes, suffix))
		return -ENIVAL;
	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;

This means the caller has to provide the suffix string in this particular order.

No, The strchr() checks that the suffix is one the caller wanted. The strcspn() is against a fixed list - so the order can be selected to make the code shorter.

Actually strcspn() isn't the function you need. There might be a function like strchr() that returns a count but I can't remember its name and it may not be in kernel. You might have to write:

	shift = 0;
	for (const char *sfp = "KkMmGgTtPp"; suffix != *sfp; sfp++, shift++) {
		if (!*sfp)
			return -EINVAL;
	}
	shift = shift/2 + 1 * 10;

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)

#8 lore raw

Re: [PATCH 1/2] lib/strtox: introduce kstrtoull_suffix() helper

Quoted 9 lines · David Laight

From: Qu Wenruo

Quoted 1 line

Sent: 19 December 2023 21:18

...

Quoted 6 lines
Quoted 5 lines

How about:

	suffix = *endptr;
	if (!strchr(suffixes, suffix))
		return -ENIVAL;
	shift = strcspn("KkMmGgTtPp", suffix)/2 * 10 + 10;
Quoted 6 lines
Quoted 2 lines

This means the caller has to provide the suffix string in this particular order.

No, The strchr() checks that the suffix is one the caller wanted. The strcspn() is against a fixed list - so the order can be selected to make the code shorter.

Ah, got it.
Although in that case, the 1st parameter should be ("KkMmGgTtPpEe"), as we still support "Ee", just not by default.

Thanks,
Qu

Quoted 17 lines

Actually strcspn() isn't the function you need. There might be a function like strchr() that returns a count but I can't remember its name and it may not be in kernel. You might have to write:

	shift = 0;
	for (const char *sfp = "KkMmGgTtPp"; suffix != *sfp; sfp++, shift++) {
		if (!*sfp)
			return -EINVAL;
	}
	shift = shift/2 + 1 * 10;

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)