Skip to contentSkip to conversation

lib/kstrtox: introduce kstrtoull_suffix() helper

PATCH In review Qu Wenruo 5 participants 7 messages Dec 20, 2023 – Dec 27, 2023

Subsystems (2)
lore
Share & tools
lore

Qu Wenruo

author
#0 lore raw

v2 cover lib/kstrtox: introduce kstrtoull_suffix() helper

[CHANGELOG]
v2:
- Use enum bitmap to define suffixes
This get rid of the upper/lower case problem, and using enum for each suffix still provides a somewhat readable result.

- Do proper suffix overflow check
The previous check for suffix overflow is incorrect at all, would still lead to incorrect values.

- Slight refactor for kstrtoull_suffix() code

Recently David Sterba <[email protected]> exposed some weird behavior of btrfs sysfs interface, which is utilizing memparse().

One example is using "echo 25e >
/sys/fs/btrfs/<uuid>/devinfo/scrub_speed_max".

The result value is not 0x25e, because there is no "0x" prefix provided. Nor (25 << 60), as that would overflow.

All these are the caveats of memparse(), which lacks:

- Overflow check
- Reasonable suffix selection
I know there may be some niche cases for memory layout, but for most callers, they just want a kstrtoull() with reasonable suffix selection.
And I don't think exabytes is a reasonable suffix.

So here we introduce a new helper, "kstrtoull_suffix", which has some the following two abilities added:

- Allow caller to select the suffix they want to enable The default list is "KkMmGgTtPp", no unreasonable "Ee" ones.

- Allow suffix parsing with overflow detection.
The int part detection is already done by _kstrtoull(), and with the extra left shift, do the check again before returning the value.

Unfortunately this new helper is still not a drop-in replacement for memparse():

- No @retptr support
It's possible to add, but I'm not sure how many call sites need that for extra separators, and even if there are, they may want to go strsep() and strtok().

- Need to check the failure properly
Which memparse() callers never seems to check anyway.

Thus the existing memparse() callers need to opt-in. For now, btrfs usage of memparse() can be easily converted to kstrtoull_suffix(), in the 2nd patch.

Qu Wenruo (2):
lib/strtox: introduce kstrtoull_suffix() helper
btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

 fs/btrfs/sysfs.c        |  20 +++-----
 include/linux/kstrtox.h |  20 ++++++++
 lib/kstrtox.c           | 108 ++++++++++++++++++++++++++++++++++++++--
 3 files changed, 131 insertions(+), 17 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v2 1/2 lib/strtox: introduce kstrtoull_suffix() helper

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

So here we introduce a new helper to address the problem, kstrtoull_suffix():

- Enhance _kstrtoull()
This allow _kstrtoull() to return even if it hits an invalid char, as long as the optional parameter @retptr is provided.

If @retptr is provided, _kstrtoull() would try its best to parse the valid part, and leave the remaining to be handled by the caller.

If @retptr is not provided, the behavior is not altered.

- New kstrtoull_suffix() helper
This new helper utilize the new @retptr capability of _kstrtoull(), and provides 2 new ability:

* Allow certain suffixes to be chosen
The recommended suffix list is "KMGTP" (using the new unit_suffix enum as a bitmap), excluding the overflow prone "E". Undermost cases there is really no need to use "E" suffix anyway. And for those who really need that exabytes suffix, they can enable that suffix pretty easily.

* Add overflow checks for the suffixes
If the original number string is fine, but with the extra left shift overflow happens, then -EOVERFLOW is returned.

Cc Andrew Morton <[email protected]>
Cc Christophe JAILLET <[email protected]>
Cc Andy Shevchenko <[email protected]>
Cc David Laight <[email protected]>

----
Changelog:
v2:
- Use enum bitmap to describe the suffixes
This gets rid of the upper/lower case problem, and enum makes it a little more readable.

- Fix the suffix overflow detection

- Move the left shift out of the switch block

- Remove the "done" tag
Since no tailing character can already be handled properly.

---
 include/linux/kstrtox.h |  20 ++++++++
 lib/kstrtox.c           | 108 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 123 insertions(+), 5 deletions(-)
include/linux/kstrtox.h +20 -0
index 7fcf29a4e0de..edac52d18a8e 100644
@@ -9,6 +9,26 @@
 int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res);
 int __must_check _kstrtol(const char *s, unsigned int base, long *res);
 
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */
+enum unit_suffix {
+	SUFFIX_K = (1 << 0),
+	SUFFIX_M = (1 << 1),
+	SUFFIX_G = (1 << 2),
+	SUFFIX_T = (1 << 3),
+	SUFFIX_P = (1 << 4),
+	SUFFIX_E = (1 << 5),
+};
+
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */
+#define KSTRTOULL_SUFFIX_DEFAULT (SUFFIX_K | SUFFIX_M | SUFFIX_G | SUFFIX_T | SUFFIX_P)
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     enum unit_suffix suffixes);
 int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res);
 int __must_check kstrtoll(const char *s, unsigned int base, long long *res);
 
lib/kstrtox.c +103 -5

138 lines not on this page · raw message

Signature · 1 line
2.43.0
author
#6 lore raw

PATCH v2 2/2 btrfs: sysfs: use kstrtoull_suffix() to replace memparse()

In reply to Qu Wenruo

Since memparse() itself can not handle overflow at all, use kstrtoull_suffix() to be extra safe.

This would introduce behavior changes to the following sysfs interfaces:

- /sys/fs/btrfs/<uuid>/allocation/(data|metadata|system)/chunk_size

- /sys/fs/btrfs/<uuid>/devinfo/<devid>/scrub_speed_max

The behavior changes would include:

- More overflow detection

- No support for "E" suffix anymore

- More strict tailing character requirement
Now the only allowed tailing character is newline. Other characters including space are no longer supported.

Although for regular "echo NUMBER >" usage it should be totally fine.

---
Changelog:
v2:
- None

---
 fs/btrfs/sysfs.c | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)
fs/btrfs/sysfs.c +8 -12
index 84c05246ffd8..089c3fc123fe 100644
@@ -760,7 +760,7 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 {
 	struct btrfs_space_info *space_info = to_space_info(kobj);
 	struct btrfs_fs_info *fs_info = to_fs_info(get_btrfs_kobj(kobj));
-	char *retptr;
+	int ret;
 	u64 val;
 
 	if (!capable(CAP_SYS_ADMIN))
@@ -776,11 +776,9 @@ static ssize_t btrfs_chunk_size_store(struct kobject *kobj,
 	if (space_info->flags & BTRFS_BLOCK_GROUP_SYSTEM)
 		return -EPERM;
 
-	val = memparse(buf, &retptr);
-	/* There could be trailing '\n', also catch any typos after the value */
-	retptr = skip_spaces(retptr);
-	if (*retptr != 0 || val == 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &val, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 
 	val = min(val, BTRFS_MAX_DATA_CHUNK_SIZE);
 
@@ -1779,14 +1777,12 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
 {
 	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
 						   devid_kobj);
-	char *endptr;
 	unsigned long long limit;
+	int ret;
 
-	limit = memparse(buf, &endptr);
-	/* There could be trailing '\n', also catch any typos after the value. */
-	endptr = skip_spaces(endptr);
-	if (*endptr != 0)
-		return -EINVAL;
+	ret = kstrtoull_suffix(buf, 0, &limit, KSTRTOULL_SUFFIX_DEFAULT);
+	if (ret < 0)
+		return ret;
 	WRITE_ONCE(device->scrub_speed_max, limit);
 	return len;
 }
Signature · 1 line
2.43.0
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] lib/strtox: introduce kstrtoull_suffix() helper

In reply to Qu Wenruo
Quoted 57 lines · Qu Wenruo

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

So here we introduce a new helper to address the problem, kstrtoull_suffix():

- Enhance _kstrtoull()
This allow _kstrtoull() to return even if it hits an invalid char, as long as the optional parameter @retptr is provided.

If @retptr is provided, _kstrtoull() would try its best to parse the valid part, and leave the remaining to be handled by the caller.

If @retptr is not provided, the behavior is not altered.

- New kstrtoull_suffix() helper
This new helper utilize the new @retptr capability of _kstrtoull(), and provides 2 new ability:

* Allow certain suffixes to be chosen
The recommended suffix list is "KMGTP" (using the new unit_suffix enum as a bitmap), excluding the overflow prone "E". Undermost cases there is really no need to use "E" suffix anyway. And for those who really need that exabytes suffix, they can enable that suffix pretty easily.

* Add overflow checks for the suffixes
If the original number string is fine, but with the extra left shift overflow happens, then -EOVERFLOW is returned.

Cc Andrew Morton <[email protected]>
Cc Christophe JAILLET <[email protected]>
Cc Andy Shevchenko <[email protected]>
Cc David Laight <[email protected]>

----
Changelog:
v2:
- Use enum bitmap to describe the suffixes
This gets rid of the upper/lower case problem, and enum makes it a little more readable.

Quoted 6 lines

- Fix the suffix overflow detection

- Move the left shift out of the switch block

- Remove the "done" tag
Since no tailing character can already be handled properly.

nit: git am puts this changelog in the commit message when applied. Please use git send-email --annotate and put it next to the diffstat, so that it gets discarded.

Quoted 20 lines
---
 include/linux/kstrtox.h |  20 ++++++++
 lib/kstrtox.c           | 108 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 123 insertions(+), 5 deletions(-)
include/linux/kstrtox.h +7 -0
index 7fcf29a4e0de..edac52d18a8e 100644
@@ -9,6 +9,26 @@
 int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res);
 int __must_check _kstrtol(const char *s, unsigned int base, long *res);
 
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */
+enum unit_suffix {
+	SUFFIX_K = (1 << 0),
+	SUFFIX_M = (1 << 1),

include/linux/kstrtox.h line 28

Quoted 10 lines
include/linux/kstrtox.h +10 -0
+	SUFFIX_G = (1 << 2),
+	SUFFIX_T = (1 << 3),
+	SUFFIX_P = (1 << 4),
+	SUFFIX_E = (1 << 5),
+};
+
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */

^ this comment is a duplicate.

include/linux/kstrtox.h line 29

Quoted 1 line
include/linux/kstrtox.h +1 -0
+#define KSTRTOULL_SUFFIX_DEFAULT (SUFFIX_K | SUFFIX_M | SUFFIX_G | SUFFIX_T | SUFFIX_P)

I think it'd be clearer if you dropped this default and had callers explicitly provide the desired suffix mask.

include/linux/kstrtox.h line 31

Quoted 2 lines
include/linux/kstrtox.h +2 -0
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     enum unit_suffix suffixes);

__must_check would be consistent with the other helpers...

Quoted 122 lines
include/linux/kstrtox.h +0 -0
 int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res);
 int __must_check kstrtoll(const char *s, unsigned int base, long long *res);
 
lib/kstrtox.c +87 -4
index d586e6af5e5a..8a2fdd1e3376 100644
@@ -93,7 +93,8 @@ unsigned int _parse_integer(const char *s, unsigned int base, unsigned long long
 	return _parse_integer_limit(s, base, p, INT_MAX);
 }
 
-static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
+static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res,
+		      char **retptr)
 {
 	unsigned long long _res;
 	unsigned int rv;
@@ -105,11 +106,19 @@ static int _kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 	if (rv == 0)
 		return -EINVAL;
 	s += rv;
-	if (*s == '\n')
+
+	/*
+	 * If @retptr is provided, caller is responsible to detect
+	 * the extra chars, otherwise we can skip one newline.
+	 */
+	if (!retptr && *s == '\n')
 		s++;
-	if (*s)
+	if (!retptr && *s)
 		return -EINVAL;
+
 	*res = _res;
+	if (retptr)
+		*retptr = (char *)s;
 	return 0;
 }
 
@@ -133,10 +142,99 @@ int kstrtoull(const char *s, unsigned int base, unsigned long long *res)
 {
 	if (s[0] == '+')
 		s++;
-	return _kstrtoull(s, base, res);
+	return _kstrtoull(s, base, res, NULL);
 }
 EXPORT_SYMBOL(kstrtoull);
 
+/**
+ * kstrtoull_suffix - convert a string to ull with suffixes support
+ * @s: The start of the string. The string must be null-terminated, and may also
+ *  include a single newline before its terminating null.
+ * @base: The number base to use. The maximum supported base is 16. If base is
+ *  given as 0, then the base of the string is automatically detected with the
+ *  conventional semantics - If it begins with 0x the number will be parsed as a
+ *  hexadecimal (case insensitive), if it otherwise begins with 0, it will be
+ *  parsed as an octal number. Otherwise it will be parsed as a decimal.
+ * @res: Where to write the result of the conversion on success.
+ * @suffixes: bitmap of acceptable suffixes, unknown bits would be ignored.
+ *
+ * Return 0 on success.
+ *
+ * Return -ERANGE on overflow or -EINVAL if invalid chars found.
+ * Return value must be checked.
+ */
+int kstrtoull_suffix(const char *s, unsigned int base, unsigned long long *res,
+		     enum unit_suffix suffixes)
+{
+	unsigned long long value;
+	int shift = 0;
+	char *endptr;
+	int ret;
+
+	ret = _kstrtoull(s, base, &value, &endptr);
+	/* Either already overflow or no number string at all. */
+	if (ret < 0)
+		return ret;
+
+	switch (*endptr) {
+	case 'K':
+	case 'k':
+		if (!(suffixes & SUFFIX_K))
+			return -EINVAL;
+		shift = 10;
+		break;
+	case 'M':
+	case 'm':
+		if (!(suffixes & SUFFIX_M))
+			return -EINVAL;
+		shift = 20;
+		break;
+	case 'G':
+	case 'g':
+		if (!(suffixes & SUFFIX_G))
+			return -EINVAL;
+		shift = 30;
+		break;
+	case 'T':
+	case 't':
+		if (!(suffixes & SUFFIX_T))
+			return -EINVAL;
+		shift = 40;
+		break;
+	case 'P':
+	case 'p':
+		if (!(suffixes & SUFFIX_P))
+			return -EINVAL;
+		shift = 50;
+		break;
+	case 'E':
+	case 'e':
+		if (!(suffixes & SUFFIX_E))
+			return -EINVAL;
+		shift = 60;
+		break;
+	}
+	if (shift)
+		endptr++;
+	if (*endptr == '\n')
+		endptr++;
+	if (*endptr)
+		return -EINVAL;

lib/kstrtox.c int kstrtoull(const char *s, unsigned int base, unsigned long long *res) · line 232

Quoted 10 lines
lib/kstrtox.c +10 -0
+
+	/*
+	 * Overflow check.
+	 *
+	 * Check @shift before doing right shift, as if right shift bit is
+	 * greater than or equal to the number of bits, the result is undefined.
+	 */
+	if (shift && value >> (64 - shift))
+		return -EOVERFLOW;
+	value <<= shift;

Might as well roll this logic in with the existing "if (shift)" above, e.g.

	if (shift) {
		/*
		 * Overflow check...
		 */
		if (value >> (64 - shift))
			return -EOVERFLOW;
		value <<= shift;
		endptr++;
	}
	if (*endptr == '\n')
	...
Quoted 9 lines
lib/kstrtox.c +5 -0
+	*res = value;
+	return 0;
+}
+EXPORT_SYMBOL(kstrtoull_suffix);
+
 /**
  * kstrtoll - convert a string to a long long
  * @s: The start of the string. The string must be null-terminated, and may also

lib/kstrtox.c int kstrtoll(const char *s, unsigned int base, long long *res) · line 263

Quoted 8 lines
lib/kstrtox.c +1 -1
@@ -159,7 +257,7 @@ int kstrtoll(const char *s, unsigned int base, long long *res)
 	int rv;
 
 	if (s[0] == '-') {
-		rv = _kstrtoull(s + 1, base, &tmp);
+		rv = _kstrtoull(s + 1, base, &tmp, NULL);
 		if (rv < 0)
 			return rv;
 		if ((long long)-tmp > 0)

With the above changes made, feel free to add

I'll leave the review of patch 2/2 up to others, as I'm still a little worried about sysfs trailing whitespace regressions.

#5 lore raw

Re: [PATCH v2 1/2] lib/strtox: introduce kstrtoull_suffix() helper

In reply to Qu Wenruo
Quoted 39 lines · Qu Wenruo

Just as mentioned in the comment of memparse(), the simple_stroull() usage can lead to overflow all by itself.

Furthermore, the suffix calculation is also super overflow prone because that some suffix like "E" itself would eat 60bits, leaving only 4 bits available.

And that suffix "E" can also lead to confusion since it's using the same char of hex Ox'E'.

One simple example to expose all the problem is to use memparse() on "25E".
The correct value should be 28823037615171174400, but the suffix E makes it super simple to overflow, resulting the incorrect value 10376293541461622784 (9E).

So here we introduce a new helper to address the problem, kstrtoull_suffix():

- Enhance _kstrtoull()
This allow _kstrtoull() to return even if it hits an invalid char, as long as the optional parameter @retptr is provided.

If @retptr is provided, _kstrtoull() would try its best to parse the valid part, and leave the remaining to be handled by the caller.

If @retptr is not provided, the behavior is not altered.

- New kstrtoull_suffix() helper
This new helper utilize the new @retptr capability of _kstrtoull(), and provides 2 new ability:

* Allow certain suffixes to be chosen
The recommended suffix list is "KMGTP" (using the new unit_suffix enum as a bitmap), excluding the overflow prone "E". Undermost cases there is really no need to use "E" suffix anyway. And for those who really need that exabytes suffix, they can enable that suffix pretty easily.

Quoted 3 lines

* Add overflow checks for the suffixes
If the original number string is fine, but with the extra left shift overflow happens, then -EOVERFLOW is returned.

NAK. Read the v1 discussion why.

Signature · 2 lines
With Best Regards,
Andy Shevchenko
#3 lore raw

Re: [PATCH v2 1/2] lib/strtox: introduce kstrtoull_suffix() helper

In reply to David Disseldorp
Quoted 2 lines · David Disseldorp

On Wed, 20 Dec 2023 10:40:00 +1030, Qu Wenruo wrote:

[...]

Quoted 10 lines
Quoted 10 lines

----
Changelog:
v2:
- Use enum bitmap to describe the suffixes
This gets rid of the upper/lower case problem, and enum makes it a little more readable.

- Fix the suffix overflow detection

Quoted 8 lines
Quoted 4 lines

- Move the left shift out of the switch block

- Remove the "done" tag
Since no tailing character can already be handled properly.

nit: git am puts this changelog in the commit message when applied. Please use git send-email --annotate and put it next to the diffstat, so that it gets discarded.

Got it.

[...]

Quoted 9 lines

include/linux/kstrtox.h line 28

Quoted 6 lines
include/linux/kstrtox.h +6 -0
+};
+
+/*
+ * The default suffix list would not include "E" since it's too easy to overflow
+ * and not much real world usage.
+ */

^ this comment is a duplicate.

Quoted 4 lines
Quoted 1 line

+#define KSTRTOULL_SUFFIX_DEFAULT (SUFFIX_K | SUFFIX_M | SUFFIX_G | SUFFIX_T | SUFFIX_P)

I think it'd be clearer if you dropped this default and had callers explicitly provide the desired suffix mask.

Well, that would be long, and would be even longer as the newer naming would be MEMPARSE_SUFFIX_*, to be more explicit on what the suffix is for...

And I really want callers to choose a saner default suffix, thus here comes the default one.

In fact, in my next version, I also found that there are some memparse() call sites benefits from the newer suffixes (although won't for the "E" one).
The example is the call site setup_elfcorehdr(). Where the comment only mentions KMG, but since memparse() silently added "PE" suffixes, maybe on some mainframes we saved some time for one or two lucky admins.

[...]

Quoted 4 lines

With the above changes made, feel free to add

Thanks for the review, but I'm afraid the newer version would be another beast.

All the ommitted comments would be addressed a in new series.

Quoted 3 lines

I'll leave the review of patch 2/2 up to others, as I'm still a little worried about sysfs trailing whitespace regressions.

That won't be a problem anymore, the new series would keep the old @retptr behavior, thus for btrfs part it won't be changed at all.

Thanks,
Qu

Quoted 1 line
#4 lore raw

Re: [PATCH v2 1/2] lib/strtox: introduce kstrtoull_suffix() helper

Quoted 1 line · Qu Wenruo

On 2023/12/20 16:08, David Disseldorp wrote:

...

Quoted 11 lines
Quoted 4 lines
Quoted 1 line

+#define KSTRTOULL_SUFFIX_DEFAULT (SUFFIX_K | SUFFIX_M | SUFFIX_G | SUFFIX_T | SUFFIX_P)

I think it'd be clearer if you dropped this default and had callers explicitly provide the desired suffix mask.

Well, that would be long, and would be even longer as the newer naming would be MEMPARSE_SUFFIX_*, to be more explicit on what the suffix is for...

And I really want callers to choose a saner default suffix, thus here comes the default one.

Quoted 6 lines

In fact, in my next version, I also found that there are some memparse() call sites benefits from the newer suffixes (although won't for the "E" one).
The example is the call site setup_elfcorehdr(). Where the comment only mentions KMG, but since memparse() silently added "PE" suffixes, maybe on some mainframes we saved some time for one or two lucky admins.

I think it's a sane default, my concern is that _DEFAULT says nothing about supported units from the caller's perspective. Perhaps MEMPARSE_SUFFIX_KMGTP or MEMPARSE_UNITS_KMGTP would be clearer.

...

Quoted 6 lines
Quoted 2 lines

With the above changes made, feel free to add

Thanks for the review, but I'm afraid the newer version would be another beast.

Quoted 7 lines

All the ommitted comments would be addressed a in new series.

Quoted 3 lines

I'll leave the review of patch 2/2 up to others, as I'm still a little worried about sysfs trailing whitespace regressions.

That won't be a problem anymore, the new series would keep the old @retptr behavior, thus for btrfs part it won't be changed at all.

Sounds good. Will follow up there.

Cheers, David