Skip to contentSkip to conversation

btrfs: drop unused memparse() parameter

PATCH In review David Disseldorp 5 participants 17 messages Dec 5, 2023 – Dec 13, 2023 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 btrfs: drop unused memparse() parameter

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

---
 fs/btrfs/sysfs.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/sysfs.c +1 -2
index e6b51fb3ddc1e..75f3b774a4d83 100644
@@ -1779,10 +1779,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
 {
 	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
 						   devid_kobj);
-	char *endptr;
 	unsigned long long limit;
 
-	limit = memparse(buf, &endptr);
+	limit = memparse(buf, NULL);
 	WRITE_ONCE(device->scrub_speed_max, limit);
 	return len;
 }
Signature · 1 line
2.35.3
#2 lore raw
In reply to David Disseldorp
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

Or should it be used for validation? memparse is also used in btrfs_chunk_size_store() that accepts whitespace as trailing characters (namely '\n' if the value is from echo).

#3 lore raw
Quoted 8 lines · David Sterba
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

Or should it be used for validation? memparse is also used in btrfs_chunk_size_store() that accepts whitespace as trailing characters (namely '\n' if the value is from echo).

It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs scrub_speed_max API. Failing on invalid input would break scripts which do things like
echo clear > /sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max

Cheers, David

#4 lore raw
Quoted 7 lines · David Disseldorp
Quoted 5 lines · David Sterba
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

Quoted 8 lines
Quoted 3 lines

Or should it be used for validation? memparse is also used in btrfs_chunk_size_store() that accepts whitespace as trailing characters (namely '\n' if the value is from echo).

It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs scrub_speed_max API. Failing on invalid input would break scripts which do things like
echo clear > /sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max

I'm not sure the 'part of the API' is a valid agrument here. It's documented that the value is in bytes and that suffixes can be passed for convenience. How come anybody would use 'clear' in the first place and expect it to work with undefined meaning?

#5 lore raw
Quoted 17 lines · David Sterba
Quoted 7 lines · David Disseldorp
Quoted 5 lines · David Sterba
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

Quoted 8 lines
Quoted 3 lines

Or should it be used for validation? memparse is also used in btrfs_chunk_size_store() that accepts whitespace as trailing characters (namely '\n' if the value is from echo).

It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs scrub_speed_max API. Failing on invalid input would break scripts which do things like
echo clear > /sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max

Quoted 4 lines

I'm not sure the 'part of the API' is a valid agrument here. It's documented that the value is in bytes and that suffixes can be passed for convenience. How come anybody would use 'clear' in the first place and expect it to work with undefined meaning?

Most people don't read documentation :). If there's a willingness to accept any fallout from adding the validation then I'm happy to do that. Will send a v2.

Cheers, David

#6 lore raw
Quoted 19 lines · David Disseldorp
Quoted 17 lines · David Sterba
Quoted 15 lines · David Disseldorp
Quoted 8 lines · David Sterba
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

Or should it be used for validation? memparse is also used in btrfs_chunk_size_store() that accepts whitespace as trailing characters (namely '\n' if the value is from echo).

It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs scrub_speed_max API. Failing on invalid input would break scripts which do things like
echo clear > /sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max

Quoted 6 lines
Quoted 4 lines

I'm not sure the 'part of the API' is a valid agrument here. It's documented that the value is in bytes and that suffixes can be passed for convenience. How come anybody would use 'clear' in the first place and expect it to work with undefined meaning?

Most people don't read documentation :).

Unfortunatelly true. Here the expected practice is that size values on input can have a suffix so it should be consistent and understandable even without reading documentation.

Quoted 3 lines

If there's a willingness to
accept any fallout from adding the validation then I'm happy to do that. Will send a v2.

Yes, I think the simple fix is to add the handling as btrfs_chunk_size_store() does and this will be the pattern for any future memparsed values in sysfs.

There's one outlier discard/kbps_limit that takes KiB directly so that's a plain integer. Eventually discard/max_discard_size can also use memparse.

#7 lore raw

PATCH 1/1 btrfs: validate scrub_speed_max sysfs string

Fail the sysfs I/O on any trailing non-space characters.

---
 fs/btrfs/sysfs.c | 3 +++
 1 file changed, 3 insertions(+)
fs/btrfs/sysfs.c +3 -0
index e6b51fb3ddc1e..4c4642ef7c5f0 100644
@@ -1783,6 +1783,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
 	unsigned long long limit;
 
 	limit = memparse(buf, &endptr);
+	endptr = skip_spaces(endptr);
+	if (*endptr != 0)
+		return -EINVAL;
 	WRITE_ONCE(device->scrub_speed_max, limit);
 	return len;
 }
Signature · 1 line
2.35.3
Reviewed-by on v1 1/1
#8 lore raw

Re: [PATCH] btrfs: validate scrub_speed_max sysfs string

Quoted 3 lines · David Disseldorp

Fail the sysfs I/O on any trailing non-space characters.

Although I have an unrelated idea.

Since memparse() provides the @endptr, can we rewind the @endptr, so that we can check if the last valid charactor is suffix 'e'. Then reject it from btrfs size.

I really don't think we need exabytes suffix for our scrub speed limit usage.

Thanks,
Qu

Quoted 18 lines
---
  fs/btrfs/sysfs.c | 3 +++
  1 file changed, 3 insertions(+)
fs/btrfs/sysfs.c +3 -0
index e6b51fb3ddc1e..4c4642ef7c5f0 100644
@@ -1783,6 +1783,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
  	unsigned long long limit;
 
  	limit = memparse(buf, &endptr);
+	endptr = skip_spaces(endptr);
+	if (*endptr != 0)
+		return -EINVAL;
  	WRITE_ONCE(device->scrub_speed_max, limit);
  	return len;
  }
#9 lore raw

Re: [PATCH] btrfs: validate scrub_speed_max sysfs string

Quoted 6 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp

Fail the sysfs I/O on any trailing non-space characters.

Thanks.

Quoted 5 lines

Although I have an unrelated idea.

Since memparse() provides the @endptr, can we rewind the @endptr, so that we can check if the last valid charactor is suffix 'e'. Then reject it from btrfs size.

Hmm we could do that, but then we'd also break hex parsing. Rejecting 'e' in the absence of a leading '0x' or '0X' would work, but it's a bit ugly.

Cheers, David

#10 lore raw

Re: [PATCH] btrfs: validate scrub_speed_max sysfs string

In reply to Qu Wenruo
Quoted 9 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp

Fail the sysfs I/O on any trailing non-space characters.

Quoted 8 lines

Although I have an unrelated idea.

Since memparse() provides the @endptr, can we rewind the @endptr, so that we can check if the last valid charactor is suffix 'e'. Then reject it from btrfs size.

I really don't think we need exabytes suffix for our scrub speed limit usage.

I think nobody will intentionally use the 'e' exabyte suffix in this case but I don't want to add an exception to parsing the values. We'd have to document that, explain why it's not accepted, it's additional work for a case I still dont understand why it's so important.

If the exabyte scale values are not properly parsed by memparse() due to simple_strtoull() as a workaround we can add our parser based on kstrtoull() or do such change in memparse() proper.

#12 lore raw
In reply to David Disseldorp
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

To me, I believe it's better to completely get rid of memparse().

As you already found out, some suffix, especially "e|E" can screw up the result.
E.g. "25e" would be interpreted as 25 with "e" as suffix, which is fine according to the rule. (without prefix, the base is 10, so only "25" is valid. Then the remaining part is interpreted as suffix).

And since btrfs is not going to do pretty size output for sysfs (as most sysfs is not directly for end users, and we need accurate output), to be consistent there isn't much need for suffix handling either.

So can't we just replace memparse() with kstrtoull()?

Thanks,
Qu

Quoted 22 lines
---
  fs/btrfs/sysfs.c | 3 +--
  1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/sysfs.c +1 -2
index e6b51fb3ddc1e..75f3b774a4d83 100644
@@ -1779,10 +1779,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,
  {
  	struct btrfs_device *device = container_of(kobj, struct btrfs_device,
  						   devid_kobj);
-	char *endptr;
  	unsigned long long limit;
 
-	limit = memparse(buf, &endptr);
+	limit = memparse(buf, NULL);
  	WRITE_ONCE(device->scrub_speed_max, limit);
  	return len;
  }
#13 lore raw
Quoted 15 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

To me, I believe it's better to completely get rid of memparse().

As you already found out, some suffix, especially "e|E" can screw up the result.
E.g. "25e" would be interpreted as 25 with "e" as suffix, which is fine according to the rule. (without prefix, the base is 10, so only "25" is valid. Then the remaining part is interpreted as suffix).

Quoted 5 lines

And since btrfs is not going to do pretty size output for sysfs (as most sysfs is not directly for end users, and we need accurate output), to be consistent there isn't much need for suffix handling either.

So can't we just replace memparse() with kstrtoull()?

The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.

However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.

This is why memparse is there and kstrtoull does not do that.

#14 lore raw
Quoted 27 lines · David Sterba
Quoted 15 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp

The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.

To me, I believe it's better to completely get rid of memparse().

As you already found out, some suffix, especially "e|E" can screw up the result.
E.g. "25e" would be interpreted as 25 with "e" as suffix, which is fine according to the rule. (without prefix, the base is 10, so only "25" is valid. Then the remaining part is interpreted as suffix).

Quoted 5 lines

And since btrfs is not going to do pretty size output for sysfs (as most sysfs is not directly for end users, and we need accurate output), to be consistent there isn't much need for suffix handling either.

So can't we just replace memparse() with kstrtoull()?

The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.

Quoted 5 lines

However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.

This is why memparse is there and kstrtoull does not do that.

That suffix is causing confusion already, just check my "25e" case. (It does not only lead to huge number, but also lead to incorrect value even if we treat "e" as a suffix)

Furthermore, the convenience argument is not that strong, you won't expect end users to do the change for a fs every time. Thus it's mostly managed by a small script or some other tool.

In that case I don't think doing extra bash calculation is a big deal anyway.

Thanks,
Qu

#15 lore raw
Quoted 15 lines · Qu Wenruo
Quoted 10 lines

The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.

However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.

This is why memparse is there and kstrtoull does not do that.

That suffix is causing confusion already, just check my "25e" case. (It does not only lead to huge number, but also lead to incorrect value even if we treat "e" as a suffix)

Quoted 6 lines

Furthermore, the convenience argument is not that strong, you won't expect end users to do the change for a fs every time. Thus it's mostly managed by a small script or some other tool.

In that case I don't think doing extra bash calculation is a big deal anyway.

This is a nice study of convenience vs extra work "that's no big deal". The sysfs files can be used by scripts, often times it's the only access to some settings (like /usr/bin/rescan-scsi-bus.sh/usr/bin/rescan-scsi-bus.sh). The value format, naming of the sysfs files is inconsistent and navigating to a particular directory and setting some magic value is I think common. Another example are trace points, or dynamic printk.

There are tools abstracting that but sometimes it's a person that needs to type it and the shortcuts like the suffix are convenient because there's no extra need to type exact syntax and numbers for a $((...)) expression. Ask 3 people if they'd prefer to type "4m" or "$((4*1024*1024))".

If you ask me I'm all for allowing "4m" and I did that several times when testing things like the discard tunables, chunk size or the scrub limits. I consider the sysfs files an interface for human interaction so even if it's used like that in 10% of time it's still saving typing or allowing one to focus on the real prolem instead of shell sytax.

#16 lore raw
Quoted 36 lines · David Sterba
Quoted 15 lines · Qu Wenruo
Quoted 10 lines

The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.

However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.

This is why memparse is there and kstrtoull does not do that.

That suffix is causing confusion already, just check my "25e" case. (It does not only lead to huge number, but also lead to incorrect value even if we treat "e" as a suffix)

Quoted 6 lines

Furthermore, the convenience argument is not that strong, you won't expect end users to do the change for a fs every time. Thus it's mostly managed by a small script or some other tool.

In that case I don't think doing extra bash calculation is a big deal anyway.

This is a nice study of convenience vs extra work "that's no big deal". The sysfs files can be used by scripts, often times it's the only access to some settings (like /usr/bin/rescan-scsi-bus.sh/usr/bin/rescan-scsi-bus.sh). The value format, naming of the sysfs files is inconsistent and navigating to a particular directory and setting some magic value is I think common. Another example are trace points, or dynamic printk.

There are tools abstracting that but sometimes it's a person that needs to type it and the shortcuts like the suffix are convenient because there's no extra need to type exact syntax and numbers for a $((...)) expression. Ask 3 people if they'd prefer to type "4m" or "$((4*1024*1024))".

Quoted 6 lines

If you ask me I'm all for allowing "4m" and I did that several times when testing things like the discard tunables, chunk size or the scrub limits. I consider the sysfs files an interface for human interaction so even if it's used like that in 10% of time it's still saving typing or allowing one to focus on the real prolem instead of shell sytax.

OK, then let me jump into the rabbit hole of stroxll family for my holiday projects, to make the memparse() itself more reliable and get rid of the caveats.

Hopefully allowing callers to select which suffixes they want to use.

(Well, at least I hope "25e" can give us a correct value after the fix)

Thanks,
Qu