btrfs: drop unused memparse() parameter
PATCH 1/1 btrfs: drop unused memparse() parameter
The @retptr parameter for memparse() is optional. btrfs_devinfo_scrub_speed_max_store() doesn't use it for any input validation, so the parameter can be dropped.
--- fs/btrfs/sysfs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
fs/btrfs/sysfs.c +1 -2
@@ -1779,10 +1779,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj, { struct btrfs_device *device = container_of(kobj, struct btrfs_device, devid_kobj); -char *endptr;unsigned long long limit; - limit =memparse (buf, &endptr); + limit =memparse (buf,NULL );WRITE_ONCE (device->scrub_speed_max, limit);return len; }
Signature · 1 line
2.35.3
Looks good,
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.
Or should it be used for validation? memparse is also used in btrfs_chunk_size_store() that accepts whitespace as trailing characters (namely '\n' if the value is from echo).
Quoted 8 lines · David Sterba
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.Or should it be used for validation? memparse is also used in
btrfs_chunk_size_store()that accepts whitespace as trailing characters (namely '\n' if the value is from echo).
It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs scrub_speed_max API. Failing on invalid input would break scripts which do things like
echo clear > /sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max
Cheers, David
Quoted 7 lines · David Disseldorp
Quoted 5 lines · David Sterba
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.
Quoted 8 lines
Quoted 3 lines
Or should it be used for validation? memparse is also used in
btrfs_chunk_size_store()that accepts whitespace as trailing characters (namely '\n' if the value is from echo).It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs
scrub_speed_maxAPI. Failing on invalid input would break scripts which do things like
echo clear >/sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max
I'm not sure the 'part of the API' is a valid agrument here. It's documented that the value is in bytes and that suffixes can be passed for convenience. How come anybody would use 'clear' in the first place and expect it to work with undefined meaning?
Quoted 17 lines · David Sterba
Quoted 7 lines · David Disseldorp
Quoted 5 lines · David Sterba
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.Quoted 8 lines
Quoted 3 lines
Or should it be used for validation? memparse is also used in
btrfs_chunk_size_store()that accepts whitespace as trailing characters (namely '\n' if the value is from echo).It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs
scrub_speed_maxAPI. Failing on invalid input would break scripts which do things like
echo clear >/sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max
Quoted 4 lines
I'm not sure the 'part of the API' is a valid agrument here. It's documented that the value is in bytes and that suffixes can be passed for convenience. How come anybody would use 'clear' in the first place and expect it to work with undefined meaning?
Most people don't read documentation :). If there's a willingness to accept any fallout from adding the validation then I'm happy to do that. Will send a v2.
Cheers, David
Quoted 19 lines · David Disseldorp
Quoted 17 lines · David Sterba
Quoted 15 lines · David Disseldorp
Quoted 8 lines · David Sterba
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.Or should it be used for validation? memparse is also used in
btrfs_chunk_size_store()that accepts whitespace as trailing characters (namely '\n' if the value is from echo).It probably should have been used for validation when originally added, but the current behaviour is now part of the sysfs
scrub_speed_maxAPI. Failing on invalid input would break scripts which do things like
echo clear >/sys/fs/btrfs/UUID/devinfo/1/scrub_speed_max
Quoted 6 lines
Quoted 4 lines
I'm not sure the 'part of the API' is a valid agrument here. It's documented that the value is in bytes and that suffixes can be passed for convenience. How come anybody would use 'clear' in the first place and expect it to work with undefined meaning?
Most people don't read documentation :).
Unfortunatelly true. Here the expected practice is that size values on input can have a suffix so it should be consistent and understandable even without reading documentation.
Quoted 3 lines
If there's a willingness to
accept any fallout from adding the validation then I'm happy to do that. Will send a v2.
Yes, I think the simple fix is to add the handling as btrfs_chunk_size_store() does and this will be the pattern for any future memparsed values in sysfs.
There's one outlier discard/kbps_limit that takes KiB directly so that's a plain integer. Eventually discard/max_discard_size can also use memparse.
PATCH 1/1 btrfs: validate scrub_speed_max sysfs string
Fail the sysfs I/O on any trailing non-space characters.
--- fs/btrfs/sysfs.c | 3 +++ 1 file changed, 3 insertions(+)
fs/btrfs/sysfs.c +3 -0
@@ -1783,6 +1783,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,unsigned long long limit; limit =memparse (buf, &endptr); + endptr =skip_spaces (endptr); +if (*endptr !=0 ) +return -EINVAL ;WRITE_ONCE (device->scrub_speed_max, limit);return len; }
Signature · 1 line
2.35.3
Re: [PATCH] btrfs: validate scrub_speed_max sysfs string
Quoted 3 lines · David Disseldorp
Fail the sysfs I/O on any trailing non-space characters.
Signed-off-by David Disseldorp <[email protected]>
Although I have an unrelated idea.
Since memparse() provides the @endptr, can we rewind the @endptr, so that we can check if the last valid charactor is suffix 'e'. Then reject it from btrfs size.
I really don't think we need exabytes suffix for our scrub speed limit usage.
Thanks,
Qu
Quoted 18 lines
--- fs/btrfs/sysfs.c | 3 +++ 1 file changed, 3 insertions(+)fs/btrfs/sysfs.c +3
-0@@ -1783,6 +1783,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj,unsigned long long limit; limit =memparse (buf, &endptr); + endptr =skip_spaces (endptr); +if (*endptr !=0 ) +return -EINVAL ;WRITE_ONCE (device->scrub_speed_max, limit);return len; }
Re: [PATCH] btrfs: validate scrub_speed_max sysfs string
Quoted 6 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp
Fail the sysfs I/O on any trailing non-space characters.
Signed-off-by David Disseldorp <[email protected]>Reviewed-by Qu Wenruo <[email protected]>
Thanks.
Quoted 5 lines
Although I have an unrelated idea.
Since
memparse()provides the @endptr, can we rewind the @endptr, so that we can check if the last valid charactor is suffix 'e'. Then reject it from btrfs size.
Hmm we could do that, but then we'd also break hex parsing. Rejecting 'e' in the absence of a leading '0x' or '0X' would work, but it's a bit ugly.
Cheers, David
Re: [PATCH] btrfs: validate scrub_speed_max sysfs string
In reply to Qu WenruoQuoted 9 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp
Fail the sysfs I/O on any trailing non-space characters.
Signed-off-by David Disseldorp <[email protected]>Reviewed-by Qu Wenruo <[email protected]>
Quoted 8 lines
Although I have an unrelated idea.
Since
memparse()provides the @endptr, can we rewind the @endptr, so that we can check if the last valid charactor is suffix 'e'. Then reject it from btrfs size.I really don't think we need exabytes suffix for our scrub speed limit usage.
I think nobody will intentionally use the 'e' exabyte suffix in this case but I don't want to add an exception to parsing the values. We'd have to document that, explain why it's not accepted, it's additional work for a case I still dont understand why it's so important.
If the exabyte scale values are not properly parsed by memparse() due to simple_strtoull() as a workaround we can add our parser based on kstrtoull() or do such change in memparse() proper.
Re: [PATCH] btrfs: validate scrub_speed_max sysfs string
In reply to David DisseldorpQuoted 3 lines · David Disseldorp
Fail the sysfs I/O on any trailing non-space characters.
Signed-off-by David Disseldorp <[email protected]>
Added to misc-next a few days ago with updated changelog with reasoning based on the recent discussions why we want the memparse and suffixes. Thanks.
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.
To me, I believe it's better to completely get rid of memparse().
As you already found out, some suffix, especially "e|E" can screw up the result.
E.g. "25e" would be interpreted as 25 with "e" as suffix, which is fine according to the rule. (without prefix, the base is 10, so only "25" is valid. Then the remaining part is interpreted as suffix).
And since btrfs is not going to do pretty size output for sysfs (as most sysfs is not directly for end users, and we need accurate output), to be consistent there isn't much need for suffix handling either.
So can't we just replace memparse() with kstrtoull()?
Thanks,
Qu
Quoted 22 lines
Signed-off-by David Disseldorp <[email protected]>--- fs/btrfs/sysfs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)fs/btrfs/sysfs.c +1
-2@@ -1779,10 +1779,9 @@ static ssize_t btrfs_devinfo_scrub_speed_max_store(struct kobject *kobj, { struct btrfs_device *device = container_of(kobj, struct btrfs_device, devid_kobj); -char *endptr;unsigned long long limit; - limit =memparse (buf, &endptr); + limit =memparse (buf,NULL );WRITE_ONCE (device->scrub_speed_max, limit);return len; }
Quoted 15 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.To me, I believe it's better to completely get rid of
memparse().As you already found out, some suffix, especially "e|E" can screw up the result.
E.g. "25e" would be interpreted as 25 with "e" as suffix, which is fine according to the rule. (without prefix, the base is 10, so only "25" is valid. Then the remaining part is interpreted as suffix).
Quoted 5 lines
And since btrfs is not going to do pretty size output for sysfs (as most sysfs is not directly for end users, and we need accurate output), to be consistent there isn't much need for suffix handling either.
So can't we just replace
memparse()withkstrtoull()?
The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.
However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.
This is why memparse is there and kstrtoull does not do that.
Quoted 27 lines · David Sterba
Quoted 15 lines · Qu Wenruo
Quoted 3 lines · David Disseldorp
The @retptr parameter for
memparse()is optional.btrfs_devinfo_scrub_speed_max_store()doesn't use it for any input validation, so the parameter can be dropped.To me, I believe it's better to completely get rid of
memparse().As you already found out, some suffix, especially "e|E" can screw up the result.
E.g. "25e" would be interpreted as 25 with "e" as suffix, which is fine according to the rule. (without prefix, the base is 10, so only "25" is valid. Then the remaining part is interpreted as suffix).Quoted 5 lines
And since btrfs is not going to do pretty size output for sysfs (as most sysfs is not directly for end users, and we need accurate output), to be consistent there isn't much need for suffix handling either.
So can't we just replace
memparse()withkstrtoull()?The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.
Quoted 5 lines
However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.
This is why memparse is there and kstrtoull does not do that.
That suffix is causing confusion already, just check my "25e" case. (It does not only lead to huge number, but also lead to incorrect value even if we treat "e" as a suffix)
Furthermore, the convenience argument is not that strong, you won't expect end users to do the change for a fs every time. Thus it's mostly managed by a small script or some other tool.
In that case I don't think doing extra bash calculation is a big deal anyway.
Thanks,
Qu
Quoted 15 lines · Qu Wenruo
Quoted 10 lines
The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.
However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.
This is why memparse is there and kstrtoull does not do that.
That suffix is causing confusion already, just check my "25e" case. (It does not only lead to huge number, but also lead to incorrect value even if we treat "e" as a suffix)
Quoted 6 lines
Furthermore, the convenience argument is not that strong, you won't expect end users to do the change for a fs every time. Thus it's mostly managed by a small script or some other tool.
In that case I don't think doing extra bash calculation is a big deal anyway.
This is a nice study of convenience vs extra work "that's no big deal". The sysfs files can be used by scripts, often times it's the only access to some settings (like /usr/bin/rescan-scsi-bus.sh/usr/bin/rescan-scsi-bus.sh). The value format, naming of the sysfs files is inconsistent and navigating to a particular directory and setting some magic value is I think common. Another example are trace points, or dynamic printk.
There are tools abstracting that but sometimes it's a person that needs to type it and the shortcuts like the suffix are convenient because there's no extra need to type exact syntax and numbers for a $((...)) expression. Ask 3 people if they'd prefer to type "4m" or "$((4*1024*1024))".
If you ask me I'm all for allowing "4m" and I did that several times when testing things like the discard tunables, chunk size or the scrub limits. I consider the sysfs files an interface for human interaction so even if it's used like that in 10% of time it's still saving typing or allowing one to focus on the real prolem instead of shell sytax.
Quoted 36 lines · David Sterba
Quoted 15 lines · Qu Wenruo
Quoted 10 lines
The value that can be read from the sysfs file is in bytes and it's so that applications do not need to interpret it, like multiplying with 1024. We'll probably never return the pretty values with suffixes in sysfs files.
However, on the input side the suffixes are a convenience, setting to limit the throughput as '32m' is better than typing '32000000' and counting zeros or $((32*1024*1024)) or 33554432.
This is why memparse is there and kstrtoull does not do that.
That suffix is causing confusion already, just check my "25e" case. (It does not only lead to huge number, but also lead to incorrect value even if we treat "e" as a suffix)
Quoted 6 lines
Furthermore, the convenience argument is not that strong, you won't expect end users to do the change for a fs every time. Thus it's mostly managed by a small script or some other tool.
In that case I don't think doing extra bash calculation is a big deal anyway.
This is a nice study of convenience vs extra work "that's no big deal". The sysfs files can be used by scripts, often times it's the only access to some settings (like
/usr/bin/rescan-scsi-bus.sh/usr/bin/rescan-scsi-bus.sh). The value format, naming of the sysfs files is inconsistent and navigating to a particular directory and setting some magic value is I think common. Another example are trace points, or dynamic printk.There are tools abstracting that but sometimes it's a person that needs to type it and the shortcuts like the suffix are convenient because there's no extra need to type exact syntax and numbers for a $((...)) expression. Ask 3 people if they'd prefer to type "4m" or "$((4*1024*1024))".
Quoted 6 lines
If you ask me I'm all for allowing "4m" and I did that several times when testing things like the discard tunables, chunk size or the scrub limits. I consider the sysfs files an interface for human interaction so even if it's used like that in 10% of time it's still saving typing or allowing one to focus on the real prolem instead of shell sytax.
OK, then let me jump into the rabbit hole of stroxll family for my holiday projects, to make the memparse() itself more reliable and get rid of the caveats.
Hopefully allowing callers to select which suffixes they want to use.
(Well, at least I hope "25e" can give us a correct value after the fix)
Thanks,
Qu