Skip to contentSkip to conversation

Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

PATCH New Sean Anderson 5 participants 8 messages Jun 7, 2020 – Jun 15, 2020 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

This reverts commit 0486497e2b5f4d36fa968a1a60fea358cbf70b65.

The strtoul has well-defined semantics. It is defined by the C standard and POSIX. To quote the relevant section of the man pages,

Quoted 4 lines

If base is zero or 16, the string may then include a "0x" prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is '0', in which case it is taken as 8 (octal).

Keeping these semantics is important for several reasons. First, it is very surprising for standard library functions to behave differently than usual. Every other implementation of strtoul has different semantics than the implementation in U-Boot at the moment. Second, it can result in very surprising results from small changes. For example, changing the string "1f" to "20" causes the parsed value to *decrease*. Forcing use of the "0x" prefix to specify hexidecimal numbers is a feature, not a bug. Lastly, this is slightly less performant, since the entire number is parsed twice.

This fixes the str_simple_strtoul test failing with

test/str_ut.c:29, run_strtoul(): expect_val == val: Expected 0x44b (1099), got 0x1099ab (1087915) test/str_ut.c:46, str_simple_strtoul(): 0 == run_strtoul(uts, str2, 0, 1099, 4): Expected 0x0 (0), got 0x1 (1)

CC Michal Simek <[email protected]>
CC Shiril Tichkule <[email protected]>

---

 lib/strto.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)
lib/strto.c +1 -17
index 3d77115d4d..c00bb5895d 100644
@@ -22,25 +22,9 @@ static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
 				*base = 16;
 			else
 				*base = 8;
-		} else {
-			int i = 0;
-			char var;
-
+		} else
 			*base = 10;
-
-			do {
-				var = tolower(s[i++]);
-				if (var >= 'a' && var <= 'f') {
-					*base = 16;
-					break;
-				}
-
-				if (!(var >= '0' && var <= '9'))
-					break;
-			} while (var);
-		}
 	}
-
 	if (*base == 16 && s[0] == '0' && tolower(s[1]) == 'x')
 		s += 2;
 	return s;
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

Quoted 19 lines · Sean Anderson

This reverts commit 0486497e2b5f4d36fa968a1a60fea358cbf70b65.

The strtoul has well-defined semantics. It is defined by the C standard and POSIX. To quote the relevant section of the man pages,

Quoted 4 lines

If base is zero or 16, the string may then include a "0x" prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is '0', in which case it is taken as 8 (octal).

Keeping these semantics is important for several reasons. First, it is very surprising for standard library functions to behave differently than usual. Every other implementation of strtoul has different semantics than the implementation in U-Boot at the moment. Second, it can result in very surprising results from small changes. For example, changing the string "1f" to "20" causes the parsed value to *decrease*. Forcing use of the "0x" prefix to specify hexidecimal numbers is a feature, not a bug. Lastly, this is slightly less performant, since the entire number is parsed twice.

Quoted 5 lines

This fixes the str_simple_strtoul test failing with

test/str_ut.c:29, run_strtoul(): expect_val == val: Expected 0x44b (1099), got 0x1099ab (1087915) test/str_ut.c:46, str_simple_strtoul(): 0 == run_strtoul(uts, str2, 0, 1099, 4): Expected 0x0 (0), got 0x1 (1)

I suggest to add a Fixes line here:

Fixes 0486497e2b5f ("lib: Improve _parse_integer_fixup_radix base 16

detection")

Cf.
https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes

Quoted 13 lines
CC Michal Simek <[email protected]>
CC Shiril Tichkule <[email protected]>

---

 lib/strto.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)
lib/strto.c +0 -0
index 3d77115d4d..c00bb5895d 100644

lib/strto.c static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base) · line 25

Quoted 8 lines
lib/strto.c +1 -4
@@ -22,25 +22,9 @@ static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
 				*base = 16;
 			else
 				*base = 8;
-		} else {
-			int i = 0;
-			char var;
-
+		} else

scripts/checkpatch.pl reports a problem:

CHECK: Unbalanced braces around else statement
#50: FILE: lib/strto.c:25:
+ } else

Best regards

Heinrich

Quoted 19 lines
lib/strto.c +0 -13
 			*base = 10;
-
-			do {
-				var = tolower(s[i++]);
-				if (var >= 'a' && var <= 'f') {
-					*base = 16;
-					break;
-				}
-
-				if (!(var >= '0' && var <= '9'))
-					break;
-			} while (var);
-		}
 	}
-
 	if (*base == 16 && s[0] == '0' && tolower(s[1]) == 'x')
 		s += 2;
 	return s;
#2 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

Quoted 22 lines · Heinrich Schuchardt
Quoted 21 lines · Sean Anderson

This reverts commit 0486497e2b5f4d36fa968a1a60fea358cbf70b65.

The strtoul has well-defined semantics. It is defined by the C standard and POSIX. To quote the relevant section of the man pages,

Quoted 4 lines

If base is zero or 16, the string may then include a "0x" prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is '0', in which case it is taken as 8 (octal).

Keeping these semantics is important for several reasons. First, it is very surprising for standard library functions to behave differently than usual. Every other implementation of strtoul has different semantics than the implementation in U-Boot at the moment. Second, it can result in very surprising results from small changes. For example, changing the string "1f" to "20" causes the parsed value to *decrease*. Forcing use of the "0x" prefix to specify hexidecimal numbers is a feature, not a bug. Lastly, this is slightly less performant, since the entire number is parsed twice.

This fixes the str_simple_strtoul test failing with

Quoted 8 lines
Quoted 3 lines

test/str_ut.c:29, run_strtoul(): expect_val == val: Expected 0x44b (1099), got 0x1099ab (1087915) test/str_ut.c:46, str_simple_strtoul(): 0 == run_strtoul(uts, str2, 0, 1099, 4): Expected 0x0 (0), got 0x1 (1)

I suggest to add a Fixes line here:

Fixes 0486497e2b5f ("lib: Improve _parse_integer_fixup_radix base 16

detection")

Thanks, I will add that.

Quoted 28 lines

Cf.
https://www.kernel.org/doc/html/v4.17/process/submitting-patches.html#using-reported-by-tested-by-reviewed-by-suggested-by-and-fixes

Quoted 13 lines
CC Michal Simek <[email protected]>
CC Shiril Tichkule <[email protected]>

---

 lib/strto.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)
lib/strto.c +0 -0
index 3d77115d4d..c00bb5895d 100644

lib/strto.c static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base) · line 25

Quoted 8 lines
lib/strto.c +1 -4
@@ -22,25 +22,9 @@ static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
 				*base = 16;
 			else
 				*base = 8;
-		} else {
-			int i = 0;
-			char var;
-
+		} else

scripts/checkpatch.pl reports a problem:

Quoted 3 lines

CHECK: Unbalanced braces around else statement
#50: FILE: lib/strto.c:25:
+ } else

I chose this form since that is what is used in Linux, and that is how it formatted before the commit this reverts. I do think that it is better form to have braces around this else block, though.

--Sean

#3 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

In reply to Sean Anderson
Quoted 54 lines · Sean Anderson

This reverts commit 0486497e2b5f4d36fa968a1a60fea358cbf70b65.

The strtoul has well-defined semantics. It is defined by the C standard and POSIX. To quote the relevant section of the man pages,

Quoted 4 lines

If base is zero or 16, the string may then include a "0x" prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is '0', in which case it is taken as 8 (octal).

Keeping these semantics is important for several reasons. First, it is very surprising for standard library functions to behave differently than usual. Every other implementation of strtoul has different semantics than the implementation in U-Boot at the moment. Second, it can result in very surprising results from small changes. For example, changing the string "1f" to "20" causes the parsed value to *decrease*. Forcing use of the "0x" prefix to specify hexidecimal numbers is a feature, not a bug. Lastly, this is slightly less performant, since the entire number is parsed twice.

This fixes the str_simple_strtoul test failing with

test/str_ut.c:29, run_strtoul(): expect_val == val: Expected 0x44b (1099), got 0x1099ab (1087915) test/str_ut.c:46, str_simple_strtoul(): 0 == run_strtoul(uts, str2, 0, 1099, 4): Expected 0x0 (0), got 0x1 (1)

CC Michal Simek <[email protected]>
CC Shiril Tichkule <[email protected]>

---

 lib/strto.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)
lib/strto.c +1 -11
index 3d77115d4d..c00bb5895d 100644
@@ -22,25 +22,9 @@ static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
 				*base = 16;
 			else
 				*base = 8;
-		} else {
-			int i = 0;
-			char var;
-
+		} else
 			*base = 10;
-
-			do {
-				var = tolower(s[i++]);
-				if (var >= 'a' && var <= 'f') {
-					*base = 16;
-					break;
-				}

lib/strto.c static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base) · line 30

Quoted 10 lines
lib/strto.c +0 -6
-
-				if (!(var >= '0' && var <= '9'))
-					break;
-			} while (var);
-		}
 	}
-
 	if (*base == 16 && s[0] == '0' && tolower(s[1]) == 'x')
 		s += 2;
 	return s;

It is in u-boot mainline from February. Then we had to fix it in April. In the middle of this I have seen IIC one patchset which improves hex handling which is likely better way then this.
I am fine with reverting this patch but I would also like to see more comments in the code to say that we shouldn't really change this.

Thanks,
Michal

#4 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

Quoted 1 line · Michal Simek

It is in u-boot mainline from February. Then we had to fix it in April.

Do you have a link/commit hash for said fix?

Quoted 2 lines

In the middle of this I have seen IIC one patchset which improves hex handling which is likely better way then this.

Same for this.

Quoted 2 lines

I am fine with reverting this patch but I would also like to see more comments in the code to say that we shouldn't really change this.

This seems like a larger issue. There is a lot of code in U-Boot implementing the C standard library, especially the string library. Should we have a comment in each file which says something like "The functions here implement the C standard, think carefully before making semantic changes"?

--Sean

#5 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

Quoted 10 lines · Sean Anderson
Quoted 1 line · Michal Simek

It is in u-boot mainline from February. Then we had to fix it in April.

Do you have a link/commit hash for said fix?

Quoted 2 lines

In the middle of this I have seen IIC one patchset which improves hex handling which is likely better way then this.

Same for this.

Quoted 8 lines
Quoted 2 lines

I am fine with reverting this patch but I would also like to see more comments in the code to say that we shouldn't really change this.

This seems like a larger issue. There is a lot of code in U-Boot implementing the C standard library, especially the string library. Should we have a comment in each file which says something like "The functions here implement the C standard, think carefully before making semantic changes"?

Honestly? Yes, it would be good to note which functions do their best to implement / conform to C standard and which _may_ deviate from it.

For example, we have an alternative printf that doesn't support all format characters. Making that clearer (and then what ones our normal printf doesn't handle) would be good.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200608/ed11b6f6/attachment.sig>
#6 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

In reply to Michal Simek

Hi,

Quoted 65 lines · Michal Simek
Quoted 63 lines · Sean Anderson

This reverts commit 0486497e2b5f4d36fa968a1a60fea358cbf70b65.

The strtoul has well-defined semantics. It is defined by the C standard and POSIX. To quote the relevant section of the man pages,

Quoted 4 lines

If base is zero or 16, the string may then include a "0x" prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is '0', in which case it is taken as 8 (octal).

Keeping these semantics is important for several reasons. First, it is very surprising for standard library functions to behave differently than usual. Every other implementation of strtoul has different semantics than the implementation in U-Boot at the moment. Second, it can result in very surprising results from small changes. For example, changing the string "1f" to "20" causes the parsed value to *decrease*. Forcing use of the "0x" prefix to specify hexidecimal numbers is a feature, not a bug. Lastly, this is slightly less performant, since the entire number is parsed twice.

This fixes the str_simple_strtoul test failing with

test/str_ut.c:29, run_strtoul(): expect_val == val: Expected 0x44b (1099), got 0x1099ab (1087915) test/str_ut.c:46, str_simple_strtoul(): 0 == run_strtoul(uts, str2, 0, 1099, 4): Expected 0x0 (0), got 0x1 (1)

CC Michal Simek <[email protected]>
CC Shiril Tichkule <[email protected]>

---

 lib/strto.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)
lib/strto.c +1 -17
index 3d77115d4d..c00bb5895d 100644
@@ -22,25 +22,9 @@ static const char *_parse_integer_fixup_radix(const char *s, unsigned int *base)
                              *base = 16;
                      else
                              *base = 8;
-             } else {
-                     int i = 0;
-                     char var;
-
+             } else
                      *base = 10;
-
-                     do {
-                             var = tolower(s[i++]);
-                             if (var >= 'a' && var <= 'f') {
-                                     *base = 16;
-                                     break;
-                             }
-
-                             if (!(var >= '0' && var <= '9'))
-                                     break;
-                     } while (var);
-             }
      }
-
      if (*base == 16 && s[0] == '0' && tolower(s[1]) == 'x')
              s += 2;
      return s;
Quoted 8 lines
Quoted 1 line

It is in u-boot mainline from February. Then we had to fix it in April. In the middle of this I have seen IIC one patchset which improves hex handling which is likely better way then this.
I am fine with reverting this patch but I would also like to see more comments in the code to say that we shouldn't really change this.

Once this revert is in I think we should refrain from changing it until there are tests to cover its current behaviour, e.g. in str_ut.c

Regards,
Simon

#7 lore raw

PATCH 1/1 Revert "lib: Improve _parse_integer_fixup_radix base 16 detection"

In reply to Sean Anderson
Quoted 21 lines · Sean Anderson

This reverts commit 0486497e2b5f4d36fa968a1a60fea358cbf70b65.

The strtoul has well-defined semantics. It is defined by the C standard and POSIX. To quote the relevant section of the man pages,

Quoted 4 lines

If base is zero or 16, the string may then include a "0x" prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is '0', in which case it is taken as 8 (octal).

Keeping these semantics is important for several reasons. First, it is very surprising for standard library functions to behave differently than usual. Every other implementation of strtoul has different semantics than the implementation in U-Boot at the moment. Second, it can result in very surprising results from small changes. For example, changing the string "1f" to "20" causes the parsed value to *decrease*. Forcing use of the "0x" prefix to specify hexidecimal numbers is a feature, not a bug. Lastly, this is slightly less performant, since the entire number is parsed twice.

This fixes the str_simple_strtoul test failing with

Quoted 7 lines

test/str_ut.c:29, run_strtoul(): expect_val == val: Expected 0x44b (1099), got 0x1099ab (1087915) test/str_ut.c:46, str_simple_strtoul(): 0 == run_strtoul(uts, str2, 0, 1099, 4): Expected 0x0 (0), got 0x1 (1)

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200615/ed1c4051/attachment.sig>