Skip to contentSkip to conversation

lib: tst_bool_expr: Add support for strings

PATCH New Cyril Hrubis 2 participants 6 messages Nov 11, 2020 – Nov 12, 2020 Library code

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lib: tst_bool_expr: Add support for strings

There are cases where special characters, e.g. parentesis appear in the value of a kernel config variable, so in order to be able to parse boolean variables such as "CONFIG_DEFAULT_HOSTNAME=\"(none)\"" the tokenizer must be able to parse strings.

The implementation is easy, when in string we do not split the input into tokens.

---
 lib/newlib_tests/config01        |  1 +
 lib/newlib_tests/config02        |  1 +
 lib/newlib_tests/config03        |  1 +
 lib/newlib_tests/config04        |  1 +
 lib/newlib_tests/config05        |  1 +
 lib/newlib_tests/test_kconfig.c  |  1 +
 lib/newlib_tests/tst_bool_expr.c |  3 +++
 lib/tst_bool_expr.c              | 18 +++++++++++++-----
 8 files changed, 22 insertions(+), 5 deletions(-)
lib/newlib_tests/config01 +1 -0
index 96d68d836..1d94d810a 100644
@@ -2,3 +2,4 @@
 CONFIG_MMU=y
 CONFIG_EXT4_FS=m
 CONFIG_PGTABLE_LEVELS=4
+CONFIG_DEFAULT_HOSTNAME="(none)"
lib/newlib_tests/config02 +1 -0
index 2de45cff8..e1b0e8086 100644
@@ -2,3 +2,4 @@
 # CONFIG_MMU is not set
 CONFIG_EXT4_FS=m
 CONFIG_PGTABLE_LEVELS=4
+CONFIG_DEFAULT_HOSTNAME="(none)"
lib/newlib_tests/config03 +1 -0
index 1a3b9e648..05c8e194a 100644
@@ -2,3 +2,4 @@
 CONFIG_MMU=y
 CONFIG_EXT4_FS=m
 CONFIG_PGTABLE_LEVELS=44
+CONFIG_DEFAULT_HOSTNAME="(none)"
lib/newlib_tests/config04 +1 -0
index cce7051ae..da01579b6 100644
@@ -2,3 +2,4 @@
 CONFIG_MMU=y
 CONFIG_EXT4_FS=y
 CONFIG_PGTABLE_LEVELS=4
+CONFIG_DEFAULT_HOSTNAME="(none)"
lib/newlib_tests/config05 +1 -0
index a9d7bab4d..490f94fa6 100644
@@ -1,3 +1,4 @@
 # Everything is wrong
 CONFIG_EXT4_FS=y
 CONFIG_PGTABLE_LEVELS=44
+CONFIG_DEFAULT_HOSTNAME=""
lib/newlib_tests/test_kconfig.c +1 -0
index 1f659b95a..9280f07ca 100644
@@ -16,6 +16,7 @@ static const char *kconfigs[] = {
 	"CONFIG_PGTABLE_LEVELS=4",
 	"CONFIG_MMU & CONFIG_EXT4_FS=m",
 	"CONFIG_EXT4_FS=m | CONFIG_MMU",
+	"CONFIG_DEFAULT_HOSTNAME=\"(none)\"",
 	NULL
 };
 
lib/newlib_tests/tst_bool_expr.c +3 -0
index f9bb1780d..8f0929d35 100644
@@ -102,6 +102,9 @@ static void do_test(void)
 	do_eval_test("False & A", 1, 0, 0, 0);
 	do_eval_test("! Undefined", 0, 0, 0, -1);
 
+	do_eval_test("\"(none)\"", 0, 0, 0, -1);
+	do_eval_test("\"(none)\" & \" \"", 0, 0, 0, -1);
+
 	parse_fail("A!");
 	parse_fail("A &");
 	parse_fail("A B");
lib/tst_bool_expr.c +13 -5
index dd147cde3..35ffa5a80 100644
@@ -64,6 +64,7 @@ static unsigned int tokenize(const char *expr, struct tst_expr_tok *last)
 {
 	size_t i, j;
 	unsigned int token_cnt = 0;
+	int in_string = 0;
 
 	for (j = i = 0; expr[i]; i++) {
 		switch (expr[i]) {
@@ -72,14 +73,21 @@ static unsigned int tokenize(const char *expr, struct tst_expr_tok *last)
 		case '!':
 		case '&':
 		case '|':
-			token_cnt += new_tok(&last, &expr[j], i - j);
-			token_cnt += new_tok(&last, &expr[i], 1);
-			j = i+1;
+			if (!in_string) {
+				token_cnt += new_tok(&last, &expr[j], i - j);
+				token_cnt += new_tok(&last, &expr[i], 1);
+				j = i+1;
+			}
 		break;
 		case '\t':
 		case ' ':
-			token_cnt += new_tok(&last, &expr[j], i - j);
-			j = i+1;
+			if (!in_string) {
+				token_cnt += new_tok(&last, &expr[j], i - j);
+				j = i+1;
+			}
+		break;
+		case '"':
+			in_string = !in_string;
 		break;
 		default:
 		break;
Signature · 1 line
2.26.2
#1 lore raw

PATCH 1/1 lib: tst_bool_expr: Add support for strings

Hello,

lib/tst_bool_expr.c static unsigned int tokenize(const char *expr, struct tst_expr_tok *last) · line 69

Quoted 11 lines · Cyril Hrubis
lib/tst_bool_expr.c +1 -0
index dd147cde3..35ffa5a80 100644
@@ -64,6 +64,7 @@ static unsigned int tokenize(const char *expr, struct tst_expr_tok *last)
 {
 	size_t i, j;
 	unsigned int token_cnt = 0;
+	int in_string = 0;
 
 	for (j = i = 0; expr[i]; i++) {

Why not skip the whole switch statement if in_string and just check for the closing '"' instead?

Quoted 30 lines
lib/tst_bool_expr.c +12 -5
 		switch (expr[i]) {
@@ -72,14 +73,21 @@ static unsigned int tokenize(const char *expr, struct tst_expr_tok *last)
 		case '!':
 		case '&':
 		case '|':
-			token_cnt += new_tok(&last, &expr[j], i - j);
-			token_cnt += new_tok(&last, &expr[i], 1);
-			j = i+1;
+			if (!in_string) {
+				token_cnt += new_tok(&last, &expr[j], i - j);
+				token_cnt += new_tok(&last, &expr[i], 1);
+				j = i+1;
+			}
 		break;
 		case '\t':
 		case ' ':
-			token_cnt += new_tok(&last, &expr[j], i - j);
-			j = i+1;
+			if (!in_string) {
+				token_cnt += new_tok(&last, &expr[j], i - j);
+				j = i+1;
+			}
+		break;
+		case '"':
+			in_string = !in_string;
 		break;
 		default:
 		break;
Signature · 1 line
2.26.2

It should probably be an error if tokenize exits with in_string=1?

Signature · 2 lines
Thank you,
Richard.
#2 lore raw

PATCH 1/1 lib: tst_bool_expr: Add support for strings

Hi!

Quoted 2 lines

Why not skip the whole switch statement if in_string and just check for the closing '"' instead?

I guess that we can instead do:

lib/tst_bool_expr.c +7 -0
index dd147cde3..387c38b91 100644
@@ -81,6 +81,13 @@ static unsigned int tokenize(const char *expr, struct tst_expr_tok *last)
                        token_cnt += new_tok(&last, &expr[j], i - j);
                        j = i+1;
                break;
+               case '"':
+                       while (expr[i+1] != '"' && expr[i+1])
+                               i++;
+
+                       if (expr[i+1] == '"')
+                               i++;
+               break;
                default:
                break;
                }

Not sure if this is more readable.

Quoted 7 lines
Quoted 5 lines
(quoted diff) +0 -0
 		break;
 		default:
 		break;
Signature · 1 line
2.26.2

It should probably be an error if tokenize exits with in_string=1?

Well my opinion is that we do not care since we will get error later on, in case of the kernel config we would get obviously wrong variable value. But if you think that this is important we can change the return type to int and return -1 on tokenizer failure.

Signature · 2 lines
Cyril Hrubis
[email protected]
#3 lore raw

PATCH 1/1 lib: tst_bool_expr: Add support for strings

Hello,

Quoted 26 lines · Cyril Hrubis

Hi!

Quoted 2 lines

Why not skip the whole switch statement if in_string and just check for the closing '"' instead?

I guess that we can instead do:

lib/tst_bool_expr.c +7 -0
index dd147cde3..387c38b91 100644
@@ -81,6 +81,13 @@ static unsigned int tokenize(const char *expr, struct tst_expr_tok *last)
                        token_cnt += new_tok(&last, &expr[j], i - j);
                        j = i+1;
                break;
+               case '"':
+                       while (expr[i+1] != '"' && expr[i+1])
+                               i++;
+
+                       if (expr[i+1] == '"')
+                               i++;
+               break;
                default:
                break;
                }
Quoted 1 line

Not sure if this is more readable.

It's probably easier to understand, it atleast keeps string processing separate.

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
lib/tst_bool_expr.c +0 -0
 		break;
 		default:
 		break;
Signature · 1 line
2.26.2
Quoted 6 lines
Quoted 1 line

It should probably be an error if tokenize exits with in_string=1?

Well my opinion is that we do not care since we will get error later on, in case of the kernel config we would get obviously wrong variable value. But if you think that this is important we can change the return type to int and return -1 on tokenizer failure.

I suppose it depends how much later and how such an error would look. If the error will happen directly after leaving this function then I don't think it matters either, but if it is possible that it manages to complete the whole evaluation process before failing with TCONF because the vars don't match then this has the potential to waste some time.

Signature · 2 lines
Thank you,
Richard.
#4 lore raw

PATCH 1/1 lib: tst_bool_expr: Add support for strings

Hi!

Quoted 5 lines

I suppose it depends how much later and how such an error would look. If the error will happen directly after leaving this function then I don't think it matters either, but if it is possible that it manages to complete the whole evaluation process before failing with TCONF because the vars don't match then this has the potential to waste some time.

Well I can easily add this kind of check here, however out of all possible mistakes that you can do with misplaced quotes this would only catch the cases where the number of quotes is odd. I.e. only a subset of all possible errors, everything else would be caught later on when we attempt to evaluate the expression.

E.g.

CONFIG_FOO"=val"
"CONFIG_FOO=val & CONFIG_BAR"
"CONFIG_FOO"
etc.

are all valid in this context, but completely wrong when evaluated.

I guess that it would make more sense to check if variable token is sane in the upper layer, i.e. in the kconfig parser since we do have the full information about how it should look like there. And this check would also cover this case as well.

Signature · 2 lines
Cyril Hrubis
[email protected]
#5 lore raw

PATCH 1/1 lib: tst_bool_expr: Add support for strings

Hello,

Quoted 20 lines · Cyril Hrubis

Hi!

Quoted 5 lines

I suppose it depends how much later and how such an error would look. If the error will happen directly after leaving this function then I don't think it matters either, but if it is possible that it manages to complete the whole evaluation process before failing with TCONF because the vars don't match then this has the potential to waste some time.

Well I can easily add this kind of check here, however out of all possible mistakes that you can do with misplaced quotes this would only catch the cases where the number of quotes is odd. I.e. only a subset of all possible errors, everything else would be caught later on when we attempt to evaluate the expression.

E.g.

CONFIG_FOO"=val"
"CONFIG_FOO=val & CONFIG_BAR"
"CONFIG_FOO"
etc.

Quoted 6 lines

are all valid in this context, but completely wrong when evaluated.

I guess that it would make more sense to check if variable token is sane in the upper layer, i.e. in the kconfig parser since we do have the full information about how it should look like there. And this check would also cover this case as well.

+1

Signature · 2 lines
Thank you,
Richard.