Skip to contentSkip to conversation

qemu-storage-daemon: Remove QemuOpts from --object

PATCH In review Markus Armbruster 3 participants 18 messages Oct 11, 2020 – Oct 19, 2020

lore
Share & tools
lore
#0 lore raw

v4 cover qemu-storage-daemon: Remove QemuOpts from --object

This replaces the QemuOpts-based help code for --object in the storage daemon with code based on the keyval parser.

Review of v3 led me to preexisting issues. Instead of posting my fixes separately, then working with Kevin to rebase his work on top of mine, I did the rebase myself and am posting it together with my fixes as v4. Hope that's okay.

Note: to test qemu-storage-daemon --object help, you need Philippe's "[PATCH v3] hw/nvram: Always register FW_CFG_DATA_GENERATOR_INTERFACE" and its prerequisites (first 10 patches of Paolo's "[PULL 00/39] SCSI, qdev, qtest, meson patches for 2020-10-10"), so it doesn't crash with "missing interface 'fw_cfg-data-generator' for object 'tls-creds'".

v4:
- PATCH 1-3: New
- PATCH 4:
* Rebased
* Commit message typos [Eric]
* Replace flawed is_help_option_n() by starts_with_help_option()

    [me]

* Update grammar and accompanying prose [me]
* Update keyval_parse_one()'s contract, tweak keyval_parse()'s [me]
* Revert the keyval_parse_one() change to a rebased version of v1, because it's simpler and the edge case that led to the more complicated version no longer exists [me].
* Rearrange tests so the simple cases come first
- PATCH 5-7: Unchanged

v3:
- Always parse help options, no matter if the caller implements help or not. If it doesn't, return an error. [Markus]
- Document changes to the keyval parser grammar [Markus]
- Support both 'help' and '?' [Eric]
- Test case fixes [Eric]
- Improved documentation of user_creatable_print_help(_from_qdict) [Markus]

v2:
- Fixed double comma by reusing the existing key and value parsers [Eric]
- More tests to cover the additional cases

Kevin Wolf (4):
keyval: Parse help options
qom: Factor out helpers from user_creatable_print_help()
qom: Add user_creatable_print_help_from_qdict()
qemu-storage-daemon: Remove QemuOpts from --object parser

Markus Armbruster (3):
keyval: Fix and clarify grammar
test-keyval: Demonstrate misparse of ',' with implied key
keyval: Fix parsing of ',' in value of implied key

 include/qemu/help_option.h           |  11 ++
 include/qemu/option.h                |   2 +-
 include/qom/object_interfaces.h      |  21 ++-
 qapi/qobject-input-visitor.c         |   2 +-
 qom/object_interfaces.c              |  99 ++++++++------
 storage-daemon/qemu-storage-daemon.c |  15 +--
 tests/test-keyval.c                  | 187 ++++++++++++++++++---------
 util/keyval.c                        | 103 +++++++++++----
 8 files changed, 297 insertions(+), 143 deletions(-)
Signature · 1 line
2.26.2
#1 lore raw

PATCH v4 1/7 keyval: Fix and clarify grammar

The grammar has a few issues:

* key-fragment = / [^=,.]* /

Prose restricts key fragments: they "must be valid QAPI names or consist only of decimal digits". Technically, '' consists only of decimal digits. The code rejects that. Fix the grammar.

* val = { / [^,]* / | ',,' }

Use + instead of *. Accepts the same language.

* val-no-key = / [^=,]* /

The code rejects an empty value. Fix the grammar.

* Section "Additional syntax for use with an implied key" is confusing. Rewrite it.

---
 util/keyval.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)
util/keyval.c +10 -6
index 13def4af54..82d8497c71 100644
@@ -16,8 +16,8 @@
  *   key-vals     = [ key-val { ',' key-val } [ ',' ] ]
  *   key-val      = key '=' val
  *   key          = key-fragment { '.' key-fragment }
- *   key-fragment = / [^=,.]* /
- *   val          = { / [^,]* / | ',,' }
+ *   key-fragment = / [^=,.]+ /
+ *   val          = { / [^,]+ / | ',,' }
  *
  * Semantics defined by reduction to JSON:
  *
@@ -71,12 +71,16 @@
  * Awkward.  Note that we carefully restrict alternate types to avoid
  * similar ambiguity.
  *
- * Additional syntax for use with an implied key:
+ * Alternative syntax for use with an implied key:
  *
- *   key-vals-ik  = val-no-key [ ',' key-vals ]
- *   val-no-key   = / [^=,]* /
+ *   key-vals     = [ key-val-1st { ',' key-val } [ ',' ] ]
+ *   key-val-1st  = val-no-key | key-val
+ *   val-no-key   = / [^=,]+ /
  *
- * where no-key is syntactic sugar for implied-key=val-no-key.
+ * where val-no-key is syntactic sugar for implied-key=val-no-key.
+ *
+ * Note that you can't use the sugared form when the value contains
+ * '=' or ','.
  */
 
 #include "qemu/osdep.h"
Signature · 1 line
2.26.2
Reviewed-by on v4 1/7
#2 lore raw

Re: [PATCH v4 1/7] keyval: Fix and clarify grammar

Quoted 29 lines · Markus Armbruster

The grammar has a few issues:

* key-fragment = / [^=,.]* /

Prose restricts key fragments: they "must be valid QAPI names or consist only of decimal digits". Technically, '' consists only of decimal digits. The code rejects that. Fix the grammar.

* val = { / [^,]* / | ',,' }

Use + instead of *. Accepts the same language.

* val-no-key = / [^=,]* /

The code rejects an empty value. Fix the grammar.

* Section "Additional syntax for use with an implied key" is confusing. Rewrite it.

---
  util/keyval.c | 16 ++++++++++------
  1 file changed, 10 insertions(+), 6 deletions(-)
util/keyval.c +0 -0
index 13def4af54..82d8497c71 100644

util/keyval.c line 19

Quoted 6 lines
util/keyval.c +1 -2
@@ -16,8 +16,8 @@
   *   key-vals     = [ key-val { ',' key-val } [ ',' ] ]
   *   key-val      = key '=' val
   *   key          = key-fragment { '.' key-fragment }
- *   key-fragment = / [^=,.]* /
- *   val          = { / [^,]* / | ',,' }
+ *   key-fragment = / [^=,.]+ /

This requires a non-empty string. Good (we don't allow an empty key).

util/keyval.c line 20

Quoted 1 line
util/keyval.c +1 -0
+ *   val          = { / [^,]+ / | ',,' }

I agree that this has no real change. Previously, you allowed zero or more repetitions of a regex that could produce zero characters; now, each outer repetition must make progress.

Quoted 11 lines
util/keyval.c +1 -2
   *
   * Semantics defined by reduction to JSON:
   *
@@ -71,12 +71,16 @@
   * Awkward.  Note that we carefully restrict alternate types to avoid
   * similar ambiguity.
   *
- * Additional syntax for use with an implied key:
+ * Alternative syntax for use with an implied key:
   *
- *   key-vals-ik  = val-no-key [ ',' key-vals ]

util/keyval.c line 83

Quoted 10 lines
util/keyval.c +7 -2
- *   val-no-key   = / [^=,]* /
+ *   key-vals     = [ key-val-1st { ',' key-val } [ ',' ] ]
+ *   key-val-1st  = val-no-key | key-val
+ *   val-no-key   = / [^=,]+ /
   *
- * where no-key is syntactic sugar for implied-key=val-no-key.
+ * where val-no-key is syntactic sugar for implied-key=val-no-key.
+ *
+ * Note that you can't use the sugared form when the value contains
+ * '=' or ','.

Nor can you use the sugared form when the value is intended to be empty (although this may be academic, as your other patches enumerate the list of clients, and none of them seem to allow an empty value even when desugared).

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#3 lore raw

Re: [PATCH v4 1/7] keyval: Fix and clarify grammar

Quoted 52 lines · Eric Blake
Quoted 20 lines · Markus Armbruster

The grammar has a few issues:
* key-fragment = / [^=,.]* /
Prose restricts key fragments: they "must be valid QAPI names or consist only of decimal digits". Technically, '' consists only of decimal digits. The code rejects that. Fix the grammar.
* val = { / [^,]* / | ',,' }
Use + instead of *. Accepts the same language.
* val-no-key = / [^=,]* /
The code rejects an empty value. Fix the grammar.
* Section "Additional syntax for use with an implied key" is confusing. Rewrite it.

---
  util/keyval.c | 16 ++++++++++------
  1 file changed, 10 insertions(+), 6 deletions(-)
util/keyval.c +0 -0
index 13def4af54..82d8497c71 100644

util/keyval.c line 19

Quoted 6 lines
util/keyval.c +1 -2
@@ -16,8 +16,8 @@
   *   key-vals     = [ key-val { ',' key-val } [ ',' ] ]
   *   key-val      = key '=' val
   *   key          = key-fragment { '.' key-fragment }
- *   key-fragment = / [^=,.]* /
- *   val          = { / [^,]* / | ',,' }
+ *   key-fragment = / [^=,.]+ /

This requires a non-empty string. Good (we don't allow an empty key).

util/keyval.c line 20

Quoted 1 line
util/keyval.c +1 -0
+ *   val          = { / [^,]+ / | ',,' }

I agree that this has no real change. Previously, you allowed zero or more repetitions of a regex that could produce zero characters; now, each outer repetition must make progress.

Quoted 16 lines
util/keyval.c +4 -3
   *
   * Semantics defined by reduction to JSON:
   *
@@ -71,12 +71,16 @@
   * Awkward.  Note that we carefully restrict alternate types to avoid
   * similar ambiguity.
   *
- * Additional syntax for use with an implied key:
+ * Alternative syntax for use with an implied key:
   *
- *   key-vals-ik  = val-no-key [ ',' key-vals ]
- *   val-no-key   = / [^=,]* /
+ *   key-vals     = [ key-val-1st { ',' key-val } [ ',' ] ]
+ *   key-val-1st  = val-no-key | key-val
+ *   val-no-key   = / [^=,]+ /
   *
Quoted 8 lines

util/keyval.c line 83

Quoted 5 lines
util/keyval.c +4 -1
- * where no-key is syntactic sugar for implied-key=val-no-key.
+ * where val-no-key is syntactic sugar for implied-key=val-no-key.
+ *
+ * Note that you can't use the sugared form when the value contains
+ * '=' or ','.

Nor can you use the sugared form when the value is intended to be empty

True. Spelling it out wouldn't hurt. Takes a follow-up patch; this one is already in master.

Quoted 5 lines

(although this may be academic, as your other patches enumerate
the list of clients, and none of them seem to allow an empty value even when desugared).

Thanks!

#4 lore raw

PATCH v4 2/7 test-keyval: Demonstrate misparse of ', ' with implied key

In reply to Markus Armbruster

Add a test for "val,,ue" with implied key. Documentation says this should parse as implied key with value "val", then fail. The code parses it as implied key with value "val,ue", then succeeds. The next commit will fix it.

---
 tests/test-keyval.c | 7 +++++++
 1 file changed, 7 insertions(+)
tests/test-keyval.c +7 -0
index e331a84149..f02bdf7029 100644
@@ -182,6 +182,13 @@ static void test_keyval_parse(void)
     error_free_or_abort(&err);
     g_assert(!qdict);
 
+    /* Implied key's value can't have comma (qemu_opts_parse(): it can) */
+    /* BUG: it can */
+    qdict = keyval_parse("val,,ue", "implied", &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 1);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "val,ue");
+    qobject_unref(qdict);
+
     /* Empty key is not an implied key */
     qdict = keyval_parse("=val", "implied", &err);
     error_free_or_abort(&err);
Signature · 1 line
2.26.2
Reviewed-by on v4 2/7
#5 lore raw

Re: [PATCH v4 2/7] test-keyval: Demonstrate misparse of ',' with implied key

Quoted 9 lines · Markus Armbruster

Add a test for "val,,ue" with implied key. Documentation says this should parse as implied key with value "val", then fail. The code parses it as implied key with value "val,ue", then succeeds. The next commit will fix it.

---
  tests/test-keyval.c | 7 +++++++
  1 file changed, 7 insertions(+)
Quoted 20 lines
tests/test-keyval.c +7 -0
index e331a84149..f02bdf7029 100644
@@ -182,6 +182,13 @@ static void test_keyval_parse(void)
      error_free_or_abort(&err);
      g_assert(!qdict);
 
+    /* Implied key's value can't have comma (qemu_opts_parse(): it can) */
+    /* BUG: it can */
+    qdict = keyval_parse("val,,ue", "implied", &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 1);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "val,ue");
+    qobject_unref(qdict);
+
      /* Empty key is not an implied key */
      qdict = keyval_parse("=val", "implied", &err);
      error_free_or_abort(&err);
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#6 lore raw

PATCH v4 3/7 keyval: Fix parsing of ',' in value of implied key

In reply to Markus Armbruster

The previous commit demonstrated documentation and code disagree on parsing of ',' in the value of an implied key. Fix the code to match the documentation.

This breaks uses of keyval_parse() that pass an implied key and accept a value containing ','. None of the existing uses does:

* audiodev: implied key "driver" is enum AudiodevDriver, none of the values contains ','

* display: implied key "type" is enum DisplayType, none of the values contains ','

* blockdev: implied key "driver is enum BlockdevDriver, none of the values contains ','

* export: implied key "type" is enum BlockExportType, none of the values contains ','

* monitor: implied key "mode" is enum MonitorMode, none of the values contains ','

* nbd-server: no implied key.

---
 tests/test-keyval.c |  8 +++-----
 util/keyval.c       | 28 +++++++++++++++++-----------
 2 files changed, 20 insertions(+), 16 deletions(-)
tests/test-keyval.c +3 -5
index f02bdf7029..04c62cf045 100644
@@ -183,11 +183,9 @@ static void test_keyval_parse(void)
     g_assert(!qdict);
 
     /* Implied key's value can't have comma (qemu_opts_parse(): it can) */
-    /* BUG: it can */
-    qdict = keyval_parse("val,,ue", "implied", &error_abort);
-    g_assert_cmpuint(qdict_size(qdict), ==, 1);
-    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "val,ue");
-    qobject_unref(qdict);
+    qdict = keyval_parse("val,,ue", "implied", &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
 
     /* Empty key is not an implied key */
     qdict = keyval_parse("=val", "implied", &err);
util/keyval.c +17 -11
index 82d8497c71..8f33a36a7c 100644
@@ -173,7 +173,7 @@ static const char *keyval_parse_one(QDict *qdict, const char *params,
                                     const char *implied_key,
                                     Error **errp)
 {
-    const char *key, *key_end, *s, *end;
+    const char *key, *key_end, *val_end, *s, *end;
     size_t len;
     char key_in_cur[128];
     QDict *cur;
@@ -182,10 +182,12 @@ static const char *keyval_parse_one(QDict *qdict, const char *params,
     QString *val;
 
     key = params;
+    val_end = NULL;
     len = strcspn(params, "=,");
     if (implied_key && len && key[len] != '=') {
         /* Desugar implied key */
         key = implied_key;
+        val_end = params + len;
         len = strlen(implied_key);
     }
     key_end = key + len;
@@ -241,7 +243,11 @@ static const char *keyval_parse_one(QDict *qdict, const char *params,
 
     if (key == implied_key) {
         assert(!*s);
-        s = params;
+        val = qstring_from_substr(params, 0, val_end - params);
+        s = val_end;
+        if (*s == ',') {
+            s++;
+        }
     } else {
         if (*s != '=') {
             error_setg(errp, "Expected '=' after parameter '%.*s'",
@@ -249,19 +255,19 @@ static const char *keyval_parse_one(QDict *qdict, const char *params,
             return NULL;
         }
         s++;
-    }
 
-    val = qstring_new();
-    for (;;) {
-        if (!*s) {
-            break;
-        } else if (*s == ',') {
-            s++;
-            if (*s != ',') {
+        val = qstring_new();
+        for (;;) {
+            if (!*s) {
                 break;
+            } else if (*s == ',') {
+                s++;
+                if (*s != ',') {
+                    break;
+                }
             }
+            qstring_append_chr(val, *s++);
         }
-        qstring_append_chr(val, *s++);
     }
 
     if (!keyval_parse_put(cur, key_in_cur, val, key, key_end, errp)) {
Signature · 1 line
2.26.2
Reviewed-by on v4 3/7
#7 lore raw

Re: [PATCH v4 3/7] keyval: Fix parsing of ', ' in value of implied key

Quoted 22 lines · Markus Armbruster

The previous commit demonstrated documentation and code disagree on parsing of ',' in the value of an implied key. Fix the code to match the documentation.

This breaks uses of keyval_parse() that pass an implied key and accept a value containing ','. None of the existing uses does:

* audiodev: implied key "driver" is enum AudiodevDriver, none of the values contains ','

* display: implied key "type" is enum DisplayType, none of the values contains ','

* blockdev: implied key "driver is enum BlockdevDriver, none of the values contains ','

* export: implied key "type" is enum BlockExportType, none of the values contains ','

* monitor: implied key "mode" is enum MonitorMode, none of the values contains ','

Quoted 8 lines

* nbd-server: no implied key.

---
  tests/test-keyval.c |  8 +++-----
  util/keyval.c       | 28 +++++++++++++++++-----------
  2 files changed, 20 insertions(+), 16 deletions(-)
Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#8 lore raw

PATCH v4 4/7 keyval: Parse help options

In reply to Markus Armbruster

From: Kevin Wolf <[email protected]>

This adds a special meaning for 'help' and '?' as options to the keyval parser. Instead of being an error (because of a missing value) or a value for an implied key, they now request help, which is a new boolean output of the parser in addition to the QDict.

A new parameter 'p_help' is added to keyval_parse() that contains on return whether help was requested. If NULL is passed, requesting help results in an error and all other cases work like before.

Turning previous error cases into help is a compatible extension. The behaviour potentially changes for implied keys: They could previously get 'help' as their value, which is now interpreted as requesting help.

This is not a problem in practice because 'help' and '?' are not a valid values for the implied key of any option parsed with keyval_parse():

* audiodev: union Audiodev, implied key "driver" is enum AudiodevDriver, "help" and "?" are not among its values

* display: union DisplayOptions, implied key "type" is enum DisplayType, "help" and "?" are not among its values

* blockdev: union BlockdevOptions, implied key "driver is enum BlockdevDriver, "help" and "?" are not among its values

* export: union BlockExport, implied key "type" is enum BlockExportType, "help" and "?" are not among its values

* monitor: struct MonitorOptions, implied key "mode" is enum MonitorMode, "help" and "?" are not among its values

* nbd-server: struct NbdServerOptions, no implied key.

---
 include/qemu/help_option.h           |  11 ++
 include/qemu/option.h                |   2 +-
 qapi/qobject-input-visitor.c         |   2 +-
 storage-daemon/qemu-storage-daemon.c |   2 +-
 tests/test-keyval.c                  | 184 ++++++++++++++++++---------
 util/keyval.c                        |  63 +++++++--
 6 files changed, 186 insertions(+), 78 deletions(-)
include/qemu/help_option.h +11 -0
index 328d2a89fd..ca6389a154 100644
@@ -19,4 +19,15 @@ static inline bool is_help_option(const char *s)
     return !strcmp(s, "?") || !strcmp(s, "help");
 }
 
+static inline int starts_with_help_option(const char *s)
+{
+    if (*s == '?') {
+        return 1;
+    }
+    if (g_str_has_prefix(s, "help")) {
+        return 4;
+    }
+    return 0;
+}
+
 #endif
include/qemu/option.h +1 -1
index 05e8a15c73..ac69352e0e 100644
@@ -149,6 +149,6 @@ void qemu_opts_free(QemuOptsList *list);
 QemuOptsList *qemu_opts_append(QemuOptsList *dst, QemuOptsList *list);
 
 QDict *keyval_parse(const char *params, const char *implied_key,
-                    Error **errp);
+                    bool *help, Error **errp);
 
 #endif
qapi/qobject-input-visitor.c +1 -1
index f918a05e5f..7b184b50a7 100644
@@ -757,7 +757,7 @@ Visitor *qobject_input_visitor_new_str(const char *str,
         assert(args);
         v = qobject_input_visitor_new(QOBJECT(args));
     } else {
-        args = keyval_parse(str, implied_key, errp);
+        args = keyval_parse(str, implied_key, NULL, errp);
         if (!args) {
             return NULL;
         }
storage-daemon/qemu-storage-daemon.c +1 -1
index 1ae1cda481..6f0e0cfb36 100644
@@ -278,7 +278,7 @@ static void process_options(int argc, char *argv[])
                 }
                 qemu_opts_del(opts);
 
-                args = keyval_parse(optarg, "qom-type", &error_fatal);
+                args = keyval_parse(optarg, "qom-type", NULL, &error_fatal);
                 user_creatable_add_dict(args, true, &error_fatal);
                 qobject_unref(args);
                 break;
tests/test-keyval.c +121 -63

489 lines not on this page · raw message

util/keyval.c +51 -12

133 lines not on this page · raw message

Signature · 1 line
2.26.2
Reviewed-by on v4 4/7
#9 lore raw

Re: [PATCH v4 4/7] keyval: Parse help options

Quoted 59 lines · Markus Armbruster

From: Kevin Wolf <[email protected]>

This adds a special meaning for 'help' and '?' as options to the keyval parser. Instead of being an error (because of a missing value) or a value for an implied key, they now request help, which is a new boolean output of the parser in addition to the QDict.

A new parameter 'p_help' is added to keyval_parse() that contains on return whether help was requested. If NULL is passed, requesting help results in an error and all other cases work like before.

tests/test-keyval.c +47 -0
+
+    /* "help" by itself, without implied key */
+    qdict = keyval_parse("help", NULL, &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 0);
+    g_assert(help);
+    qobject_unref(qdict);
+
+    /* "help" by itself, with implied key */
+    qdict = keyval_parse("help", "implied", &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 0);
+    g_assert(help);
+    qobject_unref(qdict);
+
+    /* "help" when no help is available, without implied key */
+    qdict = keyval_parse("help", NULL, NULL, &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
+
+    /* "help" when no help is available, with implied key */
+    qdict = keyval_parse("help", "implied", NULL, &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
+
+    /* Key "help" */
+    qdict = keyval_parse("help=on", NULL, &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 1);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "help"), ==, "on");
+    g_assert(!help);
+    qobject_unref(qdict);
+
+    /* "help" followed by crap, without implied key */
+    qdict = keyval_parse("help.abc", NULL, &help, &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
+
+    /* "help" followed by crap, with implied key */
+    qdict = keyval_parse("help.abc", "implied", &help, &err);
+    g_assert_cmpuint(qdict_size(qdict), ==, 1);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "help.abc");
+    g_assert(!help);
+    qobject_unref(qdict);
+
+    /* "help" with other stuff, without implied key */
+    qdict = keyval_parse("number=42,help,foo=bar", NULL, &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 2);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "number"), ==, "42");
+    g_assert_cmpstr(qdict_get_try_str(qdict, "foo"), ==, "bar");

tests/test-keyval.c static void test_keyval_parse(void) · line 252

Quoted 10 lines
tests/test-keyval.c +10 -0
+    g_assert(help);
+    qobject_unref(qdict);
+
+    /* "help" with other stuff, with implied key */
+    qdict = keyval_parse("val,help,foo=bar", "implied", &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 2);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "val");
+    g_assert_cmpstr(qdict_get_try_str(qdict, "foo"), ==, "bar");
+    g_assert(help);
+    qobject_unref(qdict);

Is it worth checking that "helper" with implied key is a value, not help?

util/keyval.c line 21

Quoted 11 lines

+++ b/util/keyval.c

util/keyval.c +2 -1
@@ -14,10 +14,11 @@
   * KEY=VALUE,... syntax:
   *
   *   key-vals     = [ key-val { ',' key-val } [ ',' ] ]
- *   key-val      = key '=' val
+ *   key-val      = key '=' val | help
   *   key          = key-fragment { '.' key-fragment }
   *   key-fragment = / [^=,.]+ /
   *   val          = { / [^,]+ / | ',,' }
+ *   help         = 'help | '?'

Missing '

Otherwise

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#10 lore raw

Re: [PATCH v4 4/7] keyval: Parse help options

Quoted 63 lines · Eric Blake
Quoted 62 lines · Markus Armbruster

From: Kevin Wolf <[email protected]>
This adds a special meaning for 'help' and '?' as options to the keyval
parser. Instead of being an error (because of a missing value) or a value for an implied key, they now request help, which is a new boolean output of the parser in addition to the QDict.
A new parameter 'p_help' is added to keyval_parse() that contains on return whether help was requested. If NULL is passed, requesting help results in an error and all other cases work like before.

tests/test-keyval.c +51 -0
+
+    /* "help" by itself, without implied key */
+    qdict = keyval_parse("help", NULL, &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 0);
+    g_assert(help);
+    qobject_unref(qdict);
+
+    /* "help" by itself, with implied key */
+    qdict = keyval_parse("help", "implied", &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 0);
+    g_assert(help);
+    qobject_unref(qdict);
+
+    /* "help" when no help is available, without implied key */
+    qdict = keyval_parse("help", NULL, NULL, &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
+
+    /* "help" when no help is available, with implied key */
+    qdict = keyval_parse("help", "implied", NULL, &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
+
+    /* Key "help" */
+    qdict = keyval_parse("help=on", NULL, &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 1);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "help"), ==, "on");
+    g_assert(!help);
+    qobject_unref(qdict);
+
+    /* "help" followed by crap, without implied key */
+    qdict = keyval_parse("help.abc", NULL, &help, &err);
+    error_free_or_abort(&err);
+    g_assert(!qdict);
+
+    /* "help" followed by crap, with implied key */
+    qdict = keyval_parse("help.abc", "implied", &help, &err);
+    g_assert_cmpuint(qdict_size(qdict), ==, 1);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "help.abc");
+    g_assert(!help);
+    qobject_unref(qdict);
+
+    /* "help" with other stuff, without implied key */
+    qdict = keyval_parse("number=42,help,foo=bar", NULL, &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 2);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "number"), ==, "42");
+    g_assert_cmpstr(qdict_get_try_str(qdict, "foo"), ==, "bar");
+    g_assert(help);
+    qobject_unref(qdict);
+
+    /* "help" with other stuff, with implied key */
Quoted 8 lines

tests/test-keyval.c static void test_keyval_parse(void) · line 252

Quoted 6 lines
tests/test-keyval.c +6 -0
+    qdict = keyval_parse("val,help,foo=bar", "implied", &help, &error_abort);
+    g_assert_cmpuint(qdict_size(qdict), ==, 2);
+    g_assert_cmpstr(qdict_get_try_str(qdict, "implied"), ==, "val");
+    g_assert_cmpstr(qdict_get_try_str(qdict, "foo"), ==, "bar");
+    g_assert(help);
+    qobject_unref(qdict);

Is it worth checking that "helper" with implied key is a value, not help?

Case /* "help" followed by crap, with implied key */ covers this, doesn't it?

Quoted 5 lines
Quoted 5 lines

+++ b/util/keyval.c

util/keyval.c +0 -0
@@ -14,10 +14,11 @@
   * KEY=VALUE,... syntax:
   *
   *   key-vals     = [ key-val { ',' key-val } [ ',' ] ]
Quoted 8 lines

util/keyval.c line 21

Quoted 6 lines
util/keyval.c +2 -1
- *   key-val      = key '=' val
+ *   key-val      = key '=' val | help
   *   key          = key-fragment { '.' key-fragment }
   *   key-fragment = / [^=,.]+ /
   *   val          = { / [^,]+ / | ',,' }
+ *   help         = 'help | '?'

Missing '

Kevin fixed it up.

Quoted 2 lines

Otherwise

Thanks!

#11 lore raw

Re: [PATCH v4 4/7] keyval: Parse help options

In reply to Markus Armbruster

Am 11.10.2020 um 09:35 hat Markus Armbruster geschrieben:

Quoted 43 lines

From: Kevin Wolf <[email protected]>

This adds a special meaning for 'help' and '?' as options to the keyval parser. Instead of being an error (because of a missing value) or a value for an implied key, they now request help, which is a new boolean output of the parser in addition to the QDict.

A new parameter 'p_help' is added to keyval_parse() that contains on return whether help was requested. If NULL is passed, requesting help results in an error and all other cases work like before.

Turning previous error cases into help is a compatible extension. The behaviour potentially changes for implied keys: They could previously get 'help' as their value, which is now interpreted as requesting help.

This is not a problem in practice because 'help' and '?' are not a valid values for the implied key of any option parsed with keyval_parse():

* audiodev: union Audiodev, implied key "driver" is enum AudiodevDriver, "help" and "?" are not among its values

* display: union DisplayOptions, implied key "type" is enum DisplayType, "help" and "?" are not among its values

* blockdev: union BlockdevOptions, implied key "driver is enum BlockdevDriver, "help" and "?" are not among its values

* export: union BlockExport, implied key "type" is enum BlockExportType, "help" and "?" are not among its values

* monitor: struct MonitorOptions, implied key "mode" is enum MonitorMode, "help" and "?" are not among its values

* nbd-server: struct NbdServerOptions, no implied key.

tests/test-keyval.c +0 -0
index 04c62cf045..7b459900af 100644

tests/test-keyval.c line 4

Quoted 4 lines
tests/test-keyval.c +1 -0
@@ -1,3 +1,4 @@
+
 /*
  * Unit tests for parsing of KEY=VALUE,... strings
  *

This hunk looks unintentional.

Kevin

#12 lore raw

Re: [PATCH v4 4/7] keyval: Parse help options

Quoted 39 lines · Kevin Wolf

Am 11.10.2020 um 09:35 hat Markus Armbruster geschrieben:

Quoted 37 lines

From: Kevin Wolf <[email protected]>

This adds a special meaning for 'help' and '?' as options to the keyval parser. Instead of being an error (because of a missing value) or a value for an implied key, they now request help, which is a new boolean output of the parser in addition to the QDict.

A new parameter 'p_help' is added to keyval_parse() that contains on return whether help was requested. If NULL is passed, requesting help results in an error and all other cases work like before.

Turning previous error cases into help is a compatible extension. The behaviour potentially changes for implied keys: They could previously get 'help' as their value, which is now interpreted as requesting help.

This is not a problem in practice because 'help' and '?' are not a valid values for the implied key of any option parsed with keyval_parse():

* audiodev: union Audiodev, implied key "driver" is enum AudiodevDriver, "help" and "?" are not among its values

* display: union DisplayOptions, implied key "type" is enum DisplayType, "help" and "?" are not among its values

* blockdev: union BlockdevOptions, implied key "driver is enum BlockdevDriver, "help" and "?" are not among its values

* export: union BlockExport, implied key "type" is enum BlockExportType, "help" and "?" are not among its values

* monitor: struct MonitorOptions, implied key "mode" is enum MonitorMode, "help" and "?" are not among its values

* nbd-server: struct NbdServerOptions, no implied key.

Quoted 11 lines

tests/test-keyval.c line 4

Quoted 9 lines
tests/test-keyval.c +1 -0
index 04c62cf045..7b459900af 100644
@@ -1,3 +1,4 @@
+
 /*
  * Unit tests for parsing of KEY=VALUE,... strings
  *

This hunk looks unintentional.

It is. Thanks for fixing it up!

#13 lore raw

PATCH v4 5/7 qom: Factor out helpers from user_creatable_print_help()

In reply to Markus Armbruster

From: Kevin Wolf <[email protected]>

This creates separate helper functions for printing a list of user creatable object types and for printing a list of properties of a given type. This will allow using these parts without having a QemuOpts.

---
 qom/object_interfaces.c | 90 ++++++++++++++++++++++++-----------------
 1 file changed, 52 insertions(+), 38 deletions(-)
qom/object_interfaces.c +52 -38
index e8e1523960..3fd1da157e 100644
@@ -214,52 +214,66 @@ char *object_property_help(const char *name, const char *type,
     return g_string_free(str, false);
 }
 
-bool user_creatable_print_help(const char *type, QemuOpts *opts)
+static void user_creatable_print_types(void)
+{
+    GSList *l, *list;
+
+    printf("List of user creatable objects:\n");
+    list = object_class_get_list_sorted(TYPE_USER_CREATABLE, false);
+    for (l = list; l != NULL; l = l->next) {
+        ObjectClass *oc = OBJECT_CLASS(l->data);
+        printf("  %s\n", object_class_get_name(oc));
+    }
+    g_slist_free(list);
+}
+
+static bool user_creatable_print_type_properites(const char *type)
 {
     ObjectClass *klass;
+    ObjectPropertyIterator iter;
+    ObjectProperty *prop;
+    GPtrArray *array;
+    int i;
 
-    if (is_help_option(type)) {
-        GSList *l, *list;
+    klass = object_class_by_name(type);
+    if (!klass) {
+        return false;
+    }
 
-        printf("List of user creatable objects:\n");
-        list = object_class_get_list_sorted(TYPE_USER_CREATABLE, false);
-        for (l = list; l != NULL; l = l->next) {
-            ObjectClass *oc = OBJECT_CLASS(l->data);
-            printf("  %s\n", object_class_get_name(oc));
+    array = g_ptr_array_new();
+    object_class_property_iter_init(&iter, klass);
+    while ((prop = object_property_iter_next(&iter))) {
+        if (!prop->set) {
+            continue;
         }
-        g_slist_free(list);
+
+        g_ptr_array_add(array,
+                        object_property_help(prop->name, prop->type,
+                                             prop->defval, prop->description));
+    }
+    g_ptr_array_sort(array, (GCompareFunc)qemu_pstrcmp0);
+    if (array->len > 0) {
+        printf("%s options:\n", type);
+    } else {
+        printf("There are no options for %s.\n", type);
+    }
+    for (i = 0; i < array->len; i++) {
+        printf("%s\n", (char *)array->pdata[i]);
+    }
+    g_ptr_array_set_free_func(array, g_free);
+    g_ptr_array_free(array, true);
+    return true;
+}
+
+bool user_creatable_print_help(const char *type, QemuOpts *opts)
+{
+    if (is_help_option(type)) {
+        user_creatable_print_types();
         return true;
     }
 
-    klass = object_class_by_name(type);
-    if (klass && qemu_opt_has_help_opt(opts)) {
-        ObjectPropertyIterator iter;
-        ObjectProperty *prop;
-        GPtrArray *array = g_ptr_array_new();
-        int i;
-
-        object_class_property_iter_init(&iter, klass);
-        while ((prop = object_property_iter_next(&iter))) {
-            if (!prop->set) {
-                continue;
-            }
-
-            g_ptr_array_add(array,
-                            object_property_help(prop->name, prop->type,
-                                                 prop->defval, prop->description));
-        }
-        g_ptr_array_sort(array, (GCompareFunc)qemu_pstrcmp0);
-        if (array->len > 0) {
-            printf("%s options:\n", type);
-        } else {
-            printf("There are no options for %s.\n", type);
-        }
-        for (i = 0; i < array->len; i++) {
-            printf("%s\n", (char *)array->pdata[i]);
-        }
-        g_ptr_array_set_free_func(array, g_free);
-        g_ptr_array_free(array, true);
-        return true;
+    if (qemu_opt_has_help_opt(opts)) {
+        return user_creatable_print_type_properites(type);
     }
 
     return false;
Signature · 1 line
2.26.2
#14 lore raw

PATCH v4 6/7 qom: Add user_creatable_print_help_from_qdict()

In reply to Markus Armbruster

From: Kevin Wolf <[email protected]>

This adds a function that, given a QDict of non-help options, prints help for user creatable objects.

---
 include/qom/object_interfaces.h | 21 ++++++++++++++++++---
 qom/object_interfaces.c         |  9 +++++++++
 2 files changed, 27 insertions(+), 3 deletions(-)
include/qom/object_interfaces.h +18 -3
index f118fb516b..07d5cc8832 100644
@@ -154,13 +154,28 @@ int user_creatable_add_opts_foreach(void *opaque,
  * @type: the QOM type to be added
  * @opts: options to create
  *
- * Prints help if requested in @opts.
+ * Prints help if requested in @type or @opts. Note that if @type is neither
+ * "help"/"?" nor a valid user creatable type, no help will be printed
+ * regardless of @opts.
  *
- * Returns: true if @opts contained a help option and help was printed, false
- * if no help option was found.
+ * Returns: true if a help option was found and help was printed, false
+ * otherwise.
  */
 bool user_creatable_print_help(const char *type, QemuOpts *opts);
 
+/**
+ * user_creatable_print_help_from_qdict:
+ * @args: options to create
+ *
+ * Prints help considering the other options given in @args (if "qom-type" is
+ * given and valid, print properties for the type, otherwise print valid types)
+ *
+ * In contrast to user_creatable_print_help(), this function can't return that
+ * no help was requested. It should only be called if we know that help is
+ * requested and it will always print some help.
+ */
+void user_creatable_print_help_from_qdict(QDict *args);
+
 /**
  * user_creatable_del:
  * @id: the unique ID for the object
qom/object_interfaces.c +9 -0
index 3fd1da157e..ed896fe764 100644
@@ -279,6 +279,15 @@ bool user_creatable_print_help(const char *type, QemuOpts *opts)
     return false;
 }
 
+void user_creatable_print_help_from_qdict(QDict *args)
+{
+    const char *type = qdict_get_try_str(args, "qom-type");
+
+    if (!type || !user_creatable_print_type_properites(type)) {
+        user_creatable_print_types();
+    }
+}
+
 bool user_creatable_del(const char *id, Error **errp)
 {
     Object *container;
Signature · 1 line
2.26.2
#15 lore raw

PATCH v4 7/7 qemu-storage-daemon: Remove QemuOpts from --object parser

In reply to Markus Armbruster

From: Kevin Wolf <[email protected]>

The command line parser for --object parses the input twice: Once into QemuOpts just for detecting help options, and then again into a QDict using the keyval parser for actually creating the object.

Now that the keyval parser can also detect help options, we can simplify this and remove the QemuOpts part.

---
 storage-daemon/qemu-storage-daemon.c | 15 ++++-----------
 1 file changed, 4 insertions(+), 11 deletions(-)
storage-daemon/qemu-storage-daemon.c +4 -11
index 6f0e0cfb36..e419ba9f19 100644
@@ -264,21 +264,14 @@ static void process_options(int argc, char *argv[])
             }
         case OPTION_OBJECT:
             {
-                QemuOpts *opts;
-                const char *type;
                 QDict *args;
+                bool help;
 
-                /* FIXME The keyval parser rejects 'help' arguments, so we must
-                 * unconditionall try QemuOpts first. */
-                opts = qemu_opts_parse(&qemu_object_opts,
-                                       optarg, true, &error_fatal);
-                type = qemu_opt_get(opts, "qom-type");
-                if (type && user_creatable_print_help(type, opts)) {
+                args = keyval_parse(optarg, "qom-type", &help, &error_fatal);
+                if (help) {
+                    user_creatable_print_help_from_qdict(args);
                     exit(EXIT_SUCCESS);
                 }
-                qemu_opts_del(opts);
-
-                args = keyval_parse(optarg, "qom-type", NULL, &error_fatal);
                 user_creatable_add_dict(args, true, &error_fatal);
                 qobject_unref(args);
                 break;
Signature · 1 line
2.26.2
#16 lore raw
In reply to Markus Armbruster

Am 11.10.2020 um 09:34 hat Markus Armbruster geschrieben:

Quoted 8 lines

This replaces the QemuOpts-based help code for --object in the storage daemon with code based on the keyval parser.

Review of v3 led me to preexisting issues. Instead of posting my fixes separately, then working with Kevin to rebase his work on top of mine, I did the rebase myself and am posting it together with my fixes as v4. Hope that's okay.

Quoted 5 lines

Note: to test qemu-storage-daemon --object help, you need Philippe's "[PATCH v3] hw/nvram: Always register FW_CFG_DATA_GENERATOR_INTERFACE" and its prerequisites (first 10 patches of Paolo's "[PULL 00/39] SCSI, qdev, qtest, meson patches for 2020-10-10"), so it doesn't crash with "missing interface 'fw_cfg-data-generator' for object 'tls-creds'".

Thanks, fixed up the typos in patch 4 and applied patches 1-4. I'm taking patches 5-7 from my original series because you didn't add your S-o-b (they are unchanged anyway, so no practical difference).

Kevin

#17 lore raw
Quoted 15 lines · Kevin Wolf

Am 11.10.2020 um 09:34 hat Markus Armbruster geschrieben:

Quoted 8 lines

This replaces the QemuOpts-based help code for --object in the storage daemon with code based on the keyval parser.

Review of v3 led me to preexisting issues. Instead of posting my fixes separately, then working with Kevin to rebase his work on top of mine, I did the rebase myself and am posting it together with my fixes as v4. Hope that's okay.

Quoted 5 lines

Note: to test qemu-storage-daemon --object help, you need Philippe's "[PATCH v3] hw/nvram: Always register FW_CFG_DATA_GENERATOR_INTERFACE" and its prerequisites (first 10 patches of Paolo's "[PULL 00/39] SCSI, qdev, qtest, meson patches for 2020-10-10"), so it doesn't crash with "missing interface 'fw_cfg-data-generator' for object 'tls-creds'".

Quoted 3 lines

Thanks, fixed up the typos in patch 4 and applied patches 1-4. I'm taking patches 5-7 from my original series because you didn't add your S-o-b (they are unchanged anyway, so no practical difference).

Makes sense. I didn't feel like slapping my S-o-b on patches I reposted unchanged.

Thanks!