Skip to contentSkip to conversation

QAPI/QOM work for non-scalar object properties

PATCH Superseded Daniel P. Berrange 3 participants 32 messages Sep 27, 2016 – Oct 21, 2016

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v14 cover QAPI/QOM work for non-scalar object properties

An update of a series previously posted

v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
v5: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg00485.html
v6: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg03876.html
v7: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg00919.html
v8: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03115.html
v9: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02653.html
v10: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02694.html
v11: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg00652.html
v12: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg03559.html
v13: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04212.html

This series provides the infrastructure to allow use of non-scalar properties with the -object CLI arg, and object_add monitor commands. eg a property which is a list of structs. The syntax used for this is intentionally compatible with the syntax used by the block layer. This will allow the qdict_crumple method to be used by the block layer to convert from QemuOpts into structured QAPI block layer structs at a future date. It is already used by one of Max's patch series, by Kevin's -blockdev series, and by my own ACL series.

This patch series has grown a fair bit larger since the previous posting (19 patches instead of 6). This was to avoid trying to squish too much into one patch, to make it easier to review.

It does not neccessarily need tobe merged in one go. Patches 1-12 deal with the QAPI visitor logic and can be merged indendepedently which would unblock other people waiting on this. Patches 13-19 deal with converting code over to use the new visitor logic and can be fed in via various maintainers trees if desired.

Once this is merged, there is also scope to further enhance the QObjectInputVisitor to deal with visiting single-properties, which would let us delete StringInputVisitor too. This would mean we have a single input visitor class, instead of the current three.

Changed in v14:

- Extend qemu_opts_to_qdict to optionally handle case of repeated keys without dropping them.
- Support parsing of integer ranges for "list of int" visits for back compat with OptsVisitor
- Support visiting nested structs which are represented as a flat keyspace for back compat with OptsVisitor
- Add trace events for visitor functions to facilitate debugging
- Add a QObjectInputVisitor constructor which takes a QemuOpts object, to avoid repeated code patterns in callers
- Greater test coverage of -object to validate correct handling of "list of ints" legacy OptsVisitor syntax
- Convert all other use of OptsVisitor to QObjectInputVisitor
- Delete OptsVisitor

Changed in v13:

- Fix typos (Kevin)
- Remove unneeded line breaks (Kevin)

Changed in v12:

- More user friendly error message for mixing dict/list keys (Kevin)
- Report error instead of assert for non-contiguous list keys (Kevin)
- Fix tests for non-contiguous list keys (Kevin)
- Add tests for escaping of '.' when crumpling (Kevin)
- Fix remaining references to Qmp(In|Out)putVisitor (Markus)
- Misc typos / whitespace fixes (Eric, Kevin)
- Avoid touching 'ret' when parsing int64 fails (Eric)
- Testing of more edge cases in QObjectInputVisitor (Eric)
- Simplify API doc format (Markus)
- Use parse_option_size instead of qemu_strtosz_suffix for consistency (Kevin)
- Use safer qobject_to_qdict casts (Eric)

Changed in v11:

- Split QAPI/QOM patches off from the access control patches

Changed in v10:

- Fixed stupid build mistake

Changed in v9:

- Rename QmpInputVisitor -> QObjectInputVisitor (Markus/Eric)
- Rename QmpOutputVisitor -> QObjectOutputVisitor (Markus/Eric)
- Drop "strict" param from qobject_string_visitor_new() (Marus)
- Misc docs typos
- Add a visitor able to use strict or string types (for Eric's netdev series)
- Add a authorization API implementation that uses PAM

Changed in v8:

- Rebase due to merge of Visitor API changes (Eric)

Changed in v7:

- Misc typos in API docs (Marc-André)
- Fix parsing of properties using type_size visitor (Marc-André)
- Mark based auth class as abstract (Marc-André)
- Fix QAPI version annotations to say 2.7 (Marc-André)

Changed in v6:

- Switch from while() to for() loop for iterating over dicts (Markus)
- Avoid redundant strdup (Markus)
- Rewrap comments at 70 chars (Markus)
- Change qdict_list_size() to qdict_is_list() (Markus)
- Misc docs changes (Markus)
- Change QmpInputVisitor so the code for handling the string types is separate from code using native scalar types (Paolo)
- Centralize code parsing bool strings (Markus)
- Centralize code parsing int strings (Markus)

Changed in v5:

- Resolved conflicts with Eric's visitor refactoring which made it stricter about struct begin/end calls
- Added support for ACLs to migration code now its TLS support is merged.
- Fixed typos in example in commit message

Changed in v4:
- Ensure examples use shell escaping for '*' (Eric)
- Add more tests for crumple impl (Eric)
- Raise error if sasl-acl/tls-acl are requested but sasl/tls auth are not enabled (Eric)
- Document return codes for auth check more clearly (Eric)
- Don't silently turn a glob match into a strcmp
- Other misc small typos/fixes (Eric)

Changed in v3:

- Created separate qdict_list_size method (Max)
- Added unit tests for case of empty dict (Max)
- Fix variable names to use underscore separator (Max)
- Fix potential free of uninitialized variables (Max)
- Use QObject APIs for casts, instead of C type casts (Max)

Changed in v2:

- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple

Daniel P. Berrange (19):
qdict: implement a qdict_crumple method for un-flattening a dict
option: make parse_option_bool/number non-static
option: allow qemu_opts_to_qdict to merge repeated options
qapi: add trace events for visitor
qapi: rename QmpInputVisitor to QObjectInputVisitor
qapi: rename QmpOutputVisitor to QObjectOutputVisitor
qapi: don't pass two copies of TestInputVisitorData to tests
qapi: permit scalar type conversions in QObjectInputVisitor
qapi: permit auto-creating single element lists
qapi: permit auto-creating nested structs
qapi: add integer range support for QObjectInputVisitor
qapi: allow QObjectInputVisitor to be created with QemuOpts
qom: support non-scalar properties with -object
hmp: support non-scalar properties with object_add
numa: convert to use QObjectInputVisitor for -numa
block: convert crypto driver to use QObjectInputVisitor
acpi: convert to QObjectInputVisitor for -acpi parsing
net: convert to QObjectInputVisitor for -net/-netdev parsing
qapi: delete unused OptsVisitor code

 Makefile.objs                                      |   1 +
 block/crypto.c                                     |  12 +-
 block/qapi.c                                       |   4 +-
 blockdev.c                                         |  11 +-
 docs/qapi-code-gen.txt                             |   4 +-
 hmp.c                                              |  25 +-
 hw/acpi/core.c                                     |  14 +-
 include/qapi/opts-visitor.h                        |  40 --
 include/qapi/qmp-input-visitor.h                   |  30 -
 include/qapi/qmp/qdict.h                           |   1 +
 include/qapi/qobject-input-visitor.h               | 124 ++++
 ...p-output-visitor.h => qobject-output-visitor.h} |  10 +-
 include/qapi/visitor.h                             |   6 +-
 include/qemu/option.h                              |  12 +-
 include/sysemu/numa_int.h                          |  11 +
 monitor.c                                          |   5 +-
 net/net.c                                          |  17 +-
 numa.c                                             |  36 +-
 qapi/Makefile.objs                                 |   6 +-
 qapi/opts-visitor.c                                | 561 ---------------
 qapi/qapi-clone-visitor.c                          |   2 +-
 qapi/qapi-visit-core.c                             |  27 +
 qapi/qmp-input-visitor.c                           | 412 -----------
 qapi/qmp-output-visitor.c                          | 256 -------
 qapi/qobject-input-visitor.c                       | 778 +++++++++++++++++++++
 qapi/qobject-output-visitor.c                      | 254 +++++++
 qapi/trace-events                                  |  33 +
 qemu-img.c                                         |  12 +-
 qemu-io-cmds.c                                     |   3 +-
 qemu-io.c                                          |   6 +-
 qemu-nbd.c                                         |   3 +-
 qmp.c                                              |   4 +-
 qobject/qdict.c                                    | 291 ++++++++
 qom/object_interfaces.c                            |  41 +-
 qom/qom-qobject.c                                  |   8 +-
 scripts/qapi-commands.py                           |   8 +-
 scripts/qapi-event.py                              |   4 +-
 stubs/Makefile.objs                                |   5 +
 stubs/exec.c                                       |   6 +
 stubs/hostmem.c                                    |  14 +
 stubs/memory.c                                     |  41 ++
 stubs/qdev.c                                       |   8 +
 stubs/vl.c                                         |   8 +
 stubs/vmstate.c                                    |   4 +
 target-s390x/cpu_models.c                          |   4 +-
 tests/.gitignore                                   |   6 +-
 tests/Makefile.include                             |  27 +-
 tests/check-qdict.c                                | 260 +++++++
 tests/check-qnull.c                                |   8 +-
 tests/check-qom-proplist.c                         | 367 +++++++++-
 tests/qemu-iotests/051.out                         |   6 +-
 tests/qemu-iotests/051.pc.out                      |   6 +-
 tests/qemu-iotests/137.out                         |   4 +-
 tests/test-numa.c                                  | 116 +++
 tests/test-opts-visitor.c                          | 268 -------
 tests/test-qemu-opts.c                             |  40 ++
 tests/test-qmp-commands.c                          |   4 +-
 ...-input-strict.c => test-qobject-input-strict.c} |   6 +-
 ...nput-visitor.c => test-qobject-input-visitor.c} | 637 +++++++++++++++--
 ...put-visitor.c => test-qobject-output-visitor.c} |   6 +-
 tests/test-replication.c                           |   9 +-
 tests/test-string-input-visitor.c                  |   2 +-
 tests/test-string-output-visitor.c                 |   2 +-
 tests/test-visitor-serialization.c                 |   8 +-
 util/qemu-option.c                                 |  68 +-
 util/qemu-sockets.c                                |   4 +-
 vl.c                                               |   1 -
 67 files changed, 3246 insertions(+), 1771 deletions(-)
 delete mode 100644 include/qapi/opts-visitor.h
 delete mode 100644 include/qapi/qmp-input-visitor.h
 create mode 100644 include/qapi/qobject-input-visitor.h
 rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
 create mode 100644 include/sysemu/numa_int.h
 delete mode 100644 qapi/opts-visitor.c
 delete mode 100644 qapi/qmp-input-visitor.c
 delete mode 100644 qapi/qmp-output-visitor.c
 create mode 100644 qapi/qobject-input-visitor.c
 create mode 100644 qapi/qobject-output-visitor.c
 create mode 100644 qapi/trace-events
 create mode 100644 stubs/exec.c
 create mode 100644 stubs/hostmem.c
 create mode 100644 stubs/memory.c
 create mode 100644 stubs/qdev.c
 create mode 100644 stubs/vl.c
 create mode 100644 tests/test-numa.c
 delete mode 100644 tests/test-opts-visitor.c
 rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
 rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (57%)
 rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)
Signature · 1 line
2.7.4
#1 lore raw

PATCH v14 1/19 qdict: implement a qdict_crumple method for un-flattening a dict

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

{
'foo': [

     { 'bar': 'one', 'wizz': '1' },
     { 'bar': 'two', 'wizz': '2' }

],
}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nesting used when the same object is defined over QMP.

---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 291 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 260 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 552 insertions(+)
include/qapi/qmp/qdict.h +1 -0
index 71b8eb0..e0d24e1 100644
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(const QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
qobject/qdict.c +291 -0

303 lines not on this page · raw message

tests/check-qdict.c +260 -0

278 lines not on this page · raw message

Signature · 1 line
2.7.4
Reviewed-by on v14 1/19
#2 lore raw

Re: [Qemu-devel] [PATCH v14 01/19] qdict: implement a qdict_crumple method for un-flattening a dict

Quoted 28 lines · Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nesting used when the same object is defined over QMP.

---

qobject/qdict.c +2 -0
+
+/**

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 698

Quoted 10 lines
qobject/qdict.c +10 -0
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to remaining suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it into two parts
+ * at the first '.' separator. Allows double dot ('..') to escape the
+ * normal separator.
+ *
+ * eg

s/eg/e.g./ or just spell it as 'for example'

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 800

Quoted 4 lines
qobject/qdict.c +3 -0
+static int qdict_is_list(QDict *maybe_list, Error **errp)
 
+
+    /* NB this isn't a perfect check - eg it won't catch

Another such use.

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 806

Quoted 6 lines
qobject/qdict.c +6 -0
+     * a list containing '1', '+1', '01', '3', but that
+     * does not matter - we've still proved that the
+     * input is a list. It is up the caller to do a
+     * stricter check if desired */
+    if (len != (max + 1)) {
+        error_setg(errp, "List indexes are not contiguous, "

s/indexes/indices/ ? (my spellchecker likes both, but indexes is a sign that modern English speakers are getting lazy and drifting away from Latin)

Quoted 11 lines

+/**
+ * qdict_crumple:
+ * @src: the original flat dictionary (only scalar values) to crumple
+ * @recursive: true to recursively crumple nested dictionaries

+ * For example, an input of:
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * will result in any output of:

s/any/an/

Quoted 8 lines

+ *
+ * {
+ * 'foo': [
+ * { 'bar': 'one', 'wizz': '1' },
+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *

Quoted 8 lines

+ * The following scenarios in the input dict will result in an
+ * error being returned:
+ *
+ * - Any values in @src are non-scalar types
+ * - If keys in @src imply that a particular level is both a
+ * list and a dict. eg, "foo.0.bar" and "foo.eek.bar".
+ * - If keys in @src imply that a particular level is a list,
+ * but the indexes are non-contigous. eg "foo.0.bar" and

s/contigous/contiguous/

and another pesky 'eg'

Modulo typo fixes and potential grammar changes,

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#3 lore raw

PATCH v14 2/19 option: make parse_option_bool/number non-static

In reply to Daniel P. Berrange

The opts-visitor.c opts_type_bool() method has code for parsing a string to set a bool value, as does the qemu-option.c parse_option_bool() method, except it handles fewer cases.

To enable consistency across the codebase, extend parse_option_bool() to handle "yes", "no", "y" and "n", and make it non-static. Convert the opts visitor to call this method directly.

Also make parse_option_number() non-static to allow for similar reuse later.

---
 include/qemu/option.h         |  4 ++++
 qapi/opts-visitor.c           | 19 +------------------
 tests/qemu-iotests/051.out    |  6 +++---
 tests/qemu-iotests/051.pc.out |  6 +++---
 tests/qemu-iotests/137.out    |  4 ++--
 util/qemu-option.c            | 27 ++++++++++++++++-----------
 6 files changed, 29 insertions(+), 37 deletions(-)
include/qemu/option.h +4 -0
index 1f9e3f9..2a5266f 100644
@@ -37,6 +37,10 @@ int get_param_value(char *buf, int buf_size,
                     const char *tag, const char *str);
 
 
+void parse_option_bool(const char *name, const char *value, bool *ret,
+                       Error **errp);
+void parse_option_number(const char *name, const char *value,
+                         uint64_t *ret, Error **errp);
 void parse_option_size(const char *name, const char *value,
                        uint64_t *ret, Error **errp);
 bool has_help_option(const char *param);
qapi/opts-visitor.c +1 -18
index 1048bbc..084f7cc 100644
@@ -334,7 +334,6 @@ opts_type_str(Visitor *v, const char *name, char **obj, Error **errp)
 }
 
 
-/* mimics qemu-option.c::parse_option_bool() */
 static void
 opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
 {
@@ -346,23 +345,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
         return;
     }
 
-    if (opt->str) {
-        if (strcmp(opt->str, "on") == 0 ||
-            strcmp(opt->str, "yes") == 0 ||
-            strcmp(opt->str, "y") == 0) {
-            *obj = true;
-        } else if (strcmp(opt->str, "off") == 0 ||
-            strcmp(opt->str, "no") == 0 ||
-            strcmp(opt->str, "n") == 0) {
-            *obj = false;
-        } else {
-            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
-                       "on|yes|y|off|no|n");
-            return;
-        }
-    } else {
-        *obj = true;
-    }
+    parse_option_bool(opt->name, opt->str, obj, errp);
 
     processed(ov, name);
 }
tests/qemu-iotests/051.out +3 -3
index 408d613..ffdc0b9 100644
@@ -86,13 +86,13 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) qququiquit
 
 Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 
 Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 
 Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 
 
 === With version 2 images enabling lazy refcounts must fail ===
tests/qemu-iotests/051.pc.out +3 -3
index ec6d222..6abf1ab 100644
@@ -86,13 +86,13 @@ QEMU X.Y.Z monitor - type 'help' for more information
 (qemu) qququiquit
 
 Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 
 Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=42: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 
 Testing: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo
-QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on' or 'off'
+QEMU_PROG: -drive file=TEST_DIR/t.qcow2,format=qcow2,lazy-refcounts=foo: Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 
 
 === With version 2 images enabling lazy refcounts must fail ===
tests/qemu-iotests/137.out +2 -2
index c0e7534..61c1f90 100644
@@ -15,7 +15,7 @@ read 33554432/33554432 bytes at offset 0
 
 === Try setting some invalid values ===
 
-Parameter 'lazy-refcounts' expects 'on' or 'off'
+Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 cache-size, l2-cache-size and refcount-cache-size may not be set the same time
 l2-cache-size may not exceed cache-size
 refcount-cache-size may not exceed cache-size
@@ -40,7 +40,7 @@ incompatible_features     0x0
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
 wrote 65536/65536 bytes at offset 0
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Parameter 'lazy-refcounts' expects 'on' or 'off'
+Parameter 'lazy-refcounts' expects 'on', 'yes', 'y', 'off', 'no' or 'n'
 qcow2: Marking image as corrupt: Preventing invalid write on metadata (overlaps with qcow2_header); further corruption events will be suppressed
 write failed: Input/output error
 *** done
util/qemu-option.c +16 -11
index 3467dc2..41b356c 100644
@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,
     return get_next_param_value(buf, buf_size, tag, &str);
 }
 
-static void parse_option_bool(const char *name, const char *value, bool *ret,
-                              Error **errp)
+void parse_option_bool(const char *name, const char *value, bool *ret,
+                       Error **errp)
 {
     if (value != NULL) {
-        if (!strcmp(value, "on")) {
-            *ret = 1;
-        } else if (!strcmp(value, "off")) {
-            *ret = 0;
+        if (strcmp(value, "on") == 0 ||
+            strcmp(value, "yes") == 0 ||
+            strcmp(value, "y") == 0) {
+            *ret = true;
+        } else if (strcmp(value, "off") == 0 ||
+            strcmp(value, "no") == 0 ||
+            strcmp(value, "n") == 0) {
+            *ret = false;
         } else {
-            error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
-                       name, "'on' or 'off'");
+            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+                       "'on', 'yes', 'y', 'off', 'no' or 'n'");
+            return;
         }
     } else {
-        *ret = 1;
+        *ret = true;
     }
 }
 
-static void parse_option_number(const char *name, const char *value,
-                                uint64_t *ret, Error **errp)
+void parse_option_number(const char *name, const char *value,
+                         uint64_t *ret, Error **errp)
 {
     char *postfix;
     uint64_t number;
Signature · 1 line
2.7.4
#4 lore raw

PATCH v14 3/19 option: allow qemu_opts_to_qdict to merge repeated options

In reply to Daniel P. Berrange

If given an option string such as

size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind

the qemu_opts_to_qdict() method will currently overwrite the values for repeated option keys, so only the last value is in the returned dict:

    size=1024
    nodes=1-2
    policy=bind

This adds the ability for the caller to ask that the repeated keys be turned into list indexes:

    size=1024
    nodes.0=10
    nodes.1=4-5
    nodes.2=1-2
    policy=bind

Note that the conversion has no way of knowing whether any given key is expected to be a list upfront - it can only figure that out when seeing the first duplicated key. Thus the caller has to be prepared to deal with the fact that if a key 'foo' is a list, then the returned qdict may contain either 'foo' (if only a single instance of the key was seen) or 'foo.NN' (if multiple instances of the key were seen).

---
 blockdev.c               |  7 ++++---
 include/qemu/option.h    |  8 +++++++-
 monitor.c                |  3 ++-
 qemu-img.c               |  4 +++-
 qemu-io-cmds.c           |  3 ++-
 qemu-io.c                |  6 ++++--
 qemu-nbd.c               |  3 ++-
 qom/object_interfaces.c  |  3 ++-
 tests/test-qemu-opts.c   | 40 ++++++++++++++++++++++++++++++++++++++++
 tests/test-replication.c |  9 ++++++---
 util/qemu-option.c       | 41 ++++++++++++++++++++++++++++++++++++++---
 11 files changed, 110 insertions(+), 17 deletions(-)
blockdev.c +4 -3
index 3010393..5ef3193 100644
@@ -911,7 +911,8 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
 
     /* Get a QDict for processing the options */
     bs_opts = qdict_new();
-    qemu_opts_to_qdict(all_opts, bs_opts);
+    qemu_opts_to_qdict(all_opts, bs_opts,
+                       QEMU_OPTS_REPEAT_POLICY_LAST);
 
     legacy_opts = qemu_opts_create(&qemu_legacy_drive_opts, NULL, 0,
                                    &error_abort);
@@ -3758,8 +3759,8 @@ void hmp_drive_add_node(Monitor *mon, const char *optstr)
         return;
     }
 
-    qdict = qemu_opts_to_qdict(opts, NULL);
-
+    qdict = qemu_opts_to_qdict(opts, NULL,
+                               QEMU_OPTS_REPEAT_POLICY_LAST);
     if (!qdict_get_try_str(qdict, "node-name")) {
         QDECREF(qdict);
         error_report("'node-name' needs to be specified");
include/qemu/option.h +7 -1
index 2a5266f..328c468 100644
@@ -125,7 +125,13 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
                             int permit_abbrev);
 QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
                                Error **errp);
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
+typedef enum {
+    QEMU_OPTS_REPEAT_POLICY_LAST,
+    QEMU_OPTS_REPEAT_POLICY_LIST,
+} QemuOptsRepeatPolicy;
+
+QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict,
+                          QemuOptsRepeatPolicy repeatPolicy);
 void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
 
 typedef int (*qemu_opts_loopfunc)(void *opaque, QemuOpts *opts, Error **errp);
monitor.c +2 -1
index 83c4edf..7dcd66b 100644
@@ -2642,7 +2642,8 @@ static QDict *monitor_parse_arguments(Monitor *mon,
                 if (!opts) {
                     goto fail;
                 }
-                qemu_opts_to_qdict(opts, qdict);
+                qemu_opts_to_qdict(opts, qdict,
+                                   QEMU_OPTS_REPEAT_POLICY_LAST);
                 qemu_opts_del(opts);
             }
             break;
qemu-img.c +3 -1
index ceffefe..b399ae5 100644
@@ -273,7 +273,9 @@ static BlockBackend *img_open_opts(const char *optstr,
     QDict *options;
     Error *local_err = NULL;
     BlockBackend *blk;
-    options = qemu_opts_to_qdict(opts, NULL);
+    options = qemu_opts_to_qdict(opts, NULL,
+                                 QEMU_OPTS_REPEAT_POLICY_LAST);
+
     blk = blk_new_open(NULL, NULL, options, flags, &local_err);
     if (!blk) {
         error_reportf_err(local_err, "Could not open '%s': ", optstr);
qemu-io-cmds.c +2 -1
index 3a3838a..e14beed 100644
@@ -1952,7 +1952,8 @@ static int reopen_f(BlockBackend *blk, int argc, char **argv)
     }
 
     qopts = qemu_opts_find(&reopen_opts, NULL);
-    opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
+    opts = qopts ? qemu_opts_to_qdict(qopts, NULL,
+                                      QEMU_OPTS_REPEAT_POLICY_LAST) : NULL;
     qemu_opts_reset(&reopen_opts);
 
     brq = bdrv_reopen_queue(NULL, bs, opts, flags);
qemu-io.c +4 -2
index db129ea..b295afa 100644
@@ -207,7 +207,8 @@ static int open_f(BlockBackend *blk, int argc, char **argv)
     }
 
     qopts = qemu_opts_find(&empty_opts, NULL);
-    opts = qopts ? qemu_opts_to_qdict(qopts, NULL) : NULL;
+    opts = qopts ? qemu_opts_to_qdict(qopts, NULL,
+                                      QEMU_OPTS_REPEAT_POLICY_LAST) : NULL;
     qemu_opts_reset(&empty_opts);
 
     if (optind == argc - 1) {
@@ -593,7 +594,8 @@ int main(int argc, char **argv)
             if (!qopts) {
                 exit(1);
             }
-            opts = qemu_opts_to_qdict(qopts, NULL);
+            opts = qemu_opts_to_qdict(qopts, NULL,
+                                      QEMU_OPTS_REPEAT_POLICY_LAST);
             openfile(NULL, flags, writethrough, opts);
         } else {
             if (format) {
qemu-nbd.c +2 -1
index 99297a5..54eb3ce 100644
@@ -859,7 +859,8 @@ int main(int argc, char **argv)
             qemu_opts_reset(&file_opts);
             exit(EXIT_FAILURE);
         }
-        options = qemu_opts_to_qdict(opts, NULL);
+        options = qemu_opts_to_qdict(opts, NULL,
+                                      QEMU_OPTS_REPEAT_POLICY_LAST);
         qemu_opts_reset(&file_opts);
         blk = blk_new_open(NULL, NULL, options, flags, &local_err);
     } else {
qom/object_interfaces.c +2 -1
index bf59846..48edf2f 100644
@@ -161,7 +161,8 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
     Object *obj = NULL;
 
     v = opts_visitor_new(opts);
-    pdict = qemu_opts_to_qdict(opts, NULL);
+    pdict = qemu_opts_to_qdict(opts, NULL,
+                               QEMU_OPTS_REPEAT_POLICY_LAST);
 
     obj = user_creatable_add(pdict, v, errp);
     visit_free(v);
tests/test-qemu-opts.c +40 -0
index a505a3e..b68ef74 100644
@@ -421,6 +421,45 @@ static void test_qemu_opts_set(void)
     g_assert(opts == NULL);
 }
 
+
+static void test_qemu_opts_to_qdict(void)
+{
+    QemuOpts *opts;
+    QDict *dict;
+
+    /* dynamically initialized (parsed) opts */
+    opts = qemu_opts_parse(&opts_list_03,
+                           "size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind",
+                           false, NULL);
+    g_assert(opts);
+
+    dict = qemu_opts_to_qdict(opts, NULL,
+                              QEMU_OPTS_REPEAT_POLICY_LAST);
+    g_assert(dict);
+
+    g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
+    g_assert_cmpstr(qdict_get_str(dict, "nodes"), ==, "1-2");
+    g_assert(!qdict_haskey(dict, "nodes.0"));
+    g_assert(!qdict_haskey(dict, "nodes.1"));
+    g_assert(!qdict_haskey(dict, "nodes.2"));
+    g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
+    QDECREF(dict);
+
+    dict = qemu_opts_to_qdict(opts, NULL,
+                              QEMU_OPTS_REPEAT_POLICY_LIST);
+    g_assert(dict);
+
+    g_assert_cmpstr(qdict_get_str(dict, "size"), ==, "1024");
+    g_assert(!qdict_haskey(dict, "nodes"));
+    g_assert_cmpstr(qdict_get_str(dict, "nodes.0"), ==, "10");
+    g_assert_cmpstr(qdict_get_str(dict, "nodes.1"), ==, "4-5");
+    g_assert_cmpstr(qdict_get_str(dict, "nodes.2"), ==, "1-2");
+    g_assert_cmpstr(qdict_get_str(dict, "policy"), ==, "bind");
+    QDECREF(dict);
+
+    qemu_opts_del(opts);
+}
+
 int main(int argc, char *argv[])
 {
     register_opts();
@@ -435,6 +474,7 @@ int main(int argc, char *argv[])
     g_test_add_func("/qemu-opts/opt_unset", test_qemu_opt_unset);
     g_test_add_func("/qemu-opts/opts_reset", test_qemu_opts_reset);
     g_test_add_func("/qemu-opts/opts_set", test_qemu_opts_set);
+    g_test_add_func("/qemu-opts/to_qdict", test_qemu_opts_to_qdict);
     g_test_run();
     return 0;
 }
tests/test-replication.c +6 -3
index 0997bd8..6165ac9 100644
@@ -181,7 +181,8 @@ static BlockBackend *start_primary(void)
     opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
     g_free(cmdline);
 
-    qdict = qemu_opts_to_qdict(opts, NULL);
+    qdict = qemu_opts_to_qdict(opts, NULL,
+                               QEMU_OPTS_REPEAT_POLICY_LAST);
     qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
     qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
 
@@ -311,7 +312,8 @@ static BlockBackend *start_secondary(void)
     opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
     g_free(cmdline);
 
-    qdict = qemu_opts_to_qdict(opts, NULL);
+    qdict = qemu_opts_to_qdict(opts, NULL,
+                               QEMU_OPTS_REPEAT_POLICY_LAST);
     qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
     qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
 
@@ -336,7 +338,8 @@ static BlockBackend *start_secondary(void)
     opts = qemu_opts_parse_noisily(&qemu_drive_opts, cmdline, false);
     g_free(cmdline);
 
-    qdict = qemu_opts_to_qdict(opts, NULL);
+    qdict = qemu_opts_to_qdict(opts, NULL,
+                               QEMU_OPTS_REPEAT_POLICY_LAST);
     qdict_set_default_str(qdict, BDRV_OPT_CACHE_DIRECT, "off");
     qdict_set_default_str(qdict, BDRV_OPT_CACHE_NO_FLUSH, "off");
 
util/qemu-option.c +38 -3

57 lines not on this page · raw message

Signature · 1 line
2.7.4
#5 lore raw

Re: [Qemu-devel] [PATCH v14 03/19] option: allow qemu_opts_to_qdict to merge repeated options

Quoted 30 lines · Daniel P. Berrange

If given an option string such as

size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind

the qemu_opts_to_qdict() method will currently overwrite the values for repeated option keys, so only the last value is in the returned dict:

    size=1024
    nodes=1-2
    policy=bind

This adds the ability for the caller to ask that the repeated keys be turned into list indexes:

    size=1024
    nodes.0=10
    nodes.1=4-5
    nodes.2=1-2
    policy=bind

Note that the conversion has no way of knowing whether any given key is expected to be a list upfront - it can only figure that out when seeing the first duplicated key. Thus the caller has to be prepared to deal with the fact that if a key 'foo' is a list, then the returned qdict may contain either 'foo' (if only a single instance of the key was seen) or 'foo.NN' (if multiple instances of the key were seen).

Quoted 2 lines

---

If I'm not mistaken, this policy adds a new policy, but all existing clients use the old policy, and the new policy is exercised only by the testsuite additions. Might be useful to mention that in the commit message, rather than making me read the whole commit before guessing that.

blockdev.c DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type) · line 915

Quoted 8 lines

+++ b/blockdev.c

blockdev.c +2 -1
@@ -911,7 +911,8 @@ DriveInfo *drive_new(QemuOpts *all_opts, BlockInterfaceType block_default_type)
 
     /* Get a QDict for processing the options */
     bs_opts = qdict_new();
-    qemu_opts_to_qdict(all_opts, bs_opts);
+    qemu_opts_to_qdict(all_opts, bs_opts,
+                       QEMU_OPTS_REPEAT_POLICY_LAST);

git send-email/format-patch -O/path/to/file (or the corresponding config option) allows you to sort the diff to put the interesting stuff first (in this case, the new enum).

include/qemu/option.h void qemu_opts_set_defaults(QemuOptsList *list, const char *params, · line 130

Quoted 9 lines
include/qemu/option.h +4 -1
+++ b/include/qemu/option.h
@@ -125,7 +125,13 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
                             int permit_abbrev);
 QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
                                Error **errp);
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
+typedef enum {
+    QEMU_OPTS_REPEAT_POLICY_LAST,
+    QEMU_OPTS_REPEAT_POLICY_LIST,

Hmm. I suspect this subtle difference (one vowel) to be the source of typo bugs. Can we come up with more obvious policy names, such as LAST_ONLY vs. INTO_LIST? Except that doing that makes it harder to fit 80 columns. So up to you if you want to ignore me here.

On the other hand, a documentation comment here would go a long ways to helping future readers:

LAST: last occurrence of a duplicate option silently overwrites all earlier
LIST: each occurrence of a duplicate option converts it into a list

maybe you also want to add:

ERROR: an occurrence of a duplicate option is considered an error

Also, while you turn 'foo=a,foo=b' into 'foo.0=a,foo.1=b', does your code correctly handle the cases of 'foo.0=a,foo=b' and 'foo=a,foo.1=b'? (And what IS the correct handling of those cases logically supposed to be?)

tests/test-qemu-opts.c static void test_qemu_opts_set(void) · line 426

Quoted 8 lines
tests/test-qemu-opts.c +4 -0
+++ b/tests/test-qemu-opts.c
@@ -421,6 +421,45 @@ static void test_qemu_opts_set(void)
     g_assert(opts == NULL);
 }
 
+
+static void test_qemu_opts_to_qdict(void)
+{

Here would be a good place to test the two mixed-use optstrings I mentioned above (inconsistent use of plain vs. list syntax).

Quoted 42 lines
tests/test-qemu-opts.c +22 -3
+}
+
 int main(int argc, char *argv[])
 {
 
+++ b/util/qemu-option.c
@@ -1058,10 +1058,12 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
  * TODO We'll want to use types appropriate for opt->desc->type, but
  * this is enough for now.
  */
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
+QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict,
+                          QemuOptsRepeatPolicy repeatPolicy)
 {
     QemuOpt *opt;
-    QObject *val;
+    QObject *val, *prevval;
+    QDict *lists = qdict_new();
 
     if (!qdict) {
         qdict = qdict_new();
@@ -1070,9 +1072,42 @@ QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict)
         qdict_put(qdict, "id", qstring_from_str(opts->id));
     }
     QTAILQ_FOREACH(opt, &opts->head, next) {
+        gchar *key = NULL;
         val = QOBJECT(qstring_from_str(opt->str));
-        qdict_put_obj(qdict, opt->name, val);
+        switch (repeatPolicy) {
+        case QEMU_OPTS_REPEAT_POLICY_LIST:
+            if (qdict_haskey(lists, opt->name)) {
+                /* Current val goes into 'foo.N' */
+                int64_t max = qdict_get_int(lists, opt->name);
+                max++;
+                key = g_strdup_printf("%s.%" PRId64, opt->name, max);
+                qdict_put_obj(lists, opt->name, QOBJECT(qint_from_int(max)));
+                qdict_put_obj(qdict, key, val);
+            } else if (qdict_haskey(qdict, opt->name)) {
+                /* Move previous val from 'foo' to 'foo.0' */
+                prevval = qdict_get(qdict, opt->name);
+                qobject_incref(prevval);
+                qdict_del(qdict, opt->name);

tests/test-qemu-opts.c QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict) · line 1100

Quoted 10 lines
tests/test-qemu-opts.c +10 -0
+                key = g_strdup_printf("%s.0", opt->name);
+                qdict_put_obj(qdict, key, prevval);
+                g_free(key);
+
+                /* Current val goes into 'foo.1' */
+                key = g_strdup_printf("%s.1", opt->name);
+                qdict_put_obj(lists, opt->name, QOBJECT(qint_from_int(1)));
+                qdict_put_obj(qdict, key, val);
+            } else {
+                qdict_put_obj(qdict, key ? key : opt->name, val);

Dead ?: here; key is always NULL.

util/qemu-option.c QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict) · line 1105

Quoted 5 lines
util/qemu-option.c +5 -0
+            }
+            break;
+
+        case QEMU_OPTS_REPEAT_POLICY_LAST:
+            qdict_put_obj(qdict, key ? key : opt->name, val);

Likewise.

util/qemu-option.c QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict) · line 1113

Quoted 9 lines
util/qemu-option.c +4 -0
+            break;
+        }
+        g_free(key);
     }
+    QDECREF(lists);
     return qdict;
 }
 

Overall, I like the idea.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#6 lore raw

Re: [Qemu-devel] [PATCH v14 03/19] option: allow qemu_opts_to_qdict to merge repeated options

Quoted 31 lines · Eric Blake
Quoted 30 lines · Daniel P. Berrange

If given an option string such as

size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind

the qemu_opts_to_qdict() method will currently overwrite the values for repeated option keys, so only the last value is in the returned dict:

    size=1024
    nodes=1-2
    policy=bind

This adds the ability for the caller to ask that the repeated keys be turned into list indexes:

    size=1024
    nodes.0=10
    nodes.1=4-5
    nodes.2=1-2
    policy=bind

Note that the conversion has no way of knowing whether any given key is expected to be a list upfront - it can only figure that out when seeing the first duplicated key. Thus the caller has to be prepared to deal with the fact that if a key 'foo' is a list, then the returned qdict may contain either 'foo' (if only a single instance of the key was seen) or 'foo.NN' (if multiple instances of the key were seen).

Quoted 7 lines
Quoted 2 lines

---

If I'm not mistaken, this policy adds a new policy, but all existing clients use the old policy, and the new policy is exercised only by the testsuite additions. Might be useful to mention that in the commit message, rather than making me read the whole commit before guessing that.

Correct, this will only be used in combination with the later qdict_crumple method.

Quoted 6 lines
Quoted 6 lines

+++ b/include/qemu/option.h

include/qemu/option.h +0 -1
@@ -125,7 +125,13 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
                             int permit_abbrev);
 QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
                                Error **errp);
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
Quoted 8 lines

include/qemu/option.h void qemu_opts_set_defaults(QemuOptsList *list, const char *params, · line 130

Quoted 3 lines
include/qemu/option.h +3 -0
+typedef enum {
+    QEMU_OPTS_REPEAT_POLICY_LAST,
+    QEMU_OPTS_REPEAT_POLICY_LIST,

Hmm. I suspect this subtle difference (one vowel) to be the source of typo bugs. Can we come up with more obvious policy names, such as LAST_ONLY vs. INTO_LIST? Except that doing that makes it harder to fit 80 columns. So up to you if you want to ignore me here.

I'll use POLICY_LAST and POLICY_ALL.

Quoted 1 line

ERROR: an occurrence of a duplicate option is considered an error

Yeah, sure can add that easily.

Quoted 3 lines

Also, while you turn 'foo=a,foo=b' into 'foo.0=a,foo.1=b', does your code correctly handle the cases of 'foo.0=a,foo=b' and 'foo=a,foo.1=b'? (And what IS the correct handling of those cases logically supposed to be?)

I'd be inclined to say that is an error. We're only doing this hack to maintain compatibility with existing OptsVisitor semantics for repeated options, and the scenario you illustrate is not possible with OptsVisitor. So I say we keep it an error. Either use the old syntax or the new syntax, but not mix them ever.

Regards,
Daniel

Signature · 3 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
#7 lore raw

Re: [Qemu-devel] [PATCH v14 03/19] option: allow qemu_opts_to_qdict to merge repeated options

Quoted 68 lines · Daniel P. Berrange
Quoted 31 lines · Eric Blake
Quoted 30 lines · Daniel P. Berrange

If given an option string such as

size=1024,nodes=10,nodes=4-5,nodes=1-2,policy=bind

the qemu_opts_to_qdict() method will currently overwrite the values for repeated option keys, so only the last value is in the returned dict:

    size=1024
    nodes=1-2
    policy=bind

This adds the ability for the caller to ask that the repeated keys be turned into list indexes:

    size=1024
    nodes.0=10
    nodes.1=4-5
    nodes.2=1-2
    policy=bind

Note that the conversion has no way of knowing whether any given key is expected to be a list upfront - it can only figure that out when seeing the first duplicated key. Thus the caller has to be prepared to deal with the fact that if a key 'foo' is a list, then the returned qdict may contain either 'foo' (if only a single instance of the key was seen) or 'foo.NN' (if multiple instances of the key were seen).

Quoted 7 lines
Quoted 2 lines

---

If I'm not mistaken, this policy adds a new policy, but all existing clients use the old policy, and the new policy is exercised only by the testsuite additions. Might be useful to mention that in the commit message, rather than making me read the whole commit before guessing that.

Correct, this will only be used in combination with the later qdict_crumple method.

Quoted 6 lines
Quoted 6 lines

+++ b/include/qemu/option.h

include/qemu/option.h +0 -1
@@ -125,7 +125,13 @@ void qemu_opts_set_defaults(QemuOptsList *list, const char *params,
                             int permit_abbrev);
 QemuOpts *qemu_opts_from_qdict(QemuOptsList *list, const QDict *qdict,
                                Error **errp);
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict);
Quoted 8 lines

include/qemu/option.h void qemu_opts_set_defaults(QemuOptsList *list, const char *params, · line 130

Quoted 3 lines
include/qemu/option.h +3 -0
+typedef enum {
+    QEMU_OPTS_REPEAT_POLICY_LAST,
+    QEMU_OPTS_REPEAT_POLICY_LIST,

Hmm. I suspect this subtle difference (one vowel) to be the source of typo bugs. Can we come up with more obvious policy names, such as LAST_ONLY vs. INTO_LIST? Except that doing that makes it harder to fit 80 columns. So up to you if you want to ignore me here.

I'll use POLICY_LAST and POLICY_ALL.

Quoted 1 line

ERROR: an occurrence of a duplicate option is considered an error

Yeah, sure can add that easily.

Quoted 3 lines

Also, while you turn 'foo=a,foo=b' into 'foo.0=a,foo.1=b', does your code correctly handle the cases of 'foo.0=a,foo=b' and 'foo=a,foo.1=b'? (And what IS the correct handling of those cases logically supposed to be?)

Quoted 6 lines

I'd be inclined to say that is an error. We're only doing this hack to maintain compatibility with existing OptsVisitor semantics for repeated options, and the scenario you illustrate is not possible with OptsVisitor. So I say we keep it an error. Either use the old syntax or the new syntax, but not mix them ever.

Having looked at this again, I don't think this code should try to detect 'foo.0=a,foo=b', because qemu_opts_to_qdict() is not placing an semantic interpretation on the keys. Such semantics are defined by the qdict_crumple() method, so that's where I'll have to do such error detection.

Regards,
Daniel

Signature · 3 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
#8 lore raw

PATCH v14 4/19 qapi: add trace events for visitor

In reply to Daniel P. Berrange

Allow tracing of the operation of visitors

---
 Makefile.objs          |  1 +
 qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
 qapi/trace-events      | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)
 create mode 100644 qapi/trace-events
Makefile.objs +1 -0
index 7301544..54da068 100644
@@ -160,3 +160,4 @@ trace-events-y += target-s390x/trace-events
 trace-events-y += target-ppc/trace-events
 trace-events-y += qom/trace-events
 trace-events-y += linux-user/trace-events
+trace-events-y += qapi/trace-events
qapi/qapi-visit-core.c +27 -0
index 55f5876..bfcb276 100644
@@ -19,10 +19,12 @@
 #include "qapi/qmp/qerror.h"
 #include "qapi/visitor.h"
 #include "qapi/visitor-impl.h"
+#include "trace.h"
 
 void visit_complete(Visitor *v, void *opaque)
 {
     assert(v->type != VISITOR_OUTPUT || v->complete);
+    trace_visit_complete(v, opaque);
     if (v->complete) {
         v->complete(v, opaque);
     }
@@ -30,6 +32,7 @@ void visit_complete(Visitor *v, void *opaque)
 
 void visit_free(Visitor *v)
 {
+    trace_visit_free(v);
     if (v) {
         v->free(v);
     }
@@ -40,6 +43,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
 {
     Error *err = NULL;
 
+    trace_visit_start_struct(v, name, obj, size);
     if (obj) {
         assert(size);
         assert(!(v->type & VISITOR_OUTPUT) || *obj);
@@ -53,6 +57,7 @@ void visit_start_struct(Visitor *v, const char *name, void **obj,
 
 void visit_check_struct(Visitor *v, Error **errp)
 {
+    trace_visit_check_struct(v);
     if (v->check_struct) {
         v->check_struct(v, errp);
     }
@@ -60,6 +65,7 @@ void visit_check_struct(Visitor *v, Error **errp)
 
 void visit_end_struct(Visitor *v, void **obj)
 {
+    trace_visit_end_struct(v, obj);
     v->end_struct(v, obj);
 }
 
@@ -69,6 +75,7 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
     Error *err = NULL;
 
     assert(!list || size >= sizeof(GenericList));
+    trace_visit_start_list(v, name, list, size);
     v->start_list(v, name, list, size, &err);
     if (list && (v->type & VISITOR_INPUT)) {
         assert(!(err && *list));
@@ -79,11 +86,13 @@ void visit_start_list(Visitor *v, const char *name, GenericList **list,
 GenericList *visit_next_list(Visitor *v, GenericList *tail, size_t size)
 {
     assert(tail && size >= sizeof(GenericList));
+    trace_visit_next_list(v, tail, size);
     return v->next_list(v, tail, size);
 }
 
 void visit_end_list(Visitor *v, void **obj)
 {
+    trace_visit_end_list(v, obj);
     v->end_list(v, obj);
 }
 
@@ -95,6 +104,7 @@ void visit_start_alternate(Visitor *v, const char *name,
 
     assert(obj && size >= sizeof(GenericAlternate));
     assert(!(v->type & VISITOR_OUTPUT) || *obj);
+    trace_visit_start_alternate(v, name, obj, size, promote_int);
     if (v->start_alternate) {
         v->start_alternate(v, name, obj, size, promote_int, &err);
     }
@@ -106,6 +116,7 @@ void visit_start_alternate(Visitor *v, const char *name,
 
 void visit_end_alternate(Visitor *v, void **obj)
 {
+    trace_visit_end_alternate(v, obj);
     if (v->end_alternate) {
         v->end_alternate(v, obj);
     }
@@ -113,6 +124,7 @@ void visit_end_alternate(Visitor *v, void **obj)
 
 bool visit_optional(Visitor *v, const char *name, bool *present)
 {
+    trace_visit_optional(v, name, present);
     if (v->optional) {
         v->optional(v, name, present);
     }
@@ -127,6 +139,7 @@ bool visit_is_input(Visitor *v)
 void visit_type_int(Visitor *v, const char *name, int64_t *obj, Error **errp)
 {
     assert(obj);
+    trace_visit_type_int(v, name, obj);
     v->type_int64(v, name, obj, errp);
 }
 
@@ -151,6 +164,7 @@ void visit_type_uint8(Visitor *v, const char *name, uint8_t *obj,
                       Error **errp)
 {
     uint64_t value = *obj;
+    trace_visit_type_uint8(v, name, obj);
     visit_type_uintN(v, &value, name, UINT8_MAX, "uint8_t", errp);
     *obj = value;
 }
@@ -159,6 +173,7 @@ void visit_type_uint16(Visitor *v, const char *name, uint16_t *obj,
                        Error **errp)
 {
     uint64_t value = *obj;
+    trace_visit_type_uint16(v, name, obj);
     visit_type_uintN(v, &value, name, UINT16_MAX, "uint16_t", errp);
     *obj = value;
 }
@@ -167,6 +182,7 @@ void visit_type_uint32(Visitor *v, const char *name, uint32_t *obj,
                        Error **errp)
 {
     uint64_t value = *obj;
+    trace_visit_type_uint32(v, name, obj);
     visit_type_uintN(v, &value, name, UINT32_MAX, "uint32_t", errp);
     *obj = value;
 }
@@ -175,6 +191,7 @@ void visit_type_uint64(Visitor *v, const char *name, uint64_t *obj,
                        Error **errp)
 {
     assert(obj);
+    trace_visit_type_uint64(v, name, obj);
     v->type_uint64(v, name, obj, errp);
 }
 
@@ -199,6 +216,7 @@ static void visit_type_intN(Visitor *v, int64_t *obj, const char *name,
 void visit_type_int8(Visitor *v, const char *name, int8_t *obj, Error **errp)
 {
     int64_t value = *obj;
+    trace_visit_type_int8(v, name, obj);
     visit_type_intN(v, &value, name, INT8_MIN, INT8_MAX, "int8_t", errp);
     *obj = value;
 }
@@ -207,6 +225,7 @@ void visit_type_int16(Visitor *v, const char *name, int16_t *obj,
                       Error **errp)
 {
     int64_t value = *obj;
+    trace_visit_type_int16(v, name, obj);
     visit_type_intN(v, &value, name, INT16_MIN, INT16_MAX, "int16_t", errp);
     *obj = value;
 }
@@ -215,6 +234,7 @@ void visit_type_int32(Visitor *v, const char *name, int32_t *obj,
                       Error **errp)
 {
     int64_t value = *obj;
+    trace_visit_type_int32(v, name, obj);
     visit_type_intN(v, &value, name, INT32_MIN, INT32_MAX, "int32_t", errp);
     *obj = value;
 }
@@ -223,6 +243,7 @@ void visit_type_int64(Visitor *v, const char *name, int64_t *obj,
                       Error **errp)
 {
     assert(obj);
+    trace_visit_type_int64(v, name, obj);
     v->type_int64(v, name, obj, errp);
 }
 
@@ -230,6 +251,7 @@ void visit_type_size(Visitor *v, const char *name, uint64_t *obj,
                      Error **errp)
 {
     assert(obj);
+    trace_visit_type_size(v, name, obj);
     if (v->type_size) {
         v->type_size(v, name, obj, errp);
     } else {
@@ -240,6 +262,7 @@ void visit_type_size(Visitor *v, const char *name, uint64_t *obj,
 void visit_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
 {
     assert(obj);
+    trace_visit_type_bool(v, name, obj);
     v->type_bool(v, name, obj, errp);
 }
 
@@ -252,6 +275,7 @@ void visit_type_str(Visitor *v, const char *name, char **obj, Error **errp)
      * can enable:
     assert(!(v->type & VISITOR_OUTPUT) || *obj);
      */
+    trace_visit_type_str(v, name, obj);
     v->type_str(v, name, obj, &err);
     if (v->type & VISITOR_INPUT) {
         assert(!err != !*obj);
@@ -263,6 +287,7 @@ void visit_type_number(Visitor *v, const char *name, double *obj,
                        Error **errp)
 {
     assert(obj);
+    trace_visit_type_number(v, name, obj);
     v->type_number(v, name, obj, errp);
 }
 
@@ -272,6 +297,7 @@ void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
 
     assert(obj);
     assert(v->type != VISITOR_OUTPUT || *obj);
+    trace_visit_type_any(v, name, obj);
     v->type_any(v, name, obj, &err);
     if (v->type == VISITOR_INPUT) {
         assert(!err != !*obj);
@@ -281,6 +307,7 @@ void visit_type_any(Visitor *v, const char *name, QObject **obj, Error **errp)
 
 void visit_type_null(Visitor *v, const char *name, Error **errp)
 {
+    trace_visit_type_null(v, name);
     v->type_null(v, name, errp);
 }
 
qapi/trace-events new file +33 -0
new file mode 100644
index 0000000..2c5d3bc
@@ -0,0 +1,33 @@
+# qapi-visit-core.c
+visit_free(void *v) "v=%p"
+visit_complete(void *v, void *opaque) "v=%p opaque=%p"
+
+visit_start_struct(void *v, const char *name, void *obj, size_t size) "v=%p name=%s obj=%p size=%zu"
+visit_check_struct(void *v) "v=%p"
+visit_end_struct(void *v, void *obj) "v=%p obj=%p"
+
+visit_start_list(void *v, const char *name, void *obj, size_t size) "v=%p name=%s obj=%p size=%zu"
+visit_next_list(void *v, void *tail, size_t size) "v=%p tail=%p size=%zu"
+visit_end_list(void *v, void *obj) "v=%p obj=%p"
+
+visit_start_alternate(void *v, const char *name, void *obj, size_t size, bool promote_int) "v=%p name=%s obj=%p size=%zu promote_int=%d"
+visit_end_alternate(void *v, void *obj) "v=%p obj=%p"
+
+visit_optional(void *v, const char *name, bool *present) "v=%p name=%s present=%p"
+
+visit_type_enum(void *v, const char *name, int *obj) "v=%p name=%s obj=%p"
+visit_type_int(void *v, const char *name, int64_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint8(void *v, const char *name, uint8_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint16(void *v, const char *name, uint16_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint32(void *v, const char *name, uint32_t *obj) "v=%p name=%s obj=%p"
+visit_type_uint64(void *v, const char *name, uint64_t *obj) "v=%p name=%s obj=%p"
+visit_type_int8(void *v, const char *name, int8_t *obj) "v=%p name=%s obj=%p"
+visit_type_int16(void *v, const char *name, int16_t *obj) "v=%p name=%s obj=%p"
+visit_type_int32(void *v, const char *name, int32_t *obj) "v=%p name=%s obj=%p"
+visit_type_int64(void *v, const char *name, int64_t *obj) "v=%p name=%s obj=%p"
+visit_type_size(void *v, const char *name, uint64_t *obj) "v=%p name=%s obj=%p"
+visit_type_bool(void *v, const char *name, bool *obj) "v=%p name=%s obj=%p"
+visit_type_str(void *v, const char *name, char **obj) "v=%p name=%s obj=%p"
+visit_type_number(void *v, const char *name, double *obj) "v=%p name=%s obj=%p"
+visit_type_any(void *v, const char *name, void *obj) "v=%p name=%s obj=%p"
+visit_type_null(void *v, const char *name) "v=%p name=%s"
Signature · 1 line
2.7.4
Reviewed-by on v14 4/19
#9 lore raw

Re: [Qemu-devel] [PATCH v14 04/19] qapi: add trace events for visitor

Quoted 1 line · Daniel P. Berrange

Allow tracing of the operation of visitors

Ooooh, shiny!

Quoted 9 lines
---
 Makefile.objs          |  1 +
 qapi/qapi-visit-core.c | 27 +++++++++++++++++++++++++++
 qapi/trace-events      | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+)
 create mode 100644 qapi/trace-events
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#10 lore raw

PATCH v14 5/19 qapi: rename QmpInputVisitor to QObjectInputVisitor

In reply to Daniel P. Berrange

The QmpInputVisitor has no direct dependency on QMP. It is valid to use it anywhere that one has a QObject. Rename it to better reflect its functionality as a generic QObject to QAPI converter.

---
 docs/qapi-code-gen.txt                             |   2 +-
 ...qmp-input-visitor.h => qobject-input-visitor.h} |  10 +-
 include/qapi/visitor.h                             |   6 +-
 monitor.c                                          |   2 +-
 qapi/Makefile.objs                                 |   2 +-
 ...qmp-input-visitor.c => qobject-input-visitor.c} | 171 +++++++++++----------
 qmp.c                                              |   4 +-
 qom/qom-qobject.c                                  |   4 +-
 scripts/qapi-commands.py                           |   4 +-
 target-s390x/cpu_models.c                          |   4 +-
 tests/.gitignore                                   |   4 +-
 tests/Makefile.include                             |  12 +-
 tests/check-qnull.c                                |   4 +-
 tests/test-qmp-commands.c                          |   4 +-
 ...-input-strict.c => test-qobject-input-strict.c} |   6 +-
 ...nput-visitor.c => test-qobject-input-visitor.c} |   6 +-
 tests/test-string-input-visitor.c                  |   2 +-
 tests/test-visitor-serialization.c                 |   4 +-
 util/qemu-sockets.c                                |   2 +-
 19 files changed, 128 insertions(+), 125 deletions(-)
 rename include/qapi/{qmp-input-visitor.h => qobject-input-visitor.h} (63%)
 rename qapi/{qmp-input-visitor.c => qobject-input-visitor.c} (56%)
 rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
 rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (99%)
docs/qapi-code-gen.txt +1 -1
index 5d4c2cd..d2604b6 100644
@@ -1024,7 +1024,7 @@ Example:
         Visitor *v;
         UserDefOneList *arg1 = NULL;
 
-        v = qmp_input_visitor_new(QOBJECT(args), true);
+        v = qobject_input_visitor_new(QOBJECT(args), true);
         visit_start_struct(v, NULL, NULL, 0, &err);
         if (err) {
             goto out;
include/qapi/qobject-input-visitor.h renamed from include/qapi/qmp-input-visitor.h +5 -5
similarity index 63%
rename from include/qapi/qmp-input-visitor.h
rename to include/qapi/qobject-input-visitor.h
index f3ff5f3..cde328d 100644
@@ -11,20 +11,20 @@
  *
  */
 
-#ifndef QMP_INPUT_VISITOR_H
-#define QMP_INPUT_VISITOR_H
+#ifndef QOBJECT_INPUT_VISITOR_H
+#define QOBJECT_INPUT_VISITOR_H
 
 #include "qapi/visitor.h"
 #include "qapi/qmp/qobject.h"
 
-typedef struct QmpInputVisitor QmpInputVisitor;
+typedef struct QObjectInputVisitor QObjectInputVisitor;
 
 /*
- * Return a new input visitor that converts QMP to QAPI.
+ * Return a new input visitor that converts a QObject to a QAPI object.
  *
  * Set @strict to reject a parse that doesn't consume all keys of a
  * dictionary; otherwise excess input is ignored.
  */
-Visitor *qmp_input_visitor_new(QObject *obj, bool strict);
+Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
 
 #endif
include/qapi/visitor.h +3 -3
index 6c77a91..9bb6cba 100644
@@ -25,14 +25,14 @@
  * for doing work at each node of a QAPI graph; it can also be used
  * for a virtual walk, where there is no actual QAPI C struct.
  *
- * There are four kinds of visitor classes: input visitors (QMP,
+ * There are four kinds of visitor classes: input visitors (QObject,
  * string, and QemuOpts) parse an external representation and build
- * the corresponding QAPI graph, output visitors (QMP and string) take
+ * the corresponding QAPI graph, output visitors (QObject and string) take
  * a completed QAPI graph and generate an external representation, the
  * dealloc visitor can take a QAPI graph (possibly partially
  * constructed) and recursively free its resources, and the clone
  * visitor performs a deep clone of one QAPI object to another.  While
- * the dealloc and QMP input/output visitors are general, the string,
+ * the dealloc and QObject input/output visitors are general, the string,
  * QemuOpts, and clone visitors have some implementation limitations;
  * see the documentation for each visitor for more details on what it
  * supports.  Also, see visitor-impl.h for the callback contracts
monitor.c +1 -1
index 7dcd66b..792b995 100644
@@ -957,7 +957,7 @@ EventInfoList *qmp_query_events(Error **errp)
  * directly into QObject instead of first parsing it with
  * visit_type_SchemaInfoList() into a SchemaInfoList, then marshal it
  * to QObject with generated output marshallers, every time.  Instead,
- * we do it in test-qmp-input-visitor.c, just to make sure
+ * we do it in test-qobject-input-visitor.c, just to make sure
  * qapi-introspect.py's output actually conforms to the schema.
  */
 static void qmp_query_qmp_schema(QDict *qdict, QObject **ret_data,
qapi/Makefile.objs +1 -1
index 7ea4aeb..6ec7bdc 100644
@@ -1,4 +1,4 @@
-util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o
+util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o
 util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
 util-obj-y += string-input-visitor.o string-output-visitor.o
 util-obj-y += opts-visitor.o qapi-clone-visitor.o
qapi/qobject-input-visitor.c renamed from qapi/qmp-input-visitor.c +87 -84

336 lines not on this page · raw message

qmp.c +2 -2
index 71f0c8b..dafd8e9 100644
@@ -31,7 +31,7 @@
 #include "qom/qom-qobject.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qobject.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "hw/boards.h"
 #include "qom/object_interfaces.h"
 #include "hw/mem/pc-dimm.h"
@@ -672,7 +672,7 @@ void qmp_object_add(const char *type, const char *id,
         }
     }
 
-    v = qmp_input_visitor_new(props, true);
+    v = qobject_input_visitor_new(props, true);
     obj = user_creatable_add_type(type, id, pdict, v, errp);
     visit_free(v);
     if (obj) {
qom/qom-qobject.c +2 -2
index c225abc..81959e0 100644
@@ -15,7 +15,7 @@
 #include "qom/object.h"
 #include "qom/qom-qobject.h"
 #include "qapi/visitor.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/qmp-output-visitor.h"
 
 void object_property_set_qobject(Object *obj, QObject *value,
@@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value,
 {
     Visitor *v;
     /* TODO: Should we reject, rather than ignore, excess input? */
-    v = qmp_input_visitor_new(value, false);
+    v = qobject_input_visitor_new(value, false);
     object_property_set(obj, v, name, errp);
     visit_free(v);
 }
scripts/qapi-commands.py +2 -2
index 2f603b0..04158a3 100644
@@ -130,7 +130,7 @@ def gen_marshal(name, arg_type, boxed, ret_type):
         push_indent()
 
     ret += mcgen('''
-    v = qmp_input_visitor_new(QOBJECT(args), true);
+    v = qobject_input_visitor_new(QOBJECT(args), true);
     visit_start_struct(v, NULL, NULL, 0, &err);
     if (err) {
         goto out;
@@ -294,7 +294,7 @@ fdef.write(mcgen('''
 #include "qapi/qmp/dispatch.h"
 #include "qapi/visitor.h"
 #include "qapi/qmp-output-visitor.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/dealloc-visitor.h"
 #include "%(prefix)sqapi-types.h"
 #include "%(prefix)sqapi-visit.h"
target-s390x/cpu_models.c +2 -2
index 3ff6a70..c1e729d 100644
@@ -17,7 +17,7 @@
 #include "qapi/visitor.h"
 #include "qemu/error-report.h"
 #include "qapi/qmp/qerror.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/qmp/qbool.h"
 #ifndef CONFIG_USER_ONLY
 #include "sysemu/arch_init.h"
@@ -345,7 +345,7 @@ static void cpu_model_from_info(S390CPUModel *model, const CpuModelInfo *info,
     }
 
     if (qdict) {
-        visitor = qmp_input_visitor_new(info->props, true);
+        visitor = qobject_input_visitor_new(info->props, true);
         visit_start_struct(visitor, NULL, NULL, 0, errp);
         if (*errp) {
             object_unref(obj);
tests/.gitignore +2 -2
index 24ac6cf..2260d8a 100644
@@ -57,8 +57,8 @@ test-qht-par
 test-qmp-commands
 test-qmp-commands.h
 test-qmp-event
-test-qmp-input-strict
-test-qmp-input-visitor
+test-qobject-input-strict
+test-qobject-input-visitor
 test-qmp-introspect.[ch]
 test-qmp-marshal.c
 test-qmp-output-visitor
tests/Makefile.include +6 -6
index d8101b3..c8d601b 100644
@@ -24,9 +24,9 @@ check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
 gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
 check-unit-y += tests/test-clone-visitor$(EXESUF)
 gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
-check-unit-y += tests/test-qmp-input-visitor$(EXESUF)
-gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c
-check-unit-y += tests/test-qmp-input-strict$(EXESUF)
+check-unit-y += tests/test-qobject-input-visitor$(EXESUF)
+gcov-files-test-qobject-input-visitor-y = qapi/qobject-input-visitor.c
+check-unit-y += tests/test-qobject-input-strict$(EXESUF)
 check-unit-y += tests/test-qmp-commands$(EXESUF)
 gcov-files-test-qmp-commands-y = qapi/qmp-dispatch.c
 check-unit-y += tests/test-string-input-visitor$(EXESUF)
@@ -440,7 +440,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
 	tests/test-coroutine.o tests/test-string-output-visitor.o \
 	tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
 	tests/test-clone-visitor.o \
-	tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \
+	tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
 	tests/test-qmp-commands.o tests/test-visitor-serialization.o \
 	tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
 	tests/test-opts-visitor.o tests/test-qmp-event.o \
@@ -543,8 +543,8 @@ tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(te
 tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
 tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
 tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
-tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y)
-tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y)
+tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y)
+tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
 tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
 tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o $(test-qapi-obj-y)
 tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
tests/check-qnull.c +2 -2
index dc906b1..eeb803a 100644
@@ -10,7 +10,7 @@
 
 #include "qapi/qmp/qobject.h"
 #include "qemu-common.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/qmp-output-visitor.h"
 #include "qapi/error.h"
 
@@ -47,7 +47,7 @@ static void qnull_visit_test(void)
 
     g_assert(qnull_.refcnt == 1);
     obj = qnull();
-    v = qmp_input_visitor_new(obj, true);
+    v = qobject_input_visitor_new(obj, true);
     qobject_decref(obj);
     visit_type_null(v, NULL, &error_abort);
     visit_free(v);
tests/test-qmp-commands.c +2 -2
index 81cbe54..ff94481 100644
@@ -4,7 +4,7 @@
 #include "test-qmp-commands.h"
 #include "qapi/qmp/dispatch.h"
 #include "qemu/module.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "tests/test-qapi-types.h"
 #include "tests/test-qapi-visit.h"
 
@@ -244,7 +244,7 @@ static void test_dealloc_partial(void)
         ud2_dict = qdict_new();
         qdict_put_obj(ud2_dict, "string0", QOBJECT(qstring_from_str(text)));
 
-        v = qmp_input_visitor_new(QOBJECT(ud2_dict), true);
+        v = qobject_input_visitor_new(QOBJECT(ud2_dict), true);
         visit_type_UserDefTwo(v, NULL, &ud2, &err);
         visit_free(v);
         QDECREF(ud2_dict);
tests/test-qobject-input-strict.c renamed from tests/test-qmp-input-strict.c +3 -3
similarity index 98%
rename from tests/test-qmp-input-strict.c
rename to tests/test-qobject-input-strict.c
index 814550a..8250365 100644
@@ -1,5 +1,5 @@
 /*
- * QMP Input Visitor unit-tests (strict mode).
+ * QObject Input Visitor unit-tests (strict mode).
  *
  * Copyright (C) 2011-2012, 2015 Red Hat Inc.
  *
@@ -15,7 +15,7 @@
 
 #include "qemu-common.h"
 #include "qapi/error.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "test-qapi-types.h"
 #include "test-qapi-visit.h"
 #include "qapi/qmp/types.h"
@@ -53,7 +53,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data,
     data->obj = qobject_from_jsonv(json_string, ap);
     g_assert(data->obj);
 
-    data->qiv = qmp_input_visitor_new(data->obj, true);
+    data->qiv = qobject_input_visitor_new(data->obj, true);
     g_assert(data->qiv);
     return data->qiv;
 }
tests/test-qobject-input-visitor.c renamed from tests/test-qmp-input-visitor.c +3 -3
similarity index 99%
rename from tests/test-qmp-input-visitor.c
rename to tests/test-qobject-input-visitor.c
index f583dce..0e65e63 100644
@@ -1,5 +1,5 @@
 /*
- * QMP Input Visitor unit-tests.
+ * QObject Input Visitor unit-tests.
  *
  * Copyright (C) 2011-2016 Red Hat Inc.
  *
@@ -14,7 +14,7 @@
 
 #include "qemu-common.h"
 #include "qapi/error.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "test-qapi-types.h"
 #include "test-qapi-visit.h"
 #include "qapi/qmp/types.h"
@@ -49,7 +49,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data,
     data->obj = qobject_from_jsonv(json_string, ap);
     g_assert(data->obj);
 
-    data->qiv = qmp_input_visitor_new(data->obj, false);
+    data->qiv = qobject_input_visitor_new(data->obj, false);
     g_assert(data->qiv);
     return data->qiv;
 }
tests/test-string-input-visitor.c +1 -1
index a679fbc..7f10e25 100644
@@ -4,7 +4,7 @@
  * Copyright (C) 2012 Red Hat Inc.
  *
  * Authors:
- *  Paolo Bonzini <[email protected]> (based on test-qmp-input-visitor)
+ *  Paolo Bonzini <[email protected]> (based on test-qobject-input-visitor)
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or later.
  * See the COPYING file in the top-level directory.
tests/test-visitor-serialization.c +2 -2
index dba4670..51df428 100644
@@ -20,7 +20,7 @@
 #include "qapi/error.h"
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qjson.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/qmp-output-visitor.h"
 #include "qapi/string-input-visitor.h"
 #include "qapi/string-output-visitor.h"
@@ -1040,7 +1040,7 @@ static void qmp_deserialize(void **native_out, void *datap,
     obj = qobject_from_json(qstring_get_str(output_json));
 
     QDECREF(output_json);
-    d->qiv = qmp_input_visitor_new(obj, true);
+    d->qiv = qobject_input_visitor_new(obj, true);
     qobject_decref(obj_orig);
     qobject_decref(obj);
     visit(d->qiv, native_out, errp);
util/qemu-sockets.c +1 -1
index 6db48b3..9e30a21 100644
@@ -21,7 +21,7 @@
 #include "qapi/error.h"
 #include "qemu/sockets.h"
 #include "qemu/main-loop.h"
-#include "qapi/qmp-input-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/qmp-output-visitor.h"
 #include "qapi-visit.h"
 #include "qemu/cutils.h"
Signature · 1 line
2.7.4
#11 lore raw

PATCH v14 6/19 qapi: rename QmpOutputVisitor to QObjectOutputVisitor

In reply to Daniel P. Berrange

The QmpOutputVisitor has no direct dependency on QMP. It is valid to use it anywhere that one wants a QObject. Rename it to better reflect its functionality as a generic QAPI to QObject converter.

---
 block/qapi.c                                       |   4 +-
 blockdev.c                                         |   4 +-
 docs/qapi-code-gen.txt                             |   2 +-
 ...p-output-visitor.h => qobject-output-visitor.h} |  10 +-
 qapi/Makefile.objs                                 |   2 +-
 qapi/qapi-clone-visitor.c                          |   2 +-
 qapi/qmp-output-visitor.c                          | 256 ---------------------
 qapi/qobject-output-visitor.c                      | 254 ++++++++++++++++++++
 qemu-img.c                                         |   8 +-
 qom/object_interfaces.c                            |   2 +-
 qom/qom-qobject.c                                  |   4 +-
 scripts/qapi-commands.py                           |   4 +-
 scripts/qapi-event.py                              |   4 +-
 tests/.gitignore                                   |   2 +-
 tests/Makefile.include                             |   8 +-
 tests/check-qnull.c                                |   4 +-
 ...put-visitor.c => test-qobject-output-visitor.c} |   6 +-
 tests/test-string-output-visitor.c                 |   2 +-
 tests/test-visitor-serialization.c                 |   4 +-
 util/qemu-sockets.c                                |   2 +-
 20 files changed, 291 insertions(+), 293 deletions(-)
 rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
 delete mode 100644 qapi/qmp-output-visitor.c
 create mode 100644 qapi/qobject-output-visitor.c
 rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)
block/qapi.c +2 -2
index 6f947e3..f35c89f 100644
@@ -29,7 +29,7 @@
 #include "block/write-threshold.h"
 #include "qmp-commands.h"
 #include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/qmp/types.h"
 #include "sysemu/block-backend.h"
 #include "qemu/cutils.h"
@@ -691,7 +691,7 @@ void bdrv_image_info_specific_dump(fprintf_function func_fprintf, void *f,
                                    ImageInfoSpecific *info_spec)
 {
     QObject *obj, *data;
-    Visitor *v = qmp_output_visitor_new(&obj);
+    Visitor *v = qobject_output_visitor_new(&obj);
 
     visit_type_ImageInfoSpecific(v, NULL, &info_spec, &error_abort);
     visit_complete(v, &obj);
blockdev.c +2 -2
index 5ef3193..3f5d528 100644
@@ -43,7 +43,7 @@
 #include "qapi/qmp/types.h"
 #include "qapi-visit.h"
 #include "qapi/qmp/qerror.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/util.h"
 #include "sysemu/sysemu.h"
 #include "block/block_int.h"
@@ -3784,7 +3784,7 @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp)
     BlockDriverState *bs;
     BlockBackend *blk = NULL;
     QObject *obj;
-    Visitor *v = qmp_output_visitor_new(&obj);
+    Visitor *v = qobject_output_visitor_new(&obj);
     QDict *qdict;
     Error *local_err = NULL;
 
docs/qapi-code-gen.txt +1 -1
index d2604b6..2841c51 100644
@@ -1005,7 +1005,7 @@ Example:
         Error *err = NULL;
         Visitor *v;
 
-        v = qmp_output_visitor_new(ret_out);
+        v = qobject_output_visitor_new(ret_out);
         visit_type_UserDefOne(v, "unused", &ret_in, &err);
         if (!err) {
             visit_complete(v, ret_out);
include/qapi/qobject-output-visitor.h renamed from include/qapi/qmp-output-visitor.h +5 -5
similarity index 66%
rename from include/qapi/qmp-output-visitor.h
rename to include/qapi/qobject-output-visitor.h
index 040fdda..358c959 100644
@@ -11,20 +11,20 @@
  *
  */
 
-#ifndef QMP_OUTPUT_VISITOR_H
-#define QMP_OUTPUT_VISITOR_H
+#ifndef QOBJECT_OUTPUT_VISITOR_H
+#define QOBJECT_OUTPUT_VISITOR_H
 
 #include "qapi/visitor.h"
 #include "qapi/qmp/qobject.h"
 
-typedef struct QmpOutputVisitor QmpOutputVisitor;
+typedef struct QObjectOutputVisitor QObjectOutputVisitor;
 
 /*
- * Create a new QMP output visitor.
+ * Create a new QOBJECT output visitor.
  *
  * If everything else succeeds, pass @result to visit_complete() to
  * collect the result of the visit.
  */
-Visitor *qmp_output_visitor_new(QObject **result);
+Visitor *qobject_output_visitor_new(QObject **result);
 
 #endif
qapi/Makefile.objs +1 -1
index 6ec7bdc..33906ff 100644
@@ -1,5 +1,5 @@
 util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o
-util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o
+util-obj-y += qobject-output-visitor.o qmp-registry.o qmp-dispatch.o
 util-obj-y += string-input-visitor.o string-output-visitor.o
 util-obj-y += opts-visitor.o qapi-clone-visitor.o
 util-obj-y += qmp-event.o
qapi/qapi-clone-visitor.c +1 -1
index 0bb8216..34086cb 100644
@@ -110,7 +110,7 @@ static void qapi_clone_type_str(Visitor *v, const char *name, char **obj,
     assert(qcv->depth);
     /*
      * Pointer was already cloned by g_memdup; create fresh copy.
-     * Note that as long as qmp-output-visitor accepts NULL instead of
+     * Note that as long as qobject-output-visitor accepts NULL instead of
      * "", then we must do likewise. However, we want to obey the
      * input visitor semantics of never producing NULL when the empty
      * string is intended.
qapi/qmp-output-visitor.c deleted +0 -256

256 lines not on this page · raw message

qapi/qobject-output-visitor.c new file +254 -0

254 lines not on this page · raw message

qemu-img.c +4 -4
index b399ae5..182a7c1 100644
@@ -25,7 +25,7 @@
 #include "qemu-version.h"
 #include "qapi/error.h"
 #include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/qjson.h"
 #include "qemu/cutils.h"
@@ -502,7 +502,7 @@ static void dump_json_image_check(ImageCheck *check, bool quiet)
 {
     QString *str;
     QObject *obj;
-    Visitor *v = qmp_output_visitor_new(&obj);
+    Visitor *v = qobject_output_visitor_new(&obj);
 
     visit_type_ImageCheck(v, NULL, &check, &error_abort);
     visit_complete(v, &obj);
@@ -2195,7 +2195,7 @@ static void dump_json_image_info_list(ImageInfoList *list)
 {
     QString *str;
     QObject *obj;
-    Visitor *v = qmp_output_visitor_new(&obj);
+    Visitor *v = qobject_output_visitor_new(&obj);
 
     visit_type_ImageInfoList(v, NULL, &list, &error_abort);
     visit_complete(v, &obj);
@@ -2211,7 +2211,7 @@ static void dump_json_image_info(ImageInfo *info)
 {
     QString *str;
     QObject *obj;
-    Visitor *v = qmp_output_visitor_new(&obj);
+    Visitor *v = qobject_output_visitor_new(&obj);
 
     visit_type_ImageInfo(v, NULL, &info, &error_abort);
     visit_complete(v, &obj);
qom/object_interfaces.c +1 -1
index 48edf2f..f7afe49 100644
@@ -3,7 +3,7 @@
 #include "qom/object_interfaces.h"
 #include "qemu/module.h"
 #include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/opts-visitor.h"
 
 void user_creatable_complete(Object *obj, Error **errp)
qom/qom-qobject.c +2 -2
index 81959e0..447e4a0 100644
@@ -16,7 +16,7 @@
 #include "qom/qom-qobject.h"
 #include "qapi/visitor.h"
 #include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 
 void object_property_set_qobject(Object *obj, QObject *value,
                                  const char *name, Error **errp)
@@ -35,7 +35,7 @@ QObject *object_property_get_qobject(Object *obj, const char *name,
     Error *local_err = NULL;
     Visitor *v;
 
-    v = qmp_output_visitor_new(&ret);
+    v = qobject_output_visitor_new(&ret);
     object_property_get(obj, v, name, &local_err);
     if (!local_err) {
         visit_complete(v, &ret);
scripts/qapi-commands.py +2 -2
index 04158a3..09e8467 100644
@@ -68,7 +68,7 @@ static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject **ret_out,
     Error *err = NULL;
     Visitor *v;
 
-    v = qmp_output_visitor_new(ret_out);
+    v = qobject_output_visitor_new(ret_out);
     visit_type_%(c_name)s(v, "unused", &ret_in, &err);
     if (!err) {
         visit_complete(v, ret_out);
@@ -293,7 +293,7 @@ fdef.write(mcgen('''
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/dispatch.h"
 #include "qapi/visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/qobject-input-visitor.h"
 #include "qapi/dealloc-visitor.h"
 #include "%(prefix)sqapi-types.h"
scripts/qapi-event.py +2 -2
index 38d8211..f4eb7f8 100644
@@ -98,7 +98,7 @@ def gen_event_send(name, arg_type, boxed):
 
     if arg_type and not arg_type.is_empty():
         ret += mcgen('''
-    v = qmp_output_visitor_new(&obj);
+    v = qobject_output_visitor_new(&obj);
 ''')
         if not arg_type.is_implicit():
             ret += mcgen('''
@@ -209,7 +209,7 @@ fdef.write(mcgen('''
 #include "qemu-common.h"
 #include "%(prefix)sqapi-event.h"
 #include "%(prefix)sqapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/qmp-event.h"
 
 ''',
tests/.gitignore +1 -1
index 2260d8a..7cb2eb0 100644
@@ -61,7 +61,7 @@ test-qobject-input-strict
 test-qobject-input-visitor
 test-qmp-introspect.[ch]
 test-qmp-marshal.c
-test-qmp-output-visitor
+test-qobject-output-visitor
 test-rcu-list
 test-replication
 test-rfifolock
tests/Makefile.include +4 -4
index c8d601b..6398678 100644
@@ -20,8 +20,8 @@ check-unit-y += tests/check-qnull$(EXESUF)
 gcov-files-check-qnull-y = qobject/qnull.c
 check-unit-y += tests/check-qjson$(EXESUF)
 gcov-files-check-qjson-y = qobject/qjson.c
-check-unit-y += tests/test-qmp-output-visitor$(EXESUF)
-gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c
+check-unit-y += tests/test-qobject-output-visitor$(EXESUF)
+gcov-files-test-qobject-output-visitor-y = qapi/qobject-output-visitor.c
 check-unit-y += tests/test-clone-visitor$(EXESUF)
 gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c
 check-unit-y += tests/test-qobject-input-visitor$(EXESUF)
@@ -438,7 +438,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
 	tests/check-qlist.o tests/check-qfloat.o tests/check-qnull.o \
 	tests/check-qjson.o \
 	tests/test-coroutine.o tests/test-string-output-visitor.o \
-	tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \
+	tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \
 	tests/test-clone-visitor.o \
 	tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
 	tests/test-qmp-commands.o tests/test-visitor-serialization.o \
@@ -541,7 +541,7 @@ $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-int
 tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y)
 tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y)
 tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y)
-tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y)
+tests/test-qobject-output-visitor$(EXESUF): tests/test-qobject-output-visitor.o $(test-qapi-obj-y)
 tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y)
 tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y)
 tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
tests/check-qnull.c +2 -2
index eeb803a..b50bb8a 100644
@@ -11,7 +11,7 @@
 #include "qapi/qmp/qobject.h"
 #include "qemu-common.h"
 #include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/error.h"
 
 /*
@@ -52,7 +52,7 @@ static void qnull_visit_test(void)
     visit_type_null(v, NULL, &error_abort);
     visit_free(v);
 
-    v = qmp_output_visitor_new(&obj);
+    v = qobject_output_visitor_new(&obj);
     visit_type_null(v, NULL, &error_abort);
     visit_complete(v, &obj);
     g_assert(obj == &qnull_);
tests/test-qobject-output-visitor.c renamed from tests/test-qmp-output-visitor.c +3 -3
similarity index 99%
rename from tests/test-qmp-output-visitor.c
rename to tests/test-qobject-output-visitor.c
index 513d71f..4e2d79c 100644
@@ -1,5 +1,5 @@
 /*
- * QMP Output Visitor unit-tests.
+ * QObject Output Visitor unit-tests.
  *
  * Copyright (C) 2011-2016 Red Hat Inc.
  *
@@ -14,7 +14,7 @@
 
 #include "qemu-common.h"
 #include "qapi/error.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "test-qapi-types.h"
 #include "test-qapi-visit.h"
 #include "qapi/qmp/types.h"
@@ -28,7 +28,7 @@ typedef struct TestOutputVisitorData {
 static void visitor_output_setup(TestOutputVisitorData *data,
                                  const void *unused)
 {
-    data->ov = qmp_output_visitor_new(&data->obj);
+    data->ov = qobject_output_visitor_new(&data->obj);
     g_assert(data->ov);
 }
 
tests/test-string-output-visitor.c +1 -1
index 444844a..e736db3 100644
@@ -4,7 +4,7 @@
  * Copyright (C) 2012 Red Hat Inc.
  *
  * Authors:
- *  Paolo Bonzini <[email protected]> (based on test-qmp-output-visitor)
+ *  Paolo Bonzini <[email protected]> (based on test-qobject-output-visitor)
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or later.
  * See the COPYING file in the top-level directory.
tests/test-visitor-serialization.c +2 -2
index 51df428..66b2b1c 100644
@@ -21,7 +21,7 @@
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qjson.h"
 #include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi/string-input-visitor.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi-types.h"
@@ -1022,7 +1022,7 @@ static void qmp_serialize(void *native_in, void **datap,
 {
     QmpSerializeData *d = g_malloc0(sizeof(*d));
 
-    d->qov = qmp_output_visitor_new(&d->obj);
+    d->qov = qobject_output_visitor_new(&d->obj);
     visit(d->qov, &native_in, errp);
     *datap = d;
 }
util/qemu-sockets.c +1 -1
index 9e30a21..4cef549 100644
@@ -22,7 +22,7 @@
 #include "qemu/sockets.h"
 #include "qemu/main-loop.h"
 #include "qapi/qobject-input-visitor.h"
-#include "qapi/qmp-output-visitor.h"
+#include "qapi/qobject-output-visitor.h"
 #include "qapi-visit.h"
 #include "qemu/cutils.h"
 
Signature · 1 line
2.7.4
#12 lore raw

PATCH v14 7/19 qapi: don't pass two copies of TestInputVisitorData to tests

In reply to Daniel P. Berrange

The input_visitor_test_add() method was accepting an instance of 'TestInputVisitorData' and passing it as the 'user_data' parameter to test functions. The main 'TestInputVisitorData' instance that was actually used, was meanwhile being allocated automatically by the test framework fixture setup.

---
 tests/test-qobject-input-visitor.c | 76 ++++++++++++++++----------------------
 1 file changed, 32 insertions(+), 44 deletions(-)
tests/test-qobject-input-visitor.c +32 -44
index 0e65e63..26c5012 100644
@@ -747,10 +747,11 @@ static void test_visitor_in_native_list_number(TestInputVisitorData *data,
 }
 
 static void input_visitor_test_add(const char *testpath,
-                                   TestInputVisitorData *data,
-                                   void (*test_func)(TestInputVisitorData *data, const void *user_data))
+                                   const void *user_data,
+                                   void (*test_func)(TestInputVisitorData *data,
+                                                     const void *user_data))
 {
-    g_test_add(testpath, TestInputVisitorData, data, NULL, test_func,
+    g_test_add(testpath, TestInputVisitorData, user_data, NULL, test_func,
                visitor_input_teardown);
 }
 
@@ -833,77 +834,64 @@ static void test_visitor_in_wrong_type(TestInputVisitorData *data,
 
 int main(int argc, char **argv)
 {
-    TestInputVisitorData in_visitor_data;
-
     g_test_init(&argc, &argv, NULL);
 
     input_visitor_test_add("/visitor/input/int",
-                           &in_visitor_data, test_visitor_in_int);
+                           NULL, test_visitor_in_int);
     input_visitor_test_add("/visitor/input/int_overflow",
-                           &in_visitor_data, test_visitor_in_int_overflow);
+                           NULL, test_visitor_in_int_overflow);
     input_visitor_test_add("/visitor/input/bool",
-                           &in_visitor_data, test_visitor_in_bool);
+                           NULL, test_visitor_in_bool);
     input_visitor_test_add("/visitor/input/number",
-                           &in_visitor_data, test_visitor_in_number);
+                           NULL, test_visitor_in_number);
     input_visitor_test_add("/visitor/input/string",
-                           &in_visitor_data, test_visitor_in_string);
+                           NULL, test_visitor_in_string);
     input_visitor_test_add("/visitor/input/enum",
-                           &in_visitor_data, test_visitor_in_enum);
+                           NULL, test_visitor_in_enum);
     input_visitor_test_add("/visitor/input/struct",
-                           &in_visitor_data, test_visitor_in_struct);
+                           NULL, test_visitor_in_struct);
     input_visitor_test_add("/visitor/input/struct-nested",
-                           &in_visitor_data, test_visitor_in_struct_nested);
+                           NULL, test_visitor_in_struct_nested);
     input_visitor_test_add("/visitor/input/list",
-                           &in_visitor_data, test_visitor_in_list);
+                           NULL, test_visitor_in_list);
     input_visitor_test_add("/visitor/input/any",
-                           &in_visitor_data, test_visitor_in_any);
+                           NULL, test_visitor_in_any);
     input_visitor_test_add("/visitor/input/null",
-                           &in_visitor_data, test_visitor_in_null);
+                           NULL, test_visitor_in_null);
     input_visitor_test_add("/visitor/input/union-flat",
-                           &in_visitor_data, test_visitor_in_union_flat);
+                           NULL, test_visitor_in_union_flat);
     input_visitor_test_add("/visitor/input/alternate",
-                           &in_visitor_data, test_visitor_in_alternate);
+                           NULL, test_visitor_in_alternate);
     input_visitor_test_add("/visitor/input/errors",
-                           &in_visitor_data, test_visitor_in_errors);
+                           NULL, test_visitor_in_errors);
     input_visitor_test_add("/visitor/input/wrong-type",
-                           &in_visitor_data, test_visitor_in_wrong_type);
+                           NULL, test_visitor_in_wrong_type);
     input_visitor_test_add("/visitor/input/alternate-number",
-                           &in_visitor_data, test_visitor_in_alternate_number);
+                           NULL, test_visitor_in_alternate_number);
     input_visitor_test_add("/visitor/input/native_list/int",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_int);
+                           NULL, test_visitor_in_native_list_int);
     input_visitor_test_add("/visitor/input/native_list/int8",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_int8);
+                           NULL, test_visitor_in_native_list_int8);
     input_visitor_test_add("/visitor/input/native_list/int16",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_int16);
+                           NULL, test_visitor_in_native_list_int16);
     input_visitor_test_add("/visitor/input/native_list/int32",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_int32);
+                           NULL, test_visitor_in_native_list_int32);
     input_visitor_test_add("/visitor/input/native_list/int64",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_int64);
+                           NULL, test_visitor_in_native_list_int64);
     input_visitor_test_add("/visitor/input/native_list/uint8",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_uint8);
+                           NULL, test_visitor_in_native_list_uint8);
     input_visitor_test_add("/visitor/input/native_list/uint16",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_uint16);
+                           NULL, test_visitor_in_native_list_uint16);
     input_visitor_test_add("/visitor/input/native_list/uint32",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_uint32);
+                           NULL, test_visitor_in_native_list_uint32);
     input_visitor_test_add("/visitor/input/native_list/uint64",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_uint64);
+                           NULL, test_visitor_in_native_list_uint64);
     input_visitor_test_add("/visitor/input/native_list/bool",
-                           &in_visitor_data, test_visitor_in_native_list_bool);
+                           NULL, test_visitor_in_native_list_bool);
     input_visitor_test_add("/visitor/input/native_list/str",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_string);
+                           NULL, test_visitor_in_native_list_string);
     input_visitor_test_add("/visitor/input/native_list/number",
-                           &in_visitor_data,
-                           test_visitor_in_native_list_number);
+                           NULL, test_visitor_in_native_list_number);
 
     g_test_run();
 
Signature · 1 line
2.7.4
Reviewed-by on v14 7/19
#13 lore raw

Re: [Qemu-devel] [PATCH v14 07/19] qapi: don't pass two copies of TestInputVisitorData to tests

Quoted 11 lines · Daniel P. Berrange

The input_visitor_test_add() method was accepting an instance of 'TestInputVisitorData' and passing it as the 'user_data' parameter to test functions. The main 'TestInputVisitorData' instance that was actually used, was meanwhile being allocated automatically by the test framework fixture setup.

---
 tests/test-qobject-input-visitor.c | 76 ++++++++++++++++----------------------
 1 file changed, 32 insertions(+), 44 deletions(-)
Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#14 lore raw

Re: [Qemu-devel] [PATCH v14 07/19] qapi: don't pass two copies of TestInputVisitorData to tests

Quoted 7 lines · Eric Blake
Quoted 6 lines · Daniel P. Berrange

The input_visitor_test_add() method was accepting an instance of 'TestInputVisitorData' and passing it as the 'user_data' parameter to test functions. The main 'TestInputVisitorData' instance that was actually used, was meanwhile being allocated automatically by the test framework fixture setup.

Quoted 8 lines
Quoted 5 lines
---
 tests/test-qobject-input-visitor.c | 76 ++++++++++++++++----------------------
 1 file changed, 32 insertions(+), 44 deletions(-)

Having said that, I note that ALL callers now pass NULL for user_data. If you plan on using it later in the series for something other than NULL for some of the (new?) tests added at that point, it would be wise to say so in the commit message; if not, I would suggest eliminating the parameter altogether.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#15 lore raw

Re: [Qemu-devel] [PATCH v14 07/19] qapi: don't pass two copies of TestInputVisitorData to tests

Quoted 14 lines · Eric Blake
Quoted 13 lines · Eric Blake
Quoted 11 lines · Daniel P. Berrange

The input_visitor_test_add() method was accepting an instance of 'TestInputVisitorData' and passing it as the 'user_data' parameter to test functions. The main 'TestInputVisitorData' instance that was actually used, was meanwhile being allocated automatically by the test framework fixture setup.

---
 tests/test-qobject-input-visitor.c | 76 ++++++++++++++++----------------------
 1 file changed, 32 insertions(+), 44 deletions(-)
Quoted 8 lines
Quoted 2 lines

Having said that, I note that ALL callers now pass NULL for user_data. If you plan on using it later in the series for something other than NULL for some of the (new?) tests added at that point, it would be wise to say so in the commit message; if not, I would suggest eliminating the parameter altogether.

Yep, later patches use this parameter for real, so I needed to move this bogus usage out of the way.

Regards,
Daniel

Signature · 3 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
#16 lore raw

PATCH v14 8/19 qapi: permit scalar type conversions in QObjectInputVisitor

In reply to Daniel P. Berrange

Currently the QObjectInputVisitor assumes that all scalar values are directly represented as the final types declared by the thing being visited. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool, etc. This is good when QObjectInputVisitor is fed a QObject that came from a JSON document on the QMP monitor, as it will strictly validate correctness.

To allow QObjectInputVisitor to be reused for visiting a QObject originating from QemuOpts, an alternative mode is needed where all the scalars types are represented as QString and converted on the fly to the final desired type.

---
 include/qapi/qobject-input-visitor.h |  32 +++++-
 qapi/qobject-input-visitor.c         | 132 ++++++++++++++++++++++++
 tests/test-qobject-input-visitor.c   | 194 ++++++++++++++++++++++++++++++++++-
 3 files changed, 350 insertions(+), 8 deletions(-)
include/qapi/qobject-input-visitor.h +28 -4
index cde328d..5022297 100644
@@ -19,12 +19,36 @@
 
 typedef struct QObjectInputVisitor QObjectInputVisitor;
 
-/*
- * Return a new input visitor that converts a QObject to a QAPI object.
+/**
+ * Create a new input visitor that converts @obj to a QAPI object.
+ *
+ * Any scalar values in the @obj input data structure should be in the
+ * required type already. i.e. if visiting a bool, the value should
+ * already be a QBool instance.
  *
- * Set @strict to reject a parse that doesn't consume all keys of a
- * dictionary; otherwise excess input is ignored.
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation. If @strict is false
+ * then extra dict keys are silently ignored.
+ *
+ * The returned input visitor should be released by calling
+ * visit_free() when no longer required.
  */
 Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
 
+/**
+ * Create a new input visitor that converts @obj to a QAPI object.
+ *
+ * Any scalar values in the @obj input data structure should always be
+ * represented as strings. i.e. if visiting a boolean, the value should
+ * be a QString whose contents represent a valid boolean.
+ *
+ * The visitor always operates in strict mode, requiring all dict keys
+ * to be consumed during visitation. An error will be reported if this
+ * does not happen.
+ *
+ * The returned input visitor should be released by calling
+ * visit_free() when no longer required.
+ */
+Visitor *qobject_input_visitor_new_autocast(QObject *obj);
+
 #endif
qapi/qobject-input-visitor.c +132 -0

171 lines not on this page · raw message

tests/test-qobject-input-visitor.c +190 -4

256 lines not on this page · raw message

Signature · 1 line
2.7.4
#17 lore raw

PATCH v14 9/19 qapi: permit auto-creating single element lists

In reply to Daniel P. Berrange

When converting QemuOpts to a QObject, there is no information about compound types available, so when visiting a list, the corresponding QObject is not guaranteed to be a QList. We therefore need to be able to auto-create a single element QList from whatever type we find.

This mode should only be enabled if you have compatibility requirements for

-arg foo=hello,foo=world

to be treated as equivalent to the preferred syntax:

-arg foo.0=hello,foo.1=world

---
 include/qapi/qobject-input-visitor.h | 16 ++++++-
 qapi/qobject-input-visitor.c         | 23 ++++++++--
 tests/test-qobject-input-visitor.c   | 88 +++++++++++++++++++++++++++++++-----
 3 files changed, 111 insertions(+), 16 deletions(-)
include/qapi/qobject-input-visitor.h +15 -1
index 5022297..fb52457 100644
@@ -42,6 +42,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  * represented as strings. i.e. if visiting a boolean, the value should
  * be a QString whose contents represent a valid boolean.
  *
+ * If @autocreate_list is true, then as an alternative to a normal QList,
+ * list values can be stored as a QString or QDict instead, which will
+ * be interpreted as representing single element lists. This should only
+ * by used if compatibility is required with the OptsVisitor which allowed
+ * repeated keys, without list indexes, to represent lists. e.g. set this
+ * to true if you have compatibility requirements for
+ *
+ *   -arg foo=hello,foo=world
+ *
+ * to be treated as equivalent to the preferred syntax:
+ *
+ *   -arg foo.0=hello,foo.1=world
+ *
  * The visitor always operates in strict mode, requiring all dict keys
  * to be consumed during visitation. An error will be reported if this
  * does not happen.
@@ -49,6 +62,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  * The returned input visitor should be released by calling
  * visit_free() when no longer required.
  */
-Visitor *qobject_input_visitor_new_autocast(QObject *obj);
+Visitor *qobject_input_visitor_new_autocast(QObject *obj,
+                                            bool autocreate_list);
 
 #endif
qapi/qobject-input-visitor.c +20 -3
index cf41df6..e8afd1e 100644
@@ -48,6 +48,10 @@ struct QObjectInputVisitor
 
     /* True to reject parse in visit_end_struct() if unvisited keys remain. */
     bool strict;
+
+    /* Whether we can auto-create single element lists when
+     * encountering a non-QList type */
+    bool autocreate_list;
 };
 
 static QObjectInputVisitor *to_qiv(Visitor *v)
@@ -108,6 +112,7 @@ static const QListEntry *qobject_input_push(QObjectInputVisitor *qiv,
     assert(obj);
     tos->obj = obj;
     tos->qapi = qapi;
+    qobject_incref(obj);
 
     if (qiv->strict && qobject_type(obj) == QTYPE_QDICT) {
         h = g_hash_table_new(g_str_hash, g_str_equal);
@@ -147,6 +152,7 @@ static void qobject_input_stack_object_free(StackObject *tos)
     if (tos->h) {
         g_hash_table_unref(tos->h);
     }
+    qobject_decref(tos->obj);
 
     g_free(tos);
 }
@@ -197,7 +203,7 @@ static void qobject_input_start_list(Visitor *v, const char *name,
     QObject *qobj = qobject_input_get_object(qiv, name, true);
     const QListEntry *entry;
 
-    if (!qobj || qobject_type(qobj) != QTYPE_QLIST) {
+    if (!qobj || (!qiv->autocreate_list && qobject_type(qobj) != QTYPE_QLIST)) {
         if (list) {
             *list = NULL;
         }
@@ -206,7 +212,16 @@ static void qobject_input_start_list(Visitor *v, const char *name,
         return;
     }
 
-    entry = qobject_input_push(qiv, qobj, list, errp);
+    if (qobject_type(qobj) != QTYPE_QLIST) {
+        QList *tmplist = qlist_new();
+        qlist_append_obj(tmplist, qobj);
+        qobject_incref(qobj);
+        entry = qobject_input_push(qiv, QOBJECT(tmplist), list, errp);
+        QDECREF(tmplist);
+    } else {
+        entry = qobject_input_push(qiv, qobj, list, errp);
+    }
+
     if (list) {
         if (entry) {
             *list = g_malloc0(size);
@@ -514,7 +529,8 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict)
     return &v->visitor;
 }
 
-Visitor *qobject_input_visitor_new_autocast(QObject *obj)
+Visitor *qobject_input_visitor_new_autocast(QObject *obj,
+                                            bool autocreate_list)
 {
     QObjectInputVisitor *v;
 
@@ -539,6 +555,7 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj)
     v->visitor.optional = qobject_input_optional;
     v->visitor.free = qobject_input_free;
     v->strict = true;
+    v->autocreate_list = autocreate_list;
 
     v->root = obj;
     qobject_incref(obj);
tests/test-qobject-input-visitor.c +76 -12

187 lines not on this page · raw message

Signature · 1 line
2.7.4
#18 lore raw

PATCH v14 10/19 qapi: permit auto-creating nested structs

In reply to Daniel P. Berrange

Some of the historical command line opts that had their keys in in a completely flat namespace are now represented by QAPI schemas that use a nested structs. When converting the QemuOpts to QObject, there is no information about compound types available, so the QObject will be completely flat, even after the qdict_crumple() call. So when starting a struct, we may not have a QDict available in the input data, so we auto-create a QDict containing all the currently unvisited input data keys. Not all historical command line opts require this, so the behaviour is opt-in, by specifying how many levels of structs are permitted to be auto-created.

Note that this only works if the child struct is the last field to the visited in the parent struct. This is always the case for currently existing legacy command line options.

The example is the NetLegacy type which has 3 levels of structs. The modern way to represent this in QemuOpts would be the dot-separated component approach

  -net vlan=1,id=foo,name=bar,opts.type=tap,\
       opts.data.fd=3,opts.data.script=ifup

The legacy syntax will just be presenting

-net vlan=1,id=foo,name=bar,type=tap,fd=3,script=ifup

So we need to auto-create 3 levels of struct when visiting.

The implementation here will enable visiting in both the modern and legacy syntax, compared to OptsVisitor which only allows the legacy syntax.

---
 include/qapi/qobject-input-visitor.h |  21 +++++-
 qapi/qobject-input-visitor.c         |  55 +++++++++++++--
 tests/test-qobject-input-visitor.c   | 130 ++++++++++++++++++++++++++++++++---
 3 files changed, 190 insertions(+), 16 deletions(-)
include/qapi/qobject-input-visitor.h +19 -2
index fb52457..90989f4 100644
@@ -45,7 +45,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  * If @autocreate_list is true, then as an alternative to a normal QList,
  * list values can be stored as a QString or QDict instead, which will
  * be interpreted as representing single element lists. This should only
- * by used if compatibility is required with the OptsVisitor which allowed
+ * be used if compatibility is required with the OptsVisitor which allowed
  * repeated keys, without list indexes, to represent lists. e.g. set this
  * to true if you have compatibility requirements for
  *
@@ -55,6 +55,22 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  *
  *   -arg foo.0=hello,foo.1=world
  *
+ * If @autocreate_struct_levels is non-zero, then when visiting structs,
+ * if the corresponding QDict is not found, it will automatically create
+ * a QDict containing all remaining unvisited options. This should only
+ * be used if compatibility is required with the OptsVisitor which flatten
+ * structs so that all keys were at the same level. e.g. set this to a
+ * non-zero number if you compatibility requirements for
+ *
+ *   -arg type=person,surname=blogs,forename=fred
+ *
+ * to be treated as equivalent to the perferred syntax
+ *
+ *   -arg type=person,data.surname=blogs,data.forename=fred
+ *
+ * The value given determines how many levels of structs are allowed to
+ * be flattened in this way.
+ *
  * The visitor always operates in strict mode, requiring all dict keys
  * to be consumed during visitation. An error will be reported if this
  * does not happen.
@@ -63,6 +79,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  * visit_free() when no longer required.
  */
 Visitor *qobject_input_visitor_new_autocast(QObject *obj,
-                                            bool autocreate_list);
+                                            bool autocreate_list,
+                                            size_t autocreate_struct_levels);
 
 #endif
qapi/qobject-input-visitor.c +51 -4

114 lines not on this page · raw message

tests/test-qobject-input-visitor.c +120 -10

174 lines not on this page · raw message

Signature · 1 line
2.7.4
#19 lore raw

PATCH v14 11/19 qapi: add integer range support for QObjectInputVisitor

In reply to Daniel P. Berrange

The traditional CLI arg syntax allows two ways to specify integer lists, either one value per key, or a range of values per key. eg the following are identical:

-arg foo=5,foo=6,foo=7
-arg foo=5-7

This extends the QObjectInputVisitor so that it is able to parse ranges and turn them into distinct list entries.

This means that

-arg foo=5-7

is treated as equivalent to

-arg foo.0=5,foo.1=6,foo.2=7

Edge case tests are copied from test-opts-visitor to ensure identical behaviour when parsing.

---
 include/qapi/qobject-input-visitor.h |  22 +++-
 qapi/qobject-input-visitor.c         | 154 +++++++++++++++++++++++++--
 tests/test-qobject-input-visitor.c   | 195 +++++++++++++++++++++++++++++++++--
 3 files changed, 356 insertions(+), 15 deletions(-)
include/qapi/qobject-input-visitor.h +21 -1
index 90989f4..63f3782 100644
@@ -19,6 +19,12 @@
 
 typedef struct QObjectInputVisitor QObjectInputVisitor;
 
+/* Inclusive upper bound on the size of any flattened range. This is a safety
+ * (= anti-annoyance) measure; wrong ranges should not cause long startup
+ * delays nor exhaust virtual memory.
+ */
+#define QIV_RANGE_MAX 65536
+
 /**
  * Create a new input visitor that converts @obj to a QAPI object.
  *
@@ -71,6 +77,19 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  * The value given determines how many levels of structs are allowed to
  * be flattened in this way.
  *
+ * If @permit_int_ranges is true, then when visiting a list of integers,
+ * the integer value strings may encode ranges eg a single element
+ * containing "5-7" is treated as if there were three elements "5", "6",
+ * "7". This should only be used if compatibility is required with the
+ * OptsVisitor which would allow integer ranges. e.g. set this to true
+ * if you have compatibility requirements for
+ *
+ *   -arg val=5-8
+ *
+ * to be treated as equivalent to the preferred syntax:
+ *
+ *   -arg val.0=5,val.1=6,val.2=7,val.3=8
+ *
  * The visitor always operates in strict mode, requiring all dict keys
  * to be consumed during visitation. An error will be reported if this
  * does not happen.
@@ -80,6 +99,7 @@ Visitor *qobject_input_visitor_new(QObject *obj, bool strict);
  */
 Visitor *qobject_input_visitor_new_autocast(QObject *obj,
                                             bool autocreate_list,
-                                            size_t autocreate_struct_levels);
+                                            size_t autocreate_struct_levels,
+                                            bool permit_int_ranges);
 
 #endif
qapi/qobject-input-visitor.c +146 -8

220 lines not on this page · raw message

tests/test-qobject-input-visitor.c +189 -6

252 lines not on this page · raw message

Signature · 1 line
2.7.4
#20 lore raw

PATCH v14 12/19 qapi: allow QObjectInputVisitor to be created with QemuOpts

In reply to Daniel P. Berrange

Instead of requiring all callers to go through the mutli-step process of turning QemuOpts into a suitable QObject for visiting, add a new constructor that encapsulates this logic. This will allow QObjectInputVisitor to be a drop-in replacement for the existing OptsVisitor with minimal code changes for callers.

---
 include/qapi/qobject-input-visitor.h | 19 +++++++++++++++++++
 include/qemu/option.h                |  2 +-
 qapi/qobject-input-visitor.c         | 29 +++++++++++++++++++++++++++++
 util/qemu-option.c                   |  2 +-
 4 files changed, 50 insertions(+), 2 deletions(-)
include/qapi/qobject-input-visitor.h +19 -0
index 63f3782..242b767 100644
@@ -102,4 +102,23 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
                                             size_t autocreate_struct_levels,
                                             bool permit_int_ranges);
 
+
+/**
+ * Create a new input visitor that converts @opts to a QAPI object.
+ *
+ * The QemuOpts will be converted into a QObject using the
+ * qdict_crumple() method to automatically create structs
+ * and lists. The resulting QDict will then be passed to the
+ * qobject_input_visitor_new_autocast() method. See the docs
+ * of that method for further details on processing behaviour.
+ *
+ * The returned input visitor should be released by calling
+ * visit_free() when no longer required.
+ */
+Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
+                                        bool autocreate_list,
+                                        size_t autocreate_struct_levels,
+                                        bool permit_int_ranges,
+                                        Error **errp);
+
 #endif
include/qemu/option.h +1 -1
index 328c468..bf1f078 100644
@@ -130,7 +130,7 @@ typedef enum {
     QEMU_OPTS_REPEAT_POLICY_LIST,
 } QemuOptsRepeatPolicy;
 
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict,
+QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
                           QemuOptsRepeatPolicy repeatPolicy);
 void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp);
 
qapi/qobject-input-visitor.c +29 -0
index a38e779..0aef20e 100644
@@ -747,3 +747,32 @@ Visitor *qobject_input_visitor_new_autocast(QObject *obj,
 
     return &v->visitor;
 }
+
+
+Visitor *qobject_input_visitor_new_opts(const QemuOpts *opts,
+                                        bool autocreate_list,
+                                        size_t autocreate_struct_levels,
+                                        bool permit_int_ranges,
+                                        Error **errp)
+{
+    QDict *pdict;
+    QObject *pobj;
+    Visitor *v = NULL;
+
+    pdict = qemu_opts_to_qdict(opts, NULL,
+                               QEMU_OPTS_REPEAT_POLICY_LIST);
+
+    pobj = qdict_crumple(pdict, true, errp);
+    if (!pobj) {
+        goto cleanup;
+    }
+
+    v = qobject_input_visitor_new_autocast(pobj,
+                                           autocreate_list,
+                                           autocreate_struct_levels,
+                                           permit_int_ranges);
+ cleanup:
+    qobject_decref(pobj);
+    QDECREF(pdict);
+    return v;
+}
util/qemu-option.c +1 -1
index ad28d4e..db0fef2 100644
@@ -1058,7 +1058,7 @@ void qemu_opts_absorb_qdict(QemuOpts *opts, QDict *qdict, Error **errp)
  * TODO We'll want to use types appropriate for opt->desc->type, but
  * this is enough for now.
  */
-QDict *qemu_opts_to_qdict(QemuOpts *opts, QDict *qdict,
+QDict *qemu_opts_to_qdict(const QemuOpts *opts, QDict *qdict,
                           QemuOptsRepeatPolicy repeatPolicy)
 {
     QemuOpt *opt;
Signature · 1 line
2.7.4
#21 lore raw

PATCH v14 13/19 qom: support non-scalar properties with -object

In reply to Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

The QObjectInputVisitor now has functionality that allows it to replace OptsVisitor, maintaining full backwards compatibility for previous CLI syntax, while also allowing use of new syntax for structs.

Thus -object can now support non-scalar properties, for example the QMP object:

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
          { "bar": "one", "wizz": "1" },
          { "bar": "two", "wizz": "2" }
        ]
      }
    }
  }

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

Notice that this syntax is intentionally compatible with that currently used by block drivers. NB the indentation seen here after line continuations should not be used in reality, it is just for clarity in this commit message.

---
 qapi/qobject-input-visitor.c |   2 +-
 qom/object_interfaces.c      |  38 ++++-
 tests/check-qom-proplist.c   | 367 ++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 397 insertions(+), 10 deletions(-)
qapi/qobject-input-visitor.c +1 -1
index 0aef20e..a353d67 100644
@@ -204,7 +204,7 @@ static void qobject_input_start_struct(Visitor *v, const char *name, void **obj,
         *obj = NULL;
     }
 
-    if (!qobj && (qiv->struct_level < qiv->autocreate_struct_levels)) {
+    if (!qobj && (qiv->struct_level <= qiv->autocreate_struct_levels)) {
         /* Create a new dict that contains all the currently
          * unvisited items */
         if (tos) {
qom/object_interfaces.c +30 -8
index f7afe49..faea1b1 100644
@@ -4,7 +4,8 @@
 #include "qemu/module.h"
 #include "qapi-visit.h"
 #include "qapi/qobject-output-visitor.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
+#include "qemu/option.h"
 
 void user_creatable_complete(Object *obj, Error **errp)
 {
@@ -63,12 +64,16 @@ Object *user_creatable_add(const QDict *qdict,
     if (local_err) {
         goto out_visit;
     }
-    visit_check_struct(v, &local_err);
+
+    obj = user_creatable_add_type(type, id, pdict, v, &local_err);
     if (local_err) {
         goto out_visit;
     }
 
-    obj = user_creatable_add_type(type, id, pdict, v, &local_err);
+    visit_check_struct(v, &local_err);
+    if (local_err) {
+        goto out_visit;
+    }
 
 out_visit:
     visit_end_struct(v, NULL);
@@ -114,7 +119,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
 
     assert(qdict);
     obj = object_new(type);
-    visit_start_struct(v, NULL, NULL, 0, &local_err);
+    visit_start_struct(v, "props", NULL, 0, &local_err);
     if (local_err) {
         goto out;
     }
@@ -158,14 +163,31 @@ Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
 {
     Visitor *v;
     QDict *pdict;
+    QObject *pobj;
     Object *obj = NULL;
 
-    v = opts_visitor_new(opts);
     pdict = qemu_opts_to_qdict(opts, NULL,
-                               QEMU_OPTS_REPEAT_POLICY_LAST);
-
-    obj = user_creatable_add(pdict, v, errp);
+                               QEMU_OPTS_REPEAT_POLICY_LIST);
+
+    pobj = qdict_crumple(pdict, true, errp);
+    if (!pobj) {
+        goto cleanup;
+    }
+    /*
+     * We need autocreate_list=true + permit_int_ranges=true
+     * in order to maintain compat with OptsVisitor creation
+     * of the 'numa' object which had an int16List property.
+     *
+     * We need autocreate_structs=1, because at the CLI level
+     * we have the object type + properties in the same flat
+     * struct, even though at the QMP level they are nested.
+     */
+    v = qobject_input_visitor_new_autocast(pobj, true, 1, true);
+
+    obj = user_creatable_add(qobject_to_qdict(pobj), v, errp);
     visit_free(v);
+    qobject_decref(pobj);
+ cleanup:
     QDECREF(pdict);
     return obj;
 }
tests/check-qom-proplist.c +366 -1

466 lines not on this page · raw message

Signature · 1 line
2.7.4
#22 lore raw

PATCH v14 14/19 hmp: support non-scalar properties with object_add

In reply to Daniel P. Berrange

The current object_add HMP syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

The QObjectInputVisitor now has functionality that allows it to replace OptsVisitor, maintaining full backwards compatibility for previous CLI syntax, while also allowing use of new syntax for structs.

Thus -object can now support non-scalar properties, for example the QMP object:

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
          { "bar": "one", "wizz": "1" },
          { "bar": "two", "wizz": "2" }
        ]
      }
    }
  }

Would be creatable via the HMP now using

   object_add demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

Notice that this syntax is intentionally compatible with that currently used by block drivers. NB the indentation seen here after line continuations should not be used in reality, it is just for clarity in this commit message.

---
 hmp.c | 25 +++++++++++++++++--------
 1 file changed, 17 insertions(+), 8 deletions(-)
hmp.c +17 -8
index ad33b44..da77d4c 100644
@@ -25,7 +25,7 @@
 #include "qemu/sockets.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi/util.h"
@@ -1695,21 +1695,30 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
 void hmp_object_add(Monitor *mon, const QDict *qdict)
 {
     Error *err = NULL;
-    QemuOpts *opts;
     Visitor *v;
     Object *obj = NULL;
+    QObject *pobj;
 
-    opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
+    pobj = qdict_crumple(qdict, true, &err);
     if (err) {
-        hmp_handle_error(mon, &err);
-        return;
+        goto cleanup;
     }
 
-    v = opts_visitor_new(opts);
-    obj = user_creatable_add(qdict, v, &err);
+    /*
+     * We need autocreate_list=true + permit_int_ranges=true
+     * in order to maintain compat with OptsVisitor creation
+     * of the 'numa' object which had an int16List property.
+     *
+     * We need autocreate_structs=1, because at the CLI level
+     * we have the object type + properties in the same flat
+     * struct, even though at the QMP level they are nested.
+     */
+    v = qobject_input_visitor_new_autocast(pobj, true, 1, true);
+    obj = user_creatable_add(qobject_to_qdict(pobj), v, &err);
     visit_free(v);
-    qemu_opts_del(opts);
 
+ cleanup:
+    qobject_decref(pobj);
     if (err) {
         hmp_handle_error(mon, &err);
     }
Signature · 1 line
2.7.4
#23 lore raw

PATCH v14 15/19 numa: convert to use QObjectInputVisitor for -numa

In reply to Daniel P. Berrange

Switch away from using OptsVisitor to parse the -numa argument processing. This enables use of the modern list syntax for specifying CPUs. e.g. the old syntax

-numa node,nodeid=0,cpus=0-3,cpus=8-11,mem=107

is equivalent to

  -numa node,nodeid=0,cpus.0=0,cpus.1=1,cpus.2=2,cpus.3=3,\
        cpus.4=8,cpus.5=9,cpus.6=10,cpus.7=11,mem=107

Furthermore, the cli arg can now follow the QAPI schema nesting, so the above is equivalent to the canonical syntax:

  -numa type=node,data.nodeid=0,data.cpus.0=0,data.cpus.1=1,\
        data.cpus.2=2,data.cpus.3=3,data.cpus.4=8,data.cpus.5=9,\
	data.cpus.6=10,data.cpus.7=11,data.mem=107

A test case is added to cover argument parsing to validate that both the old and new syntax is correctly handled.

---
 include/sysemu/numa_int.h |  11 +++++
 numa.c                    |  36 +++++++++-----
 stubs/Makefile.objs       |   5 ++
 stubs/exec.c              |   6 +++
 stubs/hostmem.c           |  14 ++++++
 stubs/memory.c            |  41 ++++++++++++++++
 stubs/qdev.c              |   8 ++++
 stubs/vl.c                |   8 ++++
 stubs/vmstate.c           |   4 ++
 tests/Makefile.include    |   2 +
 tests/test-numa.c         | 116 ++++++++++++++++++++++++++++++++++++++++++++++
 11 files changed, 240 insertions(+), 11 deletions(-)
 create mode 100644 include/sysemu/numa_int.h
 create mode 100644 stubs/exec.c
 create mode 100644 stubs/hostmem.c
 create mode 100644 stubs/memory.c
 create mode 100644 stubs/qdev.c
 create mode 100644 stubs/vl.c
 create mode 100644 tests/test-numa.c
include/sysemu/numa_int.h new file +11 -0
new file mode 100644
index 0000000..93160da
@@ -0,0 +1,11 @@
+#ifndef SYSEMU_NUMA_INT_H
+#define SYSEMU_NUMA_INT_H
+
+#include "sysemu/numa.h"
+
+extern int have_memdevs;
+extern int max_numa_nodeid;
+
+int parse_numa(void *opaque, QemuOpts *opts, Error **errp);
+
+#endif
numa.c +25 -11
index 6289f46..5daa3d1 100644
@@ -23,14 +23,14 @@
  */
 
 #include "qemu/osdep.h"
-#include "sysemu/numa.h"
+#include "sysemu/numa_int.h"
 #include "exec/cpu-common.h"
 #include "qemu/bitmap.h"
 #include "qom/cpu.h"
 #include "qemu/error-report.h"
 #include "include/exec/cpu-common.h" /* for RAM_ADDR_FMT */
 #include "qapi-visit.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "hw/boards.h"
 #include "sysemu/hostmem.h"
 #include "qmp-commands.h"
@@ -45,10 +45,10 @@ QemuOptsList qemu_numa_opts = {
     .desc = { { 0 } } /* validated with OptsVisitor */
 };
 
-static int have_memdevs = -1;
-static int max_numa_nodeid; /* Highest specified NUMA node ID, plus one.
-                             * For all nodes, nodeid < max_numa_nodeid
-                             */
+int have_memdevs = -1;
+int max_numa_nodeid; /* Highest specified NUMA node ID, plus one.
+                      * For all nodes, nodeid < max_numa_nodeid
+                      */
 int nb_numa_nodes;
 NodeInfo numa_info[MAX_NODES];
 
@@ -189,6 +189,9 @@ static void numa_node_parse(NumaNodeOptions *node, QemuOpts *opts, Error **errp)
     if (node->has_mem) {
         uint64_t mem_size = node->mem;
         const char *mem_str = qemu_opt_get(opts, "mem");
+        if (!mem_str) {
+            mem_str = qemu_opt_get(opts, "data.mem");
+        }
         /* Fix up legacy suffix-less format */
         if (g_ascii_isdigit(mem_str[strlen(mem_str) - 1])) {
             mem_size <<= 20;
@@ -211,16 +214,27 @@ static void numa_node_parse(NumaNodeOptions *node, QemuOpts *opts, Error **errp)
     max_numa_nodeid = MAX(max_numa_nodeid, nodenr + 1);
 }
 
-static int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
+int parse_numa(void *opaque, QemuOpts *opts, Error **errp)
 {
     NumaOptions *object = NULL;
     Error *err = NULL;
+    Visitor *v;
 
-    {
-        Visitor *v = opts_visitor_new(opts);
-        visit_type_NumaOptions(v, NULL, &object, &err);
-        visit_free(v);
+    /*
+     * Needs autocreate_list=true and permit_int_ranges=true
+     * in order to support existing syntax for 'cpus' parameter
+     * which is an int list.
+     *
+     * Needs autocreate_struct_levels=1, because existing syntax
+     * used a nested struct in the QMP schema with a flat namespace
+     * in the CLI args.
+     */
+    v = qobject_input_visitor_new_opts(opts, true, 1, true, &err);
+    if (err) {
+        goto end;
     }
+    visit_type_NumaOptions(v, NULL, &object, &err);
+    visit_free(v);
 
     if (err) {
         goto end;
stubs/Makefile.objs +5 -0
index c5850e8..661b48a 100644
@@ -48,3 +48,8 @@ stub-obj-y += iohandler.o
 stub-obj-y += smbios_type_38.o
 stub-obj-y += ipmi.o
 stub-obj-y += pc_madt_cpu_entry.o
+stub-obj-y += vl.o
+stub-obj-y += exec.o
+stub-obj-y += memory.o
+stub-obj-y += hostmem.o
+stub-obj-y += qdev.o
stubs/exec.c new file +6 -0
new file mode 100644
index 0000000..e37f002
@@ -0,0 +1,6 @@
+
+#include "qemu/osdep.h"
+#include "exec/cpu-common.h"
+#include "qom/cpu.h"
+
+struct CPUTailQ cpus = QTAILQ_HEAD_INITIALIZER(cpus);
stubs/hostmem.c new file +14 -0
new file mode 100644
index 0000000..301d853
@@ -0,0 +1,14 @@
+
+#include "qemu/osdep.h"
+#include "sysemu/hostmem.h"
+
+void host_memory_backend_set_mapped(HostMemoryBackend *backend, bool mapped)
+{
+}
+
+
+MemoryRegion *host_memory_backend_get_memory(HostMemoryBackend *backend,
+                                             Error **errp)
+{
+    return NULL;
+}
stubs/memory.c new file +41 -0

41 lines not on this page · raw message

stubs/qdev.c new file +8 -0
new file mode 100644
index 0000000..e28fdf2
@@ -0,0 +1,8 @@
+
+#include "qemu/osdep.h"
+#include "hw/qdev-core.h"
+
+Object *qdev_get_machine(void)
+{
+    return NULL;
+}
stubs/vl.c new file +8 -0
new file mode 100644
index 0000000..698fa80
@@ -0,0 +1,8 @@
+
+#include "qemu/osdep.h"
+#include "exec/cpu-common.h"
+
+int max_cpus = 1;
+ram_addr_t ram_size;
+const char *mem_path;
+int mem_prealloc;
stubs/vmstate.c +4 -0
index 94b831e..f2708ee 100644
@@ -23,3 +23,7 @@ void vmstate_unregister(DeviceState *dev,
                         void *opaque)
 {
 }
+
+void vmstate_register_ram_global(struct MemoryRegion *memory)
+{
+}
tests/Makefile.include +2 -0
index 6398678..7625ef7 100644
@@ -116,6 +116,7 @@ check-unit-$(CONFIG_REPLICATION) += tests/test-replication$(EXESUF)
 check-unit-y += tests/test-bufferiszero$(EXESUF)
 gcov-files-check-bufferiszero-y = util/bufferiszero.c
 check-unit-y += tests/test-uuid$(EXESUF)
+check-unit-y += tests/test-numa$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -583,6 +584,7 @@ tests/test-crypto-pbkdf$(EXESUF): tests/test-crypto-pbkdf.o $(test-crypto-obj-y)
 tests/test-crypto-ivgen$(EXESUF): tests/test-crypto-ivgen.o $(test-crypto-obj-y)
 tests/test-crypto-afsplit$(EXESUF): tests/test-crypto-afsplit.o $(test-crypto-obj-y)
 tests/test-crypto-block$(EXESUF): tests/test-crypto-block.o $(test-crypto-obj-y)
+tests/test-numa$(EXESUF): tests/test-numa.o  numa.o $(test-qom-obj-y)
 
 libqos-obj-y = tests/libqos/pci.o tests/libqos/fw_cfg.o tests/libqos/malloc.o
 libqos-obj-y += tests/libqos/i2c.o tests/libqos/libqos.o
tests/test-numa.c new file +116 -0

116 lines not on this page · raw message

Signature · 1 line
2.7.4
#24 lore raw

PATCH v14 16/19 block: convert crypto driver to use QObjectInputVisitor

In reply to Daniel P. Berrange

The crypto block driver currently uses OptsVisitor to convert from the block driver open/create options into QCryptoBlockOpenOptions/QCryptoBlockCreateOptions. This is easily replaced by use of QObjectInputVisitor with no need to enable any compatibility options, since the structs dealt with contain only scalars.

---
 block/crypto.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
block/crypto.c +9 -3
index 7aa7eb5..fee6c99 100644
@@ -23,7 +23,7 @@
 #include "block/block_int.h"
 #include "sysemu/block-backend.h"
 #include "crypto/block.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi-visit.h"
 #include "qapi/error.h"
 
@@ -206,7 +206,10 @@ block_crypto_open_opts_init(QCryptoBlockFormat format,
     ret = g_new0(QCryptoBlockOpenOptions, 1);
     ret->format = format;
 
-    v = opts_visitor_new(opts);
+    v = qobject_input_visitor_new_opts(opts, false, 0, false, &local_err);
+    if (local_err) {
+        goto out;
+    }
 
     visit_start_struct(v, NULL, NULL, 0, &local_err);
     if (local_err) {
@@ -252,7 +255,10 @@ block_crypto_create_opts_init(QCryptoBlockFormat format,
     ret = g_new0(QCryptoBlockCreateOptions, 1);
     ret->format = format;
 
-    v = opts_visitor_new(opts);
+    v = qobject_input_visitor_new_opts(opts, false, 0, false, &local_err);
+    if (local_err) {
+        goto out;
+    }
 
     visit_start_struct(v, NULL, NULL, 0, &local_err);
     if (local_err) {
Signature · 1 line
2.7.4
#25 lore raw

PATCH v14 17/19 acpi: convert to QObjectInputVisitor for -acpi parsing

In reply to Daniel P. Berrange

The -acpi command line option parsing uses the OptsVisitor currently. This is easily replaced by the QObjectInputVisitor instead. There is no need to enable any of the compatibility options, since the AcpiTableOptions QAPI struct only contains scalar properties.

---
 hw/acpi/core.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
hw/acpi/core.c +7 -7
index e890a5d..480d3dd 100644
@@ -25,7 +25,7 @@
 #include "hw/acpi/acpi.h"
 #include "hw/nvram/fw_cfg.h"
 #include "qemu/config-file.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "qapi-visit.h"
 #include "qapi-event.h"
 
@@ -237,14 +237,14 @@ void acpi_table_add(const QemuOpts *opts, Error **errp)
     char **cur;
     size_t bloblen = 0;
     char unsigned *blob = NULL;
+    Visitor *v;
 
-    {
-        Visitor *v;
-
-        v = opts_visitor_new(opts);
-        visit_type_AcpiTableOptions(v, NULL, &hdrs, &err);
-        visit_free(v);
+    v = qobject_input_visitor_new_opts(opts, false, 0, false, &err);
+    if (err) {
+        goto out;
     }
+    visit_type_AcpiTableOptions(v, NULL, &hdrs, &err);
+    visit_free(v);
 
     if (err) {
         goto out;
Signature · 1 line
2.7.4
#26 lore raw

PATCH v14 18/19 net: convert to QObjectInputVisitor for -net/-netdev parsing

In reply to Daniel P. Berrange

The -net/-netdev command line parsing code uses OptsVisitor for parsing options to populate NetLegacy or NetDev struct respectively. Although those structs have nesting, the OptsVisitor flattens them, so we must enable compatibility options to auto-create structs. This allows the legacy syntax

-net tap,id=net0,vlan=3,fd=3,script=/bin/ifup-qemu

to be treated as equivalent to the modern QAPI based syntax

-net id=net0,vlan=3,opts.type=tap,opts.data.fd=3,opts.data.script=/bin/ifup-qemu

---
 net/net.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
net/net.c +15 -2
index d51cb29..af3645b 100644
@@ -43,7 +43,7 @@
 #include "qemu/iov.h"
 #include "qemu/main-loop.h"
 #include "qapi-visit.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qobject-input-visitor.h"
 #include "sysemu/sysemu.h"
 #include "net/filter.h"
 #include "qapi/string-output-visitor.h"
@@ -1065,7 +1065,20 @@ int net_client_init(QemuOpts *opts, bool is_netdev, Error **errp)
     void *object = NULL;
     Error *err = NULL;
     int ret = -1;
-    Visitor *v = opts_visitor_new(opts);
+    /*
+     * Needs autocreate_lists=true in order support existing
+     * syntax for list options where the bare key is repeated
+     *
+     * Needs autocreate_struct_levels=3 in order to deal with
+     * 3 level nesting in NetLegacy option args, which was
+     * exposed as a flat namespace with OptVisitor
+     */
+    Visitor *v = qobject_input_visitor_new_opts(opts, true, 3, false, &err);
+
+    if (err) {
+        error_propagate(errp, err);
+        return -1;
+    }
 
     {
         /* Parse convenience option format ip6-net=fec0::0[/64] */
Signature · 1 line
2.7.4
#27 lore raw

PATCH v14 19/19 qapi: delete unused OptsVisitor code

In reply to Daniel P. Berrange

Now that all code has been converted to QObjectInputVisitor's QemuOpts compatibility mode, there is no longer any reason to keep OptsVisitor alive.

---
 include/qapi/opts-visitor.h |  40 ----
 qapi/Makefile.objs          |   2 +-
 qapi/opts-visitor.c         | 544 --------------------------------------------
 tests/Makefile.include      |   5 +-
 tests/test-opts-visitor.c   | 268 ----------------------
 vl.c                        |   1 -
 6 files changed, 2 insertions(+), 858 deletions(-)
 delete mode 100644 include/qapi/opts-visitor.h
 delete mode 100644 qapi/opts-visitor.c
 delete mode 100644 tests/test-opts-visitor.c
include/qapi/opts-visitor.h deleted +0 -40
deleted file mode 100644
index 6462c96..0000000
@@ -1,40 +0,0 @@
-/*
- * Options Visitor
- *
- * Copyright Red Hat, Inc. 2012
- *
- * Author: Laszlo Ersek <[email protected]>
- *
- * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
- * See the COPYING.LIB file in the top-level directory.
- *
- */
-
-#ifndef OPTS_VISITOR_H
-#define OPTS_VISITOR_H
-
-#include "qapi/visitor.h"
-#include "qemu/option.h"
-
-/* Inclusive upper bound on the size of any flattened range. This is a safety
- * (= anti-annoyance) measure; wrong ranges should not cause long startup
- * delays nor exhaust virtual memory.
- */
-#define OPTS_VISITOR_RANGE_MAX 65536
-
-typedef struct OptsVisitor OptsVisitor;
-
-/* Contrarily to qemu-option.c::parse_option_number(), OptsVisitor's "int"
- * parser relies on strtoll() instead of strtoull(). Consequences:
- * - string representations of negative numbers yield negative values,
- * - values below INT64_MIN or LLONG_MIN are rejected,
- * - values above INT64_MAX or LLONG_MAX are rejected.
- *
- * The Opts input visitor does not implement support for visiting QAPI
- * alternates, numbers (other than integers), null, or arbitrary
- * QTypes.  It also requires a non-null list argument to
- * visit_start_list().
- */
-Visitor *opts_visitor_new(const QemuOpts *opts);
-
-#endif
qapi/Makefile.objs +1 -1
index 33906ff..4b820a7 100644
@@ -1,6 +1,6 @@
 util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o
 util-obj-y += qobject-output-visitor.o qmp-registry.o qmp-dispatch.o
 util-obj-y += string-input-visitor.o string-output-visitor.o
-util-obj-y += opts-visitor.o qapi-clone-visitor.o
+util-obj-y += qapi-clone-visitor.o
 util-obj-y += qmp-event.o
 util-obj-y += qapi-util.o
qapi/opts-visitor.c deleted +0 -544

544 lines not on this page · raw message

tests/Makefile.include +1 -4
index 7625ef7..47c223b 100644
@@ -35,8 +35,6 @@ check-unit-y += tests/test-string-output-visitor$(EXESUF)
 gcov-files-test-string-output-visitor-y = qapi/string-output-visitor.c
 check-unit-y += tests/test-qmp-event$(EXESUF)
 gcov-files-test-qmp-event-y += qapi/qmp-event.c
-check-unit-y += tests/test-opts-visitor$(EXESUF)
-gcov-files-test-opts-visitor-y = qapi/opts-visitor.c
 check-unit-y += tests/test-coroutine$(EXESUF)
 gcov-files-test-coroutine-y = coroutine-$(CONFIG_COROUTINE_BACKEND).c
 check-unit-y += tests/test-visitor-serialization$(EXESUF)
@@ -444,7 +442,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \
 	tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \
 	tests/test-qmp-commands.o tests/test-visitor-serialization.o \
 	tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \
-	tests/test-opts-visitor.o tests/test-qmp-event.o \
+	tests/test-qmp-event.o \
 	tests/rcutorture.o tests/test-rcu-list.o \
 	tests/test-qdist.o \
 	tests/test-qht.o tests/qht-bench.o tests/test-qht-par.o
@@ -548,7 +546,6 @@ tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(
 tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
 tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y)
 tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o $(test-qapi-obj-y)
-tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
 
 tests/test-mul64$(EXESUF): tests/test-mul64.o $(test-util-obj-y)
 tests/test-bitops$(EXESUF): tests/test-bitops.o $(test-util-obj-y)
tests/test-opts-visitor.c deleted +0 -268

268 lines not on this page · raw message

vl.c +0 -1
index 215a6f9..469cf8d 100644
@@ -115,7 +115,6 @@ int main(int argc, char **argv)
 
 #include "ui/qemu-spice.h"
 #include "qapi/string-input-visitor.h"
-#include "qapi/opts-visitor.h"
 #include "qom/object_interfaces.h"
 #include "qapi-event.h"
 #include "exec/semihost.h"
Signature · 1 line
2.7.4
#28 lore raw
In reply to Daniel P. Berrange

I've warmed quite a bit to this series while reviewing it. In particular, I've come around to like structuring the command line -> QAPI pipeline exactly like the JSON -> QAPI pipeline, namely 1. parse into QObject, 2. convert to QAPI with the QObject input visitor. QObject serves as abstract syntax here. The differences between JSON and command line result in different abstract syntax, which in turn necessitates two cases in the input visitor. The series adds more than two, to cater for option visitor funnies. Perhaps we can do without some of them.

The other way to skin this cat would be an improved options visitor. Has its advantages and disadvantages, but the big one is that you already did the work for QObject input visitor solution.

The one major issue I have with the series is that it adds to the growing body of QemuOpts hacks/workarounds.

We've pushed QemuOpts beyond well its design limits. What started as a simple store for scalar configuration parameters structured as key=value lists, plus command line and configuration file syntax, has grown three ways to specify lists (repeated keys, basically an implementation accident that got pressed into service; special integer list syntax in the options visitor, later adopted by the string input visitor, hopefully compatibly; and the block layer's dotted key convention) and a way to specify arbitrary complex values (block layer's dotted key convention again). Of these, only "repeated keys" is in QemuOpts proper, all the others are bolted on and used only when needed. How they interact is not obvious.

This series marries all the bolted-on stuff and puts it in the QObject visitor. That's actually an improvement of sorts; at least it's in just two places now. But it's still a smorgasbord of syntactical/semantic options.

I feel it's time to stop working around the design limits of QemuOpts and start replacing them by something that serves our current needs. We basically need the expressive power of JSON on the command line. Syntax is debatable, but it should be *one* concrete command-line syntax, parsed by *one* parser into *one* kind of abstract syntax tree, where the only optional variations are the ones forced upon us by backward compatibility.

We can't do this for 2.8, obviously. We can try for 2.9. I have pretty specific ideas on how it should be done, so I guess it's only fair I give it a try myself.

I know the block layer wants to use bits of this series to save some coding work for certain features targeting 2.8. I have no objections as long as it doesn't create new ABI. Exception: I'm okay with applying dotted key convention to more of the same, e.g. new block drivers.

Sounds sane?

#29 lore raw
Quoted 46 lines · Markus Armbruster

I've warmed quite a bit to this series while reviewing it. In particular, I've come around to like structuring the command line -> QAPI pipeline exactly like the JSON -> QAPI pipeline, namely 1. parse into QObject, 2. convert to QAPI with the QObject input visitor. QObject serves as abstract syntax here. The differences between JSON and command line result in different abstract syntax, which in turn necessitates two cases in the input visitor. The series adds more than two, to cater for option visitor funnies. Perhaps we can do without some of them.

The other way to skin this cat would be an improved options visitor. Has its advantages and disadvantages, but the big one is that you already did the work for QObject input visitor solution.

The one major issue I have with the series is that it adds to the growing body of QemuOpts hacks/workarounds.

We've pushed QemuOpts beyond well its design limits. What started as a simple store for scalar configuration parameters structured as key=value lists, plus command line and configuration file syntax, has grown three ways to specify lists (repeated keys, basically an implementation accident that got pressed into service; special integer list syntax in the options visitor, later adopted by the string input visitor, hopefully compatibly; and the block layer's dotted key convention) and a way to specify arbitrary complex values (block layer's dotted key convention again). Of these, only "repeated keys" is in QemuOpts proper, all the others are bolted on and used only when needed. How they interact is not obvious.

This series marries all the bolted-on stuff and puts it in the QObject visitor. That's actually an improvement of sorts; at least it's in just two places now. But it's still a smorgasbord of syntactical/semantic options.

I feel it's time to stop working around the design limits of QemuOpts and start replacing them by something that serves our current needs. We basically need the expressive power of JSON on the command line. Syntax is debatable, but it should be *one* concrete command-line syntax, parsed by *one* parser into *one* kind of abstract syntax tree, where the only optional variations are the ones forced upon us by backward compatibility.

We can't do this for 2.8, obviously. We can try for 2.9. I have pretty specific ideas on how it should be done, so I guess it's only fair I give it a try myself.

Quoted 6 lines

I know the block layer wants to use bits of this series to save some coding work for certain features targeting 2.8. I have no objections as long as it doesn't create new ABI. Exception: I'm okay with applying dotted key convention to more of the same, e.g. new block drivers.

Sounds sane?

It is all fine with me.

Of the patch series proposed, I think patches 1->6 ought to be safe to take for 2.8, without exposing new ABI:

1. option: make parse_option_bool/number non-static
2. qdict: implement a qdict_crumple method for un-flattening a dict
3. qapi: add trace events for visitor
4. qapi: rename QmpInputVisitor to QObjectInputVisitor
5. qapi: rename QmpOutputVisitor to QObjectOutputVisitor
6. qapi: don't pass two copies of TestInputVisitorData to tests

The rest are clearly not an option for 2.8 since freeze is barely 1 week away, and I'm travelling all next week

Regards,
Daniel

Signature · 3 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|
#30 lore raw
In reply to Markus Armbruster

Uh, replied to the wrong v14... The one I reviewed is actually v15,
Message-Id: <[email protected]>
http://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08238.html

Quoted 52 lines · Markus Armbruster

I've warmed quite a bit to this series while reviewing it. In particular, I've come around to like structuring the command line -> QAPI pipeline exactly like the JSON -> QAPI pipeline, namely 1. parse into QObject, 2. convert to QAPI with the QObject input visitor. QObject serves as abstract syntax here. The differences between JSON and command line result in different abstract syntax, which in turn necessitates two cases in the input visitor. The series adds more than two, to cater for option visitor funnies. Perhaps we can do without some of them.

The other way to skin this cat would be an improved options visitor. Has its advantages and disadvantages, but the big one is that you already did the work for QObject input visitor solution.

The one major issue I have with the series is that it adds to the growing body of QemuOpts hacks/workarounds.

We've pushed QemuOpts beyond well its design limits. What started as a simple store for scalar configuration parameters structured as key=value lists, plus command line and configuration file syntax, has grown three ways to specify lists (repeated keys, basically an implementation accident that got pressed into service; special integer list syntax in the options visitor, later adopted by the string input visitor, hopefully compatibly; and the block layer's dotted key convention) and a way to specify arbitrary complex values (block layer's dotted key convention again). Of these, only "repeated keys" is in QemuOpts proper, all the others are bolted on and used only when needed. How they interact is not obvious.

This series marries all the bolted-on stuff and puts it in the QObject visitor. That's actually an improvement of sorts; at least it's in just two places now. But it's still a smorgasbord of syntactical/semantic options.

I feel it's time to stop working around the design limits of QemuOpts and start replacing them by something that serves our current needs. We basically need the expressive power of JSON on the command line. Syntax is debatable, but it should be *one* concrete command-line syntax, parsed by *one* parser into *one* kind of abstract syntax tree, where the only optional variations are the ones forced upon us by backward compatibility.

We can't do this for 2.8, obviously. We can try for 2.9. I have pretty specific ideas on how it should be done, so I guess it's only fair I give it a try myself.

I know the block layer wants to use bits of this series to save some coding work for certain features targeting 2.8. I have no objections as long as it doesn't create new ABI. Exception: I'm okay with applying dotted key convention to more of the same, e.g. new block drivers.

Sounds sane?

#31 lore raw
Quoted 3 lines · Markus Armbruster

Uh, replied to the wrong v14... The one I reviewed is actually v15,
Message-Id: <[email protected]>
http://lists.gnu.org/archive/html/qemu-devel/2016-09/msg08238.html

Sorry, my bad for screwing up the subject when i posted v15.

Regards,
Daniel

Signature · 3 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|