QAPI/QOM work for non-scalar object properties
v13 cover QAPI/QOM work for non-scalar object properties
This patch series contains only the QAPI/QOM bits of my previous access control patch series:
v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
v5: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg00485.html
v6: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg03876.html
v7: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg00919.html
v8: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg03115.html
v9: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02653.html
v10: https://lists.gnu.org/archive/html/qemu-devel/2016-08/msg02694.html
v11: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg00652.html
v12: https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg03559.html
This series provides the infrastructure to allow use of non-scalar properties with the -object CLI arg, and object_add monitor commands. eg a property which is a list of structs. The syntax used for this is intentionally compatible with the syntax used by the block layer. This will allow the qdict_crumple method to be used by the block layer to convert from QemuOpts into structured QAPI block layer structs at a future date. It is already used by one of Max's patch series, and recent patches for glusterfs multiple-host support could have made use of it to simplify code.
Changed in v13:
- Fix typos (Kevin)
- Remove unneeded line breaks (Kevin)
Changed in v12:
- More user friendly error message for mixing dict/list keys (Kevin)
- Report error instead of assert for non-contiguous list keys (Kevin)
- Fix tests for non-contiguous list keys (Kevin)
- Add tests for escaping of '.' when crumpling (Kevin)
- Fix remaining references to Qmp(In|Out)putVisitor (Markus)
- Misc typos / whitespace fixes (Eric, Kevin)
- Avoid touching 'ret' when parsing int64 fails (Eric)
- Testing of more edge cases in QObjectInputVisitor (Eric)
- Simplify API doc format (Markus)
- Use parse_option_size instead of qemu_strtosz_suffix for consistency (Kevin)
- Use safer qobject_to_qdict casts (Eric)
Changed in v11:
- Split QAPI/QOM patches off from the access control patches
Changed in v10:
- Fixed stupid build mistake
Changed in v9:
- Rename QmpInputVisitor -> QObjectInputVisitor (Markus/Eric)
- Rename QmpOutputVisitor -> QObjectOutputVisitor (Markus/Eric)
- Drop "strict" param from qobject_string_visitor_new() (Marus)
- Misc docs typos
- Add a visitor able to use strict or string types (for Eric's netdev series)
- Add a authorization API implementation that uses PAM
Changed in v8:
- Rebase due to merge of Visitor API changes (Eric)
Changed in v7:
- Misc typos in API docs (Marc-André)
- Fix parsing of properties using type_size visitor (Marc-André)
- Mark based auth class as abstract (Marc-André)
- Fix QAPI version annotations to say 2.7 (Marc-André)
Changed in v6:
- Switch from while() to for() loop for iterating over dicts (Markus)
- Avoid redundant strdup (Markus)
- Rewrap comments at 70 chars (Markus)
- Change qdict_list_size() to qdict_is_list() (Markus)
- Misc docs changes (Markus)
- Change QmpInputVisitor so the code for handling the string types is separate from code using native scalar types (Paolo)
- Centralize code parsing bool strings (Markus)
- Centralize code parsing int strings (Markus)
Changed in v5:
- Resolved conflicts with Eric's visitor refactoring which made it stricter about struct begin/end calls
- Added support for ACLs to migration code now its TLS support is merged.
- Fixed typos in example in commit message
Changed in v4:
- Ensure examples use shell escaping for '*' (Eric)
- Add more tests for crumple impl (Eric)
- Raise error if sasl-acl/tls-acl are requested but sasl/tls auth are not enabled (Eric)
- Document return codes for auth check more clearly (Eric)
- Don't silently turn a glob match into a strcmp
- Other misc small typos/fixes (Eric)
Changed in v3:
- Created separate qdict_list_size method (Max)
- Added unit tests for case of empty dict (Max)
- Fix variable names to use underscore separator (Max)
- Fix potential free of uninitialized variables (Max)
- Use QObject APIs for casts, instead of C type casts (Max)
Changed in v2:
- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple
Daniel P. Berrange (6):
qdict: implement a qdict_crumple method for un-flattening a dict
option: make parse_option_bool/number non-static
qapi: rename QmpInputVisitor to QObjectInputVisitor
qapi: rename QmpOutputVisitor to QObjectOutputVisitor
qapi: add a QObjectInputVisitor that does string conversion
qom: support arbitrary non-scalar properties with -object
block/qapi.c | 4 +-
blockdev.c | 4 +-
docs/qapi-code-gen.txt | 4 +-
hmp.c | 16 +-
include/qapi/qmp-input-visitor.h | 30 --
include/qapi/qmp/qdict.h | 1 +
include/qapi/qobject-input-visitor.h | 54 ++
...p-output-visitor.h => qobject-output-visitor.h} | 10 +-
include/qapi/visitor.h | 6 +-
include/qemu/option.h | 4 +
include/qom/object_interfaces.h | 10 +-
monitor.c | 2 +-
qapi/Makefile.objs | 4 +-
qapi/opts-visitor.c | 19 +-
qapi/qapi-clone-visitor.c | 2 +-
qapi/qmp-input-visitor.c | 412 ----------------
qapi/qmp-output-visitor.c | 256 ----------
qapi/qobject-input-visitor.c | 543 +++++++++++++++++++++
qapi/qobject-output-visitor.c | 254 ++++++++++
qemu-img.c | 8 +-
qmp.c | 6 +-
qobject/qdict.c | 291 +++++++++++
qom/object_interfaces.c | 47 +-
qom/qom-qobject.c | 8 +-
scripts/qapi-commands.py | 8 +-
scripts/qapi-event.py | 4 +-
target-s390x/cpu_models.c | 4 +-
tests/.gitignore | 6 +-
tests/Makefile.include | 20 +-
tests/check-qdict.c | 260 ++++++++++
tests/check-qnull.c | 8 +-
tests/check-qom-proplist.c | 334 ++++++++++++-
tests/test-qmp-commands.c | 4 +-
...-input-strict.c => test-qobject-input-strict.c} | 6 +-
...nput-visitor.c => test-qobject-input-visitor.c} | 207 +++++++-
...put-visitor.c => test-qobject-output-visitor.c} | 6 +-
tests/test-string-input-visitor.c | 2 +-
tests/test-string-output-visitor.c | 2 +-
tests/test-visitor-serialization.c | 8 +-
util/qemu-option.c | 27 +-
util/qemu-sockets.c | 4 +-
41 files changed, 2073 insertions(+), 832 deletions(-)
delete mode 100644 include/qapi/qmp-input-visitor.h
create mode 100644 include/qapi/qobject-input-visitor.h
rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
delete mode 100644 qapi/qmp-input-visitor.c
delete mode 100644 qapi/qmp-output-visitor.c
create mode 100644 qapi/qobject-input-visitor.c
create mode 100644 qapi/qobject-output-visitor.c
rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (81%)
rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)Signature · 1 line
2.7.4
PATCH v13 2/6 option: make parse_option_bool/number non-static
The opts-visitor.c opts_type_bool() method has code for parsing a string to set a bool value, as does the qemu-option.c parse_option_bool() method, except it handles fewer cases.
To enable consistency across the codebase, extend parse_option_bool() to handle "yes", "no", "y" and "n", and make it non-static. Convert the opts visitor to call this method directly.
Also make parse_option_number() non-static to allow for similar reuse later.
--- include/qemu/option.h | 4 ++++ qapi/opts-visitor.c | 19 +------------------ util/qemu-option.c | 27 ++++++++++++++++----------- 3 files changed, 21 insertions(+), 29 deletions(-)
include/qemu/option.h +4 -0
@@ -37,6 +37,10 @@ int get_param_value(char *buf, int buf_size, const char *tag, const char *str); +void parse_option_bool (const char *name,const char *value,bool *ret, +Error **errp); +void parse_option_number (const char *name,const char *value, +uint64_t *ret,Error **errp);void parse_option_size (const char *name,const char *value,uint64_t *ret,Error **errp);bool has_help_option (const char *param);
qapi/opts-visitor.c +1 -18
@@ -334,7 +334,6 @@ opts_type_str(Visitor *v, const char *name, char **obj, Error **errp) } -/* mimics qemu-option.c::parse_option_bool() */ static void opts_type_bool (Visitor *v,const char *name,bool *obj,Error **errp) {
@@ -346,23 +345,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)return ; } -if (opt->str) { -if (strcmp (opt->str,"on" ) ==0 || -strcmp (opt->str,"yes" ) ==0 || -strcmp (opt->str,"y" ) ==0 ) { - *obj = true; - }else if (strcmp (opt->str,"off" ) ==0 || -strcmp (opt->str,"no" ) ==0 || -strcmp (opt->str,"n" ) ==0 ) { - *obj = false; - }else { -error_setg (errp,QERR_INVALID_PARAMETER_VALUE , opt->name, -"on|yes|y|off|no|n" ); -return ; - } - }else { - *obj = true; - } +parse_option_bool (opt->name, opt->str, obj, errp);processed (ov, name); }
util/qemu-option.c +16 -11
@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,return get_next_param_value (buf, buf_size, tag, &str); } -static void parse_option_bool (const char *name,const char *value,bool *ret, -Error **errp) +void parse_option_bool (const char *name,const char *value,bool *ret, +Error **errp) {if (value !=NULL ) { -if (!strcmp (value,"on" )) { - *ret =1 ; - }else if (!strcmp (value,"off" )) { - *ret =0 ; +if (strcmp (value,"on" ) ==0 || +strcmp (value,"yes" ) ==0 || +strcmp (value,"y" ) ==0 ) { + *ret = true; + }else if (strcmp (value,"off" ) ==0 || +strcmp (value,"no" ) ==0 || +strcmp (value,"n" ) ==0 ) { + *ret = false; }else { -error_setg (errp,QERR_INVALID_PARAMETER_VALUE , - name,"'on' or 'off'" ); +error_setg (errp,QERR_INVALID_PARAMETER_VALUE , name, +"on|yes|y|off|no|n" ); +return ; } }else { - *ret =1 ; + *ret = true; } } -static void parse_option_number (const char *name,const char *value, -uint64_t *ret,Error **errp) +void parse_option_number (const char *name,const char *value, +uint64_t *ret,Error **errp) {char *postfix;uint64_t number;
Signature · 1 line
2.7.4
PATCH v13 1/6 qdict: implement a qdict_crumple method for un-flattening a dict
In reply to Daniel P. BerrangeThe qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.
The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.
If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.
As an example, a flat dict containing
{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}
will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.
{
'foo': [
{ 'bar': 'one', 'wizz': '1' },
{ 'bar': 'two', 'wizz': '2' } ],
}
If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict
{
'foo..bar': 'wizz',
'bar.foo..bar': 'eek',
'bar.hello': 'world'
}Will end up as
{
'foo.bar': 'wizz',
'bar': {
'foo.bar': 'eek',
'hello': 'world'
}
}The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nesting used when the same object is defined over QMP.
--- include/qapi/qmp/qdict.h | 1 + qobject/qdict.c | 291 +++++++++++++++++++++++++++++++++++++++++++++++ tests/check-qdict.c | 260 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 552 insertions(+)
include/qapi/qmp/qdict.h +1 -0
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);void qdict_extract_subqdict (QDict *src,QDict **dst,const char *start);void qdict_array_split (QDict *src,QList **dst);int qdict_array_entries (QDict *src,const char *subqdict); +QObject *qdict_crumple (QDict *src,bool recursive,Error **errp);void qdict_join (QDict *dest,QDict *src,bool overwrite);
qobject/qdict.c +291 -0
303 lines not on this page · raw message
tests/check-qdict.c +260 -0
278 lines not on this page · raw message
Signature · 1 line
2.7.4
PATCH v13 3/6 qapi: rename QmpInputVisitor to QObjectInputVisitor
In reply to Daniel P. BerrangeThe QmpInputVisitor has no direct dependency on QMP. It is valid to use it anywhere that one has a QObject. Rename it to better reflect its functionality as a generic QObject to QAPI converter.
---
docs/qapi-code-gen.txt | 2 +-
...qmp-input-visitor.h => qobject-input-visitor.h} | 10 +-
include/qapi/visitor.h | 6 +-
monitor.c | 2 +-
qapi/Makefile.objs | 2 +-
...qmp-input-visitor.c => qobject-input-visitor.c} | 171 +++++++++++----------
qmp.c | 4 +-
qom/qom-qobject.c | 4 +-
scripts/qapi-commands.py | 4 +-
target-s390x/cpu_models.c | 4 +-
tests/.gitignore | 4 +-
tests/Makefile.include | 12 +-
tests/check-qnull.c | 4 +-
tests/test-qmp-commands.c | 4 +-
...-input-strict.c => test-qobject-input-strict.c} | 6 +-
...nput-visitor.c => test-qobject-input-visitor.c} | 6 +-
tests/test-string-input-visitor.c | 2 +-
tests/test-visitor-serialization.c | 4 +-
util/qemu-sockets.c | 2 +-
19 files changed, 128 insertions(+), 125 deletions(-)
rename include/qapi/{qmp-input-visitor.h => qobject-input-visitor.h} (63%)
rename qapi/{qmp-input-visitor.c => qobject-input-visitor.c} (56%)
rename tests/{test-qmp-input-strict.c => test-qobject-input-strict.c} (98%)
rename tests/{test-qmp-input-visitor.c => test-qobject-input-visitor.c} (99%)docs/qapi-code-gen.txt +1 -1
@@ -1024,7 +1024,7 @@ Example: Visitor *v; UserDefOneList *arg1 = NULL; - v = qmp_input_visitor_new(QOBJECT(args), true); + v = qobject_input_visitor_new(QOBJECT(args), true); visit_start_struct(v, NULL, NULL, 0, &err); if (err) { goto out;
include/qapi/qobject-input-visitor.h renamed from include/qapi/qmp-input-visitor.h +5 -5
@@ -11,20 +11,20 @@* */ -#ifndef QMP_INPUT_VISITOR_H -#define QMP_INPUT_VISITOR_H +#ifndef QOBJECT_INPUT_VISITOR_H +#define QOBJECT_INPUT_VISITOR_H #include "qapi/visitor.h" #include "qapi/qmp/qobject.h" -typedef struct QmpInputVisitor QmpInputVisitor ; +typedef struct QObjectInputVisitor QObjectInputVisitor ;/* -* Return a new input visitor that converts QMP to QAPI. +* Return a new input visitor that converts a QObject to a QAPI object. * * Set @strict to reject a parse that doesn't consume all keys of a * dictionary; otherwise excess input is ignored. */ -Visitor *qmp_input_visitor_new (QObject *obj,bool strict); +Visitor *qobject_input_visitor_new (QObject *obj,bool strict);#endif
include/qapi/visitor.h +3 -3
@@ -25,14 +25,14 @@* for doing work at each node of a QAPI graph; it can also be used * for a virtual walk, where there is no actual QAPI C struct. * -* There are four kinds of visitor classes: input visitors (QMP, +* There are four kinds of visitor classes: input visitors (QObject, * string, and QemuOpts) parse an external representation and build -* the corresponding QAPI graph, output visitors (QMP and string) take +* the corresponding QAPI graph, output visitors (QObject and string) take * a completed QAPI graph and generate an external representation, the * dealloc visitor can take a QAPI graph (possibly partially * constructed) and recursively free its resources, and the clone * visitor performs a deep clone of one QAPI object to another. While -* the dealloc and QMP input/output visitors are general, the string, +* the dealloc and QObject input/output visitors are general, the string, * QemuOpts, and clone visitors have some implementation limitations; * see the documentation for each visitor for more details on what it * supports. Also, see visitor-impl.h for the callback contracts
monitor.c +1 -1
@@ -998,7 +998,7 @@ EventInfoList *qmp_query_events(Error **errp)* directly into QObject instead of first parsing it with * visit_type_SchemaInfoList() into a SchemaInfoList, then marshal it * to QObject with generated output marshallers, every time. Instead, -* we do it in test-qmp-input-visitor.c, just to make sure +* we do it in test-qobject-input-visitor.c, just to make sure * qapi-introspect.py's output actually conforms to the schema. */ static void qmp_query_qmp_schema(QDict *qdict, QObject **ret_data,
qapi/Makefile.objs +1 -1
@@ -1,4 +1,4 @@ -util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qmp-input-visitor.o +util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o util-obj-y += string-input-visitor.o string-output-visitor.o util-obj-y += opts-visitor.o qapi-clone-visitor.o
qapi/qobject-input-visitor.c renamed from qapi/qmp-input-visitor.c +87 -84
336 lines not on this page · raw message
qmp.c +2 -2
@@ -30,7 +30,7 @@#include "qom/qom-qobject.h" #include "qapi/qmp/qerror.h" #include "qapi/qmp/qobject.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "hw/boards.h" #include "qom/object_interfaces.h" #include "hw/mem/pc-dimm.h"
@@ -687,7 +687,7 @@ void qmp_object_add(const char *type, const char *id, } } - v =qmp_input_visitor_new (props, true); + v =qobject_input_visitor_new (props, true); obj =user_creatable_add_type (type, id, pdict, v, errp);visit_free (v);if (obj) {
qom/qom-qobject.c +2 -2
@@ -15,7 +15,7 @@#include "qom/object.h" #include "qom/qom-qobject.h" #include "qapi/visitor.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qmp-output-visitor.h" void object_property_set_qobject(Object *obj, QObject *value,
@@ -23,7 +23,7 @@ void object_property_set_qobject(Object *obj, QObject *value, {Visitor *v;/* TODO: Should we reject, rather than ignore, excess input? */ - v =qmp_input_visitor_new (value, false); + v =qobject_input_visitor_new (value, false);object_property_set (obj, v, name, errp);visit_free (v); }
scripts/qapi-commands.py +2 -2
@@ -117,7 +117,7 @@ def gen_marshal(name, arg_type, boxed, ret_type): Visitor *v; %(c_name)s arg = {0}; - v = qmp_input_visitor_new(QOBJECT(args), true); + v = qobject_input_visitor_new(QOBJECT(args), true); visit_start_struct(v, NULL, NULL, 0, &err); if (err) { goto out;
@@ -279,7 +279,7 @@ fdef.write(mcgen(''' #include "qapi/qmp/dispatch.h" #include "qapi/visitor.h" #include "qapi/qmp-output-visitor.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h" #include "%(prefix)sqapi-visit.h"
target-s390x/cpu_models.c +2 -2
@@ -17,7 +17,7 @@#include "qapi/visitor.h" #include "qemu/error-report.h" #include "qapi/qmp/qerror.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qmp/qbool.h" #ifndef CONFIG_USER_ONLY #include "sysemu/arch_init.h"
@@ -345,7 +345,7 @@ static void cpu_model_from_info(S390CPUModel *model, const CpuModelInfo *info, }if (qdict) { - visitor =qmp_input_visitor_new (info->props, true); + visitor =qobject_input_visitor_new (info->props, true);visit_start_struct (visitor,NULL ,NULL ,0 , errp);if (*errp) {object_unref (obj);
tests/.gitignore +2 -2
@@ -57,8 +57,8 @@ test-qht-par test-qmp-commands test-qmp-commands.h test-qmp-event -test-qmp-input-strict -test-qmp-input-visitor +test-qobject-input-strict +test-qobject-input-visitor test-qmp-introspect.[ch] test-qmp-marshal.c test-qmp-output-visitor
tests/Makefile.include +6 -6
@@ -24,9 +24,9 @@ check-unit-y += tests/test-qmp-output-visitor$(EXESUF) gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c check-unit-y += tests/test-clone-visitor$(EXESUF) gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c -check-unit-y += tests/test-qmp-input-visitor$(EXESUF) -gcov-files-test-qmp-input-visitor-y = qapi/qmp-input-visitor.c -check-unit-y += tests/test-qmp-input-strict$(EXESUF) +check-unit-y += tests/test-qobject-input-visitor$(EXESUF) +gcov-files-test-qobject-input-visitor-y = qapi/qobject-input-visitor.c +check-unit-y += tests/test-qobject-input-strict$(EXESUF) check-unit-y += tests/test-qmp-commands$(EXESUF) gcov-files-test-qmp-commands-y = qapi/qmp-dispatch.c check-unit-y += tests/test-string-input-visitor$(EXESUF)
@@ -437,7 +437,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \ tests/test-coroutine.o tests/test-string-output-visitor.o \ tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \ tests/test-clone-visitor.o \ - tests/test-qmp-input-visitor.o tests/test-qmp-input-strict.o \ + tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \ tests/test-qmp-commands.o tests/test-visitor-serialization.o \ tests/test-x86-cpuid.o tests/test-mul64.o tests/test-int128.o \ tests/test-opts-visitor.o tests/test-qmp-event.o \
@@ -540,8 +540,8 @@ tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(te tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y) tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y) -tests/test-qmp-input-visitor$(EXESUF): tests/test-qmp-input-visitor.o $(test-qapi-obj-y) -tests/test-qmp-input-strict$(EXESUF): tests/test-qmp-input-strict.o $(test-qapi-obj-y) +tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y) +tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y) tests/test-qmp-commands$(EXESUF): tests/test-qmp-commands.o tests/test-qmp-marshal.o $(test-qapi-obj-y) tests/test-visitor-serialization$(EXESUF): tests/test-visitor-serialization.o $(test-qapi-obj-y) tests/test-opts-visitor$(EXESUF): tests/test-opts-visitor.o $(test-qapi-obj-y)
tests/check-qnull.c +2 -2
@@ -10,7 +10,7 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qmp-output-visitor.h" #include "qapi/error.h"
@@ -47,7 +47,7 @@ static void qnull_visit_test(void)g_assert (qnull_.refcnt ==1 ); obj =qnull (); - v =qmp_input_visitor_new (obj, true); + v =qobject_input_visitor_new (obj, true);qobject_decref (obj);visit_type_null (v,NULL , &error_abort);visit_free (v);
tests/test-qmp-commands.c +2 -2
@@ -4,7 +4,7 @@#include "test-qmp-commands.h" #include "qapi/qmp/dispatch.h" #include "qemu/module.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "tests/test-qapi-types.h" #include "tests/test-qapi-visit.h"
@@ -229,7 +229,7 @@ static void test_dealloc_partial(void) ud2_dict =qdict_new ();qdict_put_obj (ud2_dict,"string0" ,QOBJECT (qstring_from_str (text))); - v =qmp_input_visitor_new (QOBJECT (ud2_dict), true); + v =qobject_input_visitor_new (QOBJECT (ud2_dict), true);visit_type_UserDefTwo (v,NULL , &ud2, &err);visit_free (v);QDECREF (ud2_dict);
tests/test-qobject-input-strict.c renamed from tests/test-qmp-input-strict.c +3 -3
@@ -1,5 +1,5 @@/* -* QMP Input Visitor unit-tests (strict mode). +* QObject Input Visitor unit-tests (strict mode). * * Copyright (C) 2011-2012, 2015 Red Hat Inc. *
@@ -15,7 +15,7 @@#include "qemu-common.h" #include "qapi/error.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "test-qapi-types.h" #include "test-qapi-visit.h" #include "qapi/qmp/types.h"
@@ -53,7 +53,7 @@ static Visitor *validate_test_init_internal(TestInputVisitorData *data, data->obj =qobject_from_jsonv (json_string, ap);g_assert (data->obj); - data->qiv =qmp_input_visitor_new (data->obj, true); + data->qiv =qobject_input_visitor_new (data->obj, true);g_assert (data->qiv);return data->qiv; }
tests/test-qobject-input-visitor.c renamed from tests/test-qmp-input-visitor.c +3 -3
@@ -1,5 +1,5 @@/* -* QMP Input Visitor unit-tests. +* QObject Input Visitor unit-tests. * * Copyright (C) 2011-2016 Red Hat Inc. *
@@ -14,7 +14,7 @@#include "qemu-common.h" #include "qapi/error.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "test-qapi-types.h" #include "test-qapi-visit.h" #include "qapi/qmp/types.h"
@@ -49,7 +49,7 @@ static Visitor *visitor_input_test_init_internal(TestInputVisitorData *data, data->obj =qobject_from_jsonv (json_string, ap);g_assert (data->obj); - data->qiv =qmp_input_visitor_new (data->obj, false); + data->qiv =qobject_input_visitor_new (data->obj, false);g_assert (data->qiv);return data->qiv; }
tests/test-string-input-visitor.c +1 -1
@@ -4,7 +4,7 @@* Copyright (C) 2012 Red Hat Inc. * * Authors: -* Paolo Bonzini <[email protected]> (based on test-qmp-input-visitor) +* Paolo Bonzini <[email protected]> (based on test-qobject-input-visitor) * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory.
tests/test-visitor-serialization.c +2 -2
@@ -20,7 +20,7 @@#include "qapi/error.h" #include "qapi/qmp/types.h" #include "qapi/qmp/qjson.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qmp-output-visitor.h" #include "qapi/string-input-visitor.h" #include "qapi/string-output-visitor.h"
@@ -1040,7 +1040,7 @@ static void qmp_deserialize(void **native_out, void *datap, obj =qobject_from_json (qstring_get_str (output_json));QDECREF (output_json); - d->qiv =qmp_input_visitor_new (obj, true); + d->qiv =qobject_input_visitor_new (obj, true);qobject_decref (obj_orig);qobject_decref (obj);visit (d->qiv, native_out, errp);
util/qemu-sockets.c +1 -1
@@ -21,7 +21,7 @@#include "qapi/error.h" #include "qemu/sockets.h" #include "qemu/main-loop.h" -#include "qapi/qmp-input-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qmp-output-visitor.h" #include "qapi-visit.h" #include "qemu/cutils.h"
Signature · 1 line
2.7.4
PATCH v13 5/6 qapi: add a QObjectInputVisitor that does string conversion
In reply to Daniel P. BerrangeCurrently the QObjectInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.
This adds an alternative constructor for QObjectInputVisitor that will set it up such that it expects a QString for all scalar types instead.
This makes it possible to use QObjectInputVisitor with a QDict produced from QemuOpts, where everything is in string format.
--- include/qapi/qobject-input-visitor.h | 32 +++++- qapi/qobject-input-visitor.c | 128 ++++++++++++++++++++++ tests/test-qobject-input-visitor.c | 203 ++++++++++++++++++++++++++++++++++- 3 files changed, 355 insertions(+), 8 deletions(-)
include/qapi/qobject-input-visitor.h +28 -4
@@ -19,12 +19,36 @@typedef struct QObjectInputVisitor QObjectInputVisitor ; -/* -* Return a new input visitor that converts a QObject to a QAPI object. +/** +* Create a new input visitor that converts @obj to a QAPI object. +* +* Any scalar values in the @obj input data structure should be in the +* required type already. i.e. if visiting a bool, the value should +* already be a QBool instance. * -* Set @strict to reject a parse that doesn't consume all keys of a -* dictionary; otherwise excess input is ignored. +* If @strict is set to true, then an error will be reported if any +* dict keys are not consumed during visitation. If @strict is false +* then extra dict keys are silently ignored. +* +* The returned input visitor should be released by calling +* visit_free() when no longer required. */ Visitor *qobject_input_visitor_new (QObject *obj,bool strict); +/** +* Create a new input visitor that converts a QObject to a QAPI object. +* +* Any scalar values in the @obj input data structure should always be +* represented as strings. i.e. if visiting a boolean, the value should +* be a QString whose contents represent a valid boolean. +* +* The visitor always operates in strict mode, requiring all dict keys +* to be consumed during visitation. An error will be reported if this +* does not happen. +* +* The returned input visitor should be released by calling +* visit_free() when no longer required. +*/ +Visitor *qobject_string_input_visitor_new (QObject *obj); +#endif
qapi/qobject-input-visitor.c +128 -0
167 lines not on this page · raw message
tests/test-qobject-input-visitor.c +199 -4
265 lines not on this page · raw message
Signature · 1 line
2.7.4
PATCH v13 4/6 qapi: rename QmpOutputVisitor to QObjectOutputVisitor
In reply to Daniel P. BerrangeThe QmpOutputVisitor has no direct dependency on QMP. It is valid to use it anywhere that one wants a QObject. Rename it to better reflect its functionality as a generic QAPI to QObject converter.
---
block/qapi.c | 4 +-
blockdev.c | 4 +-
docs/qapi-code-gen.txt | 2 +-
...p-output-visitor.h => qobject-output-visitor.h} | 10 +-
qapi/Makefile.objs | 2 +-
qapi/qapi-clone-visitor.c | 2 +-
qapi/qmp-output-visitor.c | 256 ---------------------
qapi/qobject-output-visitor.c | 254 ++++++++++++++++++++
qemu-img.c | 8 +-
qom/object_interfaces.c | 2 +-
qom/qom-qobject.c | 4 +-
scripts/qapi-commands.py | 4 +-
scripts/qapi-event.py | 4 +-
tests/.gitignore | 2 +-
tests/Makefile.include | 8 +-
tests/check-qnull.c | 4 +-
...put-visitor.c => test-qobject-output-visitor.c} | 6 +-
tests/test-string-output-visitor.c | 2 +-
tests/test-visitor-serialization.c | 4 +-
util/qemu-sockets.c | 2 +-
20 files changed, 291 insertions(+), 293 deletions(-)
rename include/qapi/{qmp-output-visitor.h => qobject-output-visitor.h} (66%)
delete mode 100644 qapi/qmp-output-visitor.c
create mode 100644 qapi/qobject-output-visitor.c
rename tests/{test-qmp-output-visitor.c => test-qobject-output-visitor.c} (99%)block/qapi.c +2 -2
@@ -29,7 +29,7 @@#include "block/write-threshold.h" #include "qmp-commands.h" #include "qapi-visit.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/qmp/types.h" #include "sysemu/block-backend.h" #include "qemu/cutils.h"
@@ -691,7 +691,7 @@ void bdrv_image_info_specific_dump(fprintf_function func_fprintf, void *f, ImageInfoSpecific *info_spec) {QObject *obj, *data; -Visitor *v =qmp_output_visitor_new (&obj); +Visitor *v =qobject_output_visitor_new (&obj);visit_type_ImageInfoSpecific (v,NULL , &info_spec, &error_abort);visit_complete (v, &obj);
blockdev.c +2 -2
@@ -43,7 +43,7 @@#include "qapi/qmp/types.h" #include "qapi-visit.h" #include "qapi/qmp/qerror.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/util.h" #include "sysemu/sysemu.h" #include "block/block_int.h"
@@ -3783,7 +3783,7 @@ void qmp_blockdev_add(BlockdevOptions *options, Error **errp)BlockDriverState *bs;BlockBackend *blk =NULL ;QObject *obj; -Visitor *v =qmp_output_visitor_new (&obj); +Visitor *v =qobject_output_visitor_new (&obj);QDict *qdict;Error *local_err =NULL ;
docs/qapi-code-gen.txt +1 -1
@@ -1005,7 +1005,7 @@ Example: Error *err = NULL; Visitor *v; - v = qmp_output_visitor_new(ret_out); + v = qobject_output_visitor_new(ret_out); visit_type_UserDefOne(v, "unused", &ret_in, &err); if (!err) { visit_complete(v, ret_out);
include/qapi/qobject-output-visitor.h renamed from include/qapi/qmp-output-visitor.h +5 -5
@@ -11,20 +11,20 @@* */ -#ifndef QMP_OUTPUT_VISITOR_H -#define QMP_OUTPUT_VISITOR_H +#ifndef QOBJECT_OUTPUT_VISITOR_H +#define QOBJECT_OUTPUT_VISITOR_H #include "qapi/visitor.h" #include "qapi/qmp/qobject.h" -typedef struct QmpOutputVisitor QmpOutputVisitor ; +typedef struct QObjectOutputVisitor QObjectOutputVisitor ;/* -* Create a new QMP output visitor. +* Create a new QOBJECT output visitor. * * If everything else succeeds, pass @result to visit_complete() to * collect the result of the visit. */ -Visitor *qmp_output_visitor_new (QObject **result); +Visitor *qobject_output_visitor_new (QObject **result);#endif
qapi/Makefile.objs +1 -1
@@ -1,5 +1,5 @@ util-obj-y = qapi-visit-core.o qapi-dealloc-visitor.o qobject-input-visitor.o -util-obj-y += qmp-output-visitor.o qmp-registry.o qmp-dispatch.o +util-obj-y += qobject-output-visitor.o qmp-registry.o qmp-dispatch.o util-obj-y += string-input-visitor.o string-output-visitor.o util-obj-y += opts-visitor.o qapi-clone-visitor.o util-obj-y += qmp-event.o
qapi/qapi-clone-visitor.c +1 -1
@@ -110,7 +110,7 @@ static void qapi_clone_type_str(Visitor *v, const char *name, char **obj,assert (qcv->depth);/* * Pointer was already cloned by g_memdup; create fresh copy. -* Note that as long as qmp-output-visitor accepts NULL instead of +* Note that as long as qobject-output-visitor accepts NULL instead of * "", then we must do likewise. However, we want to obey the * input visitor semantics of never producing NULL when the empty * string is intended.
qapi/qmp-output-visitor.c deleted +0 -256
256 lines not on this page · raw message
qapi/qobject-output-visitor.c new file +254 -0
254 lines not on this page · raw message
qemu-img.c +4 -4
@@ -25,7 +25,7 @@#include "qemu-version.h" #include "qapi/error.h" #include "qapi-visit.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/qmp/qerror.h" #include "qapi/qmp/qjson.h" #include "qemu/cutils.h"
@@ -492,7 +492,7 @@ static void dump_json_image_check(ImageCheck *check, bool quiet) {QString *str;QObject *obj; -Visitor *v =qmp_output_visitor_new (&obj); +Visitor *v =qobject_output_visitor_new (&obj);visit_type_ImageCheck (v,NULL , &check, &error_abort);visit_complete (v, &obj);
@@ -2185,7 +2185,7 @@ static void dump_json_image_info_list(ImageInfoList *list) {QString *str;QObject *obj; -Visitor *v =qmp_output_visitor_new (&obj); +Visitor *v =qobject_output_visitor_new (&obj);visit_type_ImageInfoList (v,NULL , &list, &error_abort);visit_complete (v, &obj);
@@ -2201,7 +2201,7 @@ static void dump_json_image_info(ImageInfo *info) {QString *str;QObject *obj; -Visitor *v =qmp_output_visitor_new (&obj); +Visitor *v =qobject_output_visitor_new (&obj);visit_type_ImageInfo (v,NULL , &info, &error_abort);visit_complete (v, &obj);
qom/object_interfaces.c +1 -1
@@ -3,7 +3,7 @@#include "qom/object_interfaces.h" #include "qemu/module.h" #include "qapi-visit.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/opts-visitor.h" void user_creatable_complete (Object *obj,Error **errp)
qom/qom-qobject.c +2 -2
@@ -16,7 +16,7 @@#include "qom/qom-qobject.h" #include "qapi/visitor.h" #include "qapi/qobject-input-visitor.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" void object_property_set_qobject (Object *obj,QObject *value,const char *name,Error **errp)
@@ -35,7 +35,7 @@ QObject *object_property_get_qobject(Object *obj, const char *name,Error *local_err =NULL ;Visitor *v; - v =qmp_output_visitor_new (&ret); + v =qobject_output_visitor_new (&ret);object_property_get (obj, v, name, &local_err);if (!local_err) {visit_complete (v, &ret);
scripts/qapi-commands.py +2 -2
@@ -68,7 +68,7 @@ static void qmp_marshal_output_%(c_name)s(%(c_type)s ret_in, QObject **ret_out, Error *err = NULL; Visitor *v; - v = qmp_output_visitor_new(ret_out); + v = qobject_output_visitor_new(ret_out); visit_type_%(c_name)s(v, "unused", &ret_in, &err); if (!err) { visit_complete(v, ret_out);
@@ -278,7 +278,7 @@ fdef.write(mcgen(''' #include "qapi/qmp/types.h" #include "qapi/qmp/dispatch.h" #include "qapi/visitor.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/qobject-input-visitor.h" #include "qapi/dealloc-visitor.h" #include "%(prefix)sqapi-types.h"
scripts/qapi-event.py +2 -2
@@ -98,7 +98,7 @@ def gen_event_send(name, arg_type, boxed): if arg_type and not arg_type.is_empty(): ret += mcgen(''' - v = qmp_output_visitor_new(&obj); + v = qobject_output_visitor_new(&obj); ''') if not arg_type.is_implicit(): ret += mcgen('''
@@ -209,7 +209,7 @@ fdef.write(mcgen(''' #include "qemu-common.h" #include "%(prefix)sqapi-event.h" #include "%(prefix)sqapi-visit.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/qmp-event.h" ''',
tests/.gitignore +1 -1
@@ -61,7 +61,7 @@ test-qobject-input-strict test-qobject-input-visitor test-qmp-introspect.[ch] test-qmp-marshal.c -test-qmp-output-visitor +test-qobject-output-visitor test-rcu-list test-replication test-rfifolock
tests/Makefile.include +4 -4
@@ -20,8 +20,8 @@ check-unit-y += tests/check-qnull$(EXESUF) gcov-files-check-qnull-y = qobject/qnull.c check-unit-y += tests/check-qjson$(EXESUF) gcov-files-check-qjson-y = qobject/qjson.c -check-unit-y += tests/test-qmp-output-visitor$(EXESUF) -gcov-files-test-qmp-output-visitor-y = qapi/qmp-output-visitor.c +check-unit-y += tests/test-qobject-output-visitor$(EXESUF) +gcov-files-test-qobject-output-visitor-y = qapi/qobject-output-visitor.c check-unit-y += tests/test-clone-visitor$(EXESUF) gcov-files-test-clone-visitor-y = qapi/qapi-clone-visitor.c check-unit-y += tests/test-qobject-input-visitor$(EXESUF)
@@ -435,7 +435,7 @@ test-obj-y = tests/check-qint.o tests/check-qstring.o tests/check-qdict.o \ tests/check-qlist.o tests/check-qfloat.o tests/check-qnull.o \ tests/check-qjson.o \ tests/test-coroutine.o tests/test-string-output-visitor.o \ - tests/test-string-input-visitor.o tests/test-qmp-output-visitor.o \ + tests/test-string-input-visitor.o tests/test-qobject-output-visitor.o \ tests/test-clone-visitor.o \ tests/test-qobject-input-visitor.o tests/test-qobject-input-strict.o \ tests/test-qmp-commands.o tests/test-visitor-serialization.o \
@@ -538,7 +538,7 @@ $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-int tests/test-string-output-visitor$(EXESUF): tests/test-string-output-visitor.o $(test-qapi-obj-y) tests/test-string-input-visitor$(EXESUF): tests/test-string-input-visitor.o $(test-qapi-obj-y) tests/test-qmp-event$(EXESUF): tests/test-qmp-event.o $(test-qapi-obj-y) -tests/test-qmp-output-visitor$(EXESUF): tests/test-qmp-output-visitor.o $(test-qapi-obj-y) +tests/test-qobject-output-visitor$(EXESUF): tests/test-qobject-output-visitor.o $(test-qapi-obj-y) tests/test-clone-visitor$(EXESUF): tests/test-clone-visitor.o $(test-qapi-obj-y) tests/test-qobject-input-visitor$(EXESUF): tests/test-qobject-input-visitor.o $(test-qapi-obj-y) tests/test-qobject-input-strict$(EXESUF): tests/test-qobject-input-strict.o $(test-qapi-obj-y)
tests/check-qnull.c +2 -2
@@ -11,7 +11,7 @@#include "qapi/qmp/qobject.h" #include "qemu-common.h" #include "qapi/qobject-input-visitor.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/error.h" /*
@@ -52,7 +52,7 @@ static void qnull_visit_test(void)visit_type_null (v,NULL , &error_abort);visit_free (v); - v =qmp_output_visitor_new (&obj); + v =qobject_output_visitor_new (&obj);visit_type_null (v,NULL , &error_abort);visit_complete (v, &obj);g_assert (obj == &qnull_);
tests/test-qobject-output-visitor.c renamed from tests/test-qmp-output-visitor.c +3 -3
@@ -1,5 +1,5 @@/* -* QMP Output Visitor unit-tests. +* QObject Output Visitor unit-tests. * * Copyright (C) 2011-2016 Red Hat Inc. *
@@ -14,7 +14,7 @@#include "qemu-common.h" #include "qapi/error.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "test-qapi-types.h" #include "test-qapi-visit.h" #include "qapi/qmp/types.h"
@@ -28,7 +28,7 @@ typedef struct TestOutputVisitorData {static void visitor_output_setup (TestOutputVisitorData *data,const void *unused) { - data->ov =qmp_output_visitor_new (&data->obj); + data->ov =qobject_output_visitor_new (&data->obj);g_assert (data->ov); }
tests/test-string-output-visitor.c +1 -1
@@ -4,7 +4,7 @@* Copyright (C) 2012 Red Hat Inc. * * Authors: -* Paolo Bonzini <[email protected]> (based on test-qmp-output-visitor) +* Paolo Bonzini <[email protected]> (based on test-qobject-output-visitor) * * This work is licensed under the terms of the GNU GPL, version 2 or later. * See the COPYING file in the top-level directory.
tests/test-visitor-serialization.c +2 -2
@@ -21,7 +21,7 @@#include "qapi/qmp/types.h" #include "qapi/qmp/qjson.h" #include "qapi/qobject-input-visitor.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi/string-input-visitor.h" #include "qapi/string-output-visitor.h" #include "qapi-types.h"
@@ -1022,7 +1022,7 @@ static void qmp_serialize(void *native_in, void **datap, {QmpSerializeData *d =g_malloc0 (sizeof (*d)); - d->qov =qmp_output_visitor_new (&d->obj); + d->qov =qobject_output_visitor_new (&d->obj);visit (d->qov, &native_in, errp); *datap = d; }
util/qemu-sockets.c +1 -1
@@ -22,7 +22,7 @@#include "qemu/sockets.h" #include "qemu/main-loop.h" #include "qapi/qobject-input-visitor.h" -#include "qapi/qmp-output-visitor.h" +#include "qapi/qobject-output-visitor.h" #include "qapi-visit.h" #include "qemu/cutils.h"
Signature · 1 line
2.7.4
PATCH v13 6/6 qom: support arbitrary non-scalar properties with -object
In reply to Daniel P. BerrangeThe current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.
It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
The previously added qdict_crumple() method is able to take a qdict containing a flat set of properties and turn that into a arbitrarily nested set of dicts and lists. By combining qemu_opts_to_qdict and qdict_crumple() together, we can turn the opt string into a data structure that is practically identical to that passed over QMP when defining an object. The only difference is that all the scalar values are represented as strings, rather than strings, ints and bools. This is sufficient to let us replace the OptsVisitor with the QObjectInputVisitor for use with -object.
Thus -object can now support non-scalar properties, for example the QMP object
{
"execute": "object-add",
"arguments": {
"qom-type": "demo",
"id": "demo0",
"parameters": {
"foo": [
{ "bar": "one", "wizz": "1" },
{ "bar": "two", "wizz": "2" }
]
}
}
}Would be creatable via the CLI now using
$QEMU \
-object demo,id=demo0,\
foo.0.bar=one,foo.0.wizz=1,\
foo.1.bar=two,foo.1.wizz=2Notice that this syntax is intentionally compatible with that currently used by block drivers.
This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.
(hmp) object_add demo,id=demo0,\
foo.0.bar=one,foo.0.wizz=1,\
foo.1.bar=two,foo.1.wizz=2NB indentation should not be used with HMP commands, this is just for convenient formatting in this commit message.
--- hmp.c | 16 +- include/qom/object_interfaces.h | 10 +- qmp.c | 2 +- qom/object_interfaces.c | 45 ++++-- tests/check-qom-proplist.c | 334 +++++++++++++++++++++++++++++++++++++++- 5 files changed, 378 insertions(+), 29 deletions(-)
hmp.c +5 -11
@@ -25,7 +25,7 @@#include "qemu/sockets.h" #include "monitor/monitor.h" #include "monitor/qdev.h" -#include "qapi/opts-visitor.h" +#include "qapi/qobject-input-visitor.h" #include "qapi/qmp/qerror.h" #include "qapi/string-output-visitor.h" #include "qapi/util.h"
@@ -1695,20 +1695,14 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)void hmp_object_add (Monitor *mon,const QDict *qdict) {Error *err =NULL ; -QemuOpts *opts;Visitor *v;Object *obj =NULL ; +QDict *pdict =qdict_clone_shallow (qdict); - opts =qemu_opts_from_qdict (qemu_find_opts ("object" ), qdict, &err); -if (err) { -hmp_handle_error (mon, &err); -return ; - } - - v =opts_visitor_new (opts); - obj =user_creatable_add (qdict, v, &err); + v =qobject_string_input_visitor_new (QOBJECT (pdict)); + obj =user_creatable_add (pdict, v, &err);visit_free (v); -qemu_opts_del (opts); +QDECREF (pdict);if (err) {hmp_handle_error (mon, &err);
include/qom/object_interfaces.h +8 -2
@@ -96,18 +96,24 @@ Object *user_creatable_add(const QDict *qdict,* user_creatable_add_type: * @type: the object type name * @id: the unique ID for the object +* @nested: whether to recurse into the visitor for properties * @qdict: the object properties * @v: the visitor * @errp: if an error occurs, a pointer to an area to store the error * * Create an instance of the user creatable object @type, placing * it in the object composition tree with name @id, initializing -* it with properties from @qdict +* it with properties from @qdict. +* +* If the visitor is already positioned to read the properties +* in @qdict, @nested should be false. Conversely, if it is +* necessary to open/close a struct to read the properties in +* @qdict, @nested should be true. * * Returns: the newly created object or NULL on error */ Object *user_creatable_add_type (const char *type,const char *id, -const QDict *qdict, +bool nested,const QDict *qdict,Visitor *v,Error **errp);/**
qmp.c +1 -1
@@ -688,7 +688,7 @@ void qmp_object_add(const char *type, const char *id, } v =qobject_input_visitor_new (props, true); - obj =user_creatable_add_type (type, id, pdict, v, errp); + obj =user_creatable_add_type (type, id, true, pdict, v, errp);visit_free (v);if (obj) {object_unref (obj);
qom/object_interfaces.c +31 -14
90 lines not on this page · raw message
tests/check-qom-proplist.c +333 -1
426 lines not on this page · raw message
Signature · 1 line
2.7.4
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
Quoted 16 lines · Daniel P. Berrange
The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.
It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\
id=ram-node2,size=24578621440,policy=bind,\
host-nodes=1-2,host-nodes=5,host-nodes=7,So I'll need to figure out a way to preserve this syntax...
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
Quoted 26 lines · Daniel P. Berrange
Quoted 16 lines · Daniel P. Berrange
The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.
It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
Quoted 8 lines
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\ id=ram-node2,size=24578621440,policy=bind,\ host-nodes=1-2,host-nodes=5,host-nodes=7,So I'll need to figure out a way to preserve this syntax...
Is there a usecase for qdict_crumple without the following de-stringification pass? If not, qdict_crumple could use a StringInputVisitor on the values directly.
Paolo
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
Quoted 36 lines · Paolo Bonzini
Quoted 33 lines · Daniel P. Berrange
Quoted 16 lines · Daniel P. Berrange
The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.
It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\ id=ram-node2,size=24578621440,policy=bind,\ host-nodes=1-2,host-nodes=5,host-nodes=7,
Quoted 5 lines
Quoted 1 line
So I'll need to figure out a way to preserve this syntax...
Is there a usecase for
qdict_crumplewithout the following de-stringification pass? If not,qdict_crumplecould use a StringInputVisitor on the values directly.
I'm not sure I understand how that would help and in any case, the string-input-visitor itself is incapable of dealing with compound properties. It too could/should be ultimately replaced by the qobject-input-visitor combined with a pre-processing step to turn the input string into a qdict.
The problem I'm facing with the above scenario though occurs before we get anywhere near the visitors, or the crumple step.
When qemu_opts_to_qdict() turns QemuOpts into QDict, it looses repeated options. So given a command line
-object memory-backend-ram,\
id=ram-node2,size=24578621440,policy=bind,\
host-nodes=1-2,host-nodes=5,host-nodes=7,we're getting an initial QDict containing
type=memory-backend-ram
id=ram-node2
size=24578621440
policy=bind
host-nodes=7What I need to do is make qemu_opts_to_qdict more intelligent so that if it sees repeated strings, it inserts them into the qdict using list index style. eg, the QemuOpts -> QDict conversion should have at minimum given us
type=memory-backend-ram
id=ram-node2
size=24578621440
policy=bind
host-nodes.0=1-2
host-nodes.1=5
host-nodes.2=7although even then there's some magic in the range value seen there. I'm unclear on whether we should try to deal with the range "1-2" in the qemu_opts_to_qdict() method, the qdict_crumple() method or the qobject input visitor code for parsing ints.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
Quoted 51 lines · Daniel P. Berrange
Quoted 36 lines · Paolo Bonzini
Quoted 33 lines · Daniel P. Berrange
Quoted 16 lines · Daniel P. Berrange
The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.
This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.
It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\ id=ram-node2,size=24578621440,policy=bind,\ host-nodes=1-2,host-nodes=5,host-nodes=7,Quoted 5 lines
Quoted 1 line
So I'll need to figure out a way to preserve this syntax...
Is there a usecase for
qdict_crumplewithout the following de-stringification pass? If not,qdict_crumplecould use a StringInputVisitor on the values directly.I'm not sure I understand how that would help and in any case, the string-input-visitor itself is incapable of dealing with compound properties.
It too could/should be ultimately replaced by the qobject-input-visitor combined with a pre-processing step to turn the input string into a qdict.
Quoted 5 lines
The problem I'm facing with the above scenario though occurs before we get anywhere near the visitors, or the crumple step.
When
qemu_opts_to_qdict()turns QemuOpts into QDict, it looses repeated options.
Ouch, I thought the problem was only with "1-2", not with the repeated host-nodes. Wow, that does really work?!?
What might work is if you convert the repeated options to "host-nodes=1-2,5,7". Then StringInputVisitor is able to parse that (but OptsVisitor isn't).
I'm not sure if there are other uses of repeated options. Probably not based on a limited inspection of callers of qemu_opt_foreach.
Paolo
Quoted 21 lines
What I need to do is make
qemu_opts_to_qdictmore intelligent so that if it sees repeated strings, it inserts them into the qdict using list index style. eg, the QemuOpts -> QDict conversion should have at minimum given ustype=memory-backend-ram id=ram-node2 size=24578621440 policy=bind host-nodes.0=1-2 host-nodes.1=5 host-nodes.2=7although even then there's some magic in the range value seen there. I'm unclear on whether we should try to deal with the range "1-2" in the
qemu_opts_to_qdict()method, theqdict_crumple()method or the qobject input visitor code for parsing ints.Regards,
Daniel
Re: [Qemu-devel] [PATCH v13 2/6] option: make parse_option_bool/number non-static
In reply to Daniel P. BerrangeAm 19.09.2016 um 13:58 hat Daniel P. Berrange geschrieben:
Quoted 38 lines
The
opts-visitor.copts_type_bool()method has code for parsing a string to set a bool value, as does theqemu-option.cparse_option_bool()method, except it handles fewer cases.To enable consistency across the codebase, extend
parse_option_bool()to handle "yes", "no", "y" and "n", and make it non-static. Convert the opts visitor to call this method directly.Also make
parse_option_number()non-static to allow for similar reuse later.Reviewed-by Kevin Wolf <[email protected]>Reviewed-by Marc-André Lureau <[email protected]>Reviewed-by Eric Blake <[email protected]>Reviewed-by Markus Armbruster <[email protected]>Signed-off-by Daniel P. Berrange <[email protected]>util/qemu-option.c +5
-6@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,return get_next_param_value (buf, buf_size, tag, &str); } -static void parse_option_bool (const char *name,const char *value,bool *ret, -Error **errp) +void parse_option_bool (const char *name,const char *value,bool *ret, +Error **errp) {if (value !=NULL ) { -if (!strcmp (value,"on" )) { - *ret =1 ; - }else if (!strcmp (value,"off" )) { - *ret =0 ; +if (strcmp (value,"on" ) ==0 || +strcmp (value,"yes" ) ==0 || +strcmp (value,"y" ) ==0 ) {
util/qemu-option.c int get_param_value(char *buf, int buf_size, · line 142
Quoted 10 lines
util/qemu-option.c +7
-2+ *ret = true; + }else if (strcmp (value,"off" ) ==0 || +strcmp (value,"no" ) ==0 || +strcmp (value,"n" ) ==0 ) { + *ret = false; }else { -error_setg (errp,QERR_INVALID_PARAMETER_VALUE , - name,"'on' or 'off'" ); +error_setg (errp,QERR_INVALID_PARAMETER_VALUE , name, +"on|yes|y|off|no|n" );
This change requires an update to the reference output of some qemu-iotests (at least 051 and 137).
Kevin
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
In reply to Daniel P. BerrangeAm 19.09.2016 um 14:12 hat Daniel P. Berrange geschrieben:
Quoted 10 lines · Daniel P. Berrange
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
Quoted 8 lines
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\ id=ram-node2,size=24578621440,policy=bind,\ host-nodes=1-2,host-nodes=5,host-nodes=7,So I'll need to figure out a way to preserve this syntax...
If this turns out rather hard, would it make sense to merge just patches 1 to 5 for now to enable -blockdev, which doesn't need this syntax? Then you can add the magic list syntax as a follow-up together with the -object work.
Kevin
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
Quoted 18 lines · Kevin Wolf
Am 19.09.2016 um 14:12 hat Daniel P. Berrange geschrieben:
Quoted 17 lines · Daniel P. Berrange
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\ id=ram-node2,size=24578621440,policy=bind,\ host-nodes=1-2,host-nodes=5,host-nodes=7,
Quoted 6 lines
Quoted 1 line
So I'll need to figure out a way to preserve this syntax...
If this turns out rather hard, would it make sense to merge just patches 1 to 5 for now to enable -blockdev, which doesn't need this syntax? Then you can add the magic list syntax as a follow-up together with the -object work.
I've got code to deal with it and will be sending a new series, probably on monday once i've done more testing.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Re: [Qemu-devel] [PATCH v13 2/6] option: make parse_option_bool/number non-static
In reply to Kevin WolfQuoted 44 lines · Kevin Wolf
Am 19.09.2016 um 13:58 hat Daniel P. Berrange geschrieben:
Quoted 43 lines
The
opts-visitor.copts_type_bool()method has code for parsing a string to set a bool value, as does theqemu-option.cparse_option_bool()method, except it handles fewer cases.To enable consistency across the codebase, extend
parse_option_bool()to handle "yes", "no", "y" and "n", and make it non-static. Convert the opts visitor to call this method directly.Also make
parse_option_number()non-static to allow for similar reuse later.Reviewed-by Kevin Wolf <[email protected]>Reviewed-by Marc-André Lureau <[email protected]>Reviewed-by Eric Blake <[email protected]>Reviewed-by Markus Armbruster <[email protected]>Signed-off-by Daniel P. Berrange <[email protected]>util/qemu-option.c +10
-6@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,return get_next_param_value (buf, buf_size, tag, &str); } -static void parse_option_bool (const char *name,const char *value,bool *ret, -Error **errp) +void parse_option_bool (const char *name,const char *value,bool *ret, +Error **errp) {if (value !=NULL ) { -if (!strcmp (value,"on" )) { - *ret =1 ; - }else if (!strcmp (value,"off" )) { - *ret =0 ; +if (strcmp (value,"on" ) ==0 || +strcmp (value,"yes" ) ==0 || +strcmp (value,"y" ) ==0 ) { + *ret = true; + }else if (strcmp (value,"off" ) ==0 || +strcmp (value,"no" ) ==0 || +strcmp (value,"n" ) ==0 ) { + *ret = false;
Quoted 8 lines
util/qemu-option.c int get_param_value(char *buf, int buf_size, · line 142
Quoted 5 lines
util/qemu-option.c +2
-2}else { -error_setg (errp,QERR_INVALID_PARAMETER_VALUE , - name,"'on' or 'off'" ); +error_setg (errp,QERR_INVALID_PARAMETER_VALUE , name, +"on|yes|y|off|no|n" );This change requires an update to the reference output of some qemu-iotests (at least 051 and 137).
Opps, yes, so it does.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
Re: [Qemu-devel] [PATCH v13 6/6] qom: support arbitrary non-scalar properties with -object
In reply to Daniel P. BerrangeAm 23.09.2016 um 13:07 hat Daniel P. Berrange geschrieben:
Quoted 21 lines · Kevin Wolf
Quoted 20 lines
Am 19.09.2016 um 14:12 hat Daniel P. Berrange geschrieben:
Quoted 18 lines · Daniel P. Berrange
Quoted 6 lines
Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.
Urgh, I've just discovered that this is not in fact true.
The 'memory-backend' object type uses uint16List which has the hacky list syntax
-object memory-backend-ram,\ id=ram-node2,size=24578621440,policy=bind,\ host-nodes=1-2,host-nodes=5,host-nodes=7,So I'll need to figure out a way to preserve this syntax...
Quoted 7 lines
Quoted 4 lines
If this turns out rather hard, would it make sense to merge just patches 1 to 5 for now to enable -blockdev, which doesn't need this syntax? Then you can add the magic list syntax as a follow-up together with the -object work.
I've got code to deal with it and will be sending a new series, probably on monday once i've done more testing.
Even better then. :-)
Kevin
Re: [Qemu-devel] [PATCH v13 2/6] option: make parse_option_bool/number non-static
In reply to Daniel P. BerrangeQuoted 9 lines · Daniel P. Berrange
Quoted 7 lines
util/qemu-option.c int get_param_value(char *buf, int buf_size, · line 142
Quoted 4 lines
util/qemu-option.c +2
-2-error_setg (errp,QERR_INVALID_PARAMETER_VALUE , - name,"'on' or 'off'" ); +error_setg (errp,QERR_INVALID_PARAMETER_VALUE , name, +"on|yes|y|off|no|n" );This change requires an update to the reference output of some qemu-iotests (at least 051 and 137).
Opps, yes, so it does.
Do any of the docker tests or buildbots catch this automatically? (I often forget to run qemu-iotests, since they are not part of 'make check')
Signature · 2 lines
Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
Re: [Qemu-devel] [PATCH v13 2/6] option: make parse_option_bool/number non-static
Quoted 7 lines · Eric Blake
Quoted 5 lines · Daniel P. Berrange
Quoted 5 lines
Quoted 4 lines
util/qemu-option.c +2
-2-error_setg (errp,QERR_INVALID_PARAMETER_VALUE , - name,"'on' or 'off'" ); +error_setg (errp,QERR_INVALID_PARAMETER_VALUE , name, +"on|yes|y|off|no|n" );
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
This change requires an update to the reference output of some qemu-iotests (at least 051 and 137).
Opps, yes, so it does.
Do any of the docker tests or buildbots catch this automatically? (I often forget to run qemu-iotests, since they are not part of 'make check')
FWIW, as mentioned here:
https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg04392.html
I've tried adding qemu-iotests to our travis config for raw, qcow2 and luks, but hit many non-deterministic failures in it. We'll have to get it stable under high load before we can have it run automatically unless we want high false-failure rate.
Regards,
Daniel
Signature · 4 lines
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|