Skip to contentSkip to conversation

Provide a QOM-based authorization API

PATCH Superseded Daniel P. Berrange 3 participants 12 messages Jul 14, 2016 – Jul 15, 2016

Subsystems (2)

v8 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v8 cover Provide a QOM-based authorization API

This is a followup of previously posted work in 2.6 cycle:

v1: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg04618.html
v2: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01454.html
v3: https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02498.html
v4: https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg01661.html
v5: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg00485.html
v6: https://lists.gnu.org/archive/html/qemu-devel/2016-06/msg03876.html
v7: https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg00919.html

Many years ago I was responsible for adding the 'qemu_acl' type and associated HMP commands. Looking back at it now, it is quite a poor facility with a couple of bad limitations. First, the responsibility for creating the ACLs was left with the QEMU network service (VNC server was only thing ever doing it). This meant you could not share ACLs across multiple services. Second, there was no way to populate ACLs on the command line, you had no choice but to use the HMP commands. Third, the API was hardcoded around the idea of an in-QEMU implementation, leaving no scope for plugging in alternative implementations backed by, for example, LDAP or PAM.

This series introduces a much better authorization API design to QEMU that addresses all these problems, and maintains back compatibility. It of course is based on the QOM framework, so that immediately gives us ability to create objects via the CLI, HMP or QMP. There is an abstract base clss "QAuthZ" which defines the basic API for QEMU network services to use, and a specific implementation "QAuthZ" simple which replicates the functionality of 'qemu_acl'. It is thus possible to add other impls, without changing any other part of QEMU in the future. Finally, the user is responsible for creating the ACL objects, so they can have one ACL associated with all their TLS enabled network services.

There was only one small problem with this, specifically the -object CLI arg and HMP 'object_add' command had no way to let the user specify non-scalar properties for objects. eg if an object had a property which is a list of structs, you are out of luck if you want to create it without using QMP.

Thus the first four patches do some work around QAPI / QOM to make it possible to specify non-scalar properties with the -object CLI arg and HMP 'object_add' command. See the respective patches for illustration of the syntax used. Some of Max's recent block patches also depend on the qdict_crumple method in patch 1.

The patches 5 and 6 introduce the new base class and specific implementation.

Patch 7 kills the old qemu_acl code, updating any existing callers of it to use the QAuthZSimple QOM class instead.

Previously there were further patches adding ACL support for chardevs, migration, nbd, etc. These will be posted later once this core code is merged, so they can flow via the respective maintainer's trees

Changed in v8:

- Rebase due to merge of Visitor API changes (Eric)

Changed in v7:

- Misc typos in API docs (Marc-André)
- Fix parsing of properties using type_size visitor (Marc-André)
- Mark based auth class as abstract (Marc-André)
- Fix QAPI version annotations to say 2.7 (Marc-André)

Changed in v6:

- Switch from while() to for() loop for iterating over dicts (Markus)
- Avoid redundant strdup (Markus)
- Rewrap comments at 70 chars (Markus)
- Change qdict_list_size() to qdict_is_list() (Markus)
- Misc docs changes (Markus)
- Change QmpInputVisitor so the code for handling the string types is separate from code using native scalar types (Paolo)
- Centralize code parsing bool strings (Markus)
- Centralize code parsing int strings (Markus)

Changed in v5:

- Resolved conflicts with Eric's visitor refactoring which made it stricter about struct begin/end calls
- Added support for ACLs to migration code now its TLS support is merged.
- Fixed typos in example in commit message

Changed in v4:

- Ensure examples use shell escaping for '*' (Eric)
- Add more tests for crumple impl (Eric)
- Raise error if sasl-acl/tls-acl are requested but sasl/tls auth are not enabled (Eric)
- Document return codes for auth check more clearly (Eric)
- Don't silently turn a glob match into a strcmp if fnmatch is not present (Eric)
- Other misc small typos/fixes (Eric)

Changed in v3:

- Created separate qdict_list_size method (Max)
- Added unit tests for case of empty dict (Max)
- Fix variable names to use underscore separator (Max)
- Fix potential free of uninitialized variables (Max)
- Use QObject APIs for casts, instead of C type casts (Max)

Changed in v2:

- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple

Daniel P. Berrange (7):
qdict: implement a qdict_crumple method for un-flattening a dict
option: make parse_option_bool/number non-static
qapi: add a QmpInputVisitor that does string conversion
qom: support arbitrary non-scalar properties with -object
util: add QAuthZ object as an authorization base class
util: add QAuthZSimple object type for a simple access control list
acl: delete existing ACL implementation

 MAINTAINERS                      |   7 +
 Makefile                         |   9 +-
 Makefile.objs                    |   2 +
 Makefile.target                  |   2 +
 crypto/tlssession.c              |  28 +++-
 hmp.c                            |  12 +-
 include/qapi/qmp-input-visitor.h |  42 +++++-
 include/qapi/qmp/qdict.h         |   1 +
 include/qemu/acl.h               |  66 --------
 include/qemu/authz-simple.h      | 115 ++++++++++++++
 include/qemu/authz.h             |  89 +++++++++++
 include/qemu/option.h            |   4 +
 include/qom/object_interfaces.h  |  10 +-
 monitor.c                        | 181 ++++++++++++++--------
 qapi-schema.json                 |   6 +-
 qapi/opts-visitor.c              |  19 +--
 qapi/qmp-input-visitor.c         | 106 +++++++++++++
 qapi/util.json                   |  47 ++++++
 qmp.c                            |   2 +-
 qobject/qdict.c                  | 283 +++++++++++++++++++++++++++++++++++
 qom/object_interfaces.c          |  45 ++++--
 tests/.gitignore                 |   1 +
 tests/Makefile.include           |   5 +-
 tests/check-qdict.c              | 241 ++++++++++++++++++++++++++++++
 tests/check-qom-proplist.c       | 314 ++++++++++++++++++++++++++++++++++++++-
 tests/test-authz-simple.c        | 172 +++++++++++++++++++++
 tests/test-crypto-tlssession.c   |  15 +-
 tests/test-io-channel-tls.c      |  16 +-
 tests/test-qmp-input-visitor.c   | 149 ++++++++++++++++++-
 ui/vnc-auth-sasl.c               |   2 +-
 ui/vnc-auth-sasl.h               |   4 +-
 ui/vnc.c                         |  11 +-
 util/Makefile.objs               |   4 +-
 util/acl.c                       | 179 ----------------------
 util/authz-simple.c              | 314 +++++++++++++++++++++++++++++++++++++++
 util/authz.c                     |  47 ++++++
 util/qemu-option.c               |  27 ++--
 37 files changed, 2180 insertions(+), 397 deletions(-)
 delete mode 100644 include/qemu/acl.h
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 include/qemu/authz.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 delete mode 100644 util/acl.c
 create mode 100644 util/authz-simple.c
 create mode 100644 util/authz.c
Signature · 1 line
2.7.4
#5 lore raw

PATCH v8 3/7 qapi: add a QmpInputVisitor that does string conversion

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This adds an alternative constructor for QmpInputVisitor that will set it up such that it expects a QString for all scalar types instead.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

---
 include/qapi/qmp-input-visitor.h |  42 +++++++++--
 qapi/qmp-input-visitor.c         | 106 ++++++++++++++++++++++++++++
 tests/test-qmp-input-visitor.c   | 149 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 290 insertions(+), 7 deletions(-)
include/qapi/qmp-input-visitor.h +38 -4
index f3ff5f3..33439b7 100644
@@ -19,12 +19,46 @@
 
 typedef struct QmpInputVisitor QmpInputVisitor;
 
-/*
- * Return a new input visitor that converts QMP to QAPI.
+/**
+ * qmp_input_visitor_new:
+ * @obj: the input object to visit
+ * @strict: whether to require that all input keys are consumed
+ *
+ * Create a new input visitor that converts QMP to QAPI.
+ *
+ * Any scalar values in the @obj input data structure should be in the
+ * required type already. ie if visiting a bool, the value should
+ * already be a QBool instance.
  *
- * Set @strict to reject a parse that doesn't consume all keys of a
- * dictionary; otherwise excess input is ignored.
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation.
+ *
+ * The returned input visitor should be released by calling
+ * qmp_input_visitor_cleanup when no longer required.
+ *
+ * Returns: a new input visitor
  */
 Visitor *qmp_input_visitor_new(QObject *obj, bool strict);
 
+/**
+ * qmp_string_input_visitor_new:
+ * @obj: the input object to visit
+ * @strict: whether to require that all input keys are consumed
+ *
+ * Create a new input visitor that converts QMP to QAPI.
+ *
+ * Any scalar values in the @obj input data structure should always be
+ * represented as strings. ie if visiting a boolean, the value should
+ * be a QString whose contents represent a valid boolean.
+ *
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation.
+ *
+ * The returned input visitor should be released by calling
+ * qmp_input_visitor_cleanup when no longer required.
+ *
+ * Returns: a new input visitor
+ */
+Visitor *qmp_string_input_visitor_new(QObject *obj, bool strict);
+
 #endif
qapi/qmp-input-visitor.c +106 -0

145 lines not on this page · raw message

tests/test-qmp-input-visitor.c +146 -3

205 lines not on this page · raw message

Signature · 1 line
2.7.4
#3 lore raw

PATCH v8 2/7 option: make parse_option_bool/number non-static

In reply to Daniel P. Berrange

The opts-visitor.c opts_type_bool() method has code for parsing a string to set a bool value, as does the qemu-option.c parse_option_bool() method, except it handles fewer cases.

To enable consistency across the codebase, extend parse_option_bool() to handle "yes", "no", "y" and "n", and make it non-static. Convert the opts visitor to call this method directly.

Also make parse_option_number() non-static to allow for similar reuse later.

---
 include/qemu/option.h |  4 ++++
 qapi/opts-visitor.c   | 19 +------------------
 util/qemu-option.c    | 27 ++++++++++++++++-----------
 3 files changed, 21 insertions(+), 29 deletions(-)
include/qemu/option.h +4 -0
index 1f9e3f9..2a5266f 100644
@@ -37,6 +37,10 @@ int get_param_value(char *buf, int buf_size,
                     const char *tag, const char *str);
 
 
+void parse_option_bool(const char *name, const char *value, bool *ret,
+                       Error **errp);
+void parse_option_number(const char *name, const char *value,
+                         uint64_t *ret, Error **errp);
 void parse_option_size(const char *name, const char *value,
                        uint64_t *ret, Error **errp);
 bool has_help_option(const char *param);
qapi/opts-visitor.c +1 -18
index 1048bbc..084f7cc 100644
@@ -334,7 +334,6 @@ opts_type_str(Visitor *v, const char *name, char **obj, Error **errp)
 }
 
 
-/* mimics qemu-option.c::parse_option_bool() */
 static void
 opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
 {
@@ -346,23 +345,7 @@ opts_type_bool(Visitor *v, const char *name, bool *obj, Error **errp)
         return;
     }
 
-    if (opt->str) {
-        if (strcmp(opt->str, "on") == 0 ||
-            strcmp(opt->str, "yes") == 0 ||
-            strcmp(opt->str, "y") == 0) {
-            *obj = true;
-        } else if (strcmp(opt->str, "off") == 0 ||
-            strcmp(opt->str, "no") == 0 ||
-            strcmp(opt->str, "n") == 0) {
-            *obj = false;
-        } else {
-            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, opt->name,
-                       "on|yes|y|off|no|n");
-            return;
-        }
-    } else {
-        *obj = true;
-    }
+    parse_option_bool(opt->name, opt->str, obj, errp);
 
     processed(ov, name);
 }
util/qemu-option.c +16 -11
index 3467dc2..6fc9ccb 100644
@@ -125,25 +125,30 @@ int get_param_value(char *buf, int buf_size,
     return get_next_param_value(buf, buf_size, tag, &str);
 }
 
-static void parse_option_bool(const char *name, const char *value, bool *ret,
-                              Error **errp)
+void parse_option_bool(const char *name, const char *value, bool *ret,
+                       Error **errp)
 {
     if (value != NULL) {
-        if (!strcmp(value, "on")) {
-            *ret = 1;
-        } else if (!strcmp(value, "off")) {
-            *ret = 0;
+        if (strcmp(value, "on") == 0 ||
+            strcmp(value, "yes") == 0 ||
+            strcmp(value, "y") == 0) {
+            *ret = true;
+        } else if (strcmp(value, "off") == 0 ||
+            strcmp(value, "no") == 0 ||
+            strcmp(value, "n") == 0) {
+            *ret = false;
         } else {
-            error_setg(errp, QERR_INVALID_PARAMETER_VALUE,
-                       name, "'on' or 'off'");
+            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+                       "on|yes|y|off|no|n");
+            return;
         }
     } else {
-        *ret = 1;
+        *ret = true;
     }
 }
 
-static void parse_option_number(const char *name, const char *value,
-                                uint64_t *ret, Error **errp)
+void parse_option_number(const char *name, const char *value,
+                         uint64_t *ret, Error **errp)
 {
     char *postfix;
     uint64_t number;
Signature · 1 line
2.7.4
#1 lore raw

PATCH v8 1/7 qdict: implement a qdict_crumple method for un-flattening a dict

In reply to Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

{
'foo': [

     { 'bar': 'one', 'wizz': '1' },
     { 'bar': 'two', 'wizz': '2' }

],
}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nesting used when the same object is defined over QMP.

---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 283 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 241 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 525 insertions(+)
include/qapi/qmp/qdict.h +1 -0
index 71b8eb0..8a3ac13 100644
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
qobject/qdict.c +283 -0

295 lines not on this page · raw message

tests/check-qdict.c +241 -0

259 lines not on this page · raw message

Signature · 1 line
2.7.4
#8 lore raw

PATCH v8 4/7 qom: support arbitrary non-scalar properties with -object

In reply to Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.

The previously added qdict_crumple() method is able to take a qdict containing a flat set of properties and turn that into a arbitrarily nested set of dicts and lists. By combining qemu_opts_to_qdict and qdict_crumple() together, we can turn the opt string into a data structure that is practically identical to that passed over QMP when defining an object. The only difference is that all the scalar values are represented as strings, rather than strings, ints and bools. This is sufficient to let us replace the OptsVisitor with the QMPInputVisitor for use with -object.

Thus -object can now support non-scalar properties, for example the QMP object

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
          { "bar": "one", "wizz": "1" },
          { "bar": "two", "wizz": "2" }
        ]
      }
    }
  }

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

Notice that this syntax is intentionally compatible with that currently used by block drivers.

This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.

  (hmp) object_add demo,id=demo0,\
                   foo.0.bar=one,foo.0.wizz=1,\
                   foo.1.bar=two,foo.1.wizz=2

NB indentation should not be used with HMP commands, this is just for convenient formatting in this commit message.

---
 hmp.c                           |  12 +-
 include/qom/object_interfaces.h |  10 +-
 qmp.c                           |   2 +-
 qom/object_interfaces.c         |  45 ++++--
 tests/check-qom-proplist.c      | 314 +++++++++++++++++++++++++++++++++++++++-
 5 files changed, 355 insertions(+), 28 deletions(-)
hmp.c +2 -10
index 3ca79c3..5d30e6f 100644
@@ -25,7 +25,7 @@
 #include "qemu/sockets.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi/util.h"
@@ -1721,20 +1721,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
 void hmp_object_add(Monitor *mon, const QDict *qdict)
 {
     Error *err = NULL;
-    QemuOpts *opts;
     Visitor *v;
     Object *obj = NULL;
 
-    opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
-    if (err) {
-        hmp_handle_error(mon, &err);
-        return;
-    }
-
-    v = opts_visitor_new(opts);
+    v = qmp_string_input_visitor_new((QObject *)qdict, true);
     obj = user_creatable_add(qdict, v, &err);
     visit_free(v);
-    qemu_opts_del(opts);
 
     if (err) {
         hmp_handle_error(mon, &err);
include/qom/object_interfaces.h +8 -2
index 8b17f4d..2466e53 100644
@@ -96,18 +96,24 @@ Object *user_creatable_add(const QDict *qdict,
  * user_creatable_add_type:
  * @type: the object type name
  * @id: the unique ID for the object
+ * @nested: whether to recurse into the visitor for properties
  * @qdict: the object properties
  * @v: the visitor
  * @errp: if an error occurs, a pointer to an area to store the error
  *
  * Create an instance of the user creatable object @type, placing
  * it in the object composition tree with name @id, initializing
- * it with properties from @qdict
+ * it with properties from @qdict.
+ *
+ * If the visitor is already positioned to read the properties
+ * in @qdict, @nested should be false. Conversely, if it is
+ * necessary to open/close a struct to read the properties in
+ * @qdict, @nested should be true.
  *
  * Returns: the newly created object or NULL on error
  */
 Object *user_creatable_add_type(const char *type, const char *id,
-                                const QDict *qdict,
+                                bool nested, const QDict *qdict,
                                 Visitor *v, Error **errp);
 
 /**
qmp.c +1 -1
index b6d531e..f310278 100644
@@ -667,7 +667,7 @@ void qmp_object_add(const char *type, const char *id,
     }
 
     v = qmp_input_visitor_new(props, true);
-    obj = user_creatable_add_type(type, id, pdict, v, errp);
+    obj = user_creatable_add_type(type, id, true, pdict, v, errp);
     visit_free(v);
     if (obj) {
         object_unref(obj);
qom/object_interfaces.c +31 -14

90 lines not on this page · raw message

tests/check-qom-proplist.c +313 -1

406 lines not on this page · raw message

Signature · 1 line
2.7.4
#9 lore raw

PATCH v8 5/7 util: add QAuthZ object as an authorization base class

In reply to Daniel P. Berrange

The current qemu_acl module provides a simple access control list facility inside QEMU, which is used via a set of monitor commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially integrate with external authorization systems like polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be subclassed for creating different authorization providers.

---
 MAINTAINERS          |  7 +++++
 Makefile             |  1 +
 Makefile.objs        |  2 ++
 Makefile.target      |  2 ++
 include/qemu/authz.h | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 util/Makefile.objs   |  2 ++
 util/authz.c         | 47 +++++++++++++++++++++++++++
 7 files changed, 150 insertions(+)
 create mode 100644 include/qemu/authz.h
 create mode 100644 util/authz.c
MAINTAINERS +7 -0
index 1d0e2c3..349af5f 100644
@@ -1325,6 +1325,13 @@ F: include/qemu/throttle.h
 F: util/throttle.c
 L: [email protected]
 
+Authorization
+M: Daniel P. Berrange <[email protected]>
+S: Maintained
+F: util/authz*
+F: include/qemu/authz*
+F: tests/test-authz-*
+
 Usermode Emulation
 ------------------
 Overall
Makefile +1 -0
index c054bc6..b3ac210 100644
@@ -148,6 +148,7 @@ endif
 dummy := $(call unnest-vars,, \
                 stub-obj-y \
                 util-obj-y \
+                util-qom-obj-y \
                 qga-obj-y \
                 ivshmem-client-obj-y \
                 ivshmem-server-obj-y \
Makefile.objs +2 -0
index 7f1f0a3..1d7bad7 100644
@@ -4,6 +4,8 @@ stub-obj-y = stubs/ crypto/
 util-obj-y = util/ qobject/ qapi/
 util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
 
+util-qom-obj-y += util/
+
 #######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img
 
Makefile.target +2 -0
index a440bcb..118c502 100644
@@ -173,6 +173,7 @@ include $(SRC_PATH)/Makefile.objs
 dummy := $(call unnest-vars,,target-obj-y)
 target-obj-y-save := $(target-obj-y)
 dummy := $(call unnest-vars,.., \
+               util-qom-obj-y \
                block-obj-y \
                block-obj-m \
                crypto-obj-y \
@@ -185,6 +186,7 @@ target-obj-y := $(target-obj-y-save)
 all-obj-y += $(common-obj-y)
 all-obj-y += $(target-obj-y)
 all-obj-y += $(qom-obj-y)
+all-obj-y += $(util-qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
include/qemu/authz.h new file +89 -0

89 lines not on this page · raw message

util/Makefile.objs +2 -0
index 96cb1e0..006ac63 100644
@@ -35,3 +35,5 @@ util-obj-y += log.o
 util-obj-y += qdist.o
 util-obj-y += qht.o
 util-obj-y += range.o
+
+util-qom-obj-y += authz.o
util/authz.c new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
2.7.4
#11 lore raw

PATCH v8 7/7 acl: delete existing ACL implementation

In reply to Daniel P. Berrange

The 'qemu_acl' type was a previous non-QOM based attempt to provide an authorization facility in QEMU. Because it is non-QOM based it cannot be created via the command line and requires special monitor commands to manipulate it.

The new QAuthZ and QAuthZSimple QOM classes provide a superset of the functionality in qemu_acl, so the latter can now be deleted. The HMP 'acl_*' monitor commands are converted to use the new QAuthZSimple data type instead in order to provide backwards compatibility, but their use is discouraged.

---
 Makefile                       |   6 +-
 crypto/tlssession.c            |  28 +++++--
 include/qemu/acl.h             |  66 ---------------
 monitor.c                      | 181 +++++++++++++++++++++++++++--------------
 tests/Makefile.include         |   2 +-
 tests/test-crypto-tlssession.c |  15 ++--
 tests/test-io-channel-tls.c    |  16 ++--
 ui/vnc-auth-sasl.c             |   2 +-
 ui/vnc-auth-sasl.h             |   4 +-
 ui/vnc.c                       |  11 ++-
 util/Makefile.objs             |   1 -
 util/acl.c                     | 179 ----------------------------------------
 12 files changed, 180 insertions(+), 331 deletions(-)
 delete mode 100644 include/qemu/acl.h
 delete mode 100644 util/acl.c
Makefile +3 -3
index 3a1f4c1..f90e1fc 100644
@@ -253,9 +253,9 @@ util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
 
 qemu-img.o: qemu-img-cmds.h
 
-qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
+qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
 
 qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o libqemuutil.a libqemustub.a
 
crypto/tlssession.c +23 -5
index 2de42c6..2edf5ab 100644
@@ -23,7 +23,7 @@
 #include "crypto/tlscredsanon.h"
 #include "crypto/tlscredsx509.h"
 #include "qapi/error.h"
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "trace.h"
 
 #ifdef CONFIG_GNUTLS
@@ -220,6 +220,7 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
     unsigned int nCerts, i;
     time_t now;
     gnutls_x509_crt_t cert = NULL;
+    Error *err = NULL;
 
     now = time(NULL);
     if (now == ((time_t)-1)) {
@@ -308,16 +309,33 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
                 goto error;
             }
             if (session->aclname) {
-                qemu_acl *acl = qemu_acl_find(session->aclname);
-                int allow;
-                if (!acl) {
+                QAuthZ *acl;
+                Object *obj;
+                Object *container;
+                bool allow;
+
+                container = object_get_objects_root();
+                obj = object_resolve_path_component(container,
+                                                    session->aclname);
+                if (!obj) {
                     error_setg(errp, "Cannot find ACL %s",
                                session->aclname);
                     goto error;
                 }
 
-                allow = qemu_acl_party_is_allowed(acl, session->peername);
+                if (!object_dynamic_cast(obj, TYPE_QAUTHZ)) {
+                    error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                               session->aclname);
+                    goto error;
+                }
 
+                acl = QAUTHZ(obj);
+
+                allow = qauthz_is_allowed(acl, session->peername, &err);
+                if (err) {
+                    error_propagate(errp, err);
+                    goto error;
+                }
                 if (!allow) {
                     error_setg(errp, "TLS x509 ACL check for %s is denied",
                                session->peername);
include/qemu/acl.h deleted +0 -66

66 lines not on this page · raw message

monitor.c +121 -60

240 lines not on this page · raw message

tests/Makefile.include +1 -1
index 38298db..9f43b3b 100644
@@ -424,7 +424,7 @@ test-qom-obj-y = $(qom-obj-y) $(test-util-obj-y)
 test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
 	tests/test-qapi-event.o tests/test-qmp-introspect.o \
 	$(test-qom-obj-y)
-test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
+test-crypto-obj-y = $(crypto-obj-y) $(util-qom-obj-y) $(test-qom-obj-y)
 test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
 test-block-obj-y = $(block-obj-y) $(test-io-obj-y)
 
tests/test-crypto-tlssession.c +10 -5
index 1a4a066..b7463e7 100644
@@ -26,7 +26,7 @@
 #include "qom/object_interfaces.h"
 #include "qapi/error.h"
 #include "qemu/sockets.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 
 #ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
 
@@ -111,7 +111,7 @@ static void test_crypto_tls_session(const void *opaque)
     QCryptoTLSCreds *serverCreds;
     QCryptoTLSSession *clientSess = NULL;
     QCryptoTLSSession *serverSess = NULL;
-    qemu_acl *acl;
+    QAuthZSimple *auth;
     const char * const *wildcards;
     int channel[2];
     bool clientShake = false;
@@ -170,11 +170,15 @@ static void test_crypto_tls_session(const void *opaque)
         &err);
     g_assert(serverCreds != NULL);
 
-    acl = qemu_acl_init("tlssessionacl");
-    qemu_acl_reset(acl);
+    auth = qauthz_simple_new("tlssessionacl",
+                             QAUTHZ_SIMPLE_POLICY_DENY,
+                             &error_abort);
     wildcards = data->wildcards;
     while (wildcards && *wildcards) {
-        qemu_acl_append(acl, 0, *wildcards);
+        qauthz_simple_append_rule(auth, *wildcards,
+                                  QAUTHZ_SIMPLE_POLICY_ALLOW,
+                                  QAUTHZ_SIMPLE_FORMAT_GLOB,
+                                  &error_abort);
         wildcards++;
     }
 
@@ -264,6 +268,7 @@ static void test_crypto_tls_session(const void *opaque)
 
     object_unparent(OBJECT(serverCreds));
     object_unparent(OBJECT(clientCreds));
+    object_unparent(OBJECT(auth));
 
     qcrypto_tls_session_free(serverSess);
     qcrypto_tls_session_free(clientSess);
tests/test-io-channel-tls.c +11 -5
index 3c361a7..ff472a0 100644
@@ -28,7 +28,7 @@
 #include "io/channel-socket.h"
 #include "io-channel-helpers.h"
 #include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 #include "qom/object_interfaces.h"
 
 #ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
@@ -115,7 +115,7 @@ static void test_io_channel_tls(const void *opaque)
     QIOChannelTLS *serverChanTLS;
     QIOChannelSocket *clientChanSock;
     QIOChannelSocket *serverChanSock;
-    qemu_acl *acl;
+    QAuthZSimple *auth;
     const char * const *wildcards;
     int channel[2];
     struct QIOChannelTLSHandshakeData clientHandshake = { false, false };
@@ -166,11 +166,15 @@ static void test_io_channel_tls(const void *opaque)
         &err);
     g_assert(serverCreds != NULL);
 
-    acl = qemu_acl_init("channeltlsacl");
-    qemu_acl_reset(acl);
+    auth = qauthz_simple_new("channeltlsacl",
+                             QAUTHZ_SIMPLE_POLICY_DENY,
+                             &error_abort);
     wildcards = data->wildcards;
     while (wildcards && *wildcards) {
-        qemu_acl_append(acl, 0, *wildcards);
+        qauthz_simple_append_rule(auth, *wildcards,
+                                  QAUTHZ_SIMPLE_POLICY_ALLOW,
+                                  QAUTHZ_SIMPLE_FORMAT_GLOB,
+                                  &error_abort);
         wildcards++;
     }
 
@@ -256,6 +260,8 @@ static void test_io_channel_tls(const void *opaque)
     object_unref(OBJECT(serverChanSock));
     object_unref(OBJECT(clientChanSock));
 
+    object_unparent(OBJECT(auth));
+
     close(channel[0]);
     close(channel[1]);
 }
ui/vnc-auth-sasl.c +1 -1
index 5ae29c1..b1064a7 100644
@@ -150,7 +150,7 @@ static int vnc_auth_sasl_check_access(VncState *vs)
         return 0;
     }
 
-    allow = qemu_acl_party_is_allowed(vs->vd->sasl.acl, vs->sasl.username);
+    allow = qauthz_is_allowed(vs->vd->sasl.acl, vs->sasl.username, NULL);
 
     VNC_DEBUG("SASL client %s %s by ACL\n", vs->sasl.username,
               allow ? "allowed" : "denied");
ui/vnc-auth-sasl.h +2 -2
index cb42745..12b099b 100644
@@ -30,7 +30,7 @@
 typedef struct VncStateSASL VncStateSASL;
 typedef struct VncDisplaySASL VncDisplaySASL;
 
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "qemu/main-loop.h"
 
 struct VncStateSASL {
@@ -59,7 +59,7 @@ struct VncStateSASL {
 };
 
 struct VncDisplaySASL {
-    qemu_acl *acl;
+    QAuthZ *acl;
 };
 
 void vnc_sasl_client_cleanup(VncState *vs);
ui/vnc.c +8 -3
index e3f857c..f1a234a 100644
@@ -33,7 +33,7 @@
 #include "qemu/error-report.h"
 #include "qemu/sockets.h"
 #include "qemu/timer.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 #include "qemu/config-file.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/types.h"
@@ -3744,7 +3744,9 @@ void vnc_display_open(const char *id, Error **errp)
         } else {
             vs->tlsaclname = g_strdup_printf("vnc.%s.x509dname", vs->id);
         }
-        qemu_acl_init(vs->tlsaclname);
+        qauthz_simple_new(vs->tlsaclname,
+                          QAUTHZ_SIMPLE_POLICY_DENY,
+                          &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
     if (acl && sasl) {
@@ -3755,7 +3757,10 @@ void vnc_display_open(const char *id, Error **errp)
         } else {
             aclname = g_strdup_printf("vnc.%s.username", vs->id);
         }
-        vs->sasl.acl = qemu_acl_init(aclname);
+        vs->sasl.acl =
+            QAUTHZ(qauthz_simple_new(aclname,
+                                     QAUTHZ_SIMPLE_POLICY_DENY,
+                                     &error_abort));
         g_free(aclname);
     }
 #endif
util/Makefile.objs +0 -1
index f71a7b6..08b5f91 100644
@@ -13,7 +13,6 @@ util-obj-y += envlist.o path.o module.o
 util-obj-$(call lnot,$(CONFIG_INT128)) += host-utils.o
 util-obj-y += bitmap.o bitops.o hbitmap.o
 util-obj-y += fifo8.o
-util-obj-y += acl.o
 util-obj-y += error.o qemu-error.o
 util-obj-y += id.o
 util-obj-y += iov.o qemu-config.o qemu-sockets.o uri.o notify.o
util/acl.c deleted +0 -179

179 lines not on this page · raw message

Signature · 1 line
2.7.4
#10 lore raw

PATCH v8 6/7 util: add QAuthZSimple object type for a simple access control list

In reply to Daniel P. Berrange

Add a QAuthZSimple object type that implements the QAuthZ interface. This simple built-in implementation maintains a trivial access control list with a sequence of match rules and a final default policy. This replicates the functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor, the syntax used would be

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "authz-simple",
      "id": "auth0",
      "parameters": {
        "rules": [
           { "match": "fred", "policy": "allow", "format": "exact" },
           { "match": "bob", "policy": "allow", "format": "exact" },
           { "match": "danb", "policy": "deny", "format": "glob" },
           { "match": "dan*", "policy": "allow", "format": "exact" },
        ],
        "policy": "deny"
      }
    }
  }

Or via the -object command line

  $QEMU \
     -object authz-simple,id=acl0,policy=deny,\
             rules.0.match=fred,rules.0.policy=allow,rules.0.format=exact,\
             rules.1.match=bob,rules.1.policy=allow,rules.1.format=exact,\
             rules.2.match=danb,rules.2.policy=deny,rules.2.format=glob,\
             rules.3.match=dan\*,rules.3.policy=allow,rules.3.format=exact

This sets up an authorization rule that allows 'fred', 'bob' and anyone whose name starts with 'dan', except for 'danb'. Everyone unmatched is denied.

---
 Makefile                    |   2 +-
 include/qemu/authz-simple.h | 115 ++++++++++++++++
 qapi-schema.json            |   6 +-
 qapi/util.json              |  47 +++++++
 tests/.gitignore            |   1 +
 tests/Makefile.include      |   3 +
 tests/test-authz-simple.c   | 172 ++++++++++++++++++++++++
 util/Makefile.objs          |   1 +
 util/authz-simple.c         | 314 ++++++++++++++++++++++++++++++++++++++++++++
 9 files changed, 659 insertions(+), 2 deletions(-)
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 create mode 100644 util/authz-simple.c
Makefile +1 -1
index b3ac210..3a1f4c1 100644
@@ -292,7 +292,7 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
                $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
                $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
                $(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
-               $(SRC_PATH)/qapi/trace.json
+               $(SRC_PATH)/qapi/trace.json $(SRC_PATH)/qapi/util.json
 
 qapi-types.c qapi-types.h :\
 $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
include/qemu/authz-simple.h new file +115 -0

115 lines not on this page · raw message

qapi-schema.json +5 -1
index d2d6506..814372c 100644
@@ -5,6 +5,9 @@
 # QAPI common definitions
 { 'include': 'qapi/common.json' }
 
+# QAPI util definitions
+{ 'include': 'qapi/util.json' }
+
 # QAPI crypto definitions
 { 'include': 'qapi/crypto.json' }
 
@@ -3740,7 +3743,8 @@
 # Since 2.5
 ##
 { 'struct': 'DummyForceArrays',
-  'data': { 'unused': ['X86CPUFeatureWordInfo'] } }
+  'data': { 'unused1': ['X86CPUFeatureWordInfo'],
+            'unused2': ['QAuthZSimpleRule'] } }
 
 
 ##
qapi/util.json new file +47 -0

47 lines not on this page · raw message

tests/.gitignore +1 -0
index dbb5263..f0fbdb7 100644
@@ -10,6 +10,7 @@ check-qom-proplist
 qht-bench
 rcutorture
 test-aio
+test-authz-simple
 test-base64
 test-bitops
 test-blockjob-txn
tests/Makefile.include +3 -0
index 2010b11..38298db 100644
@@ -111,6 +111,7 @@ check-unit-y += tests/test-crypto-xts$(EXESUF)
 check-unit-y += tests/test-crypto-block$(EXESUF)
 gcov-files-test-logging-y = tests/test-logging.c
 check-unit-y += tests/test-logging$(EXESUF)
+check-unit-y += tests/test-authz-simple$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -469,6 +470,8 @@ tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
 	$(test-util-obj-y)
 tests/test-base64$(EXESUF): tests/test-base64.o \
 	libqemuutil.a libqemustub.a
+tests/test-authz-simple$(EXESUF): tests/test-authz-simple.o \
+	libqemuutil.a libqemustub.a $(util-qom-obj-y) $(qom-obj-y)
 
 tests/test-logging$(EXESUF): tests/test-logging.o $(test-util-obj-y)
 
tests/test-authz-simple.c new file +172 -0

172 lines not on this page · raw message

util/Makefile.objs +1 -0
index 006ac63..f71a7b6 100644
@@ -37,3 +37,4 @@ util-obj-y += qht.o
 util-obj-y += range.o
 
 util-qom-obj-y += authz.o
+util-qom-obj-y += authz-simple.o
util/authz-simple.c new file +314 -0

314 lines not on this page · raw message

Signature · 1 line
2.7.4
#2 lore raw

Re: [Qemu-devel] [PATCH v8 1/7] qdict: implement a qdict_crumple method for un-flattening a dict

In reply to Daniel P. Berrange
Quoted 202 lines · Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

 {
   'foo.0.bar': 'one',
   'foo.0.wizz': '1',
   'foo.1.bar': 'two',
   'foo.1.wizz': '2'

}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

 {
   'foo': [
     { 'bar': 'one', 'wizz': '1' },
     { 'bar': 'two', 'wizz': '2' }
   ],

}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nesting used when the same object is defined over QMP.

---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 283 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 241 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 525 insertions(+)
include/qapi/qmp/qdict.h +1 -0
index 71b8eb0..8a3ac13 100644
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
qobject/qdict.c +109 -0
index 60f158c..fd2d0e3 100644
@@ -17,6 +17,7 @@
 #include "qapi/qmp/qbool.h"
 #include "qapi/qmp/qstring.h"
 #include "qapi/qmp/qobject.h"
+#include "qapi/error.h"
 #include "qemu/queue.h"
 #include "qemu-common.h"
 #include "qemu/cutils.h"
@@ -683,6 +684,288 @@ void qdict_array_split(QDict *src, QList **dst)
     }
 }
 
+
+/**
+ * qdict_split_flat_key:
+ * @key: the key string to split
+ * @prefix: non-NULL pointer to hold extracted prefix
+ * @suffix: non-NULL pointer to remaining suffix
+ *
+ * Given a flattened key such as 'foo.0.bar', split it into two parts
+ * at the first '.' separator. Allows double dot ('..') to escape the
+ * normal separator.
+ *
+ * eg
+ *    'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ *    'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned 'prefix'
+ * string. The 'suffix' string will be left in escaped format, so it
+ * can be fed back into the qdict_split_flat_key() key as the input
+ * later.
+ *
+ * The caller is responsible for freeing the string returned in @prefix
+ * using g_free().
+ */
+static void qdict_split_flat_key(const char *key, char **prefix,
+                                 const char **suffix)
+{
+    const char *separator;
+    size_t i, j;
+
+    /* Find first '.' separator, but if there is a pair '..'
+     * that acts as an escape, so skip over '..' */
+    separator = NULL;
+    do {
+        if (separator) {
+            separator += 2;
+        } else {
+            separator = key;
+        }
+        separator = strchr(separator, '.');
+    } while (separator && separator[1] == '.');
+
+    if (separator) {
+        *prefix = g_strndup(key,
+                            separator - key);
+        *suffix = separator + 1;
+    } else {
+        *prefix = g_strdup(key);
+        *suffix = NULL;
+    }
+
+    /* Unescape the '..' sequence into '.' */
+    for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+        if ((*prefix)[i] == '.') {
+            assert((*prefix)[i + 1] == '.');
+            i++;
+        }
+        (*prefix)[j] = (*prefix)[i];
+    }
+    (*prefix)[j] = '\0';
+}
+
+
+/**
+ * qdict_is_list:
+ * @maybe_list: dict to check if keys represent list elements.
+ *
+ * Determine whether all keys in @maybe_list are valid list elements.
+ * If @maybe_list is non-zero in length and all the keys look like
+ * valid list indexes, this will return 1. If @maybe_list is zero
+ * length or all keys are non-numeric then it will return 0 to indicate
+ * it is a normal qdict. If there is a mix of numeric and non-numeric
+ * keys, or the list indexes are non-contiguous, an error is reported.
+ *
+ * Returns: 1 if a valid list, 0 if a dict, -1 on error
+ */
+static int qdict_is_list(QDict *maybe_list, Error **errp)
+{
+    const QDictEntry *ent;
+    ssize_t len = 0;
+    ssize_t max = -1;
+    int is_list = -1;
+    int64_t val;
+
+    for (ent = qdict_first(maybe_list); ent != NULL;
+         ent = qdict_next(maybe_list, ent)) {
+
+        if (qemu_strtoll(ent->key, NULL, 10, &val) == 0) {
+            if (is_list == -1) {
+                is_list = 1;
+            } else if (!is_list) {
+                error_setg(errp,
+                           "Cannot crumple a dictionary with a mix of list "
+                           "and non-list keys");
+                return -1;
+            }
+            len++;
+            if (val > max) {
+                max = val;
+            }
+        } else {
+            if (is_list == -1) {
+                is_list = 0;
+            } else if (is_list) {
+                error_setg(errp,
+                           "Cannot crumple a dictionary with a mix of list "
+                           "and non-list keys");
+                return -1;
+            }

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 804

Quoted 10 lines
qobject/qdict.c +10 -0
+        }
+    }
+
+    if (is_list == -1) {
+        assert(!qdict_size(maybe_list));
+        is_list = 0;
+    }
+
+    if (len != (max + 1)) {
+        error_setg(errp, "List indexes are not contigous, "

contiguous

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 810

Quoted 6 lines
qobject/qdict.c +6 -0
+                   "saw %zd elements but %zd largest index",
+                   len, max);
+        return -1;
+    }
+
+    return is_list ? 1 : 0;

Why not simply return is_list?

Quoted 1 line
qobject/qdict.c +1 -0
+}

[...]

Reviewed-by on v8 2/7
#4 lore raw

Re: [Qemu-devel] [PATCH v8 2/7] option: make parse_option_bool/number non-static

In reply to Daniel P. Berrange
Quoted 9 lines · Daniel P. Berrange

The opts-visitor.c opts_type_bool() method has code for parsing a string to set a bool value, as does the qemu-option.c parse_option_bool() method, except it handles fewer cases.

To enable consistency across the codebase, extend parse_option_bool() to handle "yes", "no", "y" and "n", and make it non-static. Convert the opts visitor to call this method directly.

Baroque. But you're merely making things consistently baroque.

Quoted 6 lines

Also make parse_option_number() non-static to allow for similar reuse later.

#6 lore raw

Re: [Qemu-devel] [PATCH v8 3/7] qapi: add a QmpInputVisitor that does string conversion

In reply to Daniel P. Berrange
Quoted 27 lines · Daniel P. Berrange

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This adds an alternative constructor for QmpInputVisitor that will set it up such that it expects a QString for all scalar types instead.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

---
 include/qapi/qmp-input-visitor.h |  42 +++++++++--
 qapi/qmp-input-visitor.c         | 106 ++++++++++++++++++++++++++++
 tests/test-qmp-input-visitor.c   | 149 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 290 insertions(+), 7 deletions(-)
include/qapi/qmp-input-visitor.h +0 -0
index f3ff5f3..33439b7 100644
@@ -19,12 +19,46 @@
 

include/qapi/qmp-input-visitor.h line 27

Quoted 10 lines
include/qapi/qmp-input-visitor.h +6 -2
 typedef struct QmpInputVisitor QmpInputVisitor;
 
-/*
- * Return a new input visitor that converts QMP to QAPI.
+/**
+ * qmp_input_visitor_new:
+ * @obj: the input object to visit
+ * @strict: whether to require that all input keys are consumed
+ *
+ * Create a new input visitor that converts QMP to QAPI.

Actually, it converts a QObject to a QAPI object.

include/qapi/qmp-input-visitor.h line 30

Quoted 3 lines
include/qapi/qmp-input-visitor.h +3 -0
+ *
+ * Any scalar values in the @obj input data structure should be in the
+ * required type already. ie if visiting a bool, the value should

i.e.

Quoted 13 lines
include/qapi/qmp-input-visitor.h +8 -2
+ * already be a QBool instance.
  *
- * Set @strict to reject a parse that doesn't consume all keys of a
- * dictionary; otherwise excess input is ignored.
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation.
+ *
+ * The returned input visitor should be released by calling
+ * qmp_input_visitor_cleanup when no longer required.
+ *
+ * Returns: a new input visitor
  */
 Visitor *qmp_input_visitor_new(QObject *obj, bool strict);

include/qapi/qmp-input-visitor.h line 51

Quoted 10 lines
include/qapi/qmp-input-visitor.h +9 -0
 
+/**
+ * qmp_string_input_visitor_new:
+ * @obj: the input object to visit
+ * @strict: whether to require that all input keys are consumed
+ *
+ * Create a new input visitor that converts QMP to QAPI.
+ *
+ * Any scalar values in the @obj input data structure should always be
+ * represented as strings. ie if visiting a boolean, the value should

i.e.

include/qapi/qmp-input-visitor.h line 62

Quoted 11 lines
include/qapi/qmp-input-visitor.h +11 -0
+ * be a QString whose contents represent a valid boolean.
+ *
+ * If @strict is set to true, then an error will be reported if any
+ * dict keys are not consumed during visitation.
+ *
+ * The returned input visitor should be released by calling
+ * qmp_input_visitor_cleanup when no longer required.
+ *
+ * Returns: a new input visitor
+ */
+Visitor *qmp_string_input_visitor_new(QObject *obj, bool strict);

We have a QMP input visitor that isn't really about QMP, and a string visitor. You're adding a QMP string input visitor that's even less about QMP (using it with QMP would be wrong, in fact), and that is related to the string visitor only insofar as both parse strings. Differently, of course; this is QEMU.

Can't think of a better name, and I'm running out of Friday.

Should we specify how strings are parsed?

Do you actually need both strict = true and strict = false here?

Quoted 112 lines
include/qapi/qmp-input-visitor.h +1 -0
+
 #endif
qapi/qmp-input-visitor.c +67 -0
index 21edb39..307155f 100644
@@ -20,6 +20,7 @@
 #include "qemu-common.h"
 #include "qapi/qmp/types.h"
 #include "qapi/qmp/qerror.h"
+#include "qemu/cutils.h"
 
 #define QIV_STACK_SIZE 1024
 
@@ -268,6 +269,17 @@ static void qmp_input_type_int64(Visitor *v, const char *name, int64_t *obj,
     *obj = qint_get_int(qint);
 }
 
+static void qmp_input_type_int64_str(Visitor *v, const char *name, int64_t *obj,
+                                     Error **errp)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+    QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));
+    uint64_t ret;
+
+    parse_option_number(name, qstr ? qstr->string : NULL, &ret, errp);
+    *obj = ret;
+}
+
 static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
                                   Error **errp)
 {
@@ -284,6 +296,15 @@ static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
     *obj = qint_get_int(qint);
 }
 
+static void qmp_input_type_uint64_str(Visitor *v, const char *name,
+                                      uint64_t *obj, Error **errp)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+    QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));
+
+    parse_option_number(name, qstr ? qstr->string : NULL, obj, errp);
+}
+
 static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
                                 Error **errp)
 {
@@ -299,6 +320,15 @@ static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
     *obj = qbool_get_bool(qbool);
 }
 
+static void qmp_input_type_bool_str(Visitor *v, const char *name, bool *obj,
+                                    Error **errp)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+    QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));
+
+    parse_option_bool(name, qstr ? qstr->string : NULL, obj, errp);
+}
+
 static void qmp_input_type_str(Visitor *v, const char *name, char **obj,
                                Error **errp)
 {
@@ -339,6 +369,25 @@ static void qmp_input_type_number(Visitor *v, const char *name, double *obj,
                "number");
 }
 
+static void qmp_input_type_number_str(Visitor *v, const char *name, double *obj,
+                                      Error **errp)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+    QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));
+    char *endp;
+
+    if (qstr && qstr->string) {
+        errno = 0;
+        *obj = strtod(qstr->string, &endp);
+        if (errno == 0 && endp != qstr->string && *endp == '\0') {
+            return;
+        }
+    }
+
+    error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+               "number");
+}
+
 static void qmp_input_type_any(Visitor *v, const char *name, QObject **obj,
                                Error **errp)
 {
@@ -360,6 +409,31 @@ static void qmp_input_type_null(Visitor *v, const char *name, Error **errp)
     }
 }
 
+static void qmp_input_type_size_str(Visitor *v, const char *name, uint64_t *obj,
+                                    Error **errp)
+{
+    QmpInputVisitor *qiv = to_qiv(v);
+    QString *qstr = qobject_to_qstring(qmp_input_get_object(qiv, name, true));
+    int64_t val;
+    char *endptr;
+
+    if (qstr && qstr->string) {
+        val = qemu_strtosz_suffix(qstr->string, &endptr,
+                                  QEMU_STRTOSZ_DEFSUFFIX_B);
+        if (val < 0 || *endptr) {
+            error_setg(errp, QERR_INVALID_PARAMETER_VALUE, name,
+                       "a size value representible as a non-negative int64");
+            return;
+        }
+
+        *obj = val;

qapi/qmp-input-visitor.c static void qmp_input_type_null(Visitor *v, const char *name, Error **errp) · line 439

Quoted 10 lines
qapi/qmp-input-visitor.c +7 -0
+        return;
+    }
+
+    error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
+               "size");
+}
+
 static void qmp_input_optional(Visitor *v, const char *name, bool *present)
 {
     QmpInputVisitor *qiv = to_qiv(v);

Separate callbacks result in cleaner code than what I reviewed last. Cleaner semantics, too: either all the scalars have sane types, or all are strings.

Quoted 36 lines
qapi/qmp-input-visitor.c +32 -0
@@ -411,3 +485,35 @@ Visitor *qmp_input_visitor_new(QObject *obj, bool strict)
 
     return &v->visitor;
 }
+
+Visitor *qmp_string_input_visitor_new(QObject *obj, bool strict)
+{
+    QmpInputVisitor *v;
+
+    v = g_malloc0(sizeof(*v));
+
+    v->visitor.type = VISITOR_INPUT;
+    v->visitor.start_struct = qmp_input_start_struct;
+    v->visitor.check_struct = qmp_input_check_struct;
+    v->visitor.end_struct = qmp_input_pop;
+    v->visitor.start_list = qmp_input_start_list;
+    v->visitor.next_list = qmp_input_next_list;
+    v->visitor.end_list = qmp_input_pop;
+    v->visitor.start_alternate = qmp_input_start_alternate;
+    v->visitor.type_int64 = qmp_input_type_int64_str;
+    v->visitor.type_uint64 = qmp_input_type_uint64_str;
+    v->visitor.type_bool = qmp_input_type_bool_str;
+    v->visitor.type_str = qmp_input_type_str;
+    v->visitor.type_number = qmp_input_type_number_str;
+    v->visitor.type_any = qmp_input_type_any;
+    v->visitor.type_null = qmp_input_type_null;
+    v->visitor.type_size = qmp_input_type_size_str;
+    v->visitor.optional = qmp_input_optional;
+    v->visitor.free = qmp_input_free;
+    v->strict = strict;
+
+    v->root = obj;
+    qobject_incref(obj);
+
+    return &v->visitor;
+}

[...]

#7 lore raw

Re: [Qemu-devel] [PATCH v8 3/7] qapi: add a QmpInputVisitor that does string conversion

Quoted 21 lines · Markus Armbruster
Quoted 13 lines · Daniel P. Berrange

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This adds an alternative constructor for QmpInputVisitor that will set it up such that it expects a QString for all scalar types instead.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

Quoted 5 lines

---

+Visitor *qmp_string_input_visitor_new(QObject *obj, bool strict);

Quoted 8 lines

We have a QMP input visitor that isn't really about QMP, and a string visitor. You're adding a QMP string input visitor that's even less about QMP (using it with QMP would be wrong, in fact), and that is related to the string visitor only insofar as both parse strings. Differently, of course; this is QEMU.

Can't think of a better name, and I'm running out of Friday.

Sounds like we might want a prereq patch that just does a global rename of s/qmp-\(input-visitor\|output-visitor\)/qobj-\1/ through all affected files. Since it really is a QObject visitor, the rename will make it easier to give the new visitor a nicer name.

Quoted 3 lines

Should we specify how strings are parsed?

Do you actually need both strict = true and strict = false here?

I'd be fine if we just enforced that the new QObject-string parser is always strict. Non-strict parsers should only be used where we are worried about back-compat for hand-written code that knows how to deal with unparsed keys (such as device_add, where we MUST accept arguments not part of the QAPI schema, so long as we don't have a schema that fully describes it).

Quoted 7 lines
Quoted 3 lines

static void qmp_input_optional(Visitor *v, const char *name, bool *present)

 {
     QmpInputVisitor *qiv = to_qiv(v);

Separate callbacks result in cleaner code than what I reviewed last. Cleaner semantics, too: either all the scalars have sane types, or all are strings.

In other words, it forces us to strongly be tied to the input source, where command line is strongly-typed to strings, and QMP is strongly-typed to native types. It doesn't solve the problem of netdev_add previously being loose and taking both types, but as we've mentioned in other threads, we may not care (it may be sufficient to state that the undocumented looseness of netdev_add is not worth fixing); and Dan did have the nice followup proposal of adding yet another callback with peek semantics that can then delegate to the correct strongly-typed callback, rather than my approach that munged it all into a single callback.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org