Skip to contentSkip to conversation

Provide a QOM-based authorization API

PATCH Superseded Daniel P. Berrange 2 participants 14 messages Mar 7, 2016 – Mar 9, 2016

Subsystems (2)

v2 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#1 lore raw

PATCH v2 1/10 qdict: implement a qdict_crumple method for un-flattening a dict

In reply to Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

{
'foo' => [

     { 'bar': 'one', 'wizz': '1' }
     { 'bar': 'two', 'wizz': '2' }

],
}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nested used when the same object is defined over QMP.

---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 237 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 114 +++++++++++++++++++++++
 3 files changed, 352 insertions(+)
include/qapi/qmp/qdict.h +1 -0
index 71b8eb0..8a3ac13 100644
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
qobject/qdict.c +237 -0

243 lines not on this page · raw message

tests/check-qdict.c +114 -0

126 lines not on this page · raw message

Signature · 1 line
2.5.0
#0 lore raw

v2 cover Provide a QOM-based authorization API

Many years ago I was responsible for adding the 'qemu_acl' type and associated HMP commands. Looking back at it now, it is quite a poor facility with a couple of bad limitations. First, the responsibility for creating the ACLs was left with the QEMU network service (VNC server was only thing ever doing it). This meant you could not share ACLs across multiple services. Second, there was no way to populate ACLs on the command line, you had no choice but to use the HMP commands. Third, the API was hardcoded around the idea of an in-QEMU implementation, leaving no scope for plugging in alternative implementations backed by, for exmaple, LDAP or PAM.

This series introduces a much better authorization API design to QEMU that addresses all these problems, and maintains back compatibility. It of course is based on the QOM framework, so that immediately gives us ability to create objects via the CLI, HMP or QMP. There is an abstract base clss "QAuthZ" which defines the basic API for QEMU network services to use, and a specific implementation "QAuthZ" simple which replicates the functionality of 'qemu_acl'. It is thus possible to add other impls, without changing any other part of QEMU in the future. Finally, the user is responsible for creating the ACL objects, so they can have one ACL associated with all their TLS enabled network services.

There was only one small problem with this, specifically the -object CLI arg and HMP 'object_add' command had no way to let the user specify non-scalar properties for objects. eg if an object had a property which is a list of structs, you are out of luck if you want to create it without using QMP.

Thus the first three patches do some work around QAPI / QOM to make it possible to specify non-scalar properties with the -object CLI arg and HMP 'object_add' command. See the respective patches for illustration of the syntax used.

The patches 4 and 5 introduce the new base class and specific implementation.

Patch 6 kills the old qemu_acl code, updating any existing callers of it to use the QAuthZSimple QOM class instead.

Patches 7-10 add support for associating ACLs with the network services supporting TLS encryption (NBD, chardev and VNC).

Aside from the outstanding migration TLS patches, this series wraps up the feature based work I have for TLS in this release cycle.

Changed in v2:

- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple

Daniel P. Berrange (10):
qdict: implement a qdict_crumple method for un-flattening a dict
qapi: allow QmpInputVisitor to auto-cast types
qom: support arbitrary non-scalar properties with -object
util: add QAuthZ object as an authorization base class
util: add QAuthZSimple object type for a simple access control list
acl: delete existing ACL implementation
qemu-nbd: add support for ACLs for TLS clients
nbd: allow an ACL to be set with nbd-server-start QMP command
chardev: add support for ACLs for TLS clients
vnc: allow specifying a custom ACL object name

 MAINTAINERS                      |   7 +
 Makefile                         |   9 +-
 Makefile.objs                    |   2 +
 Makefile.target                  |   2 +
 blockdev-nbd.c                   |  10 +-
 crypto/tlssession.c              |  28 +++-
 hmp.c                            |  20 +--
 include/qapi/qmp-input-visitor.h |   3 +
 include/qapi/qmp/qdict.h         |   1 +
 include/qemu/acl.h               |  74 ----------
 include/qemu/authz-simple.h      | 107 ++++++++++++++
 include/qemu/authz.h             |  81 +++++++++++
 monitor.c                        | 161 +++++++++++++--------
 qapi-schema.json                 |   8 +-
 qapi/block.json                  |   4 +-
 qapi/qmp-input-visitor.c         |  96 +++++++++++--
 qapi/util.json                   |  31 ++++
 qemu-char.c                      |  11 +-
 qemu-nbd.c                       |  13 +-
 qemu-nbd.texi                    |   4 +
 qmp-commands.hx                  |   2 +-
 qobject/qdict.c                  | 237 +++++++++++++++++++++++++++++++
 qom/object_interfaces.c          |  20 ++-
 tests/.gitignore                 |   1 +
 tests/Makefile                   |   5 +-
 tests/check-qdict.c              | 114 +++++++++++++++
 tests/check-qom-proplist.c       | 295 ++++++++++++++++++++++++++++++++++++++-
 tests/test-authz-simple.c        | 156 +++++++++++++++++++++
 tests/test-crypto-tlssession.c   |  13 +-
 tests/test-io-channel-tls.c      |  14 +-
 tests/test-qmp-input-visitor.c   | 115 ++++++++++++++-
 ui/vnc-auth-sasl.c               |   2 +-
 ui/vnc-auth-sasl.h               |   4 +-
 ui/vnc.c                         |  76 ++++++++--
 util/Makefile.objs               |   4 +-
 util/acl.c                       | 188 -------------------------
 util/authz-simple.c              | 256 +++++++++++++++++++++++++++++++++
 util/authz.c                     |  46 ++++++
 38 files changed, 1817 insertions(+), 403 deletions(-)
 delete mode 100644 include/qemu/acl.h
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 include/qemu/authz.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 delete mode 100644 util/acl.c
 create mode 100644 util/authz-simple.c
 create mode 100644 util/authz.c
Signature · 1 line
2.5.0
#6 lore raw

PATCH v2 3/10 qom: support arbitrary non-scalar properties with -object

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.

The previously added qdict_crumple() method is able to take a qdict containing a flat set of properties and turn that into a arbitrarily nested set of dicts and lists. By combining qemu_opts_to_qdict and qdict_crumple() together, we can turn the opt string into a data structure that is practically identical to that passed over QMP when defining an object. The only difference is that all the scalar values are represented as strings, rather than strings, ints and bools. This is sufficient to let us replace the OptsVisitor with the QMPInputVisitor for use with -object.

Thus -object can now support non-scalar properties, for example the QMP object

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
	  { "bar": "one", "wizz": "1" },
	  { "bar": "two", "wizz": "2" }
        ]
      }
    }

}

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.

  (hmp) object_add demo,id=demo0,\
                   foo.0.bar=one,foo.0.wizz=1,\
		   foo.1.bar=two,foo.1.wizz=2
---
 hmp.c                      |  18 +--
 qom/object_interfaces.c    |  20 ++-
 tests/check-qom-proplist.c | 295 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 313 insertions(+), 20 deletions(-)
hmp.c +5 -13
index 5b6084a..7a98726 100644
@@ -25,7 +25,7 @@
 #include "qemu/sockets.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi/util.h"
@@ -1673,20 +1673,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
 void hmp_object_add(Monitor *mon, const QDict *qdict)
 {
     Error *err = NULL;
-    QemuOpts *opts;
-    OptsVisitor *ov;
+    QmpInputVisitor *qiv;
     Object *obj = NULL;
 
-    opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
-    if (err) {
-        hmp_handle_error(mon, &err);
-        return;
-    }
-
-    ov = opts_visitor_new(opts);
-    obj = user_creatable_add(qdict, opts_get_visitor(ov), &err);
-    opts_visitor_cleanup(ov);
-    qemu_opts_del(opts);
+    qiv = qmp_input_visitor_new_full((QObject *)qdict, true, true);
+    obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), &err);
+    qmp_input_visitor_cleanup(qiv);
 
     if (err) {
         hmp_handle_error(mon, &err);
qom/object_interfaces.c +14 -6
index c2f6e29..9c41730 100644
@@ -1,9 +1,9 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 #include "qemu/module.h"
+#include "qemu/option.h"
 #include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 
 void user_creatable_complete(Object *obj, Error **errp)
 {
@@ -120,6 +120,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
     obj = object_new(type);
     if (qdict) {
         for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+
             object_property_set(obj, v, e->key, &local_err);
             if (local_err) {
                 goto out;
@@ -151,15 +152,22 @@ out:
 
 Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
 {
-    OptsVisitor *ov;
+    QmpInputVisitor *qiv;
     QDict *pdict;
+    QObject *pobj;
     Object *obj = NULL;
 
-    ov = opts_visitor_new(opts);
     pdict = qemu_opts_to_qdict(opts, NULL);
+    pobj = qdict_crumple(pdict, true, errp);
+    if (!pobj) {
+        goto cleanup;
+    }
+    qiv = qmp_input_visitor_new_full(pobj, true, true);
 
-    obj = user_creatable_add(pdict, opts_get_visitor(ov), errp);
-    opts_visitor_cleanup(ov);
+    obj = user_creatable_add((QDict *)pobj, qmp_input_get_visitor(qiv), errp);
+    qmp_input_visitor_cleanup(qiv);
+    qobject_decref(pobj);
+ cleanup:
     QDECREF(pdict);
     return obj;
 }
tests/check-qom-proplist.c +294 -1

387 lines not on this page · raw message

Signature · 1 line
2.5.0
#7 lore raw

PATCH v2 4/10 util: add QAuthZ object as an authorization base class

In reply to Daniel P. Berrange

The current qemu_acl module provides a simple access control list facility inside QEMU, which is used via a set of monitor commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially integrate with external authorization systems like polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be subclassed for creating different authorization providers.

---
 MAINTAINERS          |  7 +++++
 Makefile             |  1 +
 Makefile.objs        |  2 ++
 Makefile.target      |  2 ++
 include/qemu/authz.h | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 util/Makefile.objs   |  2 ++
 util/authz.c         | 46 +++++++++++++++++++++++++++++
 7 files changed, 141 insertions(+)
 create mode 100644 include/qemu/authz.h
 create mode 100644 util/authz.c
MAINTAINERS +7 -0
index 2f5a338..9dcbde9 100644
@@ -1297,6 +1297,13 @@ F: include/qemu/throttle.h
 F: util/throttle.c
 L: [email protected]
 
+Authorization
+M: Daniel P. Berrange <[email protected]>
+S: Maintained
+F: util/authz*
+F: include/qemu/authz*
+F: tests/test-authz-*
+
 Usermode Emulation
 ------------------
 Overall
Makefile +1 -0
index 70e3ebc..903dc35 100644
@@ -150,6 +150,7 @@ endif
 dummy := $(call unnest-vars,, \
                 stub-obj-y \
                 util-obj-y \
+                util-qom-obj-y \
                 qga-obj-y \
                 ivshmem-client-obj-y \
                 ivshmem-server-obj-y \
Makefile.objs +2 -0
index fbcaa74..8bc9a77 100644
@@ -4,6 +4,8 @@ stub-obj-y = stubs/
 util-obj-y = util/ qobject/ qapi/
 util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
 
+util-qom-obj-y += util/
+
 #######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img
 
Makefile.target +2 -0
index 34ddb7e..9728f86 100644
@@ -171,6 +171,7 @@ include $(SRC_PATH)/Makefile.objs
 dummy := $(call unnest-vars,,target-obj-y)
 target-obj-y-save := $(target-obj-y)
 dummy := $(call unnest-vars,.., \
+               util-qom-obj-y \
                block-obj-y \
                block-obj-m \
                crypto-obj-y \
@@ -183,6 +184,7 @@ target-obj-y := $(target-obj-y-save)
 all-obj-y += $(common-obj-y)
 all-obj-y += $(target-obj-y)
 all-obj-y += $(qom-obj-y)
+all-obj-y += $(util-qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
include/qemu/authz.h new file +81 -0

81 lines not on this page · raw message

util/Makefile.objs +2 -0
index a8a777e..b29fb45 100644
@@ -32,3 +32,5 @@ util-obj-y += buffer.o
 util-obj-y += timed-average.o
 util-obj-y += base64.o
 util-obj-y += log.o
+
+util-qom-obj-y += authz.o
util/authz.c new file +46 -0

46 lines not on this page · raw message

Signature · 1 line
2.5.0
#5 lore raw

PATCH v2 2/10 qapi: allow QmpInputVisitor to auto-cast types

In reply to Daniel P. Berrange

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This extends it so that QString is optionally permitted for any of the non-string scalar types. This behaviour is turned on by requesting the 'autocast' flag in the constructor.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

---
 include/qapi/qmp-input-visitor.h |   3 +
 qapi/qmp-input-visitor.c         |  96 +++++++++++++++++++++++++++-----
 tests/test-qmp-input-visitor.c   | 115 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 196 insertions(+), 18 deletions(-)
include/qapi/qmp-input-visitor.h +3 -0
index 3ed499c..c25cb7c 100644
@@ -21,6 +21,9 @@ typedef struct QmpInputVisitor QmpInputVisitor;
 
 QmpInputVisitor *qmp_input_visitor_new(QObject *obj);
 QmpInputVisitor *qmp_input_visitor_new_strict(QObject *obj);
+QmpInputVisitor *qmp_input_visitor_new_full(QObject *obj,
+                                            bool strict,
+                                            bool autocast);
 
 void qmp_input_visitor_cleanup(QmpInputVisitor *v);
 
qapi/qmp-input-visitor.c +81 -15

147 lines not on this page · raw message

tests/test-qmp-input-visitor.c +112 -3

173 lines not on this page · raw message

Signature · 1 line
2.5.0
#8 lore raw

PATCH v2 5/10 util: add QAuthZSimple object type for a simple access control list

In reply to Daniel P. Berrange

Add a QAuthZSimple object type that implements the QAuthZ interface. This simple built-in implementation maintains a trivial access control list with a sequence of match rules and a final default policy. This replicates the functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor, the syntax used would be

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "authz-simple",
      "id": "auth0",
      "parameters": {
        "rules": [
           { "match": "fred", "policy": "allow" },
           { "match": "bob", "policy": "allow" },
           { "match": "danb", "policy": "deny" },
           { "match": "dan*", "policy": "allow" }
        ],
        "policy": "deny"
      }
    }
  }

Or via the -object command line

  $QEMU \
     -object authz-simple,id=acl0,policy=deny,\
             match.0.name=fred,match.0.policy=allow, \
             match.1.name=bob,match.1.policy=allow, \
             match.2.name=danb,match.2.policy=deny, \
             match.3.name=dan*,match.3.policy=allow

This sets up an authorization rule that allows 'fred', 'bob' and anyone whose name starts with 'dan', except for 'danb'. Everyone unmatched is denied.

---
 Makefile                    |   2 +-
 include/qemu/authz-simple.h | 107 ++++++++++++++++++
 qapi-schema.json            |   6 +-
 qapi/util.json              |  31 ++++++
 tests/.gitignore            |   1 +
 tests/Makefile              |   3 +
 tests/test-authz-simple.c   | 156 +++++++++++++++++++++++++++
 util/Makefile.objs          |   1 +
 util/authz-simple.c         | 256 ++++++++++++++++++++++++++++++++++++++++++++
 9 files changed, 561 insertions(+), 2 deletions(-)
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 create mode 100644 util/authz-simple.c
Makefile +1 -1
index 903dc35..60ad13e 100644
@@ -274,7 +274,7 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
                $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
                $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
                $(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
-               $(SRC_PATH)/qapi/trace.json
+               $(SRC_PATH)/qapi/trace.json $(SRC_PATH)/qapi/util.json
 
 qapi-types.c qapi-types.h :\
 $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
include/qemu/authz-simple.h new file +107 -0

107 lines not on this page · raw message

qapi-schema.json +5 -1
index 362c9d8..b6769de 100644
@@ -5,6 +5,9 @@
 # QAPI common definitions
 { 'include': 'qapi/common.json' }
 
+# QAPI util definitions
+{ 'include': 'qapi/util.json' }
+
 # QAPI crypto definitions
 { 'include': 'qapi/crypto.json' }
 
@@ -3652,7 +3655,8 @@
 # Since 2.5
 ##
 { 'struct': 'DummyForceArrays',
-  'data': { 'unused': ['X86CPUFeatureWordInfo'] } }
+  'data': { 'unused': ['X86CPUFeatureWordInfo'],
+            'iamalsounused': ['QAuthZSimpleRule'] } }
 
 
 ##
qapi/util.json new file +31 -0
new file mode 100644
index 0000000..d903497
@@ -0,0 +1,31 @@
+# -*- Mode: Python -*-
+#
+# QAPI util definitions
+
+##
+# QAuthZSimplePolicy:
+#
+# The authorization policy result
+#
+# @deny: deny access
+# @allow: allow access
+#
+# Since: 2.6
+##
+{ 'enum': 'QAuthZSimplePolicy',
+  'prefix': 'QAUTHZ_SIMPLE_POLICY',
+  'data': ['deny', 'allow']}
+
+##
+# QAuthZSimpleRule:
+#
+# A single authorization rule.
+#
+# @match: a glob to match against a user identity
+# @policy: the result to return if @match evaluates to true
+#
+# Since: 2.6
+##
+{ 'struct': 'QAuthZSimpleRule',
+  'data': {'match': 'str',
+           'policy': 'QAuthZSimplePolicy'}}
tests/.gitignore +1 -0
index 787c95c..1e73a2f 100644
@@ -8,6 +8,7 @@ check-qom-interface
 check-qom-proplist
 rcutorture
 test-aio
+test-authz-simple
 test-base64
 test-bitops
 test-blockjob-txn
tests/Makefile +3 -0
index cd4bbd4..7a1d959 100644
@@ -92,6 +92,7 @@ check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
 check-unit-y += tests/test-io-channel-command$(EXESUF)
 check-unit-y += tests/test-io-channel-buffer$(EXESUF)
 check-unit-y += tests/test-base64$(EXESUF)
+check-unit-y += tests/test-authz-simple$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -430,6 +431,8 @@ tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
 	$(test-util-obj-y)
 tests/test-base64$(EXESUF): tests/test-base64.o \
 	libqemuutil.a libqemustub.a
+tests/test-authz-simple$(EXESUF): tests/test-authz-simple.o \
+	libqemuutil.a libqemustub.a $(util-qom-obj-y) $(qom-obj-y)
 
 tests/test-qapi-types.c tests/test-qapi-types.h :\
 $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
tests/test-authz-simple.c new file +156 -0

156 lines not on this page · raw message

util/Makefile.objs +1 -0
index b29fb45..4870905 100644
@@ -34,3 +34,4 @@ util-obj-y += base64.o
 util-obj-y += log.o
 
 util-qom-obj-y += authz.o
+util-qom-obj-y += authz-simple.o
util/authz-simple.c new file +256 -0

256 lines not on this page · raw message

Signature · 1 line
2.5.0
#9 lore raw

PATCH v2 6/10 acl: delete existing ACL implementation

In reply to Daniel P. Berrange

The 'qemu_acl' type was a previous non-QOM based attempt to provide an authorization facility in QEMU. Because it is non-QOM based it cannot be created via the command line and requires special monitor commands to manipulate it.

The new QAuthZ and QAuthZSimple QOM classes provide a superset of the functionality in qemu_acl, so the latter can now be deleted. The HMP 'acl_*' monitor commands are converted to use the new QAuthZSimple data type instead in order to provide backwards compatibility, but their use is discouraged.

---
 Makefile                       |   6 +-
 crypto/tlssession.c            |  28 ++++--
 include/qemu/acl.h             |  74 ----------------
 monitor.c                      | 161 ++++++++++++++++++++++-------------
 tests/Makefile                 |   2 +-
 tests/test-crypto-tlssession.c |  13 +--
 tests/test-io-channel-tls.c    |  14 +--
 ui/vnc-auth-sasl.c             |   2 +-
 ui/vnc-auth-sasl.h             |   4 +-
 ui/vnc.c                       |  11 ++-
 util/Makefile.objs             |   1 -
 util/acl.c                     | 188 -----------------------------------------
 12 files changed, 156 insertions(+), 348 deletions(-)
 delete mode 100644 include/qemu/acl.h
 delete mode 100644 util/acl.c
Makefile +3 -3
index 60ad13e..8f7ffd3 100644
@@ -235,9 +235,9 @@ util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
 
 qemu-img.o: qemu-img-cmds.h
 
-qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
+qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
 
 qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o
 
crypto/tlssession.c +23 -5
index e0d9658..26e8097 100644
@@ -22,7 +22,7 @@
 #include "crypto/tlssession.h"
 #include "crypto/tlscredsanon.h"
 #include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "trace.h"
 
 #ifdef CONFIG_GNUTLS
@@ -207,6 +207,7 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
     unsigned int nCerts, i;
     time_t now;
     gnutls_x509_crt_t cert = NULL;
+    Error *err = NULL;
 
     now = time(NULL);
     if (now == ((time_t)-1)) {
@@ -295,16 +296,33 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
                 goto error;
             }
             if (session->aclname) {
-                qemu_acl *acl = qemu_acl_find(session->aclname);
-                int allow;
-                if (!acl) {
+                QAuthZ *acl;
+                Object *obj;
+                Object *container;
+                bool allow;
+
+                container = object_get_objects_root();
+                obj = object_resolve_path_component(container,
+                                                    session->aclname);
+                if (!obj) {
                     error_setg(errp, "Cannot find ACL %s",
                                session->aclname);
                     goto error;
                 }
 
-                allow = qemu_acl_party_is_allowed(acl, session->peername);
+                if (!object_dynamic_cast(obj, TYPE_QAUTHZ)) {
+                    error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                               session->aclname);
+                    goto error;
+                }
 
+                acl = QAUTHZ(obj);
+
+                allow = qauthz_is_allowed(acl, session->peername, &err);
+                if (err) {
+                    error_propagate(errp, err);
+                    goto error;
+                }
                 if (!allow) {
                     error_setg(errp, "TLS x509 ACL check for %s is denied",
                                session->peername);
include/qemu/acl.h deleted +0 -74

74 lines not on this page · raw message

monitor.c +101 -60

221 lines not on this page · raw message

tests/Makefile +1 -1
index 7a1d959..647f30f 100644
@@ -391,7 +391,7 @@ test-qom-obj-y = $(qom-obj-y) $(test-util-obj-y)
 test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
 	tests/test-qapi-event.o tests/test-qmp-introspect.o \
 	$(test-qom-obj-y)
-test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
+test-crypto-obj-y = $(crypto-obj-y) $(util-qom-obj-y) $(test-qom-obj-y)
 test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
 test-block-obj-y = $(block-obj-y) $(test-io-obj-y)
 
tests/test-crypto-tlssession.c +8 -5
index 036a86b..e76b249 100644
@@ -25,7 +25,7 @@
 #include "crypto/tlssession.h"
 #include "qom/object_interfaces.h"
 #include "qemu/sockets.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 
 #ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
 
@@ -110,7 +110,7 @@ static void test_crypto_tls_session(const void *opaque)
     QCryptoTLSCreds *serverCreds;
     QCryptoTLSSession *clientSess = NULL;
     QCryptoTLSSession *serverSess = NULL;
-    qemu_acl *acl;
+    QAuthZSimple *auth;
     const char * const *wildcards;
     int channel[2];
     bool clientShake = false;
@@ -169,11 +169,13 @@ static void test_crypto_tls_session(const void *opaque)
         &err);
     g_assert(serverCreds != NULL);
 
-    acl = qemu_acl_init("tlssessionacl");
-    qemu_acl_reset(acl);
+    auth = qauthz_simple_new("tlssessionacl",
+                             QAUTHZ_SIMPLE_POLICY_DENY,
+                             &error_abort);
     wildcards = data->wildcards;
     while (wildcards && *wildcards) {
-        qemu_acl_append(acl, 0, *wildcards);
+        qauthz_simple_append_rule(auth, *wildcards,
+                                  QAUTHZ_SIMPLE_POLICY_ALLOW);
         wildcards++;
     }
 
@@ -263,6 +265,7 @@ static void test_crypto_tls_session(const void *opaque)
 
     object_unparent(OBJECT(serverCreds));
     object_unparent(OBJECT(clientCreds));
+    object_unparent(OBJECT(auth));
 
     qcrypto_tls_session_free(serverSess);
     qcrypto_tls_session_free(clientSess);
tests/test-io-channel-tls.c +9 -5
index 3c361a7..390d3e9 100644
@@ -28,7 +28,7 @@
 #include "io/channel-socket.h"
 #include "io-channel-helpers.h"
 #include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 #include "qom/object_interfaces.h"
 
 #ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
@@ -115,7 +115,7 @@ static void test_io_channel_tls(const void *opaque)
     QIOChannelTLS *serverChanTLS;
     QIOChannelSocket *clientChanSock;
     QIOChannelSocket *serverChanSock;
-    qemu_acl *acl;
+    QAuthZSimple *auth;
     const char * const *wildcards;
     int channel[2];
     struct QIOChannelTLSHandshakeData clientHandshake = { false, false };
@@ -166,11 +166,13 @@ static void test_io_channel_tls(const void *opaque)
         &err);
     g_assert(serverCreds != NULL);
 
-    acl = qemu_acl_init("channeltlsacl");
-    qemu_acl_reset(acl);
+    auth = qauthz_simple_new("channeltlsacl",
+                             QAUTHZ_SIMPLE_POLICY_DENY,
+                             &error_abort);
     wildcards = data->wildcards;
     while (wildcards && *wildcards) {
-        qemu_acl_append(acl, 0, *wildcards);
+        qauthz_simple_append_rule(auth, *wildcards,
+                                  QAUTHZ_SIMPLE_POLICY_ALLOW);
         wildcards++;
     }
 
@@ -256,6 +258,8 @@ static void test_io_channel_tls(const void *opaque)
     object_unref(OBJECT(serverChanSock));
     object_unref(OBJECT(clientChanSock));
 
+    object_unparent(OBJECT(auth));
+
     close(channel[0]);
     close(channel[1]);
 }
ui/vnc-auth-sasl.c +1 -1
index 13a59f5..53f9a7c 100644
@@ -149,7 +149,7 @@ static int vnc_auth_sasl_check_access(VncState *vs)
         return 0;
     }
 
-    allow = qemu_acl_party_is_allowed(vs->vd->sasl.acl, vs->sasl.username);
+    allow = qauthz_is_allowed(vs->vd->sasl.acl, vs->sasl.username, NULL);
 
     VNC_DEBUG("SASL client %s %s by ACL\n", vs->sasl.username,
               allow ? "allowed" : "denied");
ui/vnc-auth-sasl.h +2 -2
index 3f59da6..a6ffb7e 100644
@@ -32,7 +32,7 @@
 typedef struct VncStateSASL VncStateSASL;
 typedef struct VncDisplaySASL VncDisplaySASL;
 
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "qemu/main-loop.h"
 
 struct VncStateSASL {
@@ -61,7 +61,7 @@ struct VncStateSASL {
 };
 
 struct VncDisplaySASL {
-    qemu_acl *acl;
+    QAuthZ *acl;
 };
 
 void vnc_sasl_client_cleanup(VncState *vs);
ui/vnc.c +8 -3
index 6cd6314..19bc396 100644
@@ -33,7 +33,7 @@
 #include "qemu/error-report.h"
 #include "qemu/sockets.h"
 #include "qemu/timer.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 #include "qemu/config-file.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/types.h"
@@ -3702,7 +3702,9 @@ void vnc_display_open(const char *id, Error **errp)
         } else {
             vs->tlsaclname = g_strdup_printf("vnc.%s.x509dname", vs->id);
         }
-        qemu_acl_init(vs->tlsaclname);
+        qauthz_simple_new(vs->tlsaclname,
+                          QAUTHZ_SIMPLE_POLICY_DENY,
+                          &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
     if (acl && sasl) {
@@ -3713,7 +3715,10 @@ void vnc_display_open(const char *id, Error **errp)
         } else {
             aclname = g_strdup_printf("vnc.%s.username", vs->id);
         }
-        vs->sasl.acl = qemu_acl_init(aclname);
+        vs->sasl.acl =
+            QAUTHZ(qauthz_simple_new(aclname,
+                                     QAUTHZ_SIMPLE_POLICY_DENY,
+                                     &error_abort));
         g_free(aclname);
     }
 #endif
util/Makefile.objs +0 -1
index 4870905..44078c1 100644
@@ -13,7 +13,6 @@ util-obj-y += envlist.o path.o module.o
 util-obj-$(call lnot,$(CONFIG_INT128)) += host-utils.o
 util-obj-y += bitmap.o bitops.o hbitmap.o
 util-obj-y += fifo8.o
-util-obj-y += acl.o
 util-obj-y += error.o qemu-error.o
 util-obj-y += id.o
 util-obj-y += iov.o qemu-config.o qemu-sockets.o uri.o notify.o
util/acl.c deleted +0 -188

188 lines not on this page · raw message

Signature · 1 line
2.5.0
#10 lore raw

PATCH v2 7/10 qemu-nbd: add support for ACLs for TLS clients

In reply to Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly weak bar.

This adds a '--tls-acl ACL-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the NBD server.

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

  qemu-nbd -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                   endpoint=server,verify-peer=yes \
           -object authz-simple,id=acl0,policy=deny,\
	           rules.0.match=*CN=fred,rules.0.policy=allow \
           -tls-creds tls0 \
           -tls-acl acl0
	   ....other qemu-nbd args...
---
 qemu-nbd.c    | 13 ++++++++++++-
 qemu-nbd.texi |  4 ++++
 2 files changed, 16 insertions(+), 1 deletion(-)
qemu-nbd.c +12 -1
index a5c1d95..d70960f 100644
@@ -42,6 +42,7 @@
 #define QEMU_NBD_OPT_OBJECT        260
 #define QEMU_NBD_OPT_TLSCREDS      261
 #define QEMU_NBD_OPT_IMAGE_OPTS    262
+#define QEMU_NBD_OPT_TLSACL        263
 
 static NBDExport *exp;
 static bool newproto;
@@ -55,6 +56,7 @@ static int nb_fds;
 static QIOChannelSocket *server_ioc;
 static int server_watch = -1;
 static QCryptoTLSCreds *tlscreds;
+static const char *tlsacl;
 
 static void usage(const char *name)
 {
@@ -344,7 +346,7 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition cond, gpointer opaque)
     nb_fds++;
     nbd_update_server_watch();
     nbd_client_new(newproto ? NULL : exp, cioc,
-                   tlscreds, NULL, nbd_client_closed);
+                   tlscreds, tlsacl, nbd_client_closed);
     object_unref(OBJECT(cioc));
 
     return TRUE;
@@ -488,6 +490,7 @@ int main(int argc, char **argv)
         { "export-name", required_argument, NULL, 'x' },
         { "tls-creds", required_argument, NULL, QEMU_NBD_OPT_TLSCREDS },
         { "image-opts", no_argument, NULL, QEMU_NBD_OPT_IMAGE_OPTS },
+        { "tls-acl", no_argument, NULL, QEMU_NBD_OPT_TLSACL },
         { NULL, 0, NULL, 0 }
     };
     int ch;
@@ -689,6 +692,9 @@ int main(int argc, char **argv)
         case QEMU_NBD_OPT_IMAGE_OPTS:
             imageOpts = true;
             break;
+        case QEMU_NBD_OPT_TLSACL:
+            tlsacl = optarg;
+            break;
         }
     }
 
@@ -725,6 +731,11 @@ int main(int argc, char **argv)
                          error_get_pretty(local_err));
             exit(EXIT_FAILURE);
         }
+    } else {
+        if (tlsacl) {
+            error_report("--tls-acl is not permitted without --tls-creds");
+            exit(EXIT_FAILURE);
+        }
     }
 
     if (disconnect) {
qemu-nbd.texi +4 -0
index 9f23343..69f32cb 100644
@@ -86,6 +86,10 @@ the new style NBD protocol negotiation
 Enable mandatory TLS encryption for the server by setting the ID
 of the TLS credentials object previously created with the --object
 option.
+@item --tls-acl=ID
+Specify the ID of a qauthz object previously created with the
+--object option. This will be used to authorize users who
+connect against their x509 distinguish name.
 @item -v, --verbose
 Display extra debugging information
 @item -h, --help
Signature · 1 line
2.5.0
#11 lore raw

PATCH v2 8/10 nbd: allow an ACL to be set with nbd-server-start QMP command

In reply to Daniel P. Berrange

As with the previous patch to qemu-nbd, the nbd-server-start QMP command also needs to be able to specify an ACL when enabling TLS encryption.

First the client must create a QAuthZ object instance using the 'object-add' command:

   {
     'execute': 'object-add',
     'arguments': {
       'qom-type': 'authz-simple',
       'id': 'tls0',
       'parameters': {
         'policy': 'deny',
         'rules': [
           {
             'match': '*CN=fred',
             'policy': 'allow'
           }
         ]
       }
     }
   }

They can then reference this in the new 'tls-acl' parameter when executing the 'nbd-server-start' command.

   {
     'execute': 'nbd-server-start',
     'arguments': {
       'addr': {
           'type': 'inet',
           'host': '127.0.0.1',
           'port': '9000'
       },
       'tls-creds': 'tls0',
       'tls-acl': 'tlsacl0'
     }
   }
---
 blockdev-nbd.c  | 10 +++++++++-
 hmp.c           |  2 +-
 qapi/block.json |  4 +++-
 qmp-commands.hx |  2 +-
 4 files changed, 14 insertions(+), 4 deletions(-)
blockdev-nbd.c +9 -1
index 12cae0e..ae5335e 100644
@@ -24,6 +24,7 @@ typedef struct NBDServerData {
     QIOChannelSocket *listen_ioc;
     int watch;
     QCryptoTLSCreds *tlscreds;
+    char *tlsacl;
 } NBDServerData;
 
 static NBDServerData *nbd_server;
@@ -45,7 +46,8 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition condition,
     }
 
     nbd_client_new(NULL, cioc,
-                   nbd_server->tlscreds, NULL,
+                   nbd_server->tlscreds,
+                   nbd_server->tlsacl,
                    nbd_client_put);
     object_unref(OBJECT(cioc));
     return TRUE;
@@ -65,6 +67,7 @@ static void nbd_server_free(NBDServerData *server)
     if (server->tlscreds) {
         object_unref(OBJECT(server->tlscreds));
     }
+    g_free(server->tlsacl);
 
     g_free(server);
 }
@@ -101,6 +104,7 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
 
 void qmp_nbd_server_start(SocketAddress *addr,
                           bool has_tls_creds, const char *tls_creds,
+                          bool has_tls_acl, const char *tls_acl,
                           Error **errp)
 {
     if (nbd_server) {
@@ -128,6 +132,10 @@ void qmp_nbd_server_start(SocketAddress *addr,
         }
     }
 
+    if (has_tls_acl) {
+        nbd_server->tlsacl = g_strdup(tls_acl);
+    }
+
     nbd_server->watch = qio_channel_add_watch(
         QIO_CHANNEL(nbd_server->listen_ioc),
         G_IO_IN,
hmp.c +1 -1
index 7a98726..20703fd 100644
@@ -1802,7 +1802,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
         goto exit;
     }
 
-    qmp_nbd_server_start(addr, false, NULL, &local_err);
+    qmp_nbd_server_start(addr, false, NULL, false, NULL, &local_err);
     qapi_free_SocketAddress(addr);
     if (local_err != NULL) {
         goto exit;
qapi/block.json +3 -1
index 58e6b30..6b209e1 100644
@@ -147,6 +147,7 @@
 #
 # @addr: Address on which to listen.
 # @tls-creds: (optional) ID of the TLS credentials object. Since 2.6
+# @tls-acl: (optional) ID of the QAuthZ authorization object. Since 2.6
 #
 # Returns: error if the server is already running.
 #
@@ -154,7 +155,8 @@
 ##
 { 'command': 'nbd-server-start',
   'data': { 'addr': 'SocketAddress',
-            '*tls-creds': 'str'} }
+            '*tls-creds': 'str',
+            '*tls-acl': 'str'} }
 
 ##
 # @nbd-server-add:
qmp-commands.hx +1 -1
index b629673..7a3fa26 100644
@@ -3859,7 +3859,7 @@ EQMP
 
     {
         .name       = "nbd-server-start",
-        .args_type  = "addr:q,tls-creds:s?",
+        .args_type  = "addr:q,tls-creds:s?,tls-acl:s?",
         .mhandler.cmd_new = qmp_marshal_nbd_server_start,
     },
     {
Signature · 1 line
2.5.0
#12 lore raw

PATCH v2 9/10 chardev: add support for ACLs for TLS clients

In reply to Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use a chardev server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the chardev server. This is still a fairly weak bar.

This adds a 'tls-acl=ACL-ID' option to the socket chardev backend which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the chardev server.

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

  $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                endpoint=server,verify-peer=yes \
        -object authz-simple,id=acl0,policy=deny,\
                rules.0.match=*CN=fred,rules.0.policy=allow \
        -chardev socket,host=127.0.0.1,port=9000,server,\
	         tls-creds=tls0,tls-acl=acl0 \
        ...other qemud args...
---
 qapi-schema.json |  2 ++
 qemu-char.c      | 11 ++++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)
qapi-schema.json +2 -0
index b6769de..a6a7205 100644
@@ -3202,6 +3202,7 @@
 # @addr: socket address to listen on (server=true)
 #        or connect to (server=false)
 # @tls-creds: #optional the ID of the TLS credentials object (since 2.6)
+# @tls-acl: #optional the ID of the QAuthZ authorization object (since 2.6)
 # @server: #optional create server socket (default: true)
 # @wait: #optional wait for incoming connection on server
 #        sockets (default: false).
@@ -3217,6 +3218,7 @@
 ##
 { 'struct': 'ChardevSocket', 'data': { 'addr'       : 'SocketAddress',
                                      '*tls-creds'  : 'str',
+                                     '*tls-acl'    : 'str',
                                      '*server'    : 'bool',
                                      '*wait'      : 'bool',
                                      '*nodelay'   : 'bool',
qemu-char.c +10 -1
index e0147f3..9533e7e 100644
@@ -2533,6 +2533,7 @@ typedef struct {
     QIOChannelSocket *listen_ioc;
     guint listen_tag;
     QCryptoTLSCreds *tls_creds;
+    char *tls_acl;
     int connected;
     int max_size;
     int do_telnetopt;
@@ -2963,7 +2964,7 @@ static void tcp_chr_tls_init(CharDriverState *chr)
     if (s->is_listen) {
         tioc = qio_channel_tls_new_server(
             s->ioc, s->tls_creds,
-            NULL, /* XXX Use an ACL */
+            s->tls_acl,
             &err);
     } else {
         tioc = qio_channel_tls_new_client(
@@ -3084,6 +3085,7 @@ static void tcp_chr_close(CharDriverState *chr)
     if (s->tls_creds) {
         object_unref(OBJECT(s->tls_creds));
     }
+    g_free(s->tls_acl);
     if (s->write_msgfds_num) {
         g_free(s->write_msgfds);
     }
@@ -3623,6 +3625,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
     const char *host = qemu_opt_get(opts, "host");
     const char *port = qemu_opt_get(opts, "port");
     const char *tls_creds = qemu_opt_get(opts, "tls-creds");
+    const char *tls_acl = qemu_opt_get(opts, "tls-acl");
     SocketAddress *addr;
     ChardevSocket *sock;
 
@@ -3656,6 +3659,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
     sock->has_reconnect = true;
     sock->reconnect = reconnect;
     sock->tls_creds = g_strdup(tls_creds);
+    sock->tls_acl = g_strdup(tls_acl);
 
     addr = g_new0(SocketAddress, 1);
     if (path) {
@@ -4094,6 +4098,9 @@ QemuOptsList qemu_chardev_opts = {
             .name = "tls-creds",
             .type = QEMU_OPT_STRING,
         },{
+            .name = "tls-acl",
+            .type = QEMU_OPT_STRING,
+        },{
             .name = "width",
             .type = QEMU_OPT_NUMBER,
         },{
@@ -4341,6 +4348,7 @@ static CharDriverState *qmp_chardev_open_socket(const char *id,
             }
         }
     }
+    s->tls_acl = g_strdup(sock->tls_acl);
 
     qapi_copy_SocketAddress(&s->addr, sock->addr);
 
@@ -4386,6 +4394,7 @@ static CharDriverState *qmp_chardev_open_socket(const char *id,
     if (s->tls_creds) {
         object_unref(OBJECT(s->tls_creds));
     }
+    g_free(s->tls_acl);
     g_free(s);
     qemu_chr_free_common(chr);
     return NULL;
Signature · 1 line
2.5.0
#13 lore raw

PATCH v2 10/10 vnc: allow specifying a custom ACL object name

In reply to Daniel P. Berrange

The VNC server has historically had support for ACLs to check both the SASL username and the TLS x509 distinguished name. The VNC server was responsible for creating the initial ACL, and the client app was then responsible for populating it with rules using the HMP 'acl_add' command.

This is not satisfactory for a variety of reasons. There is no way to populate the ACLs from the command line, users are forced to use the HMP. With multiple network services all supporting TLS and ACLs now, it is desirable to be able to define a single ACL that is referenced by all services.

To address these limitations, two new options are added to the VNC server CLI. The 'tls-acl' option takes the ID of a QAuthZ object to use for checking TLS x509 distinguished names, and the 'sasl-acl' option takes the ID of another object to use for checking SASL usernames.

In this example, we setup two ACLs. The first allows any client with a certificate issued by the 'RedHat' organization in the 'London' locality. The second ACL allows clients with either the '[email protected]' or '[email protected]' kerberos usernames. Both ACLs must pass for the user to be allowed.

    $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                  endpoint=server,verify-peer=yes \
          -object authz-simple,id=acl0,policy=deny,\
                  rules.0.match=O=RedHat,,L=London,rules.0.policy=allow \
          -object authz-simple,id=acl0,policy=deny,\
                  [email protected],rules.0.policy=allow \
                  [email protected],rules.0.policy=allow \
          -vnc 0.0.0.0:1,tls-creds=tls0,tls-acl=tlsacl0,
	       sasl,sasl-acl=saslacl0 \
          ...other QEMU args...
---
 ui/vnc.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 60 insertions(+), 13 deletions(-)
ui/vnc.c +60 -13
index 19bc396..96a5213 100644
@@ -3258,6 +3258,12 @@ static QemuOptsList qemu_vnc_opts = {
             .name = "acl",
             .type = QEMU_OPT_BOOL,
         },{
+            .name = "tls-acl",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "sasl-acl",
+            .type = QEMU_OPT_STRING,
+        },{
             .name = "lossy",
             .type = QEMU_OPT_BOOL,
         },{
@@ -3480,6 +3486,10 @@ void vnc_display_open(const char *id, Error **errp)
     int saslErr;
 #endif
     int acl = 0;
+    const char *tlsacl;
+#ifdef CONFIG_VNC_SASL
+    const char *saslacl;
+#endif
     int lock_key_sync = 1;
 
     if (!vs) {
@@ -3667,6 +3677,21 @@ void vnc_display_open(const char *id, Error **errp)
         }
     }
     acl = qemu_opt_get_bool(opts, "acl", false);
+    tlsacl = qemu_opt_get(opts, "tls-acl");
+    if (acl && tlsacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'tls-acl' options");
+        goto fail;
+    }
+
+#ifdef CONFIG_VNC_SASL
+    saslacl = qemu_opt_get(opts, "sasl-acl");
+    if (acl && saslacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'sasl-acl' options");
+        goto fail;
+    }
+#endif
 
     share = qemu_opt_get(opts, "share");
     if (share) {
@@ -3696,7 +3721,9 @@ void vnc_display_open(const char *id, Error **errp)
         vs->non_adaptive = true;
     }
 
-    if (acl) {
+    if (tlsacl) {
+        vs->tlsaclname = g_strdup(tlsacl);
+    } else if (acl) {
         if (strcmp(vs->id, "default") == 0) {
             vs->tlsaclname = g_strdup("vnc.x509dname");
         } else {
@@ -3707,19 +3734,39 @@ void vnc_display_open(const char *id, Error **errp)
                           &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
-    if (acl && sasl) {
-        char *aclname;
+    if (sasl) {
+        if (saslacl) {
+            Object *container, *acl;
+            container = object_get_objects_root();
+            acl = object_resolve_path_component(container, saslacl);
+            if (!acl) {
+                error_setg(errp, "Cannot find ACL %s", saslacl);
+                goto fail;
+            }
 
-        if (strcmp(vs->id, "default") == 0) {
-            aclname = g_strdup("vnc.username");
-        } else {
-            aclname = g_strdup_printf("vnc.%s.username", vs->id);
-        }
-        vs->sasl.acl =
-            QAUTHZ(qauthz_simple_new(aclname,
-                                     QAUTHZ_SIMPLE_POLICY_DENY,
-                                     &error_abort));
-        g_free(aclname);
+            if (!object_dynamic_cast(acl, TYPE_QAUTHZ)) {
+                error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                           saslacl);
+                goto fail;
+            }
+            vs->sasl.acl = QAUTHZ(acl);
+        } else if (acl) {
+            char *aclname;
+
+            if (strcmp(vs->id, "default") == 0) {
+                aclname = g_strdup("vnc.username");
+            } else {
+                aclname = g_strdup_printf("vnc.%s.username", vs->id);
+            }
+            vs->sasl.acl =
+                QAUTHZ(qauthz_simple_new(aclname,
+                                         QAUTHZ_SIMPLE_POLICY_DENY,
+                                         &error_abort));
+            g_free(aclname);
+        }
+    } else if (saslacl) {
+        error_setg(errp, "SASL ACL provided when SASL is disabled");
+        goto fail;
     }
 #endif
 
Signature · 1 line
2.5.0
#2 lore raw

Re: [Qemu-devel] [PATCH v2 01/10] qdict: implement a qdict_crumple method for un-flattening a dict

In reply to Daniel P. Berrange
Quoted 58 lines · Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

 {
   'foo.0.bar': 'one',
   'foo.0.wizz': '1',
   'foo.1.bar': 'two',
   'foo.1.wizz': '2'

}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

 {
   'foo' => [
     { 'bar': 'one', 'wizz': '1' }
     { 'bar': 'two', 'wizz': '2' }
   ],

}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nested used when the same object is defined over QMP.

Quoted 6 lines
---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 237 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 114 +++++++++++++++++++++++
 3 files changed, 352 insertions(+)

Under the assumption that we are going to replace qdict_array_split() with this function later on: Looks good overall, some comments below, the biggest of which regarding design is me nagging again how nice step 3 could be as an own function.

Quoted 72 lines
include/qapi/qmp/qdict.h +1 -0
index 71b8eb0..8a3ac13 100644
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
qobject/qdict.c +52 -0
index 9833bd0..b0f3447 100644
@@ -682,6 +682,243 @@ void qdict_array_split(QDict *src, QList **dst)
     }
 }
 
+
+/**
+ * qdict_split_flat_key:
+ *
+ * Given a flattened key such as 'foo.0.bar', split it
+ * into two parts at the first '.' separator. Allows
+ * double dot ('..') to escape the normal separator.
+ *
+ * eg
+ *    'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ *    'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned
+ * 'prefix' string. The 'suffix' string will be left
+ * in escaped format, so it can be fed back into the
+ * qdict_split_flat_key() key as the input later.
+ */
+static void qdict_split_flat_key(const char *key, char **prefix, char **suffix)
+{
+    const char *separator;
+    size_t i, j;
+
+    /* Find first '.' separator, but if there is a pair '..'
+     * that acts as an escape, so skip over '..' */
+    separator = NULL;
+    do {
+        if (separator) {
+            separator += 2;
+        } else {
+            separator = key;
+        }
+        separator = strchr(separator, '.');
+    } while (separator && *(separator + 1) == '.');
+
+    if (separator) {
+        *prefix = g_strndup(key,
+                            separator - key);
+        *suffix = g_strdup(separator + 1);
+    } else {
+        *prefix = g_strdup(key);
+        *suffix = NULL;
+    }
+
+    /* Unescape the '..' sequence into '.' */
+    for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+        if ((*prefix)[i] == '.' &&
+            (*prefix)[i + 1] == '.') {
+            i++;
+        }
+        (*prefix)[j] = (*prefix)[i];
+    }
+    (*prefix)[j] = '\0';

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 746

Quoted 10 lines
qobject/qdict.c +10 -0
+}
+
+
+/**
+ * qdict_crumple:
+ *
+ * Reverses the flattening done by qdict_flatten by
+ * crumpling the dicts into a nested structure. Similar
+ * qdict_array_split, but copes with arbitrary nesting
+ * of dicts & arrays, not meely one level of arrays

s/meely/merely/

Quoted 10 lines
qobject/qdict.c +10 -0
+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ *   'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * =>
+ *
+ * {
+ *   'foo' => [
+ *      { 'bar': 'one', 'wizz': '1' }
+ *      { 'bar': 'two', 'wizz': '2' }

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 766

Quoted 10 lines
qobject/qdict.c +10 -0
+ *   ],
+ * }
+ *
+ */
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
+{
+    const QDictEntry *entry, *next;
+    QDict *two_level, *multi_level = NULL;
+    QObject *dst = NULL, *child;
+    bool isList = false;

*cough* :-)

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 771

Quoted 5 lines
qobject/qdict.c +5 -0
+    ssize_t list_max = -1;
+    size_t list_len = 0;
+    size_t i;
+    int64_t val;
+    char *prefix, *suffix;

These should be initialized to NULL...

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 784

Quoted 13 lines
qobject/qdict.c +13 -0
+
+    two_level = qdict_new();
+    entry = qdict_first(src);
+
+    /* Step 1: split our totally flat dict into a two level dict */
+    while (entry != NULL) {
+        next = qdict_next(src, entry);
+
+        if (qobject_type(entry->value) == QTYPE_QDICT ||
+            qobject_type(entry->value) == QTYPE_QLIST) {
+            error_setg(errp, "Value %s is not a scalar",
+                       entry->key);
+            goto error;

...otherwise this might result in uninitialized values being passed to g_free() in the error path.

Quoted 4 lines
qobject/qdict.c +4 -0
+        }
+
+        qdict_split_flat_key(entry->key, &prefix, &suffix);
+

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 798

Quoted 10 lines
qobject/qdict.c +10 -0
+        child = qdict_get(two_level, prefix);
+        if (suffix) {
+            if (child) {
+                if (qobject_type(child) != QTYPE_QDICT) {
+                    error_setg(errp, "Key %s prefix is already set as a scalar",
+                               prefix);
+                    goto error;
+                }
+            } else {
+                child = (QObject *)qdict_new();

Works, but I'd prefer QOBJECT(qdict_new()).

Quoted 46 lines
qobject/qdict.c +46 -0
+                qdict_put_obj(two_level, prefix, child);
+            }
+            qobject_incref(entry->value);
+            qdict_put_obj(qobject_to_qdict(child), suffix, entry->value);
+        } else {
+            if (child) {
+                error_setg(errp, "Key %s prefix is already set as a dict",
+                           prefix);
+                goto error;
+            }
+            qobject_incref(entry->value);
+            qdict_put_obj(two_level, prefix, entry->value);
+        }
+
+        g_free(suffix);
+        g_free(prefix);
+        suffix = prefix = NULL;
+
+        entry = next;
+    }
+
+    /* Step 2: optionally process the two level dict recursively
+     * into a multi-level dict */
+    if (recursive) {
+        multi_level = qdict_new();
+        entry = qdict_first(two_level);
+        while (entry != NULL) {
+            next = qdict_next(two_level, entry);
+
+            if (qobject_type(entry->value) == QTYPE_QDICT) {
+                child = qdict_crumple(qobject_to_qdict(entry->value),
+                                      recursive, errp);
+                if (!child) {
+                    goto error;
+                }
+
+                qdict_put_obj(multi_level, entry->key, child);
+            } else {
+                qobject_incref(entry->value);
+                qdict_put_obj(multi_level, entry->key, entry->value);
+            }
+
+            entry = next;
+        }
+        QDECREF(two_level);
+    } else {

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 854

Quoted 10 lines
qobject/qdict.c +10 -0
+        multi_level = two_level;
+    }
+    two_level = NULL;
+
+    /* Step 3: detect if we need to turn our dict into list */
+    entry = qdict_first(multi_level);
+    while (entry != NULL) {
+        next = qdict_next(multi_level, entry);
+
+        errno = 0;

This appears unnecessary to me, qemu_strtoll() works fine without.

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 857

Quoted 3 lines
qobject/qdict.c +3 -0
+        if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
+            if (!dst) {
+                dst = (QObject *)qlist_new();

Again, works just fine, but QOBJECT(qlist_new()) would be nicer.

Quoted 4 lines
qobject/qdict.c +4 -0
+                isList = true;
+            } else if (!isList) {
+                error_setg(errp,
+                           "Key '%s' is for a list, but previous key is "

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 871

Quoted 10 lines
qobject/qdict.c +10 -0
+                           "for a dict", entry->key);
+                goto error;
+            }
+            list_len++;
+            if (val > list_max) {
+                list_max = val;
+            }
+        } else {
+            if (!dst) {
+                dst = (QObject *)multi_level;

Similarly, QOBJECT(multi_level).

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 883

Quoted 12 lines
qobject/qdict.c +12 -0
+                qobject_incref(dst);
+                isList = false;
+            } else if (isList) {
+                error_setg(errp,
+                           "Key '%s' is for a dict, but previous key is "
+                           "for a list", entry->key);
+                goto error;
+            }
+        }
+
+        entry = next;
+    }

If the QDict is empty, dst will be NULL and this function will return NULL. Though nothing is leaked, is this intended behavior?

If not, I think it would be a bit simpler if the loop just yielded is_list being true or false, and then dst is set afterwards. For this to work, inside the loop we'd simply need to know whether we are in the first iteration or not (is_list may be changed in the first iteration only).

Pulling out the setting of dst would be a nice side-effect (IMO).

(And, again, I think this decision of whether the QDict should be made a QList or not can be put really nicely into a dedicated function. Besides that boolean, it only needs to return the list_size (actually, it could just return the list_size and a size of 0 means it should stay a QDict). The test whether list_len != list_max + 1 can be done inside of that function, too.)

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 889

Quoted 6 lines
qobject/qdict.c +6 -0
+
+    /* Step 4: Turn the dict into a list */
+    if (isList) {
+        if (list_len != (list_max + 1)) {
+            error_setg(errp, "List indexes are not continuous, "
+                       "saw %zu elements but %zu largest index",

The second should be %zi or %zd (unfortunately, as I have been told, some systems have sizeof(size_t) != sizeof(ssize_t)).

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 902

Quoted 13 lines
qobject/qdict.c +13 -0
+                       list_len, list_max);
+            goto error;
+        }
+
+        for (i = 0; i < list_len; i++) {
+            char *key = g_strdup_printf("%zu", i);
+
+            child = qdict_get(multi_level, key);
+            g_free(key);
+            if (!child) {
+                error_setg(errp, "Unexpected missing list entry %zu", i);
+                goto error;
+            }

Could be made an assert(child);, but doesn't need to be, of course.

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 905

Quoted 3 lines
qobject/qdict.c +3 -0
+
+            qobject_incref(child);
+            qlist_append_obj((QList *)dst, child);

As with the (QObject *) casts, qobject_to_qlist(dst) would be nicer.

Quoted 30 lines
qobject/qdict.c +16 -0
+        }
+    }
+    QDECREF(multi_level);
+
+    return dst;
+
+ error:
+    g_free(suffix);
+    g_free(prefix);
+    QDECREF(multi_level);
+    QDECREF(two_level);
+    qobject_decref(dst);
+    return NULL;
+}
+
+
 /**
  * qdict_array_entries(): Returns the number of direct array entries if the
  * sub-QDict of src specified by the prefix in subqdict (or src itself for
tests/check-qdict.c +3 -0
index a43056c..20aa1af 100644
@@ -596,6 +596,113 @@ static void qdict_join_test(void)
     QDECREF(dict2);
 }
 
+
+static void qdict_crumple_test_nonrecursive(void)
+{

tests/check-qdict.c static void qdict_join_test(void) · line 611

Quoted 10 lines
tests/check-qdict.c +10 -0
+    QDict *src, *dst, *rules;
+    QObject *child;
+
+    src = qdict_new();
+    qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+    qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+    qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+    qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+
+    dst = (QDict *)qdict_crumple(src, false, &error_abort);

It's a test, so anything is fine that works, but still... :-)

tests/check-qdict.c static void qdict_join_test(void) · line 618

Quoted 7 lines
tests/check-qdict.c +7 -0
+
+    g_assert_cmpint(qdict_size(dst), ==, 1);
+
+    child = qdict_get(dst, "rule");
+    g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+
+    rules = qdict_get_qdict(dst, "rule");

g_assert_cmpint(qdict_size(rules), ==, 4); ?

Quoted 24 lines
tests/check-qdict.c +24 -0
+
+    g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
+    g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
+    g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
+    g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
+
+    QDECREF(src);
+    QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_recursive(void)
+{
+    QDict *src, *dst, *rule;
+    QObject *child;
+    QList *rules;
+
+    src = qdict_new();
+    qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+    qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+    qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+    qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+
+    dst = (QDict *)qdict_crumple(src, true, &error_abort);

tests/check-qdict.c static void qdict_join_test(void) · line 652

Quoted 10 lines
tests/check-qdict.c +10 -0
+
+    g_assert_cmpint(qdict_size(dst), ==, 1);
+
+    child = qdict_get(dst, "rule");
+    g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+
+    rules = qdict_get_qlist(dst, "rule");
+    g_assert_cmpint(qlist_size(rules), ==, 2);
+
+    rule = qobject_to_qdict(qlist_pop(rules));

g_assert_cmpint(qdict_size(rule), ==, 2); ?

tests/check-qdict.c static void qdict_join_test(void) · line 657

Quoted 5 lines
tests/check-qdict.c +5 -0
+    g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+    g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+    QDECREF(rule);
+
+    rule = qobject_to_qdict(qlist_pop(rules));

g_assert_cmpint(qdict_size(rule), ==, 2); ?

tests/check-qdict.c static void qdict_join_test(void) · line 663

Quoted 6 lines
tests/check-qdict.c +6 -0
+    g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+    g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+    QDECREF(rule);
+
+    QDECREF(src);
+    QDECREF(dst);

QDECREF(rules);

Max

Quoted 60 lines
tests/check-qdict.c +49 -0
+}
+
+
+static void qdict_crumple_test_bad_inputs(void)
+{
+    QDict *src;
+    Error *error = NULL;
+
+    src = qdict_new();
+    /* rule.0 can't be both a string and a dict */
+    qdict_put(src, "rule.0", qstring_from_str("fred"));
+    qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+
+    g_assert(qdict_crumple(src, true, &error) == NULL);
+    g_assert(error != NULL);
+    error_free(error);
+    error = NULL;
+    QDECREF(src);
+
+    src = qdict_new();
+    /* rule can't be both a list and a dict */
+    qdict_put(src, "rule.0", qstring_from_str("fred"));
+    qdict_put(src, "rule.a", qstring_from_str("allow"));
+
+    g_assert(qdict_crumple(src, true, &error) == NULL);
+    g_assert(error != NULL);
+    error_free(error);
+    error = NULL;
+    QDECREF(src);
+
+    src = qdict_new();
+    /* The input should be flat, ie no dicts or lists */
+    qdict_put(src, "rule.0", qdict_new());
+    qdict_put(src, "rule.a", qstring_from_str("allow"));
+
+    g_assert(qdict_crumple(src, true, &error) == NULL);
+    g_assert(error != NULL);
+    error_free(error);
+    error = NULL;
+    QDECREF(src);
+}
+
 /*
  * Errors test-cases
  */
@@ -743,6 +850,13 @@ int main(int argc, char **argv)
     g_test_add_func("/errors/put_exists", qdict_put_exists_test);
     g_test_add_func("/errors/get_not_exists", qdict_get_not_exists_test);
 
+    g_test_add_func("/public/crumple/nonrecursive",
+                    qdict_crumple_test_nonrecursive);
+    g_test_add_func("/public/crumple/recursive",
+                    qdict_crumple_test_recursive);
+    g_test_add_func("/public/crumple/bad_inputs",
+                    qdict_crumple_test_bad_inputs);
+
     /* The Big one */
     if (g_test_slow()) {
         g_test_add_func("/stress/test", qdict_stress_test);
#3 lore raw

Re: [Qemu-devel] [PATCH v2 01/10] qdict: implement a qdict_crumple method for un-flattening a dict

Quoted 81 lines · Max Reitz
Quoted 58 lines · Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

 {
   'foo.0.bar': 'one',
   'foo.0.wizz': '1',
   'foo.1.bar': 'two',
   'foo.1.wizz': '2'

}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

 {
   'foo' => [
     { 'bar': 'one', 'wizz': '1' }
     { 'bar': 'two', 'wizz': '2' }
   ],

}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nested used when the same object is defined over QMP.

Quoted 6 lines
---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 237 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 114 +++++++++++++++++++++++
 3 files changed, 352 insertions(+)

Under the assumption that we are going to replace qdict_array_split() with this function later on: Looks good overall, some comments below, the biggest of which regarding design is me nagging again how nice step 3 could be as an own function.

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 766

Quoted 6 lines
qobject/qdict.c +6 -0
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
+{
+    const QDictEntry *entry, *next;
+    QDict *two_level, *multi_level = NULL;
+    QObject *dst = NULL, *child;
+    bool isList = false;

*cough* :-)

Quoted 7 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 771

Quoted 5 lines
qobject/qdict.c +5 -0
+    ssize_t list_max = -1;
+    size_t list_len = 0;
+    size_t i;
+    int64_t val;
+    char *prefix, *suffix;

These should be initialized to NULL...

Opps, yes.

Quoted 16 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 784

Quoted 12 lines
qobject/qdict.c +12 -0
+    two_level = qdict_new();
+    entry = qdict_first(src);
+
+    /* Step 1: split our totally flat dict into a two level dict */
+    while (entry != NULL) {
+        next = qdict_next(src, entry);
+
+        if (qobject_type(entry->value) == QTYPE_QDICT ||
+            qobject_type(entry->value) == QTYPE_QLIST) {
+            error_setg(errp, "Value %s is not a scalar",
+                       entry->key);
+            goto error;

...otherwise this might result in uninitialized values being passed to g_free() in the error path.

Quoted 16 lines
Quoted 4 lines
qobject/qdict.c +4 -0
+        }
+
+        qdict_split_flat_key(entry->key, &prefix, &suffix);
+

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 798

Quoted 10 lines
qobject/qdict.c +10 -0
+        child = qdict_get(two_level, prefix);
+        if (suffix) {
+            if (child) {
+                if (qobject_type(child) != QTYPE_QDICT) {
+                    error_setg(errp, "Key %s prefix is already set as a scalar",
+                               prefix);
+                    goto error;
+                }
+            } else {
+                child = (QObject *)qdict_new();

Works, but I'd prefer QOBJECT(qdict_new()).

Ok.

Quoted 8 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 854

Quoted 6 lines
qobject/qdict.c +6 -0
+    /* Step 3: detect if we need to turn our dict into list */
+    entry = qdict_first(multi_level);
+    while (entry != NULL) {
+        next = qdict_next(multi_level, entry);
+
+        errno = 0;

This appears unnecessary to me, qemu_strtoll() works fine without.

Left over from when i used bare strtoll.

Quoted 5 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 857

Quoted 3 lines
qobject/qdict.c +3 -0
+        if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
+            if (!dst) {
+                dst = (QObject *)qlist_new();

Again, works just fine, but QOBJECT(qlist_new()) would be nicer.

Ok

Quoted 15 lines
Quoted 4 lines
qobject/qdict.c +4 -0
+                isList = true;
+            } else if (!isList) {
+                error_setg(errp,
+                           "Key '%s' is for a list, but previous key is "

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 871

Quoted 10 lines
qobject/qdict.c +10 -0
+                           "for a dict", entry->key);
+                goto error;
+            }
+            list_len++;
+            if (val > list_max) {
+                list_max = val;
+            }
+        } else {
+            if (!dst) {
+                dst = (QObject *)multi_level;
Quoted 1 line

Similarly, QOBJECT(multi_level).

Ok.

Quoted 21 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 883

Quoted 12 lines
qobject/qdict.c +12 -0
+                qobject_incref(dst);
+                isList = false;
+            } else if (isList) {
+                error_setg(errp,
+                           "Key '%s' is for a dict, but previous key is "
+                           "for a list", entry->key);
+                goto error;
+            }
+        }
+
+        entry = next;
+    }

If the QDict is empty, dst will be NULL and this function will return NULL. Though nothing is leaked, is this intended behavior?

If not, I think it would be a bit simpler if the loop just yielded is_list being true or false, and then dst is set afterwards. For this to work, inside the loop we'd simply need to know whether we are in the first iteration or not (is_list may be changed in the first iteration only).

Quoted 8 lines

Pulling out the setting of dst would be a nice side-effect (IMO).

(And, again, I think this decision of whether the QDict should be made a QList or not can be put really nicely into a dedicated function. Besides that boolean, it only needs to return the list_size (actually, it could just return the list_size and a size of 0 means it should stay a QDict). The test whether list_len != list_max + 1 can be done inside of that function, too.)

Yes, I have split the code out now and it is nicer like that. I also added a test for the empty dict case.

Quoted 26 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 889

Quoted 6 lines
qobject/qdict.c +6 -0
+
+    /* Step 4: Turn the dict into a list */
+    if (isList) {
+        if (list_len != (list_max + 1)) {
+            error_setg(errp, "List indexes are not continuous, "
+                       "saw %zu elements but %zu largest index",

The second should be %zi or %zd (unfortunately, as I have been told, some systems have sizeof(size_t) != sizeof(ssize_t)).

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 902

Quoted 13 lines
qobject/qdict.c +13 -0
+                       list_len, list_max);
+            goto error;
+        }
+
+        for (i = 0; i < list_len; i++) {
+            char *key = g_strdup_printf("%zu", i);
+
+            child = qdict_get(multi_level, key);
+            g_free(key);
+            if (!child) {
+                error_setg(errp, "Unexpected missing list entry %zu", i);
+                goto error;
+            }

Could be made an assert(child);, but doesn't need to be, of course.

Quoted 5 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 905

Quoted 3 lines
qobject/qdict.c +3 -0
+
+            qobject_incref(child);
+            qlist_append_obj((QList *)dst, child);

As with the (QObject *) casts, qobject_to_qlist(dst) would be nicer.

Ok

Quoted 71 lines

tests/check-qdict.c static void qdict_join_test(void) · line 611

Quoted 12 lines
tests/check-qdict.c +12 -0
+static void qdict_crumple_test_nonrecursive(void)
+{
+    QDict *src, *dst, *rules;
+    QObject *child;
+
+    src = qdict_new();
+    qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+    qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+    qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+    qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+
+    dst = (QDict *)qdict_crumple(src, false, &error_abort);

It's a test, so anything is fine that works, but still... :-)

tests/check-qdict.c static void qdict_join_test(void) · line 618

Quoted 7 lines
tests/check-qdict.c +7 -0
+
+    g_assert_cmpint(qdict_size(dst), ==, 1);
+
+    child = qdict_get(dst, "rule");
+    g_assert_cmpint(qobject_type(child), ==, QTYPE_QDICT);
+
+    rules = qdict_get_qdict(dst, "rule");

g_assert_cmpint(qdict_size(rules), ==, 4); ?

Quoted 24 lines
tests/check-qdict.c +24 -0
+
+    g_assert_cmpstr("fred", ==, qdict_get_str(rules, "0.match"));
+    g_assert_cmpstr("allow", ==, qdict_get_str(rules, "0.policy"));
+    g_assert_cmpstr("bob", ==, qdict_get_str(rules, "1.match"));
+    g_assert_cmpstr("deny", ==, qdict_get_str(rules, "1.policy"));
+
+    QDECREF(src);
+    QDECREF(dst);
+}
+
+
+static void qdict_crumple_test_recursive(void)
+{
+    QDict *src, *dst, *rule;
+    QObject *child;
+    QList *rules;
+
+    src = qdict_new();
+    qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+    qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+    qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+    qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+
+    dst = (QDict *)qdict_crumple(src, true, &error_abort);

tests/check-qdict.c static void qdict_join_test(void) · line 652

Quoted 10 lines
tests/check-qdict.c +10 -0
+
+    g_assert_cmpint(qdict_size(dst), ==, 1);
+
+    child = qdict_get(dst, "rule");
+    g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+
+    rules = qdict_get_qlist(dst, "rule");
+    g_assert_cmpint(qlist_size(rules), ==, 2);
+
+    rule = qobject_to_qdict(qlist_pop(rules));

g_assert_cmpint(qdict_size(rule), ==, 2); ?

tests/check-qdict.c static void qdict_join_test(void) · line 657

Quoted 5 lines
tests/check-qdict.c +5 -0
+    g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+    g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+    QDECREF(rule);
+
+    rule = qobject_to_qdict(qlist_pop(rules));

g_assert_cmpint(qdict_size(rule), ==, 2); ?

Quoted 8 lines

tests/check-qdict.c static void qdict_join_test(void) · line 663

Quoted 6 lines
tests/check-qdict.c +6 -0
+    g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+    g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+    QDECREF(rule);
+
+    QDECREF(src);
+    QDECREF(dst);

QDECREF(rules);

rules is a borrowed reference, so we can't decrement it.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#4 lore raw

Re: [Qemu-devel] [PATCH v2 01/10] qdict: implement a qdict_crumple method for un-flattening a dict

Quoted 2 lines · Daniel P. Berrange
Quoted 1 line · Max Reitz

On 07.03.2016 16:43, Daniel P. Berrange wrote:

[...]

Quoted 42 lines
Quoted 42 lines
Quoted 13 lines
tests/check-qdict.c +13 -0
+static void qdict_crumple_test_recursive(void)
+{
+    QDict *src, *dst, *rule;
+    QObject *child;
+    QList *rules;
+
+    src = qdict_new();
+    qdict_put(src, "rule.0.match", qstring_from_str("fred"));
+    qdict_put(src, "rule.0.policy", qstring_from_str("allow"));
+    qdict_put(src, "rule.1.match", qstring_from_str("bob"));
+    qdict_put(src, "rule.1.policy", qstring_from_str("deny"));
+
+    dst = (QDict *)qdict_crumple(src, true, &error_abort);

tests/check-qdict.c static void qdict_join_test(void) · line 652

Quoted 10 lines
tests/check-qdict.c +10 -0
+
+    g_assert_cmpint(qdict_size(dst), ==, 1);
+
+    child = qdict_get(dst, "rule");
+    g_assert_cmpint(qobject_type(child), ==, QTYPE_QLIST);
+
+    rules = qdict_get_qlist(dst, "rule");
+    g_assert_cmpint(qlist_size(rules), ==, 2);
+
+    rule = qobject_to_qdict(qlist_pop(rules));

g_assert_cmpint(qdict_size(rule), ==, 2); ?

tests/check-qdict.c static void qdict_join_test(void) · line 657

Quoted 5 lines
tests/check-qdict.c +5 -0
+    g_assert_cmpstr("fred", ==, qdict_get_str(rule, "match"));
+    g_assert_cmpstr("allow", ==, qdict_get_str(rule, "policy"));
+    QDECREF(rule);
+
+    rule = qobject_to_qdict(qlist_pop(rules));

g_assert_cmpint(qdict_size(rule), ==, 2); ?

Quoted 6 lines
tests/check-qdict.c +6 -0
+    g_assert_cmpstr("bob", ==, qdict_get_str(rule, "match"));
+    g_assert_cmpstr("deny", ==, qdict_get_str(rule, "policy"));
+    QDECREF(rule);
+
+    QDECREF(src);
+    QDECREF(dst);
Quoted 3 lines
Quoted 1 line

QDECREF(rules);

rules is a borrowed reference, so we can't decrement it.

Oops, right. :-)

I suppose it looked so tempting to me because of the QDECREF(rule) for the pop'ed rules.

Max