Skip to contentSkip to conversation

Provide a QOM-based authorization API

PATCH Superseded Daniel P. Berrange 3 participants 32 messages Mar 10, 2016 – Mar 23, 2016

Subsystems (2)

v3 of 14 Superseded Latest is v14 →

lore
Share & tools
lore
#0 lore raw

v3 cover Provide a QOM-based authorization API

Many years ago I was responsible for adding the 'qemu_acl' type and associated HMP commands. Looking back at it now, it is quite a poor facility with a couple of bad limitations. First, the responsibility for creating the ACLs was left with the QEMU network service (VNC server was only thing ever doing it). This meant you could not share ACLs across multiple services. Second, there was no way to populate ACLs on the command line, you had no choice but to use the HMP commands. Third, the API was hardcoded around the idea of an in-QEMU implementation, leaving no scope for plugging in alternative implementations backed by, for exmaple, LDAP or PAM.

This series introduces a much better authorization API design to QEMU that addresses all these problems, and maintains back compatibility. It of course is based on the QOM framework, so that immediately gives us ability to create objects via the CLI, HMP or QMP. There is an abstract base clss "QAuthZ" which defines the basic API for QEMU network services to use, and a specific implementation "QAuthZ" simple which replicates the functionality of 'qemu_acl'. It is thus possible to add other impls, without changing any other part of QEMU in the future. Finally, the user is responsible for creating the ACL objects, so they can have one ACL associated with all their TLS enabled network services.

There was only one small problem with this, specifically the -object CLI arg and HMP 'object_add' command had no way to let the user specify non-scalar properties for objects. eg if an object had a property which is a list of structs, you are out of luck if you want to create it without using QMP.

Thus the first three patches do some work around QAPI / QOM to make it possible to specify non-scalar properties with the -object CLI arg and HMP 'object_add' command. See the respective patches for illustration of the syntax used.

The patches 4 and 5 introduce the new base class and specific implementation.

Patch 6 kills the old qemu_acl code, updating any existing callers of it to use the QAuthZSimple QOM class instead.

Patches 7-10 add support for associating ACLs with the network services supporting TLS encryption (NBD, chardev and VNC).

Aside from the outstanding migration TLS patches, this series wraps up the feature based work I have for TLS in this release cycle.

Changed in v3:

- Created separate qdict_list_size method (Max)
- Added unit tests for case of empty dict (Max)
- Fix variable names to use underscore separator (Max)
- Fix potential free of uninitialized variables (Max)
- Use QObject APIs for casts, instead of C type casts (Max)

Changed in v2:

- Adapt to changes in qapi visitor APIs
- Add a 'bool recursive' flag to qdict_crumple (Max)
- Fix memory leaks in qdict_crumple (Max)
- Split out key splitting code from qdict_crumple (Max)
- Use saner variable names in qdict_crumple (Max)
- Added some tests for bad inputs to qdict_crumple

Daniel P. Berrange (10):
qdict: implement a qdict_crumple method for un-flattening a dict
qapi: allow QmpInputVisitor to auto-cast types
qom: support arbitrary non-scalar properties with -object
util: add QAuthZ object as an authorization base class
util: add QAuthZSimple object type for a simple access control list
acl: delete existing ACL implementation
qemu-nbd: add support for ACLs for TLS clients
nbd: allow an ACL to be set with nbd-server-start QMP command
chardev: add support for ACLs for TLS clients
vnc: allow specifying a custom ACL object name

 MAINTAINERS                      |   7 +
 Makefile                         |   9 +-
 Makefile.objs                    |   2 +
 Makefile.target                  |   2 +
 blockdev-nbd.c                   |  10 +-
 crypto/tlssession.c              |  28 +++-
 hmp.c                            |  20 +--
 include/qapi/qmp-input-visitor.h |   3 +
 include/qapi/qmp/qdict.h         |   1 +
 include/qemu/acl.h               |  74 ----------
 include/qemu/authz-simple.h      | 107 ++++++++++++++
 include/qemu/authz.h             |  81 +++++++++++
 monitor.c                        | 161 +++++++++++++--------
 qapi-schema.json                 |   8 +-
 qapi/block.json                  |   4 +-
 qapi/qmp-input-visitor.c         |  96 +++++++++++--
 qapi/util.json                   |  31 ++++
 qemu-char.c                      |  11 +-
 qemu-nbd.c                       |  13 +-
 qemu-nbd.texi                    |   4 +
 qmp-commands.hx                  |   2 +-
 qobject/qdict.c                  | 267 +++++++++++++++++++++++++++++++++++
 qom/object_interfaces.c          |  20 ++-
 tests/.gitignore                 |   1 +
 tests/Makefile                   |   5 +-
 tests/check-qdict.c              | 143 +++++++++++++++++++
 tests/check-qom-proplist.c       | 295 ++++++++++++++++++++++++++++++++++++++-
 tests/test-authz-simple.c        | 156 +++++++++++++++++++++
 tests/test-crypto-tlssession.c   |  13 +-
 tests/test-io-channel-tls.c      |  14 +-
 tests/test-qmp-input-visitor.c   | 115 ++++++++++++++-
 ui/vnc-auth-sasl.c               |   2 +-
 ui/vnc-auth-sasl.h               |   4 +-
 ui/vnc.c                         |  76 ++++++++--
 util/Makefile.objs               |   4 +-
 util/acl.c                       | 188 -------------------------
 util/authz-simple.c              | 256 +++++++++++++++++++++++++++++++++
 util/authz.c                     |  46 ++++++
 38 files changed, 1876 insertions(+), 403 deletions(-)
 delete mode 100644 include/qemu/acl.h
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 include/qemu/authz.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 delete mode 100644 util/acl.c
 create mode 100644 util/authz-simple.c
 create mode 100644 util/authz.c
Signature · 1 line
2.5.0
#1 lore raw

PATCH v3 1/10 qdict: implement a qdict_crumple method for un-flattening a dict

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

If the keys are a mixture of numeric and non-numeric, or the numeric keys are not in strictly ascending order, an error will be reported.

As an example, a flat dict containing

{
'foo.0.bar': 'one',
'foo.0.wizz': '1',
'foo.1.bar': 'two',
'foo.1.wizz': '2'
}

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

{
'foo' => [

     { 'bar': 'one', 'wizz': '1' }
     { 'bar': 'two', 'wizz': '2' }

],
}

If the key is intended to contain a literal '.', then it must be escaped as '..'. ie a flat dict

  {
     'foo..bar': 'wizz',
     'bar.foo..bar': 'eek',
     'bar.hello': 'world'
  }

Will end up as

  {
     'foo.bar': 'wizz',
     'bar': {
        'foo.bar': 'eek',
        'hello': 'world'
     }
  }

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nested used when the same object is defined over QMP.

---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 267 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 143 +++++++++++++++++++++++++
 3 files changed, 411 insertions(+)
include/qapi/qmp/qdict.h +1 -0
index 71b8eb0..8a3ac13 100644
@@ -73,6 +73,7 @@ void qdict_flatten(QDict *qdict);
 void qdict_extract_subqdict(QDict *src, QDict **dst, const char *start);
 void qdict_array_split(QDict *src, QList **dst);
 int qdict_array_entries(QDict *src, const char *subqdict);
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp);
 
 void qdict_join(QDict *dest, QDict *src, bool overwrite);
 
qobject/qdict.c +267 -0

273 lines not on this page · raw message

tests/check-qdict.c +143 -0

155 lines not on this page · raw message

Signature · 1 line
2.5.0
#2 lore raw

PATCH v3 2/10 qapi: allow QmpInputVisitor to auto-cast types

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This extends it so that QString is optionally permitted for any of the non-string scalar types. This behaviour is turned on by requesting the 'autocast' flag in the constructor.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

---
 include/qapi/qmp-input-visitor.h |   3 +
 qapi/qmp-input-visitor.c         |  96 +++++++++++++++++++++++++++-----
 tests/test-qmp-input-visitor.c   | 115 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 196 insertions(+), 18 deletions(-)
include/qapi/qmp-input-visitor.h +3 -0
index 3ed499c..c25cb7c 100644
@@ -21,6 +21,9 @@ typedef struct QmpInputVisitor QmpInputVisitor;
 
 QmpInputVisitor *qmp_input_visitor_new(QObject *obj);
 QmpInputVisitor *qmp_input_visitor_new_strict(QObject *obj);
+QmpInputVisitor *qmp_input_visitor_new_full(QObject *obj,
+                                            bool strict,
+                                            bool autocast);
 
 void qmp_input_visitor_cleanup(QmpInputVisitor *v);
 
qapi/qmp-input-visitor.c +81 -15

147 lines not on this page · raw message

tests/test-qmp-input-visitor.c +112 -3

173 lines not on this page · raw message

Signature · 1 line
2.5.0
#6 lore raw

PATCH v3 3/10 qom: support arbitrary non-scalar properties with -object

In reply to Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.

The previously added qdict_crumple() method is able to take a qdict containing a flat set of properties and turn that into a arbitrarily nested set of dicts and lists. By combining qemu_opts_to_qdict and qdict_crumple() together, we can turn the opt string into a data structure that is practically identical to that passed over QMP when defining an object. The only difference is that all the scalar values are represented as strings, rather than strings, ints and bools. This is sufficient to let us replace the OptsVisitor with the QMPInputVisitor for use with -object.

Thus -object can now support non-scalar properties, for example the QMP object

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
	  { "bar": "one", "wizz": "1" },
	  { "bar": "two", "wizz": "2" }
        ]
      }
    }

}

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.

  (hmp) object_add demo,id=demo0,\
                   foo.0.bar=one,foo.0.wizz=1,\
		   foo.1.bar=two,foo.1.wizz=2
---
 hmp.c                      |  18 +--
 qom/object_interfaces.c    |  20 ++-
 tests/check-qom-proplist.c | 295 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 313 insertions(+), 20 deletions(-)
hmp.c +5 -13
index 5b6084a..7a98726 100644
@@ -25,7 +25,7 @@
 #include "qemu/sockets.h"
 #include "monitor/monitor.h"
 #include "monitor/qdev.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/string-output-visitor.h"
 #include "qapi/util.h"
@@ -1673,20 +1673,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
 void hmp_object_add(Monitor *mon, const QDict *qdict)
 {
     Error *err = NULL;
-    QemuOpts *opts;
-    OptsVisitor *ov;
+    QmpInputVisitor *qiv;
     Object *obj = NULL;
 
-    opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
-    if (err) {
-        hmp_handle_error(mon, &err);
-        return;
-    }
-
-    ov = opts_visitor_new(opts);
-    obj = user_creatable_add(qdict, opts_get_visitor(ov), &err);
-    opts_visitor_cleanup(ov);
-    qemu_opts_del(opts);
+    qiv = qmp_input_visitor_new_full((QObject *)qdict, true, true);
+    obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), &err);
+    qmp_input_visitor_cleanup(qiv);
 
     if (err) {
         hmp_handle_error(mon, &err);
qom/object_interfaces.c +14 -6
index c2f6e29..9c41730 100644
@@ -1,9 +1,9 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 #include "qemu/module.h"
+#include "qemu/option.h"
 #include "qapi-visit.h"
-#include "qapi/qmp-output-visitor.h"
-#include "qapi/opts-visitor.h"
+#include "qapi/qmp-input-visitor.h"
 
 void user_creatable_complete(Object *obj, Error **errp)
 {
@@ -120,6 +120,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
     obj = object_new(type);
     if (qdict) {
         for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+
             object_property_set(obj, v, e->key, &local_err);
             if (local_err) {
                 goto out;
@@ -151,15 +152,22 @@ out:
 
 Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
 {
-    OptsVisitor *ov;
+    QmpInputVisitor *qiv;
     QDict *pdict;
+    QObject *pobj;
     Object *obj = NULL;
 
-    ov = opts_visitor_new(opts);
     pdict = qemu_opts_to_qdict(opts, NULL);
+    pobj = qdict_crumple(pdict, true, errp);
+    if (!pobj) {
+        goto cleanup;
+    }
+    qiv = qmp_input_visitor_new_full(pobj, true, true);
 
-    obj = user_creatable_add(pdict, opts_get_visitor(ov), errp);
-    opts_visitor_cleanup(ov);
+    obj = user_creatable_add((QDict *)pobj, qmp_input_get_visitor(qiv), errp);
+    qmp_input_visitor_cleanup(qiv);
+    qobject_decref(pobj);
+ cleanup:
     QDECREF(pdict);
     return obj;
 }
tests/check-qom-proplist.c +294 -1

387 lines not on this page · raw message

Signature · 1 line
2.5.0
#20 lore raw

PATCH v3 7/10 qemu-nbd: add support for ACLs for TLS clients

In reply to Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly weak bar.

This adds a '--tls-acl ACL-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the NBD server.

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

  qemu-nbd -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                   endpoint=server,verify-peer=yes \
           -object authz-simple,id=acl0,policy=deny,\
	           rules.0.match=*CN=fred,rules.0.policy=allow \
           -tls-creds tls0 \
           -tls-acl acl0
	   ....other qemu-nbd args...
---
 qemu-nbd.c    | 13 ++++++++++++-
 qemu-nbd.texi |  4 ++++
 2 files changed, 16 insertions(+), 1 deletion(-)
qemu-nbd.c +12 -1
index a5c1d95..d70960f 100644
@@ -42,6 +42,7 @@
 #define QEMU_NBD_OPT_OBJECT        260
 #define QEMU_NBD_OPT_TLSCREDS      261
 #define QEMU_NBD_OPT_IMAGE_OPTS    262
+#define QEMU_NBD_OPT_TLSACL        263
 
 static NBDExport *exp;
 static bool newproto;
@@ -55,6 +56,7 @@ static int nb_fds;
 static QIOChannelSocket *server_ioc;
 static int server_watch = -1;
 static QCryptoTLSCreds *tlscreds;
+static const char *tlsacl;
 
 static void usage(const char *name)
 {
@@ -344,7 +346,7 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition cond, gpointer opaque)
     nb_fds++;
     nbd_update_server_watch();
     nbd_client_new(newproto ? NULL : exp, cioc,
-                   tlscreds, NULL, nbd_client_closed);
+                   tlscreds, tlsacl, nbd_client_closed);
     object_unref(OBJECT(cioc));
 
     return TRUE;
@@ -488,6 +490,7 @@ int main(int argc, char **argv)
         { "export-name", required_argument, NULL, 'x' },
         { "tls-creds", required_argument, NULL, QEMU_NBD_OPT_TLSCREDS },
         { "image-opts", no_argument, NULL, QEMU_NBD_OPT_IMAGE_OPTS },
+        { "tls-acl", no_argument, NULL, QEMU_NBD_OPT_TLSACL },
         { NULL, 0, NULL, 0 }
     };
     int ch;
@@ -689,6 +692,9 @@ int main(int argc, char **argv)
         case QEMU_NBD_OPT_IMAGE_OPTS:
             imageOpts = true;
             break;
+        case QEMU_NBD_OPT_TLSACL:
+            tlsacl = optarg;
+            break;
         }
     }
 
@@ -725,6 +731,11 @@ int main(int argc, char **argv)
                          error_get_pretty(local_err));
             exit(EXIT_FAILURE);
         }
+    } else {
+        if (tlsacl) {
+            error_report("--tls-acl is not permitted without --tls-creds");
+            exit(EXIT_FAILURE);
+        }
     }
 
     if (disconnect) {
qemu-nbd.texi +4 -0
index 9f23343..69f32cb 100644
@@ -86,6 +86,10 @@ the new style NBD protocol negotiation
 Enable mandatory TLS encryption for the server by setting the ID
 of the TLS credentials object previously created with the --object
 option.
+@item --tls-acl=ID
+Specify the ID of a qauthz object previously created with the
+--object option. This will be used to authorize users who
+connect against their x509 distinguish name.
 @item -v, --verbose
 Display extra debugging information
 @item -h, --help
Signature · 1 line
2.5.0
#11 lore raw

PATCH v3 4/10 util: add QAuthZ object as an authorization base class

In reply to Daniel P. Berrange

The current qemu_acl module provides a simple access control list facility inside QEMU, which is used via a set of monitor commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially integrate with external authorization systems like polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be subclassed for creating different authorization providers.

---
 MAINTAINERS          |  7 +++++
 Makefile             |  1 +
 Makefile.objs        |  2 ++
 Makefile.target      |  2 ++
 include/qemu/authz.h | 81 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 util/Makefile.objs   |  2 ++
 util/authz.c         | 46 +++++++++++++++++++++++++++++
 7 files changed, 141 insertions(+)
 create mode 100644 include/qemu/authz.h
 create mode 100644 util/authz.c
MAINTAINERS +7 -0
index dc0aa54..73bc431 100644
@@ -1301,6 +1301,13 @@ F: include/qemu/throttle.h
 F: util/throttle.c
 L: [email protected]
 
+Authorization
+M: Daniel P. Berrange <[email protected]>
+S: Maintained
+F: util/authz*
+F: include/qemu/authz*
+F: tests/test-authz-*
+
 Usermode Emulation
 ------------------
 Overall
Makefile +1 -0
index 70e3ebc..903dc35 100644
@@ -150,6 +150,7 @@ endif
 dummy := $(call unnest-vars,, \
                 stub-obj-y \
                 util-obj-y \
+                util-qom-obj-y \
                 qga-obj-y \
                 ivshmem-client-obj-y \
                 ivshmem-server-obj-y \
Makefile.objs +2 -0
index fbcaa74..8bc9a77 100644
@@ -4,6 +4,8 @@ stub-obj-y = stubs/
 util-obj-y = util/ qobject/ qapi/
 util-obj-y += qmp-introspect.o qapi-types.o qapi-visit.o qapi-event.o
 
+util-qom-obj-y += util/
+
 #######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img
 
Makefile.target +2 -0
index 34ddb7e..9728f86 100644
@@ -171,6 +171,7 @@ include $(SRC_PATH)/Makefile.objs
 dummy := $(call unnest-vars,,target-obj-y)
 target-obj-y-save := $(target-obj-y)
 dummy := $(call unnest-vars,.., \
+               util-qom-obj-y \
                block-obj-y \
                block-obj-m \
                crypto-obj-y \
@@ -183,6 +184,7 @@ target-obj-y := $(target-obj-y-save)
 all-obj-y += $(common-obj-y)
 all-obj-y += $(target-obj-y)
 all-obj-y += $(qom-obj-y)
+all-obj-y += $(util-qom-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(block-obj-y)
 all-obj-$(CONFIG_USER_ONLY) += $(crypto-aes-obj-y)
 all-obj-$(CONFIG_SOFTMMU) += $(crypto-obj-y)
include/qemu/authz.h new file +81 -0

81 lines not on this page · raw message

util/Makefile.objs +2 -0
index a8a777e..b29fb45 100644
@@ -32,3 +32,5 @@ util-obj-y += buffer.o
 util-obj-y += timed-average.o
 util-obj-y += base64.o
 util-obj-y += log.o
+
+util-qom-obj-y += authz.o
util/authz.c new file +46 -0

46 lines not on this page · raw message

Signature · 1 line
2.5.0
#15 lore raw

PATCH v3 5/10 util: add QAuthZSimple object type for a simple access control list

In reply to Daniel P. Berrange

Add a QAuthZSimple object type that implements the QAuthZ interface. This simple built-in implementation maintains a trivial access control list with a sequence of match rules and a final default policy. This replicates the functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor, the syntax used would be

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "authz-simple",
      "id": "auth0",
      "parameters": {
        "rules": [
           { "match": "fred", "policy": "allow" },
           { "match": "bob", "policy": "allow" },
           { "match": "danb", "policy": "deny" },
           { "match": "dan*", "policy": "allow" }
        ],
        "policy": "deny"
      }
    }
  }

Or via the -object command line

  $QEMU \
     -object authz-simple,id=acl0,policy=deny,\
             match.0.name=fred,match.0.policy=allow, \
             match.1.name=bob,match.1.policy=allow, \
             match.2.name=danb,match.2.policy=deny, \
             match.3.name=dan*,match.3.policy=allow

This sets up an authorization rule that allows 'fred', 'bob' and anyone whose name starts with 'dan', except for 'danb'. Everyone unmatched is denied.

---
 Makefile                    |   2 +-
 include/qemu/authz-simple.h | 107 ++++++++++++++++++
 qapi-schema.json            |   6 +-
 qapi/util.json              |  31 ++++++
 tests/.gitignore            |   1 +
 tests/Makefile              |   3 +
 tests/test-authz-simple.c   | 156 +++++++++++++++++++++++++++
 util/Makefile.objs          |   1 +
 util/authz-simple.c         | 256 ++++++++++++++++++++++++++++++++++++++++++++
 9 files changed, 561 insertions(+), 2 deletions(-)
 create mode 100644 include/qemu/authz-simple.h
 create mode 100644 qapi/util.json
 create mode 100644 tests/test-authz-simple.c
 create mode 100644 util/authz-simple.c
Makefile +1 -1
index 903dc35..60ad13e 100644
@@ -274,7 +274,7 @@ qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
                $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
                $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
                $(SRC_PATH)/qapi/crypto.json $(SRC_PATH)/qapi/rocker.json \
-               $(SRC_PATH)/qapi/trace.json
+               $(SRC_PATH)/qapi/trace.json $(SRC_PATH)/qapi/util.json
 
 qapi-types.c qapi-types.h :\
 $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
include/qemu/authz-simple.h new file +107 -0

107 lines not on this page · raw message

qapi-schema.json +5 -1
index 362c9d8..b6769de 100644
@@ -5,6 +5,9 @@
 # QAPI common definitions
 { 'include': 'qapi/common.json' }
 
+# QAPI util definitions
+{ 'include': 'qapi/util.json' }
+
 # QAPI crypto definitions
 { 'include': 'qapi/crypto.json' }
 
@@ -3652,7 +3655,8 @@
 # Since 2.5
 ##
 { 'struct': 'DummyForceArrays',
-  'data': { 'unused': ['X86CPUFeatureWordInfo'] } }
+  'data': { 'unused': ['X86CPUFeatureWordInfo'],
+            'iamalsounused': ['QAuthZSimpleRule'] } }
 
 
 ##
qapi/util.json new file +31 -0
new file mode 100644
index 0000000..d903497
@@ -0,0 +1,31 @@
+# -*- Mode: Python -*-
+#
+# QAPI util definitions
+
+##
+# QAuthZSimplePolicy:
+#
+# The authorization policy result
+#
+# @deny: deny access
+# @allow: allow access
+#
+# Since: 2.6
+##
+{ 'enum': 'QAuthZSimplePolicy',
+  'prefix': 'QAUTHZ_SIMPLE_POLICY',
+  'data': ['deny', 'allow']}
+
+##
+# QAuthZSimpleRule:
+#
+# A single authorization rule.
+#
+# @match: a glob to match against a user identity
+# @policy: the result to return if @match evaluates to true
+#
+# Since: 2.6
+##
+{ 'struct': 'QAuthZSimpleRule',
+  'data': {'match': 'str',
+           'policy': 'QAuthZSimplePolicy'}}
tests/.gitignore +1 -0
index 787c95c..1e73a2f 100644
@@ -8,6 +8,7 @@ check-qom-interface
 check-qom-proplist
 rcutorture
 test-aio
+test-authz-simple
 test-base64
 test-bitops
 test-blockjob-txn
tests/Makefile +3 -0
index cd4bbd4..7a1d959 100644
@@ -92,6 +92,7 @@ check-unit-$(CONFIG_GNUTLS) += tests/test-io-channel-tls$(EXESUF)
 check-unit-y += tests/test-io-channel-command$(EXESUF)
 check-unit-y += tests/test-io-channel-buffer$(EXESUF)
 check-unit-y += tests/test-base64$(EXESUF)
+check-unit-y += tests/test-authz-simple$(EXESUF)
 
 check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh
 
@@ -430,6 +431,8 @@ tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
 	$(test-util-obj-y)
 tests/test-base64$(EXESUF): tests/test-base64.o \
 	libqemuutil.a libqemustub.a
+tests/test-authz-simple$(EXESUF): tests/test-authz-simple.o \
+	libqemuutil.a libqemustub.a $(util-qom-obj-y) $(qom-obj-y)
 
 tests/test-qapi-types.c tests/test-qapi-types.h :\
 $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
tests/test-authz-simple.c new file +156 -0

156 lines not on this page · raw message

util/Makefile.objs +1 -0
index b29fb45..4870905 100644
@@ -34,3 +34,4 @@ util-obj-y += base64.o
 util-obj-y += log.o
 
 util-qom-obj-y += authz.o
+util-qom-obj-y += authz-simple.o
util/authz-simple.c new file +256 -0

256 lines not on this page · raw message

Signature · 1 line
2.5.0
#27 lore raw

PATCH v3 10/10 vnc: allow specifying a custom ACL object name

In reply to Daniel P. Berrange

The VNC server has historically had support for ACLs to check both the SASL username and the TLS x509 distinguished name. The VNC server was responsible for creating the initial ACL, and the client app was then responsible for populating it with rules using the HMP 'acl_add' command.

This is not satisfactory for a variety of reasons. There is no way to populate the ACLs from the command line, users are forced to use the HMP. With multiple network services all supporting TLS and ACLs now, it is desirable to be able to define a single ACL that is referenced by all services.

To address these limitations, two new options are added to the VNC server CLI. The 'tls-acl' option takes the ID of a QAuthZ object to use for checking TLS x509 distinguished names, and the 'sasl-acl' option takes the ID of another object to use for checking SASL usernames.

In this example, we setup two ACLs. The first allows any client with a certificate issued by the 'RedHat' organization in the 'London' locality. The second ACL allows clients with either the '[email protected]' or '[email protected]' kerberos usernames. Both ACLs must pass for the user to be allowed.

    $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                  endpoint=server,verify-peer=yes \
          -object authz-simple,id=acl0,policy=deny,\
                  rules.0.match=O=RedHat,,L=London,rules.0.policy=allow \
          -object authz-simple,id=acl0,policy=deny,\
                  [email protected],rules.0.policy=allow \
                  [email protected],rules.0.policy=allow \
          -vnc 0.0.0.0:1,tls-creds=tls0,tls-acl=tlsacl0,
	       sasl,sasl-acl=saslacl0 \
          ...other QEMU args...
---
 ui/vnc.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 60 insertions(+), 13 deletions(-)
ui/vnc.c +60 -13
index 324512d..7090f0b 100644
@@ -3261,6 +3261,12 @@ static QemuOptsList qemu_vnc_opts = {
             .name = "acl",
             .type = QEMU_OPT_BOOL,
         },{
+            .name = "tls-acl",
+            .type = QEMU_OPT_STRING,
+        },{
+            .name = "sasl-acl",
+            .type = QEMU_OPT_STRING,
+        },{
             .name = "lossy",
             .type = QEMU_OPT_BOOL,
         },{
@@ -3483,6 +3489,10 @@ void vnc_display_open(const char *id, Error **errp)
     int saslErr;
 #endif
     int acl = 0;
+    const char *tlsacl;
+#ifdef CONFIG_VNC_SASL
+    const char *saslacl;
+#endif
     int lock_key_sync = 1;
 
     if (!vs) {
@@ -3670,6 +3680,21 @@ void vnc_display_open(const char *id, Error **errp)
         }
     }
     acl = qemu_opt_get_bool(opts, "acl", false);
+    tlsacl = qemu_opt_get(opts, "tls-acl");
+    if (acl && tlsacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'tls-acl' options");
+        goto fail;
+    }
+
+#ifdef CONFIG_VNC_SASL
+    saslacl = qemu_opt_get(opts, "sasl-acl");
+    if (acl && saslacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'sasl-acl' options");
+        goto fail;
+    }
+#endif
 
     share = qemu_opt_get(opts, "share");
     if (share) {
@@ -3699,7 +3724,9 @@ void vnc_display_open(const char *id, Error **errp)
         vs->non_adaptive = true;
     }
 
-    if (acl) {
+    if (tlsacl) {
+        vs->tlsaclname = g_strdup(tlsacl);
+    } else if (acl) {
         if (strcmp(vs->id, "default") == 0) {
             vs->tlsaclname = g_strdup("vnc.x509dname");
         } else {
@@ -3710,19 +3737,39 @@ void vnc_display_open(const char *id, Error **errp)
                           &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
-    if (acl && sasl) {
-        char *aclname;
+    if (sasl) {
+        if (saslacl) {
+            Object *container, *acl;
+            container = object_get_objects_root();
+            acl = object_resolve_path_component(container, saslacl);
+            if (!acl) {
+                error_setg(errp, "Cannot find ACL %s", saslacl);
+                goto fail;
+            }
 
-        if (strcmp(vs->id, "default") == 0) {
-            aclname = g_strdup("vnc.username");
-        } else {
-            aclname = g_strdup_printf("vnc.%s.username", vs->id);
-        }
-        vs->sasl.acl =
-            QAUTHZ(qauthz_simple_new(aclname,
-                                     QAUTHZ_SIMPLE_POLICY_DENY,
-                                     &error_abort));
-        g_free(aclname);
+            if (!object_dynamic_cast(acl, TYPE_QAUTHZ)) {
+                error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                           saslacl);
+                goto fail;
+            }
+            vs->sasl.acl = QAUTHZ(acl);
+        } else if (acl) {
+            char *aclname;
+
+            if (strcmp(vs->id, "default") == 0) {
+                aclname = g_strdup("vnc.username");
+            } else {
+                aclname = g_strdup_printf("vnc.%s.username", vs->id);
+            }
+            vs->sasl.acl =
+                QAUTHZ(qauthz_simple_new(aclname,
+                                         QAUTHZ_SIMPLE_POLICY_DENY,
+                                         &error_abort));
+            g_free(aclname);
+        }
+    } else if (saslacl) {
+        error_setg(errp, "SASL ACL provided when SASL is disabled");
+        goto fail;
     }
 #endif
 
Signature · 1 line
2.5.0
#25 lore raw

PATCH v3 9/10 chardev: add support for ACLs for TLS clients

In reply to Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use a chardev server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the chardev server. This is still a fairly weak bar.

This adds a 'tls-acl=ACL-ID' option to the socket chardev backend which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the chardev server.

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

  $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                endpoint=server,verify-peer=yes \
        -object authz-simple,id=acl0,policy=deny,\
                rules.0.match=*CN=fred,rules.0.policy=allow \
        -chardev socket,host=127.0.0.1,port=9000,server,\
	         tls-creds=tls0,tls-acl=acl0 \
        ...other qemud args...
---
 qapi-schema.json |  2 ++
 qemu-char.c      | 11 ++++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)
qapi-schema.json +2 -0
index b6769de..a6a7205 100644
@@ -3202,6 +3202,7 @@
 # @addr: socket address to listen on (server=true)
 #        or connect to (server=false)
 # @tls-creds: #optional the ID of the TLS credentials object (since 2.6)
+# @tls-acl: #optional the ID of the QAuthZ authorization object (since 2.6)
 # @server: #optional create server socket (default: true)
 # @wait: #optional wait for incoming connection on server
 #        sockets (default: false).
@@ -3217,6 +3218,7 @@
 ##
 { 'struct': 'ChardevSocket', 'data': { 'addr'       : 'SocketAddress',
                                      '*tls-creds'  : 'str',
+                                     '*tls-acl'    : 'str',
                                      '*server'    : 'bool',
                                      '*wait'      : 'bool',
                                      '*nodelay'   : 'bool',
qemu-char.c +10 -1
index e0147f3..9533e7e 100644
@@ -2533,6 +2533,7 @@ typedef struct {
     QIOChannelSocket *listen_ioc;
     guint listen_tag;
     QCryptoTLSCreds *tls_creds;
+    char *tls_acl;
     int connected;
     int max_size;
     int do_telnetopt;
@@ -2963,7 +2964,7 @@ static void tcp_chr_tls_init(CharDriverState *chr)
     if (s->is_listen) {
         tioc = qio_channel_tls_new_server(
             s->ioc, s->tls_creds,
-            NULL, /* XXX Use an ACL */
+            s->tls_acl,
             &err);
     } else {
         tioc = qio_channel_tls_new_client(
@@ -3084,6 +3085,7 @@ static void tcp_chr_close(CharDriverState *chr)
     if (s->tls_creds) {
         object_unref(OBJECT(s->tls_creds));
     }
+    g_free(s->tls_acl);
     if (s->write_msgfds_num) {
         g_free(s->write_msgfds);
     }
@@ -3623,6 +3625,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
     const char *host = qemu_opt_get(opts, "host");
     const char *port = qemu_opt_get(opts, "port");
     const char *tls_creds = qemu_opt_get(opts, "tls-creds");
+    const char *tls_acl = qemu_opt_get(opts, "tls-acl");
     SocketAddress *addr;
     ChardevSocket *sock;
 
@@ -3656,6 +3659,7 @@ static void qemu_chr_parse_socket(QemuOpts *opts, ChardevBackend *backend,
     sock->has_reconnect = true;
     sock->reconnect = reconnect;
     sock->tls_creds = g_strdup(tls_creds);
+    sock->tls_acl = g_strdup(tls_acl);
 
     addr = g_new0(SocketAddress, 1);
     if (path) {
@@ -4094,6 +4098,9 @@ QemuOptsList qemu_chardev_opts = {
             .name = "tls-creds",
             .type = QEMU_OPT_STRING,
         },{
+            .name = "tls-acl",
+            .type = QEMU_OPT_STRING,
+        },{
             .name = "width",
             .type = QEMU_OPT_NUMBER,
         },{
@@ -4341,6 +4348,7 @@ static CharDriverState *qmp_chardev_open_socket(const char *id,
             }
         }
     }
+    s->tls_acl = g_strdup(sock->tls_acl);
 
     qapi_copy_SocketAddress(&s->addr, sock->addr);
 
@@ -4386,6 +4394,7 @@ static CharDriverState *qmp_chardev_open_socket(const char *id,
     if (s->tls_creds) {
         object_unref(OBJECT(s->tls_creds));
     }
+    g_free(s->tls_acl);
     g_free(s);
     qemu_chr_free_common(chr);
     return NULL;
Signature · 1 line
2.5.0
#23 lore raw

PATCH v3 8/10 nbd: allow an ACL to be set with nbd-server-start QMP command

In reply to Daniel P. Berrange

As with the previous patch to qemu-nbd, the nbd-server-start QMP command also needs to be able to specify an ACL when enabling TLS encryption.

First the client must create a QAuthZ object instance using the 'object-add' command:

   {
     'execute': 'object-add',
     'arguments': {
       'qom-type': 'authz-simple',
       'id': 'tls0',
       'parameters': {
         'policy': 'deny',
         'rules': [
           {
             'match': '*CN=fred',
             'policy': 'allow'
           }
         ]
       }
     }
   }

They can then reference this in the new 'tls-acl' parameter when executing the 'nbd-server-start' command.

   {
     'execute': 'nbd-server-start',
     'arguments': {
       'addr': {
           'type': 'inet',
           'host': '127.0.0.1',
           'port': '9000'
       },
       'tls-creds': 'tls0',
       'tls-acl': 'tlsacl0'
     }
   }
---
 blockdev-nbd.c  | 10 +++++++++-
 hmp.c           |  2 +-
 qapi/block.json |  4 +++-
 qmp-commands.hx |  2 +-
 4 files changed, 14 insertions(+), 4 deletions(-)
blockdev-nbd.c +9 -1
index 12cae0e..ae5335e 100644
@@ -24,6 +24,7 @@ typedef struct NBDServerData {
     QIOChannelSocket *listen_ioc;
     int watch;
     QCryptoTLSCreds *tlscreds;
+    char *tlsacl;
 } NBDServerData;
 
 static NBDServerData *nbd_server;
@@ -45,7 +46,8 @@ static gboolean nbd_accept(QIOChannel *ioc, GIOCondition condition,
     }
 
     nbd_client_new(NULL, cioc,
-                   nbd_server->tlscreds, NULL,
+                   nbd_server->tlscreds,
+                   nbd_server->tlsacl,
                    nbd_client_put);
     object_unref(OBJECT(cioc));
     return TRUE;
@@ -65,6 +67,7 @@ static void nbd_server_free(NBDServerData *server)
     if (server->tlscreds) {
         object_unref(OBJECT(server->tlscreds));
     }
+    g_free(server->tlsacl);
 
     g_free(server);
 }
@@ -101,6 +104,7 @@ static QCryptoTLSCreds *nbd_get_tls_creds(const char *id, Error **errp)
 
 void qmp_nbd_server_start(SocketAddress *addr,
                           bool has_tls_creds, const char *tls_creds,
+                          bool has_tls_acl, const char *tls_acl,
                           Error **errp)
 {
     if (nbd_server) {
@@ -128,6 +132,10 @@ void qmp_nbd_server_start(SocketAddress *addr,
         }
     }
 
+    if (has_tls_acl) {
+        nbd_server->tlsacl = g_strdup(tls_acl);
+    }
+
     nbd_server->watch = qio_channel_add_watch(
         QIO_CHANNEL(nbd_server->listen_ioc),
         G_IO_IN,
hmp.c +1 -1
index 7a98726..20703fd 100644
@@ -1802,7 +1802,7 @@ void hmp_nbd_server_start(Monitor *mon, const QDict *qdict)
         goto exit;
     }
 
-    qmp_nbd_server_start(addr, false, NULL, &local_err);
+    qmp_nbd_server_start(addr, false, NULL, false, NULL, &local_err);
     qapi_free_SocketAddress(addr);
     if (local_err != NULL) {
         goto exit;
qapi/block.json +3 -1
index 58e6b30..6b209e1 100644
@@ -147,6 +147,7 @@
 #
 # @addr: Address on which to listen.
 # @tls-creds: (optional) ID of the TLS credentials object. Since 2.6
+# @tls-acl: (optional) ID of the QAuthZ authorization object. Since 2.6
 #
 # Returns: error if the server is already running.
 #
@@ -154,7 +155,8 @@
 ##
 { 'command': 'nbd-server-start',
   'data': { 'addr': 'SocketAddress',
-            '*tls-creds': 'str'} }
+            '*tls-creds': 'str',
+            '*tls-acl': 'str'} }
 
 ##
 # @nbd-server-add:
qmp-commands.hx +1 -1
index b629673..7a3fa26 100644
@@ -3859,7 +3859,7 @@ EQMP
 
     {
         .name       = "nbd-server-start",
-        .args_type  = "addr:q,tls-creds:s?",
+        .args_type  = "addr:q,tls-creds:s?,tls-acl:s?",
         .mhandler.cmd_new = qmp_marshal_nbd_server_start,
     },
     {
Signature · 1 line
2.5.0
#18 lore raw

PATCH v3 6/10 acl: delete existing ACL implementation

In reply to Daniel P. Berrange

The 'qemu_acl' type was a previous non-QOM based attempt to provide an authorization facility in QEMU. Because it is non-QOM based it cannot be created via the command line and requires special monitor commands to manipulate it.

The new QAuthZ and QAuthZSimple QOM classes provide a superset of the functionality in qemu_acl, so the latter can now be deleted. The HMP 'acl_*' monitor commands are converted to use the new QAuthZSimple data type instead in order to provide backwards compatibility, but their use is discouraged.

---
 Makefile                       |   6 +-
 crypto/tlssession.c            |  28 ++++--
 include/qemu/acl.h             |  74 ----------------
 monitor.c                      | 161 ++++++++++++++++++++++-------------
 tests/Makefile                 |   2 +-
 tests/test-crypto-tlssession.c |  13 +--
 tests/test-io-channel-tls.c    |  14 +--
 ui/vnc-auth-sasl.c             |   2 +-
 ui/vnc-auth-sasl.h             |   4 +-
 ui/vnc.c                       |  11 ++-
 util/Makefile.objs             |   1 -
 util/acl.c                     | 188 -----------------------------------------
 12 files changed, 156 insertions(+), 348 deletions(-)
 delete mode 100644 include/qemu/acl.h
 delete mode 100644 util/acl.c
Makefile +3 -3
index 60ad13e..8f7ffd3 100644
@@ -235,9 +235,9 @@ util/module.o-cflags = -D'CONFIG_BLOCK_MODULES=$(block-modules)'
 
 qemu-img.o: qemu-img-cmds.h
 
-qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
-qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) libqemuutil.a libqemustub.a
+qemu-img$(EXESUF): qemu-img.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-nbd$(EXESUF): qemu-nbd.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
+qemu-io$(EXESUF): qemu-io.o $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(util-qom-obj-y) libqemuutil.a libqemustub.a
 
 qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o
 
crypto/tlssession.c +23 -5
index e0d9658..26e8097 100644
@@ -22,7 +22,7 @@
 #include "crypto/tlssession.h"
 #include "crypto/tlscredsanon.h"
 #include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "trace.h"
 
 #ifdef CONFIG_GNUTLS
@@ -207,6 +207,7 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
     unsigned int nCerts, i;
     time_t now;
     gnutls_x509_crt_t cert = NULL;
+    Error *err = NULL;
 
     now = time(NULL);
     if (now == ((time_t)-1)) {
@@ -295,16 +296,33 @@ qcrypto_tls_session_check_certificate(QCryptoTLSSession *session,
                 goto error;
             }
             if (session->aclname) {
-                qemu_acl *acl = qemu_acl_find(session->aclname);
-                int allow;
-                if (!acl) {
+                QAuthZ *acl;
+                Object *obj;
+                Object *container;
+                bool allow;
+
+                container = object_get_objects_root();
+                obj = object_resolve_path_component(container,
+                                                    session->aclname);
+                if (!obj) {
                     error_setg(errp, "Cannot find ACL %s",
                                session->aclname);
                     goto error;
                 }
 
-                allow = qemu_acl_party_is_allowed(acl, session->peername);
+                if (!object_dynamic_cast(obj, TYPE_QAUTHZ)) {
+                    error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                               session->aclname);
+                    goto error;
+                }
 
+                acl = QAUTHZ(obj);
+
+                allow = qauthz_is_allowed(acl, session->peername, &err);
+                if (err) {
+                    error_propagate(errp, err);
+                    goto error;
+                }
                 if (!allow) {
                     error_setg(errp, "TLS x509 ACL check for %s is denied",
                                session->peername);
include/qemu/acl.h deleted +0 -74

74 lines not on this page · raw message

monitor.c +101 -60

221 lines not on this page · raw message

tests/Makefile +1 -1
index 7a1d959..647f30f 100644
@@ -391,7 +391,7 @@ test-qom-obj-y = $(qom-obj-y) $(test-util-obj-y)
 test-qapi-obj-y = tests/test-qapi-visit.o tests/test-qapi-types.o \
 	tests/test-qapi-event.o tests/test-qmp-introspect.o \
 	$(test-qom-obj-y)
-test-crypto-obj-y = $(crypto-obj-y) $(test-qom-obj-y)
+test-crypto-obj-y = $(crypto-obj-y) $(util-qom-obj-y) $(test-qom-obj-y)
 test-io-obj-y = $(io-obj-y) $(test-crypto-obj-y)
 test-block-obj-y = $(block-obj-y) $(test-io-obj-y)
 
tests/test-crypto-tlssession.c +8 -5
index 036a86b..e76b249 100644
@@ -25,7 +25,7 @@
 #include "crypto/tlssession.h"
 #include "qom/object_interfaces.h"
 #include "qemu/sockets.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 
 #ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
 
@@ -110,7 +110,7 @@ static void test_crypto_tls_session(const void *opaque)
     QCryptoTLSCreds *serverCreds;
     QCryptoTLSSession *clientSess = NULL;
     QCryptoTLSSession *serverSess = NULL;
-    qemu_acl *acl;
+    QAuthZSimple *auth;
     const char * const *wildcards;
     int channel[2];
     bool clientShake = false;
@@ -169,11 +169,13 @@ static void test_crypto_tls_session(const void *opaque)
         &err);
     g_assert(serverCreds != NULL);
 
-    acl = qemu_acl_init("tlssessionacl");
-    qemu_acl_reset(acl);
+    auth = qauthz_simple_new("tlssessionacl",
+                             QAUTHZ_SIMPLE_POLICY_DENY,
+                             &error_abort);
     wildcards = data->wildcards;
     while (wildcards && *wildcards) {
-        qemu_acl_append(acl, 0, *wildcards);
+        qauthz_simple_append_rule(auth, *wildcards,
+                                  QAUTHZ_SIMPLE_POLICY_ALLOW);
         wildcards++;
     }
 
@@ -263,6 +265,7 @@ static void test_crypto_tls_session(const void *opaque)
 
     object_unparent(OBJECT(serverCreds));
     object_unparent(OBJECT(clientCreds));
+    object_unparent(OBJECT(auth));
 
     qcrypto_tls_session_free(serverSess);
     qcrypto_tls_session_free(clientSess);
tests/test-io-channel-tls.c +9 -5
index 3c361a7..390d3e9 100644
@@ -28,7 +28,7 @@
 #include "io/channel-socket.h"
 #include "io-channel-helpers.h"
 #include "crypto/tlscredsx509.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 #include "qom/object_interfaces.h"
 
 #ifdef QCRYPTO_HAVE_TLS_TEST_SUPPORT
@@ -115,7 +115,7 @@ static void test_io_channel_tls(const void *opaque)
     QIOChannelTLS *serverChanTLS;
     QIOChannelSocket *clientChanSock;
     QIOChannelSocket *serverChanSock;
-    qemu_acl *acl;
+    QAuthZSimple *auth;
     const char * const *wildcards;
     int channel[2];
     struct QIOChannelTLSHandshakeData clientHandshake = { false, false };
@@ -166,11 +166,13 @@ static void test_io_channel_tls(const void *opaque)
         &err);
     g_assert(serverCreds != NULL);
 
-    acl = qemu_acl_init("channeltlsacl");
-    qemu_acl_reset(acl);
+    auth = qauthz_simple_new("channeltlsacl",
+                             QAUTHZ_SIMPLE_POLICY_DENY,
+                             &error_abort);
     wildcards = data->wildcards;
     while (wildcards && *wildcards) {
-        qemu_acl_append(acl, 0, *wildcards);
+        qauthz_simple_append_rule(auth, *wildcards,
+                                  QAUTHZ_SIMPLE_POLICY_ALLOW);
         wildcards++;
     }
 
@@ -256,6 +258,8 @@ static void test_io_channel_tls(const void *opaque)
     object_unref(OBJECT(serverChanSock));
     object_unref(OBJECT(clientChanSock));
 
+    object_unparent(OBJECT(auth));
+
     close(channel[0]);
     close(channel[1]);
 }
ui/vnc-auth-sasl.c +1 -1
index 13a59f5..53f9a7c 100644
@@ -149,7 +149,7 @@ static int vnc_auth_sasl_check_access(VncState *vs)
         return 0;
     }
 
-    allow = qemu_acl_party_is_allowed(vs->vd->sasl.acl, vs->sasl.username);
+    allow = qauthz_is_allowed(vs->vd->sasl.acl, vs->sasl.username, NULL);
 
     VNC_DEBUG("SASL client %s %s by ACL\n", vs->sasl.username,
               allow ? "allowed" : "denied");
ui/vnc-auth-sasl.h +2 -2
index 3f59da6..a6ffb7e 100644
@@ -32,7 +32,7 @@
 typedef struct VncStateSASL VncStateSASL;
 typedef struct VncDisplaySASL VncDisplaySASL;
 
-#include "qemu/acl.h"
+#include "qemu/authz.h"
 #include "qemu/main-loop.h"
 
 struct VncStateSASL {
@@ -61,7 +61,7 @@ struct VncStateSASL {
 };
 
 struct VncDisplaySASL {
-    qemu_acl *acl;
+    QAuthZ *acl;
 };
 
 void vnc_sasl_client_cleanup(VncState *vs);
ui/vnc.c +8 -3
index 729f630..324512d 100644
@@ -33,7 +33,7 @@
 #include "qemu/error-report.h"
 #include "qemu/sockets.h"
 #include "qemu/timer.h"
-#include "qemu/acl.h"
+#include "qemu/authz-simple.h"
 #include "qemu/config-file.h"
 #include "qapi/qmp/qerror.h"
 #include "qapi/qmp/types.h"
@@ -3705,7 +3705,9 @@ void vnc_display_open(const char *id, Error **errp)
         } else {
             vs->tlsaclname = g_strdup_printf("vnc.%s.x509dname", vs->id);
         }
-        qemu_acl_init(vs->tlsaclname);
+        qauthz_simple_new(vs->tlsaclname,
+                          QAUTHZ_SIMPLE_POLICY_DENY,
+                          &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
     if (acl && sasl) {
@@ -3716,7 +3718,10 @@ void vnc_display_open(const char *id, Error **errp)
         } else {
             aclname = g_strdup_printf("vnc.%s.username", vs->id);
         }
-        vs->sasl.acl = qemu_acl_init(aclname);
+        vs->sasl.acl =
+            QAUTHZ(qauthz_simple_new(aclname,
+                                     QAUTHZ_SIMPLE_POLICY_DENY,
+                                     &error_abort));
         g_free(aclname);
     }
 #endif
util/Makefile.objs +0 -1
index 4870905..44078c1 100644
@@ -13,7 +13,6 @@ util-obj-y += envlist.o path.o module.o
 util-obj-$(call lnot,$(CONFIG_INT128)) += host-utils.o
 util-obj-y += bitmap.o bitops.o hbitmap.o
 util-obj-y += fifo8.o
-util-obj-y += acl.o
 util-obj-y += error.o qemu-error.o
 util-obj-y += id.o
 util-obj-y += iov.o qemu-config.o qemu-sockets.o uri.o notify.o
util/acl.c deleted +0 -188

188 lines not on this page · raw message

Signature · 1 line
2.5.0
#30 lore raw

Re: [Qemu-devel] [PATCH v3 01/10] qdict: implement a qdict_crumple method for un-flattening a dict

In reply to Daniel P. Berrange
Quoted 13 lines · Daniel P. Berrange

The qdict_flatten() method will take a dict whose elements are further nested dicts/lists and flatten them by concatenating keys.

The qdict_crumple() method aims to do the reverse, taking a flat qdict, and turning it into a set of nested dicts/lists. It will apply nesting based on the key name, with a '.' indicating a new level in the hierarchy. If the keys in the nested structure are all numeric, it will create a list, otherwise it will create a dict.

Quoted 6 lines

will get turned into a dict with one element 'foo' whose value is a list. The list elements will each in turn be dicts.

 {
   'foo' => [

s/=>/:/

Quoted 1 line
     { 'bar': 'one', 'wizz': '1' }

s/$/,/

Quoted 7 lines
     { 'bar': 'two', 'wizz': '2' }

],
}

The intent of this function is that it allows a set of QemuOpts to be turned into a nested data structure that mirrors the nested

s/the nested/the nesting/

Quoted 16 lines

used when the same object is defined over QMP.

---
 include/qapi/qmp/qdict.h |   1 +
 qobject/qdict.c          | 267 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/check-qdict.c      | 143 +++++++++++++++++++++++++
 3 files changed, 411 insertions(+)
qobject/qdict.c +7 -0
+
+/**
+ * qdict_split_flat_key:
+ *
+ * Given a flattened key such as 'foo.0.bar', split it
+ * into two parts at the first '.' separator. Allows
+ * double dot ('..') to escape the normal separator.

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 701

Quoted 10 lines
qobject/qdict.c +10 -0
+ *
+ * eg
+ *    'foo.0.bar' -> prefix='foo' and suffix='0.bar'
+ *    'foo..0.bar' -> prefix='foo.0' and suffix='bar'
+ *
+ * The '..' sequence will be unescaped in the returned
+ * 'prefix' string. The 'suffix' string will be left
+ * in escaped format, so it can be fed back into the
+ * qdict_split_flat_key() key as the input later.
+ */

Might be worth mentioning that prefix and suffix must both be non-NULL, and that the caller must g_free() the two resulting strings.

Quoted 6 lines
qobject/qdict.c +6 -0
+static void qdict_split_flat_key(const char *key, char **prefix, char **suffix)
+{
+    const char *separator;
+    size_t i, j;
+
+    /* Find first '.' separator, but if there is a pair '..'

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 717

Quoted 10 lines
qobject/qdict.c +10 -0
+     * that acts as an escape, so skip over '..' */
+    separator = NULL;
+    do {
+        if (separator) {
+            separator += 2;
+        } else {
+            separator = key;
+        }
+        separator = strchr(separator, '.');
+    } while (separator && *(separator + 1) == '.');

I'd probably have written separator[1] == '.', but your approach is synonymous.

Quoted 4 lines
qobject/qdict.c +4 -0
+
+    if (separator) {
+        *prefix = g_strndup(key,
+                            separator - key);

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 731

Quoted 10 lines
qobject/qdict.c +10 -0
+        *suffix = g_strdup(separator + 1);
+    } else {
+        *prefix = g_strdup(key);
+        *suffix = NULL;
+    }
+
+    /* Unescape the '..' sequence into '.' */
+    for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+        if ((*prefix)[i] == '.' &&
+            (*prefix)[i + 1] == '.') {

Technically, if (*prefix)[i] == '.', we could assert((*prefix)[i + 1] == '.'), since the only way to get a '.' in prefix is via escaping. For that matter, you could short-circuit (part of) the loop by doing a strchr for '.' (if not found, the loop is not needed; if found, start the reduction at that point rather on the bytes leading up to that point).

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 742

Quoted 11 lines
qobject/qdict.c +11 -0
+            i++;
+        }
+        (*prefix)[j] = (*prefix)[i];
+    }
+    (*prefix)[j] = '\0';
+}
+
+
+/**
+ * qdict_list_size:
+ * @maybe_List: dict that may be only list elements

s/List/list/

Quoted 3 lines

+ *
+ * Determine whether all keys in @maybe_list are
+ * valid list elements. They they are all valid,

s/They they/If they/

Quoted 7 lines

+ * then this returns the number of elements. If
+ * they all look like non-numeric keys, then returns
+ * zero. If there is a mix of numeric and non-numeric
+ * keys, then an error is set as it is both a list
+ * and a dict at once.
+ *
+ * Returns: number of list elemets, 0 if a dict, -1 on error

s/elemets/elements/

Quoted 10 lines

+ */

qobject/qdict.c +9 -0
+static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
+{
+    const QDictEntry *entry, *next;
+    ssize_t len = 0;
+    ssize_t max = -1;
+    int is_list = -1;
+    int64_t val;
+
+    entry = qdict_first(maybe_list);

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 772

Quoted 10 lines
qobject/qdict.c +10 -0
+    while (entry != NULL) {
+        next = qdict_next(maybe_list, entry);
+
+        if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
+            if (is_list == -1) {
+                is_list = 1;
+            } else if (!is_list) {
+                error_setg(errp,
+                           "Key '%s' is for a list, but previous key is "
+                           "for a dict", entry->key);

Keys are unsorted, so it's a bit hard to call it "previous key". Maybe a better error message would be along the lines of "cannot crumple dictionary because of a mix of list and non-list keys"? I dunno...

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 785

Quoted 13 lines
qobject/qdict.c +13 -0
+                return -1;
+            }
+            len++;
+            if (val > max) {
+                max = val;
+            }
+        } else {
+            if (is_list == -1) {
+                is_list = 0;
+            } else if (is_list) {
+                error_setg(errp,
+                           "Key '%s' is for a dict, but previous key is "
+                           "for a list", entry->key);

...same argument. If we can wordsmith something that makes sense, it might work for both places. Otherwise, I can live with your messages.

Quoted 3 lines

+/**
+ * qdict_crumple:
+ *

Worth documenting the 'recursive' parameter?

Quoted 4 lines

+ * Reverses the flattening done by qdict_flatten by
+ * crumpling the dicts into a nested structure. Similar
+ * qdict_array_split, but copes with arbitrary nesting
+ * of dicts & arrays, not merely one level of arrays

Quoted 8 lines

+ *
+ * { 'foo.0.bar': 'one', 'foo.0.wizz': '1',
+ * 'foo.1.bar': 'two', 'foo.1.wizz': '2' }
+ *
+ * =>
+ *
+ * {
+ * 'foo' => [

s/=>/:/

Quoted 1 line

+ * { 'bar': 'one', 'wizz': '1' }

s/$/,/

Quoted 4 lines

+ * { 'bar': 'two', 'wizz': '2' }
+ * ],
+ * }
+ *

Worth mentioning the escaping of '.' in key names?

Quoted 27 lines

+ */

qobject/qdict.c +24 -0
+QObject *qdict_crumple(QDict *src, bool recursive, Error **errp)
+{
+    const QDictEntry *entry, *next;
+    QDict *two_level, *multi_level = NULL;
+    QObject *dst = NULL, *child;
+    ssize_t list_len;
+    size_t i;
+    char *prefix = NULL, *suffix = NULL;
+
+    two_level = qdict_new();
+    entry = qdict_first(src);
+
+    /* Step 1: split our totally flat dict into a two level dict */
 
+
+    /* Step 2: optionally process the two level dict recursively
+     * into a multi-level dict */
+    if (recursive) {
 
+
+    /* Step 3: detect if we need to turn our dict into list */
+    list_len = qdict_list_size(multi_level, errp);
+    if (list_len < 0) {
+        goto error;
+    }
+

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 928

Quoted 10 lines
qobject/qdict.c +10 -0
+    if (list_len) {
+        dst = QOBJECT(qlist_new());
+
+        for (i = 0; i < list_len; i++) {
+            char *key = g_strdup_printf("%zu", i);
+
+            child = qdict_get(multi_level, key);
+            g_free(key);
+            if (!child) {
+                error_setg(errp, "Unexpected missing list entry %zu", i);

Couldn't we assert() this, since it is a programming bug if qdict_list_size() let us get this far but then the key disappeared?

Overall looks like it does the trick.

tests/check-qdict.c static void qdict_join_test(void) · line 601

Quoted 8 lines

+++ b/tests/check-qdict.c

tests/check-qdict.c +3 -0
@@ -596,6 +596,140 @@ static void qdict_join_test(void)
     QDECREF(dict2);
 }
 
+
+static void qdict_crumple_test_nonrecursive(void)
+{

This only covers a single layer of collapse, but not turning a dict into a list. Is it also worth covering a case where no list indices are involved, such as the four keys "a.b.d", "a.b.e", "a.c.d", "a.d.e" being crumpled non-recursively into a single dict "a" with keys "b.d", "b.e", "c.d", and "d.e"?

tests/check-qdict.c static void qdict_join_test(void) · line 641

Quoted 4 lines
tests/check-qdict.c +4 -0
+
+static void qdict_crumple_test_recursive(void)
+{
+

This only covers a list of dict collapse, not a true multi-layer dict collapse. Is it also worth covering the same four keys as above, but this time that dict "a" has keys "b" and "c", each of which is a dict in turn with keys "d" and "e"?

Review static void qdict_join_test(void) · line 607

Quoted 2 lines
(quoted diff) +2 -0
+static void qdict_crumple_test_empty(void)
+{

So an empty dict is never crumpled to an empty list. I guess that shouldn't matter.

tests/check-qdict.c static void qdict_join_test(void) · line 724

Quoted 12 lines
tests/check-qdict.c +11 -0
+
+static void qdict_crumple_test_bad_inputs(void)
+{
+    QDict *src;
+    Error *error = NULL;
+
 
+
+    src = qdict_new();
+    /* The input should be flat, ie no dicts or lists */
+    qdict_put(src, "rule.0", qdict_new());
+    qdict_put(src, "rule.a", qstring_from_str("allow"));

I'd use "rule.a" and "rule.b" here, so that you aren't confusing this with the earlier test that you can't mix list and dict.

I'd also add a negative test for "rule.1" without "rule.0" being invalid (missing a list index).

I'll wait to give R-b until I get further into the series, and/or you post a v4, but it's mostly there.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#3 lore raw

Re: [Qemu-devel] [PATCH v3 02/10] qapi: allow QmpInputVisitor to auto-cast types

In reply to Daniel P. Berrange
Quoted 26 lines · Daniel P. Berrange

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This extends it so that QString is optionally permitted for any of the non-string scalar types. This behaviour is turned on by requesting the 'autocast' flag in the constructor.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

---
 include/qapi/qmp-input-visitor.h |   3 +
 qapi/qmp-input-visitor.c         |  96 +++++++++++++++++++++++++++-----
 tests/test-qmp-input-visitor.c   | 115 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 196 insertions(+), 18 deletions(-)
include/qapi/qmp-input-visitor.h +0 -0
index 3ed499c..c25cb7c 100644

include/qapi/qmp-input-visitor.h typedef struct QmpInputVisitor QmpInputVisitor; · line 26

Quoted 6 lines
include/qapi/qmp-input-visitor.h +3 -0
@@ -21,6 +21,9 @@ typedef struct QmpInputVisitor QmpInputVisitor;
 
 QmpInputVisitor *qmp_input_visitor_new(QObject *obj);
 QmpInputVisitor *qmp_input_visitor_new_strict(QObject *obj);
+QmpInputVisitor *qmp_input_visitor_new_full(QObject *obj,
+                                            bool strict,
+                                            bool autocast);

We have so few uses of qmp_input_visitor_new* that it might be worth just having a single prototype, and maybe using an 'int flags' instead of a string of bool. But not a show-stopper for this patch (rather, an idea for a future patch).

include/qapi/qmp-input-visitor.h typedef struct QmpInputVisitor QmpInputVisitor; · line 29

Quoted 4 lines
include/qapi/qmp-input-visitor.h +3 -1
-    *obj = qint_get_int(qint);
+    qstr = qobject_to_qstring(qobj);
+    if (qstr && qstr->string && qiv->autocast) {
+        errno = 0;

Dead setting of errno, since...

Quoted 1 line

+ if (qemu_strtoll(qstr->string, NULL, 10, obj) == 0) {

qemu_strtoll() handles it on your behalf, and you aren't using error_setg_errno().

Quoted 13 lines
include/qapi/qmp-input-visitor.h +4 -4
@@ -233,30 +245,61 @@ static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj,
 {
     /* FIXME: qobject_to_qint mishandles values over INT64_MAX */
     QmpInputVisitor *qiv = to_qiv(v);
-    QInt *qint = qobject_to_qint(qmp_input_get_object(qiv, name, true));
+    QObject *qobj = qmp_input_get_object(qiv, name, true);
+    QInt *qint;
+    QString *qstr;
 
-    if (!qint) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "integer");
+    qint = qobject_to_qint(qobj);

include/qapi/qmp-input-visitor.h static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj, · line 261

Quoted 10 lines
include/qapi/qmp-input-visitor.h +6 -1
+    if (qint) {
+        *obj = qint_get_int(qint);
         return;
     }
 
-    *obj = qint_get_int(qint);
+    qstr = qobject_to_qstring(qobj);
+    if (qstr && qstr->string && qiv->autocast) {
+        errno = 0;
+        if (qemu_strtoull(qstr->string, NULL, 10, obj) == 0) {

And again.

Hmm. Do we need to worry about partial asymmetry? That is, qint_get_int() returns a signed number, but qemu_strtoull() parses unsigned; if the original conversion from JSON to qint uses a different parser, then we could have funny results where we get different results for things like:
"key1":9223372036854775807, "key2":"9223372036854775807",
even though the same string of digits is being parsed, based on whether the different parsers handle numbers larger than INT64_MAX differently.

[Ultimately, I'd like QInt to be enhanced to track whether the input was signed or unsigned, and automatically make the output match the input when converting back to string; that is, track 65 bits of information instead of 64; but that's no sooner than 2.7 material]

Quoted 18 lines
include/qapi/qmp-input-visitor.h +6 -4
 static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
                                 Error **errp)
 {
     QmpInputVisitor *qiv = to_qiv(v);
-    QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
+    QObject *qobj = qmp_input_get_object(qiv, name, true);
+    QBool *qbool;
+    QString *qstr;
 
-    if (!qbool) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "boolean");
+    qbool = qobject_to_qbool(qobj);
+    if (qbool) {
+        *obj = qbool_get_bool(qbool);
         return;
     }
 

include/qapi/qmp-input-visitor.h static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj, · line 284

Quoted 10 lines
include/qapi/qmp-input-visitor.h +9 -1
-    *obj = qbool_get_bool(qbool);
+
+    qstr = qobject_to_qstring(qobj);
+    if (qstr && qstr->string && qiv->autocast) {
+        if (!strcasecmp(qstr->string, "on") ||
+            !strcasecmp(qstr->string, "yes") ||
+            !strcasecmp(qstr->string, "true")) {
+            *obj = true;
+            return;
+        }

Do we also want to allow "0"/"1" for true/false?

Overall, I'm a big fan of this patch.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#7 lore raw

Re: [Qemu-devel] [PATCH v3 03/10] qom: support arbitrary non-scalar properties with -object

In reply to Daniel P. Berrange
Quoted 11 lines · Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

Quoted 4 lines

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Zoltán Kővágó tried earlier with his GSoC patches for the audio subsystem last year, but those got stalled waiting for qapi enhancements to go in. But I think your approach is indeed a bit nicer (rather than making the warty OptsVisitor even wartier, just avoid it).

Quoted 8 lines

Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.

Quoted 11 lines

The previously added qdict_crumple() method is able to take a qdict containing a flat set of properties and turn that into a arbitrarily nested set of dicts and lists. By combining qemu_opts_to_qdict and qdict_crumple() together, we can turn the opt string into a data structure that is practically identical to that passed over QMP when defining an object. The only difference is that all the scalar values are represented as strings, rather than strings, ints and bools. This is sufficient to let us replace the OptsVisitor with the QMPInputVisitor for use with -object.

Indeed, nice replacement.

Quoted 25 lines

Thus -object can now support non-scalar properties, for example the QMP object

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
	  { "bar": "one", "wizz": "1" },
	  { "bar": "two", "wizz": "2" }
        ]
      }
    }

}

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2
Quoted 6 lines

This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.

  (hmp) object_add demo,id=demo0,\
                   foo.0.bar=one,foo.0.wizz=1,\
		   foo.1.bar=two,foo.1.wizz=2

Maybe mention that the indentation is not actually present in the real command lines typed.

Quoted 10 lines
---
 hmp.c                      |  18 +--
 qom/object_interfaces.c    |  20 ++-
 tests/check-qom-proplist.c | 295 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 313 insertions(+), 20 deletions(-)

qom/object_interfaces.c Object *user_creatable_add_type(const char *type, const char *id, · line 126

Quoted 7 lines
qom/object_interfaces.c +1 -0
@@ -120,6 +120,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
     obj = object_new(type);
     if (qdict) {
         for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+
             object_property_set(obj, v, e->key, &local_err);
             if (local_err) {
                 goto out;

Spurious hunk?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#8 lore raw

Re: [Qemu-devel] [PATCH v3 03/10] qom: support arbitrary non-scalar properties with -object

Quoted 12 lines · Eric Blake
Quoted 11 lines · Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

Quoted 8 lines
Quoted 4 lines

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Zoltán Kővágó tried earlier with his GSoC patches for the audio subsystem last year, but those got stalled waiting for qapi enhancements to go in.

Yet another series stalled on the big QAPI rework. Hitting a GSoC student that way is really unfortunate.

Quoted 2 lines

But I think your approach is indeed a bit nicer (rather than
making the warty OptsVisitor even wartier, just avoid it).

QemuOpts defines an important part of the command line language, namely (most of) the syntax of many option arguments. It parses them into a set of (name, value) pairs.

"Most of", because additional syntax may hide in the parameter value.

Parameter values are typed, except when they aren't. Types are limited to string, bool, uint64_t number (accepts negative numbers and casts them) and uint64_t size (rejects negative numbers, accepts suffixes).

OptsVisitor adds special list syntax. It's used with untyped values.

Bypassing OptsVisitor risks adding different special syntax. Doesn't mean it's a bad idea, only that we need to keep close watch on what it does to the language. See below.

I merely scratched the surprising amount of complexity that has accrued over time. If you doubt me, study how the merge_lists flag works, and how it interacts with the several ways we do defaults.

The gap between QemuOpts' and QAPI's type system is obvious. But it's not just a gap, it's also contraditions: QAPI does only *signed* integers.

Nevertheless, the path from command line to QAPI is through QemuOpts for now.

Aside: for historical reasons, we have a few QMP commands that detour through QemuOpts. Mistakes, do not add more.

Ceterum censeo QemuOpts esse delendam. And I say that as author of 35 out of 117 commits touching qemu-option.c.

Quoted 19 lines
Quoted 7 lines

Fortunately there is no existing object that implements the UserCreatable interface that relies on the list handling behaviour, so it is possible to swap out the OptsVisitor for a different visitor implementation, so -object supports non-scalar properties, thus leaving other users of OptsVisitor unaffected.

Quoted 11 lines

The previously added qdict_crumple() method is able to take a qdict containing a flat set of properties and turn that into a arbitrarily nested set of dicts and lists. By combining qemu_opts_to_qdict and qdict_crumple() together, we can turn the opt string into a data structure that is practically identical to that passed over QMP when defining an object. The only difference is that all the scalar values are represented as strings, rather than strings, ints and bools. This is sufficient to let us replace the OptsVisitor with the QMPInputVisitor for use with -object.

Quoted 26 lines

Indeed, nice replacement.

Quoted 24 lines

Thus -object can now support non-scalar properties, for example the QMP object

  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "demo",
      "id": "demo0",
      "parameters": {
        "foo": [
	  { "bar": "one", "wizz": "1" },
	  { "bar": "two", "wizz": "2" }
        ]
      }
    }

}

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

Okay, this adds a bare minimum of syntax, and it's not even new: we use similar syntax for block stuff already.

Quoted 29 lines
Quoted 6 lines

This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.

  (hmp) object_add demo,id=demo0,\
                   foo.0.bar=one,foo.0.wizz=1,\
		   foo.1.bar=two,foo.1.wizz=2

Maybe mention that the indentation is not actually present in the real command lines typed.

Quoted 10 lines
---
 hmp.c                      |  18 +--
 qom/object_interfaces.c    |  20 ++-
 tests/check-qom-proplist.c | 295 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 313 insertions(+), 20 deletions(-)

qom/object_interfaces.c Object *user_creatable_add_type(const char *type, const char *id, · line 126

Quoted 7 lines
qom/object_interfaces.c +1 -0
@@ -120,6 +120,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
     obj = object_new(type);
     if (qdict) {
         for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+
             object_property_set(obj, v, e->key, &local_err);
             if (local_err) {
                 goto out;

Spurious hunk?

#9 lore raw

Re: [Qemu-devel] [PATCH v3 03/10] qom: support arbitrary non-scalar properties with -object

Quoted 39 lines · Markus Armbruster
Quoted 12 lines · Eric Blake
Quoted 11 lines · Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

Quoted 8 lines
Quoted 4 lines

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Zoltán Kővágó tried earlier with his GSoC patches for the audio subsystem last year, but those got stalled waiting for qapi enhancements to go in.

Yet another series stalled on the big QAPI rework. Hitting a GSoC student that way is really unfortunate.

Quoted 2 lines

But I think your approach is indeed a bit nicer (rather than
making the warty OptsVisitor even wartier, just avoid it).

QemuOpts defines an important part of the command line language, namely (most of) the syntax of many option arguments. It parses them into a set of (name, value) pairs.

"Most of", because additional syntax may hide in the parameter value.

Parameter values are typed, except when they aren't. Types are limited to string, bool, uint64_t number (accepts negative numbers and casts them) and uint64_t size (rejects negative numbers, accepts suffixes).

Quoted 5 lines

OptsVisitor adds special list syntax. It's used with untyped values.

Bypassing OptsVisitor risks adding different special syntax. Doesn't mean it's a bad idea, only that we need to keep close watch on what it does to the language. See below.

FWIW, when I first started attacking this problem I actually went down the path of extending the OptsVisitor to cope with arbitrarily nested structs + lists. I quickly discovered that special list syntax supported by the OptsVisitor. I tried to hack support for nested structs + lists on top of that, but ultimately the way the special list syntax is designed makes that an impossible problem without breaking back compat, or having OptsVisitor support two completely different lists syntaxes at the same time with ambiguous parsing results. None the less I did implement that all and it was a huge amount of work. I then took a look at QMPInputVisitor and realized that if we could converts a QemuOpts into a QDict, we could just reuse the QMPInputVisitor.

In the specific case of -object the fact that QMPInputVisitor does not support the special list syntax from OptsVisitor is not a problem because we don't have any existing user defined object type that uses list props yet. So if we want to change -object from OptsVisitor to QMPInputVisitor the sooner we change it the better - it is only a matter of time before something comes along that depends on the existing special list syntax and then we'll be locked into OptsVisitor's non-extensible approach.

Quoted 9 lines
Quoted 6 lines
Quoted 6 lines

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

Okay, this adds a bare minimum of syntax, and it's not even new: we use similar syntax for block stuff already.

Yes, indeed that usage by the block layer is what motivated me to ditch OptsVisitor and take the approach of converting QemuOpts to QDict and then using QMPInputVisitor.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#31 lore raw

Re: [Qemu-devel] [PATCH v3 01/10] qdict: implement a qdict_crumple method for un-flattening a dict

In reply to Eric Blake
Quoted 7 lines · Eric Blake

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 731

Quoted 4 lines · Daniel P. Berrange
qobject/qdict.c +4 -0
+    /* Unescape the '..' sequence into '.' */
+    for (i = 0, j = 0; (*prefix)[i] != '\0'; i++, j++) {
+        if ((*prefix)[i] == '.' &&
+            (*prefix)[i + 1] == '.') {
Quoted 5 lines

Technically, if (*prefix)[i] == '.', we could assert((*prefix)[i + 1] == '.'), since the only way to get a '.' in prefix is via escaping. For that matter, you could short-circuit (part of) the loop by doing a strchr for '.' (if not found, the loop is not needed; if found, start the reduction at that point rather on the bytes leading up to that point).

I'm not seeing obvious benefit in trying to short-circuit the loop using a strchr, as both ways you still end up iterating over all chars in the string - its just that you're hiding the iteration in strchr instead.

Quoted 15 lines
Quoted 15 lines
qobject/qdict.c +15 -0
+static ssize_t qdict_list_size(QDict *maybe_list, Error **errp)
+{
+    const QDictEntry *entry, *next;
+    ssize_t len = 0;
+    ssize_t max = -1;
+    int is_list = -1;
+    int64_t val;
+
+    entry = qdict_first(maybe_list);
+    while (entry != NULL) {
+        next = qdict_next(maybe_list, entry);
+
+        if (qemu_strtoll(entry->key, NULL, 10, &val) == 0) {
+            if (is_list == -1) {
+                is_list = 1;
Quoted 8 lines

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 772

Quoted 4 lines
qobject/qdict.c +4 -0
+            } else if (!is_list) {
+                error_setg(errp,
+                           "Key '%s' is for a list, but previous key is "
+                           "for a dict", entry->key);

Keys are unsorted, so it's a bit hard to call it "previous key". Maybe a better error message would be along the lines of "cannot crumple dictionary because of a mix of list and non-list keys"? I dunno...

Yeah, I'll use

"Cannot crumple a dictionary with a mix of list and non-list keys"

Quoted 24 lines
Quoted 4 lines
qobject/qdict.c +3 -0
+                return -1;
+            }
+            len++;

qobject/qdict.c void qdict_array_split(QDict *src, QList **dst) · line 785

Quoted 10 lines
qobject/qdict.c +10 -0
+            if (val > max) {
+                max = val;
+            }
+        } else {
+            if (is_list == -1) {
+                is_list = 0;
+            } else if (is_list) {
+                error_setg(errp,
+                           "Key '%s' is for a dict, but previous key is "
+                           "for a list", entry->key);

...same argument. If we can wordsmith something that makes sense, it might work for both places. Otherwise, I can live with your messages.

Quoted 5 lines

+++ b/tests/check-qdict.c

tests/check-qdict.c +0 -0
@@ -596,6 +596,140 @@ static void qdict_join_test(void)
     QDECREF(dict2);
 }
 
Quoted 9 lines

tests/check-qdict.c static void qdict_join_test(void) · line 601

Quoted 3 lines
tests/check-qdict.c +3 -0
+
+static void qdict_crumple_test_nonrecursive(void)
+{

This only covers a single layer of collapse, but not turning a dict into a list. Is it also worth covering a case where no list indices are involved, such as the four keys "a.b.d", "a.b.e", "a.c.d", "a.d.e" being crumpled non-recursively into a single dict "a" with keys "b.d", "b.e", "c.d", and "d.e"?

I'll add an explicit rule to test dict -> list conversion, and some extra dict items here to cover proper nested dicts.

Quoted 10 lines

tests/check-qdict.c static void qdict_join_test(void) · line 641

Quoted 5 lines
tests/check-qdict.c +4 -0
+
+static void qdict_crumple_test_recursive(void)
+{
+

This only covers a list of dict collapse, not a true multi-layer dict collapse. Is it also worth covering the same four keys as above, but this time that dict "a" has keys "b" and "c", each of which is a dict in turn with keys "d" and "e"?

I'll add some more dict items to properly cover nested dicts

Quoted 6 lines
Quoted 5 lines
tests/check-qdict.c +5 -0
+static void qdict_crumple_test_bad_inputs(void)
+{
+    QDict *src;
+    Error *error = NULL;
+
Quoted 8 lines

tests/check-qdict.c static void qdict_join_test(void) · line 724

Quoted 5 lines
tests/check-qdict.c +5 -0
+
+    src = qdict_new();
+    /* The input should be flat, ie no dicts or lists */
+    qdict_put(src, "rule.0", qdict_new());
+    qdict_put(src, "rule.a", qstring_from_str("allow"));

I'd use "rule.a" and "rule.b" here, so that you aren't confusing this with the earlier test that you can't mix list and dict.

Good point.

Quoted 2 lines

I'd also add a negative test for "rule.1" without "rule.0" being invalid (missing a list index).

Yep, I'll add that.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#4 lore raw

Re: [Qemu-devel] [PATCH v3 02/10] qapi: allow QmpInputVisitor to auto-cast types

In reply to Eric Blake
Quoted 31 lines · Eric Blake
Quoted 22 lines · Daniel P. Berrange

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This extends it so that QString is optionally permitted for any of the non-string scalar types. This behaviour is turned on by requesting the 'autocast' flag in the constructor.

This makes it possible to use QmpInputVisitor with a QDict produced from QemuOpts, where everything is in string format.

---
 include/qapi/qmp-input-visitor.h |   3 +
 qapi/qmp-input-visitor.c         |  96 +++++++++++++++++++++++++++-----
 tests/test-qmp-input-visitor.c   | 115 ++++++++++++++++++++++++++++++++++++++-
 3 files changed, 196 insertions(+), 18 deletions(-)

qapi/qmp-input-visitor.c static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj, · line 261

Quoted 5 lines
qapi/qmp-input-visitor.c +4 -1
-    *obj = qint_get_int(qint);
+    qstr = qobject_to_qstring(qobj);
+    if (qstr && qstr->string && qiv->autocast) {
+        errno = 0;
+        if (qemu_strtoull(qstr->string, NULL, 10, obj) == 0) {

And again.

Quoted 8 lines

Hmm. Do we need to worry about partial asymmetry? That is, qint_get_int() returns a signed number, but qemu_strtoull() parses unsigned; if the original conversion from JSON to qint uses a different parser, then we could have funny results where we get different results for things like:
"key1":9223372036854775807, "key2":"9223372036854775807",
even though the same string of digits is being parsed, based on whether the different parsers handle numbers larger than INT64_MAX differently.

Is this something you want me to change for re-post, or just a general point for future ? ie should I be using qemu_strtoll instead of
qemu_strtoull or something else ? qint itself doesn't seem
to concern itself with parsing ints from strintgs, so presumably this is from json code ?

Quoted 35 lines

[Ultimately, I'd like QInt to be enhanced to track whether the input was signed or unsigned, and automatically make the output match the input when converting back to string; that is, track 65 bits of information instead of 64; but that's no sooner than 2.7 material]

Quoted 18 lines

static void qmp_input_type_bool(Visitor *v, const char *name, bool *obj,
Error **errp)

 {
     QmpInputVisitor *qiv = to_qiv(v);
qapi/qmp-input-visitor.c +6 -4
-    QBool *qbool = qobject_to_qbool(qmp_input_get_object(qiv, name, true));
+    QObject *qobj = qmp_input_get_object(qiv, name, true);
+    QBool *qbool;
+    QString *qstr;
 
-    if (!qbool) {
-        error_setg(errp, QERR_INVALID_PARAMETER_TYPE, name ? name : "null",
-                   "boolean");
+    qbool = qobject_to_qbool(qobj);
+    if (qbool) {
+        *obj = qbool_get_bool(qbool);
         return;
     }
 

qapi/qmp-input-visitor.c static void qmp_input_type_uint64(Visitor *v, const char *name, uint64_t *obj, · line 292

Quoted 10 lines
qapi/qmp-input-visitor.c +9 -1
-    *obj = qbool_get_bool(qbool);
+
+    qstr = qobject_to_qstring(qobj);
+    if (qstr && qstr->string && qiv->autocast) {
+        if (!strcasecmp(qstr->string, "on") ||
+            !strcasecmp(qstr->string, "yes") ||
+            !strcasecmp(qstr->string, "true")) {
+            *obj = true;
+            return;
+        }
Quoted 1 line

Do we also want to allow "0"/"1" for true/false?

These 3 strings I took from opts-visitor.c, so to maintain compat we probably should not allow 0/1, unless we decide to extend opts-visitor too

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#10 lore raw

Re: [Qemu-devel] [PATCH v3 03/10] qom: support arbitrary non-scalar properties with -object

In reply to Eric Blake
Quoted 17 lines · Eric Blake
Quoted 11 lines · Daniel P. Berrange

The current -object command line syntax only allows for creation of objects with scalar properties, or a list with a fixed scalar element type. Objects which have properties that are represented as structs in the QAPI schema cannot be created using -object.

This is a design limitation of the way the OptsVisitor is written. It simply iterates over the QemuOpts values as a flat list. The support for lists is enabled by allowing the same key to be repeated in the opts string.

Quoted 4 lines

It is not practical to extend the OptsVisitor to support more complex data structures while also maintaining the existing list handling behaviour that is relied upon by other areas of QEMU.

Quoted 4 lines

Zoltán Kővágó tried earlier with his GSoC patches for the audio subsystem last year, but those got stalled waiting for qapi enhancements to go in. But I think your approach is indeed a bit nicer (rather than making the warty OptsVisitor even wartier, just avoid it).

My first attempt did indeed modify OptsVisitor, but I quickly abandoned it since it ended up being quite complex code to make it fit in with the pre-existing hack to supports lists of scalars in OptsVisitor. The QmpInputVisitor approach is cleaner and simpler overall

Quoted 10 lines
Quoted 10 lines

Would be creatable via the CLI now using

    $QEMU \
      -object demo,id=demo0,\
              foo.0.bar=one,foo.0.wizz=1,\
              foo.1.bar=two,foo.1.wizz=2

This is also wired up to work for the 'object_add' command in the HMP monitor with the same syntax.

Quoted 6 lines
Quoted 3 lines
  (hmp) object_add demo,id=demo0,\
                   foo.0.bar=one,foo.0.wizz=1,\
		   foo.1.bar=two,foo.1.wizz=2

Maybe mention that the indentation is not actually present in the real command lines typed.

Heh, yeah

Quoted 10 lines

qom/object_interfaces.c Object *user_creatable_add_type(const char *type, const char *id, · line 126

Quoted 8 lines
qom/object_interfaces.c +1 -0
@@ -120,6 +120,7 @@ Object *user_creatable_add_type(const char *type, const char *id,
     obj = object_new(type);
     if (qdict) {
         for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
+
             object_property_set(obj, v, e->key, &local_err);
             if (local_err) {
                 goto out;

Spurious hunk?

Indeed

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#5 lore raw

Re: [Qemu-devel] [PATCH v3 02/10] qapi: allow QmpInputVisitor to auto-cast types

In reply to Daniel P. Berrange
Quoted 21 lines · Daniel P. Berrange
Quoted 11 lines · Eric Blake
Quoted 10 lines · Daniel P. Berrange

Currently the QmpInputVisitor assumes that all scalar values are directly represented as their final types. ie it assumes an 'int' is using QInt, and a 'bool' is using QBool.

This extends it so that QString is optionally permitted for any of the non-string scalar types. This behaviour is turned on by requesting the 'autocast' flag in the constructor.

Quoted 8 lines

Hmm. Do we need to worry about partial asymmetry? That is, qint_get_int() returns a signed number, but qemu_strtoull() parses unsigned; if the original conversion from JSON to qint uses a different parser, then we could have funny results where we get different results for things like:
"key1":9223372036854775807, "key2":"9223372036854775807",
even though the same string of digits is being parsed, based on whether the different parsers handle numbers larger than INT64_MAX differently.

Quoted 5 lines

Is this something you want me to change for re-post, or just a general point for future ? ie should I be using qemu_strtoll instead of
qemu_strtoull or something else ? qint itself doesn't seem
to concern itself with parsing ints from strintgs, so presumably this is from json code ?

General comment for now. We already know we need a bigger audit of handling of values larger than INT64_MAX, so any cleanups related to that can be deferred to that later audit. But maybe a FIXME or TODO comment in the code in your submission, to remind us to think about it during the later audit, would help.

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
qapi/qmp-input-visitor.c +8 -0
+    qstr = qobject_to_qstring(qobj);
+    if (qstr && qstr->string && qiv->autocast) {
+        if (!strcasecmp(qstr->string, "on") ||
+            !strcasecmp(qstr->string, "yes") ||
+            !strcasecmp(qstr->string, "true")) {
+            *obj = true;
+            return;
+        }
Quoted 5 lines
Quoted 1 line

Do we also want to allow "0"/"1" for true/false?

These 3 strings I took from opts-visitor.c, so to maintain compat we probably should not allow 0/1, unless we decide to extend opts-visitor too

Good point. Maybe a comment pointing in both places pointing out that we should keep them in sync is a worthwhile addition.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#12 lore raw

Re: [Qemu-devel] [PATCH v3 04/10] util: add QAuthZ object as an authorization base class

In reply to Daniel P. Berrange
Quoted 19 lines · Daniel P. Berrange

The current qemu_acl module provides a simple access control list facility inside QEMU, which is used via a set of monitor commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially integrate with external authorization systems like polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be subclassed for creating different authorization providers.

---

+++ b/include/qemu/authz.h

include/qemu/authz.h line 49

Quoted 7 lines
include/qemu/authz.h new file +7 -0
+
+/**
+ * QAuthZ:
+ *
+ * The QAuthZ class defines an API contract to be used
+ * for providing an authorization driver for network
+ * services.

Just network services? Or is it broader than that?

Quoted 9 lines

+/**
+ * qauthz_is_allowed:
+ * @authz: the authorization object
+ * @identity: the user identity to authorize
+ * @errp: pointer to a NULL initialized error object
+ *
+ * Check if a user @identity is authorized
+ *
+ * Returns: true if @identity is authorizd, false otherwise

s/authorizd/authorized/

I think you need more documentation on return semantics. Do we have strict binary return (either we returned true and errp is unset, or we returned false and errp is set), or is it a ternary (we return true and errp is unset: permission is explicitly granted; we return false and errp is unset: permission is explicitly denied; or we set errp: we could not determine permission). And if a ternary, do we also want to require that setting 'errp' also requires a return of false, or is the return undefined in that case?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#13 lore raw

Re: [Qemu-devel] [PATCH v3 04/10] util: add QAuthZ object as an authorization base class

Quoted 42 lines · Eric Blake
Quoted 19 lines · Daniel P. Berrange

The current qemu_acl module provides a simple access control list facility inside QEMU, which is used via a set of monitor commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially integrate with external authorization systems like polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be subclassed for creating different authorization providers.

---

+++ b/include/qemu/authz.h

include/qemu/authz.h line 49

Quoted 7 lines
include/qemu/authz.h new file +7 -0
+
+/**
+ * QAuthZ:
+ *
+ * The QAuthZ class defines an API contract to be used
+ * for providing an authorization driver for network
+ * services.

Just network services? Or is it broader than that?

Quoted 9 lines

+/**
+ * qauthz_is_allowed:
+ * @authz: the authorization object
+ * @identity: the user identity to authorize
+ * @errp: pointer to a NULL initialized error object
+ *
+ * Check if a user @identity is authorized
+ *
+ * Returns: true if @identity is authorizd, false otherwise

s/authorizd/authorized/

Quoted 8 lines

I think you need more documentation on return semantics. Do we have strict binary return (either we returned true and errp is unset, or we returned false and errp is set), or is it a ternary (we return true and errp is unset: permission is explicitly granted; we return false and errp is unset: permission is explicitly denied; or we set errp: we could not determine permission). And if a ternary, do we also want to require that setting 'errp' also requires a return of false, or is the return undefined in that case?

It is intended to be ternary, and if errp is set, the return value should be false.

ie you should be able todo

if (qauthz_is_allowed(authz, identity, NULL))

     ....

safe in the knowledge that any error that you're ignoring will result in denial of permission

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#14 lore raw

Re: [Qemu-devel] [PATCH v3 04/10] util: add QAuthZ object as an authorization base class

In reply to Eric Blake
Quoted 19 lines · Eric Blake
Quoted 17 lines · Daniel P. Berrange

The current qemu_acl module provides a simple access control list facility inside QEMU, which is used via a set of monitor commands acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially integrate with external authorization systems like polkit, pam, ldap lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be subclassed for creating different authorization providers.

---

Quoted 10 lines

include/qemu/authz.h line 49

Quoted 8 lines

+++ b/include/qemu/authz.h

include/qemu/authz.h new file +7 -0
+
+/**
+ * QAuthZ:
+ *
+ * The QAuthZ class defines an API contract to be used
+ * for providing an authorization driver for network
+ * services.

Just network services? Or is it broader than that?

Any service that requires authentication. It is actually nothing specific to networking

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#16 lore raw

Re: [Qemu-devel] [PATCH v3 05/10] util: add QAuthZSimple object type for a simple access control list

In reply to Daniel P. Berrange
Quoted 9 lines · Daniel P. Berrange

Add a QAuthZSimple object type that implements the QAuthZ interface. This simple built-in implementation maintains a trivial access control list with a sequence of match rules and a final default policy. This replicates the functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor, the syntax used would be

Quoted 16 lines
  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "authz-simple",
      "id": "auth0",
      "parameters": {
        "rules": [
           { "match": "fred", "policy": "allow" },
           { "match": "bob", "policy": "allow" },
           { "match": "danb", "policy": "deny" },
           { "match": "dan*", "policy": "allow" }
        ],
        "policy": "deny"
      }
    }
  }

So "match" appears to be a glob (as opposed to a regex). And order is important (the first rule matched ends the lookup), so you correctly used an array for the list of rules (a dict does not have to maintain order).

Quoted 9 lines

Or via the -object command line

  $QEMU \
     -object authz-simple,id=acl0,policy=deny,\
             match.0.name=fred,match.0.policy=allow, \
             match.1.name=bob,match.1.policy=allow, \
             match.2.name=danb,match.2.policy=deny, \
             match.3.name=dan*,match.3.policy=allow

The '*' in the command line will require shell quoting.

Quoted 8 lines

This sets up an authorization rule that allows 'fred', 'bob' and anyone whose name starts with 'dan', except for 'danb'. Everyone unmatched is denied.

---

Quoted 9 lines

+/**
+ * QAuthZSimple:
+ *
+ * This authorization driver provides a simple mechanism
+ * for granting access by matching user names against a
+ * list of globs. Each match rule has an associated policy
+ * and a catch all policy applies if no rule matches
+ *
+ * To create an instace of this class via QMP:

s/instace/instance/

Quoted 9 lines

+ *
+ * Or via the CLI:
+ *

+ *   $QEMU                                                  \
+ *    -object authz-simple,id=acl0,policy=deny,             \
+ *            match.0.name=fred,match.0.policy=allow,       \
+ *            match.1.name=bob,match.1.policy=allow,        \
+ *            match.2.name=danb,match.2.policy=deny,        \

+ * match.3.name=dan*,match.3.policy=allow

Again, quoting the "*" is important, and maybe a comment that the whitespace is for display purposes but must be omitted on the command line.

Quoted 12 lines

+++ b/qapi-schema.json

qapi-schema.json +3 -0
@@ -5,6 +5,9 @@
 # QAPI common definitions
 { 'include': 'qapi/common.json' }
 
+# QAPI util definitions
+{ 'include': 'qapi/util.json' }
+
 # QAPI crypto definitions
 { 'include': 'qapi/crypto.json' }
 

qapi-schema.json line 3659

Quoted 6 lines
qapi-schema.json +2 -1
@@ -3652,7 +3655,8 @@
 # Since 2.5
 ##
 { 'struct': 'DummyForceArrays',
-  'data': { 'unused': ['X86CPUFeatureWordInfo'] } }
+  'data': { 'unused': ['X86CPUFeatureWordInfo'],
+            'iamalsounused': ['QAuthZSimpleRule'] } }

Cute. I might have renamed things and gone 'unused1' and 'unused2'.

Quoted 9 lines
qapi/util.json new file +8 -0
+++ b/qapi/util.json
@@ -0,0 +1,31 @@
+# -*- Mode: Python -*-
+#
+# QAPI util definitions
+
+##
+# QAuthZSimplePolicy:
+#

qapi/util.json line 17

Quoted 10 lines
qapi/util.json new file +10 -0
+# The authorization policy result
+#
+# @deny: deny access
+# @allow: allow access
+#
+# Since: 2.6
+##
+{ 'enum': 'QAuthZSimplePolicy',
+  'prefix': 'QAUTHZ_SIMPLE_POLICY',
+  'data': ['deny', 'allow']}

I know Markus isn't a big fan of 'prefix', but I don't mind it.

We're awfully late in the 2.6 cycle; this feels like a feature addition rather than a bug fix, so should it be 2.7?

Quoted 4 lines
qapi/util.json new file +4 -0
+
+##
+# QAuthZSimpleRule:
+#

qapi/util.json line 31

Quoted 10 lines
qapi/util.json new file +10 -0
+# A single authorization rule.
+#
+# @match: a glob to match against a user identity
+# @policy: the result to return if @match evaluates to true
+#
+# Since: 2.6
+##
+{ 'struct': 'QAuthZSimpleRule',
+  'data': {'match': 'str',
+           'policy': 'QAuthZSimplePolicy'}}

Hmm. I was expecting something like:

{ 'struct': 'QAuthZSimple',
'data': { 'rules': [ 'QAuthZSimpleRule' ],

            'policy': 'QAuthZSimplePolicy' } }

but I guess that's one of our short-comings of QOM: the top-level structure does not have to be specified anywhere in QAPI.

Quoted 6 lines

+++ b/tests/test-authz-simple.c

tests/test-authz-simple.c new file +4 -0
@@ -0,0 +1,156 @@
+/*
+ * QEMU simple authorization object
+ *
+ * Copyright (c) 2016 Red Hat, Inc.

tests/test-authz-simple.c line 33

Quoted 10 lines
tests/test-authz-simple.c new file +9 -0
+ *
 
+static void test_authz_default_deny(void)
+{
+    QAuthZSimple *auth = qauthz_simple_new("auth0",
+                                           QAUTHZ_SIMPLE_POLICY_DENY,
+                                           &error_abort);
+
+    g_assert(!qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+

Okay, so you definitely intend to return false without setting errp, so it is a ternary result.

tests/test-authz-simple.c line 77

Quoted 4 lines
tests/test-authz-simple.c new file +4 -0
+
+#ifdef CONFIG_FNMATCH
+static void test_authz_complex(void)
+{

Wait - the behavior depends on whether fnmatch() is available? That is, a name is a literal match if fnmatch() is not present, and glob if present? I'd argue that if fnmatch() is not present, we must explicitly reject any "match" with glob metacharacters, rather than accidentally matching only a literal "*" when a glob was intended.

util/authz-simple.c line 33

Quoted 11 lines

+++ b/util/authz-simple.c

util/authz-simple.c new file +9 -0
+
+/* If no fnmatch, fallback to exact string matching
+ * instead of allowing wildcards */
+#ifdef CONFIG_FNMATCH
+#include <fnmatch.h>
+#define qauthz_match(pattern, string) fnmatch(pattern, string, 0)
+#else
+#define qauthz_match(pattern, string) strcmp(pattern, string)
+#endif

As mentioned above, I'd be more comfortable if you also explicitly reject any attempt to add a pattern that resembles a glob when globbing is not enabled. Or maybe it's worth a more complex QAPI definition:

# How to interpret 'match', as a literal string or a glob { 'enum': 'QAuthZSimpleStyle', 'data': [ 'literal', 'glob' ] } # @style: #optional, default to 'literal'
{ 'struct': 'QAuthZSimpleRule',
'data': { 'match': 'str', '*style': 'QAuthZSimpleStyle',

            'policy': 'QAuthZSimplePolicy' } }

and used as:

         "rules": [
            { "match": "fred", "policy": "allow" },
            { "match": "bob", "policy": "allow" },
            { "match": "danb", "policy": "deny" },
            { "match": "dan*", "policy": "allow", "style": "glob" }

where you can then gracefully error out for ALL attempts to use "style":"glob" if fnmatch() is not present, and use strcmp() even when fnmatch() is present for rules that have the (default) "style":"literal".

Quoted 11 lines
util/authz-simple.c new file +11 -0
+static void
+qauthz_simple_class_init(ObjectClass *oc, void *data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+    QAuthZClass *authz = QAUTHZ_CLASS(oc);
+
+    ucc->complete = qauthz_simple_complete;
+    authz->is_allowed = qauthz_simple_is_allowed;
+
+    object_class_property_add_enum(oc, "policy",
+                                   "QAuthZSimplePolicy",

util/authz-simple.c line 130

Quoted 10 lines
util/authz-simple.c new file +10 -0
+                                   QAuthZSimplePolicy_lookup,
+                                   qauthz_simple_prop_get_policy,
+                                   qauthz_simple_prop_set_policy,
+                                   NULL);
+
+    object_class_property_add(oc, "rules", "QAuthZSimpleRule",
+                              qauthz_simple_prop_get_rules,
+                              qauthz_simple_prop_set_rules,
+                              NULL, NULL, NULL);
+}

Not for this patch, but it would be cool if we had enough framework to just tell QOM to instantiate an object with callbacks by merely pointing to the name of a QAPI type that the object implements (referring back to my comment that I'm a bit surprised we didn't need a QAPI type for the overall QAuthZSimple).

Quoted 13 lines
util/authz-simple.c new file +13 -0
+
+size_t qauthz_simple_append_rule(QAuthZSimple *auth, const char *match,
+                                 QAuthZSimplePolicy policy)
+{
+    QAuthZSimpleRule *rule;
+    QAuthZSimpleRuleList *rules, *tmp;
+    size_t i = 0;
+
+    rule = g_new0(QAuthZSimpleRule, 1);
+    rule->policy = policy;
+    rule->match = g_strdup(match);
+
+    tmp = g_new0(QAuthZSimpleRuleList, 1);

util/authz-simple.c line 167

Quoted 10 lines
util/authz-simple.c new file +10 -0
+    tmp->value = rule;
+
+    rules = auth->rules;
+    if (rules) {
+        while (rules->next) {
+            i++;
+            rules = rules->next;
+        }
+        rules->next = tmp;
+        return i + 1;

No checks whether 'match' is already an existing rule...

Quoted 9 lines
util/authz-simple.c new file +9 -0
+ssize_t qauthz_simple_delete_rule(QAuthZSimple *auth, const char *match)
+{
+    QAuthZSimpleRule *rule;
+    QAuthZSimpleRuleList *rules, *prev;
+    size_t i = 0;
+
+    prev = NULL;
+    rules = auth->rules;
+    while (rules) {

util/authz-simple.c line 224

Quoted 10 lines
util/authz-simple.c new file +10 -0
+        rule = rules->value;
+        if (g_str_equal(rule->match, match)) {
+            if (prev) {
+                prev->next = rules->next;
+            } else {
+                auth->rules = rules->next;
+            }
+            rules->next = NULL;
+            qapi_free_QAuthZSimpleRuleList(rules);
+            return i;

...which means a rule can be inserted more than once, and this only removes the first instance of the rule. Do we care enough to add in additional checking that we aren't permitting duplicate 'match' strings in the list of rules?

If you add the style=literal/glob to a rule, would you also want to be able to constrain deletion to a particular style (delete the glob 'dan*' but leave the literal 'dan*' intact)?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
Reviewed-by on v3 6/10
#19 lore raw

Re: [Qemu-devel] [PATCH v3 06/10] acl: delete existing ACL implementation

In reply to Daniel P. Berrange
Quoted 5 lines · Daniel P. Berrange

The 'qemu_acl' type was a previous non-QOM based attempt to provide an authorization facility in QEMU. Because it is non-QOM based it cannot be created via the command line and requires special monitor commands to manipulate it.

Quoted 8 lines

The new QAuthZ and QAuthZSimple QOM classes provide a superset of the functionality in qemu_acl, so the latter can now be deleted. The HMP 'acl_*' monitor commands are converted to use the new QAuthZSimple data type instead in order to provide backwards compatibility, but their use is discouraged.

---

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
Reviewed-by on v3 7/10
#21 lore raw

Re: [Qemu-devel] [PATCH v3 07/10] qemu-nbd: add support for ACLs for TLS clients

In reply to Daniel P. Berrange
Quoted 18 lines · Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly weak bar.

This adds a '--tls-acl ACL-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the NBD server.

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

Quoted 7 lines
  qemu-nbd -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                   endpoint=server,verify-peer=yes \
           -object authz-simple,id=acl0,policy=deny,\
	           rules.0.match=*CN=fred,rules.0.policy=allow \
           -tls-creds tls0 \
           -tls-acl acl0
	   ....other qemu-nbd args...

Ah, so you are arguing that this is feature-completion of work started in 2.6, continuing work started before soft-freeze, and not a new feature to be delayed to 2.7.

Quoted 10 lines
---
 qemu-nbd.c    | 13 ++++++++++++-
 qemu-nbd.texi |  4 ++++
 2 files changed, 16 insertions(+), 1 deletion(-)

+++ b/qemu-nbd.texi

qemu-nbd.texi the new style NBD protocol negotiation · line 92

Quoted 7 lines
qemu-nbd.texi +4 -0
@@ -86,6 +86,10 @@ the new style NBD protocol negotiation
 Enable mandatory TLS encryption for the server by setting the ID
 of the TLS credentials object previously created with the --object
 option.
+@item --tls-acl=ID
+Specify the ID of a qauthz object previously created with the
+--object option. This will be used to authorize users who
+connect against their x509 distinguish name.

s/distinguish/distinguished/

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
Reviewed-by on v3 8/10
#24 lore raw

Re: [Qemu-devel] [PATCH v3 08/10] nbd: allow an ACL to be set with nbd-server-start QMP command

In reply to Daniel P. Berrange
Quoted 21 lines · Daniel P. Berrange

As with the previous patch to qemu-nbd, the nbd-server-start QMP command also needs to be able to specify an ACL when enabling TLS encryption.

First the client must create a QAuthZ object instance using the 'object-add' command:

---

+++ b/qapi/block.json

qapi/block.json +1 -0
@@ -147,6 +147,7 @@
 #
 # @addr: Address on which to listen.
 # @tls-creds: (optional) ID of the TLS credentials object. Since 2.6
+# @tls-acl: (optional) ID of the QAuthZ authorization object. Since 2.6
 #
 # Returns: error if the server is already running.
 #

qapi/block.json line 160

Quoted 7 lines
qapi/block.json +2 -1
@@ -154,7 +155,8 @@
 ##
 { 'command': 'nbd-server-start',
   'data': { 'addr': 'SocketAddress',
-            '*tls-creds': 'str'} }
+            '*tls-creds': 'str',
+            '*tls-acl': 'str'} }
 

Interface change is deceptively simple :)

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
Reviewed-by on v3 9/10
#26 lore raw

Re: [Qemu-devel] [PATCH v3 09/10] chardev: add support for ACLs for TLS clients

In reply to Daniel P. Berrange
Quoted 14 lines · Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use a chardev server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the chardev server. This is still a fairly weak bar.

This adds a 'tls-acl=ACL-ID' option to the socket chardev backend which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the chardev server.

Quoted 8 lines

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

  $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                endpoint=server,verify-peer=yes \
        -object authz-simple,id=acl0,policy=deny,\
                rules.0.match=*CN=fred,rules.0.policy=allow \

Needs shell quoting for *, and also the same recurring comment about whitespace for presentation not actually being in the command line.

Food for thought: should we enhance QemuOpts to skip all whitespace after ',', since we _know_ that valid key names start with a letter rather than a space? Then, we could represent command lines as:

$QEMU -object 'name,
param1=value,
param2=value'

with the same semantics as:

$QEMU -object name,param1=value,param2=value

and without having to worry about backslash-newline-whitespace formatting. Obviously, such an enhancement would be a separate patch.

Quoted 10 lines
        -chardev socket,host=127.0.0.1,port=9000,server,\
	         tls-creds=tls0,tls-acl=acl0 \
        ...other qemud args...
---
 qapi-schema.json |  2 ++
 qemu-char.c      | 11 ++++++++++-
 2 files changed, 12 insertions(+), 1 deletion(-)

Code is fine; my only comments were on the commit message.

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#28 lore raw

Re: [Qemu-devel] [PATCH v3 10/10] vnc: allow specifying a custom ACL object name

In reply to Daniel P. Berrange
Quoted 24 lines · Daniel P. Berrange

The VNC server has historically had support for ACLs to check both the SASL username and the TLS x509 distinguished name. The VNC server was responsible for creating the initial ACL, and the client app was then responsible for populating it with rules using the HMP 'acl_add' command.

This is not satisfactory for a variety of reasons. There is no way to populate the ACLs from the command line, users are forced to use the HMP. With multiple network services all supporting TLS and ACLs now, it is desirable to be able to define a single ACL that is referenced by all services.

To address these limitations, two new options are added to the VNC server CLI. The 'tls-acl' option takes the ID of a QAuthZ object to use for checking TLS x509 distinguished names, and the 'sasl-acl' option takes the ID of another object to use for checking SASL usernames.

In this example, we setup two ACLs. The first allows any client with a certificate issued by the 'RedHat' organization in the 'London' locality. The second ACL allows clients with either the '[email protected]' or '[email protected]' kerberos usernames. Both ACLs must pass for the user to be allowed.

Quoted 5 lines
    $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                  endpoint=server,verify-peer=yes \
          -object authz-simple,id=acl0,policy=deny,\
                  rules.0.match=O=RedHat,,L=London,rules.0.policy=allow \
          -object authz-simple,id=acl0,policy=deny,\

Umm, you can't reuse 'acl0' as the id.

Quoted 4 lines
                  [email protected],rules.0.policy=allow \
                  [email protected],rules.0.policy=allow \
          -vnc 0.0.0.0:1,tls-creds=tls0,tls-acl=tlsacl0,
	       sasl,sasl-acl=saslacl0 \

And this fails because the ids don't exist. I think you meant authz-simple,id=tlsacl0 in the first instance, and authz-simple,id=saslacl0 in the second instance.

Quoted 16 lines
          ...other QEMU args...
---
 ui/vnc.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 60 insertions(+), 13 deletions(-)
ui/vnc.c +5 -0
@@ -3670,6 +3680,21 @@ void vnc_display_open(const char *id, Error **errp)
         }
     }
     acl = qemu_opt_get_bool(opts, "acl", false);
+    tlsacl = qemu_opt_get(opts, "tls-acl");
+    if (acl && tlsacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'tls-acl' options");
+        goto fail;

ui/vnc.c void vnc_display_open(const char *id, Error **errp) · line 3697

Quoted 10 lines
ui/vnc.c +10 -0
+    }
+
+#ifdef CONFIG_VNC_SASL
+    saslacl = qemu_opt_get(opts, "sasl-acl");
+    if (acl && saslacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'sasl-acl' options");
+        goto fail;
+    }
+#endif

Do we explicitly fail if sasl-acl was provided but CONFIG_VNC_SASL is not defined? It looks here like you silently ignore it, which would not be good.

Quoted 42 lines
ui/vnc.c +25 -12
@@ -3710,19 +3737,39 @@ void vnc_display_open(const char *id, Error **errp)
                           &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
-    if (acl && sasl) {
-        char *aclname;
+    if (sasl) {
+        if (saslacl) {
+            Object *container, *acl;
+            container = object_get_objects_root();
+            acl = object_resolve_path_component(container, saslacl);
+            if (!acl) {
+                error_setg(errp, "Cannot find ACL %s", saslacl);
+                goto fail;
+            }
 
-        if (strcmp(vs->id, "default") == 0) {
-            aclname = g_strdup("vnc.username");
-        } else {
-            aclname = g_strdup_printf("vnc.%s.username", vs->id);
-        }
-        vs->sasl.acl =
-            QAUTHZ(qauthz_simple_new(aclname,
-                                     QAUTHZ_SIMPLE_POLICY_DENY,
-                                     &error_abort));
-        g_free(aclname);
+            if (!object_dynamic_cast(acl, TYPE_QAUTHZ)) {
+                error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                           saslacl);
+                goto fail;
+            }
+            vs->sasl.acl = QAUTHZ(acl);
+        } else if (acl) {
+            char *aclname;
+
+            if (strcmp(vs->id, "default") == 0) {
+                aclname = g_strdup("vnc.username");
+            } else {
+                aclname = g_strdup_printf("vnc.%s.username", vs->id);
+            }
+            vs->sasl.acl =
+                QAUTHZ(qauthz_simple_new(aclname,

ui/vnc.c void vnc_display_open(const char *id, Error **errp) · line 3775

Quoted 10 lines
ui/vnc.c +7 -0
+                                         QAUTHZ_SIMPLE_POLICY_DENY,
+                                         &error_abort));
+            g_free(aclname);
+        }
+    } else if (saslacl) {
+        error_setg(errp, "SASL ACL provided when SASL is disabled");
+        goto fail;
     }
 #endif
 

Again, the saslacl check is only mentioned inside the #if; what happens when the #if is not compiled?

Signature · 2 lines
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org
#17 lore raw

Re: [Qemu-devel] [PATCH v3 05/10] util: add QAuthZSimple object type for a simple access control list

In reply to Eric Blake
Quoted 27 lines · Eric Blake
Quoted 9 lines · Daniel P. Berrange

Add a QAuthZSimple object type that implements the QAuthZ interface. This simple built-in implementation maintains a trivial access control list with a sequence of match rules and a final default policy. This replicates the functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor, the syntax used would be

Quoted 16 lines
  {
    "execute": "object-add",
    "arguments": {
      "qom-type": "authz-simple",
      "id": "auth0",
      "parameters": {
        "rules": [
           { "match": "fred", "policy": "allow" },
           { "match": "bob", "policy": "allow" },
           { "match": "danb", "policy": "deny" },
           { "match": "dan*", "policy": "allow" }
        ],
        "policy": "deny"
      }
    }
  }
Quoted 4 lines

So "match" appears to be a glob (as opposed to a regex). And order is important (the first rule matched ends the lookup), so you correctly used an array for the list of rules (a dict does not have to maintain order).

Yes, its a glob (as defined by fnmatch)

Quoted 11 lines
Quoted 9 lines

Or via the -object command line

  $QEMU \
     -object authz-simple,id=acl0,policy=deny,\
             match.0.name=fred,match.0.policy=allow, \
             match.1.name=bob,match.1.policy=allow, \
             match.2.name=danb,match.2.policy=deny, \
             match.3.name=dan*,match.3.policy=allow

The '*' in the command line will require shell quoting.

Yeah, CLI syntax escaping from shell is horrible, but fortunately libvirt doesn't use shell :-) None the less I'll update the example.

Quoted 7 lines
Quoted 7 lines
qapi/util.json new file +7 -0
+##
+# QAuthZSimplePolicy:
+#
+# The authorization policy result
+#
+# @deny: deny access
+# @allow: allow access
Quoted 8 lines

qapi/util.json line 17

Quoted 6 lines
qapi/util.json new file +6 -0
+#
+# Since: 2.6
+##
+{ 'enum': 'QAuthZSimplePolicy',
+  'prefix': 'QAUTHZ_SIMPLE_POLICY',
+  'data': ['deny', 'allow']}

I know Markus isn't a big fan of 'prefix', but I don't mind it.

It doesn't affect public API anyway, so we can change it at will later if desired.

Quoted 2 lines

We're awfully late in the 2.6 cycle; this feels like a feature addition rather than a bug fix, so should it be 2.7?

It is definitely a feature addition. I was hoping to get it into 2.6 since it is logically associated with the TLS enhancements I've been doing. It isn't the end of the world if it waits until 2.7 though, so I'm open minded either way.

Basically TLS provides encryption, x509 certs provide authentication, and this ACL stuff provides the authorization piece of the puzzel.

If we miss the authorization piece in 2.6 we're still in a far better position than we were historically because we at least have encryption and authentication still :-) You can mitigate the lack of authorization by having a dedicated CA certificate just for QEMU services, so that you limit who has access to client certs. This is a crude authorization setup that is none the less acceptable in many scenarios.

Quoted 16 lines

qapi/util.json line 31

Quoted 13 lines
qapi/util.json new file +13 -0
+##
+# QAuthZSimpleRule:
+#
+# A single authorization rule.
+#
+# @match: a glob to match against a user identity
+# @policy: the result to return if @match evaluates to true
+#
+# Since: 2.6
+##
+{ 'struct': 'QAuthZSimpleRule',
+  'data': {'match': 'str',
+           'policy': 'QAuthZSimplePolicy'}}

Hmm. I was expecting something like:

Quoted 6 lines

{ 'struct': 'QAuthZSimple',
'data': { 'rules': [ 'QAuthZSimpleRule' ],

            'policy': 'QAuthZSimplePolicy' } }

but I guess that's one of our short-comings of QOM: the top-level structure does not have to be specified anywhere in QAPI.

I'm not sure I'd call it a short-coming but rather its just a difference in approach. For anything that is a user-defined object, the QAPI schema is defined implicitly by the QOM object or class definition. With the ability to define QOM properties against the class instad of object, we are actally in a position where we could auto-generate a QAPI schema to represent each user-defined object type if desired. Alternatively we could equally extend QAPI to have an 'object' type (which is a specialization of the 'struct' type) and auto-generate the basic boilerplate code to define a QOM object class from it.

Quoted 11 lines

tests/test-authz-simple.c line 33

Quoted 8 lines
tests/test-authz-simple.c new file +8 -0
+static void test_authz_default_deny(void)
+{
+    QAuthZSimple *auth = qauthz_simple_new("auth0",
+                                           QAUTHZ_SIMPLE_POLICY_DENY,
+                                           &error_abort);
+
+    g_assert(!qauthz_is_allowed(QAUTHZ(auth), "fred", &error_abort));
+

Okay, so you definitely intend to return false without setting errp, so it is a ternary result.

Yep

Quoted 9 lines

tests/test-authz-simple.c line 77

Quoted 3 lines
tests/test-authz-simple.c new file +3 -0
+#ifdef CONFIG_FNMATCH
+static void test_authz_complex(void)
+{

Wait - the behavior depends on whether fnmatch() is available? That is, a name is a literal match if fnmatch() is not present, and glob if present? I'd argue that if fnmatch() is not present, we must explicitly reject any "match" with glob metacharacters, rather than accidentally matching only a literal "*" when a glob was intended.

Historically we have the qemu/acl.c code which uses fnmatch and falls back to plain strcmp if not available. Since this was intended to be an exact drop-in replacement, I tried to implement the same logic here.

We could do something slightly different though - define a property against the QAuthzSimple object which enumerates the type of matching scheme - exact, glob or regex. The acl.c compatibility layer could then simply set matchtype=exact when !CONFIG_FNMATCH, which would in turn allow the QAuthzSimple impl to raise a fatall error for use of globbing when fnmatch is missing.

Quoted 15 lines

# How to interpret 'match', as a literal string or a glob { 'enum': 'QAuthZSimpleStyle', 'data': [ 'literal', 'glob' ] } # @style: #optional, default to 'literal'
{ 'struct': 'QAuthZSimpleRule',
'data': { 'match': 'str', '*style': 'QAuthZSimpleStyle',

            'policy': 'QAuthZSimplePolicy' } }

and used as:

         "rules": [
            { "match": "fred", "policy": "allow" },
            { "match": "bob", "policy": "allow" },
            { "match": "danb", "policy": "deny" },
            { "match": "dan*", "policy": "allow", "style": "glob" }
Quoted 3 lines

where you can then gracefully error out for ALL attempts to use "style":"glob" if fnmatch() is not present, and use strcmp() even when fnmatch() is present for rules that have the (default) "style":"literal".

Yep, allowing style to be set per rule is another option - I wonder if it is better todo it per-rule or per-ACL applying to all rules.

Quoted 19 lines
Quoted 19 lines
util/authz-simple.c new file +19 -0
+static void
+qauthz_simple_class_init(ObjectClass *oc, void *data)
+{
+    UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+    QAuthZClass *authz = QAUTHZ_CLASS(oc);
+
+    ucc->complete = qauthz_simple_complete;
+    authz->is_allowed = qauthz_simple_is_allowed;
+
+    object_class_property_add_enum(oc, "policy",
+                                   "QAuthZSimplePolicy",
+                                   QAuthZSimplePolicy_lookup,
+                                   qauthz_simple_prop_get_policy,
+                                   qauthz_simple_prop_set_policy,
+                                   NULL);
+
+    object_class_property_add(oc, "rules", "QAuthZSimpleRule",
+                              qauthz_simple_prop_get_rules,
+                              qauthz_simple_prop_set_rules,
Quoted 8 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+                              NULL, NULL, NULL);
+}

Not for this patch, but it would be cool if we had enough framework to just tell QOM to instantiate an object with callbacks by merely pointing to the name of a QAPI type that the object implements (referring back to my comment that I'm a bit surprised we didn't need a QAPI type for the overall QAuthZSimple).

Yep, as mentioned above this is just the difference between QAPI and QOM user creatable objects that needs resolving somehow.

Quoted 42 lines
Quoted 12 lines
util/authz-simple.c new file +12 -0
+size_t qauthz_simple_append_rule(QAuthZSimple *auth, const char *match,
+                                 QAuthZSimplePolicy policy)
+{
+    QAuthZSimpleRule *rule;
+    QAuthZSimpleRuleList *rules, *tmp;
+    size_t i = 0;
+
+    rule = g_new0(QAuthZSimpleRule, 1);
+    rule->policy = policy;
+    rule->match = g_strdup(match);
+
+    tmp = g_new0(QAuthZSimpleRuleList, 1);

util/authz-simple.c line 167

Quoted 10 lines
util/authz-simple.c new file +10 -0
+    tmp->value = rule;
+
+    rules = auth->rules;
+    if (rules) {
+        while (rules->next) {
+            i++;
+            rules = rules->next;
+        }
+        rules->next = tmp;
+        return i + 1;

No checks whether 'match' is already an existing rule...

Quoted 16 lines
util/authz-simple.c new file +16 -0
+ssize_t qauthz_simple_delete_rule(QAuthZSimple *auth, const char *match)
+{
+    QAuthZSimpleRule *rule;
+    QAuthZSimpleRuleList *rules, *prev;
+    size_t i = 0;
+
+    prev = NULL;
+    rules = auth->rules;
+    while (rules) {
+        rule = rules->value;
+        if (g_str_equal(rule->match, match)) {
+            if (prev) {
+                prev->next = rules->next;
+            } else {
+                auth->rules = rules->next;
+            }
Quoted 8 lines

util/authz-simple.c line 224

Quoted 3 lines
util/authz-simple.c new file +3 -0
+            rules->next = NULL;
+            qapi_free_QAuthZSimpleRuleList(rules);
+            return i;

...which means a rule can be inserted more than once, and this only removes the first instance of the rule. Do we care enough to add in additional checking that we aren't permitting duplicate 'match' strings in the list of rules?

I don't think it really matters much - if you add 2 rules with the same string, you still need 2 calls to delete it. The order of deletion can be left undefined I think.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#22 lore raw

Re: [Qemu-devel] [PATCH v3 07/10] qemu-nbd: add support for ACLs for TLS clients

In reply to Eric Blake
Quoted 27 lines · Eric Blake
Quoted 18 lines · Daniel P. Berrange

Currently any client which can complete the TLS handshake is able to use the NBD server. The server admin can turn on the 'verify-peer' option for the x509 creds to require the client to provide a x509 certificate. This means the client will have to acquire a certificate from the CA before they are permitted to use the NBD server. This is still a fairly weak bar.

This adds a '--tls-acl ACL-ID' option to the qemu-nbd command which takes the ID of a previously added 'QAuthZ' object instance. This ACL will be used to validate the client's x509 distinguished name. Clients failing the ACL will not be permitted to use the NBD server.

For example to setup an ACL that only allows connection from a client whose x509 certificate distinguished name contains 'CN=fred', you would use:

Quoted 7 lines
  qemu-nbd -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                   endpoint=server,verify-peer=yes \
           -object authz-simple,id=acl0,policy=deny,\
	           rules.0.match=*CN=fred,rules.0.policy=allow \
           -tls-creds tls0 \
           -tls-acl acl0
	   ....other qemu-nbd args...
Quoted 3 lines

Ah, so you are arguing that this is feature-completion of work started in 2.6, continuing work started before soft-freeze, and not a new feature to be delayed to 2.7.

Yes and v1 of this patch series was in fact posted just a few days before the soft-freeze deadline.

That said, as mentioned in the earlier patch, I'm open minded about whether this goes in 2.6 or 2.7. It would be nice to have in 2.6 but not the end of the world if it misses it, as overall we're still waaaaaaaay better off compared to 2.5 even if this doesn't mege :-)

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#29 lore raw

Re: [Qemu-devel] [PATCH v3 10/10] vnc: allow specifying a custom ACL object name

In reply to Eric Blake
Quoted 33 lines · Eric Blake
Quoted 24 lines · Daniel P. Berrange

The VNC server has historically had support for ACLs to check both the SASL username and the TLS x509 distinguished name. The VNC server was responsible for creating the initial ACL, and the client app was then responsible for populating it with rules using the HMP 'acl_add' command.

This is not satisfactory for a variety of reasons. There is no way to populate the ACLs from the command line, users are forced to use the HMP. With multiple network services all supporting TLS and ACLs now, it is desirable to be able to define a single ACL that is referenced by all services.

To address these limitations, two new options are added to the VNC server CLI. The 'tls-acl' option takes the ID of a QAuthZ object to use for checking TLS x509 distinguished names, and the 'sasl-acl' option takes the ID of another object to use for checking SASL usernames.

In this example, we setup two ACLs. The first allows any client with a certificate issued by the 'RedHat' organization in the 'London' locality. The second ACL allows clients with either the '[email protected]' or '[email protected]' kerberos usernames. Both ACLs must pass for the user to be allowed.

Quoted 5 lines
    $QEMU -object tls-creds-x509,id=tls0,dir=/home/berrange/qemutls,\
                  endpoint=server,verify-peer=yes \
          -object authz-simple,id=acl0,policy=deny,\
                  rules.0.match=O=RedHat,,L=London,rules.0.policy=allow \
          -object authz-simple,id=acl0,policy=deny,\

Umm, you can't reuse 'acl0' as the id.

Quoted 8 lines
Quoted 4 lines
                  [email protected],rules.0.policy=allow \
                  [email protected],rules.0.policy=allow \
          -vnc 0.0.0.0:1,tls-creds=tls0,tls-acl=tlsacl0,
	       sasl,sasl-acl=saslacl0 \

And this fails because the ids don't exist. I think you meant authz-simple,id=tlsacl0 in the first instance, and authz-simple,id=saslacl0 in the second instance.

Heh, yeah, I really ought to try the examples I put in the commit message tomake sure they work :-)

Quoted 20 lines
Quoted 20 lines
---
 ui/vnc.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++------------
 1 file changed, 60 insertions(+), 13 deletions(-)
ui/vnc.c +11 -0
@@ -3670,6 +3680,21 @@ void vnc_display_open(const char *id, Error **errp)
         }
     }
     acl = qemu_opt_get_bool(opts, "acl", false);
+    tlsacl = qemu_opt_get(opts, "tls-acl");
+    if (acl && tlsacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
+                   "'tls-acl' options");
+        goto fail;
+    }
+
+#ifdef CONFIG_VNC_SASL
+    saslacl = qemu_opt_get(opts, "sasl-acl");
+    if (acl && saslacl) {
+        error_setg(errp, "'acl' option is mutually exclusive with the "
Quoted 8 lines

ui/vnc.c void vnc_display_open(const char *id, Error **errp) · line 3697

Quoted 4 lines
ui/vnc.c +4 -0
+                   "'sasl-acl' options");
+        goto fail;
+    }
+#endif

Do we explicitly fail if sasl-acl was provided but CONFIG_VNC_SASL is not defined? It looks here like you silently ignore it, which would not be good.

Yes, we should really raise the error unconditionally.

Quoted 47 lines
Quoted 47 lines
ui/vnc.c +30 -12
@@ -3710,19 +3737,39 @@ void vnc_display_open(const char *id, Error **errp)
                           &error_abort);
     }
 #ifdef CONFIG_VNC_SASL
-    if (acl && sasl) {
-        char *aclname;
+    if (sasl) {
+        if (saslacl) {
+            Object *container, *acl;
+            container = object_get_objects_root();
+            acl = object_resolve_path_component(container, saslacl);
+            if (!acl) {
+                error_setg(errp, "Cannot find ACL %s", saslacl);
+                goto fail;
+            }
 
-        if (strcmp(vs->id, "default") == 0) {
-            aclname = g_strdup("vnc.username");
-        } else {
-            aclname = g_strdup_printf("vnc.%s.username", vs->id);
-        }
-        vs->sasl.acl =
-            QAUTHZ(qauthz_simple_new(aclname,
-                                     QAUTHZ_SIMPLE_POLICY_DENY,
-                                     &error_abort));
-        g_free(aclname);
+            if (!object_dynamic_cast(acl, TYPE_QAUTHZ)) {
+                error_setg(errp, "Object '%s' is not a QAuthZ subclass",
+                           saslacl);
+                goto fail;
+            }
+            vs->sasl.acl = QAUTHZ(acl);
+        } else if (acl) {
+            char *aclname;
+
+            if (strcmp(vs->id, "default") == 0) {
+                aclname = g_strdup("vnc.username");
+            } else {
+                aclname = g_strdup_printf("vnc.%s.username", vs->id);
+            }
+            vs->sasl.acl =
+                QAUTHZ(qauthz_simple_new(aclname,
+                                         QAUTHZ_SIMPLE_POLICY_DENY,
+                                         &error_abort));
+            g_free(aclname);
+        }
+    } else if (saslacl) {
Quoted 8 lines

ui/vnc.c void vnc_display_open(const char *id, Error **errp) · line 3775

Quoted 5 lines
ui/vnc.c +2 -0
+        error_setg(errp, "SASL ACL provided when SASL is disabled");
+        goto fail;
     }
 #endif
 

Again, the saslacl check is only mentioned inside the #if; what happens when the #if is not compiled?

Yeah, I should fix that.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|