Skip to contentSkip to conversation

Basic device state visualization

PATCH In review Jan Kiszka 10 participants 53 messages May 14, 2010 – May 19, 2010 Kbuild

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

cover Basic device state visualization

While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.

This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show <qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:

(qemu) device_show /i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.

    version_id:         00000002
    config:             a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00
                        ...
    irq_state:          00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00
  num_ports_vmstate:    02
  ports[00].
      ctrl:             0083
  ports[01].
      ctrl:             0080
  cmd:                  00c1
  status:               0000
  intr:                 0000
  frnum:                0077
  fl_base_addr:         0fffc000
  sof_timing:           40
  status2:              00
  frame_timer:          0000000000cb2bd0

Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...

Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!

Jan Kiszka (8):
qdev: Allow device addressing via 'driver.instance' Add base64 encoder/decoder
Add QBuffer
monitor: Add basic device state visualization
qmp: Teach basic capability negotiation to python example
qmp: Fix python helper /wrt long return strings Add QLIST_INSERT_TAIL
qdev: Add new devices/buses at the tail

 Makefile        |    3 +-
 Makefile.objs   |    4 +-
 QMP/qmp-shell   |    1 +
 QMP/qmp.py      |    2 +-
 QMP/vm-info     |    1 +
 base64.c        |  202 +++++++++++++++++++++++++++++++++++
 base64.h        |   18 +++
 check-qbuffer.c |  172 ++++++++++++++++++++++++++++++
 configure       |    2 +-
 hw/hw.h         |    2 +
 hw/qdev.c       |  314 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 hw/qdev.h       |    2 +
 qbuffer.c       |  116 ++++++++++++++++++++
 qbuffer.h       |   33 ++++++
 qemu-monitor.hx |   16 +++
 qemu-queue.h    |    9 ++
 qjson.c         |   16 +++
 qobject.h       |    1 +
 18 files changed, 902 insertions(+), 12 deletions(-)
 create mode 100644 base64.c
 create mode 100644 base64.h
 create mode 100644 check-qbuffer.c
 create mode 100644 qbuffer.c
 create mode 100644 qbuffer.h
author
#1 lore raw

PATCH 1/8 qdev: Allow device addressing via 'driver.instance'

Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.

---
 hw/qdev.c |   23 ++++++++++++++++++-----
 1 files changed, 18 insertions(+), 5 deletions(-)
hw/qdev.c +18 -5
index d3bf0fa..fe49e71 100644
@@ -515,28 +515,41 @@ static BusState *qbus_find_bus(DeviceState *dev, char *elem)
     return NULL;
 }
 
-static DeviceState *qbus_find_dev(BusState *bus, char *elem)
+static DeviceState *qbus_find_dev(BusState *bus, const char *elem)
 {
     DeviceState *dev;
+    int instance, n;
+    char buf[128];
 
     /*
      * try to match in order:
      *   (1) instance id, if present
-     *   (2) driver name
-     *   (3) driver alias, if present
+     *   (2) driver name [.instance]
+     *   (3) driver alias [.instance], if present
      */
     QLIST_FOREACH(dev, &bus->children, sibling) {
         if (dev->id  &&  strcmp(dev->id, elem) == 0) {
             return dev;
         }
     }
+
+    if (sscanf(elem, "%127[^.].%u", buf, &instance) == 2) {
+        elem = buf;
+    } else {
+        instance = 0;
+    }
+
+    n = 0;
     QLIST_FOREACH(dev, &bus->children, sibling) {
-        if (strcmp(dev->info->name, elem) == 0) {
+        if (strcmp(dev->info->name, elem) == 0 && n++ == instance) {
             return dev;
         }
     }
+
+    n = 0;
     QLIST_FOREACH(dev, &bus->children, sibling) {
-        if (dev->info->alias && strcmp(dev->info->alias, elem) == 0) {
+        if (dev->info->alias && strcmp(dev->info->alias, elem) == 0 &&
+            n++ == instance) {
             return dev;
         }
     }
Signature · 1 line
1.6.0.2
#2 lore raw

Re: [Qemu-devel] [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'

Quoted 5 lines · Jan Kiszka

Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.

How's the instance number defined? Should be documented.

Is an instance number a good user interface? Gerd, what do you think?

[...]

#3 lore raw

Re: [Qemu-devel] [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'

Quoted 9 lines · Markus Armbruster
Quoted 5 lines · Jan Kiszka

Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.

How's the instance number defined? Should be documented.

savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think. Drivers can also specify one. Most don't do that. IIRC some ISA drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.

It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.

cheers,
Gerd

#4 lore raw

[Qemu-devel] Re: [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'

Quoted 18 lines · Gerd Hoffmann
Quoted 9 lines · Markus Armbruster
Quoted 5 lines · Jan Kiszka

Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.

How's the instance number defined? Should be documented.

savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think. Drivers can also specify one. Most don't do that. IIRC some ISA drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.

Quoted 3 lines

It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.

Agreed. The problem here is that we sent the instance_id on the wire, so for "legacy" devices that used an instance_id != -1, we are stuck with it :(

Guess why a friend calls "backwards compatibility": A Curse of Bible proportions :(

Later, Juan.

#5 lore raw

[Qemu-devel] Re: [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'

Hi,

Quoted 3 lines

Agreed. The problem here is that we sent the instance_id on the wire, so for "legacy" devices that used an instance_id != -1, we are stuck with it :(

Another way would be to fill the instance id with something useful, say encode the pci address there for pci devices. That should be good enough to reliably identify devices even after hot-plugging them in and out.

Of course this has transition isses too. But maybe it is easier to handle than replacing instance_id with something entirely different.

And it also makes the instance id less useful to address devices in a human-friendly way.

cheers,
Gerd

#6 lore raw

[Qemu-devel] Re: [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'

In reply to Gerd Hoffmann
Quoted 10 lines · Gerd Hoffmann
Quoted 9 lines · Markus Armbruster
Quoted 5 lines · Jan Kiszka

Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.

How's the instance number defined? Should be documented.

OK, will do. For now it is the per-bus instance ID. I think that makes most sense and is easily handleable. Still, we should probably print it also via "info qtree" (and a future "query-qtree").

Quoted 7 lines

savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think. Drivers can also specify one. Most don't do that. IIRC some ISA drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.

That io-address-based instance numbers have just been deprecated, see 4d2ffa08b601bdd40d9ccf225480c0a7e90ca078. ISA devices are already converted, there are just a few non-PC devices remaining that don't use the (auto-generated) vmstate instance number. But that is actually a different, user-invisible numbering scheme.

Quoted 4 lines

It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.

You are always free to address devices via a unique user-defined ID.

Jan

#7 lore raw

Re: [Qemu-devel] [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'

In reply to Gerd Hoffmann
Quoted 7 lines · Gerd Hoffmann
Quoted 6 lines · Markus Armbruster
Quoted 4 lines · Jan Kiszka

Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

How's the instance number defined? Should be documented.

savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think.

That's an implementation detail that's being exposed in an external interface. Granted, users shouldn't expect this to remain stable across invocations, yet it makes me uncomfortable.

Why not always use topology to locate devices?

Quoted 4 lines

Drivers can also specify one. Most don't do that. IIRC some ISA
drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.

It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.

Oh, that's what you're suggesting. So we agree.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
author
#8 lore raw

PATCH 2/8 Add base64 encoder/decoder

In reply to Jan Kiszka

Will be used by QBuffer.

---
 Makefile.objs |    2 +-
 base64.c      |  202 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 base64.h      |   18 +++++
 3 files changed, 221 insertions(+), 1 deletions(-)
 create mode 100644 base64.c
 create mode 100644 base64.h
Makefile.objs +1 -1
index ecdd53e..3d2a27a 100644
@@ -2,7 +2,7 @@
 # QObject
 qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o
 qobject-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o
-qobject-obj-y += qerror.o
+qobject-obj-y += qerror.o base64.o
 
 #######################################################################
 # block-obj-y is code used by both qemu system emulation and qemu-img
base64.c new file +202 -0

202 lines not on this page · raw message

base64.h new file +18 -0
new file mode 100644
index 0000000..9a0e03a
@@ -0,0 +1,18 @@
+/*
+ * Base64 encoder/decoder conforming to RFC 4648
+ * (based on Mozilla's nsprpub/lib/libc/src/base64.c)
+ *
+ * Copyright (C) 2010 Siemens AG
+ *
+ * Authors:
+ *  Jan Kiszka <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#include <unistd.h>
+
+void base64_encode(const void *src, size_t srclen, char *dest);
+int base64_decode(const char *src, size_t srclen, void *dest);
Signature · 1 line
1.6.0.2
author
#9 lore raw

PATCH 3/8 Add QBuffer

In reply to Jan Kiszka

This introduces a buffer object for use with QMP. As a buffer is not natively encodable in JSON, we encode it as a base64 string. To decode this kind of strings back to a QBuffer, the receiving side has to be aware of their semantic, which is normally no problem within QMP.

The first use case of this type is pushing the content of buffers that are part of a device state into a qdict.

---
 Makefile        |    3 +-
 Makefile.objs   |    2 +-
 check-qbuffer.c |  172 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 configure       |    2 +-
 qbuffer.c       |  116 +++++++++++++++++++++++++++++++++++++
 qbuffer.h       |   33 +++++++++++
 qjson.c         |   16 +++++
 qobject.h       |    1 +
 8 files changed, 342 insertions(+), 3 deletions(-)
 create mode 100644 check-qbuffer.c
 create mode 100644 qbuffer.c
 create mode 100644 qbuffer.h
Makefile +2 -1
index eb9e02b..065c1a5 100644
@@ -149,7 +149,8 @@ check-qstring: check-qstring.o qstring.o qemu-malloc.o
 check-qdict: check-qdict.o qdict.o qfloat.o qint.o qstring.o qbool.o qemu-malloc.o qlist.o
 check-qlist: check-qlist.o qlist.o qint.o qemu-malloc.o
 check-qfloat: check-qfloat.o qfloat.o qemu-malloc.o
-check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o qemu-malloc.o
+check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qbuffer.o base64.o qjson.o json-streamer.o json-lexer.o json-parser.o qemu-malloc.o
+check-qbuffer: check-qbuffer.o qbuffer.o base64.o qstring.o qemu-malloc.o
 
 clean:
 # avoid old build problems by removing potentially incorrect old files
Makefile.objs +1 -1
index 3d2a27a..52c8ec7 100644
@@ -1,6 +1,6 @@
 #######################################################################
 # QObject
-qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o
+qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o qbuffer.o
 qobject-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o
 qobject-obj-y += qerror.o base64.o
 
check-qbuffer.c new file +172 -0

172 lines not on this page · raw message

configure +1 -1
index 36d028f..9ff9308 100755
@@ -2280,7 +2280,7 @@ if test `expr "$target_list" : ".*softmmu.*"` != 0 ; then
   if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then
       tools="qemu-nbd\$(EXESUF) $tools"
     if [ "$check_utests" = "yes" ]; then
-      tools="check-qint check-qstring check-qdict check-qlist $tools"
+      tools="check-qint check-qstring check-qdict check-qlist check-qbuffer $tools"
       tools="check-qfloat check-qjson $tools"
     fi
   fi
qbuffer.c new file +116 -0

116 lines not on this page · raw message

qbuffer.h new file +33 -0
new file mode 100644
index 0000000..2e01078
@@ -0,0 +1,33 @@
+/*
+ * QBuffer Module
+ *
+ * Copyright (C) 2010 Siemens AG
+ *
+ * Authors:
+ *  Jan Kiszka <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ *
+ */
+
+#ifndef QBUFFER_H
+#define QBUFFER_H
+
+#include <stdint.h>
+#include "qobject.h"
+#include "qstring.h"
+
+typedef struct QBuffer {
+    QObject_HEAD;
+    void *data;
+    size_t size;
+} QBuffer;
+
+QBuffer *qbuffer_from_data(const void *data, size_t size);
+QBuffer *qbuffer_from_qstring(const QString *string);
+const void *qbuffer_get_data(const QBuffer *qb);
+size_t qbuffer_get_size(const QBuffer *qb);
+QBuffer *qobject_to_qbuffer(const QObject *obj);
+
+#endif /* QBUFFER_H */
qjson.c +16 -0
index 483c667..4d1c21a 100644
@@ -19,7 +19,9 @@
 #include "qlist.h"
 #include "qbool.h"
 #include "qfloat.h"
+#include "qbuffer.h"
 #include "qdict.h"
+#include "base64.h"
 
 typedef struct JSONParsingState
 {
@@ -235,6 +237,20 @@ static void to_json(const QObject *obj, QString *str)
         }
         break;
     }
+    case QTYPE_QBUFFER: {
+        QBuffer *val = qobject_to_qbuffer(obj);
+        size_t data_size = qbuffer_get_size(val);
+        size_t str_len = ((data_size + 2) / 3) * 4;
+        char *buffer = qemu_malloc(str_len + 3);
+
+        buffer[0] = '"';
+        base64_encode(qbuffer_get_data(val), data_size, buffer + 1);
+        buffer[str_len + 1] = '"';
+        buffer[str_len + 2] = 0;
+        qstring_append(str, buffer);
+        qemu_free(buffer);
+        break;
+    }
     case QTYPE_QERROR:
         /* XXX: should QError be emitted? */
     case QTYPE_NONE:
qobject.h +1 -0
index 07de211..45c4fa0 100644
@@ -44,6 +44,7 @@ typedef enum {
     QTYPE_QFLOAT,
     QTYPE_QBOOL,
     QTYPE_QERROR,
+    QTYPE_QBUFFER,
 } qtype_code;
 
 struct QObject;
Signature · 1 line
1.6.0.2
#10 lore raw

Re: [Qemu-devel] [PATCH 3/8] Add QBuffer

Quoted 23 lines · Jan Kiszka
qjson.c +7 -0
index 483c667..4d1c21a 100644
@@ -19,7 +19,9 @@
  #include "qlist.h"
  #include "qbool.h"
  #include "qfloat.h"
+#include "qbuffer.h"
  #include "qdict.h"
+#include "base64.h"
 
  typedef struct JSONParsingState
  {
@@ -235,6 +237,20 @@ static void to_json(const QObject *obj, QString *str)
          }
          break;
      }
+    case QTYPE_QBUFFER: {
+        QBuffer *val = qobject_to_qbuffer(obj);
+        size_t data_size = qbuffer_get_size(val);
+        size_t str_len = ((data_size + 2) / 3) * 4;
+        char *buffer = qemu_malloc(str_len + 3);

qjson.c static void to_json(const QObject *obj, QString *str) · line 254

Quoted 10 lines
qjson.c +9 -0
+
+        buffer[0] = '"';
+        base64_encode(qbuffer_get_data(val), data_size, buffer + 1);
+        buffer[str_len + 1] = '"';
+        buffer[str_len + 2] = 0;
+        qstring_append(str, buffer);
+        qemu_free(buffer);
+        break;
+    }
 

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.

The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.

Regards,

Anthony Liguori

Quoted 16 lines
qjson.c +0 -0
      case QTYPE_QERROR:
          /* XXX: should QError be emitted? */

case QTYPE_NONE:

qobject.h +1 -0
index 07de211..45c4fa0 100644
@@ -44,6 +44,7 @@ typedef enum {
      QTYPE_QFLOAT,
      QTYPE_QBOOL,
      QTYPE_QERROR,
+    QTYPE_QBUFFER,
  } qtype_code;
 
  struct QObject;
#11 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 41 lines · Anthony Liguori
Quoted 24 lines · Jan Kiszka
qjson.c +7 -0
index 483c667..4d1c21a 100644
@@ -19,7 +19,9 @@
  #include "qlist.h"
  #include "qbool.h"
  #include "qfloat.h"
+#include "qbuffer.h"
  #include "qdict.h"
+#include "base64.h"
 
  typedef struct JSONParsingState
  {
@@ -235,6 +237,20 @@ static void to_json(const QObject *obj, QString
 *str)
          }
          break;
      }
+    case QTYPE_QBUFFER: {
+        QBuffer *val = qobject_to_qbuffer(obj);
+        size_t data_size = qbuffer_get_size(val);
+        size_t str_len = ((data_size + 2) / 3) * 4;
+        char *buffer = qemu_malloc(str_len + 3);

qjson.c static void to_json(const QObject *obj, QString · line 255

Quoted 10 lines
qjson.c +9 -0
+
+        buffer[0] = '"';
+        base64_encode(qbuffer_get_data(val), data_size, buffer + 1);
+        buffer[str_len + 1] = '"';
+        buffer[str_len + 2] = 0;
+        qstring_append(str, buffer);
+        qemu_free(buffer);
+        break;
+    }
 

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

Quoted 7 lines

We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.

The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.

Indeed, was amazingly simple to add and works nicely with qmp-shell as demonstrator. Will repost, also to fix a few remaining glitches I came across in the meantime.

Jan

#12 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 10 lines · Jan Kiszka
Quoted 10 lines

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.

Quoted 9 lines
Quoted 4 lines

The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.

Indeed, was amazingly simple to add and works nicely with qmp-shell as demonstrator. Will repost, also to fix a few remaining glitches I came across in the meantime.

Is this __class__ stuff documented anywhere?

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#13 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 15 lines · Avi Kivity
Quoted 14 lines · Jan Kiszka
Quoted 14 lines

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.

The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.

Quoted 8 lines
Quoted 5 lines

Indeed, was amazingly simple to add and works nicely with qmp-shell as demonstrator. Will repost, also to fix a few remaining glitches I came across in the meantime.

Is this __class__ stuff documented anywhere?

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Jan

#14 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 9 lines · Jan Kiszka
Quoted 4 lines

Is this __class__ stuff documented anywhere?

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Why use such weird names at all? What's wrong with 'class'?

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#15 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines

Is this __class__ stuff documented anywhere?

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Why use such weird names at all? What's wrong with 'class'?

That it conflicts with e.g. PCI classes?

Paolo

#16 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Avi Kivity
Quoted 3 lines · Jan Kiszka
Quoted 3 lines

Is this __class__ stuff documented anywhere?

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Why use such weird names at all? What's wrong with 'class'?

That it conflicts with e.g. PCI classes?

Won't the context tell it apart? When you expect a pci function, 'class': 'video' means one thing, when you read a buffer it means another.

The only reason to use a special name is if it's a protocol level feature that can happen in all contexts. But in that case we're better off with a schema, we don't want to push class descriptors everywhere.

Signature · 1 line
error compiling committee.c: too many arguments to function
#17 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 11 lines · Avi Kivity
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines

Is this __class__ stuff documented anywhere?

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Why use such weird names at all? What's wrong with 'class'?

That it conflicts with e.g. PCI classes?

Won't the context tell it apart? When you expect a pci function, 'class': 'video' means one thing, when you read a buffer it means another.

The point is to make this notation context-independent so that you can do

    def __json_obj_hook(self, dct):
        if '__class__' in dct:
            if dct['__class__'] == 'buffer':
                return base64.b64decode(dct['data'])
            else:
                return
        return dct

and

line = json.loads(self.sockfile.readline(),
object_hook=self.__json_obj_hook)

i.e. parse the QMP stream into a proper representation without known QMP at all.

Quoted 5 lines

The only reason to use a special name is if it's a protocol level feature that can happen in all contexts. But in that case we're better off with a schema, we don't want to push class descriptors everywhere.

Not everywhere, just into those nodes that aren't expressible with native JSON types.

Jan

#18 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

In reply to Avi Kivity
Quoted 11 lines · Avi Kivity
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines

Is this __class__ stuff documented anywhere?

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Why use such weird names at all? What's wrong with 'class'?

That it conflicts with e.g. PCI classes?

Won't the context tell it apart?

Yes, of course, it you need to know the schema. If you don't know the schema you don't know the context.

This QBuffer thing is something that a client QMP library could create automatically. Keys in a separate namespace (like '__class__') have the advantage of being easily picked up automatically by a wrapper of the JSON parser; if you used simply 'class' such as layer would need to know a schema, or it wouldn't know that "context".

(BTW I'd prefer something like '__encoding__'; the word "class" suggests much more than what it is in reality).

Paolo

#19 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 14 lines · Paolo Bonzini
Quoted 13 lines · Avi Kivity
Quoted 12 lines · Paolo Bonzini
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines

Is this __class__ stuff documented anywhere?

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Why use such weird names at all? What's wrong with 'class'?

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

That it conflicts with e.g. PCI classes?

Won't the context tell it apart?

Yes, of course, it you need to know the schema. If you don't know the schema you don't know the context.

This QBuffer thing is something that a client QMP library could create automatically. Keys in a separate namespace (like '__class__') have the advantage of being easily picked up automatically by a wrapper of the JSON parser; if you used simply 'class' such as layer would need to know a schema, or it wouldn't know that "context".

Makes sense. So this is a protocol feature and needs to be documented as such.

Quoted 1 line

(BTW I'd prefer something like '__encoding__'; the word "class" suggests much more than what it is in reality).

Agreed.

Signature · 1 line
error compiling committee.c: too many arguments to function
#20 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

In reply to Avi Kivity
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka
Quoted 4 lines

Is this __class__ stuff documented anywhere?

Quoted 7 lines
Quoted 5 lines

Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').

Why use such weird names at all? What's wrong with 'class'?

It's too generic and we may too easily create dict keys that will be misinterpreted.

Jan

#21 lore raw

Re: [Qemu-devel] [PATCH 3/8] Add QBuffer

In reply to Anthony Liguori
Quoted 4 lines · Anthony Liguori

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

Is there a benefit to the class indirection, over simply a keyword?:

{'__base64__': ...}

__class__ seems to suggest much more than it's being used for here.

-- Jamie

#22 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 4 lines · Jamie Lokier
Quoted 3 lines · Anthony Liguori

Instead of encoding just as a string, it would be a good idea to encode it as something like:

Quoted 8 lines
Quoted 1 line

{'__class__': 'base64', 'data': ...}

Is there a benefit to the class indirection, over simply a keyword?:

{'__base64__': ...}

__class__ seems to suggest much more than it's being used for here.

Depending on how sophisticated your parser is, you could directly push the result into an object of the proper type. And we can add more complex objects in the future that do not only consists of a single data key. Note that this extension is not just about encoding, it is about typecasting (dict -> custom type).

Jan

#23 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 11 lines · Jan Kiszka
Quoted 10 lines · Jamie Lokier
Quoted 4 lines · Anthony Liguori

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

Is there a benefit to the class indirection, over simply a keyword?:

{'__base64__': ...}

Quoted 8 lines
Quoted 2 lines

__class__ seems to suggest much more than it's being used for here.

Depending on how sophisticated your parser is, you could directly push the result into an object of the proper type. And we can add more complex objects in the future that do not only consists of a single data key. Note that this extension is not just about encoding, it is about typecasting (dict -> custom type).

Sure, if that's the plan.

Does it make sense to combine encoding and custom types in this way? It looks like mixing syntax and semantics, which has consequences for code using generic parsers with separate semantic layer, but I realise there's no "correct" answer.

Back to the syntax: I'm under the impression from earlier discussion that the '__*__' keyspace reserved, so even types could use the compact syntax?

Or is there something Javascript-ish (and not merely JSON-ish) about '__class__' in particular which makes it appropriate?

-- Jamie

#24 lore raw

Re: [Qemu-devel] [PATCH 3/8] Add QBuffer

In reply to Jamie Lokier
Quoted 11 lines · Jamie Lokier
Quoted 4 lines · Anthony Liguori

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

Is there a benefit to the class indirection, over simply a keyword?:

{'__base64__': ...}

__class__ seems to suggest much more than it's being used for here.

Yes. The problem with JSON is that it's based on JavaScript and JavaScript is goofy :-)

JavaScript's object mechanism doesn't map well to most other languages since it's prototype based. What we're calling QDict's are really objects in JavaScript and they carry mostly no type information. With JS, it's very simple to treat a generic object as a specialized class after instantiation which means objects don't need type information.

For non-prototype languages, which is the vast majority of clients, it's necessary to have type information at instantiation time since monkey patching is awkward at best. That's why we need a special, reserved, object member to carry type information. The remainder of the object members represent the serialized state of the object.

Another way to think of it, is that we're already transmitting objects so we really just need a way to say, no, this isn't just a Dictionary, it's really an instance of the following class.

Regards,

Anthony Liguori

Quoted 2 lines

-- Jamie

#25 lore raw

Re: [Qemu-devel] [PATCH 3/8] Add QBuffer

Quoted 9 lines · Anthony Liguori
Quoted 7 lines · Jamie Lokier
Quoted 5 lines · Anthony Liguori

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

Quoted 10 lines
Quoted 6 lines

Is there a benefit to the class indirection, over simply a keyword?:

{'__base64__': ...}

__class__ seems to suggest much more than it's being used for here.

Yes. The problem with JSON is that it's based on JavaScript and JavaScript is goofy :-)

I suggest completely ignoring JavaScript. JSON is simply an encoding for numbers, strings, arrays, and key-value stores. Where's the goofiness?

Quoted 6 lines

JavaScript's object mechanism doesn't map well to most other languages since it's prototype based. What we're calling QDict's are really objects in JavaScript and they carry mostly no type information. With JS, it's very simple to treat a generic object as a specialized class after instantiation which means objects don't need type information.

Quoted 6 lines

For non-prototype languages, which is the vast majority of clients, it's necessary to have type information at instantiation time since monkey patching is awkward at best. That's why we need a special, reserved, object member to carry type information. The remainder of the object members represent the serialized state of the object.

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

Quoted 4 lines

Another way to think of it, is that we're already transmitting objects so we really just need a way to say, no, this isn't just a Dictionary, it's really an instance of the following class.

Are there cases where the receiver cannot infer this from the context?

As I see it, dynamic type information is easiest for dynamicically typed languages. You just have a dict of class names -> object constructor and call the constructors at runtime. Statically typed languages will need a schema to use objects, since the field types have to be known at compile time, not just run time.

Another wart is arrays: statically typed languages usually contain only objects of the same type, but here this isn't known until we process the first member.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
author
#26 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 38 lines · Avi Kivity
Quoted 10 lines · Anthony Liguori
Quoted 7 lines · Jamie Lokier
Quoted 5 lines · Anthony Liguori

Instead of encoding just as a string, it would be a good idea to encode it as something like:

{'__class__': 'base64', 'data': ...}

Quoted 10 lines
Quoted 6 lines

Is there a benefit to the class indirection, over simply a keyword?:

{'__base64__': ...}

__class__ seems to suggest much more than it's being used for here.

Yes. The problem with JSON is that it's based on JavaScript and JavaScript is goofy :-)

I suggest completely ignoring JavaScript. JSON is simply an encoding for numbers, strings, arrays, and key-value stores. Where's the goofiness?

Quoted 6 lines

JavaScript's object mechanism doesn't map well to most other languages since it's prototype based. What we're calling QDict's are really objects in JavaScript and they carry mostly no type information. With JS, it's very simple to treat a generic object as a specialized class after instantiation which means objects don't need type information.

Quoted 6 lines

For non-prototype languages, which is the vast majority of clients, it's necessary to have type information at instantiation time since monkey patching is awkward at best. That's why we need a special, reserved, object member to carry type information. The remainder of the object members represent the serialized state of the object.

Quoted 6 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#27 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 5 lines · Jan Kiszka
Quoted 5 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Quoted 9 lines
Quoted 3 lines

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

What is the receiver to do with it?

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

My worry is that __class__ will make the protocol more ad-hoc.

Quoted 3 lines

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
author
#28 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 15 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka
Quoted 4 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Quoted 9 lines
Quoted 3 lines

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

Quoted 5 lines

What is the receiver to do with it?

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

There is a schema describing the fields (name, size, number of elements), but their types (int, buffer, sub-field, array of X) are derived from the JSON objects (ie. the JSON parser does this job).

Quoted 10 lines

My worry is that __class__ will make the protocol more ad-hoc.

Quoted 3 lines

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.

What clients do you have in mind?

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#29 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 19 lines · Jan Kiszka
Quoted 17 lines · Avi Kivity
Quoted 15 lines · Jan Kiszka
Quoted 8 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

Quoted 8 lines
Quoted 6 lines

What is the receiver to do with it?

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

There is a schema describing the fields (name, size, number of elements),

Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.

Quoted 3 lines

but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).

The names of fields are also type information.

Quoted 11 lines
Quoted 8 lines
Quoted 4 lines

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.

What clients do you have in mind?

Any client that doesn't allow object types to be created dynamically; C, C++, Java, and the like could all benefit from a schema and wouldn't be able to do much with __class__ unless all classes were predefined. Python, JavaScript, and the like wouldn't care.

Another way of looking at it: if the client sees { __class__: foo, f1: 10, f2: 9 }, it cannot derive any information from __class__ unless it was aware of foo beforehand. If that's the case, let's make it part of the schema so it is available at compile time instead of runtime.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#30 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 3 lines · Avi Kivity

Another way of looking at it: if the client sees { __class__: foo, f1: 10, f2: 9 }, it cannot derive any information from __class__ unless it was aware of foo beforehand. If that's the case, let's make it part of the schema so it is available at compile time instead of runtime.

Or both. That's similar to a non-object field's type being described both in the schema at during runtime. A static parser can verify __class__ matches the expectations, a dynamic parser can look up __class__ and call the appropriate constructor.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
author
#31 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

In reply to Avi Kivity
Quoted 29 lines · Avi Kivity
Quoted 19 lines · Jan Kiszka
Quoted 17 lines · Avi Kivity
Quoted 15 lines · Jan Kiszka
Quoted 8 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

Quoted 8 lines
Quoted 6 lines

What is the receiver to do with it?

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

There is a schema describing the fields (name, size, number of elements),

Quoted 8 lines

Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.

Quoted 3 lines

but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).

The names of fields are also type information.

Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.

Quoted 18 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

Quoted 7 lines
Quoted 4 lines

The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.

What clients do you have in mind?

Any client that doesn't allow object types to be created dynamically; C, C++, Java, and the like could all benefit from a schema and wouldn't be able to do much with __class__ unless all classes were predefined. Python, JavaScript, and the like wouldn't care.

Quoted 5 lines

Another way of looking at it: if the client sees { __class__: foo, f1:
10, f2: 9 }, it cannot derive any information from __class__ unless it was aware of foo beforehand. If that's the case, let's make it part of the schema so it is available at compile time instead of runtime.

Maybe a misunderstanding on my side: I'm not arguing against predefining what __class__ values exists and how dicts that carry such keys are encoded.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#32 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 7 lines · Jan Kiszka
Quoted 3 lines

The names of fields are also type information.

Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.

For that use case, I agree. Maybe we should send both the parsed and unparsed information.

But if the client isn't going to interpret the object and only display it, then there is no need for __class__?

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
author
#33 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 10 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka
Quoted 2 lines

The names of fields are also type information.

Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.

For that use case, I agree. Maybe we should send both the parsed and unparsed information.

Now I can't parse what you mean.

Quoted 3 lines

But if the client isn't going to interpret the object and only display it, then there is no need for __class__?

For that uncommon case, yes. But the common one is to perform a bit more than raw JSON dictionary printing.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#34 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 12 lines · Jan Kiszka
Quoted 10 lines · Avi Kivity
Quoted 8 lines · Jan Kiszka
Quoted 3 lines

The names of fields are also type information.

Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.

Quoted 5 lines
Quoted 3 lines

For that use case, I agree. Maybe we should send both the parsed and unparsed information.

Now I can't parse what you mean.

Sorry. I meant that if we have a raw buffer that we can decode (like pci config space and its fields) maybe it makes sense to send both formats. The raw buffer is something likely to be stable, and the decoded fields are more readable. But I realize now that doesn't make sense in the context of base64 encoding which started all this.

Quoted 6 lines
Quoted 3 lines

But if the client isn't going to interpret the object and only display it, then there is no need for __class__?

For that uncommon case, yes. But the common one is to perform a bit more than raw JSON dictionary printing.

But the common one is likely to know the type of the object beforehand (or it can't do much beyond printing, since it has no idea what fields to expect).

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#35 lore raw

Re: [Qemu-devel] Re: [PATCH 3/8] Add QBuffer

In reply to Jan Kiszka
Quoted 27 lines · Jan Kiszka
Quoted 26 lines · Avi Kivity
Quoted 25 lines · Jan Kiszka
Quoted 23 lines · Avi Kivity
Quoted 8 lines · Jan Kiszka
Quoted 8 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

Quoted 7 lines

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

What is the receiver to do with it?

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines

There is a schema describing the fields (name, size, number of elements),

Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.

Doesn't that tie you to a specific version of QMP at compile-time?

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).

The names of fields are also type information.

Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.

qdev device properties are similar. Right now, they occur only as arguments of device_add. When do_info_qtree() gets converted, they'll appear in results.

[...]

#36 lore raw

Re: [Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 6 lines · Markus Armbruster
Quoted 4 lines
Quoted 4 lines

Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.

Doesn't that tie you to a specific version of QMP at compile-time?

The client needs to ignore anything not provided by the schema to be forward compatible.

(alternatively we make sure anything new is explicitly enabled by the client, so it can parse strictly according to the schema).

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).

The names of fields are also type information.

Quoted 8 lines
Quoted 4 lines

Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.

qdev device properties are similar. Right now, they occur only as arguments of device_add. When do_info_qtree() gets converted, they'll appear in results.

This sounds like a horror movie in the making, "sysfs: the return". The qdev tree is completely undocumented, so once again the code dictates the protocol. Any bug in the qdev hierarchy will be hardcoded forever and ever, or we have to add a new layer of indirection to have a separate internal qdev and an external qdev-for-qmp tree(s).

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#37 lore raw

[Qemu-devel] Re: [PATCH 3/8] Add QBuffer

In reply to Avi Kivity
Quoted 16 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.

Quoted 7 lines
Quoted 2 lines

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

What is the receiver to do with it?

Quoted 8 lines

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

My worry is that __class__ will make the protocol more ad-hoc.

Quoted 2 lines

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.

Whether a protocol is self-describing is orthogonal to whether it's well defined (ala a schema). A self-describing protocol is very convenient for dynamic languages like Python. We should also provide a formal schema though for languages that require IDL to generate bindings (like C).

Regards,

Anthony Liguori

#38 lore raw

Re: [Qemu-devel] Re: [PATCH 3/8] Add QBuffer

Quoted 33 lines · Anthony Liguori
Quoted 24 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 7 lines

The alternative is to have a schema. Sun RPC/XDR doesn't carry any type
information (you can't even distinguish between a number and text) yet C
clients have to problem extracting typed information from it.

Quoted 7 lines
Quoted 2 lines

Having __class__ everywhere means we're carrying the schema in every message instead of just once.

The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.

What is the receiver to do with it?

If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.

My worry is that __class__ will make the protocol more ad-hoc.

Quoted 7 lines
Quoted 2 lines

I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.

The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.

Quoted 5 lines

Whether a protocol is self-describing is orthogonal to whether it's well defined (ala a schema). A self-describing protocol is very convenient for dynamic languages like Python. We should also provide a formal schema though for languages that require IDL to generate bindings (like C).

And that schema should be available over QMP.

author
#39 lore raw

PATCH 4/8 monitor: Add basic device state visualization

In reply to Jan Kiszka

This introduces device_show, a monitor command that saves the vmstate of a qdev device and visualizes it. QMP is also supported. Buffers are cut after 16 byte by default, but the full content can be requested via '-f'. To pretty-print sub-arrays, vmstate is extended to store the start index name.

---
 hw/hw.h         |    2 +
 hw/qdev.c       |  287 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 hw/qdev.h       |    2 +
 qemu-monitor.hx |   16 +++
 4 files changed, 307 insertions(+), 0 deletions(-)
hw/hw.h +2 -0
index 328b704..1ff8e40 100644
@@ -299,6 +299,7 @@ enum VMStateFlags {
 
 typedef struct {
     const char *name;
+    const char *start_index;
     size_t offset;
     size_t size;
     size_t start;
@@ -414,6 +415,7 @@ extern const VMStateInfo vmstate_info_unused_buffer;
     .size       = sizeof(_type),                                     \
     .flags      = VMS_ARRAY,                                         \
     .offset     = vmstate_offset_sub_array(_state, _field, _type, _start), \
+    .start_index = (stringify(_start)),                              \
 }
 
 #define VMSTATE_VARRAY_INT32(_field, _state, _field_num, _version, _info, _type) {\
hw/qdev.c +287 -0

296 lines not on this page · raw message

hw/qdev.h +2 -0
index d8fbc73..f8436ec 100644
@@ -177,6 +177,8 @@ void do_info_qtree(Monitor *mon);
 void do_info_qdm(Monitor *mon);
 int do_device_add(Monitor *mon, const QDict *qdict, QObject **ret_data);
 int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data);
+void device_user_print(Monitor *mon, const QObject *data);
+int do_device_show(Monitor *mon, const QDict *qdict, QObject **ret_data);
 
 /*** qdev-properties.c ***/
 
qemu-monitor.hx +16 -0
index a8f194c..449f012 100644
@@ -599,6 +599,22 @@ Remove device @var{id}.
 ETEXI
 
     {
+        .name       = "device_show",
+        .args_type  = "path:s,full:-f",
+        .params     = "device [-f]",
+        .help       = "show device state (specify -f for full buffer dumping)",
+        .user_print = device_user_print,
+        .mhandler.cmd_new = do_device_show,
+    },
+
+STEXI
+@item device_show @var{id} [@code{-f}]
+
+Show state of device @var{id} in a human-readable form. Buffers are cut after
+16 bytes unless a full dump is requested via @code{-f} 
+ETEXI
+
+    {
         .name       = "cpu",
         .args_type  = "index:i",
         .params     = "index",
Signature · 1 line
1.6.0.2
#40 lore raw

Re: [Qemu-devel] [PATCH 4/8] monitor: Add basic device state visualization

Quoted 7 lines · Jan Kiszka

This introduces device_show, a monitor command that saves the vmstate of a qdev device and visualizes it. QMP is also supported. Buffers are cut after 16 byte by default, but the full content can be requested via '-f'. To pretty-print sub-arrays, vmstate is extended to store the start index name.

Neato! Comments inline.

Quoted 12 lines
---
 hw/hw.h         |    2 +
 hw/qdev.c       |  287 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 hw/qdev.h       |    2 +
 qemu-monitor.hx |   16 +++
 4 files changed, 307 insertions(+), 0 deletions(-)
hw/hw.h +0 -0
index 328b704..1ff8e40 100644

hw/hw.h enum VMStateFlags { · line 305

Quoted 7 lines
hw/hw.h +1 -0
@@ -299,6 +299,7 @@ enum VMStateFlags {
 
 typedef struct {
     const char *name;
+    const char *start_index;
     size_t offset;
     size_t size;
     size_t start;

Why is start_index a string?

Quoted 30 lines
hw/hw.h +1 -0
@@ -414,6 +415,7 @@ extern const VMStateInfo vmstate_info_unused_buffer;
     .size       = sizeof(_type),                                     \
     .flags      = VMS_ARRAY,                                         \
     .offset     = vmstate_offset_sub_array(_state, _field, _type, _start), \
+    .start_index = (stringify(_start)),                              \
 }
 
 #define VMSTATE_VARRAY_INT32(_field, _state, _field_num, _version, _info, _type) {\
hw/qdev.c +7 -0
index fe49e71..c989010 100644
@@ -29,6 +29,9 @@
 #include "qdev.h"
 #include "sysemu.h"
 #include "monitor.h"
+#include "qjson.h"
+#include "qint.h"
+#include "qbuffer.h"
 
 static int qdev_hotplug = 0;
 
@@ -824,3 +827,287 @@ int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data)
     }
     return qdev_unplug(dev);
 }
+
+static int print_array_index(Monitor *mon, const char *start, int index)
+{
+    char index_str[32];

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 843

Quoted 10 lines
hw/qdev.c +10 -0
+    int len;
+
+    if (start) {
+        len = snprintf(index_str, sizeof(index_str), "[%s+%02x]", start,
+                       index);
+    } else {
+        len = snprintf(index_str, sizeof(index_str), "[%02x]", index);
+    }
+    monitor_printf(mon, index_str);
+    return len;

You go via the snprintf() detour just because monitor_printf() doesn't return the number of characters printed. Wouldn't it be better to fix that?

Quoted 40 lines
hw/qdev.c +40 -0
+}
+
+#define NAME_COLUMN_WIDTH 23
+
+static void print_field(Monitor *mon, const QDict *qfield, int indent);
+
+static void print_elem(Monitor *mon, const QObject *qelem, size_t size,
+                       int column_pos, int indent)
+{
+    int64_t data_size;
+    const void *data;
+    int n;
+
+    if (qobject_type(qelem) == QTYPE_QDICT) {
+        if (column_pos >= 0) {
+            monitor_printf(mon, ".\n");
+        }
+    } else {
+        monitor_printf(mon, ":");
+        column_pos++;
+        if (column_pos < NAME_COLUMN_WIDTH) {
+            monitor_printf(mon, "%*c", NAME_COLUMN_WIDTH - column_pos, ' ');
+        }
+    }
+
+    switch (qobject_type(qelem)) {
+    case QTYPE_QDICT:
+        print_field(mon, qobject_to_qdict(qelem), indent + 2);
+        break;
+    case QTYPE_QBUFFER:
+        data = qbuffer_get_data(qobject_to_qbuffer(qelem));
+        data_size = qbuffer_get_size(qobject_to_qbuffer(qelem));
+        for (n = 0; n < data_size; ) {
+            monitor_printf(mon, " %02x", *((uint8_t *)data+n));
+            if (++n < size) {
+                if (n % 16 == 0) {
+                    monitor_printf(mon, "\n%*c", NAME_COLUMN_WIDTH, ' ');
+                } else if (n % 8 == 0) {
+                    monitor_printf(mon, " -");
+                }

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 893

Quoted 10 lines
hw/qdev.c +10 -0
+            }
+        }
+        if (data_size < size) {
+            monitor_printf(mon, " ...");
+        }
+        monitor_printf(mon, "\n");
+        break;
+    case QTYPE_QINT:
+        monitor_printf(mon, " %0*x\n", (int)size * 2,
+                       (int)qint_get_int(qobject_to_qint(qelem)));

I'm confused. Doesn't casting qint_get_int() to int lose bits?

Quoted 178 lines
hw/qdev.c +178 -0
+        break;
+    default:
+        assert(0);
+    }
+}
+
+static void print_field(Monitor *mon, const QDict *qfield, int indent)
+{
+    const char *name = qdict_get_str(qfield, "name");
+    const char *start = qdict_get_try_str(qfield, "start");
+    int64_t size = qdict_get_int(qfield, "size");
+    QList *qlist = qdict_get_qlist(qfield, "elems");
+    QListEntry *entry, *sub_entry;
+    QList *sub_list;
+    int elem_no = 0;
+
+    QLIST_FOREACH_ENTRY(qlist, entry) {
+        QObject *qelem = qlist_entry_obj(entry);
+        int pos = indent + strlen(name);
+
+        if (qobject_type(qelem) == QTYPE_QLIST) {
+            monitor_printf(mon, "%*c%s", indent, ' ', name);
+            pos += print_array_index(mon, start, elem_no);
+            sub_list = qobject_to_qlist(qelem);
+            QLIST_FOREACH_ENTRY(sub_list, sub_entry) {
+                print_elem(mon, qlist_entry_obj(sub_entry), size, pos,
+                           indent + 2);
+                pos = -1;
+            }
+        } else {
+            if (elem_no == 0) {
+                monitor_printf(mon, "%*c%s", indent, ' ', name);
+            } else {
+                pos = -1;
+            }
+            print_elem(mon, qelem, size, pos, indent);
+        }
+        elem_no++;
+    }
+}
+
+void device_user_print(Monitor *mon, const QObject *data)
+{
+    QDict *qdict = qobject_to_qdict(data);
+    QList *qlist = qdict_get_qlist(qdict, "fields");
+    QListEntry *entry;
+
+    monitor_printf(mon, "dev: %s, id \"%s\"\n",
+                   qdict_get_str(qdict, "device"),
+                   qdict_get_str(qdict, "id"));
+
+    QLIST_FOREACH_ENTRY(qlist, entry) {
+        print_field(mon, qobject_to_qdict(qlist_entry_obj(entry)), 2);
+    }
+}
+
+static void parse_vmstate(const VMStateDescription *vmsd, void *opaque,
+                          QList *qlist, int full_buffers)
+{
+    VMStateField *field = vmsd->fields;
+
+    if (vmsd->pre_save) {
+        vmsd->pre_save(opaque);
+    }
+    while(field->name) {
+        if (!field->field_exists ||
+            field->field_exists(opaque, vmsd->version_id)) {
+            void *base_addr = opaque + field->offset;
+            int i, n_elems = 1;
+            int is_array = 1;
+            size_t size = field->size;
+            QDict *qfield = qdict_new();
+            QList *qelems = qlist_new();
+
+            qlist_append_obj(qlist, QOBJECT(qfield));
+
+            qdict_put_obj(qfield, "name",
+                          QOBJECT(qstring_from_str(field->name)));
+            qdict_put_obj(qfield, "elems", QOBJECT(qelems));
+
+            if (field->flags & VMS_VBUFFER) {
+                size = *(int32_t *)(opaque + field->size_offset);
+                if (field->flags & VMS_MULTIPLY) {
+                    size *= field->size;
+                }
+            }
+            qdict_put_obj(qfield, "size", QOBJECT(qint_from_int(size)));
+            if (field->start_index) {
+                qdict_put_obj(qfield, "start",
+                              QOBJECT(qstring_from_str(field->start_index)));
+            }
+
+            if (field->flags & VMS_ARRAY) {
+                n_elems = field->num;
+            } else if (field->flags & VMS_VARRAY_INT32) {
+                n_elems = *(int32_t *)(opaque + field->num_offset);
+            } else if (field->flags & VMS_VARRAY_UINT16) {
+                n_elems = *(uint16_t *)(opaque + field->num_offset);
+            } else {
+                is_array = 0;
+            }
+            if (field->flags & VMS_POINTER) {
+                base_addr = *(void **)base_addr + field->start;
+            }
+            for (i = 0; i < n_elems; i++) {
+                void *addr = base_addr + size * i;
+                QList *sub_elems = qelems;
+                int val;
+
+                if (is_array) {
+                    sub_elems = qlist_new();
+                    qlist_append_obj(qelems, QOBJECT(sub_elems));
+                }
+                if (field->flags & VMS_ARRAY_OF_POINTER) {
+                    addr = *(void **)addr;
+                }
+                if (field->flags & VMS_STRUCT) {
+                    parse_vmstate(field->vmsd, addr, sub_elems, full_buffers);
+                } else {
+                    if (field->flags & (VMS_BUFFER | VMS_VBUFFER)) {
+                        if (!full_buffers && size > 16) {
+                            size = 16;
+                        }
+                        qlist_append_obj(sub_elems,
+                                         QOBJECT(qbuffer_from_data(addr,
+                                                                   size)));
+                    } else {
+                        switch (size) {
+                        case 1:
+                            val = *(uint8_t *)addr;
+                            break;
+                        case 2:
+                            val = *(uint16_t *)addr;
+                            break;
+                        case 4:
+                            val = *(uint32_t *)addr;
+                            break;
+                        case 8:
+                            val = *(uint64_t *)addr;
+                            break;
+                        default:
+                            assert(0);
+                        }
+                        qlist_append_obj(sub_elems,
+                                         QOBJECT(qint_from_int(val)));
+                    }
+                }
+            }
+        }
+        field++;
+    }
+}
+
+static DeviceState *qdev_find(const char *path)
+{
+    const char *dev_name;
+    DeviceState *dev;
+    char *bus_path;
+    BusState *bus;
+
+    dev_name = strrchr(path, '/');
+    if (!dev_name) {
+        bus = main_system_bus;
+        dev_name = path;
+    } else {
+        dev_name++;
+        bus_path = qemu_strdup(path);
+        bus_path[dev_name - path] = 0;
+
+        bus = qbus_find(bus_path);
+        qemu_free(bus_path);
+
+        if (!bus) {
+            /* qbus_find already reported the error */
+            return NULL;
+        }
+    }
+    dev = qbus_find_dev(bus, dev_name);

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1081

Quoted 10 lines
hw/qdev.c +10 -0
+    if (!dev) {
+        qerror_report(QERR_DEVICE_NOT_FOUND, dev_name);
+        if (!monitor_cur_is_qmp()) {
+            qbus_list_dev(bus);
+        }
+    }
+    return dev;
+}
+
+int do_device_show(Monitor *mon, const QDict *qdict, QObject **ret_data)

Command documentation missing.

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1094

Quoted 13 lines
hw/qdev.c +13 -0
+{
+    const char *path = qdict_get_str(qdict, "path");
+    const VMStateDescription *vmsd;
+    DeviceState *dev;
+    QList *qlist;
+
+    dev = qdev_find_recursive(main_system_bus, path);
+    if (!dev) {
+        dev = qdev_find(path);
+        if (!dev) {
+            return -1;
+        }
+    }

This is inconsistent with how do_device_del() looks up devices by ID. Let's agree on one lookup, put it into a function, and use it everywhere.

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1102

Quoted 8 lines
hw/qdev.c +8 -0
+
+    vmsd = dev->info->vmsd;
+    if (!vmsd) {
+        qerror_report(QERR_UNDEFINED_ERROR);
+        if (!monitor_cur_is_qmp()) {
+            error_printf_unless_qmp("Device '%s' does not support state "
+                                    "dumping\n", path);
+        }

Please use a suitable QERR_. Define one if necessary.

Quoted 51 lines
hw/qdev.c +11 -0
+        return -1;
+    }
+
+    *ret_data = qobject_from_jsonf("{ 'device': %s, 'id': %s }",
+                                   dev->info->name, dev->id ? : "");
+    qlist = qlist_new();
+    parse_vmstate(vmsd, dev, qlist, qdict_get_int(qdict, "full"));
+    qdict_put_obj(qobject_to_qdict(*ret_data), "fields", QOBJECT(qlist));
+
+    return 0;
+}
hw/qdev.h +2 -0
index d8fbc73..f8436ec 100644
@@ -177,6 +177,8 @@ void do_info_qtree(Monitor *mon);
 void do_info_qdm(Monitor *mon);
 int do_device_add(Monitor *mon, const QDict *qdict, QObject **ret_data);
 int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data);
+void device_user_print(Monitor *mon, const QObject *data);
+int do_device_show(Monitor *mon, const QDict *qdict, QObject **ret_data);
 
 /*** qdev-properties.c ***/
 
qemu-monitor.hx +16 -0
index a8f194c..449f012 100644
@@ -599,6 +599,22 @@ Remove device @var{id}.
 ETEXI
 
     {
+        .name       = "device_show",
+        .args_type  = "path:s,full:-f",
+        .params     = "device [-f]",
+        .help       = "show device state (specify -f for full buffer dumping)",
+        .user_print = device_user_print,
+        .mhandler.cmd_new = do_device_show,
+    },
+
+STEXI
+@item device_show @var{id} [@code{-f}]
+
+Show state of device @var{id} in a human-readable form. Buffers are cut after
+16 bytes unless a full dump is requested via @code{-f} 
+ETEXI
+
+    {
         .name       = "cpu",
         .args_type  = "index:i",
         .params     = "index",
#41 lore raw

[Qemu-devel] Re: [PATCH 4/8] monitor: Add basic device state visualization

Quoted 25 lines · Markus Armbruster
Quoted 7 lines · Jan Kiszka

This introduces device_show, a monitor command that saves the vmstate of a qdev device and visualizes it. QMP is also supported. Buffers are cut after 16 byte by default, but the full content can be requested via '-f'. To pretty-print sub-arrays, vmstate is extended to store the start index name.

Neato! Comments inline.

Quoted 13 lines
---
 hw/hw.h         |    2 +
 hw/qdev.c       |  287 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 hw/qdev.h       |    2 +
 qemu-monitor.hx |   16 +++
 4 files changed, 307 insertions(+), 0 deletions(-)
hw/hw.h +0 -0
index 328b704..1ff8e40 100644
@@ -299,6 +299,7 @@ enum VMStateFlags {
 
Quoted 8 lines

hw/hw.h enum VMStateFlags { · line 305

Quoted 6 lines
hw/hw.h +1 -0
 typedef struct {
     const char *name;
+    const char *start_index;
     size_t offset;
     size_t size;
     size_t start;

Why is start_index a string?

It can, actually should be a symbolic constant (from e1000:
"mac_reg[RA+03]: ...", ie. "RA" is the start index here).

Quoted 37 lines
Quoted 37 lines
hw/hw.h +1 -0
@@ -414,6 +415,7 @@ extern const VMStateInfo vmstate_info_unused_buffer;
     .size       = sizeof(_type),                                     \
     .flags      = VMS_ARRAY,                                         \
     .offset     = vmstate_offset_sub_array(_state, _field, _type, _start), \
+    .start_index = (stringify(_start)),                              \
 }
 
 #define VMSTATE_VARRAY_INT32(_field, _state, _field_num, _version, _info, _type) {\
hw/qdev.c +13 -0
index fe49e71..c989010 100644
@@ -29,6 +29,9 @@
 #include "qdev.h"
 #include "sysemu.h"
 #include "monitor.h"
+#include "qjson.h"
+#include "qint.h"
+#include "qbuffer.h"
 
 static int qdev_hotplug = 0;
 
@@ -824,3 +827,287 @@ int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data)
     }
     return qdev_unplug(dev);
 }
+
+static int print_array_index(Monitor *mon, const char *start, int index)
+{
+    char index_str[32];
+    int len;
+
+    if (start) {
+        len = snprintf(index_str, sizeof(index_str), "[%s+%02x]", start,
+                       index);
+    } else {
Quoted 8 lines

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 843

Quoted 4 lines
hw/qdev.c +4 -0
+        len = snprintf(index_str, sizeof(index_str), "[%02x]", index);
+    }
+    monitor_printf(mon, index_str);
+    return len;

You go via the snprintf() detour just because monitor_printf() doesn't return the number of characters printed. Wouldn't it be better to fix that?

Yeah, good point. Will address this.

Quoted 45 lines
Quoted 45 lines
hw/qdev.c +44 -0
+}
+
+#define NAME_COLUMN_WIDTH 23
+
+static void print_field(Monitor *mon, const QDict *qfield, int indent);
+
+static void print_elem(Monitor *mon, const QObject *qelem, size_t size,
+                       int column_pos, int indent)
+{
+    int64_t data_size;
+    const void *data;
+    int n;
+
+    if (qobject_type(qelem) == QTYPE_QDICT) {
+        if (column_pos >= 0) {
+            monitor_printf(mon, ".\n");
+        }
+    } else {
+        monitor_printf(mon, ":");
+        column_pos++;
+        if (column_pos < NAME_COLUMN_WIDTH) {
+            monitor_printf(mon, "%*c", NAME_COLUMN_WIDTH - column_pos, ' ');
+        }
+    }
+
+    switch (qobject_type(qelem)) {
+    case QTYPE_QDICT:
+        print_field(mon, qobject_to_qdict(qelem), indent + 2);
+        break;
+    case QTYPE_QBUFFER:
+        data = qbuffer_get_data(qobject_to_qbuffer(qelem));
+        data_size = qbuffer_get_size(qobject_to_qbuffer(qelem));
+        for (n = 0; n < data_size; ) {
+            monitor_printf(mon, " %02x", *((uint8_t *)data+n));
+            if (++n < size) {
+                if (n % 16 == 0) {
+                    monitor_printf(mon, "\n%*c", NAME_COLUMN_WIDTH, ' ');
+                } else if (n % 8 == 0) {
+                    monitor_printf(mon, " -");
+                }
+            }
+        }
+        if (data_size < size) {
+            monitor_printf(mon, " ...");
Quoted 8 lines

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 893

Quoted 6 lines
hw/qdev.c +6 -0
+        }
+        monitor_printf(mon, "\n");
+        break;
+    case QTYPE_QINT:
+        monitor_printf(mon, " %0*x\n", (int)size * 2,
+                       (int)qint_get_int(qobject_to_qint(qelem)));

I'm confused. Doesn't casting qint_get_int() to int lose bits?

You are right, will fix!

Quoted 183 lines
Quoted 183 lines
hw/qdev.c +182 -0
+        break;
+    default:
+        assert(0);
+    }
+}
+
+static void print_field(Monitor *mon, const QDict *qfield, int indent)
+{
+    const char *name = qdict_get_str(qfield, "name");
+    const char *start = qdict_get_try_str(qfield, "start");
+    int64_t size = qdict_get_int(qfield, "size");
+    QList *qlist = qdict_get_qlist(qfield, "elems");
+    QListEntry *entry, *sub_entry;
+    QList *sub_list;
+    int elem_no = 0;
+
+    QLIST_FOREACH_ENTRY(qlist, entry) {
+        QObject *qelem = qlist_entry_obj(entry);
+        int pos = indent + strlen(name);
+
+        if (qobject_type(qelem) == QTYPE_QLIST) {
+            monitor_printf(mon, "%*c%s", indent, ' ', name);
+            pos += print_array_index(mon, start, elem_no);
+            sub_list = qobject_to_qlist(qelem);
+            QLIST_FOREACH_ENTRY(sub_list, sub_entry) {
+                print_elem(mon, qlist_entry_obj(sub_entry), size, pos,
+                           indent + 2);
+                pos = -1;
+            }
+        } else {
+            if (elem_no == 0) {
+                monitor_printf(mon, "%*c%s", indent, ' ', name);
+            } else {
+                pos = -1;
+            }
+            print_elem(mon, qelem, size, pos, indent);
+        }
+        elem_no++;
+    }
+}
+
+void device_user_print(Monitor *mon, const QObject *data)
+{
+    QDict *qdict = qobject_to_qdict(data);
+    QList *qlist = qdict_get_qlist(qdict, "fields");
+    QListEntry *entry;
+
+    monitor_printf(mon, "dev: %s, id \"%s\"\n",
+                   qdict_get_str(qdict, "device"),
+                   qdict_get_str(qdict, "id"));
+
+    QLIST_FOREACH_ENTRY(qlist, entry) {
+        print_field(mon, qobject_to_qdict(qlist_entry_obj(entry)), 2);
+    }
+}
+
+static void parse_vmstate(const VMStateDescription *vmsd, void *opaque,
+                          QList *qlist, int full_buffers)
+{
+    VMStateField *field = vmsd->fields;
+
+    if (vmsd->pre_save) {
+        vmsd->pre_save(opaque);
+    }
+    while(field->name) {
+        if (!field->field_exists ||
+            field->field_exists(opaque, vmsd->version_id)) {
+            void *base_addr = opaque + field->offset;
+            int i, n_elems = 1;
+            int is_array = 1;
+            size_t size = field->size;
+            QDict *qfield = qdict_new();
+            QList *qelems = qlist_new();
+
+            qlist_append_obj(qlist, QOBJECT(qfield));
+
+            qdict_put_obj(qfield, "name",
+                          QOBJECT(qstring_from_str(field->name)));
+            qdict_put_obj(qfield, "elems", QOBJECT(qelems));
+
+            if (field->flags & VMS_VBUFFER) {
+                size = *(int32_t *)(opaque + field->size_offset);
+                if (field->flags & VMS_MULTIPLY) {
+                    size *= field->size;
+                }
+            }
+            qdict_put_obj(qfield, "size", QOBJECT(qint_from_int(size)));
+            if (field->start_index) {
+                qdict_put_obj(qfield, "start",
+                              QOBJECT(qstring_from_str(field->start_index)));
+            }
+
+            if (field->flags & VMS_ARRAY) {
+                n_elems = field->num;
+            } else if (field->flags & VMS_VARRAY_INT32) {
+                n_elems = *(int32_t *)(opaque + field->num_offset);
+            } else if (field->flags & VMS_VARRAY_UINT16) {
+                n_elems = *(uint16_t *)(opaque + field->num_offset);
+            } else {
+                is_array = 0;
+            }
+            if (field->flags & VMS_POINTER) {
+                base_addr = *(void **)base_addr + field->start;
+            }
+            for (i = 0; i < n_elems; i++) {
+                void *addr = base_addr + size * i;
+                QList *sub_elems = qelems;
+                int val;
+
+                if (is_array) {
+                    sub_elems = qlist_new();
+                    qlist_append_obj(qelems, QOBJECT(sub_elems));
+                }
+                if (field->flags & VMS_ARRAY_OF_POINTER) {
+                    addr = *(void **)addr;
+                }
+                if (field->flags & VMS_STRUCT) {
+                    parse_vmstate(field->vmsd, addr, sub_elems, full_buffers);
+                } else {
+                    if (field->flags & (VMS_BUFFER | VMS_VBUFFER)) {
+                        if (!full_buffers && size > 16) {
+                            size = 16;
+                        }
+                        qlist_append_obj(sub_elems,
+                                         QOBJECT(qbuffer_from_data(addr,
+                                                                   size)));
+                    } else {
+                        switch (size) {
+                        case 1:
+                            val = *(uint8_t *)addr;
+                            break;
+                        case 2:
+                            val = *(uint16_t *)addr;
+                            break;
+                        case 4:
+                            val = *(uint32_t *)addr;
+                            break;
+                        case 8:
+                            val = *(uint64_t *)addr;
+                            break;
+                        default:
+                            assert(0);
+                        }
+                        qlist_append_obj(sub_elems,
+                                         QOBJECT(qint_from_int(val)));
+                    }
+                }
+            }
+        }
+        field++;
+    }
+}
+
+static DeviceState *qdev_find(const char *path)
+{
+    const char *dev_name;
+    DeviceState *dev;
+    char *bus_path;
+    BusState *bus;
+
+    dev_name = strrchr(path, '/');
+    if (!dev_name) {
+        bus = main_system_bus;
+        dev_name = path;
+    } else {
+        dev_name++;
+        bus_path = qemu_strdup(path);
+        bus_path[dev_name - path] = 0;
+
+        bus = qbus_find(bus_path);
+        qemu_free(bus_path);
+
+        if (!bus) {
+            /* qbus_find already reported the error */
+            return NULL;
+        }
+    }
+    dev = qbus_find_dev(bus, dev_name);
+    if (!dev) {
+        qerror_report(QERR_DEVICE_NOT_FOUND, dev_name);
+        if (!monitor_cur_is_qmp()) {
+            qbus_list_dev(bus);
Quoted 8 lines

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1081

Quoted 6 lines
hw/qdev.c +6 -0
+        }
+    }
+    return dev;
+}
+
+int do_device_show(Monitor *mon, const QDict *qdict, QObject **ret_data)

Command documentation missing.

In the make (depends on QMP doc infrastructure).

Quoted 10 lines
Quoted 10 lines
hw/qdev.c +9 -0
+{
+    const char *path = qdict_get_str(qdict, "path");
+    const VMStateDescription *vmsd;
+    DeviceState *dev;
+    QList *qlist;
+
+    dev = qdev_find_recursive(main_system_bus, path);
+    if (!dev) {
+        dev = qdev_find(path);
Quoted 8 lines

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1094

Quoted 4 lines
hw/qdev.c +4 -0
+        if (!dev) {
+            return -1;
+        }
+    }

This is inconsistent with how do_device_del() looks up devices by ID. Let's agree on one lookup, put it into a function, and use it everywhere.

Will adjust device_del to the same scheme.

BTW, qdev path completion will also be available for both commands in my next series.

Quoted 11 lines

hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1102

Quoted 9 lines
hw/qdev.c +8 -0
+
+    vmsd = dev->info->vmsd;
+    if (!vmsd) {
+        qerror_report(QERR_UNDEFINED_ERROR);
+        if (!monitor_cur_is_qmp()) {
+            error_printf_unless_qmp("Device '%s' does not support state "
+                                    "dumping\n", path);
+        }

Please use a suitable QERR_. Define one if necessary.

Already fixed in my tree (forgot this bit for v1). Just this redundant qmp check was still present.

Quoted 44 lines
Quoted 44 lines
hw/qdev.c +11 -0
+        return -1;
+    }
+
+    *ret_data = qobject_from_jsonf("{ 'device': %s, 'id': %s }",
+                                   dev->info->name, dev->id ? : "");
+    qlist = qlist_new();
+    parse_vmstate(vmsd, dev, qlist, qdict_get_int(qdict, "full"));
+    qdict_put_obj(qobject_to_qdict(*ret_data), "fields", QOBJECT(qlist));
+
+    return 0;
+}
hw/qdev.h +2 -0
index d8fbc73..f8436ec 100644
@@ -177,6 +177,8 @@ void do_info_qtree(Monitor *mon);
 void do_info_qdm(Monitor *mon);
 int do_device_add(Monitor *mon, const QDict *qdict, QObject **ret_data);
 int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data);
+void device_user_print(Monitor *mon, const QObject *data);
+int do_device_show(Monitor *mon, const QDict *qdict, QObject **ret_data);
 
 /*** qdev-properties.c ***/
 
qemu-monitor.hx +11 -0
index a8f194c..449f012 100644
@@ -599,6 +599,22 @@ Remove device @var{id}.
 ETEXI
 
     {
+        .name       = "device_show",
+        .args_type  = "path:s,full:-f",
+        .params     = "device [-f]",
+        .help       = "show device state (specify -f for full buffer dumping)",
+        .user_print = device_user_print,
+        .mhandler.cmd_new = do_device_show,
+    },
+
+STEXI
+@item device_show @var{id} [@code{-f}]
+
Quoted 10 lines

qemu-monitor.hx Remove device @var{id}. · line 620

Quoted 8 lines
qemu-monitor.hx +5 -0
+Show state of device @var{id} in a human-readable form. Buffers are cut after
+16 bytes unless a full dump is requested via @code{-f} 
+ETEXI
+
+    {
         .name       = "cpu",
         .args_type  = "index:i",
         .params     = "index",

Thanks for the comments!

Jan

author
#42 lore raw

PATCH 5/8 qmp: Teach basic capability negotiation to python example

In reply to Jan Kiszka

As sending "qmp_capabilities" on session start became mandatory, both python examples were broken.

---
 QMP/qmp-shell |    1 +
 QMP/vm-info   |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)
QMP/qmp-shell +1 -0
index f89b9af..a5b72d1 100755
@@ -42,6 +42,7 @@ def main():
 
     qemu = qmp.QEMUMonitorProtocol(argv[1])
     qemu.connect()
+    qemu.send("qmp_capabilities")
 
     print 'Connected!'
 
QMP/vm-info +1 -0
index b150d82..d29e7f5 100755
@@ -24,6 +24,7 @@ def main():
 
     qemu = qmp.QEMUMonitorProtocol(argv[1])
     qemu.connect()
+    qemu.send("qmp_capabilities")
 
     for cmd in [ 'version', 'hpet', 'kvm', 'status', 'uuid', 'balloon' ]:
         print cmd + ': ' + str(qemu.send('query-' + cmd))
Signature · 1 line
1.6.0.2
author
#43 lore raw

PATCH 6/8 qmp: Fix python helper /wrt long return strings

In reply to Jan Kiszka

Remove the arbitrary limitation of 1024 characters per return string and read complete lines instead. Required for device_show.

---
 QMP/qmp.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
QMP/qmp.py +1 -1
index d9da603..b8f1741 100644
@@ -63,7 +63,7 @@ class QEMUMonitorProtocol:
 
     def __json_read(self):
         try:
-            return json.loads(self.sock.recv(1024))
+            return json.loads(self.sock.makefile().readline())
         except ValueError:
             return
 
Signature · 1 line
1.6.0.2
author
#44 lore raw

PATCH 7/8 Add QLIST_INSERT_TAIL

In reply to Jan Kiszka

As the QLIST has not tail pointer, this requires list walking. Still useful when lists are short or insertion time doesn't matter.

---
 qemu-queue.h |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
qemu-queue.h +9 -0
index 1d07745..99ed1f6 100644
@@ -122,6 +122,15 @@ struct {                                                                \
         (elm)->field.le_prev = &(head)->lh_first;                       \
 } while (/*CONSTCOND*/0)
 
+#define QLIST_INSERT_TAIL(head, elm, field) do {                        \
+        typeof((head)->lh_first) *qlist_lastptr = &(head)->lh_first;    \
+        while (*qlist_lastptr)                                          \
+            qlist_lastptr = &(*qlist_lastptr)->field.le_next;           \
+        (elm)->field.le_next = NULL;                                    \
+        *qlist_lastptr = (elm);                                         \
+        (elm)->field.le_prev = qlist_lastptr;                           \
+} while (/*CONSTCOND*/0)
+
 #define QLIST_REMOVE(elm, field) do {                                   \
         if ((elm)->field.le_next != NULL)                               \
                 (elm)->field.le_next->field.le_prev =                   \
Signature · 1 line
1.6.0.2
#45 lore raw

[Qemu-devel] Re: [PATCH 7/8] Add QLIST_INSERT_TAIL

Quoted 2 lines · Jan Kiszka

As the QLIST has not tail pointer, this requires list walking. Still useful when lists are short or insertion time doesn't matter.

True, but why not switch to QTAIL instead?

Paolo

#46 lore raw

[Qemu-devel] Re: [PATCH 7/8] Add QLIST_INSERT_TAIL

Quoted 5 lines · Paolo Bonzini
Quoted 2 lines · Jan Kiszka

As the QLIST has not tail pointer, this requires list walking. Still useful when lists are short or insertion time doesn't matter.

True, but why not switch to QTAIL instead?

Would be more invasive, but I can do so if it's preferred.

Jan

author
#47 lore raw

PATCH 8/8 qdev: Add new devices/buses at the tail

In reply to Jan Kiszka

Cosmetic change to align the instance number assignment with bus ordering. The current ordering is a bit annoying when you dump the qtree or address devices via 'driver.instance'.

---
 hw/qdev.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
hw/qdev.c +2 -2
index c989010..3864478 100644
@@ -91,7 +91,7 @@ static DeviceState *qdev_create_from_info(BusState *bus, DeviceInfo *info)
     qdev_prop_set_defaults(dev, dev->info->props);
     qdev_prop_set_defaults(dev, dev->parent_bus->info->props);
     qdev_prop_set_globals(dev);
-    QLIST_INSERT_HEAD(&bus->children, dev, sibling);
+    QLIST_INSERT_TAIL(&bus->children, dev, sibling);
     if (qdev_hotplug) {
         assert(bus->allow_hotplug);
         dev->hotplugged = 1;
@@ -677,7 +677,7 @@ void qbus_create_inplace(BusState *bus, BusInfo *info,
 
     QLIST_INIT(&bus->children);
     if (parent) {
-        QLIST_INSERT_HEAD(&parent->child_bus, bus, sibling);
+        QLIST_INSERT_TAIL(&parent->child_bus, bus, sibling);
         parent->num_child_bus++;
     }
 
Signature · 1 line
1.6.0.2
#48 lore raw
In reply to Jan Kiszka
Quoted 6 lines · Jan Kiszka

While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.

Wonderful! may even motivate some more qmp conversions.

Quoted 26 lines

This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:

(qemu) device_show /i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.

     version_id:         00000002
     config:             a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00
                         ...
     irq_state:          00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00
   num_ports_vmstate:    02
   ports[00].
       ctrl:             0083
   ports[01].
       ctrl:             0080
   cmd:                  00c1
   status:               0000
   intr:                 0000
   frnum:                0077
   fl_base_addr:         0fffc000
   sof_timing:           40
   status2:              00
   frame_timer:          0000000000cb2bd0
Quoted 7 lines

Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...

Will be interesting to pass these annotations via qmp as well.

Quoted 3 lines

Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!

I'd like to see qmp command documentation for this.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
author
#49 lore raw
Quoted 9 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka

While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.

Wonderful! may even motivate some more qmp conversions.

Yeah, a few low handing fruits are already gone (the vmstate series I sent out recently). I've some stuff for hpet here, but I did not tough [IO]APIC yet as that should be based on top of Glauber's work for in-kernel irqchip support.

Quoted 35 lines
Quoted 27 lines

This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:

(qemu) device_show /i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.

     version_id:         00000002
     config:             a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00
                         ...
     irq_state:          00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00
   num_ports_vmstate:    02
   ports[00].
       ctrl:             0083
   ports[01].
       ctrl:             0080
   cmd:                  00c1
   status:               0000
   intr:                 0000
   frnum:                0077
   fl_base_addr:         0fffc000
   sof_timing:           40
   status2:              00
   frame_timer:          0000000000cb2bd0
Quoted 7 lines

Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...

Quoted 1 line

Will be interesting to pass these annotations via qmp as well.

For sure. We could e.g. convert QBuffers to QInts or QInts to QBools once we know their names and positions. This is where the harder work starts (that's why I proposed it for GSOC :) ).

Quoted 7 lines
Quoted 4 lines

Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!

I'd like to see qmp command documentation for this.

Will add it once the doc baseline is merged.

Jan

Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux
#50 lore raw
Quoted 7 lines · Jan Kiszka
Quoted 5 lines

I'd like to see qmp command documentation for this.

Will add it once the doc baseline is merged.

How about posting it as a patch on top of the documentation patch? That allows us to review it (which IMO should be a condition for accepting any patch that adds or changes a monitor command) while not requiring merging to be delayed until after the doc patches makes it in.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#51 lore raw
In reply to Jan Kiszka
Quoted 32 lines · Jan Kiszka

While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.

This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:

(qemu) device_show /i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.

     version_id:         00000002
     config:             a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00
                         ...
     irq_state:          00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00
   num_ports_vmstate:    02
   ports[00].
       ctrl:             0083
   ports[01].
       ctrl:             0080
   cmd:                  00c1
   status:               0000
   intr:                 0000
   frnum:                0077
   fl_base_addr:         0fffc000
   sof_timing:           40
   status2:              00
   frame_timer:          0000000000cb2bd0
Quoted 9 lines

Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...

Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!

Very neat!

Regards,

Anthony Liguori

Quoted 37 lines

Jan Kiszka (8):
qdev: Allow device addressing via 'driver.instance' Add base64 encoder/decoder
Add QBuffer
monitor: Add basic device state visualization
qmp: Teach basic capability negotiation to python example
qmp: Fix python helper /wrt long return strings Add QLIST_INSERT_TAIL
qdev: Add new devices/buses at the tail

  Makefile        |    3 +-
  Makefile.objs   |    4 +-
  QMP/qmp-shell   |    1 +
  QMP/qmp.py      |    2 +-
  QMP/vm-info     |    1 +
  base64.c        |  202 +++++++++++++++++++++++++++++++++++
  base64.h        |   18 +++
  check-qbuffer.c |  172 ++++++++++++++++++++++++++++++
  configure       |    2 +-
  hw/hw.h         |    2 +
  hw/qdev.c       |  314 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
  hw/qdev.h       |    2 +
  qbuffer.c       |  116 ++++++++++++++++++++
  qbuffer.h       |   33 ++++++
  qemu-monitor.hx |   16 +++
  qemu-queue.h    |    9 ++
  qjson.c         |   16 +++
  qobject.h       |    1 +
  18 files changed, 902 insertions(+), 12 deletions(-)

create mode 100644 base64.c
create mode 100644 base64.h
create mode 100644 check-qbuffer.c
create mode 100644 qbuffer.c
create mode 100644 qbuffer.h

#52 lore raw
In reply to Jan Kiszka
Quoted 32 lines · Jan Kiszka

While recently fixing the SCSI reset issues, I once again had the need
for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.

This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show <qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:

(qemu) device_show /i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.

    version_id:         00000002
    config:             a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00
                        ...
    irq_state:          00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00
  num_ports_vmstate:    02
  ports[00].
      ctrl:             0083
  ports[01].
      ctrl:             0080
  cmd:                  00c1
  status:               0000
  intr:                 0000
  frnum:                0077
  fl_base_addr:         0fffc000
  sof_timing:           40
  status2:              00
  frame_timer:          0000000000cb2bd0
Quoted 8 lines

Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...

Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!

Nice! I think this confirms that there is no future for the qdev side of my yesterday's patch, this is much better.