Basic device state visualization
cover Basic device state visualization
While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.
This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show <qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:
(qemu) device_show /i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.
version_id: 00000002
config: a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00
...
irq_state: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00
num_ports_vmstate: 02
ports[00].
ctrl: 0083
ports[01].
ctrl: 0080
cmd: 00c1
status: 0000
intr: 0000
frnum: 0077
fl_base_addr: 0fffc000
sof_timing: 40
status2: 00
frame_timer: 0000000000cb2bd0Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...
Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!
Jan Kiszka (8):
qdev: Allow device addressing via 'driver.instance' Add base64 encoder/decoder
Add QBuffer
monitor: Add basic device state visualization
qmp: Teach basic capability negotiation to python example
qmp: Fix python helper /wrt long return strings Add QLIST_INSERT_TAIL
qdev: Add new devices/buses at the tail
Makefile | 3 +- Makefile.objs | 4 +- QMP/qmp-shell | 1 + QMP/qmp.py | 2 +- QMP/vm-info | 1 + base64.c | 202 +++++++++++++++++++++++++++++++++++ base64.h | 18 +++ check-qbuffer.c | 172 ++++++++++++++++++++++++++++++ configure | 2 +- hw/hw.h | 2 + hw/qdev.c | 314 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- hw/qdev.h | 2 + qbuffer.c | 116 ++++++++++++++++++++ qbuffer.h | 33 ++++++ qemu-monitor.hx | 16 +++ qemu-queue.h | 9 ++ qjson.c | 16 +++ qobject.h | 1 + 18 files changed, 902 insertions(+), 12 deletions(-) create mode 100644 base64.c create mode 100644 base64.h create mode 100644 check-qbuffer.c create mode 100644 qbuffer.c create mode 100644 qbuffer.h
PATCH 1/8 qdev: Allow device addressing via 'driver.instance'
Extend qbus_find_dev to allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.
--- hw/qdev.c | 23 ++++++++++++++++++----- 1 files changed, 18 insertions(+), 5 deletions(-)
hw/qdev.c +18 -5
@@ -515,28 +515,41 @@ static BusState *qbus_find_bus(DeviceState *dev, char *elem)return NULL ; } -static DeviceState *qbus_find_dev (BusState *bus,char *elem) +static DeviceState *qbus_find_dev (BusState *bus,const char *elem) {DeviceState *dev; +int instance, n; +char buf[128 ];/* * try to match in order: * (1) instance id, if present -* (2) driver name -* (3) driver alias, if present +* (2) driver name [.instance] +* (3) driver alias [.instance], if present */ QLIST_FOREACH(dev, &bus->children, sibling) {if (dev->id &&strcmp (dev->id, elem) ==0 ) {return dev; } } + +if (sscanf (elem,"%127[^.].%u" , buf, &instance) ==2 ) { + elem = buf; + }else { + instance =0 ; + } + + n =0 ; QLIST_FOREACH(dev, &bus->children, sibling) { -if (strcmp (dev->info->name, elem) ==0 ) { +if (strcmp (dev->info->name, elem) ==0 && n++ == instance) {return dev; } } + + n =0 ; QLIST_FOREACH(dev, &bus->children, sibling) { -if (dev->info->alias &&strcmp (dev->info->alias, elem) ==0 ) { +if (dev->info->alias &&strcmp (dev->info->alias, elem) ==0 && + n++ == instance) {return dev; } }
Signature · 1 line
1.6.0.2
Re: [Qemu-devel] [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'
Quoted 5 lines · Jan Kiszka
Extend
qbus_find_devto allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.Signed-off-by Jan Kiszka <[email protected]>
How's the instance number defined? Should be documented.
Is an instance number a good user interface? Gerd, what do you think?
[...]
Re: [Qemu-devel] [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'
Quoted 9 lines · Markus Armbruster
Quoted 5 lines · Jan Kiszka
Extend
qbus_find_devto allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.Signed-off-by Jan Kiszka<[email protected]>How's the instance number defined? Should be documented.
savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think. Drivers can also specify one. Most don't do that. IIRC some ISA drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.
It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.
cheers,
Gerd
[Qemu-devel] Re: [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'
Quoted 18 lines · Gerd Hoffmann
Quoted 9 lines · Markus Armbruster
Quoted 5 lines · Jan Kiszka
Extend
qbus_find_devto allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.Signed-off-by Jan Kiszka<[email protected]>How's the instance number defined? Should be documented.
savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think. Drivers can also specify one. Most don't do that. IIRC some ISA drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.
Quoted 3 lines
It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.
Agreed. The problem here is that we sent the instance_id on the wire, so for "legacy" devices that used an instance_id != -1, we are stuck with it :(
Guess why a friend calls "backwards compatibility": A Curse of Bible proportions :(
Later, Juan.
[Qemu-devel] Re: [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'
Hi,
Quoted 3 lines
Agreed. The problem here is that we sent the
instance_idon the wire, so for "legacy" devices that used aninstance_id!= -1, we are stuck with it :(
Another way would be to fill the instance id with something useful, say encode the pci address there for pci devices. That should be good enough to reliably identify devices even after hot-plugging them in and out.
Of course this has transition isses too. But maybe it is easier to handle than replacing instance_id with something entirely different.
And it also makes the instance id less useful to address devices in a human-friendly way.
cheers,
Gerd
[Qemu-devel] Re: [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'
In reply to Gerd HoffmannQuoted 10 lines · Gerd Hoffmann
Quoted 9 lines · Markus Armbruster
Quoted 5 lines · Jan Kiszka
Extend
qbus_find_devto allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.Signed-off-by Jan Kiszka<[email protected]>How's the instance number defined? Should be documented.
OK, will do. For now it is the per-bus instance ID. I think that makes most sense and is easily handleable. Still, we should probably print it also via "info qtree" (and a future "query-qtree").
Quoted 7 lines
savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think. Drivers can also specify one. Most don't do that. IIRC some ISA drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.
That io-address-based instance numbers have just been deprecated, see 4d2ffa08b601bdd40d9ccf225480c0a7e90ca078. ISA devices are already converted, there are just a few non-PC devices remaining that don't use the (auto-generated) vmstate instance number. But that is actually a different, user-invisible numbering scheme.
Quoted 4 lines
It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.
You are always free to address devices via a unique user-defined ID.
Jan
Re: [Qemu-devel] [PATCH 1/8] qdev: Allow device addressing via 'driver.instance'
In reply to Gerd HoffmannQuoted 7 lines · Gerd Hoffmann
Quoted 6 lines · Markus Armbruster
Quoted 4 lines · Jan Kiszka
Extend
qbus_find_devto allow addressing of devices without an unique id via an optional instance number. The new formats are 'driver.instance' and 'alias.instance'.
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Jan Kiszka<[email protected]>How's the instance number defined? Should be documented.
savevm instance id, used to identify multiple instances of some device on loadvm. By default is just incrementing (0,1,2,...) for each new device instance I think.
That's an implementation detail that's being exposed in an external interface. Granted, users shouldn't expect this to remain stable across invocations, yet it makes me uncomfortable.
Why not always use topology to locate devices?
Quoted 4 lines
Drivers can also specify one. Most don't do that. IIRC some ISA
drivers use the base ioport as instance id, which sort-of makes sense as it makes sure the id identifies the correct device no matter what the initialization order is.It probably makes sense to replace the instance id with the device path once all devices are converted over to qdev+vmstate, so we avoid the nasty ordering issues altogether.
Oh, that's what you're suggesting. So we agree.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
PATCH 2/8 Add base64 encoder/decoder
In reply to Jan KiszkaWill be used by QBuffer.
--- Makefile.objs | 2 +- base64.c | 202 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ base64.h | 18 +++++ 3 files changed, 221 insertions(+), 1 deletions(-) create mode 100644 base64.c create mode 100644 base64.h
Makefile.objs +1 -1
@@ -2,7 +2,7 @@ # QObject qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o qobject-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o -qobject-obj-y += qerror.o +qobject-obj-y += qerror.o base64.o ####################################################################### # block-obj-y is code used by both qemu system emulation and qemu-img
base64.c new file +202 -0
202 lines not on this page · raw message
base64.h new file +18 -0
@@ -0,0 +1,18 @@ +/* +* Base64 encoder/decoder conforming to RFC 4648 +* (based on Mozilla's nsprpub/lib/libc/src/base64.c) +* +* Copyright (C) 2010 Siemens AG +* +* Authors: +* Jan Kiszka <[email protected]> +* +* This work is licensed under the terms of the GNU LGPL, version 2.1 or later. +* See the COPYING.LIB file in the top-level directory. +* +*/ + +#include <unistd.h> + +void base64_encode (const void *src,size_t srclen,char *dest); +int base64_decode (const char *src,size_t srclen,void *dest);
Signature · 1 line
1.6.0.2
PATCH 3/8 Add QBuffer
In reply to Jan KiszkaThis introduces a buffer object for use with QMP. As a buffer is not natively encodable in JSON, we encode it as a base64 string. To decode this kind of strings back to a QBuffer, the receiving side has to be aware of their semantic, which is normally no problem within QMP.
The first use case of this type is pushing the content of buffers that are part of a device state into a qdict.
--- Makefile | 3 +- Makefile.objs | 2 +- check-qbuffer.c | 172 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ configure | 2 +- qbuffer.c | 116 +++++++++++++++++++++++++++++++++++++ qbuffer.h | 33 +++++++++++ qjson.c | 16 +++++ qobject.h | 1 + 8 files changed, 342 insertions(+), 3 deletions(-) create mode 100644 check-qbuffer.c create mode 100644 qbuffer.c create mode 100644 qbuffer.h
Makefile +2 -1
@@ -149,7 +149,8 @@ check-qstring: check-qstring.o qstring.o qemu-malloc.o check-qdict: check-qdict.o qdict.o qfloat.o qint.o qstring.o qbool.o qemu-malloc.o qlist.o check-qlist: check-qlist.o qlist.o qint.o qemu-malloc.o check-qfloat: check-qfloat.o qfloat.o qemu-malloc.o -check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qjson.o json-streamer.o json-lexer.o json-parser.o qemu-malloc.o +check-qjson: check-qjson.o qfloat.o qint.o qdict.o qstring.o qlist.o qbool.o qbuffer.o base64.o qjson.o json-streamer.o json-lexer.o json-parser.o qemu-malloc.o +check-qbuffer: check-qbuffer.o qbuffer.o base64.o qstring.o qemu-malloc.o clean: # avoid old build problems by removing potentially incorrect old files
Makefile.objs +1 -1
@@ -1,6 +1,6 @@ ####################################################################### # QObject -qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o +qobject-obj-y = qint.o qstring.o qdict.o qlist.o qfloat.o qbool.o qbuffer.o qobject-obj-y += qjson.o json-lexer.o json-streamer.o json-parser.o qobject-obj-y += qerror.o base64.o
check-qbuffer.c new file +172 -0
172 lines not on this page · raw message
configure +1 -1
@@ -2280,7 +2280,7 @@ if test `expr "$target_list" : ".*softmmu.*"` != 0 ; then if [ "$linux" = "yes" -o "$bsd" = "yes" -o "$solaris" = "yes" ] ; then tools="qemu-nbd\$(EXESUF) $tools" if [ "$check_utests" = "yes" ]; then - tools="check-qint check-qstring check-qdict check-qlist $tools" + tools="check-qint check-qstring check-qdict check-qlist check-qbuffer $tools" tools="check-qfloat check-qjson $tools" fi fi
qbuffer.c new file +116 -0
116 lines not on this page · raw message
qbuffer.h new file +33 -0
@@ -0,0 +1,33 @@ +/* +* QBuffer Module +* +* Copyright (C) 2010 Siemens AG +* +* Authors: +* Jan Kiszka <[email protected]> +* +* This work is licensed under the terms of the GNU LGPL, version 2.1 or later. +* See the COPYING.LIB file in the top-level directory. +* +*/ + +#ifndef QBUFFER_H +#define QBUFFER_H + +#include <stdint.h> +#include "qobject.h" +#include "qstring.h" + +typedef struct QBuffer { +QObject_HEAD ; +void *data; +size_t size; +}QBuffer ; + +QBuffer *qbuffer_from_data (const void *data,size_t size); +QBuffer *qbuffer_from_qstring (const QString *string); +const void *qbuffer_get_data (const QBuffer *qb); +size_t qbuffer_get_size (const QBuffer *qb); +QBuffer *qobject_to_qbuffer (const QObject *obj); + +#endif /* QBUFFER_H */
qjson.c +16 -0
@@ -19,7 +19,9 @@#include "qlist.h" #include "qbool.h" #include "qfloat.h" +#include "qbuffer.h" #include "qdict.h" +#include "base64.h" typedef struct JSONParsingState {
@@ -235,6 +237,20 @@ static void to_json(const QObject *obj, QString *str) }break ; } +case QTYPE_QBUFFER : { +QBuffer *val =qobject_to_qbuffer (obj); +size_t data_size =qbuffer_get_size (val); +size_t str_len = ((data_size +2 ) /3 ) *4 ; +char *buffer =qemu_malloc (str_len +3 ); + + buffer[0 ] ='"' ; +base64_encode (qbuffer_get_data (val), data_size, buffer +1 ); + buffer[str_len +1 ] ='"' ; + buffer[str_len +2 ] =0 ; +qstring_append (str, buffer); +qemu_free (buffer); +break ; + }case QTYPE_QERROR :/* XXX: should QError be emitted? */ case QTYPE_NONE :
qobject.h +1 -0
@@ -44,6 +44,7 @@ typedef enum {QTYPE_QFLOAT ,QTYPE_QBOOL ,QTYPE_QERROR , +QTYPE_QBUFFER , } qtype_code;struct QObject ;
Signature · 1 line
1.6.0.2
Re: [Qemu-devel] [PATCH 3/8] Add QBuffer
Quoted 23 lines · Jan Kiszka
qjson.c +7
-0@@ -19,7 +19,9 @@#include "qlist.h" #include "qbool.h" #include "qfloat.h" +#include "qbuffer.h" #include "qdict.h" +#include "base64.h" typedef struct JSONParsingState {@@ -235,6 +237,20 @@ static void to_json(const QObject *obj, QString *str) }break ; } +case QTYPE_QBUFFER : { +QBuffer *val =qobject_to_qbuffer (obj); +size_t data_size =qbuffer_get_size (val); +size_t str_len = ((data_size +2 ) /3 ) *4 ; +char *buffer =qemu_malloc (str_len +3 );
qjson.c static void to_json(const QObject *obj, QString *str) · line 254
Quoted 10 lines
qjson.c +9
-0+ + buffer[0 ] ='"' ; +base64_encode (qbuffer_get_data (val), data_size, buffer +1 ); + buffer[str_len +1 ] ='"' ; + buffer[str_len +2 ] =0 ; +qstring_append (str, buffer); +qemu_free (buffer); +break ; + }
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'__class__': 'base64', 'data': ...}
We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.
The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.
Regards,
Anthony Liguori
Quoted 16 lines
qjson.c +0
-0case QTYPE_QERROR :/* XXX: should QError be emitted? */ case
QTYPE_NONE:qobject.h +1
-0@@ -44,6 +44,7 @@ typedef enum {QTYPE_QFLOAT ,QTYPE_QBOOL ,QTYPE_QERROR , +QTYPE_QBUFFER , } qtype_code;struct QObject ;
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 41 lines · Anthony Liguori
Quoted 24 lines · Jan Kiszka
qjson.c +7
-0@@ -19,7 +19,9 @@#include "qlist.h" #include "qbool.h" #include "qfloat.h" +#include "qbuffer.h" #include "qdict.h" +#include "base64.h" typedef struct JSONParsingState {@@ -235,6 +237,20 @@ static void to_json(const QObject *obj, QString *str) }break ; } +case QTYPE_QBUFFER : { +QBuffer *val =qobject_to_qbuffer (obj); +size_t data_size =qbuffer_get_size (val); +size_t str_len = ((data_size +2 ) /3 ) *4 ; +char *buffer =qemu_malloc (str_len +3 );qjson.c static void to_json(const QObject *obj, QString · line 255
Quoted 10 lines
qjson.c +9
-0+ + buffer[0 ] ='"' ; +base64_encode (qbuffer_get_data (val), data_size, buffer +1 ); + buffer[str_len +1 ] ='"' ; + buffer[str_len +2 ] =0 ; +qstring_append (str, buffer); +qemu_free (buffer); +break ; + }Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}
Quoted 7 lines
We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.
The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.
Indeed, was amazingly simple to add and works nicely with qmp-shell as demonstrator. Will repost, also to fix a few remaining glitches I came across in the meantime.
Jan
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 10 lines · Jan Kiszka
Quoted 10 lines
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.
Quoted 9 lines
Quoted 4 lines
The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.
Indeed, was amazingly simple to add and works nicely with qmp-shell as demonstrator. Will repost, also to fix a few remaining glitches I came across in the meantime.
Is this __class__ stuff documented anywhere?
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 15 lines · Avi Kivity
Quoted 14 lines · Jan Kiszka
Quoted 14 lines
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}We've discussed using hidden properties to describe special things like abstract classes and since we already have this namespace reserved, I think it's a good time to use it.
The advantage is that in a dynamic language like Python, the parser can convert base64 to binary strings automatically without having to understand the QMP protocol.
Quoted 8 lines
Quoted 5 lines
Indeed, was amazingly simple to add and works nicely with qmp-shell as demonstrator. Will repost, also to fix a few remaining glitches I came across in the meantime.
Is this
__class__stuff documented anywhere?
Not yet. Also, we should clarify the proposed private extension section that only "__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').
Jan
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 9 lines · Jan Kiszka
Quoted 4 lines
Is this
__class__stuff documented anywhere?Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').
Why use such weird names at all? What's wrong with 'class'?
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines
Is this
__class__stuff documented anywhere?Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').Why use such weird names at all? What's wrong with 'class'?
That it conflicts with e.g. PCI classes?
Paolo
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 5 lines · Paolo Bonzini
Quoted 4 lines · Avi Kivity
Quoted 3 lines · Jan Kiszka
Quoted 3 lines
Is this
__class__stuff documented anywhere?
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').Why use such weird names at all? What's wrong with 'class'?
That it conflicts with e.g. PCI classes?
Won't the context tell it apart? When you expect a pci function, 'class': 'video' means one thing, when you read a buffer it means another.
The only reason to use a special name is if it's a protocol level feature that can happen in all contexts. But in that case we're better off with a schema, we don't want to push class descriptors everywhere.
Signature · 1 line
error compiling committee.c: too many arguments to function
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 11 lines · Avi Kivity
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines
Is this
__class__stuff documented anywhere?Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Why use such weird names at all? What's wrong with 'class'?
That it conflicts with e.g. PCI classes?
Won't the context tell it apart? When you expect a pci function, 'class': 'video' means one thing, when you read a buffer it means another.
The point is to make this notation context-independent so that you can do
def __json_obj_hook(self, dct):
if '__class__' in dct:
if dct['__class__'] == 'buffer':
return base64.b64decode(dct['data'])
else:
return
return dctand
line = json.loads(self.sockfile.readline(),
object_hook=self.__json_obj_hook)
i.e. parse the QMP stream into a proper representation without known QMP at all.
Quoted 5 lines
The only reason to use a special name is if it's a protocol level feature that can happen in all contexts. But in that case we're better off with a schema, we don't want to push class descriptors everywhere.
Not everywhere, just into those nodes that aren't expressible with native JSON types.
Jan
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
In reply to Avi KivityQuoted 11 lines · Avi Kivity
Quoted 10 lines · Paolo Bonzini
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines
Is this
__class__stuff documented anywhere?Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
Why use such weird names at all? What's wrong with 'class'?
That it conflicts with e.g. PCI classes?
Won't the context tell it apart?
Yes, of course, it you need to know the schema. If you don't know the schema you don't know the context.
This QBuffer thing is something that a client QMP library could create automatically. Keys in a separate namespace (like '__class__') have the advantage of being easily picked up automatically by a wrapper of the JSON parser; if you used simply 'class' such as layer would need to know a schema, or it wouldn't know that "context".
(BTW I'd prefer something like '__encoding__'; the word "class" suggests much more than what it is in reality).
Paolo
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 14 lines · Paolo Bonzini
Quoted 13 lines · Avi Kivity
Quoted 12 lines · Paolo Bonzini
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 3 lines
Is this
__class__stuff documented anywhere?Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').Why use such weird names at all? What's wrong with 'class'?
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
That it conflicts with e.g. PCI classes?
Won't the context tell it apart?
Yes, of course, it you need to know the schema. If you don't know the schema you don't know the context.
This QBuffer thing is something that a client QMP library could create automatically. Keys in a separate namespace (like '
__class__') have the advantage of being easily picked up automatically by a wrapper of the JSON parser; if you used simply 'class' such as layer would need to know a schema, or it wouldn't know that "context".
Makes sense. So this is a protocol feature and needs to be documented as such.
Quoted 1 line
(BTW I'd prefer something like '
__encoding__'; the word "class" suggests much more than what it is in reality).
Agreed.
Signature · 1 line
error compiling committee.c: too many arguments to function
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
In reply to Avi KivityQuoted 5 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka
Quoted 4 lines
Is this
__class__stuff documented anywhere?
Quoted 7 lines
Quoted 5 lines
Not yet. Also, we should clarify the proposed private extension section that only "
__some_key" is reserved for downstream, not '__some_other_key__' (i.e. downstream names must not end with '__').Why use such weird names at all? What's wrong with 'class'?
It's too generic and we may too easily create dict keys that will be misinterpreted.
Jan
Re: [Qemu-devel] [PATCH 3/8] Add QBuffer
In reply to Anthony LiguoriQuoted 4 lines · Anthony Liguori
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}
Is there a benefit to the class indirection, over simply a keyword?:
{'__base64__': ...}
__class__ seems to suggest much more than it's being used for here.
-- Jamie
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 4 lines · Jamie Lokier
Quoted 3 lines · Anthony Liguori
Instead of encoding just as a string, it would be a good idea to encode it as something like:
Quoted 8 lines
Quoted 1 line
{'
__class__': 'base64', 'data': ...}Is there a benefit to the class indirection, over simply a keyword?:
{'
__base64__': ...}
__class__seems to suggest much more than it's being used for here.
Depending on how sophisticated your parser is, you could directly push the result into an object of the proper type. And we can add more complex objects in the future that do not only consists of a single data key. Note that this extension is not just about encoding, it is about typecasting (dict -> custom type).
Jan
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 11 lines · Jan Kiszka
Quoted 10 lines · Jamie Lokier
Quoted 4 lines · Anthony Liguori
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}Is there a benefit to the class indirection, over simply a keyword?:
{'
__base64__': ...}
Quoted 8 lines
Quoted 2 lines
__class__seems to suggest much more than it's being used for here.Depending on how sophisticated your parser is, you could directly push the result into an object of the proper type. And we can add more complex objects in the future that do not only consists of a single data key. Note that this extension is not just about encoding, it is about typecasting (dict -> custom type).
Sure, if that's the plan.
Does it make sense to combine encoding and custom types in this way? It looks like mixing syntax and semantics, which has consequences for code using generic parsers with separate semantic layer, but I realise there's no "correct" answer.
Back to the syntax: I'm under the impression from earlier discussion that the '__*__' keyspace reserved, so even types could use the compact syntax?
Or is there something Javascript-ish (and not merely JSON-ish) about '__class__' in particular which makes it appropriate?
-- Jamie
Re: [Qemu-devel] [PATCH 3/8] Add QBuffer
In reply to Jamie LokierQuoted 11 lines · Jamie Lokier
Quoted 4 lines · Anthony Liguori
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}Is there a benefit to the class indirection, over simply a keyword?:
{'
__base64__': ...}
__class__seems to suggest much more than it's being used for here.
Yes. The problem with JSON is that it's based on JavaScript and JavaScript is goofy :-)
JavaScript's object mechanism doesn't map well to most other languages since it's prototype based. What we're calling QDict's are really objects in JavaScript and they carry mostly no type information. With JS, it's very simple to treat a generic object as a specialized class after instantiation which means objects don't need type information.
For non-prototype languages, which is the vast majority of clients, it's necessary to have type information at instantiation time since monkey patching is awkward at best. That's why we need a special, reserved, object member to carry type information. The remainder of the object members represent the serialized state of the object.
Another way to think of it, is that we're already transmitting objects so we really just need a way to say, no, this isn't just a Dictionary, it's really an instance of the following class.
Regards,
Anthony Liguori
Quoted 2 lines
-- Jamie
Re: [Qemu-devel] [PATCH 3/8] Add QBuffer
Quoted 9 lines · Anthony Liguori
Quoted 7 lines · Jamie Lokier
Quoted 5 lines · Anthony Liguori
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}
Quoted 10 lines
Quoted 6 lines
Is there a benefit to the class indirection, over simply a keyword?:
{'
__base64__': ...}
__class__seems to suggest much more than it's being used for here.Yes. The problem with JSON is that it's based on JavaScript and JavaScript is goofy :-)
I suggest completely ignoring JavaScript. JSON is simply an encoding for numbers, strings, arrays, and key-value stores. Where's the goofiness?
Quoted 6 lines
JavaScript's object mechanism doesn't map well to most other languages since it's prototype based. What we're calling QDict's are really objects in JavaScript and they carry mostly no type information. With JS, it's very simple to treat a generic object as a specialized class after instantiation which means objects don't need type information.
Quoted 6 lines
For non-prototype languages, which is the vast majority of clients, it's necessary to have type information at instantiation time since monkey patching is awkward at best. That's why we need a special, reserved, object member to carry type information. The remainder of the object members represent the serialized state of the object.
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Having __class__ everywhere means we're carrying the schema in every message instead of just once.
Quoted 4 lines
Another way to think of it, is that we're already transmitting objects so we really just need a way to say, no, this isn't just a Dictionary, it's really an instance of the following class.
Are there cases where the receiver cannot infer this from the context?
As I see it, dynamic type information is easiest for dynamicically typed languages. You just have a dict of class names -> object constructor and call the constructors at runtime. Statically typed languages will need a schema to use objects, since the field types have to be known at compile time, not just run time.
Another wart is arrays: statically typed languages usually contain only objects of the same type, but here this isn't known until we process the first member.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 38 lines · Avi Kivity
Quoted 10 lines · Anthony Liguori
Quoted 7 lines · Jamie Lokier
Quoted 5 lines · Anthony Liguori
Instead of encoding just as a string, it would be a good idea to encode it as something like:
{'
__class__': 'base64', 'data': ...}Quoted 10 lines
Quoted 6 lines
Is there a benefit to the class indirection, over simply a keyword?:
{'
__base64__': ...}
__class__seems to suggest much more than it's being used for here.Yes. The problem with JSON is that it's based on JavaScript and JavaScript is goofy :-)
I suggest completely ignoring JavaScript. JSON is simply an encoding for numbers, strings, arrays, and key-value stores. Where's the goofiness?
Quoted 6 lines
JavaScript's object mechanism doesn't map well to most other languages since it's prototype based. What we're calling QDict's are really objects in JavaScript and they carry mostly no type information. With JS, it's very simple to treat a generic object as a specialized class after instantiation which means objects don't need type information.
Quoted 6 lines
For non-prototype languages, which is the vast majority of clients, it's necessary to have type information at instantiation time since monkey patching is awkward at best. That's why we need a special, reserved, object member to carry type information. The remainder of the object members represent the serialized state of the object.
Quoted 6 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Having
__class__everywhere means we're carrying the schema in every message instead of just once.
The device_show command is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.
I really don't see the problem with __class__. Being a text protocol, JSON is already fairly verbose.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 5 lines · Jan Kiszka
Quoted 5 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Quoted 9 lines
Quoted 3 lines
Having
__class__everywhere means we're carrying the schema in every message instead of just once.The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.
What is the receiver to do with it?
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then __class__ is unnecessary.
My worry is that __class__ will make the protocol more ad-hoc.
Quoted 3 lines
I really don't see the problem with
__class__. Being a text protocol, JSON is already fairly verbose.
The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 15 lines · Avi Kivity
Quoted 4 lines · Jan Kiszka
Quoted 4 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Quoted 9 lines
Quoted 3 lines
Having
__class__everywhere means we're carrying the schema in every message instead of just once.The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.
Quoted 5 lines
What is the receiver to do with it?
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then
__class__is unnecessary.
There is a schema describing the fields (name, size, number of elements), but their types (int, buffer, sub-field, array of X) are derived from the JSON objects (ie. the JSON parser does this job).
Quoted 10 lines
My worry is that
__class__will make the protocol more ad-hoc.Quoted 3 lines
I really don't see the problem with
__class__. Being a text protocol, JSON is already fairly verbose.The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.
What clients do you have in mind?
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 19 lines · Jan Kiszka
Quoted 17 lines · Avi Kivity
Quoted 15 lines · Jan Kiszka
Quoted 8 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Having
__class__everywhere means we're carrying the schema in every message instead of just once.The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.
Quoted 8 lines
Quoted 6 lines
What is the receiver to do with it?
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then
__class__is unnecessary.There is a schema describing the fields (name, size, number of elements),
Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.
Quoted 3 lines
but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).
The names of fields are also type information.
Quoted 11 lines
Quoted 8 lines
Quoted 4 lines
I really don't see the problem with
__class__. Being a text protocol, JSON is already fairly verbose.The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.
What clients do you have in mind?
Any client that doesn't allow object types to be created dynamically; C, C++, Java, and the like could all benefit from a schema and wouldn't be able to do much with __class__ unless all classes were predefined. Python, JavaScript, and the like wouldn't care.
Another way of looking at it: if the client sees { __class__: foo, f1: 10, f2: 9 }, it cannot derive any information from __class__ unless it was aware of foo beforehand. If that's the case, let's make it part of the schema so it is available at compile time instead of runtime.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 3 lines · Avi Kivity
Another way of looking at it: if the client sees {
__class__: foo, f1: 10, f2: 9 }, it cannot derive any information from__class__unless it was aware of foo beforehand. If that's the case, let's make it part of the schema so it is available at compile time instead of runtime.
Or both. That's similar to a non-object field's type being described both in the schema at during runtime. A static parser can verify __class__ matches the expectations, a dynamic parser can look up __class__ and call the appropriate constructor.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
In reply to Avi KivityQuoted 29 lines · Avi Kivity
Quoted 19 lines · Jan Kiszka
Quoted 17 lines · Avi Kivity
Quoted 15 lines · Jan Kiszka
Quoted 8 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Having
__class__everywhere means we're carrying the schema in every message instead of just once.The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.Quoted 8 lines
Quoted 6 lines
What is the receiver to do with it?
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then
__class__is unnecessary.There is a schema describing the fields (name, size, number of elements),
Quoted 8 lines
Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.
Quoted 3 lines
but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).The names of fields are also type information.
Not in the case of device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.
Quoted 18 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
I really don't see the problem with
__class__. Being a text protocol, JSON is already fairly verbose.Quoted 7 lines
Quoted 4 lines
The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.
What clients do you have in mind?
Any client that doesn't allow object types to be created dynamically; C, C++, Java, and the like could all benefit from a schema and wouldn't be able to do much with
__class__unless all classes were predefined. Python, JavaScript, and the like wouldn't care.
Quoted 5 lines
Another way of looking at it: if the client sees {
__class__: foo, f1:
10, f2: 9 }, it cannot derive any information from__class__unless it was aware of foo beforehand. If that's the case, let's make it part of the schema so it is available at compile time instead of runtime.
Maybe a misunderstanding on my side: I'm not arguing against predefining what __class__ values exists and how dicts that carry such keys are encoded.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 7 lines · Jan Kiszka
Quoted 3 lines
The names of fields are also type information.
Not in the case of
device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.
For that use case, I agree. Maybe we should send both the parsed and unparsed information.
But if the client isn't going to interpret the object and only display it, then there is no need for __class__?
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 10 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka
Quoted 2 lines
The names of fields are also type information.
Not in the case of
device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.For that use case, I agree. Maybe we should send both the parsed and unparsed information.
Now I can't parse what you mean.
Quoted 3 lines
But if the client isn't going to interpret the object and only display it, then there is no need for
__class__?
For that uncommon case, yes. But the common one is to perform a bit more than raw JSON dictionary printing.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 12 lines · Jan Kiszka
Quoted 10 lines · Avi Kivity
Quoted 8 lines · Jan Kiszka
Quoted 3 lines
The names of fields are also type information.
Not in the case of
device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.
Quoted 5 lines
Quoted 3 lines
For that use case, I agree. Maybe we should send both the parsed and unparsed information.
Now I can't parse what you mean.
Sorry. I meant that if we have a raw buffer that we can decode (like pci config space and its fields) maybe it makes sense to send both formats. The raw buffer is something likely to be stable, and the decoded fields are more readable. But I realize now that doesn't make sense in the context of base64 encoding which started all this.
Quoted 6 lines
Quoted 3 lines
But if the client isn't going to interpret the object and only display it, then there is no need for
__class__?For that uncommon case, yes. But the common one is to perform a bit more than raw JSON dictionary printing.
But the common one is likely to know the type of the object beforehand (or it can't do much beyond printing, since it has no idea what fields to expect).
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Re: [Qemu-devel] Re: [PATCH 3/8] Add QBuffer
In reply to Jan KiszkaQuoted 27 lines · Jan Kiszka
Quoted 26 lines · Avi Kivity
Quoted 25 lines · Jan Kiszka
Quoted 23 lines · Avi Kivity
Quoted 8 lines · Jan Kiszka
Quoted 8 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Having
__class__everywhere means we're carrying the schema in every message instead of just once.Quoted 7 lines
The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.What is the receiver to do with it?
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then
__class__is unnecessary.
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
There is a schema describing the fields (name, size, number of elements),
Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.
Doesn't that tie you to a specific version of QMP at compile-time?
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).The names of fields are also type information.
Not in the case of
device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.
qdev device properties are similar. Right now, they occur only as arguments of device_add. When do_info_qtree() gets converted, they'll appear in results.
[...]
Re: [Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 6 lines · Markus Armbruster
Quoted 4 lines
Quoted 4 lines
Surely the schema has to describe the type as well? If it does, you can use the schema to generate a classes at compile time.
Doesn't that tie you to a specific version of QMP at compile-time?
The client needs to ignore anything not provided by the schema to be forward compatible.
(alternatively we make sure anything new is explicitly enabled by the client, so it can parse strictly according to the schema).
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
but their types (int, buffer, sub-field, array of X) are
derived from the JSON objects (ie. the JSON parser does this job).The names of fields are also type information.
Quoted 8 lines
Quoted 4 lines
Not in the case of
device_show. The clients have no idea of the vmstate structures before they were transfered. Granted, that will likely remain a special case in the QMP command set.qdev device properties are similar. Right now, they occur only as arguments of
device_add. Whendo_info_qtree()gets converted, they'll appear in results.
This sounds like a horror movie in the making, "sysfs: the return". The qdev tree is completely undocumented, so once again the code dictates the protocol. Any bug in the qdev hierarchy will be hardcoded forever and ever, or we have to add a new layer of indirection to have a separate internal qdev and an external qdev-for-qmp tree(s).
Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this signature is too narrow to contain.
[Qemu-devel] Re: [PATCH 3/8] Add QBuffer
In reply to Avi KivityQuoted 16 lines · Avi Kivity
Quoted 5 lines · Jan Kiszka
Quoted 5 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type information (you can't even distinguish between a number and text) yet C clients have to problem extracting typed information from it.
Quoted 7 lines
Quoted 2 lines
Having
__class__everywhere means we're carrying the schema in every message instead of just once.The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.What is the receiver to do with it?
Quoted 8 lines
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then
__class__is unnecessary.My worry is that
__class__will make the protocol more ad-hoc.Quoted 2 lines
I really don't see the problem with
__class__. Being a text protocol, JSON is already fairly verbose.The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.
Whether a protocol is self-describing is orthogonal to whether it's well defined (ala a schema). A self-describing protocol is very convenient for dynamic languages like Python. We should also provide a formal schema though for languages that require IDL to generate bindings (like C).
Regards,
Anthony Liguori
Re: [Qemu-devel] Re: [PATCH 3/8] Add QBuffer
Quoted 33 lines · Anthony Liguori
Quoted 24 lines · Avi Kivity
Quoted 7 lines · Jan Kiszka
Quoted 7 lines
The alternative is to have a schema. Sun RPC/XDR doesn't carry any type
information (you can't even distinguish between a number and text) yet C
clients have to problem extracting typed information from it.Quoted 7 lines
Quoted 2 lines
Having
__class__everywhere means we're carrying the schema in every message instead of just once.The
device_showcommand is already an example where you don't have a predefined schema. It is derived from the data stream the encodes the vmstate fields. So far we have no collision between base64-encoded buffers and real strings, but this may actually change when we start annotating the fields with symbolic constants.What is the receiver to do with it?
If it doesn't know the schema (and there is no schema), then all it can do is display the key/values. If it does know the schema, then
__class__is unnecessary.My worry is that
__class__will make the protocol more ad-hoc.Quoted 7 lines
Quoted 2 lines
I really don't see the problem with
__class__. Being a text protocol, JSON is already fairly verbose.The problem is not the verbosity, it's that information is carried too late. Many clients want to know this information at compile time or initialization time, and we are providing it at object instantiating time.
Quoted 5 lines
Whether a protocol is self-describing is orthogonal to whether it's well defined (ala a schema). A self-describing protocol is very convenient for dynamic languages like Python. We should also provide a formal schema though for languages that require IDL to generate bindings (like C).
And that schema should be available over QMP.
PATCH 4/8 monitor: Add basic device state visualization
In reply to Jan KiszkaThis introduces device_show, a monitor command that saves the vmstate of a qdev device and visualizes it. QMP is also supported. Buffers are cut after 16 byte by default, but the full content can be requested via '-f'. To pretty-print sub-arrays, vmstate is extended to store the start index name.
--- hw/hw.h | 2 + hw/qdev.c | 287 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/qdev.h | 2 + qemu-monitor.hx | 16 +++ 4 files changed, 307 insertions(+), 0 deletions(-)
hw/hw.h +2 -0
@@ -299,6 +299,7 @@ enum VMStateFlags { typedef struct { const char *name; + const char *start_index; size_t offset; size_t size; size_t start;
@@ -414,6 +415,7 @@ extern const VMStateInfo vmstate_info_unused_buffer; .size = sizeof(_type), \ .flags = VMS_ARRAY, \ .offset = vmstate_offset_sub_array(_state, _field, _type, _start), \ + .start_index = (stringify(_start)), \ }#define VMSTATE_VARRAY_INT32 (_field, _state, _field_num, _version, _info, _type) {\
hw/qdev.c +287 -0
296 lines not on this page · raw message
hw/qdev.h +2 -0
@@ -177,6 +177,8 @@ void do_info_qtree(Monitor *mon);void do_info_qdm (Monitor *mon);int do_device_add (Monitor *mon,const QDict *qdict,QObject **ret_data);int do_device_del (Monitor *mon,const QDict *qdict,QObject **ret_data); +void device_user_print (Monitor *mon,const QObject *data); +int do_device_show (Monitor *mon,const QDict *qdict,QObject **ret_data);/*** qdev-properties.c ***/
qemu-monitor.hx +16 -0
@@ -599,6 +599,22 @@ Remove device @var{id}. ETEXI { + .name = "device_show", + .args_type = "path:s,full:-f", + .params = "device [-f]", + .help = "show device state (specify -f for full buffer dumping)", + .user_print = device_user_print, + .mhandler.cmd_new = do_device_show, + }, + +STEXI +@item device_show @var{id} [@code{-f}] + +Show state of device @var{id} in a human-readable form. Buffers are cut after +16 bytes unless a full dump is requested via @code{-f} +ETEXI + + { .name = "cpu", .args_type = "index:i", .params = "index",
Signature · 1 line
1.6.0.2
Re: [Qemu-devel] [PATCH 4/8] monitor: Add basic device state visualization
Quoted 7 lines · Jan Kiszka
This introduces
device_show, a monitor command that saves the vmstate of a qdev device and visualizes it. QMP is also supported. Buffers are cut after 16 byte by default, but the full content can be requested via '-f'. To pretty-print sub-arrays, vmstate is extended to store the start index name.Signed-off-by Jan Kiszka <[email protected]>
Neato! Comments inline.
Quoted 12 lines
--- hw/hw.h | 2 + hw/qdev.c | 287 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/qdev.h | 2 + qemu-monitor.hx | 16 +++ 4 files changed, 307 insertions(+), 0 deletions(-)hw/hw.h +0
-0
hw/hw.h enum VMStateFlags { · line 305
Quoted 7 lines
hw/hw.h +1
-0@@ -299,6 +299,7 @@ enum VMStateFlags { typedef struct { const char *name; + const char *start_index; size_t offset; size_t size; size_t start;
Why is start_index a string?
Quoted 30 lines
hw/hw.h +1
-0@@ -414,6 +415,7 @@ extern const VMStateInfo vmstate_info_unused_buffer; .size = sizeof(_type), \ .flags = VMS_ARRAY, \ .offset = vmstate_offset_sub_array(_state, _field, _type, _start), \ + .start_index = (stringify(_start)), \ }#define VMSTATE_VARRAY_INT32 (_field, _state, _field_num, _version, _info, _type) {\hw/qdev.c +7
-0@@ -29,6 +29,9 @@#include "qdev.h" #include "sysemu.h" #include "monitor.h" +#include "qjson.h" +#include "qint.h" +#include "qbuffer.h" static int qdev_hotplug =0 ;@@ -824,3 +827,287 @@ int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) }return qdev_unplug (dev); } + +static int print_array_index (Monitor *mon,const char *start,int index) +{ +char index_str[32 ];
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 843
Quoted 10 lines
hw/qdev.c +10
-0+int len; + +if (start) { + len =snprintf (index_str,sizeof (index_str),"[%s+%02x]" , start, + index); + }else { + len =snprintf (index_str,sizeof (index_str),"[%02x]" , index); + } +monitor_printf (mon, index_str); +return len;
You go via the snprintf() detour just because monitor_printf() doesn't return the number of characters printed. Wouldn't it be better to fix that?
Quoted 40 lines
hw/qdev.c +40
-0+} + +#define NAME_COLUMN_WIDTH 23 + +static void print_field (Monitor *mon,const QDict *qfield,int indent); + +static void print_elem (Monitor *mon,const QObject *qelem,size_t size, +int column_pos,int indent) +{ +int64_t data_size; +const void *data; +int n; + +if (qobject_type (qelem) ==QTYPE_QDICT ) { +if (column_pos >=0 ) { +monitor_printf (mon,".\n" ); + } + }else { +monitor_printf (mon,":" ); + column_pos++; +if (column_pos <NAME_COLUMN_WIDTH ) { +monitor_printf (mon,"%*c" ,NAME_COLUMN_WIDTH - column_pos,' ' ); + } + } + +switch (qobject_type (qelem)) { +case QTYPE_QDICT : +print_field (mon,qobject_to_qdict (qelem), indent +2 ); +break ; +case QTYPE_QBUFFER : + data =qbuffer_get_data (qobject_to_qbuffer (qelem)); + data_size =qbuffer_get_size (qobject_to_qbuffer (qelem)); +for (n =0 ; n < data_size; ) { +monitor_printf (mon," %02x" , *((uint8_t *)data+n)); +if (++n < size) { +if (n %16 ==0 ) { +monitor_printf (mon,"\n%*c" ,NAME_COLUMN_WIDTH ,' ' ); + }else if (n %8 ==0 ) { +monitor_printf (mon," -" ); + }
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 893
Quoted 10 lines
hw/qdev.c +10
-0+ } + } +if (data_size < size) { +monitor_printf (mon," ..." ); + } +monitor_printf (mon,"\n" ); +break ; +case QTYPE_QINT : +monitor_printf (mon," %0*x\n" , (int )size *2 , + (int )qint_get_int (qobject_to_qint (qelem)));
I'm confused. Doesn't casting qint_get_int() to int lose bits?
Quoted 178 lines
hw/qdev.c +178
-0+break ; +default : +assert (0 ); + } +} + +static void print_field (Monitor *mon,const QDict *qfield,int indent) +{ +const char *name =qdict_get_str (qfield,"name" ); +const char *start =qdict_get_try_str (qfield,"start" ); +int64_t size =qdict_get_int (qfield,"size" ); +QList *qlist =qdict_get_qlist (qfield,"elems" ); +QListEntry *entry, *sub_entry; +QList *sub_list; +int elem_no =0 ; + + QLIST_FOREACH_ENTRY(qlist, entry) { +QObject *qelem =qlist_entry_obj (entry); +int pos = indent +strlen (name); + +if (qobject_type (qelem) ==QTYPE_QLIST ) { +monitor_printf (mon,"%*c%s" , indent,' ' , name); + pos +=print_array_index (mon, start, elem_no); + sub_list =qobject_to_qlist (qelem); + QLIST_FOREACH_ENTRY(sub_list, sub_entry) { +print_elem (mon,qlist_entry_obj (sub_entry), size, pos, + indent +2 ); + pos =-1 ; + } + }else { +if (elem_no ==0 ) { +monitor_printf (mon,"%*c%s" , indent,' ' , name); + }else { + pos =-1 ; + } +print_elem (mon, qelem, size, pos, indent); + } + elem_no++; + } +} + +void device_user_print (Monitor *mon,const QObject *data) +{ +QDict *qdict =qobject_to_qdict (data); +QList *qlist =qdict_get_qlist (qdict,"fields" ); +QListEntry *entry; + +monitor_printf (mon,"dev: %s, id \"%s\"\n" , +qdict_get_str (qdict,"device" ), +qdict_get_str (qdict,"id" )); + + QLIST_FOREACH_ENTRY(qlist, entry) { +print_field (mon,qobject_to_qdict (qlist_entry_obj (entry)),2 ); + } +} + +static void parse_vmstate (const VMStateDescription *vmsd,void *opaque, +QList *qlist,int full_buffers) +{ +VMStateField *field = vmsd->fields; + +if (vmsd->pre_save) { + vmsd->pre_save (opaque); + } +while (field->name) { +if (!field->field_exists || + field->field_exists (opaque, vmsd->version_id)) { +void *base_addr = opaque + field->offset; +int i, n_elems =1 ; +int is_array =1 ; +size_t size = field->size; +QDict *qfield =qdict_new (); +QList *qelems =qlist_new (); + +qlist_append_obj (qlist,QOBJECT (qfield)); + +qdict_put_obj (qfield,"name" , +QOBJECT (qstring_from_str (field->name))); +qdict_put_obj (qfield,"elems" ,QOBJECT (qelems)); + +if (field->flags &VMS_VBUFFER ) { + size = *(int32_t *)(opaque + field->size_offset); +if (field->flags &VMS_MULTIPLY ) { + size *= field->size; + } + } +qdict_put_obj (qfield,"size" ,QOBJECT (qint_from_int (size))); +if (field->start_index) { +qdict_put_obj (qfield,"start" , +QOBJECT (qstring_from_str (field->start_index))); + } + +if (field->flags &VMS_ARRAY ) { + n_elems = field->num; + }else if (field->flags &VMS_VARRAY_INT32 ) { + n_elems = *(int32_t *)(opaque + field->num_offset); + }else if (field->flags &VMS_VARRAY_UINT16 ) { + n_elems = *(uint16_t *)(opaque + field->num_offset); + }else { + is_array =0 ; + } +if (field->flags &VMS_POINTER ) { + base_addr = *(void **)base_addr + field->start; + } +for (i =0 ; i < n_elems; i++) { +void *addr = base_addr + size * i; +QList *sub_elems = qelems; +int val; + +if (is_array) { + sub_elems =qlist_new (); +qlist_append_obj (qelems,QOBJECT (sub_elems)); + } +if (field->flags &VMS_ARRAY_OF_POINTER ) { + addr = *(void **)addr; + } +if (field->flags &VMS_STRUCT ) { +parse_vmstate (field->vmsd, addr, sub_elems, full_buffers); + }else { +if (field->flags & (VMS_BUFFER |VMS_VBUFFER )) { +if (!full_buffers && size >16 ) { + size =16 ; + } +qlist_append_obj (sub_elems, +QOBJECT (qbuffer_from_data (addr, + size))); + }else { +switch (size) { +case 1 : + val = *(uint8_t *)addr; +break ; +case 2 : + val = *(uint16_t *)addr; +break ; +case 4 : + val = *(uint32_t *)addr; +break ; +case 8 : + val = *(uint64_t *)addr; +break ; +default : +assert (0 ); + } +qlist_append_obj (sub_elems, +QOBJECT (qint_from_int (val))); + } + } + } + } + field++; + } +} + +static DeviceState *qdev_find (const char *path) +{ +const char *dev_name; +DeviceState *dev; +char *bus_path; +BusState *bus; + + dev_name =strrchr (path,'/' ); +if (!dev_name) { + bus = main_system_bus; + dev_name = path; + }else { + dev_name++; + bus_path =qemu_strdup (path); + bus_path[dev_name - path] =0 ; + + bus =qbus_find (bus_path); +qemu_free (bus_path); + +if (!bus) { +/* qbus_find already reported the error */ +return NULL ; + } + } + dev =qbus_find_dev (bus, dev_name);
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1081
Quoted 10 lines
hw/qdev.c +10
-0+if (!dev) { +qerror_report (QERR_DEVICE_NOT_FOUND , dev_name); +if (!monitor_cur_is_qmp ()) { +qbus_list_dev (bus); + } + } +return dev; +} + +int do_device_show (Monitor *mon,const QDict *qdict,QObject **ret_data)
Command documentation missing.
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1094
Quoted 13 lines
hw/qdev.c +13
-0+{ +const char *path =qdict_get_str (qdict,"path" ); +const VMStateDescription *vmsd; +DeviceState *dev; +QList *qlist; + + dev =qdev_find_recursive (main_system_bus, path); +if (!dev) { + dev =qdev_find (path); +if (!dev) { +return -1 ; + } + }
This is inconsistent with how do_device_del() looks up devices by ID. Let's agree on one lookup, put it into a function, and use it everywhere.
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1102
Quoted 8 lines
hw/qdev.c +8
-0+ + vmsd = dev->info->vmsd; +if (!vmsd) { +qerror_report (QERR_UNDEFINED_ERROR ); +if (!monitor_cur_is_qmp ()) { +error_printf_unless_qmp ("Device '%s' does not support state " +"dumping\n" , path); + }
Please use a suitable QERR_. Define one if necessary.
Quoted 51 lines
hw/qdev.c +11
-0+return -1 ; + } + + *ret_data =qobject_from_jsonf ("{ 'device': %s, 'id': %s }" , + dev->info->name, dev->id ? :"" ); + qlist =qlist_new (); +parse_vmstate (vmsd, dev, qlist,qdict_get_int (qdict,"full" )); +qdict_put_obj (qobject_to_qdict (*ret_data),"fields" ,QOBJECT (qlist)); + +return 0 ; +}hw/qdev.h +2
-0@@ -177,6 +177,8 @@ void do_info_qtree(Monitor *mon);void do_info_qdm (Monitor *mon);int do_device_add (Monitor *mon,const QDict *qdict,QObject **ret_data);int do_device_del (Monitor *mon,const QDict *qdict,QObject **ret_data); +void device_user_print (Monitor *mon,const QObject *data); +int do_device_show (Monitor *mon,const QDict *qdict,QObject **ret_data);/*** qdev-properties.c ***/ qemu-monitor.hx +16
-0@@ -599,6 +599,22 @@ Remove device @var{id}. ETEXI { + .name = "device_show", + .args_type = "path:s,full:-f", + .params = "device [-f]", + .help = "show device state (specify -f for full buffer dumping)", + .user_print = device_user_print, + .mhandler.cmd_new = do_device_show, + }, + +STEXI +@item device_show @var{id} [@code{-f}] + +Show state of device @var{id} in a human-readable form. Buffers are cut after +16 bytes unless a full dump is requested via @code{-f} +ETEXI + + { .name = "cpu", .args_type = "index:i", .params = "index",
[Qemu-devel] Re: [PATCH 4/8] monitor: Add basic device state visualization
Quoted 25 lines · Markus Armbruster
Quoted 7 lines · Jan Kiszka
This introduces
device_show, a monitor command that saves the vmstate of a qdev device and visualizes it. QMP is also supported. Buffers are cut after 16 byte by default, but the full content can be requested via '-f'. To pretty-print sub-arrays, vmstate is extended to store the start index name.Signed-off-by Jan Kiszka <[email protected]>Neato! Comments inline.
Quoted 13 lines
--- hw/hw.h | 2 + hw/qdev.c | 287 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ hw/qdev.h | 2 + qemu-monitor.hx | 16 +++ 4 files changed, 307 insertions(+), 0 deletions(-)hw/hw.h +0
-0@@ -299,6 +299,7 @@ enum VMStateFlags {
Quoted 8 lines
hw/hw.h enum VMStateFlags { · line 305
Quoted 6 lines
hw/hw.h +1
-0typedef struct { const char *name; + const char *start_index; size_t offset; size_t size; size_t start;Why is
start_indexa string?
It can, actually should be a symbolic constant (from e1000:
"mac_reg[RA+03]: ...", ie. "RA" is the start index here).
Quoted 37 lines
Quoted 37 lines
hw/hw.h +1
-0@@ -414,6 +415,7 @@ extern const VMStateInfo vmstate_info_unused_buffer; .size = sizeof(_type), \ .flags = VMS_ARRAY, \ .offset = vmstate_offset_sub_array(_state, _field, _type, _start), \ + .start_index = (stringify(_start)), \ }#define VMSTATE_VARRAY_INT32 (_field, _state, _field_num, _version, _info, _type) {\hw/qdev.c +13
-0@@ -29,6 +29,9 @@#include "qdev.h" #include "sysemu.h" #include "monitor.h" +#include "qjson.h" +#include "qint.h" +#include "qbuffer.h" static int qdev_hotplug =0 ;@@ -824,3 +827,287 @@ int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) }return qdev_unplug (dev); } + +static int print_array_index (Monitor *mon,const char *start,int index) +{ +char index_str[32 ]; +int len; + +if (start) { + len =snprintf (index_str,sizeof (index_str),"[%s+%02x]" , start, + index); + }else {
Quoted 8 lines
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 843
Quoted 4 lines
hw/qdev.c +4
-0+ len =snprintf (index_str,sizeof (index_str),"[%02x]" , index); + } +monitor_printf (mon, index_str); +return len;You go via the
snprintf()detour just becausemonitor_printf()doesn't return the number of characters printed. Wouldn't it be better to fix that?
Yeah, good point. Will address this.
Quoted 45 lines
Quoted 45 lines
hw/qdev.c +44
-0+} + +#define NAME_COLUMN_WIDTH 23 + +static void print_field (Monitor *mon,const QDict *qfield,int indent); + +static void print_elem (Monitor *mon,const QObject *qelem,size_t size, +int column_pos,int indent) +{ +int64_t data_size; +const void *data; +int n; + +if (qobject_type (qelem) ==QTYPE_QDICT ) { +if (column_pos >=0 ) { +monitor_printf (mon,".\n" ); + } + }else { +monitor_printf (mon,":" ); + column_pos++; +if (column_pos <NAME_COLUMN_WIDTH ) { +monitor_printf (mon,"%*c" ,NAME_COLUMN_WIDTH - column_pos,' ' ); + } + } + +switch (qobject_type (qelem)) { +case QTYPE_QDICT : +print_field (mon,qobject_to_qdict (qelem), indent +2 ); +break ; +case QTYPE_QBUFFER : + data =qbuffer_get_data (qobject_to_qbuffer (qelem)); + data_size =qbuffer_get_size (qobject_to_qbuffer (qelem)); +for (n =0 ; n < data_size; ) { +monitor_printf (mon," %02x" , *((uint8_t *)data+n)); +if (++n < size) { +if (n %16 ==0 ) { +monitor_printf (mon,"\n%*c" ,NAME_COLUMN_WIDTH ,' ' ); + }else if (n %8 ==0 ) { +monitor_printf (mon," -" ); + } + } + } +if (data_size < size) { +monitor_printf (mon," ..." );
Quoted 8 lines
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 893
Quoted 6 lines
hw/qdev.c +6
-0+ } +monitor_printf (mon,"\n" ); +break ; +case QTYPE_QINT : +monitor_printf (mon," %0*x\n" , (int )size *2 , + (int )qint_get_int (qobject_to_qint (qelem)));I'm confused. Doesn't casting
qint_get_int()to int lose bits?
You are right, will fix!
Quoted 183 lines
Quoted 183 lines
hw/qdev.c +182
-0+break ; +default : +assert (0 ); + } +} + +static void print_field (Monitor *mon,const QDict *qfield,int indent) +{ +const char *name =qdict_get_str (qfield,"name" ); +const char *start =qdict_get_try_str (qfield,"start" ); +int64_t size =qdict_get_int (qfield,"size" ); +QList *qlist =qdict_get_qlist (qfield,"elems" ); +QListEntry *entry, *sub_entry; +QList *sub_list; +int elem_no =0 ; + + QLIST_FOREACH_ENTRY(qlist, entry) { +QObject *qelem =qlist_entry_obj (entry); +int pos = indent +strlen (name); + +if (qobject_type (qelem) ==QTYPE_QLIST ) { +monitor_printf (mon,"%*c%s" , indent,' ' , name); + pos +=print_array_index (mon, start, elem_no); + sub_list =qobject_to_qlist (qelem); + QLIST_FOREACH_ENTRY(sub_list, sub_entry) { +print_elem (mon,qlist_entry_obj (sub_entry), size, pos, + indent +2 ); + pos =-1 ; + } + }else { +if (elem_no ==0 ) { +monitor_printf (mon,"%*c%s" , indent,' ' , name); + }else { + pos =-1 ; + } +print_elem (mon, qelem, size, pos, indent); + } + elem_no++; + } +} + +void device_user_print (Monitor *mon,const QObject *data) +{ +QDict *qdict =qobject_to_qdict (data); +QList *qlist =qdict_get_qlist (qdict,"fields" ); +QListEntry *entry; + +monitor_printf (mon,"dev: %s, id \"%s\"\n" , +qdict_get_str (qdict,"device" ), +qdict_get_str (qdict,"id" )); + + QLIST_FOREACH_ENTRY(qlist, entry) { +print_field (mon,qobject_to_qdict (qlist_entry_obj (entry)),2 ); + } +} + +static void parse_vmstate (const VMStateDescription *vmsd,void *opaque, +QList *qlist,int full_buffers) +{ +VMStateField *field = vmsd->fields; + +if (vmsd->pre_save) { + vmsd->pre_save (opaque); + } +while (field->name) { +if (!field->field_exists || + field->field_exists (opaque, vmsd->version_id)) { +void *base_addr = opaque + field->offset; +int i, n_elems =1 ; +int is_array =1 ; +size_t size = field->size; +QDict *qfield =qdict_new (); +QList *qelems =qlist_new (); + +qlist_append_obj (qlist,QOBJECT (qfield)); + +qdict_put_obj (qfield,"name" , +QOBJECT (qstring_from_str (field->name))); +qdict_put_obj (qfield,"elems" ,QOBJECT (qelems)); + +if (field->flags &VMS_VBUFFER ) { + size = *(int32_t *)(opaque + field->size_offset); +if (field->flags &VMS_MULTIPLY ) { + size *= field->size; + } + } +qdict_put_obj (qfield,"size" ,QOBJECT (qint_from_int (size))); +if (field->start_index) { +qdict_put_obj (qfield,"start" , +QOBJECT (qstring_from_str (field->start_index))); + } + +if (field->flags &VMS_ARRAY ) { + n_elems = field->num; + }else if (field->flags &VMS_VARRAY_INT32 ) { + n_elems = *(int32_t *)(opaque + field->num_offset); + }else if (field->flags &VMS_VARRAY_UINT16 ) { + n_elems = *(uint16_t *)(opaque + field->num_offset); + }else { + is_array =0 ; + } +if (field->flags &VMS_POINTER ) { + base_addr = *(void **)base_addr + field->start; + } +for (i =0 ; i < n_elems; i++) { +void *addr = base_addr + size * i; +QList *sub_elems = qelems; +int val; + +if (is_array) { + sub_elems =qlist_new (); +qlist_append_obj (qelems,QOBJECT (sub_elems)); + } +if (field->flags &VMS_ARRAY_OF_POINTER ) { + addr = *(void **)addr; + } +if (field->flags &VMS_STRUCT ) { +parse_vmstate (field->vmsd, addr, sub_elems, full_buffers); + }else { +if (field->flags & (VMS_BUFFER |VMS_VBUFFER )) { +if (!full_buffers && size >16 ) { + size =16 ; + } +qlist_append_obj (sub_elems, +QOBJECT (qbuffer_from_data (addr, + size))); + }else { +switch (size) { +case 1 : + val = *(uint8_t *)addr; +break ; +case 2 : + val = *(uint16_t *)addr; +break ; +case 4 : + val = *(uint32_t *)addr; +break ; +case 8 : + val = *(uint64_t *)addr; +break ; +default : +assert (0 ); + } +qlist_append_obj (sub_elems, +QOBJECT (qint_from_int (val))); + } + } + } + } + field++; + } +} + +static DeviceState *qdev_find (const char *path) +{ +const char *dev_name; +DeviceState *dev; +char *bus_path; +BusState *bus; + + dev_name =strrchr (path,'/' ); +if (!dev_name) { + bus = main_system_bus; + dev_name = path; + }else { + dev_name++; + bus_path =qemu_strdup (path); + bus_path[dev_name - path] =0 ; + + bus =qbus_find (bus_path); +qemu_free (bus_path); + +if (!bus) { +/* qbus_find already reported the error */ +return NULL ; + } + } + dev =qbus_find_dev (bus, dev_name); +if (!dev) { +qerror_report (QERR_DEVICE_NOT_FOUND , dev_name); +if (!monitor_cur_is_qmp ()) { +qbus_list_dev (bus);
Quoted 8 lines
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1081
Quoted 6 lines
hw/qdev.c +6
-0+ } + } +return dev; +} + +int do_device_show (Monitor *mon,const QDict *qdict,QObject **ret_data)Command documentation missing.
In the make (depends on QMP doc infrastructure).
Quoted 10 lines
Quoted 10 lines
hw/qdev.c +9
-0+{ +const char *path =qdict_get_str (qdict,"path" ); +const VMStateDescription *vmsd; +DeviceState *dev; +QList *qlist; + + dev =qdev_find_recursive (main_system_bus, path); +if (!dev) { + dev =qdev_find (path);
Quoted 8 lines
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1094
Quoted 4 lines
hw/qdev.c +4
-0+if (!dev) { +return -1 ; + } + }This is inconsistent with how
do_device_del()looks up devices by ID. Let's agree on one lookup, put it into a function, and use it everywhere.
Will adjust device_del to the same scheme.
BTW, qdev path completion will also be available for both commands in my next series.
Quoted 11 lines
hw/qdev.c int do_device_del(Monitor *mon, const QDict *qdict, QObject **ret_data) · line 1102
Quoted 9 lines
hw/qdev.c +8
-0+ + vmsd = dev->info->vmsd; +if (!vmsd) { +qerror_report (QERR_UNDEFINED_ERROR ); +if (!monitor_cur_is_qmp ()) { +error_printf_unless_qmp ("Device '%s' does not support state " +"dumping\n" , path); + }Please use a suitable
QERR_. Define one if necessary.
Already fixed in my tree (forgot this bit for v1). Just this redundant qmp check was still present.
Quoted 44 lines
Quoted 44 lines
hw/qdev.c +11
-0+return -1 ; + } + + *ret_data =qobject_from_jsonf ("{ 'device': %s, 'id': %s }" , + dev->info->name, dev->id ? :"" ); + qlist =qlist_new (); +parse_vmstate (vmsd, dev, qlist,qdict_get_int (qdict,"full" )); +qdict_put_obj (qobject_to_qdict (*ret_data),"fields" ,QOBJECT (qlist)); + +return 0 ; +}hw/qdev.h +2
-0@@ -177,6 +177,8 @@ void do_info_qtree(Monitor *mon);void do_info_qdm (Monitor *mon);int do_device_add (Monitor *mon,const QDict *qdict,QObject **ret_data);int do_device_del (Monitor *mon,const QDict *qdict,QObject **ret_data); +void device_user_print (Monitor *mon,const QObject *data); +int do_device_show (Monitor *mon,const QDict *qdict,QObject **ret_data);/*** qdev-properties.c ***/ qemu-monitor.hx +11
-0@@ -599,6 +599,22 @@ Remove device @var{id}. ETEXI { + .name = "device_show", + .args_type = "path:s,full:-f", + .params = "device [-f]", + .help = "show device state (specify -f for full buffer dumping)", + .user_print = device_user_print, + .mhandler.cmd_new = do_device_show, + }, + +STEXI +@item device_show @var{id} [@code{-f}] +
Quoted 10 lines
qemu-monitor.hx Remove device @var{id}. · line 620
Quoted 8 lines
qemu-monitor.hx +5
-0+Show state of device @var{id} in a human-readable form. Buffers are cut after +16 bytes unless a full dump is requested via @code{-f} +ETEXI + + { .name = "cpu", .args_type = "index:i", .params = "index",
Thanks for the comments!
Jan
PATCH 5/8 qmp: Teach basic capability negotiation to python example
In reply to Jan KiszkaAs sending "qmp_capabilities" on session start became mandatory, both python examples were broken.
--- QMP/qmp-shell | 1 + QMP/vm-info | 1 + 2 files changed, 2 insertions(+), 0 deletions(-)
QMP/qmp-shell +1 -0
@@ -42,6 +42,7 @@ def main(): qemu = qmp.QEMUMonitorProtocol(argv[1]) qemu.connect() + qemu.send("qmp_capabilities") print 'Connected!'
QMP/vm-info +1 -0
@@ -24,6 +24,7 @@ def main(): qemu = qmp.QEMUMonitorProtocol(argv[1]) qemu.connect() + qemu.send("qmp_capabilities") for cmd in [ 'version', 'hpet', 'kvm', 'status', 'uuid', 'balloon' ]: print cmd + ': ' + str(qemu.send('query-' + cmd))
Signature · 1 line
1.6.0.2
PATCH 6/8 qmp: Fix python helper /wrt long return strings
In reply to Jan KiszkaRemove the arbitrary limitation of 1024 characters per return string and read complete lines instead. Required for device_show.
--- QMP/qmp.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
QMP/qmp.py +1 -1
@@ -63,7 +63,7 @@ class QEMUMonitorProtocol: def __json_read(self): try: - return json.loads(self.sock.recv(1024)) + return json.loads(self.sock.makefile().readline()) except ValueError: return
Signature · 1 line
1.6.0.2
PATCH 7/8 Add QLIST_INSERT_TAIL
In reply to Jan KiszkaAs the QLIST has not tail pointer, this requires list walking. Still useful when lists are short or insertion time doesn't matter.
--- qemu-queue.h | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-)
qemu-queue.h +9 -0
@@ -122,6 +122,15 @@ struct { \ (elm)->field.le_prev = &(head)->lh_first; \ } while (/*CONSTCOND*/0) +#define QLIST_INSERT_TAIL(head, elm, field) do { \ + typeof((head)->lh_first) *qlist_lastptr = &(head)->lh_first; \ + while (*qlist_lastptr) \ + qlist_lastptr = &(*qlist_lastptr)->field.le_next; \ + (elm)->field.le_next = NULL; \ + *qlist_lastptr = (elm); \ + (elm)->field.le_prev = qlist_lastptr; \ +} while (/*CONSTCOND*/0) +#define QLIST_REMOVE (elm, field) do { \ if ((elm)->field.le_next != NULL) \ (elm)->field.le_next->field.le_prev = \
Signature · 1 line
1.6.0.2
[Qemu-devel] Re: [PATCH 7/8] Add QLIST_INSERT_TAIL
Quoted 2 lines · Jan Kiszka
As the QLIST has not tail pointer, this requires list walking. Still useful when lists are short or insertion time doesn't matter.
True, but why not switch to QTAIL instead?
Paolo
[Qemu-devel] Re: [PATCH 7/8] Add QLIST_INSERT_TAIL
Quoted 5 lines · Paolo Bonzini
Quoted 2 lines · Jan Kiszka
As the QLIST has not tail pointer, this requires list walking. Still useful when lists are short or insertion time doesn't matter.
True, but why not switch to QTAIL instead?
Would be more invasive, but I can do so if it's preferred.
Jan
PATCH 8/8 qdev: Add new devices/buses at the tail
In reply to Jan KiszkaCosmetic change to align the instance number assignment with bus ordering. The current ordering is a bit annoying when you dump the qtree or address devices via 'driver.instance'.
--- hw/qdev.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
hw/qdev.c +2 -2
@@ -91,7 +91,7 @@ static DeviceState *qdev_create_from_info(BusState *bus, DeviceInfo *info)qdev_prop_set_defaults (dev, dev->info->props);qdev_prop_set_defaults (dev, dev->parent_bus->info->props);qdev_prop_set_globals (dev); -QLIST_INSERT_HEAD (&bus->children, dev, sibling); +QLIST_INSERT_TAIL (&bus->children, dev, sibling);if (qdev_hotplug) {assert (bus->allow_hotplug); dev->hotplugged =1 ;
@@ -677,7 +677,7 @@ void qbus_create_inplace(BusState *bus, BusInfo *info,QLIST_INIT (&bus->children);if (parent) { -QLIST_INSERT_HEAD (&parent->child_bus, bus, sibling); +QLIST_INSERT_TAIL (&parent->child_bus, bus, sibling); parent->num_child_bus++; }
Signature · 1 line
1.6.0.2
Quoted 6 lines · Jan Kiszka
While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.
Wonderful! may even motivate some more qmp conversions.
Quoted 26 lines
This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:
(qemu)
device_show/i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.version_id: 00000002 config: a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00 ... irq_state: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 num_ports_vmstate: 02 ports[00]. ctrl: 0083 ports[01]. ctrl: 0080 cmd: 00c1 status: 0000 intr: 0000 frnum: 0077 fl_base_addr: 0fffc000 sof_timing: 40 status2: 00 frame_timer: 0000000000cb2bd0
Quoted 7 lines
Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...
Will be interesting to pass these annotations via qmp as well.
Quoted 3 lines
Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!
I'd like to see qmp command documentation for this.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Quoted 9 lines · Avi Kivity
Quoted 6 lines · Jan Kiszka
While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.
Wonderful! may even motivate some more qmp conversions.
Yeah, a few low handing fruits are already gone (the vmstate series I sent out recently). I've some stuff for hpet here, but I did not tough [IO]APIC yet as that should be based on top of Glauber's work for in-kernel irqchip support.
Quoted 35 lines
Quoted 27 lines
This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:
(qemu)
device_show/i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.version_id: 00000002 config: a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00 ... irq_state: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 num_ports_vmstate: 02 ports[00]. ctrl: 0083 ports[01]. ctrl: 0080 cmd: 00c1 status: 0000 intr: 0000 frnum: 0077 fl_base_addr: 0fffc000 sof_timing: 40 status2: 00 frame_timer: 0000000000cb2bd0Quoted 7 lines
Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...
Quoted 1 line
Will be interesting to pass these annotations via qmp as well.
For sure. We could e.g. convert QBuffers to QInts or QInts to QBools once we know their names and positions. This is where the harder work starts (that's why I proposed it for GSOC :) ).
Quoted 7 lines
Quoted 4 lines
Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!
I'd like to see qmp command documentation for this.
Will add it once the doc baseline is merged.
Jan
Signature · 2 lines
Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux
Quoted 7 lines · Jan Kiszka
Quoted 5 lines
I'd like to see qmp command documentation for this.
Will add it once the doc baseline is merged.
How about posting it as a patch on top of the documentation patch? That allows us to review it (which IMO should be a condition for accepting any patch that adds or changes a monitor command) while not requiring merging to be delayed until after the doc patches makes it in.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Quoted 32 lines · Jan Kiszka
While recently fixing the SCSI reset issues, I once again had the need for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.
This series now lays the ground for more sophisticated visulization. It adds the monitor command 'device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:
(qemu)
device_show/i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.version_id: 00000002 config: a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00 ... irq_state: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 num_ports_vmstate: 02 ports[00]. ctrl: 0083 ports[01]. ctrl: 0080 cmd: 00c1 status: 0000 intr: 0000 frnum: 0077 fl_base_addr: 0fffc000 sof_timing: 40 status2: 00 frame_timer: 0000000000cb2bd0
Quoted 9 lines
Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...
Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!
Very neat!
Regards,
Anthony Liguori
Quoted 37 lines
Jan Kiszka (8):
qdev: Allow device addressing via 'driver.instance' Add base64 encoder/decoder
Add QBuffer
monitor: Add basic device state visualization
qmp: Teach basic capability negotiation to python example
qmp: Fix python helper /wrt long return strings AddQLIST_INSERT_TAIL
qdev: Add new devices/buses at the tailMakefile | 3 +- Makefile.objs | 4 +- QMP/qmp-shell | 1 + QMP/qmp.py | 2 +- QMP/vm-info | 1 + base64.c | 202 +++++++++++++++++++++++++++++++++++ base64.h | 18 +++ check-qbuffer.c | 172 ++++++++++++++++++++++++++++++ configure | 2 +- hw/hw.h | 2 + hw/qdev.c | 314 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- hw/qdev.h | 2 + qbuffer.c | 116 ++++++++++++++++++++ qbuffer.h | 33 ++++++ qemu-monitor.hx | 16 +++ qemu-queue.h | 9 ++ qjson.c | 16 +++ qobject.h | 1 + 18 files changed, 902 insertions(+), 12 deletions(-)create mode 100644
base64.c
create mode 100644base64.h
create mode 100644check-qbuffer.c
create mode 100644qbuffer.c
create mode 100644qbuffer.h
Quoted 32 lines · Jan Kiszka
While recently fixing the SCSI reset issues, I once again had the need
for displaying the state of involved devices. So far the common approach is to attach gdb to qemu (or even inject some printf). But that time I hacked up a 30-minute patch to dump the vmstate of any (fully converted) qdev device.This series now lays the ground for more sophisticated visulization. It adds the monitor command '
device_show<qdev-path>', freezes the vmstate of the addressed device, sticks it into a QMP dict, and either transmit this via QMP or pretty-prints it on a monitor console. Some example:(qemu)
device_show/i440FX-pcihost/pci.0/piix3-usb-uhci
dev: piix3-usb-uhci, id ""
dev.version_id: 00000002 config: a0 7d d1 00 00 00 00 00 - b0 7e d1 00 00 00 00 00 ... irq_state: 00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00 num_ports_vmstate: 02 ports[00]. ctrl: 0083 ports[01]. ctrl: 0080 cmd: 00c1 status: 0000 intr: 0000 frnum: 0077 fl_base_addr: 0fffc000 sof_timing: 40 status2: 00 frame_timer: 0000000000cb2bd0
Quoted 8 lines
Basically, this is the level of support I recently saw in a demonstration of some commercial simulator as well. We are just lacking support for the yet unconverted devices. And I think we can even do better on the long term, e.g. by annotating state variables that contain flags, or by pretty-printing buffers like the PCI config space, or...
Let's give this a start, I bet it will be helpful while adding complex device models like AHCI or EHCI. Looking forward to feedback!
Nice! I think this confirms that there is no future for the qdev side of my yesterday's patch, this is much better.