Skip to contentSkip to conversation

virtio: migrate new properties

RFC PATCH Superseded Greg Kurz 8 participants 63 messages May 14, 2014 – Jun 12, 2014 3 messages not in the archive

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Greg Kurz

author
#0 lore raw

cover virtio: migrate new properties

The work on virtio endianness unveiled the need to add new properties to VirtIODevice and be able to migrate without breaking compatibility. There was a similar concern with the introduction of a VirtIODevice.broken property.

This patch set is a tentative to address the issue, following a suggestion from Alex:
- patches 1 to 7 adds optional subsections in the virtio migration stream
- patch 8 belongs to the virtio endianness serie, it is sent as an example

Thanks for your comments.

---

Greg Kurz (8):
virtio: add subsections to the migration stream
virtio-net: migrate subsections
virtio-blk: migrate subsections
virtio-scsi: migrate subsections
virtio-serial: migrate subsections
virtio-balloon: migrate subsections
virtio-rng: migrate subsections
virtio: add endian-ambivalent support to VirtIODevice

 exec.c                      |    8 ---
 hw/block/virtio-blk.c       |    4 +
 hw/char/virtio-serial-bus.c |   10 ++-
 hw/net/virtio-net.c         |    7 ++
 hw/scsi/virtio-scsi.c       |    3 +
 hw/virtio/virtio-balloon.c  |    4 +
 hw/virtio/virtio-pci.c      |   11 +---
 hw/virtio/virtio-rng.c      |   13 ++++
 hw/virtio/virtio.c          |  129 +++++++++++++++++++++++++++++++++++++++----
 include/exec/cpu-common.h   |    1 
 include/hw/virtio/virtio.h  |   17 ++++++
 11 files changed, 175 insertions(+), 32 deletions(-)
Signature · 1 line
Greg
author
#1 lore raw

RFC PATCH 1/8 virtio: add subsections to the migration stream

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

---
 hw/virtio/virtio.c         |   65 ++++++++++++++++++++++++++++++++++++++++++++
 include/hw/virtio/virtio.h |    4 +++
 2 files changed, 69 insertions(+)
hw/virtio/virtio.c +65 -0
index aeabf3a..f4eaa3f 100644
@@ -19,6 +19,7 @@
 #include "hw/virtio/virtio.h"
 #include "qemu/atomic.h"
 #include "hw/virtio/virtio-bus.h"
+#include "migration/migration.h"
 
 /*
  * The alignment to use between consumer and producer parts of vring.
@@ -833,6 +834,70 @@ void virtio_notify_config(VirtIODevice *vdev)
     virtio_notify_vector(vdev, vdev->config_vector);
 }
 
+static const struct VirtIOSubsectionDescStruct {
+    const char *name;
+    int version_id;
+    void (*save)(VirtIODevice *vdev, QEMUFile *f);
+    int (*load)(VirtIODevice *vdev, QEMUFile *f);
+} virtio_subsection[] = {
+    { .name = NULL }
+};
+
+void virtio_save_subsections(VirtIODevice *vdev, QEMUFile *f)
+{
+    int i;
+
+    for (i = 0; virtio_subsection[i].name; i++) {
+        const char *name = virtio_subsection[i].name;
+        uint8_t len = strlen(name);
+
+        qemu_put_byte(f, QEMU_VM_SUBSECTION);
+        qemu_put_byte(f, len);
+        qemu_put_buffer(f, (uint8_t *) name, len);
+        qemu_put_be32(f, virtio_subsection[i].version_id);
+        (*virtio_subsection[i].save)(vdev, f);
+    }
+}
+
+int virtio_load_subsections(VirtIODevice *vdev, QEMUFile *f)
+{
+    while (qemu_peek_byte(f, 0) == QEMU_VM_SUBSECTION) {
+        char idstr[256];
+        uint8_t len, size;
+        int i;
+
+        len = qemu_peek_byte(f, 1);
+        size = qemu_peek_buffer(f, (uint8_t *) idstr, len, 2);
+        if (size != len) {
+            break;
+        }
+
+        idstr[size] = 0;
+
+        for (i = 0; virtio_subsection[i].name; i++) {
+            if (strcmp(virtio_subsection[i].name, idstr) == 0) {
+                uint32_t version_id;
+                int ret;
+
+                qemu_file_skip(f, 1); /* subsection */
+                qemu_file_skip(f, 1); /* len */
+                qemu_file_skip(f, len); /* idstr */
+
+                version_id = qemu_get_be32(f);
+                if (version_id > virtio_subsection[i].version_id) {
+                    return -EINVAL;
+                }
+                ret = (*virtio_subsection[i].load)(vdev, f);
+                if (ret) {
+                    return ret;
+                }
+            }
+        }
+    }
+
+    return 0;
+}
+
 void virtio_save(VirtIODevice *vdev, QEMUFile *f)
 {
     BusState *qbus = qdev_get_parent_bus(DEVICE(vdev));
include/hw/virtio/virtio.h +4 -0
index 3e54e90..82f852f 100644
@@ -186,6 +186,10 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f);
 
 int virtio_load(VirtIODevice *vdev, QEMUFile *f);
 
+void virtio_save_subsections(VirtIODevice *vdev, QEMUFile *f);
+
+int virtio_load_subsections(VirtIODevice *vdev, QEMUFile *f);
+
 void virtio_notify_config(VirtIODevice *vdev);
 
 void virtio_queue_set_notification(VirtQueue *vq, int enable);
#2 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 8 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Quoted 7 lines

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Amit

#3 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 21 lines · Amit Shah
Quoted 8 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Quoted 7 lines

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 8 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Amit

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

Signature · 1 line
MST
#4 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

BTW Greg, do you plan on working on vmstate for virtio?

Quoted 15 lines
Quoted 15 lines
Quoted 7 lines

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Quoted 7 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Amit

author
#5 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Quoted 2 lines

BTW Greg, do you plan on working on vmstate for virtio?

Yes.

Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
Quoted 7 lines

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

Quoted 2 lines

Amit

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#6 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 10 lines
Quoted 8 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines · Amit Shah
Quoted 5 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.

Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

Would touch on Stefan's alias properties for anything but virtio-mmio.

Regards,
Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#7 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 14 lines · Andreas Färber

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

So yes it's a useful feature, and no not supporting it does not help anyway.

Quoted 42 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.

Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

Would touch on Stefan's alias properties for anything but virtio-mmio.

Regards,
Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#8 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

It seems you misunderstand. I was not saying it's not useful.

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#9 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 46 lines · Andreas Färber

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

Quoted 7 lines

It seems you misunderstand. I was not saying it's not useful.

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Andreas

Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.

Quoted 3 lines
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#10 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:

Quoted 49 lines · Andreas Färber
Quoted 48 lines

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

It seems you misunderstand. I was not saying it's not useful.

Quoted 7 lines
Quoted 3 lines

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.

OK, but upstream always sends the latest version today. So isn't that just two ifs that you would need to add in save and load functions added here for the downstream? x86_64 is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#11 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 58 lines · Andreas Färber

Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:

Quoted 49 lines · Andreas Färber
Quoted 48 lines

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

It seems you misunderstand. I was not saying it's not useful.

Quoted 7 lines
Quoted 3 lines

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.

Quoted 8 lines

OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream? x86_64 is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?

Andreas

I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.

With *any* change, there's always the temptation to say "oh but old behaviour was too buggy we should just ignore it". Seems to make sense in each case but does not scale, you end up with breakages in all version without exception. So this should be a very high bar to overcome.

Quoted 3 lines
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#12 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Am 15.05.2014 12:16, schrieb Michael S. Tsirkin:

Quoted 66 lines · Andreas Färber
Quoted 58 lines

Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:

Quoted 49 lines · Andreas Färber
Quoted 48 lines

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

It seems you misunderstand. I was not saying it's not useful.

Quoted 7 lines
Quoted 3 lines

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.

Quoted 6 lines

OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream? x86_64 is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?

Quoted 4 lines

I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.

Whatever the version_id fields are for, upstream QEMU *always* saves the newest version_id format that it knows. There is no mechanism that I'm aware of in upstream QEMU for migrating device fields dependent on -M. So a device in QEMU only migrates exactly like an old QEMU did if neither fields nor subsections were added.

Subsections are usually migrated based on whether the state differs from the default state (didn't check whether the final patch does this correctly here, but doesn't matter for 1/8 concept). So for x86 the subsection should *never* get written and thus not be a problem for you. For ppc64 it should not get written either, unless you care about migration from SLES12 or upstream ppc64le to RHEL ppc64.
As I understood the IRC discussion Alex and me had with Greg about this, this is copying the exact code VMState uses to write its subsections, so there would be no forward migration problems at all once we convert to VMState and VMStateSubsections. The only question remaining is how does your downstream react when it encounters a subsection marker for subsections it doesn't know - which is not something upstream can solve for you unless you contribute backwards migration support upstream.

Don't forget that Greg is introducing subsection support *because of* your backwards migration wish, so telling him not to add subsections because of backwards migration is really weird! Either subsections work with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#13 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 72 lines · Andreas Färber

Am 15.05.2014 12:16, schrieb Michael S. Tsirkin:

Quoted 66 lines · Andreas Färber
Quoted 58 lines

Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:

Quoted 49 lines · Andreas Färber
Quoted 48 lines

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

It seems you misunderstand. I was not saying it's not useful.

Quoted 7 lines
Quoted 3 lines

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.

Quoted 6 lines

OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream? x86_64 is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?

Quoted 4 lines

I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.

Quoted 5 lines

Whatever the version_id fields are for, upstream QEMU *always* saves the newest version_id format that it knows. There is no mechanism that I'm aware of in upstream QEMU for migrating device fields dependent on -M. So a device in QEMU only migrates exactly like an old QEMU did if neither fields nor subsections were added.

Ah you are talking about version ids.
Yes, they can not support cross-version migration.
For this reason these should only be incremented as the last resort, normally new fields (possibly subsections) should be added, with field_exists callback checking for compatibility mode.

Quoted 3 lines

Subsections are usually migrated based on whether the state differs from the default state (didn't check whether the final patch does this correctly here, but doesn't matter for 1/8 concept).

Yes, this would be fine, but it's not what's implemented IIUC.

Quoted 12 lines

So for x86 the
subsection should *never* get written and thus not be a problem for you. For ppc64 it should not get written either, unless you care about migration from SLES12 or upstream ppc64le to RHEL ppc64.
As I understood the IRC discussion Alex and me had with Greg about this, this is copying the exact code VMState uses to write its subsections, so there would be no forward migration problems at all once we convert to VMState and VMStateSubsections. The only question remaining is how does your downstream react when it encounters a subsection marker for subsections it doesn't know - which is not something upstream can solve for you unless you contribute backwards migration support upstream.

Quoted 3 lines

Don't forget that Greg is introducing subsection support *because of* your backwards migration wish, so telling him not to add subsections because of backwards migration is really weird!

I don't really care whether it's a subsection or just plain qemu_put_byte, but I'm not saying not to use subsection either.
I am merely saying this should have "exists" callback checking for compability mode for old machine types.

Quoted 9 lines

Either subsections work
with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.

Andreas

I only mentioned downstream because it's a requirement coming from users most of whom never interact with upstream.
There shouldn't be any difference and people building clusters using upstream qemu will have same needs: clusters can't be completely brought down for version upgrades, so need to be able to run a mix of versions, and be able to migrate in any direction while in the process.

Quoted 3 lines
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#14 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 1 line · Michael S. Tsirkin

On Thu, May 15, 2014 at 02:00:15PM +0200, Andreas Färber wrote:

[...]

Quoted 10 lines
Quoted 9 lines

Either subsections work
with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.

Andreas

Quoted 7 lines

I only mentioned downstream because it's a requirement coming from users most of whom never interact with upstream.
There shouldn't be any difference and people building clusters using upstream qemu will have same needs: clusters can't be completely brought down for version upgrades, so need to be able to run a mix of versions, and be able to migrate in any direction while in the process.

Okay, I'll have world peace, a cluster, and a pony, please.

Seriously, no matter how much your cluster management software wants to migrate backwards, it's not going to work reliably, except in very narrow cases, at least not today.

If you want to migrate freely, stick to an ultra-stable (downstream) branch of QEMU. When that branch EOLs, scrap your cluster and start over.

If you want to upgrade QEMU in more interesting ways, keep track of what your cluster nodes are running, and migrate only forward.

author
#15 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Andreas Färber
Quoted 78 lines · On Thu, 15 May 2014 14:00:15 +0200 Andreas Färber <[email protected]> wrote:

Am 15.05.2014 12:16, schrieb Michael S. Tsirkin:

Quoted 66 lines · Andreas Färber
Quoted 58 lines

Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:

Quoted 49 lines · Andreas Färber
Quoted 48 lines

Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:

Quoted 42 lines · Andreas Färber
Quoted 14 lines

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

It seems you misunderstand. I was not saying it's not useful.

Quoted 7 lines
Quoted 3 lines

My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?

Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.

Quoted 6 lines

OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream? x86_64 is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?

Quoted 4 lines

I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.

Whatever the version_id fields are for, upstream QEMU *always* saves the newest version_id format that it knows. There is no mechanism that I'm aware of in upstream QEMU for migrating device fields dependent on -M. So a device in QEMU only migrates exactly like an old QEMU did if neither fields nor subsections were added.

Quoted 3 lines

Subsections are usually migrated based on whether the state differs from the default state (didn't check whether the final patch does this correctly here, but doesn't matter for 1/8 concept). So for x86 the

For this first round, subsections are sent unconditionally. I will change that.

Quoted 23 lines

subsection should *never* get written and thus not be a problem for you. For ppc64 it should not get written either, unless you care about migration from SLES12 or upstream ppc64le to RHEL ppc64.
As I understood the IRC discussion Alex and me had with Greg about this, this is copying the exact code VMState uses to write its subsections, so there would be no forward migration problems at all once we convert to VMState and VMStateSubsections. The only question remaining is how does your downstream react when it encounters a subsection marker for subsections it doesn't know - which is not something upstream can solve for you unless you contribute backwards migration support upstream.

Don't forget that Greg is introducing subsection support *because of* your backwards migration wish, so telling him not to add subsections because of backwards migration is really weird! Either subsections work with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.

Andreas

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#16 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Michael S. Tsirkin
Quoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Andreas Färber

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 6 lines
Quoted 3 lines

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly.

Can I have world peace and a pony with that?

Given the current state of things, attempting to support backward migration is trying to run before you can walk. We need to put migration on a more solid footing first.

The migration format is crap, and needs to be replaced.

Reasoning on migration compatibility is entirely manual.

Systematic testing of migration compatibility is done downstream.

Fortunately, there's progress being made on all of the above. Let's not sabotage it by biting off yet another mouthful.

Quoted 9 lines

So some downstreams support this
and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Non sequitur.

Quoted 4 lines

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :)

However, it is.

Quoted 4 lines

Once you do the hard part, breaking migration because of the
trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Bold claim; citation needed.

Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

Nobody denies reliable backward migration would be useful. However, attempting to do every useful feature at once just because they're all useful is foolish.

Treating backward migration as strictly secondary concern while we're up to the ass in other alligators *can* help, by letting us focus on the said other alligators.

I'm not opposed to coding things in ways that help backward migration. Speaking of "support", however, is clearly premature and misleading.

#17 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 60 lines · Markus Armbruster
Quoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Andreas Färber

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 6 lines
Quoted 3 lines

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly.

Can I have world peace and a pony with that?

Given the current state of things, attempting to support backward migration is trying to run before you can walk. We need to put migration on a more solid footing first.

The migration format is crap, and needs to be replaced.

Reasoning on migration compatibility is entirely manual.

Systematic testing of migration compatibility is done downstream.

Fortunately, there's progress being made on all of the above. Let's not sabotage it by biting off yet another mouthful.

Quoted 9 lines

So some downstreams support this
and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Non sequitur.

Quoted 4 lines

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :)

Quoted 8 lines

However, it is.

Quoted 4 lines

Once you do the hard part, breaking migration because of the
trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Bold claim; citation needed.

You are asking for examples of ugly assymetry?
It's easy: grep for .load_state_old.
You have here a bunch of functions loading format that qemu
can no longer produce, with any set of flags.
The only way to make them run is to install two qemu versions side by side, save from old one and load in the new one.
What, would you guess, is the chance that they actually work?

I'm going to send a patch removing all this stuff, it's effectively dead code, but this is just one, biggest example.

Quoted 7 lines
Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

Nobody denies reliable backward migration would be useful. However, attempting to do every useful feature at once just because they're all useful is foolish.

Quoted 6 lines

Treating backward migration as strictly secondary concern while we're up to the ass in other alligators *can* help, by letting us focus on the said other alligators.

I'm not opposed to coding things in ways that help backward migration. Speaking of "support", however, is clearly premature and misleading.

I agree it's a secondary concern upstream and your comments really apply to migration generally. We can't claim that it's properly supported by the upstream QEMU.

I am merely asking that we don't break cross-version migration intentionally. I and others also try fix it when we notice it's broken. In particular I'm not asking that submitters test it.

Signature · 1 line
MST
author
#18 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 97 lines · On Thu, 15 May 2014 15:58:16 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 60 lines · Markus Armbruster
Quoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Andreas Färber

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 6 lines
Quoted 3 lines

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Backwards and forwards migration need to be supported, customers told us repeatedly.

Can I have world peace and a pony with that?

Given the current state of things, attempting to support backward migration is trying to run before you can walk. We need to put migration on a more solid footing first.

The migration format is crap, and needs to be replaced.

Reasoning on migration compatibility is entirely manual.

Systematic testing of migration compatibility is done downstream.

Fortunately, there's progress being made on all of the above. Let's not sabotage it by biting off yet another mouthful.

Quoted 9 lines

So some downstreams support this
and not supporting it upstream just means downstreams need to do their own thing.

As importantly, ping-pong migration is the only
reliable way to stress migration.

So if we want to test cross-version we need it to work both way.

Non sequitur.

Quoted 4 lines

Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :)

Quoted 8 lines

However, it is.

Quoted 4 lines

Once you do the hard part, breaking migration because of the
trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.

Bold claim; citation needed.

You are asking for examples of ugly assymetry?
It's easy: grep for .load_state_old.
You have here a bunch of functions loading format that qemu
can no longer produce, with any set of flags.
The only way to make them run is to install two qemu versions side by side, save from old one and load in the new one.
What, would you guess, is the chance that they actually work?

I'm going to send a patch removing all this stuff, it's effectively dead code, but this is just one, biggest example.

Quoted 7 lines
Quoted 2 lines

So yes it's a useful feature, and no not supporting it does not help anyway.

Nobody denies reliable backward migration would be useful. However, attempting to do every useful feature at once just because they're all useful is foolish.

Quoted 6 lines

Treating backward migration as strictly secondary concern while we're up to the ass in other alligators *can* help, by letting us focus on the said other alligators.

I'm not opposed to coding things in ways that help backward migration. Speaking of "support", however, is clearly premature and misleading.

Quoted 8 lines

I agree it's a secondary concern upstream and your comments really apply to migration generally. We can't claim that it's properly supported by the upstream QEMU.

I am merely asking that we don't break cross-version migration intentionally. I and others also try fix it when we notice it's broken. In particular I'm not asking that submitters test it.

Michael,

I fully understand your concern and I will take some time to study backward migration. This being said, the task is not that simple and I will need several rounds to have the job done...

Cheers.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
author
#19 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Andreas Färber
Quoted 14 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.

That makes a lot of sense.

Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

Quoted 7 lines

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

IIRW the "broken" state was proposed as a per-device property...

Fam,

Do you have plans about the "broken" property ? Is it still needed ?

Quoted 2 lines

Would touch on Stefan's alias properties for anything but virtio-mmio.

OMG... maybe I should hold on then.

Quoted 3 lines

Regards,
Andreas

Thanks !

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#20 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 69 lines · Greg Kurz
Quoted 14 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.

That makes a lot of sense.

Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

Quoted 7 lines

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

IIRW the "broken" state was proposed as a per-device property...

Quoted 8 lines

Fam,

Do you have plans about the "broken" property ? Is it still needed ?

Quoted 2 lines

Would touch on Stefan's alias properties for anything but virtio-mmio.

OMG... maybe I should hold on then.

No need to wait imho.
Can this be made even simpler - call this stuff
from virtio_save/virtio_load?

Why not?

Quoted 14 lines
Quoted 3 lines

Regards,
Andreas

Thanks !

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
author
#21 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 74 lines · On Thu, 15 May 2014 13:12:12 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 73 lines · Greg Kurz
Quoted 14 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.

That makes a lot of sense.

Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

Quoted 7 lines

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

IIRW the "broken" state was proposed as a per-device property...

Fam,

Do you have plans about the "broken" property ? Is it still needed ?

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

Would touch on Stefan's alias properties for anything but virtio-mmio.

OMG... maybe I should hold on then.

No need to wait imho.
Can this be made even simpler - call this stuff
from virtio_save/virtio_load?

Andreas already suggested this inversion of control.

Quoted 2 lines

Why not?

No reason indeed. I'll rewrite the code that way ! :)

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

Regards,
Andreas

Thanks !

Quoted 9 lines
Quoted 8 lines
Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.

Thnaks !

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
author
#22 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Greg Kurz
Quoted 20 lines · On Thu, 15 May 2014 12:08:26 +0200 Greg Kurz <[email protected]> wrote:
Quoted 18 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:

Am 15.05.2014 09:04, schrieb Greg Kurz:

Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 9 lines
Quoted 4 lines

Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?

Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?

Heh just got the clarification thanks to Michael's mail... Sorry for the confusion and noise. :P

Quoted 55 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.

That makes a lot of sense.

Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

Quoted 7 lines

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

IIRW the "broken" state was proposed as a per-device property...

Fam,

Do you have plans about the "broken" property ? Is it still needed ?

Quoted 2 lines

Would touch on Stefan's alias properties for anything but virtio-mmio.

OMG... maybe I should hold on then.

Quoted 3 lines

Regards,
Andreas

Thanks !

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#23 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Greg Kurz
Quoted 13 lines · Greg Kurz
Quoted 5 lines
Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

Quoted 7 lines

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

Quoted 1 line

IIRW the "broken" state was proposed as a per-device property...

Yes.

Quoted 4 lines

Fam,

Do you have plans about the "broken" property ? Is it still needed ?

It is on my todo list. We will need it. I hope the next revision of virtio specification has a proper device status bit that the "broken" property can utilize.

Thanks,
Fam

#24 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Am 16.05.2014 11:14, schrieb Fam Zheng:

Quoted 14 lines · Greg Kurz
Quoted 13 lines
Quoted 12 lines
Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

Quoted 3 lines
Quoted 1 line

IIRW the "broken" state was proposed as a per-device property...

Yes.

Sure, and that makes sense, but we do have a 1:1 relation of bus/device, or does virtio-mmio support more? If device doesn't work for some reason, we could (mis)use the bus as fallback then.

Cheers,
Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#25 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 15 lines · Andreas Färber

Am 16.05.2014 11:14, schrieb Fam Zheng:

Quoted 14 lines · Greg Kurz
Quoted 13 lines
Quoted 12 lines
Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

IIRW the "broken" state was proposed as a per-device property...

Yes.

Sure, and that makes sense, but we do have a 1:1 relation of bus/device, or does virtio-mmio support more? If device doesn't work for some reason, we could (mis)use the bus as fallback then.

FWIW, I just realized that "broken" may be loaded from device status bit in the future, so we don't need to migrate the field separately:

https://tools.oasis-open.org/issues/browse/VIRTIO-98

Fam

author
#26 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 25 lines · On Fri, 16 May 2014 17:40:00 +0800 Fam Zheng <[email protected]> wrote:
Quoted 15 lines · Andreas Färber

Am 16.05.2014 11:14, schrieb Fam Zheng:

Quoted 14 lines · Greg Kurz
Quoted 13 lines
Quoted 12 lines
Quoted 4 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.

There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

IIRW the "broken" state was proposed as a per-device property...

Yes.

Sure, and that makes sense, but we do have a 1:1 relation of bus/device, or does virtio-mmio support more? If device doesn't work for some reason, we could (mis)use the bus as fallback then.

Quoted 7 lines

FWIW, I just realized that "broken" may be loaded from device status bit in the future, so we don't need to migrate the field separately:

https://tools.oasis-open.org/issues/browse/VIRTIO-98

Fam

Oh, if you are targetting virtio 1.0... all the virtio endian stuff I am working on is for legacy virtio. Unless you plan to do something for legacy, I guess our objectives differ.

Cheers.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#27 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Greg Kurz
Quoted 47 lines · Greg Kurz
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Quoted 2 lines

BTW Greg, do you plan on working on vmstate for virtio?

Yes.

Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
Quoted 7 lines

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

Quoted 9 lines
Quoted 3 lines

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Quoted 3 lines

The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks...

I see.
I think it's all not a big deal.
People here suggested many ways to deal with it, but IMHO the only thing that does matter is the functionality.

Functionality-wise I think the only two things that were mentioned were
- decent chance that migration to a wrong machine
version fails gracefully
- migrate in a compatible way with correct legacy machine version

Quoted 16 lines

I need to study how the virtio-bus gets
migrated to ensure the endian section is streamed before the devices.

Quoted 2 lines

Amit

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#28 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Amit Shah
Quoted 36 lines · Amit Shah
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

BTW Greg, do you plan on working on vmstate for virtio?

Quoted 15 lines
Quoted 15 lines
Quoted 7 lines

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Quoted 7 lines
Quoted 3 lines

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.

Quoted 6 lines

No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.

Amit

I don't mind but the gain isn't very big here.

Signature · 1 line
MST
author
#29 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Amit Shah
Quoted 14 lines · On Thu, 15 May 2014 11:34:25 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Quoted 7 lines
Quoted 2 lines

Unknown savevm section type 5
Error -22 while loading VM state

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Hmmm... you mean we support migration from a newer QEMU to an older one ?

Quoted 7 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

That would be nice if possible. I will have a closer look.

Quoted 2 lines

Amit

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#30 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

Quoted 20 lines · Greg Kurz
Quoted 17 lines · On Thu, 15 May 2014 11:34:25 +0530 Amit Shah <[email protected]> wrote:
Quoted 15 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 6 lines
Quoted 4 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Hmmm... you mean we support migration from a newer QEMU to an older one ?

Well, not really "support", since many things don't work anyway, but let's make that easier on downstreams that may want to.

Quoted 9 lines
Quoted 7 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

That would be nice if possible. I will have a closer look.

Thanks!

Amit

#31 lore raw

Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream

In reply to Greg Kurz
Quoted 20 lines · Greg Kurz
Quoted 17 lines · On Thu, 15 May 2014 11:34:25 +0530 Amit Shah <[email protected]> wrote:
Quoted 15 lines · Greg Kurz

There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.

Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:

Unknown savevm section type 5
Error -22 while loading VM state

Quoted 6 lines
Quoted 4 lines

Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.

Hmmm... you mean we support migration from a newer QEMU to an older one ?

In theory yes, that's why we have the -M switch. This area isn't well tested unfortunately, but there are downstreams that test and productize it.

Quoted 23 lines
Quoted 7 lines
Quoted 2 lines

All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.

Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.

That would be nice if possible. I will have a closer look.

Quoted 2 lines

Amit

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
author
#32 lore raw

RFC PATCH 2/8 virtio-net: migrate subsections

In reply to Greg Kurz
---
 hw/net/virtio-net.c |    7 +++++++
 1 file changed, 7 insertions(+)
hw/net/virtio-net.c +7 -0
index fd23c46..4004d02 100644
@@ -1300,6 +1300,8 @@ static void virtio_net_save(QEMUFile *f, void *opaque)
     if ((1 << VIRTIO_NET_F_CTRL_GUEST_OFFLOADS) & vdev->guest_features) {
         qemu_put_be64(f, n->curr_guest_offloads);
     }
+
+    virtio_save_subsections(vdev, f);
 }
 
 static int virtio_net_load(QEMUFile *f, void *opaque, int version_id)
@@ -1396,6 +1398,11 @@ static int virtio_net_load(QEMUFile *f, void *opaque, int version_id)
         n->curr_guest_offloads = virtio_net_supported_guest_offloads(n);
     }
 
+    ret = virtio_load_subsections(vdev, f);
+    if (ret) {
+        return ret;
+    }
+
     if (peer_has_vnet_hdr(n)) {
         virtio_net_apply_guest_offloads(n);
     }
author
#33 lore raw

RFC PATCH 3/8 virtio-blk: migrate subsections

In reply to Greg Kurz
---
 hw/block/virtio-blk.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hw/block/virtio-blk.c +3 -1
index 8a568e5..5a0d224 100644
@@ -600,6 +600,8 @@ static void virtio_blk_save(QEMUFile *f, void *opaque)
         req = req->next;
     }
     qemu_put_sbyte(f, 0);
+
+    virtio_save_subsections(vdev, f);
 }
 
 static int virtio_blk_load(QEMUFile *f, void *opaque, int version_id)
@@ -628,7 +630,7 @@ static int virtio_blk_load(QEMUFile *f, void *opaque, int version_id)
             req->elem.out_num, 0);
     }
 
-    return 0;
+    return virtio_load_subsections(vdev, f);
 }
 
 static void virtio_blk_resize(void *opaque)
author
#34 lore raw

RFC PATCH 4/8 virtio-scsi: migrate subsections

In reply to Greg Kurz
---
 hw/scsi/virtio-scsi.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/scsi/virtio-scsi.c +2 -1
index b0d7517..989e4ed 100644
@@ -471,6 +471,7 @@ static void virtio_scsi_save(QEMUFile *f, void *opaque)
 {
     VirtIODevice *vdev = VIRTIO_DEVICE(opaque);
     virtio_save(vdev, f);
+    virtio_save_subsections(vdev, f);
 }
 
 static int virtio_scsi_load(QEMUFile *f, void *opaque, int version_id)
@@ -482,7 +483,7 @@ static int virtio_scsi_load(QEMUFile *f, void *opaque, int version_id)
     if (ret) {
         return ret;
     }
-    return 0;
+    return virtio_load_subsections(vdev, f);
 }
 
 static void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
author
#35 lore raw

RFC PATCH 5/8 virtio-serial: migrate subsections

In reply to Greg Kurz
---
 hw/char/virtio-serial-bus.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
hw/char/virtio-serial-bus.c +7 -3
index 2b647b6..98f32fd 100644
@@ -522,12 +522,13 @@ static void vser_reset(VirtIODevice *vdev)
 static void virtio_serial_save(QEMUFile *f, void *opaque)
 {
     VirtIOSerial *s = VIRTIO_SERIAL(opaque);
+    VirtIODevice *vdev = VIRTIO_DEVICE(s);
     VirtIOSerialPort *port;
     uint32_t nr_active_ports;
     unsigned int i, max_nr_ports;
 
     /* The virtio device */
-    virtio_save(VIRTIO_DEVICE(s), f);
+    virtio_save(vdev, f);
 
     /* The config space */
     qemu_put_be16s(f, &s->config.cols);
@@ -573,6 +574,8 @@ static void virtio_serial_save(QEMUFile *f, void *opaque)
                             sizeof(port->elem));
         }
     }
+
+    virtio_save_subsections(vdev, f);
 }
 
 static void virtio_serial_post_load_timer_cb(void *opaque)
@@ -667,6 +670,7 @@ static int fetch_active_ports_list(QEMUFile *f, int version_id,
 static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)
 {
     VirtIOSerial *s = VIRTIO_SERIAL(opaque);
+    VirtIODevice *vdev = VIRTIO_DEVICE(s);
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
@@ -676,7 +680,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)
     }
 
     /* The virtio device */
-    ret = virtio_load(VIRTIO_DEVICE(s), f);
+    ret = virtio_load(vdev, f);
     if (ret) {
         return ret;
     }
@@ -716,7 +720,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)
             return ret;
         }
     }
-    return 0;
+    return virtio_load_subsections(vdev, f);
 }
 
 static void virtser_bus_dev_print(Monitor *mon, DeviceState *qdev, int indent);
author
#36 lore raw

RFC PATCH 6/8 virtio-balloon: migrate subsections

In reply to Greg Kurz
---
 hw/virtio/virtio-balloon.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
hw/virtio/virtio-balloon.c +3 -1
index a470a0b..41f7ae1 100644
@@ -316,6 +316,8 @@ static void virtio_balloon_save(QEMUFile *f, void *opaque)
 
     qemu_put_be32(f, s->num_pages);
     qemu_put_be32(f, s->actual);
+
+    virtio_save_subsections(vdev, f);
 }
 
 static int virtio_balloon_load(QEMUFile *f, void *opaque, int version_id)
@@ -334,7 +336,7 @@ static int virtio_balloon_load(QEMUFile *f, void *opaque, int version_id)
 
     s->num_pages = qemu_get_be32(f);
     s->actual = qemu_get_be32(f);
-    return 0;
+    return virtio_load_subsections(vdev, f);
 }
 
 static void virtio_balloon_device_realize(DeviceState *dev, Error **errp)
author
#37 lore raw

RFC PATCH 7/8 virtio-rng: migrate subsections

In reply to Greg Kurz

While we are here, we also check virtio_load() return value.

---
 hw/virtio/virtio-rng.c |   13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
hw/virtio/virtio-rng.c +12 -1
index b6ab361..0371103 100644
@@ -103,17 +103,28 @@ static void virtio_rng_save(QEMUFile *f, void *opaque)
     VirtIODevice *vdev = opaque;
 
     virtio_save(vdev, f);
+    virtio_save_subsections(vdev, f);
 }
 
 static int virtio_rng_load(QEMUFile *f, void *opaque, int version_id)
 {
     VirtIORNG *vrng = opaque;
     VirtIODevice *vdev = VIRTIO_DEVICE(vrng);
+    int ret;
 
     if (version_id != 1) {
         return -EINVAL;
     }
-    virtio_load(vdev, f);
+
+    ret = virtio_load(vdev, f);
+    if (ret) {
+        return ret;
+    }
+
+    ret = virtio_load_subsections(vdev, f);
+    if (ret) {
+        return ret;
+    }
 
     /* We may have an element ready but couldn't process it due to a quota
      * limit.  Make sure to try again after live migration when the quota may
author
#38 lore raw

RFC PATCH 8/8 virtio: add endian-ambivalent support to VirtIODevice

In reply to Greg Kurz

Some CPU families can dynamically change their endianness. This means we can have little endian ppc or big endian arm guests for example. This has an impact on legacy virtio data structures since they are target endian. We hence introduce a new property to track the endianness of each virtio device. It is reasonnably assumed that endianness won't change while the device is in use : we hence capture the device endianness when it gets reset.

Of course this property must be part of the migration stream as most of the virtio code will depend on it. It is hence migrated in a subsection to preserve compatibility with older migration streams. The tricky part is that the endianness gets known quite late and we must ensure no access is made to virtio data before that. It is for example invalid to call vring_avail_idx() as does the actual migration code: the VQ indexes sanity checks had to be moved from virtio_load() to virtio_load_subsections() because of that.

In fact, we have two conditions where the endianness property is stale:
- from initialization time (virtio_init) to first reset time (virtio_reset)
- from incoming migration time (virtio_load) to the time the property is restored (virtio_load_device_endian)
We enforce some paranoia by making the endianness a 3-value enum so that we can poison it when it should not be used.

---
 exec.c                     |    8 +-----
 hw/virtio/virtio-pci.c     |   11 +++-----
 hw/virtio/virtio.c         |   64 ++++++++++++++++++++++++++++++++++++--------
 include/exec/cpu-common.h  |    1 +
 include/hw/virtio/virtio.h |   13 +++++++++
 5 files changed, 72 insertions(+), 25 deletions(-)
exec.c +1 -7
index 91513c6..4e83588 100644
@@ -2740,13 +2740,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
 #endif
 
 #if !defined(CONFIG_USER_ONLY)
-
-/*
- * A helper function for the _utterly broken_ virtio device model to find out if
- * it's running on a big endian machine. Don't do this at home kids!
- */
-bool virtio_is_big_endian(void);
-bool virtio_is_big_endian(void)
+bool target_words_bigendian(void)
 {
 #if defined(TARGET_WORDS_BIGENDIAN)
     return true;
hw/virtio/virtio-pci.c +4 -7
index ce97514..2ffceb8 100644
@@ -89,9 +89,6 @@
 /* Flags track per-device state like workarounds for quirks in older guests. */
 #define VIRTIO_PCI_FLAG_BUS_MASTER_BUG  (1 << 0)
 
-/* HACK for virtio to determine if it's running a big endian guest */
-bool virtio_is_big_endian(void);
-
 static void virtio_pci_bus_new(VirtioBusState *bus, size_t bus_size,
                                VirtIOPCIProxy *dev);
 
@@ -409,13 +406,13 @@ static uint64_t virtio_pci_config_read(void *opaque, hwaddr addr,
         break;
     case 2:
         val = virtio_config_readw(vdev, addr);
-        if (virtio_is_big_endian()) {
+        if (virtio_is_big_endian(vdev)) {
             val = bswap16(val);
         }
         break;
     case 4:
         val = virtio_config_readl(vdev, addr);
-        if (virtio_is_big_endian()) {
+        if (virtio_is_big_endian(vdev)) {
             val = bswap32(val);
         }
         break;
@@ -443,13 +440,13 @@ static void virtio_pci_config_write(void *opaque, hwaddr addr,
         virtio_config_writeb(vdev, addr, val);
         break;
     case 2:
-        if (virtio_is_big_endian()) {
+        if (virtio_is_big_endian(vdev)) {
             val = bswap16(val);
         }
         virtio_config_writew(vdev, addr, val);
         break;
     case 4:
-        if (virtio_is_big_endian()) {
+        if (virtio_is_big_endian(vdev)) {
             val = bswap32(val);
         }
         virtio_config_writel(vdev, addr, val);
hw/virtio/virtio.c +53 -11

113 lines not on this page · raw message

include/exec/cpu-common.h +1 -0
index a21b65a..eb798c1 100644
@@ -122,4 +122,5 @@ void qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);
 
 #endif
 
+bool target_words_bigendian(void);
 #endif /* !CPU_COMMON_H */
include/hw/virtio/virtio.h +13 -0
index 82f852f..0012470 100644
@@ -104,6 +104,12 @@ typedef struct VirtQueueElement
 #define VIRTIO_DEVICE(obj) \
         OBJECT_CHECK(VirtIODevice, (obj), TYPE_VIRTIO_DEVICE)
 
+enum virtio_device_endian {
+    VIRTIO_DEVICE_ENDIAN_UNKNOWN,
+    VIRTIO_DEVICE_ENDIAN_LITTLE,
+    VIRTIO_DEVICE_ENDIAN_BIG,
+};
+
 struct VirtIODevice
 {
     DeviceState parent_obj;
@@ -121,6 +127,7 @@ struct VirtIODevice
     bool vm_running;
     VMChangeStateEntry *vmstate;
     char *bus_name;
+    enum virtio_device_endian device_endian;
 };
 
 typedef struct VirtioDeviceClass {
@@ -257,4 +264,10 @@ void virtio_queue_set_host_notifier_fd_handler(VirtQueue *vq, bool assign,
                                                bool set_handler);
 void virtio_queue_notify_vq(VirtQueue *vq);
 void virtio_irq(VirtQueue *vq);
+
+static inline bool virtio_is_big_endian(VirtIODevice *vdev)
+{
+    assert(vdev->device_endian != VIRTIO_DEVICE_ENDIAN_UNKNOWN);
+    return vdev->device_endian == VIRTIO_DEVICE_ENDIAN_BIG;
+}
 #endif

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#42 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

            nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                       ^^^^^^^^^^^
            /* Check it isn't doing very strange things with descriptor numbers. */
            if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

    /* The config space */
    qemu_get_be16s(f, &s->config.cols);
    qemu_get_be16s(f, &s->config.rows);

    qemu_get_be32s(f, &max_nr_ports);
    tswap32s(&max_nr_ports);
     ^^^^^^
    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

     /* These three used to be fetched in target endianness and then
      * stored as big endian.  It ended up as little endian if host and
      * target endianness doesn't match.
      *
      * Starting with qemu 2.1, we always store as big endian.  The
      * version wasn't bumped to avoid breaking backwards compatibility.
      * We check the validity of max_nr_ports, and the incorrect-
      * endianness max_nr_ports will be huge, which will abort migration
      * anyway.
      */
     uint16_t cols = tswap16(s->config.cols);
     uint16_t rows = tswap16(s->config.rows);
     uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

     qemu_put_be16s(f, &cols);
     qemu_put_be16s(f, &rows);
     qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
     uint16_t cols, rows;

     qemu_get_be16s(f, &cols);
     qemu_get_be16s(f, &rows);
     qemu_get_be32s(f, &max_nr_ports);

     /* Convert back to target endianness when storing into the config
      * space.
      */

Paolo,

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\
If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
     s->config.cols = tswap16(cols);
     s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
     if (max_nr_ports > tswap32(s->config.max_nr_ports) {
         ...
     }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 10 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Paolo

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#43 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 78 lines · Greg Kurz
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

            nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                       ^^^^^^^^^^^
            /* Check it isn't doing very strange things with descriptor numbers. */
            if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

    /* The config space */
    qemu_get_be16s(f, &s->config.cols);
    qemu_get_be16s(f, &s->config.rows);

    qemu_get_be32s(f, &max_nr_ports);
    tswap32s(&max_nr_ports);
     ^^^^^^
    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

     /* These three used to be fetched in target endianness and then
      * stored as big endian.  It ended up as little endian if host and
      * target endianness doesn't match.
      *
      * Starting with qemu 2.1, we always store as big endian.  The
      * version wasn't bumped to avoid breaking backwards compatibility.
      * We check the validity of max_nr_ports, and the incorrect-
      * endianness max_nr_ports will be huge, which will abort migration
      * anyway.
      */
     uint16_t cols = tswap16(s->config.cols);
     uint16_t rows = tswap16(s->config.rows);
     uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

     qemu_put_be16s(f, &cols);
     qemu_put_be16s(f, &rows);
     qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
     uint16_t cols, rows;

     qemu_get_be16s(f, &cols);
     qemu_get_be16s(f, &rows);
     qemu_get_be32s(f, &max_nr_ports);

     /* Convert back to target endianness when storing into the config
      * space.
      */
Quoted 8 lines

Paolo,

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 29 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
     s->config.cols = tswap16(cols);
     s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
     if (max_nr_ports > tswap32(s->config.max_nr_ports) {
         ...
     }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 4 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

Quoted 8 lines
Quoted 6 lines

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Paolo

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

Quoted 8 lines
Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
author
#44 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 81 lines · On Thu, 12 Jun 2014 10:54:48 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 80 lines · Greg Kurz
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

            nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                       ^^^^^^^^^^^
            /* Check it isn't doing very strange things with descriptor numbers. */
            if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

    /* The config space */
    qemu_get_be16s(f, &s->config.cols);
    qemu_get_be16s(f, &s->config.rows);

    qemu_get_be32s(f, &max_nr_ports);
    tswap32s(&max_nr_ports);
     ^^^^^^
    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

     /* These three used to be fetched in target endianness and then
      * stored as big endian.  It ended up as little endian if host and
      * target endianness doesn't match.
      *
      * Starting with qemu 2.1, we always store as big endian.  The
      * version wasn't bumped to avoid breaking backwards compatibility.
      * We check the validity of max_nr_ports, and the incorrect-
      * endianness max_nr_ports will be huge, which will abort migration
      * anyway.
      */
     uint16_t cols = tswap16(s->config.cols);
     uint16_t rows = tswap16(s->config.rows);
     uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

     qemu_put_be16s(f, &cols);
     qemu_put_be16s(f, &rows);
     qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
     uint16_t cols, rows;

     qemu_get_be16s(f, &cols);
     qemu_get_be16s(f, &rows);
     qemu_get_be32s(f, &max_nr_ports);

     /* Convert back to target endianness when storing into the config
      * space.
      */

Paolo,

Quoted 8 lines
Quoted 6 lines

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Store it earlier then, using plain put/get.

Not sure I follow... this will break compatibility, no ?

Quoted 39 lines

You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 36 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
     s->config.cols = tswap16(cols);
     s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
     if (max_nr_ports > tswap32(s->config.max_nr_ports) {
         ...
     }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 10 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Paolo

Quoted 7 lines
Quoted 1 line

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

Hmmm... AFAIC some QEMU maintainers expressed interest in supporting legacy virtio in the case we have endian-changing targets. Patches to run a ppc64le guests have been accepted in KVM, Linux and QEMU... the only missing block is virtio. I don't especially care in supporting old drivers at all cost: this request was expressed on the list. I just want people to be able to run a ppc64le ubuntu-14.04 (and soon other distros) guest on a ppc64 box and be able to migrate.

Would it be acceptable to break compatibility for ppc64 (and maybe ARM) only with a target specific hook being called from the virtio code ?

Quoted 10 lines
Quoted 9 lines
Signature · 8 lines
-- 
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#45 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 90 lines · Greg Kurz
Quoted 88 lines · On Thu, 12 Jun 2014 10:54:48 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 86 lines · Greg Kurz
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

            nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                       ^^^^^^^^^^^
            /* Check it isn't doing very strange things with descriptor numbers. */
            if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

    /* The config space */
    qemu_get_be16s(f, &s->config.cols);
    qemu_get_be16s(f, &s->config.rows);

    qemu_get_be32s(f, &max_nr_ports);
    tswap32s(&max_nr_ports);
     ^^^^^^
    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

     /* These three used to be fetched in target endianness and then
      * stored as big endian.  It ended up as little endian if host and
      * target endianness doesn't match.
      *
      * Starting with qemu 2.1, we always store as big endian.  The
      * version wasn't bumped to avoid breaking backwards compatibility.
      * We check the validity of max_nr_ports, and the incorrect-
      * endianness max_nr_ports will be huge, which will abort migration
      * anyway.
      */
     uint16_t cols = tswap16(s->config.cols);
     uint16_t rows = tswap16(s->config.rows);
     uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

     qemu_put_be16s(f, &cols);
     qemu_put_be16s(f, &rows);
     qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
     uint16_t cols, rows;

     qemu_get_be16s(f, &cols);
     qemu_get_be16s(f, &rows);
     qemu_get_be32s(f, &max_nr_ports);

     /* Convert back to target endianness when storing into the config
      * space.
      */

Paolo,

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Quoted 3 lines
Quoted 1 line

Store it earlier then, using plain put/get.

Not sure I follow... this will break compatibility, no ?

Do it only on the ambialent targets :)

Quoted 47 lines
Quoted 39 lines

You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 36 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
     s->config.cols = tswap16(cols);
     s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
     if (max_nr_ports > tswap32(s->config.max_nr_ports) {
         ...
     }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 10 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Paolo

Quoted 7 lines
Quoted 1 line

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

Quoted 6 lines

Hmmm... AFAIC some QEMU maintainers expressed interest in supporting legacy virtio in the case we have endian-changing targets. Patches to run a ppc64le guests have been accepted in KVM, Linux and QEMU... the only missing block is virtio. I don't especially care in supporting old drivers at all cost: this request was expressed on the list. I just want people to be able to run a ppc64le ubuntu-14.04 (and soon other distros) guest on a ppc64 box and be able to migrate.

Sigh. So ubuntu ships known-broken virtio drivers in their LTS release and now we are trying to make them work.
There's no working qemu configuration at the moment, do we really have to build new hardware around known-broken drivers
and then maintain it all for years?
Why not build good hardware and backport new drivers?

Quoted 2 lines

Would it be acceptable to break compatibility for ppc64 (and maybe ARM) only with a target specific hook being called from the virtio code ?

I think so.
Can we just add a function that tells us whether target is ambivalent? Only migrate new stuff if that's the case.

Quoted 21 lines
Quoted 10 lines
Quoted 9 lines
Signature · 8 lines
-- 
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.

Thanks.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#46 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Greg Kurz
Quoted 137 lines · Greg Kurz
Quoted 78 lines · On Thu, 12 Jun 2014 10:54:48 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 77 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

             nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                        ^^^^^^^^^^^
             /* Check it isn't doing very strange things with descriptor numbers. */
             if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

     /* The config space */
     qemu_get_be16s(f, &s->config.cols);
     qemu_get_be16s(f, &s->config.rows);

     qemu_get_be32s(f, &max_nr_ports);
     tswap32s(&max_nr_ports);
      ^^^^^^
     if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

      /* These three used to be fetched in target endianness and then
       * stored as big endian.  It ended up as little endian if host and
       * target endianness doesn't match.
       *
       * Starting with qemu 2.1, we always store as big endian.  The
       * version wasn't bumped to avoid breaking backwards compatibility.
       * We check the validity of max_nr_ports, and the incorrect-
       * endianness max_nr_ports will be huge, which will abort migration
       * anyway.
       */
      uint16_t cols = tswap16(s->config.cols);
      uint16_t rows = tswap16(s->config.rows);
      uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

      qemu_put_be16s(f, &cols);
      qemu_put_be16s(f, &rows);
      qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
      uint16_t cols, rows;

      qemu_get_be16s(f, &cols);
      qemu_get_be16s(f, &rows);
      qemu_get_be32s(f, &max_nr_ports);

      /* Convert back to target endianness when storing into the config
       * space.
       */

Paolo,

Quoted 7 lines
Quoted 6 lines

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Store it earlier then, using plain put/get.

Not sure I follow... this will break compatibility, no ?

Quoted 33 lines

You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 30 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
      s->config.cols = tswap16(cols);
      s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
      if (max_nr_ports > tswap32(s->config.max_nr_ports) {
          ...
      }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 7 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Quoted 8 lines
Quoted 3 lines
Quoted 2 lines

Paolo

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

Hmmm... AFAIC some QEMU maintainers expressed interest in supporting legacy virtio in the case we have endian-changing targets. Patches to run a ppc64le guests have been accepted in KVM, Linux and QEMU... the only missing block is virtio. I don't especially care in supporting old drivers at all cost: this request was expressed on the list. I just want people to be able to run a ppc64le ubuntu-14.04 (and soon other distros) guest on a ppc64 box and be able to migrate.

Quoted 2 lines

Would it be acceptable to break compatibility for ppc64 (and maybe ARM) only with a target specific hook being called from the virtio code ?

I don't see the problem. Let's just make the config fields UNUSED in vmstate and keep the virtio-serial config space in good endian locally. Then you don't care about ordering of sections anymore.

Then we only need the subsection code for "endian is not target endian" which makes migration backwards compatible with non-switchable QEMU versions and everyone's happy. End of story :).

Alex

#47 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Michael S. Tsirkin
Quoted 118 lines · Michael S. Tsirkin
Quoted 75 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

             nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                        ^^^^^^^^^^^
             /* Check it isn't doing very strange things with descriptor numbers. */
             if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

     /* The config space */
     qemu_get_be16s(f, &s->config.cols);
     qemu_get_be16s(f, &s->config.rows);

     qemu_get_be32s(f, &max_nr_ports);
     tswap32s(&max_nr_ports);
      ^^^^^^
     if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

      /* These three used to be fetched in target endianness and then
       * stored as big endian.  It ended up as little endian if host and
       * target endianness doesn't match.
       *
       * Starting with qemu 2.1, we always store as big endian.  The
       * version wasn't bumped to avoid breaking backwards compatibility.
       * We check the validity of max_nr_ports, and the incorrect-
       * endianness max_nr_ports will be huge, which will abort migration
       * anyway.
       */
      uint16_t cols = tswap16(s->config.cols);
      uint16_t rows = tswap16(s->config.rows);
      uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

      qemu_put_be16s(f, &cols);
      qemu_put_be16s(f, &rows);
      qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
      uint16_t cols, rows;

      qemu_get_be16s(f, &cols);
      qemu_get_be16s(f, &rows);
      qemu_get_be32s(f, &max_nr_ports);

      /* Convert back to target endianness when storing into the config
       * space.
       */
Quoted 8 lines

Paolo,

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 30 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
      s->config.cols = tswap16(cols);
      s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
      if (max_nr_ports > tswap32(s->config.max_nr_ports) {
          ...
      }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 7 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

Paolo

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

There are already released and working Linux distributions (Ubuntu, openSUSE, maybe others) that don't have virtio 1.0 drivers. Putting our heads into the sand is not an option ;).

Alex

#48 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 128 lines · Alexander Graf
Quoted 118 lines · Michael S. Tsirkin
Quoted 75 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

            nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                       ^^^^^^^^^^^
            /* Check it isn't doing very strange things with descriptor numbers. */
            if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

    /* The config space */
    qemu_get_be16s(f, &s->config.cols);
    qemu_get_be16s(f, &s->config.rows);

    qemu_get_be32s(f, &max_nr_ports);
    tswap32s(&max_nr_ports);
     ^^^^^^
    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

     /* These three used to be fetched in target endianness and then
      * stored as big endian.  It ended up as little endian if host and
      * target endianness doesn't match.
      *
      * Starting with qemu 2.1, we always store as big endian.  The
      * version wasn't bumped to avoid breaking backwards compatibility.
      * We check the validity of max_nr_ports, and the incorrect-
      * endianness max_nr_ports will be huge, which will abort migration
      * anyway.
      */
     uint16_t cols = tswap16(s->config.cols);
     uint16_t rows = tswap16(s->config.rows);
     uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

     qemu_put_be16s(f, &cols);
     qemu_put_be16s(f, &rows);
     qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
     uint16_t cols, rows;

     qemu_get_be16s(f, &cols);
     qemu_get_be16s(f, &rows);
     qemu_get_be32s(f, &max_nr_ports);

     /* Convert back to target endianness when storing into the config
      * space.
      */
Quoted 8 lines

Paolo,

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 30 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
     s->config.cols = tswap16(cols);
     s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
     if (max_nr_ports > tswap32(s->config.max_nr_ports) {
         ...
     }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 7 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Quoted 7 lines
Quoted 3 lines
Quoted 2 lines

Paolo

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

Quoted 6 lines

There are already released and working Linux distributions (Ubuntu, openSUSE, maybe others) that don't have virtio 1.0 drivers. Putting our heads into the sand is not an option ;).

Alex

I don't get it. Does virtio work there at the moment?

Signature · 1 line
MST
#49 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 127 lines · Michael S. Tsirkin
Quoted 126 lines · Alexander Graf
Quoted 125 lines · Michael S. Tsirkin
Quoted 75 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:

Il 29/05/2014 11:12, Greg Kurz ha scritto:

Quoted 23 lines

int virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]

             nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx;
                        ^^^^^^^^^^^
             /* Check it isn't doing very strange things with descriptor numbers. */
             if (nheads > vdev->vq[i].vring.num) {

[...]
}

and

static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) {
[...]

     /* The config space */
     qemu_get_be16s(f, &s->config.cols);
     qemu_get_be16s(f, &s->config.rows);

     qemu_get_be32s(f, &max_nr_ports);
     tswap32s(&max_nr_ports);
      ^^^^^^
     if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
Quoted 8 lines

[...]
}

If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.

The first one can be easily dealt with: just defer the sanity check to a post_load function.

Good, we're lucky here.

Quoted 5 lines

The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumes max_nr_ports from the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\

Ouch.

I guess we could break migration in the case of host endianness != target endianness, like this:

      /* These three used to be fetched in target endianness and then
       * stored as big endian.  It ended up as little endian if host and
       * target endianness doesn't match.
       *
       * Starting with qemu 2.1, we always store as big endian.  The
       * version wasn't bumped to avoid breaking backwards compatibility.
       * We check the validity of max_nr_ports, and the incorrect-
       * endianness max_nr_ports will be huge, which will abort migration
       * anyway.
       */
      uint16_t cols = tswap16(s->config.cols);
      uint16_t rows = tswap16(s->config.rows);
      uint32_t max_nr_ports = tswap32(s->config.max_nr_ports);

      qemu_put_be16s(f, &cols);
      qemu_put_be16s(f, &rows);
      qemu_put_be32s(f, &max_nr_ports);

...

Quoted 9 lines
      uint16_t cols, rows;

      qemu_get_be16s(f, &cols);
      qemu_get_be16s(f, &rows);
      qemu_get_be32s(f, &max_nr_ports);

      /* Convert back to target endianness when storing into the config
       * space.
       */
Quoted 8 lines

Paolo,

The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\

Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.

Quoted 26 lines

If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...

Quoted 2 lines
      s->config.cols = tswap16(cols);
      s->config.rows = tswap16(rows);

Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.

Quoted 4 lines
      if (max_nr_ports > tswap32(s->config.max_nr_ports) {
          ...
      }

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 3 lines
Quoted 3 lines

In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)

Quoted 7 lines
Quoted 6 lines

As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.

Paolo

Thanks.

One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?

Quoted 7 lines
Quoted 6 lines

There are already released and working Linux distributions (Ubuntu, openSUSE, maybe others) that don't have virtio 1.0 drivers. Putting our heads into the sand is not an option ;).

Alex

I don't get it. Does virtio work there at the moment?

With the LE enable patch, yes, sure. Imagine the same would happen for Windows and e1000 - would you still argue the same way?

Alex

#50 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Greg Kurz

Il 12/06/2014 09:43, Greg Kurz ha scritto:

Quoted 10 lines

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}

Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and virtio_tswap on top of the previous change.

Paolo

#51 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 8 lines · Paolo Bonzini

Il 12/06/2014 09:43, Greg Kurz ha scritto:

Quoted 7 lines

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
    max_nr_ports = bswap32(max_nr_ports);
}
Quoted 5 lines
Quoted 3 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
    return -EINVAL;
}

Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and virtio_tswap on top of the previous change.

Why don't we just ignore those config fields? They won't change during the lifetime of the guest anyway - and configuration needs to be the same on source and dest to work.

Alex

author
#52 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Paolo Bonzini
Quoted 12 lines · On Thu, 12 Jun 2014 10:55:42 +0200 Paolo Bonzini <[email protected]> wrote:

Il 12/06/2014 09:43, Greg Kurz ha scritto:

Quoted 10 lines

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 6 lines

Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and virtio_tswap on top of the previous change.

Paolo

BTW, can someone explain why we stream the device config ?

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#53 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Il 12/06/2014 11:06, Greg Kurz ha scritto:

Quoted 15 lines
Quoted 12 lines · On Thu, 12 Jun 2014 10:55:42 +0200 Paolo Bonzini <[email protected]> wrote:

Il 12/06/2014 09:43, Greg Kurz ha scritto:

Quoted 10 lines

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 5 lines
Quoted 3 lines

Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and virtio_tswap on top of the previous change.

BTW, can someone explain why we stream the device config ?

For max_nr_ports it is useless indeed. Let's keep storing it for backwards compatibility, but you can remove its load.

The cols and rows values should be defined by the host and thus could even change on migration, there is no need to store them. As of now, however, they are unused in QEMU and should always be zero, because VIRTIO_CONSOLE_F_SIZE is not supported.

Paolo

#54 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 25 lines · Paolo Bonzini

Il 12/06/2014 11:06, Greg Kurz ha scritto:

Quoted 15 lines
Quoted 12 lines · On Thu, 12 Jun 2014 10:55:42 +0200 Paolo Bonzini <[email protected]> wrote:

Il 12/06/2014 09:43, Greg Kurz ha scritto:

Quoted 10 lines

Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?

if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	max_nr_ports = bswap32(max_nr_ports);
}
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
	return -EINVAL;
}
Quoted 5 lines
Quoted 3 lines

Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and virtio_tswap on top of the previous change.

BTW, can someone explain why we stream the device config ?

For max_nr_ports it is useless indeed. Let's keep storing it for backwards compatibility, but you can remove its load.

Quoted 6 lines

The cols and rows values should be defined by the host and thus could even change on migration, there is no need to store them. As of now, however, they are unused in QEMU and should always be zero, because VIRTIO_CONSOLE_F_SIZE is not supported.

Paolo

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

Signature · 1 line
MST
#55 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

Paolo

#56 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 5 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +6 -10
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
+    uint32_t tmp;
 
     if (version_id > 3) {
         return -EINVAL;
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)
         return 0;
     }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
+    max_nr_ports = tswap32(s->config.max_nr_ports);
     for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
         qemu_get_be32s(f, &ports_map);
author
#57 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 49 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 6 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
      uint32_t max_nr_ports, nr_active_ports, ports_map;
      unsigned int i;
      int ret;
+    uint32_t tmp;
 
      if (version_id > 3) {
          return -EINVAL;
hw/char/virtio-serial-bus.c +0 -8
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
          return 0;
      }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697

Quoted 10 lines
hw/char/virtio-serial-bus.c +5 -2
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
+    max_nr_ports = tswap32(s->config.max_nr_ports);
      for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
          qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#58 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 61 lines

Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:

Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
+    uint32_t tmp;
 
     if (version_id > 3) {
         return -EINVAL;
hw/char/virtio-serial-bus.c +4 -10
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
         return 0;
     }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
Quoted 9 lines

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696

Quoted 3 lines
hw/char/virtio-serial-bus.c +1 -0
+    max_nr_ports = tswap32(s->config.max_nr_ports);
     for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
         qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.

Alex

author
#59 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 71 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 61 lines

Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:

Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
+    uint32_t tmp;
 
     if (version_id > 3) {
         return -EINVAL;
hw/char/virtio-serial-bus.c +4 -10
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
         return 0;
     }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
Quoted 9 lines

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696

Quoted 3 lines
hw/char/virtio-serial-bus.c +1 -0
+    max_nr_ports = tswap32(s->config.max_nr_ports);
     for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
         qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

Quoted 2 lines

Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.

I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#60 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 78 lines

Am 12.06.2014 um 12:50 schrieb Greg Kurz <[email protected]>:

Quoted 71 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 70 lines

Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:

Quoted 47 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
    uint32_t max_nr_ports, nr_active_ports, ports_map;
    unsigned int i;
    int ret;
+    uint32_t tmp;
 
    if (version_id > 3) {
        return -EINVAL;
hw/char/virtio-serial-bus.c +0 -8
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
        return 0;
    }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697

Quoted 10 lines
hw/char/virtio-serial-bus.c +5 -2
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
+    max_nr_ports = tswap32(s->config.max_nr_ports);
    for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
        qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

Quoted 6 lines
Quoted 1 line

Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.

I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?

The destination has be be executed in compat mode to the older qemu version via a versioned machine type. This should ensure that limits are kept.

Alex

#61 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Greg Kurz
Quoted 74 lines · Greg Kurz
Quoted 71 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 70 lines

Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:

Quoted 47 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
+    uint32_t tmp;
 
     if (version_id > 3) {
         return -EINVAL;
hw/char/virtio-serial-bus.c +0 -8
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
         return 0;
     }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697

Quoted 10 lines
hw/char/virtio-serial-bus.c +5 -2
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
+    max_nr_ports = tswap32(s->config.max_nr_ports);
     for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
         qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

Quoted 7 lines
Quoted 2 lines

Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.

I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?

When the destination machine is started with -M 2.1, it knows that it has to read 32 bit.
If started with -M 3.0 it reads in 42 bits :)

Quoted 8 lines
Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
author
#62 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

Quoted 78 lines · On Thu, 12 Jun 2014 13:59:59 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 77 lines · Greg Kurz
Quoted 73 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 61 lines

Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:

Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
+    uint32_t tmp;
 
     if (version_id > 3) {
         return -EINVAL;
hw/char/virtio-serial-bus.c +4 -10
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
         return 0;
     }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
Quoted 9 lines

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696

Quoted 3 lines
hw/char/virtio-serial-bus.c +1 -0
+    max_nr_ports = tswap32(s->config.max_nr_ports);
     for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
         qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.

Quoted 9 lines
Quoted 4 lines

I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?

When the destination machine is started with -M 2.1, it knows that it has to read 32 bit.
If started with -M 3.0 it reads in 42 bits :)

Okay ! I was completely missing this point... now things make sense at last ! :) About Alex's patch, will you or Amit or Anthony apply it directly or should I send it along with my patches for a full review ?

Thanks for your time.

Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
#63 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Greg Kurz
Quoted 57 lines · Greg Kurz
Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 6 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
      uint32_t max_nr_ports, nr_active_ports, ports_map;
      unsigned int i;
      int ret;
+    uint32_t tmp;
 
      if (version_id > 3) {
          return -EINVAL;
hw/char/virtio-serial-bus.c +4 -10
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void 
 *opaque, int version_id)
          return 0;
      }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
Quoted 11 lines

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696

Quoted 5 lines
hw/char/virtio-serial-bus.c +1 -0
+    max_nr_ports = tswap32(s->config.max_nr_ports);
      for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
          qemu_get_be32s(f, &ports_map);

For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...

I don't think we need to worry.
We won't be able to change max_nr_ports in compat machine types.

Quoted 8 lines
Signature · 7 lines
Gregory Kurz                                     [email protected]
                                                 [email protected]
Software Engineer @ IBM/Meiosys                  http://www.ibm.com
Tel +33 (0)562 165 496

"Anarchy is about taking complete responsibility for yourself."
        Alan Moore.
Reviewed-by
#64 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Alexander Graf
Quoted 48 lines · Alexander Graf
Quoted 6 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.

Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).

Alex

hw/char/virtio-serial-bus.c +1 -0
index 2b647b6..73cb9b7 100644
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque,
 int version_id)
     uint32_t max_nr_ports, nr_active_ports, ports_map;
     unsigned int i;
     int ret;
+    uint32_t tmp;
 
     if (version_id > 3) {
         return -EINVAL;
hw/char/virtio-serial-bus.c +0 -8
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void
 *opaque, int version_id)
         return 0;
     }
 
-    /* The config space */
-    qemu_get_be16s(f, &s->config.cols);
-    qemu_get_be16s(f, &s->config.rows);
-
-    qemu_get_be32s(f, &max_nr_ports);
-    tswap32s(&max_nr_ports);
-    if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
-        /* Source could have had more ports than us. Fail migration. */

hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697

Quoted 10 lines
hw/char/virtio-serial-bus.c +5 -2
-        return -EINVAL;
-    }
+    /* Unused */
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be16s(f, &tmp);
+    qemu_get_be32s(f, &tmp);
 
+    max_nr_ports = tswap32(s->config.max_nr_ports);
     for (i = 0; i < (max_nr_ports + 31) / 32; i++) {
         qemu_get_be32s(f, &ports_map);

Exactly.

#65 lore raw

Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice

In reply to Paolo Bonzini
Quoted 8 lines · Paolo Bonzini

Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:

Quoted 2 lines

Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.

For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.

Paolo

Ah, good idea, so simply disregard the values in load.

Signature · 1 line
MST