virtio: migrate new properties
cover virtio: migrate new properties
The work on virtio endianness unveiled the need to add new properties to VirtIODevice and be able to migrate without breaking compatibility. There was a similar concern with the introduction of a VirtIODevice.broken property.
This patch set is a tentative to address the issue, following a suggestion from Alex:
- patches 1 to 7 adds optional subsections in the virtio migration stream
- patch 8 belongs to the virtio endianness serie, it is sent as an example
Thanks for your comments.
---
Greg Kurz (8):
virtio: add subsections to the migration stream
virtio-net: migrate subsections
virtio-blk: migrate subsections
virtio-scsi: migrate subsections
virtio-serial: migrate subsections
virtio-balloon: migrate subsections
virtio-rng: migrate subsections
virtio: add endian-ambivalent support to VirtIODevice
exec.c | 8 --- hw/block/virtio-blk.c | 4 + hw/char/virtio-serial-bus.c | 10 ++- hw/net/virtio-net.c | 7 ++ hw/scsi/virtio-scsi.c | 3 + hw/virtio/virtio-balloon.c | 4 + hw/virtio/virtio-pci.c | 11 +--- hw/virtio/virtio-rng.c | 13 ++++ hw/virtio/virtio.c | 129 +++++++++++++++++++++++++++++++++++++++---- include/exec/cpu-common.h | 1 include/hw/virtio/virtio.h | 17 ++++++ 11 files changed, 175 insertions(+), 32 deletions(-)
Signature · 1 line
Greg
RFC PATCH 1/8 virtio: add subsections to the migration stream
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM state
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
--- hw/virtio/virtio.c | 65 ++++++++++++++++++++++++++++++++++++++++++++ include/hw/virtio/virtio.h | 4 +++ 2 files changed, 69 insertions(+)
hw/virtio/virtio.c +65 -0
@@ -19,6 +19,7 @@#include "hw/virtio/virtio.h" #include "qemu/atomic.h" #include "hw/virtio/virtio-bus.h" +#include "migration/migration.h" /* * The alignment to use between consumer and producer parts of vring.
@@ -833,6 +834,70 @@ void virtio_notify_config(VirtIODevice *vdev)virtio_notify_vector (vdev, vdev->config_vector); } +static const struct VirtIOSubsectionDescStruct { +const char *name; +int version_id; +void (*save)(VirtIODevice *vdev,QEMUFile *f); +int (*load)(VirtIODevice *vdev,QEMUFile *f); +} virtio_subsection[] = { + { .name =NULL } +}; + +void virtio_save_subsections (VirtIODevice *vdev,QEMUFile *f) +{ +int i; + +for (i =0 ; virtio_subsection[i].name; i++) { +const char *name = virtio_subsection[i].name; +uint8_t len =strlen (name); + +qemu_put_byte (f,QEMU_VM_SUBSECTION ); +qemu_put_byte (f, len); +qemu_put_buffer (f, (uint8_t *) name, len); +qemu_put_be32 (f, virtio_subsection[i].version_id); + (*virtio_subsection[i].save)(vdev, f); + } +} + +int virtio_load_subsections (VirtIODevice *vdev,QEMUFile *f) +{ +while (qemu_peek_byte (f,0 ) ==QEMU_VM_SUBSECTION ) { +char idstr[256 ]; +uint8_t len, size; +int i; + + len =qemu_peek_byte (f,1 ); + size =qemu_peek_buffer (f, (uint8_t *) idstr, len,2 ); +if (size != len) { +break ; + } + + idstr[size] =0 ; + +for (i =0 ; virtio_subsection[i].name; i++) { +if (strcmp (virtio_subsection[i].name, idstr) ==0 ) { +uint32_t version_id; +int ret; + +qemu_file_skip (f,1 );/* subsection */ +qemu_file_skip (f,1 );/* len */ +qemu_file_skip (f, len);/* idstr */ + + version_id =qemu_get_be32 (f); +if (version_id > virtio_subsection[i].version_id) { +return -EINVAL ; + } + ret = (*virtio_subsection[i].load)(vdev, f); +if (ret) { +return ret; + } + } + } + } + +return 0 ; +} +void virtio_save (VirtIODevice *vdev,QEMUFile *f) {BusState *qbus =qdev_get_parent_bus (DEVICE (vdev));
include/hw/virtio/virtio.h +4 -0
@@ -186,6 +186,10 @@ void virtio_save(VirtIODevice *vdev, QEMUFile *f);int virtio_load (VirtIODevice *vdev,QEMUFile *f); +void virtio_save_subsections (VirtIODevice *vdev,QEMUFile *f); + +int virtio_load_subsections (VirtIODevice *vdev,QEMUFile *f); +void virtio_notify_config (VirtIODevice *vdev);void virtio_queue_set_notification (VirtQueue *vq,int enable);
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 8 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Quoted 7 lines
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM state
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Amit
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 21 lines · Amit Shah
Quoted 8 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Quoted 7 lines
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 8 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Amit
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
BTW Greg, do you plan on working on vmstate for virtio?
Quoted 15 lines
Quoted 15 lines
Quoted 7 lines
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Quoted 7 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.
No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Amit
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Quoted 2 lines
BTW Greg, do you plan on working on vmstate for virtio?
Yes.
Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
Quoted 7 lines
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
Quoted 2 lines
Amit
Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 10 lines
Quoted 8 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines · Amit Shah
Quoted 5 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
Would touch on Stefan's alias properties for anything but virtio-mmio.
Regards,
Andreas
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 14 lines · Andreas Färber
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM state
Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.
So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
So yes it's a useful feature, and no not supporting it does not help anyway.
Quoted 42 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
Would touch on Stefan's alias properties for anything but virtio-mmio.
Regards,
AndreasSignature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
It seems you misunderstand. I was not saying it's not useful.
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Andreas
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 46 lines · Andreas Färber
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
Quoted 7 lines
It seems you misunderstand. I was not saying it's not useful.
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Andreas
Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.
Quoted 3 lines
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:
Quoted 49 lines · Andreas Färber
Quoted 48 lines
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
It seems you misunderstand. I was not saying it's not useful.
Quoted 7 lines
Quoted 3 lines
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.
OK, but upstream always sends the latest version today. So isn't that just two ifs that you would need to add in save and load functions added here for the downstream? x86_64 is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?
Andreas
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 58 lines · Andreas Färber
Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:
Quoted 49 lines · Andreas Färber
Quoted 48 lines
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
It seems you misunderstand. I was not saying it's not useful.
Quoted 7 lines
Quoted 3 lines
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.
Quoted 8 lines
OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream?x86_64is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?Andreas
I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.
With *any* change, there's always the temptation to say "oh but old behaviour was too buggy we should just ignore it". Seems to make sense in each case but does not scale, you end up with breakages in all version without exception. So this should be a very high bar to overcome.
Quoted 3 lines
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Am 15.05.2014 12:16, schrieb Michael S. Tsirkin:
Quoted 66 lines · Andreas Färber
Quoted 58 lines
Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:
Quoted 49 lines · Andreas Färber
Quoted 48 lines
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
It seems you misunderstand. I was not saying it's not useful.
Quoted 7 lines
Quoted 3 lines
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.Quoted 6 lines
OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream?x86_64is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?
Quoted 4 lines
I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.
Whatever the version_id fields are for, upstream QEMU *always* saves the newest version_id format that it knows. There is no mechanism that I'm aware of in upstream QEMU for migrating device fields dependent on -M. So a device in QEMU only migrates exactly like an old QEMU did if neither fields nor subsections were added.
Subsections are usually migrated based on whether the state differs from the default state (didn't check whether the final patch does this correctly here, but doesn't matter for 1/8 concept). So for x86 the subsection should *never* get written and thus not be a problem for you. For ppc64 it should not get written either, unless you care about migration from SLES12 or upstream ppc64le to RHEL ppc64.
As I understood the IRC discussion Alex and me had with Greg about this, this is copying the exact code VMState uses to write its subsections, so there would be no forward migration problems at all once we convert to VMState and VMStateSubsections. The only question remaining is how does your downstream react when it encounters a subsection marker for subsections it doesn't know - which is not something upstream can solve for you unless you contribute backwards migration support upstream.
Don't forget that Greg is introducing subsection support *because of* your backwards migration wish, so telling him not to add subsections because of backwards migration is really weird! Either subsections work with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.
Andreas
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 72 lines · Andreas Färber
Am 15.05.2014 12:16, schrieb Michael S. Tsirkin:
Quoted 66 lines · Andreas Färber
Quoted 58 lines
Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:
Quoted 49 lines · Andreas Färber
Quoted 48 lines
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
It seems you misunderstand. I was not saying it's not useful.
Quoted 7 lines
Quoted 3 lines
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.Quoted 6 lines
OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream?x86_64is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?Quoted 4 lines
I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.
Quoted 5 lines
Whatever the
version_idfields are for, upstream QEMU *always* saves the newestversion_idformat that it knows. There is no mechanism that I'm aware of in upstream QEMU for migrating device fields dependent on -M. So a device in QEMU only migrates exactly like an old QEMU did if neither fields nor subsections were added.
Ah you are talking about version ids.
Yes, they can not support cross-version migration.
For this reason these should only be incremented as the last resort, normally new fields (possibly subsections) should be added, with field_exists callback checking for compatibility mode.
Quoted 3 lines
Subsections are usually migrated based on whether the state differs from the default state (didn't check whether the final patch does this correctly here, but doesn't matter for 1/8 concept).
Yes, this would be fine, but it's not what's implemented IIUC.
Quoted 12 lines
So for x86 the
subsection should *never* get written and thus not be a problem for you. For ppc64 it should not get written either, unless you care about migration from SLES12 or upstream ppc64le to RHEL ppc64.
As I understood the IRC discussion Alex and me had with Greg about this, this is copying the exact code VMState uses to write its subsections, so there would be no forward migration problems at all once we convert to VMState and VMStateSubsections. The only question remaining is how does your downstream react when it encounters a subsection marker for subsections it doesn't know - which is not something upstream can solve for you unless you contribute backwards migration support upstream.
Quoted 3 lines
Don't forget that Greg is introducing subsection support *because of* your backwards migration wish, so telling him not to add subsections because of backwards migration is really weird!
I don't really care whether it's a subsection or just plain qemu_put_byte, but I'm not saying not to use subsection either.
I am merely saying this should have "exists" callback checking for compability mode for old machine types.
Quoted 9 lines
Either subsections work
with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.Andreas
I only mentioned downstream because it's a requirement coming from users most of whom never interact with upstream.
There shouldn't be any difference and people building clusters using upstream qemu will have same needs: clusters can't be completely brought down for version upgrades, so need to be able to run a mix of versions, and be able to migrate in any direction while in the process.
Quoted 3 lines
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 1 line · Michael S. Tsirkin
On Thu, May 15, 2014 at 02:00:15PM +0200, Andreas Färber wrote:
[...]
Quoted 10 lines
Quoted 9 lines
Either subsections work
with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.Andreas
Quoted 7 lines
I only mentioned downstream because it's a requirement coming from users most of whom never interact with upstream.
There shouldn't be any difference and people building clusters using upstream qemu will have same needs: clusters can't be completely brought down for version upgrades, so need to be able to run a mix of versions, and be able to migrate in any direction while in the process.
Okay, I'll have world peace, a cluster, and a pony, please.
Seriously, no matter how much your cluster management software wants to migrate backwards, it's not going to work reliably, except in very narrow cases, at least not today.
If you want to migrate freely, stick to an ultra-stable (downstream) branch of QEMU. When that branch EOLs, scrap your cluster and start over.
If you want to upgrade QEMU in more interesting ways, keep track of what your cluster nodes are running, and migrate only forward.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Andreas FärberQuoted 78 lines · On Thu, 15 May 2014 14:00:15 +0200 Andreas Färber <[email protected]> wrote:
Am 15.05.2014 12:16, schrieb Michael S. Tsirkin:
Quoted 66 lines · Andreas Färber
Quoted 58 lines
Am 15.05.2014 12:03, schrieb Michael S. Tsirkin:
Quoted 49 lines · Andreas Färber
Quoted 48 lines
Am 15.05.2014 11:52, schrieb Michael S. Tsirkin:
Quoted 42 lines · Andreas Färber
Quoted 14 lines
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 7 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly. So some downstreams support this and not supporting it upstream just means downstreams need to do their own thing.
As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :) Once you do the hard part, breaking migration because of the trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
It seems you misunderstand. I was not saying it's not useful.
Quoted 7 lines
Quoted 3 lines
My point is that VMStateSubsections added in newer versions (and thus not existing in older versions) need to be handled for any VMState-converted devices. So why can't we make that work for virtio too?
Sure.
AFAIK the way to this works is by adding an "field_exists" callback, and not sending the section when we are in a compat mode.Quoted 6 lines
OK, but upstream always sends the latest version today.
So isn't that
just two ifs that you would need to add in save and load functions added here for the downstream?x86_64is unaffected from ppc's endianness changes and you still do ppc64 BE, so there's no real functional problem for RHEL, is there?Quoted 4 lines
I'm sorry I don't understand what you are saying here. Simply put, if you specify a compatible -M then your device should behave, and migrate, exactly like and old qemu did.
Whatever the
version_idfields are for, upstream QEMU *always* saves the newestversion_idformat that it knows. There is no mechanism that I'm aware of in upstream QEMU for migrating device fields dependent on -M. So a device in QEMU only migrates exactly like an old QEMU did if neither fields nor subsections were added.
Quoted 3 lines
Subsections are usually migrated based on whether the state differs from the default state (didn't check whether the final patch does this correctly here, but doesn't matter for 1/8 concept). So for x86 the
For this first round, subsections are sent unconditionally. I will change that.
Quoted 23 lines
subsection should *never* get written and thus not be a problem for you. For ppc64 it should not get written either, unless you care about migration from SLES12 or upstream ppc64le to RHEL ppc64.
As I understood the IRC discussion Alex and me had with Greg about this, this is copying the exact code VMState uses to write its subsections, so there would be no forward migration problems at all once we convert to VMState and VMStateSubsections. The only question remaining is how does your downstream react when it encounters a subsection marker for subsections it doesn't know - which is not something upstream can solve for you unless you contribute backwards migration support upstream.Don't forget that Greg is introducing subsection support *because of* your backwards migration wish, so telling him not to add subsections because of backwards migration is really weird! Either subsections work with your downstream or they don't; if they don't, then you have much larger problems than can be solved by blocking this one section for ppc. Therefore I was saying if your downstream forgot to handle the case of a non-VMState device having subsections, then it may be easier to fix exactly that than make Greg jump through more hoops here for a theoretical problem you are unlikely to encounter on your downstream.
Andreas
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Michael S. TsirkinQuoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Andreas Färber
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 6 lines
Quoted 3 lines
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly.
Can I have world peace and a pony with that?
Given the current state of things, attempting to support backward migration is trying to run before you can walk. We need to put migration on a more solid footing first.
The migration format is crap, and needs to be replaced.
Reasoning on migration compatibility is entirely manual.
Systematic testing of migration compatibility is done downstream.
Fortunately, there's progress being made on all of the above. Let's not sabotage it by biting off yet another mouthful.
Quoted 9 lines
So some downstreams support this
and not supporting it upstream just means downstreams need to do their own thing.As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Non sequitur.
Quoted 4 lines
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :)
However, it is.
Quoted 4 lines
Once you do the hard part, breaking migration because of the
trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.
Bold claim; citation needed.
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
Nobody denies reliable backward migration would be useful. However, attempting to do every useful feature at once just because they're all useful is foolish.
Treating backward migration as strictly secondary concern while we're up to the ass in other alligators *can* help, by letting us focus on the said other alligators.
I'm not opposed to coding things in ways that help backward migration. Speaking of "support", however, is clearly premature and misleading.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 60 lines · Markus Armbruster
Quoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Andreas Färber
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 6 lines
Quoted 3 lines
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly.
Can I have world peace and a pony with that?
Given the current state of things, attempting to support backward migration is trying to run before you can walk. We need to put migration on a more solid footing first.
The migration format is crap, and needs to be replaced.
Reasoning on migration compatibility is entirely manual.
Systematic testing of migration compatibility is done downstream.
Fortunately, there's progress being made on all of the above. Let's not sabotage it by biting off yet another mouthful.
Quoted 9 lines
So some downstreams support this
and not supporting it upstream just means downstreams need to do their own thing.As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Non sequitur.
Quoted 4 lines
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :)
Quoted 8 lines
However, it is.
Quoted 4 lines
Once you do the hard part, breaking migration because of the
trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.Bold claim; citation needed.
You are asking for examples of ugly assymetry?
It's easy: grep for .load_state_old.
You have here a bunch of functions loading format that qemu
can no longer produce, with any set of flags.
The only way to make them run is to install two qemu versions side by side, save from old one and load in the new one.
What, would you guess, is the chance that they actually work?
I'm going to send a patch removing all this stuff, it's effectively dead code, but this is just one, biggest example.
Quoted 7 lines
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
Nobody denies reliable backward migration would be useful. However, attempting to do every useful feature at once just because they're all useful is foolish.
Quoted 6 lines
Treating backward migration as strictly secondary concern while we're up to the ass in other alligators *can* help, by letting us focus on the said other alligators.
I'm not opposed to coding things in ways that help backward migration. Speaking of "support", however, is clearly premature and misleading.
I agree it's a secondary concern upstream and your comments really apply to migration generally. We can't claim that it's properly supported by the upstream QEMU.
I am merely asking that we don't break cross-version migration intentionally. I and others also try fix it when we notice it's broken. In particular I'm not asking that submitters test it.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 97 lines · On Thu, 15 May 2014 15:58:16 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 60 lines · Markus Armbruster
Quoted 19 lines · Michael S. Tsirkin
Quoted 18 lines · Andreas Färber
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 6 lines
Quoted 3 lines
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Backwards and forwards migration need to be supported, customers told us repeatedly.
Can I have world peace and a pony with that?
Given the current state of things, attempting to support backward migration is trying to run before you can walk. We need to put migration on a more solid footing first.
The migration format is crap, and needs to be replaced.
Reasoning on migration compatibility is entirely manual.
Systematic testing of migration compatibility is done downstream.
Fortunately, there's progress being made on all of the above. Let's not sabotage it by biting off yet another mouthful.
Quoted 9 lines
So some downstreams support this
and not supporting it upstream just means downstreams need to do their own thing.As importantly, ping-pong migration is the only
reliable way to stress migration.So if we want to test cross-version we need it to work both way.
Non sequitur.
Quoted 4 lines
Finally, the real issue and difficulty with cross-version migration is making VM behave in a backwards compatible way. Serializing in a compatible way is a trivial problem, or would be if the code wasn't a mess :)
Quoted 8 lines
However, it is.
Quoted 4 lines
Once you do the hard part, breaking migration because of the
trivial serialization issue is just silly. And special-casing forward migration does not make code simpler, it really only leads to proliferation of hacks and lack of symmetry.Bold claim; citation needed.
You are asking for examples of ugly assymetry?
It's easy: grep for .load_state_old.
You have here a bunch of functions loading format that qemu
can no longer produce, with any set of flags.
The only way to make them run is to install two qemu versions side by side, save from old one and load in the new one.
What, would you guess, is the chance that they actually work?I'm going to send a patch removing all this stuff, it's effectively dead code, but this is just one, biggest example.
Quoted 7 lines
Quoted 2 lines
So yes it's a useful feature, and no not supporting it does not help anyway.
Nobody denies reliable backward migration would be useful. However, attempting to do every useful feature at once just because they're all useful is foolish.
Quoted 6 lines
Treating backward migration as strictly secondary concern while we're up to the ass in other alligators *can* help, by letting us focus on the said other alligators.
I'm not opposed to coding things in ways that help backward migration. Speaking of "support", however, is clearly premature and misleading.
Quoted 8 lines
I agree it's a secondary concern upstream and your comments really apply to migration generally. We can't claim that it's properly supported by the upstream QEMU.
I am merely asking that we don't break cross-version migration intentionally. I and others also try fix it when we notice it's broken. In particular I'm not asking that submitters test it.
Michael,
I fully understand your concern and I will take some time to study backward migration. This being said, the task is not that simple and I will need several rounds to have the job done...
Cheers.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Andreas FärberQuoted 14 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM state
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.
That makes a lot of sense.
Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
Quoted 7 lines
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
IIRW the "broken" state was proposed as a per-device property...
Fam,
Do you have plans about the "broken" property ? Is it still needed ?
Quoted 2 lines
Would touch on Stefan's alias properties for anything but virtio-mmio.
OMG... maybe I should hold on then.
Quoted 3 lines
Regards,
Andreas
Thanks !
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 69 lines · Greg Kurz
Quoted 14 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.
That makes a lot of sense.
Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
Quoted 7 lines
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
IIRW the "broken" state was proposed as a per-device property...
Quoted 8 lines
Fam,
Do you have plans about the "broken" property ? Is it still needed ?
Quoted 2 lines
Would touch on Stefan's alias properties for anything but virtio-mmio.
OMG... maybe I should hold on then.
No need to wait imho.
Can this be made even simpler - call this stuff
from virtio_save/virtio_load?
Why not?
Quoted 14 lines
Quoted 3 lines
Regards,
AndreasThanks !
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 74 lines · On Thu, 15 May 2014 13:12:12 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 73 lines · Greg Kurz
Quoted 14 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 13 lines
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 10 lines · Michael S. Tsirkin
Quoted 9 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM stateQuoted 8 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.
That makes a lot of sense.
Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
Quoted 7 lines
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
IIRW the "broken" state was proposed as a per-device property...
Fam,
Do you have plans about the "broken" property ? Is it still needed ?
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
Would touch on Stefan's alias properties for anything but virtio-mmio.
OMG... maybe I should hold on then.
No need to wait imho.
Can this be made even simpler - call this stuff
from virtio_save/virtio_load?
Andreas already suggested this inversion of control.
Quoted 2 lines
Why not?
No reason indeed. I'll rewrite the code that way ! :)
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Regards,
AndreasThanks !
Quoted 9 lines
Quoted 8 lines
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Thnaks !
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Greg KurzQuoted 20 lines · On Thu, 15 May 2014 12:08:26 +0200 Greg Kurz <[email protected]> wrote:
Quoted 18 lines · On Thu, 15 May 2014 11:20:18 +0200 Andreas Färber <[email protected]> wrote:
Am 15.05.2014 09:04, schrieb Greg Kurz:
Quoted 16 lines
Quoted 14 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Amit Shah
Quoted 7 lines · Greg Kurz
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 9 lines
Quoted 4 lines
Since backwards migration is not supported upstream, wouldn't it be easiest to just add support for the subsection marker and skipping to the end of section in that downstream?
Not sure I understand well... Do you suggest to stream the markers first, then the device, then the subsections ? And then there would be a way we can have the subsections restored before the device ?
Heh just got the clarification thanks to Michael's mail... Sorry for the confusion and noise. :P
Quoted 55 lines
Quoted 8 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
IMO this is calling for inversion of control - i.e. let virtio devices call generic load/save functions that then dispatch to device-specific code and let us add common stuff in a central place without forgetting to add calls in some new device.
That makes a lot of sense.
Quoted 19 lines
Quoted 14 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
Quoted 7 lines
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
IIRW the "broken" state was proposed as a per-device property...
Fam,
Do you have plans about the "broken" property ? Is it still needed ?
Quoted 2 lines
Would touch on Stefan's alias properties for anything but virtio-mmio.
OMG... maybe I should hold on then.
Quoted 3 lines
Regards,
AndreasThanks !
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Greg KurzQuoted 13 lines · Greg Kurz
Quoted 5 lines
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
Quoted 7 lines
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
Quoted 1 line
IIRW the "broken" state was proposed as a per-device property...
Yes.
Quoted 4 lines
Fam,
Do you have plans about the "broken" property ? Is it still needed ?
It is on my todo list. We will need it. I hope the next revision of virtio specification has a proper device status bit that the "broken" property can utilize.
Thanks,
Fam
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Am 16.05.2014 11:14, schrieb Fam Zheng:
Quoted 14 lines · Greg Kurz
Quoted 13 lines
Quoted 12 lines
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
Quoted 3 lines
Quoted 1 line
IIRW the "broken" state was proposed as a per-device property...
Yes.
Sure, and that makes sense, but we do have a 1:1 relation of bus/device, or does virtio-mmio support more? If device doesn't work for some reason, we could (mis)use the bus as fallback then.
Cheers,
Andreas
Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 15 lines · Andreas Färber
Am 16.05.2014 11:14, schrieb Fam Zheng:
Quoted 14 lines · Greg Kurz
Quoted 13 lines
Quoted 12 lines
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
IIRW the "broken" state was proposed as a per-device property...
Yes.
Sure, and that makes sense, but we do have a 1:1 relation of bus/device, or does virtio-mmio support more? If device doesn't work for some reason, we could (mis)use the bus as fallback then.
FWIW, I just realized that "broken" may be loaded from device status bit in the future, so we don't need to migrate the field separately:
https://tools.oasis-open.org/issues/browse/VIRTIO-98
Fam
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 25 lines · On Fri, 16 May 2014 17:40:00 +0800 Fam Zheng <[email protected]> wrote:
Quoted 15 lines · Andreas Färber
Am 16.05.2014 11:14, schrieb Fam Zheng:
Quoted 14 lines · Greg Kurz
Quoted 13 lines
Quoted 12 lines
Quoted 4 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks... I need to study how the virtio-bus gets migrated to ensure the endian section is streamed before the devices.
There is no ordering guarantee. The state needs to be migrated in the device or bus where it sits, if post-load processing is required; i.e., if it's in VirtIODevice then something like this series, if it were on VirtioBus exclusively (device asking bus for its endianness each time and does not do post-load stuff) then endianness could be migrated as a new bus section. Not sure if that would help the "broken" state though?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
IIRW the "broken" state was proposed as a per-device property...
Yes.
Sure, and that makes sense, but we do have a 1:1 relation of bus/device, or does virtio-mmio support more? If device doesn't work for some reason, we could (mis)use the bus as fallback then.
Quoted 7 lines
FWIW, I just realized that "broken" may be loaded from device status bit in the future, so we don't need to migrate the field separately:
https://tools.oasis-open.org/issues/browse/VIRTIO-98
Fam
Oh, if you are targetting virtio 1.0... all the virtio endian stuff I am working on is for legacy virtio. Unless you plan to do something for legacy, I guess our objectives differ.
Cheers.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Greg KurzQuoted 47 lines · Greg Kurz
Quoted 11 lines · On Thu, 15 May 2014 12:16:35 +0530 Amit Shah <[email protected]> wrote:
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Quoted 2 lines
BTW Greg, do you plan on working on vmstate for virtio?
Yes.
Quoted 19 lines
Quoted 19 lines
Quoted 18 lines
Quoted 7 lines
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
Quoted 9 lines
Quoted 3 lines
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Quoted 3 lines
The main problem I see is that virtio sucks: as you see in patch 8, we have to be careful not to call vring or virtqueue stuff before the device knows its endianness or it breaks...
I see.
I think it's all not a big deal.
People here suggested many ways to deal with it, but IMHO the only thing that does matter is the functionality.
Functionality-wise I think the only two things that were mentioned were
- decent chance that migration to a wrong machine
version fails gracefully
- migrate in a compatible way with correct legacy machine version
Quoted 16 lines
I need to study how the virtio-bus gets
migrated to ensure the endian section is streamed before the devices.Quoted 2 lines
Amit
Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Amit ShahQuoted 36 lines · Amit Shah
Quoted 9 lines · Michael S. Tsirkin
Quoted 8 lines · Amit Shah
Quoted 7 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
BTW Greg, do you plan on working on vmstate for virtio?
Quoted 15 lines
Quoted 15 lines
Quoted 7 lines
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Quoted 7 lines
Quoted 3 lines
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
This calls for a way for devices to inherit properties from the bus, which doesn't exist ATM.
Fine but let's not hold up this patchset because of this.
Quoted 6 lines
No, only suggestion is to add a migration section in the bus, and then it's easier to do this in the post-migrate functions for each device -- so only one new section gets introduced instead of all devices being modified to send a new subsection.
Amit
I don't mind but the gain isn't very big here.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Amit ShahQuoted 14 lines · On Thu, 15 May 2014 11:34:25 +0530 Amit Shah <[email protected]> wrote:
Quoted 13 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Quoted 7 lines
Quoted 2 lines
Unknown savevm section type 5
Error -22 while loading VM statePlease make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Hmmm... you mean we support migration from a newer QEMU to an older one ?
Quoted 7 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
That would be nice if possible. I will have a closer look.
Quoted 2 lines
Amit
Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
Quoted 20 lines · Greg Kurz
Quoted 17 lines · On Thu, 15 May 2014 11:34:25 +0530 Amit Shah <[email protected]> wrote:
Quoted 15 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM state
Quoted 6 lines
Quoted 4 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Hmmm... you mean we support migration from a newer QEMU to an older one ?
Well, not really "support", since many things don't work anyway, but let's make that easier on downstreams that may want to.
Quoted 9 lines
Quoted 7 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
That would be nice if possible. I will have a closer look.
Thanks!
Amit
Re: [Qemu-devel] [PATCH RFC 1/8] virtio: add subsections to the migration stream
In reply to Greg KurzQuoted 20 lines · Greg Kurz
Quoted 17 lines · On Thu, 15 May 2014 11:34:25 +0530 Amit Shah <[email protected]> wrote:
Quoted 15 lines · Greg Kurz
There is a need to add some more fields to VirtIODevice that should be migrated (broken status, endianness). The problem is that we do not want to break compatibility while adding a new feature... This issue has been addressed in the generic VMState code with the use of optional subsections. As a *temporary* alternative to port the whole virtio migration code to VMState, this patch mimics a similar subsectionning ability for virtio.
Since each virtio device is streamed in its own section, the idea is to stream subsections between the end of the device section and the start of the next sections. This allows an older QEMU to complain and exit when fed with subsections:
Unknown savevm section type 5
Error -22 while loading VM state
Quoted 6 lines
Quoted 4 lines
Please make this configurable -- either via configure or device properties. That avoids having to break existing configurations that work without this patch.
Hmmm... you mean we support migration from a newer QEMU to an older one ?
In theory yes, that's why we have the -M switch. This area isn't well tested unfortunately, but there are downstreams that test and productize it.
Quoted 23 lines
Quoted 7 lines
Quoted 2 lines
All users of virtio_load()/virtio_save() need to be patched because the subsections are streamed AFTER the device itself.
Since all have the same fixup, I'm wondering if a new section can be added to the virtio-bus itself, which gets propagated to all devices upon load in the dest.
That would be nice if possible. I will have a closer look.
Quoted 2 lines
Amit
Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
RFC PATCH 2/8 virtio-net: migrate subsections
In reply to Greg Kurz--- hw/net/virtio-net.c | 7 +++++++ 1 file changed, 7 insertions(+)
hw/net/virtio-net.c +7 -0
@@ -1300,6 +1300,8 @@ static void virtio_net_save(QEMUFile *f, void *opaque)if ((1 <<VIRTIO_NET_F_CTRL_GUEST_OFFLOADS ) & vdev->guest_features) {qemu_put_be64 (f, n->curr_guest_offloads); } + +virtio_save_subsections (vdev, f); }static int virtio_net_load (QEMUFile *f,void *opaque,int version_id)
@@ -1396,6 +1398,11 @@ static int virtio_net_load(QEMUFile *f, void *opaque, int version_id) n->curr_guest_offloads =virtio_net_supported_guest_offloads (n); } + ret =virtio_load_subsections (vdev, f); +if (ret) { +return ret; + } +if (peer_has_vnet_hdr (n)) {virtio_net_apply_guest_offloads (n); }
RFC PATCH 3/8 virtio-blk: migrate subsections
In reply to Greg Kurz--- hw/block/virtio-blk.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
hw/block/virtio-blk.c +3 -1
@@ -600,6 +600,8 @@ static void virtio_blk_save(QEMUFile *f, void *opaque) req = req->next; }qemu_put_sbyte (f,0 ); + +virtio_save_subsections (vdev, f); }static int virtio_blk_load (QEMUFile *f,void *opaque,int version_id)
@@ -628,7 +630,7 @@ static int virtio_blk_load(QEMUFile *f, void *opaque, int version_id) req->elem.out_num, 0); } -return 0 ; +return virtio_load_subsections (vdev, f); }static void virtio_blk_resize (void *opaque)
RFC PATCH 4/8 virtio-scsi: migrate subsections
In reply to Greg Kurz--- hw/scsi/virtio-scsi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
hw/scsi/virtio-scsi.c +2 -1
@@ -471,6 +471,7 @@ static void virtio_scsi_save(QEMUFile *f, void *opaque) {VirtIODevice *vdev =VIRTIO_DEVICE (opaque);virtio_save (vdev, f); +virtio_save_subsections (vdev, f); }static int virtio_scsi_load (QEMUFile *f,void *opaque,int version_id)
@@ -482,7 +483,7 @@ static int virtio_scsi_load(QEMUFile *f, void *opaque, int version_id)if (ret) {return ret; } -return 0 ; +return virtio_load_subsections (vdev, f); } static void virtio_scsi_push_event(VirtIOSCSI *s, SCSIDevice *dev,
RFC PATCH 5/8 virtio-serial: migrate subsections
In reply to Greg Kurz--- hw/char/virtio-serial-bus.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
hw/char/virtio-serial-bus.c +7 -3
@@ -522,12 +522,13 @@ static void vser_reset(VirtIODevice *vdev)static void virtio_serial_save (QEMUFile *f,void *opaque) {VirtIOSerial *s =VIRTIO_SERIAL (opaque); +VirtIODevice *vdev =VIRTIO_DEVICE (s);VirtIOSerialPort *port;uint32_t nr_active_ports;unsigned int i, max_nr_ports;/* The virtio device */ -virtio_save (VIRTIO_DEVICE (s), f); +virtio_save (vdev, f);/* The config space */ qemu_put_be16s (f, &s->config.cols);
@@ -573,6 +574,8 @@ static void virtio_serial_save(QEMUFile *f, void *opaque) sizeof(port->elem)); } } + +virtio_save_subsections (vdev, f); }static void virtio_serial_post_load_timer_cb (void *opaque)
@@ -667,6 +670,7 @@ static int fetch_active_ports_list(QEMUFile *f, int version_id,static int virtio_serial_load (QEMUFile *f,void *opaque,int version_id) {VirtIOSerial *s =VIRTIO_SERIAL (opaque); +VirtIODevice *vdev =VIRTIO_DEVICE (s);uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret;
@@ -676,7 +680,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) }/* The virtio device */ - ret =virtio_load (VIRTIO_DEVICE (s), f); + ret =virtio_load (vdev, f);if (ret) {return ret; }
@@ -716,7 +720,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return ret; } } -return 0 ; +return virtio_load_subsections (vdev, f); }static void virtser_bus_dev_print (Monitor *mon,DeviceState *qdev,int indent);
RFC PATCH 6/8 virtio-balloon: migrate subsections
In reply to Greg Kurz--- hw/virtio/virtio-balloon.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
hw/virtio/virtio-balloon.c +3 -1
@@ -316,6 +316,8 @@ static void virtio_balloon_save(QEMUFile *f, void *opaque)qemu_put_be32 (f, s->num_pages);qemu_put_be32 (f, s->actual); + +virtio_save_subsections (vdev, f); }static int virtio_balloon_load (QEMUFile *f,void *opaque,int version_id)
@@ -334,7 +336,7 @@ static int virtio_balloon_load(QEMUFile *f, void *opaque, int version_id) s->num_pages =qemu_get_be32 (f); s->actual =qemu_get_be32 (f); -return 0 ; +return virtio_load_subsections (vdev, f); }static void virtio_balloon_device_realize (DeviceState *dev,Error **errp)
RFC PATCH 7/8 virtio-rng: migrate subsections
In reply to Greg KurzWhile we are here, we also check virtio_load() return value.
--- hw/virtio/virtio-rng.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
hw/virtio/virtio-rng.c +12 -1
@@ -103,17 +103,28 @@ static void virtio_rng_save(QEMUFile *f, void *opaque)VirtIODevice *vdev = opaque;virtio_save (vdev, f); +virtio_save_subsections (vdev, f); }static int virtio_rng_load (QEMUFile *f,void *opaque,int version_id) {VirtIORNG *vrng = opaque;VirtIODevice *vdev =VIRTIO_DEVICE (vrng); +int ret;if (version_id !=1 ) {return -EINVAL ; } -virtio_load (vdev, f); + + ret =virtio_load (vdev, f); +if (ret) { +return ret; + } + + ret =virtio_load_subsections (vdev, f); +if (ret) { +return ret; + }/* We may have an element ready but couldn't process it due to a quota * limit. Make sure to try again after live migration when the quota may
RFC PATCH 8/8 virtio: add endian-ambivalent support to VirtIODevice
In reply to Greg KurzSome CPU families can dynamically change their endianness. This means we can have little endian ppc or big endian arm guests for example. This has an impact on legacy virtio data structures since they are target endian. We hence introduce a new property to track the endianness of each virtio device. It is reasonnably assumed that endianness won't change while the device is in use : we hence capture the device endianness when it gets reset.
Of course this property must be part of the migration stream as most of the virtio code will depend on it. It is hence migrated in a subsection to preserve compatibility with older migration streams. The tricky part is that the endianness gets known quite late and we must ensure no access is made to virtio data before that. It is for example invalid to call vring_avail_idx() as does the actual migration code: the VQ indexes sanity checks had to be moved from virtio_load() to virtio_load_subsections() because of that.
In fact, we have two conditions where the endianness property is stale:
- from initialization time (virtio_init) to first reset time (virtio_reset)
- from incoming migration time (virtio_load) to the time the property is restored (virtio_load_device_endian)
We enforce some paranoia by making the endianness a 3-value enum so that we can poison it when it should not be used.
--- exec.c | 8 +----- hw/virtio/virtio-pci.c | 11 +++----- hw/virtio/virtio.c | 64 ++++++++++++++++++++++++++++++++++++-------- include/exec/cpu-common.h | 1 + include/hw/virtio/virtio.h | 13 +++++++++ 5 files changed, 72 insertions(+), 25 deletions(-)
exec.c +1 -7
@@ -2740,13 +2740,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,#endif #if !defined(CONFIG_USER_ONLY ) - -/* -* A helper function for the _utterly broken_ virtio device model to find out if -* it's running on a big endian machine. Don't do this at home kids! -*/ -bool virtio_is_big_endian (void ); -bool virtio_is_big_endian (void ) +bool target_words_bigendian (void ) {#if defined(TARGET_WORDS_BIGENDIAN )return true;
hw/virtio/virtio-pci.c +4 -7
@@ -89,9 +89,6 @@/* Flags track per-device state like workarounds for quirks in older guests. */ #define VIRTIO_PCI_FLAG_BUS_MASTER_BUG (1 << 0) -/* HACK for virtio to determine if it's running a big endian guest */ -bool virtio_is_big_endian (void ); -static void virtio_pci_bus_new (VirtioBusState *bus,size_t bus_size,VirtIOPCIProxy *dev);
@@ -409,13 +406,13 @@ static uint64_t virtio_pci_config_read(void *opaque, hwaddr addr,break ;case 2 : val =virtio_config_readw (vdev, addr); -if (virtio_is_big_endian ()) { +if (virtio_is_big_endian (vdev)) { val =bswap16 (val); }break ;case 4 : val =virtio_config_readl (vdev, addr); -if (virtio_is_big_endian ()) { +if (virtio_is_big_endian (vdev)) { val =bswap32 (val); }break ;
@@ -443,13 +440,13 @@ static void virtio_pci_config_write(void *opaque, hwaddr addr,virtio_config_writeb (vdev, addr, val);break ;case 2 : -if (virtio_is_big_endian ()) { +if (virtio_is_big_endian (vdev)) { val =bswap16 (val); }virtio_config_writew (vdev, addr, val);break ;case 4 : -if (virtio_is_big_endian ()) { +if (virtio_is_big_endian (vdev)) { val =bswap32 (val); }virtio_config_writel (vdev, addr, val);
hw/virtio/virtio.c +53 -11
113 lines not on this page · raw message
include/exec/cpu-common.h +1 -0
@@ -122,4 +122,5 @@ void qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);#endif +bool target_words_bigendian (void );#endif /* !CPU_COMMON_H */
include/hw/virtio/virtio.h +13 -0
@@ -104,6 +104,12 @@ typedef struct VirtQueueElement#define VIRTIO_DEVICE (obj) \ OBJECT_CHECK(VirtIODevice, (obj), TYPE_VIRTIO_DEVICE) +enum virtio_device_endian { +VIRTIO_DEVICE_ENDIAN_UNKNOWN , +VIRTIO_DEVICE_ENDIAN_LITTLE , +VIRTIO_DEVICE_ENDIAN_BIG , +}; +struct VirtIODevice {DeviceState parent_obj;
@@ -121,6 +127,7 @@ struct VirtIODevicebool vm_running;VMChangeStateEntry *vmstate;char *bus_name; +enum virtio_device_endian device_endian; }; typedef struct VirtioDeviceClass {
@@ -257,4 +264,10 @@ void virtio_queue_set_host_notifier_fd_handler(VirtQueue *vq, bool assign, bool set_handler);void virtio_queue_notify_vq (VirtQueue *vq);void virtio_irq (VirtQueue *vq); + +static inline bool virtio_is_big_endian (VirtIODevice *vdev) +{ +assert (vdev->device_endian !=VIRTIO_DEVICE_ENDIAN_UNKNOWN ); +return vdev->device_endian ==VIRTIO_DEVICE_ENDIAN_BIG ; +}#endif
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows;
qemu_get_be16s(f, &cols);
qemu_get_be16s(f, &rows);
qemu_get_be32s(f, &max_nr_ports);
/* Convert back to target endianness when storing into the config
* space.
*/Paolo,
The patch set to support endian changing targets adds a device_endian field to the VirtIODevice structure to be used instead of the default target endianness as it happens with tswap() macros. It also introduces virtio_tswap() helpers for this purpose, but they can only be used when the device_endian field has been restored... in a subsection after the device descriptor... :-\
If the scenario is ppc64le-on-ppc64: tswap() macros don't do anything and we cannot convert back to LE...
Quoted 2 lines
s->config.cols = tswap16(cols);
s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) {
...
}Since we know that 0 < max_nr_ports < 32, is it acceptable to guess the correct endianness with a heuristic ?
if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
max_nr_ports = bswap32(max_nr_ports);
}if (max_nr_ports > tswap32(s->config.max_nr_ports)) {
return -EINVAL;
}Quoted 10 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Paolo
Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 78 lines · Greg Kurz
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */
Quoted 8 lines
Paolo,
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\
Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 29 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 4 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
Quoted 8 lines
Quoted 6 lines
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Paolo
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
Quoted 8 lines
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 81 lines · On Thu, 12 Jun 2014 10:54:48 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 80 lines · Greg Kurz
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */Paolo,
Quoted 8 lines
Quoted 6 lines
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\Store it earlier then, using plain put/get.
Not sure I follow... this will break compatibility, no ?
Quoted 39 lines
You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 36 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 10 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Paolo
Quoted 7 lines
Quoted 1 line
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
Hmmm... AFAIC some QEMU maintainers expressed interest in supporting legacy virtio in the case we have endian-changing targets. Patches to run a ppc64le guests have been accepted in KVM, Linux and QEMU... the only missing block is virtio. I don't especially care in supporting old drivers at all cost: this request was expressed on the list. I just want people to be able to run a ppc64le ubuntu-14.04 (and soon other distros) guest on a ppc64 box and be able to migrate.
Would it be acceptable to break compatibility for ppc64 (and maybe ARM) only with a target specific hook being called from the virtio code ?
Quoted 10 lines
Quoted 9 lines
Signature · 8 lines
-- Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 90 lines · Greg Kurz
Quoted 88 lines · On Thu, 12 Jun 2014 10:54:48 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 86 lines · Greg Kurz
Quoted 66 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */Paolo,
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\
Quoted 3 lines
Quoted 1 line
Store it earlier then, using plain put/get.
Not sure I follow... this will break compatibility, no ?
Do it only on the ambialent targets :)
Quoted 47 lines
Quoted 39 lines
You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 36 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 10 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Paolo
Quoted 7 lines
Quoted 1 line
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
Quoted 6 lines
Hmmm... AFAIC some QEMU maintainers expressed interest in supporting legacy virtio in the case we have endian-changing targets. Patches to run a ppc64le guests have been accepted in KVM, Linux and QEMU... the only missing block is virtio. I don't especially care in supporting old drivers at all cost: this request was expressed on the list. I just want people to be able to run a ppc64le ubuntu-14.04 (and soon other distros) guest on a ppc64 box and be able to migrate.
Sigh. So ubuntu ships known-broken virtio drivers in their LTS release and now we are trying to make them work.
There's no working qemu configuration at the moment, do we really have to build new hardware around known-broken drivers
and then maintain it all for years?
Why not build good hardware and backport new drivers?
Quoted 2 lines
Would it be acceptable to break compatibility for ppc64 (and maybe ARM) only with a target specific hook being called from the virtio code ?
I think so.
Can we just add a function that tells us whether target is ambivalent? Only migrate new stuff if that's the case.
Quoted 21 lines
Quoted 10 lines
Quoted 9 lines
Signature · 8 lines
-- Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.Thanks.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Greg KurzQuoted 137 lines · Greg Kurz
Quoted 78 lines · On Thu, 12 Jun 2014 10:54:48 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 77 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */Paolo,
Quoted 7 lines
Quoted 6 lines
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\Store it earlier then, using plain put/get.
Not sure I follow... this will break compatibility, no ?
Quoted 33 lines
You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 30 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 7 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Quoted 8 lines
Quoted 3 lines
Quoted 2 lines
Paolo
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
Hmmm... AFAIC some QEMU maintainers expressed interest in supporting legacy virtio in the case we have endian-changing targets. Patches to run a ppc64le guests have been accepted in KVM, Linux and QEMU... the only missing block is virtio. I don't especially care in supporting old drivers at all cost: this request was expressed on the list. I just want people to be able to run a ppc64le ubuntu-14.04 (and soon other distros) guest on a ppc64 box and be able to migrate.
Quoted 2 lines
Would it be acceptable to break compatibility for ppc64 (and maybe ARM) only with a target specific hook being called from the virtio code ?
I don't see the problem. Let's just make the config fields UNUSED in vmstate and keep the virtio-serial config space in good endian locally. Then you don't care about ordering of sections anymore.
Then we only need the subsection code for "endian is not target endian" which makes migration backwards compatible with non-switchable QEMU versions and everyone's happy. End of story :).
Alex
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Michael S. TsirkinQuoted 118 lines · Michael S. Tsirkin
Quoted 75 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */Quoted 8 lines
Paolo,
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 30 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 7 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Paolo
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
There are already released and working Linux distributions (Ubuntu, openSUSE, maybe others) that don't have virtio 1.0 drivers. Putting our heads into the sand is not an option ;).
Alex
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 128 lines · Alexander Graf
Quoted 118 lines · Michael S. Tsirkin
Quoted 75 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */Quoted 8 lines
Paolo,
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 30 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 7 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Paolo
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
Quoted 6 lines
There are already released and working Linux distributions (Ubuntu, openSUSE, maybe others) that don't have virtio 1.0 drivers. Putting our heads into the sand is not an option ;).
Alex
I don't get it. Does virtio work there at the moment?
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 127 lines · Michael S. Tsirkin
Quoted 126 lines · Alexander Graf
Quoted 125 lines · Michael S. Tsirkin
Quoted 75 lines · Greg Kurz
Quoted 64 lines · On Thu, 29 May 2014 12:16:26 +0200 Paolo Bonzini <[email protected]> wrote:
Il 29/05/2014 11:12, Greg Kurz ha scritto:
Quoted 23 lines
int
virtio_load(VirtIODevice *vdev, QEMUFile *f)
{
[...]nheads = vring_avail_idx(&vdev->vq[i]) - vdev->vq[i].last_avail_idx; ^^^^^^^^^^^ /* Check it isn't doing very strange things with descriptor numbers. */ if (nheads > vdev->vq[i].vring.num) {[...]
}and
static int
virtio_serial_load(QEMUFile *f, void *opaque, int version_id){
[...]/* The config space */ qemu_get_be16s(f, &s->config.cols); qemu_get_be16s(f, &s->config.rows); qemu_get_be32s(f, &max_nr_ports); tswap32s(&max_nr_ports); ^^^^^^ if (max_nr_ports > tswap32(s->config.max_nr_ports)) {Quoted 8 lines
[...]
}If we stream subsections after the device descriptor as it is done in VMState, these two will break because the device endian is stale.
The first one can be easily dealt with: just defer the sanity check to a
post_loadfunction.Good, we're lucky here.
Quoted 5 lines
The second is a bit more tricky: the
virtio serial migrates its config space (target endian) and the active ports bitmap. The load code assumesmax_nr_portsfrom the config space tells the size of the ports bitmap... that means the virtio migration protocol is also contaminated by target endianness. :-\Ouch.
I guess we could break migration in the case of host endianness != target endianness, like this:
/* These three used to be fetched in target endianness and then * stored as big endian. It ended up as little endian if host and * target endianness doesn't match. * * Starting with qemu 2.1, we always store as big endian. The * version wasn't bumped to avoid breaking backwards compatibility. * We check the validity of max_nr_ports, and the incorrect- * endianness max_nr_ports will be huge, which will abort migration * anyway. */ uint16_t cols = tswap16(s->config.cols); uint16_t rows = tswap16(s->config.rows); uint32_t max_nr_ports = tswap32(s->config.max_nr_ports); qemu_put_be16s(f, &cols); qemu_put_be16s(f, &rows); qemu_put_be32s(f, &max_nr_ports);...
Quoted 9 lines
uint16_t cols, rows; qemu_get_be16s(f, &cols); qemu_get_be16s(f, &rows); qemu_get_be32s(f, &max_nr_ports); /* Convert back to target endianness when storing into the config * space. */Quoted 8 lines
Paolo,
The patch set to support endian changing targets adds a
device_endianfield to the VirtIODevice structure to be used instead of the default target endianness as it happens withtswap()macros. It also introducesvirtio_tswap()helpers for this purpose, but they can only be used when thedevice_endianfield has been restored... in a subsection after the device descriptor... :-\Store it earlier then, using plain put/get. You can still add a section conditionally to cause a cleaner failure in broken cross-version scenarios.
Quoted 26 lines
If the scenario is ppc64le-on-ppc64:
tswap()macros don't do anything and we cannot convert back to LE...Quoted 2 lines
s->config.cols = tswap16(cols); s->config.rows = tswap16(rows);Since cols and rows are not involved in the protocol, we can safely defer the conversion to post load.
Quoted 4 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports) { ... }Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 3 lines
Quoted 3 lines
In the case the answer for above is "legacy virtio really sucks" then, is it acceptable to not honor bug-compatibility with older versions and fix the code ? :)
Quoted 7 lines
Quoted 6 lines
As long as the common cases don't break, yes. The question is what are the common cases. Here I think the only non-obscure case that could break is x86-on-PPC, and it's not that common.
Paolo
Thanks.
One starts doubting whether all this hackery is worth it. virtio 1.0 should be out real soon now, it makes everything LE so the problem goes away. It's not like PPC LE is so popular that we must support old drivers at all costs. Won't time be better spent backporting virtio 1.0 drivers?
Quoted 7 lines
Quoted 6 lines
There are already released and working Linux distributions (Ubuntu, openSUSE, maybe others) that don't have virtio 1.0 drivers. Putting our heads into the sand is not an option ;).
Alex
I don't get it. Does virtio work there at the moment?
With the LE enable patch, yes, sure. Imagine the same would happen for Windows and e1000 - would you still argue the same way?
Alex
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Greg KurzIl 12/06/2014 09:43, Greg Kurz ha scritto:
Quoted 10 lines
Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }
Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and virtio_tswap on top of the previous change.
Paolo
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 8 lines · Paolo Bonzini
Il 12/06/2014 09:43, Greg Kurz ha scritto:
Quoted 7 lines
Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }
Quoted 5 lines
Quoted 3 lines
if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and
virtio_tswapon top of the previous change.
Why don't we just ignore those config fields? They won't change during the lifetime of the guest anyway - and configuration needs to be the same on source and dest to work.
Alex
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Paolo BonziniQuoted 12 lines · On Thu, 12 Jun 2014 10:55:42 +0200 Paolo Bonzini <[email protected]> wrote:
Il 12/06/2014 09:43, Greg Kurz ha scritto:
Quoted 10 lines
Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }
Quoted 6 lines
Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and
virtio_tswapon top of the previous change.Paolo
BTW, can someone explain why we stream the device config ?
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Il 12/06/2014 11:06, Greg Kurz ha scritto:
Quoted 15 lines
Quoted 12 lines · On Thu, 12 Jun 2014 10:55:42 +0200 Paolo Bonzini <[email protected]> wrote:
Il 12/06/2014 09:43, Greg Kurz ha scritto:
Quoted 10 lines
Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }
Quoted 5 lines
Quoted 3 lines
Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and
virtio_tswapon top of the previous change.BTW, can someone explain why we stream the device config ?
For max_nr_ports it is useless indeed. Let's keep storing it for backwards compatibility, but you can remove its load.
The cols and rows values should be defined by the host and thus could even change on migration, there is no need to store them. As of now, however, they are unused in QEMU and should always be zero, because VIRTIO_CONSOLE_F_SIZE is not supported.
Paolo
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 25 lines · Paolo Bonzini
Il 12/06/2014 11:06, Greg Kurz ha scritto:
Quoted 15 lines
Quoted 12 lines · On Thu, 12 Jun 2014 10:55:42 +0200 Paolo Bonzini <[email protected]> wrote:
Il 12/06/2014 09:43, Greg Kurz ha scritto:
Quoted 10 lines
Since we know that 0 <
max_nr_ports< 32, is it acceptable to guess the correct endianness with a heuristic ?if (max_nr_ports > tswap32(s->config.max_nr_ports)) { max_nr_ports = bswap32(max_nr_ports); }if (max_nr_ports > tswap32(s->config.max_nr_ports)) { return -EINVAL; }Quoted 5 lines
Quoted 3 lines
Yes, I guess it is acceptable. So first you should fix the code to always serialize fields as big-endian, and then apply this little hack and
virtio_tswapon top of the previous change.BTW, can someone explain why we stream the device config ?
For
max_nr_portsit is useless indeed. Let's keep storing it for backwards compatibility, but you can remove its load.
Quoted 6 lines
The cols and rows values should be defined by the host and thus could even change on migration, there is no need to store them. As of now, however, they are unused in QEMU and should always be zero, because
VIRTIO_CONSOLE_F_SIZEis not supported.Paolo
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
Signature · 1 line
MST
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
Paolo
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 5 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +6 -10
@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL ;
@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ -return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp); + max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 49 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 6 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +0
-8@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */
hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697
Quoted 10 lines
hw/char/virtio-serial-bus.c +5
-2-return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp); + max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);
For the moment, we have 0 < max_nr_ports < 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machine max_nr_ports value can give the information...
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 61 lines
Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:
Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +4
-10@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ -return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp);
Quoted 9 lines
hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696
Quoted 3 lines
hw/char/virtio-serial-bus.c +1
-0+ max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);For the moment, we have 0 <
max_nr_ports< 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machinemax_nr_portsvalue can give the information...
Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.
Alex
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 71 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 61 lines
Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:
Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +4
-10@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ -return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp);Quoted 9 lines
hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696
Quoted 3 lines
hw/char/virtio-serial-bus.c +1
-0+ max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);For the moment, we have 0 <
max_nr_ports< 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machinemax_nr_portsvalue can give the information...
Quoted 2 lines
Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.
I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 78 lines
Am 12.06.2014 um 12:50 schrieb Greg Kurz <[email protected]>:
Quoted 71 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 70 lines
Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:
Quoted 47 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines
Quoted 5 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +0
-8@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697
Quoted 10 lines
hw/char/virtio-serial-bus.c +5
-2-return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp); + max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);For the moment, we have 0 <
max_nr_ports< 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machinemax_nr_portsvalue can give the information...
Quoted 6 lines
Quoted 1 line
Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.
I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?
The destination has be be executed in compat mode to the older qemu version via a versioned machine type. This should ensure that limits are kept.
Alex
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Greg KurzQuoted 74 lines · Greg Kurz
Quoted 71 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 70 lines
Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:
Quoted 47 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +0
-8@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697
Quoted 10 lines
hw/char/virtio-serial-bus.c +5
-2-return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp); + max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);For the moment, we have 0 <
max_nr_ports< 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machinemax_nr_portsvalue can give the information...
Quoted 7 lines
Quoted 2 lines
Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.
I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?
When the destination machine is started with -M 2.1, it knows that it has to read 32 bit.
If started with -M 3.0 it reads in 42 bits :)
Quoted 8 lines
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
Quoted 78 lines · On Thu, 12 Jun 2014 13:59:59 +0300 "Michael S. Tsirkin" <[email protected]> wrote:
Quoted 77 lines · Greg Kurz
Quoted 73 lines · On Thu, 12 Jun 2014 12:39:27 +0200 Alexander Graf <[email protected]> wrote:
Quoted 61 lines
Am 12.06.2014 um 12:14 schrieb Greg Kurz <[email protected]>:
Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 8 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +4
-10@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ -return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp);Quoted 9 lines
hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696
Quoted 3 lines
hw/char/virtio-serial-bus.c +1
-0+ max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);For the moment, we have 0 <
max_nr_ports< 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machinemax_nr_portsvalue can give the information...Why? This value only ever gets set in realize, so it will not change during the lifetime of the device - which means we don't need to migrate it.
Quoted 9 lines
Quoted 4 lines
I agree with the fact that the value does not change and should not be migrated in the first place. I am just worried about the size of the active ports bitmap that is streamed in the for loop... it is only 32 bit as of today, because we are limited to 32 ports. How would this work if the limit is raised ? How can the destination machine know how many bits have to be read ?
When the destination machine is started with -M 2.1, it knows that it has to read 32 bit.
If started with -M 3.0 it reads in 42 bits :)
Okay ! I was completely missing this point... now things make sense at last ! :) About Alex's patch, will you or Amit or Anthony apply it directly or should I send it along with my patches for a full review ?
Thanks for your time.
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Greg KurzQuoted 57 lines · Greg Kurz
Quoted 54 lines · On Thu, 12 Jun 2014 11:43:20 +0200 Alexander Graf <[email protected]> wrote:
Quoted 6 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +4
-10@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */ -return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp);
Quoted 11 lines
hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id) · line 696
Quoted 5 lines
hw/char/virtio-serial-bus.c +1
-0+ max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);For the moment, we have 0 <
max_nr_ports< 32 so the source machine only stores a single 32 bit value... If this limit gets raised, we can end up sending more than that... and only the source machinemax_nr_portsvalue can give the information...
I don't think we need to worry.
We won't be able to change max_nr_ports in compat machine types.
Quoted 8 lines
Signature · 7 lines
Gregory Kurz [email protected] [email protected] Software Engineer @ IBM/Meiosys http://www.ibm.com Tel +33 (0)562 165 496 "Anarchy is about taking complete responsibility for yourself." Alan Moore.
Michael S. Tsirkin
Reviewed-byRe: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Alexander GrafQuoted 48 lines · Alexander Graf
Quoted 6 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
How about a patch like this before the actual LE awareness ones? With this we should make virtio-serial config space completely independent of live migration.
Also since QEMU versions that do read these swapped values during migration are not bi-endian aware, we can never get into a case where a cross-endian save needs to be considered ;).
Alex
hw/char/virtio-serial-bus.c +1
-0@@ -670,6 +670,7 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)uint32_t max_nr_ports, nr_active_ports, ports_map;unsigned int i;int ret; +uint32_t tmp;if (version_id >3 ) {return -EINVAL;hw/char/virtio-serial-bus.c +0
-8@@ -685,17 +686,12 @@ static int virtio_serial_load(QEMUFile *f, void *opaque, int version_id)return 0 ; } -/* The config space */ -qemu_get_be16s (f, &s->config.cols); -qemu_get_be16s (f, &s->config.rows); - -qemu_get_be32s (f, &max_nr_ports); -tswap32s (&max_nr_ports); -if (max_nr_ports >tswap32 (s->config.max_nr_ports)) { -/* Source could have had more ports than us. Fail migration. */
hw/char/virtio-serial-bus.c static int virtio_serial_load(QEMUFile *f, void · line 697
Quoted 10 lines
hw/char/virtio-serial-bus.c +5
-2-return -EINVAL ; - } +/* Unused */ +qemu_get_be16s (f, &tmp); +qemu_get_be16s (f, &tmp); +qemu_get_be32s (f, &tmp); + max_nr_ports =tswap32 (s->config.max_nr_ports);for (i =0 ; i < (max_nr_ports +31 ) /32 ; i++) {qemu_get_be32s (f, &ports_map);
Exactly.
Re: [Qemu-devel] [PATCH RFC 8/8] virtio: add endian-ambivalent support to VirtIODevice
In reply to Paolo BonziniQuoted 8 lines · Paolo Bonzini
Il 12/06/2014 11:37, Michael S. Tsirkin ha scritto:
Quoted 2 lines
Maybe just drop unnecessary stuff for new machine types? Then we won't need hacks to migrate it.
For any machine type it's trivial to migrate it. All machine types including old ones can disregard the migrated values.
Paolo
Ah, good idea, so simply disregard the values in load.
Signature · 1 line
MST