failover: unregister ROM on unplug
PATCH v3 1/1 failover: unregister ROM on unplug
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...
(qemu) migrate ...
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance 0x0 of device 'ram' load of migration failed: Invalid argument
This happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
---
Notes:
v3:
remove useless space before comma
v2:
reset has_rom to false
update commit log message
hw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)
hw/net/virtio-net.c +4 -0
@@ -3256,6 +3256,10 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s)if (migration_in_setup (s) && !should_be_hidden) {if (failover_unplug_primary (n, dev)) {vmstate_unregister (VMSTATE_IF (dev),qdev_get_vmsd (dev), dev); +if (PCI_DEVICE (dev)->has_rom) { +PCI_DEVICE (dev)->has_rom = false; +vmstate_unregister_ram (&PCI_DEVICE (dev)->rom, dev); + }qapi_event_send_unplug_primary (dev->id);qatomic_set (&n->failover_primary_hidden, true); }else {
Signature · 1 line
2.31.1
Quoted 49 lines · * Laurent Vivier ([email protected]) wrote:
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...(qemu) migrate ...
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance
0x0of device 'ram' load of migration failed: Invalid argumentThis happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
Signed-off-by Laurent Vivier <[email protected]>---
Notes:
v3:
remove useless space before commav2:
resethas_romto false
update commit log messagehw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/virtio-net.c +0
-0
hw/net/virtio-net.c static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) · line 3262
Quoted 7 lines
hw/net/virtio-net.c +4
-0@@ -3256,6 +3256,10 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s)if (migration_in_setup (s) && !should_be_hidden) {if (failover_unplug_primary (n, dev)) {vmstate_unregister (VMSTATE_IF (dev),qdev_get_vmsd (dev), dev); +if (PCI_DEVICE (dev)->has_rom) { +PCI_DEVICE (dev)->has_rom = false; +vmstate_unregister_ram (&PCI_DEVICE (dev)->rom, dev); + }
Not actually originated by your fix, but....
Why doesn't failover_replug_primary re-add the vmstates?
(I did wonder if passing rom-file="" to the e1000 would help in your testing case, but it still creates the RAM image).
Dave
Quoted 6 lines
hw/net/virtio-net.c +0
-0qapi_event_send_unplug_primary (dev->id);qatomic_set (&n->failover_primary_hidden, true); }else {Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 60 lines · Dr. David Alan Gilbert
Quoted 49 lines · * Laurent Vivier ([email protected]) wrote:
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...(qemu) migrate ...
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance
0x0of device 'ram' load of migration failed: Invalid argumentThis happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
Signed-off-by Laurent Vivier <[email protected]>---
Notes:
v3:
remove useless space before commav2:
resethas_romto false
update commit log messagehw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/virtio-net.c +0
-0hw/net/virtio-net.c static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) · line 3262
Quoted 7 lines
hw/net/virtio-net.c +4
-0@@ -3256,6 +3256,10 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s)if (migration_in_setup (s) && !should_be_hidden) {if (failover_unplug_primary (n, dev)) {vmstate_unregister (VMSTATE_IF (dev),qdev_get_vmsd (dev), dev); +if (PCI_DEVICE (dev)->has_rom) { +PCI_DEVICE (dev)->has_rom = false; +vmstate_unregister_ram (&PCI_DEVICE (dev)->rom, dev); + }Not actually originated by your fix, but....
Quoted 1 line
Why doesn't
failover_replug_primaryre-add the vmstates?
Good point.
I'm going to check but I think the vmstates are re-added by the hotplug operations that are used in failover_replug_primary().
Thanks,
Laurent
Quoted 60 lines · Dr. David Alan Gilbert
Quoted 49 lines · * Laurent Vivier ([email protected]) wrote:
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...(qemu) migrate ...
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance
0x0of device 'ram' load of migration failed: Invalid argumentThis happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
Signed-off-by Laurent Vivier <[email protected]>---
Notes:
v3:
remove useless space before commav2:
resethas_romto false
update commit log messagehw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/virtio-net.c +0
-0hw/net/virtio-net.c static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) · line 3262
Quoted 7 lines
hw/net/virtio-net.c +4
-0@@ -3256,6 +3256,10 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s)if (migration_in_setup (s) && !should_be_hidden) {if (failover_unplug_primary (n, dev)) {vmstate_unregister (VMSTATE_IF (dev),qdev_get_vmsd (dev), dev); +if (PCI_DEVICE (dev)->has_rom) { +PCI_DEVICE (dev)->has_rom = false; +vmstate_unregister_ram (&PCI_DEVICE (dev)->rom, dev); + }Not actually originated by your fix, but....
Quoted 1 line
Why doesn't
failover_replug_primaryre-add the vmstates?
Because we can't migrate until the "unplug" has happened. Yes, it is a mess.
I think this is the saner patch that I can think of for that functionality.
What I wonder is why we register rom as ram, but I guess that the rom can be updated from userspace, or who knows.
Later, Juan.
Quoted 11 lines
(I did wonder if passing rom-file="" to the e1000 would help in your testing case, but it still creates the RAM image).
Dave
Quoted 6 lines
hw/net/virtio-net.c +0
-0qapi_event_send_unplug_primary (dev->id);qatomic_set (&n->failover_primary_hidden, true); }else {Signature · 1 line
2.31.1
Quoted 61 lines · * Juan Quintela ([email protected]) wrote:
Quoted 60 lines · Dr. David Alan Gilbert
Quoted 49 lines · * Laurent Vivier ([email protected]) wrote:
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...(qemu) migrate ...
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance
0x0of device 'ram' load of migration failed: Invalid argumentThis happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
Signed-off-by Laurent Vivier <[email protected]>---
Notes:
v3:
remove useless space before commav2:
resethas_romto false
update commit log messagehw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/virtio-net.c +0
-0hw/net/virtio-net.c static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) · line 3262
Quoted 7 lines
hw/net/virtio-net.c +4
-0@@ -3256,6 +3256,10 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s)if (migration_in_setup (s) && !should_be_hidden) {if (failover_unplug_primary (n, dev)) {vmstate_unregister (VMSTATE_IF (dev),qdev_get_vmsd (dev), dev); +if (PCI_DEVICE (dev)->has_rom) { +PCI_DEVICE (dev)->has_rom = false; +vmstate_unregister_ram (&PCI_DEVICE (dev)->rom, dev); + }Not actually originated by your fix, but....
Quoted 4 lines
Quoted 1 line
Why doesn't
failover_replug_primaryre-add the vmstates?Because we can't migrate until the "unplug" has happened. Yes, it is a mess.
But if the migrate fails, shouldn't it add it back?
Dave
Quoted 20 lines
I think this is the saner patch that I can think of for that functionality.
What I wonder is why we register rom as ram, but I guess that the rom can be updated from userspace, or who knows.
Later, Juan.
Quoted 11 lines
(I did wonder if passing rom-file="" to the e1000 would help in your testing case, but it still creates the RAM image).
Dave
Quoted 6 lines
hw/net/virtio-net.c +0
-0qapi_event_send_unplug_primary (dev->id);qatomic_set (&n->failover_primary_hidden, true); }else {Signature · 1 line
2.31.1
Signature · 1 line
Dr. David Alan Gilbert / [email protected] / Manchester, UK
Quoted 62 lines · Dr. David Alan Gilbert
Quoted 61 lines · * Juan Quintela ([email protected]) wrote:
Quoted 60 lines · Dr. David Alan Gilbert
Quoted 49 lines · * Laurent Vivier ([email protected]) wrote:
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...(qemu) migrate ...
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance
0x0of device 'ram' load of migration failed: Invalid argumentThis happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
Signed-off-by Laurent Vivier <[email protected]>---
Notes:
v3:
remove useless space before commav2:
resethas_romto false
update commit log messagehw/net/virtio-net.c | 4 ++++ 1 file changed, 4 insertions(+)hw/net/virtio-net.c +0
-0hw/net/virtio-net.c static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) · line 3262
Quoted 7 lines
hw/net/virtio-net.c +4
-0@@ -3256,6 +3256,10 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s)if (migration_in_setup (s) && !should_be_hidden) {if (failover_unplug_primary (n, dev)) {vmstate_unregister (VMSTATE_IF (dev),qdev_get_vmsd (dev), dev); +if (PCI_DEVICE (dev)->has_rom) { +PCI_DEVICE (dev)->has_rom = false; +vmstate_unregister_ram (&PCI_DEVICE (dev)->rom, dev); + }Not actually originated by your fix, but....
Quoted 14 lines
Quoted 4 lines
Quoted 1 line
Why doesn't
failover_replug_primaryre-add the vmstates?Because we can't migrate until the "unplug" has happened. Yes, it is a mess.
But if the migrate fails, shouldn't it add it back?
Dave
Quoted 5 lines
I think this is the saner patch that I can think of for that functionality.
What I wonder is why we register rom as ram, but I guess that the rom can be updated from userspace, or who knows.
Unlikely, and if we can, this is a bug.
We call memory_region_init_rom() in pci_add_option_rom().
* memory_region_init_rom: Initialize a ROM memory region. *
* This has the same effect as calling memory_region_init_ram()
* and then marking the resulting region read-only with
* memory_region_set_readonly(). This includes arranging for the
* contents to be migrated.
I agree it would be clearer to have a vmstate_unregister_rom() function internally calling vmstate_unregister_ram().
Quoted 15 lines
Quoted 15 lines
Later, Juan.
Quoted 11 lines
(I did wonder if passing rom-file="" to the e1000 would help in your testing case, but it still creates the RAM image).
Dave
Quoted 6 lines
hw/net/virtio-net.c +0
-0qapi_event_send_unplug_primary (dev->id);qatomic_set (&n->failover_primary_hidden, true); }else {Signature · 1 line
2.31.1
Quoted 23 lines · Laurent Vivier
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration.
This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover.
This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device.
Without this change the migration of a system configured with failover fails with:
... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ...(qemu) migrate ...
Quoted 8 lines
Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance
0x0of device 'ram' load of migration failed: Invalid argumentThis happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that.
Signed-off-by Laurent Vivier <[email protected]>
As this is only for testing.