Skip to contentSkip to conversation

vfio: Set the priority of VFIO VM state change handler explicitly

RFC PATCH In review Shenming Lu 3 participants 3 messages Nov 17, 2020 – Nov 23, 2020

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 vfio: Set the priority of VFIO VM state change handler explicitly

In VFIO VM state change handler, VFIO devices are transitioned in _SAVING state, which should keep them from sending interrupts. Then we can save the pending states of all interrupts in GIC VM state change handler (on ARM).

So we have to set the priority of VFIO VM state change handler explicitly (like virtio devices) to ensure it is called before GIC's in saving.

---
 hw/vfio/migration.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/vfio/migration.c +2 -1
index 55261562d4..d0d30864ba 100644
@@ -857,7 +857,8 @@ static int vfio_migration_init(VFIODevice *vbasedev,
     register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_handlers,
                          vbasedev);
 
-    migration->vm_state = qemu_add_vm_change_state_handler(vfio_vmstate_change,
+    migration->vm_state = qdev_add_vm_change_state_handler(vbasedev->dev,
+                                                           vfio_vmstate_change,
                                                            vbasedev);
     migration->migration_state.notify = vfio_migration_state_notifier;
     add_migration_state_change_notifier(&migration->migration_state);
Signature · 1 line
2.19.1
Reviewed-by on v1 1/1
#1 lore raw
Quoted 20 lines · Shenming Lu

In VFIO VM state change handler, VFIO devices are transitioned in _SAVING state, which should keep them from sending interrupts. Then we can save the pending states of all interrupts in GIC VM state change handler (on ARM).

So we have to set the priority of VFIO VM state change handler explicitly (like virtio devices) to ensure it is called before GIC's in saving.

---
  hw/vfio/migration.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
hw/vfio/migration.c +0 -0
index 55261562d4..d0d30864ba 100644

hw/vfio/migration.c static int vfio_migration_init(VFIODevice *vbasedev, · line 864

Quoted 9 lines
hw/vfio/migration.c +2 -1
@@ -857,7 +857,8 @@ static int vfio_migration_init(VFIODevice *vbasedev,
      register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_handlers,
                           vbasedev);
 
-    migration->vm_state = qemu_add_vm_change_state_handler(vfio_vmstate_change,
+    migration->vm_state = qdev_add_vm_change_state_handler(vbasedev->dev,
+                                                           vfio_vmstate_change,
                                                             vbasedev);
      migration->migration_state.notify = vfio_migration_state_notifier;
      add_migration_state_change_notifier(&migration->migration_state);

Looks good to me.

#2 lore raw
In reply to Shenming Lu
Quoted 8 lines · On Tue, 17 Nov 2020 09:40:50 +0800 Shenming Lu <[email protected]> wrote:

In VFIO VM state change handler, VFIO devices are transitioned in _SAVING state, which should keep them from sending interrupts. Then we can save the pending states of all interrupts in GIC VM state change handler (on ARM).

So we have to set the priority of VFIO VM state change handler explicitly (like virtio devices) to ensure it is called before GIC's in saving.

I'm wondering whether this is the case for other irq chips as well, and the GIC is just the first one where it was noticed.

Quoted 11 lines
---
 hw/vfio/migration.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/vfio/migration.c +0 -0
index 55261562d4..d0d30864ba 100644

hw/vfio/migration.c static int vfio_migration_init(VFIODevice *vbasedev, · line 864

Quoted 9 lines
hw/vfio/migration.c +2 -1
@@ -857,7 +857,8 @@ static int vfio_migration_init(VFIODevice *vbasedev,
     register_savevm_live(id, VMSTATE_INSTANCE_ID_ANY, 1, &savevm_vfio_handlers,
                          vbasedev);
 
-    migration->vm_state = qemu_add_vm_change_state_handler(vfio_vmstate_change,
+    migration->vm_state = qdev_add_vm_change_state_handler(vbasedev->dev,
+                                                           vfio_vmstate_change,
                                                            vbasedev);
     migration->migration_state.notify = vfio_migration_state_notifier;
     add_migration_state_change_notifier(&migration->migration_state);

I think this patch does what the patch description says, but I'm wondering whether we might miss other interdependencies as well (which would also be fixed by pushing the vm state handling of vfio devices to that point in the sequence?)