Skip to contentSkip to conversation

vfio/pci: Guard accel_irqchip_begin_route_changes() calls

PATCH In review Cédric Le Goater 2 participants 4 messages Jul 21 – Jul 29

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 vfio/pci: Guard accel_irqchip_begin_route_changes() calls

Since commit 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes"), accel_irqchip_begin_route_changes() aborts when no accelerator irqchip is available. This causes a fatal error when running VFIO passthrough devices under TCG emulation:

qemu-system-aarch64: can't initiate route change, no accel irqchip available

The previous kvm_irqchip_begin_route_changes() was a simple inline that did not have a fatal path. The VFIO code already handles the absence of KVM MSI routing gracefully by falling back to userspace handling, but the new generic function aborts before that fallback can take effect.

Guard the call sites in hw/vfio/pci.c with accel_msi_via_irqfd_enabled() so that route changes are only initiated when an accelerator irqchip is actually present.

Fixes 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes")
Cc Magnus Kulke <[email protected]>
---
 hw/vfio/pci.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
hw/vfio/pci.c +4 -2
index 380dd8c15f8d5bb98b725075978eef2e4e1e6c2d..a3147d28665abd29fd804bd08bdffc3c7440033c 100644
@@ -699,7 +699,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
         if (msg) {
             if (vdev->defer_kvm_irq_routing) {
                 vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);
-            } else {
+            } else if (accel_msi_via_irqfd_enabled()) {
                 vfio_route_change = accel_irqchip_begin_route_changes();
                 vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);
                 accel_irqchip_commit_route_changes(&vfio_route_change);
@@ -801,7 +801,9 @@ void vfio_pci_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
 {
     assert(!vdev->defer_kvm_irq_routing);
     vdev->defer_kvm_irq_routing = true;
-    vfio_route_change = accel_irqchip_begin_route_changes();
+    if (accel_msi_via_irqfd_enabled()) {
+        vfio_route_change = accel_irqchip_begin_route_changes();
+    }
 }
 
 void vfio_pci_commit_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
Signature · 1 line
2.55.0
#1 lore raw
Quoted 29 lines · Cédric Le Goater

Since commit 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes"), accel_irqchip_begin_route_changes() aborts when no accelerator irqchip is available. This causes a fatal error when running VFIO passthrough devices under TCG emulation:

qemu-system-aarch64: can't initiate route change, no accel irqchip available

The previous kvm_irqchip_begin_route_changes() was a simple inline that did not have a fatal path. The VFIO code already handles the absence of KVM MSI routing gracefully by falling back to userspace handling, but the new generic function aborts before that fallback can take effect.

Guard the call sites in hw/vfio/pci.c with accel_msi_via_irqfd_enabled() so that route changes are only initiated when an accelerator irqchip is actually present.

Fixes 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes")
Cc Magnus Kulke <[email protected]>
---
  hw/vfio/pci.c | 6 ++++--
  1 file changed, 4 insertions(+), 2 deletions(-)
hw/vfio/pci.c +0 -0
index 380dd8c15f8d5bb98b725075978eef2e4e1e6c2d..a3147d28665abd29fd804bd08bdffc3c7440033c 100644

hw/vfio/pci.c static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr, · line 703

Quoted 6 lines
hw/vfio/pci.c +1 -1
@@ -699,7 +699,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
          if (msg) {
              if (vdev->defer_kvm_irq_routing) {
                  vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);
-            } else {
+            } else if (accel_msi_via_irqfd_enabled()) {
                  vfio_route_change = accel_irqchip_begin_route_changes();

Pre-existing, either we use the accel_*() API which is agnostic to the host hw accelerator implementation (KVM, MSHV), or we directly use the hw API.

hw/vfio/pci.c static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr, · line 704

Quoted 1 line
hw/vfio/pci.c +0 -0
                  vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);

Here there is an implicit KVM dependency, so I'm not sure why we have these accel_* calls. Maybe "system/accel-irq.h" is incomplete in that regard.

hw/vfio/pci.c void vfio_pci_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev) · line 804

Quoted 7 lines
hw/vfio/pci.c +1 -1
                  accel_irqchip_commit_route_changes(&vfio_route_change);
@@ -801,7 +801,9 @@ void vfio_pci_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
  {
      assert(!vdev->defer_kvm_irq_routing);
      vdev->defer_kvm_irq_routing = true;
-    vfio_route_change = accel_irqchip_begin_route_changes();
+    if (accel_msi_via_irqfd_enabled()) {

(IOW here I'd rather a kvm_msi_via_irqfd_enabled() call)

Quoted 5 lines
hw/vfio/pci.c +2 -0
+        vfio_route_change = accel_irqchip_begin_route_changes();
+    }
  }
 
  void vfio_pci_commit_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
#2 lore raw
Quoted 25 lines · Philippe Mathieu-Daudé
Quoted 24 lines · Cédric Le Goater

Since commit 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes"), accel_irqchip_begin_route_changes() aborts when no accelerator irqchip is available. This causes a fatal error when running VFIO passthrough devices under TCG emulation:

   qemu-system-aarch64: can't initiate route change, no accel irqchip available

The previous kvm_irqchip_begin_route_changes() was a simple inline that did not have a fatal path. The VFIO code already handles the absence of KVM MSI routing gracefully by falling back to userspace handling, but the new generic function aborts before that fallback can take effect.

Guard the call sites in hw/vfio/pci.c with accel_msi_via_irqfd_enabled() so that route changes are only initiated when an accelerator irqchip is actually present.

Fixes 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes")
Cc Magnus Kulke <[email protected]>

---
  hw/vfio/pci.c | 6 ++++--

  1 file changed, 4 insertions(+), 2 deletions(-)
Quoted 14 lines

hw/vfio/pci.c static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr, · line 703

Quoted 11 lines
hw/vfio/pci.c +1 -1
index 380dd8c15f8d5bb98b725075978eef2e4e1e6c2d..a3147d28665abd29fd804bd08bdffc3c7440033c 100644
@@ -699,7 +699,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
           if (msg) {
               if (vdev->defer_kvm_irq_routing) {
                   vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);
-            } else {
+            } else if (accel_msi_via_irqfd_enabled()) {
                   vfio_route_change = accel_irqchip_begin_route_changes();

Pre-existing, either we use the accel_*() API which is agnostic to the host hw accelerator implementation (KVM, MSHV), or we directly use the hw API.

That would be a partial revert. I am fine with that.

C.

Quoted 22 lines

hw/vfio/pci.c static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr, · line 704

Quoted 2 lines
hw/vfio/pci.c +0 -0
                   vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);

Here there is an implicit KVM dependency, so I'm not sure why we have these accel_* calls. Maybe "system/accel-irq.h" is incomplete in that regard.

hw/vfio/pci.c void vfio_pci_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev) · line 804

Quoted 7 lines
hw/vfio/pci.c +1 -1
                   accel_irqchip_commit_route_changes(&vfio_route_change);
@@ -801,7 +801,9 @@ void vfio_pci_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
   {
       assert(!vdev->defer_kvm_irq_routing);
       vdev->defer_kvm_irq_routing = true;
-    vfio_route_change = accel_irqchip_begin_route_changes();
+    if (accel_msi_via_irqfd_enabled()) {

(IOW here I'd rather a kvm_msi_via_irqfd_enabled() call)

Quoted 4 lines
hw/vfio/pci.c +2 -0
+        vfio_route_change = accel_irqchip_begin_route_changes();
+    }
   }
   void vfio_pci_commit_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
#3 lore raw
In reply to Cédric Le Goater

This is a regression with TCG.

VFIO needs these changes to correctly fall back to userspace interrupt routing, as before without aborting.

Thanks,

C.

Quoted 48 lines · Cédric Le Goater

Since commit 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes"), accel_irqchip_begin_route_changes() aborts when no accelerator irqchip is available. This causes a fatal error when running VFIO passthrough devices under TCG emulation:

qemu-system-aarch64: can't initiate route change, no accel irqchip available

The previous kvm_irqchip_begin_route_changes() was a simple inline that did not have a fatal path. The VFIO code already handles the absence of KVM MSI routing gracefully by falling back to userspace handling, but the new generic function aborts before that fallback can take effect.

Guard the call sites in hw/vfio/pci.c with accel_msi_via_irqfd_enabled() so that route changes are only initiated when an accelerator irqchip is actually present.

Fixes 49b2dcbd2422 ("accel/accel-irq: add generic begin_route_changes")
Cc Magnus Kulke <[email protected]>
---
  hw/vfio/pci.c | 6 ++++--
  1 file changed, 4 insertions(+), 2 deletions(-)
hw/vfio/pci.c +4 -2
index 380dd8c15f8d5bb98b725075978eef2e4e1e6c2d..a3147d28665abd29fd804bd08bdffc3c7440033c 100644
@@ -699,7 +699,7 @@ static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
          if (msg) {
              if (vdev->defer_kvm_irq_routing) {
                  vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);
-            } else {
+            } else if (accel_msi_via_irqfd_enabled()) {
                  vfio_route_change = accel_irqchip_begin_route_changes();
                  vfio_pci_add_kvm_msi_virq(vdev, vector, nr, true);
                  accel_irqchip_commit_route_changes(&vfio_route_change);
@@ -801,7 +801,9 @@ void vfio_pci_prepare_kvm_msi_virq_batch(VFIOPCIDevice *vdev)
  {
      assert(!vdev->defer_kvm_irq_routing);
      vdev->defer_kvm_irq_routing = true;
-    vfio_route_change = accel_irqchip_begin_route_changes();
+    if (accel_msi_via_irqfd_enabled()) {
+        vfio_route_change = accel_irqchip_begin_route_changes();
+    }
  }
 
  void vfio_pci_commit_kvm_msi_virq_batch(VFIOPCIDevice *vdev)