Skip to contentSkip to conversation

util/vfio-helpers: Fixes to allow using multiple IRQs

RFC PATCH Superseded Philippe Mathieu-Daudé 2 participants 7 messages Aug 11, 2020 Block layer

v1 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

cover util/vfio-helpers: Fixes to allow using multiple IRQs

A pair of fixes to allow binding notifiers on different IRQs (NVMe block driver series will follow).

Philippe Mathieu-Daudé (3):
util/vfio-helpers: Store eventfd using int32_t type util/vfio-helpers: Add trace event to display device IRQs available util/vfio-helpers: Let qemu_vfio_pci_init_irq take IRQ index argument

 include/qemu/vfio-helpers.h |  2 +-
 block/nvme.c                |  2 +-
 util/vfio-helpers.c         | 17 ++++++++++++-----
 util/trace-events           |  1 +
 4 files changed, 15 insertions(+), 7 deletions(-)
Signature · 1 line
2.21.3
#1 lore raw

RFC PATCH 1/3 util/vfio-helpers: Store eventfd using int32_t type

Per the documentation in linux-headers/linux/vfio.h:

VFIO_DEVICE_SET_IRQS

* DATA_EVENTFD binds the specified ACTION to the provided __s32 eventfd.

Replace the 'int' by an 'int32_t' to match the documentation.

Fixes 418026ca43 ("util: Introduce vfio helpers")
---
 util/vfio-helpers.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
util/vfio-helpers.c +2 -2
index 583bdfb36f..6defefcc01 100644
@@ -193,7 +193,7 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         return -EINVAL;
     }
 
-    irq_set_size = sizeof(*irq_set) + sizeof(int);
+    irq_set_size = sizeof(*irq_set) + sizeof(int32_t);
     irq_set = g_malloc0(irq_set_size);
 
     /* Get to a known IRQ state */
@@ -205,7 +205,7 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         .count = 1,
     };
 
-    *(int *)&irq_set->data = event_notifier_get_fd(e);
+    *(int32_t *)&irq_set->data = event_notifier_get_fd(e);
     r = ioctl(s->device, VFIO_DEVICE_SET_IRQS, irq_set);
     g_free(irq_set);
     if (r) {
Signature · 1 line
2.21.3
#2 lore raw

RFC PATCH 2/3 util/vfio-helpers: Add trace event to display device IRQs available

In reply to Philippe Mathieu-Daudé

Add a trace event to display the amount of IRQs available on the device.

---
 util/vfio-helpers.c | 1 +
 util/trace-events   | 1 +
 2 files changed, 2 insertions(+)
util/vfio-helpers.c +1 -0
index 6defefcc01..3ad7e6be52 100644
@@ -192,6 +192,7 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         error_setg(errp, "Device interrupt doesn't support eventfd");
         return -EINVAL;
     }
+    trace_qemu_vfio_pci_init_irq(irq_info.count);
 
     irq_set_size = sizeof(*irq_set) + sizeof(int32_t);
     irq_set = g_malloc0(irq_set_size);
util/trace-events +1 -0
index 0ce42822eb..351dbdbe3c 100644
@@ -83,3 +83,4 @@ qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova
 qemu_vfio_do_mapping(void *s, void *host, size_t size, uint64_t iova) "s %p host %p size %zu iova 0x%"PRIx64
 qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t *iova) "s %p host %p size %zu temporary %d iova %p"
 qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p"
+qemu_vfio_pci_init_irq(uint32_t count) "device interrupt count: %"PRIu32
Signature · 1 line
2.21.3
#3 lore raw

Re: [RFC PATCH 2/3] util/vfio-helpers: Add trace event to display device IRQs available

Quoted 26 lines · On Tue, 11 Aug 2020 19:28:44 +0200 Philippe Mathieu-Daudé <[email protected]>…

Add a trace event to display the amount of IRQs available on the device.

---
 util/vfio-helpers.c | 1 +
 util/trace-events   | 1 +
 2 files changed, 2 insertions(+)
util/vfio-helpers.c +1 -0
index 6defefcc01..3ad7e6be52 100644
@@ -192,6 +192,7 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         error_setg(errp, "Device interrupt doesn't support eventfd");
         return -EINVAL;
     }
+    trace_qemu_vfio_pci_init_irq(irq_info.count);
 
     irq_set_size = sizeof(*irq_set) + sizeof(int32_t);
     irq_set = g_malloc0(irq_set_size);
util/trace-events +0 -0
index 0ce42822eb..351dbdbe3c 100644

util/trace-events qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova · line 86

Quoted 4 lines
util/trace-events +1 -0
@@ -83,3 +83,4 @@ qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova
 qemu_vfio_do_mapping(void *s, void *host, size_t size, uint64_t iova) "s %p host %p size %zu iova 0x%"PRIx64
 qemu_vfio_dma_map(void *s, void *host, size_t size, bool temporary, uint64_t *iova) "s %p host %p size %zu temporary %d iova %p"
 qemu_vfio_dma_unmap(void *s, void *host) "s %p host %p"
+qemu_vfio_pci_init_irq(uint32_t count) "device interrupt count: %"PRIu32

Knowing the count independent of which index we're looking at doesn't seem very useful. Thanks,

Alex

#4 lore raw

RFC PATCH 3/3 util/vfio-helpers: Let qemu_vfio_pci_init_irq take IRQ index argument

In reply to Philippe Mathieu-Daudé

Add a new 'index' argument to qemu_vfio_pci_init_irq() to be able to initialize other IRQs than IRQ #0. Adapt the single user of this API in nvme_init().

---
 include/qemu/vfio-helpers.h |  2 +-
 block/nvme.c                |  2 +-
 util/vfio-helpers.c         | 12 +++++++++---
 3 files changed, 11 insertions(+), 5 deletions(-)
include/qemu/vfio-helpers.h +1 -1
index 1f057c2b9e..ff63e75096 100644
@@ -27,6 +27,6 @@ void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index,
 void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar,
                              uint64_t offset, uint64_t size);
 int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
-                           int irq_type, Error **errp);
+                           int irq_type, unsigned index, Error **errp);
 
 #endif
block/nvme.c +1 -1
index 374e268915..2b3986b66d 100644
@@ -757,7 +757,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     }
 
     ret = qemu_vfio_pci_init_irq(s->vfio, &s->irq_notifier,
-                                 VFIO_PCI_MSIX_IRQ_INDEX, errp);
+                                 VFIO_PCI_MSIX_IRQ_INDEX, 0, errp);
     if (ret) {
         goto out;
     }
util/vfio-helpers.c +9 -3
index 3ad7e6be52..ba9a869364 100644
@@ -173,10 +173,11 @@ void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar,
 }
 
 /**
- * Initialize device IRQ with @irq_type and register an event notifier.
+ * Initialize device IRQ @index with @irq_type
+ * and register an event notifier.
  */
 int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
-                           int irq_type, Error **errp)
+                           int irq_type, unsigned index, Error **errp)
 {
     int r;
     struct vfio_irq_set *irq_set;
@@ -193,6 +194,11 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         return -EINVAL;
     }
     trace_qemu_vfio_pci_init_irq(irq_info.count);
+    if (index >= irq_info.count) {
+        error_setg(errp, "Device has %"PRIu32" interrupts (requested index %u)",
+                   irq_info.count, index);
+        return -EINVAL;
+    }
 
     irq_set_size = sizeof(*irq_set) + sizeof(int32_t);
     irq_set = g_malloc0(irq_set_size);
@@ -202,7 +208,7 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         .argsz = irq_set_size,
         .flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER,
         .index = irq_info.index,
-        .start = 0,
+        .start = index,
         .count = 1,
     };
 
Signature · 1 line
2.21.3
#5 lore raw

Re: [RFC PATCH 3/3] util/vfio-helpers: Let qemu_vfio_pci_init_irq take IRQ index argument

Quoted 3 lines · On Tue, 11 Aug 2020 19:28:45 +0200 Philippe Mathieu-Daudé <[email protected]>…

Add a new 'index' argument to qemu_vfio_pci_init_irq() to be able to initialize other IRQs than IRQ #0. Adapt the single user of this API in nvme_init().

This is actually addressing the what the vfio uAPI refers to as the subindex, the index is the interrupt type, ex. MSI-X.

Quoted 39 lines
---
 include/qemu/vfio-helpers.h |  2 +-
 block/nvme.c                |  2 +-
 util/vfio-helpers.c         | 12 +++++++++---
 3 files changed, 11 insertions(+), 5 deletions(-)
include/qemu/vfio-helpers.h +1 -1
index 1f057c2b9e..ff63e75096 100644
@@ -27,6 +27,6 @@ void *qemu_vfio_pci_map_bar(QEMUVFIOState *s, int index,
 void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar,
                              uint64_t offset, uint64_t size);
 int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
-                           int irq_type, Error **errp);
+                           int irq_type, unsigned index, Error **errp);
 
 #endif
block/nvme.c +1 -1
index 374e268915..2b3986b66d 100644
@@ -757,7 +757,7 @@ static int nvme_init(BlockDriverState *bs, const char *device, int namespace,
     }
 
     ret = qemu_vfio_pci_init_irq(s->vfio, &s->irq_notifier,
-                                 VFIO_PCI_MSIX_IRQ_INDEX, errp);
+                                 VFIO_PCI_MSIX_IRQ_INDEX, 0, errp);
     if (ret) {
         goto out;
     }
util/vfio-helpers.c +0 -0
index 3ad7e6be52..ba9a869364 100644
@@ -173,10 +173,11 @@ void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar,
 }

util/vfio-helpers.c void qemu_vfio_pci_unmap_bar(QEMUVFIOState *s, int index, void *bar, · line 181

Quoted 10 lines
util/vfio-helpers.c +3 -2
 
 /**
- * Initialize device IRQ with @irq_type and register an event notifier.
+ * Initialize device IRQ @index with @irq_type
+ * and register an event notifier.
  */
 int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
-                           int irq_type, Error **errp)
+                           int irq_type, unsigned index, Error **errp)
 {

But MSI-X will expose the VFIO_IRQ_INFO_NORESIZE flag, which means that we cannot incrementally enable additional subindexes, aka vectors, without first disabling and re-enabling the entire index/irq_type. Therefore this is exposing an API that isn't actually supported. Thanks,

Alex

Quoted 23 lines
util/vfio-helpers.c +6 -1
     int r;
     struct vfio_irq_set *irq_set;
@@ -193,6 +194,11 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         return -EINVAL;
     }
     trace_qemu_vfio_pci_init_irq(irq_info.count);
+    if (index >= irq_info.count) {
+        error_setg(errp, "Device has %"PRIu32" interrupts (requested index %u)",
+                   irq_info.count, index);
+        return -EINVAL;
+    }
 
     irq_set_size = sizeof(*irq_set) + sizeof(int32_t);
     irq_set = g_malloc0(irq_set_size);
@@ -202,7 +208,7 @@ int qemu_vfio_pci_init_irq(QEMUVFIOState *s, EventNotifier *e,
         .argsz = irq_set_size,
         .flags = VFIO_IRQ_SET_DATA_EVENTFD | VFIO_IRQ_SET_ACTION_TRIGGER,
         .index = irq_info.index,
-        .start = 0,
+        .start = index,
         .count = 1,
     };
 
#6 lore raw
In reply to Philippe Mathieu-Daudé
Quoted 8 lines · Philippe Mathieu-Daudé

A pair of fixes to allow binding notifiers on different IRQs (NVMe block driver series will follow).

Philippe Mathieu-Daudé (3):
util/vfio-helpers: Store eventfd using int32_t type util/vfio-helpers: Add trace event to display device IRQs available util/vfio-helpers: Let qemu_vfio_pci_init_irq take IRQ index argument

Quoted 6 lines
 include/qemu/vfio-helpers.h |  2 +-
 block/nvme.c                |  2 +-
 util/vfio-helpers.c         | 17 ++++++++++++-----
 util/trace-events           |  1 +
 4 files changed, 15 insertions(+), 7 deletions(-)

Oops I forgot, it is based on "util/vfio-helpers: Fix typo in description":
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg02073.html
Based-on: <[email protected]>