Skip to contentSkip to conversation

Add support for MSI injection on Arm

PATCH In review Mykyta Poturai 4 participants 10 messages Apr 14, 2025 – Apr 16, 2025

lore
Share & tools
lore
#0 lore raw

v2 cover Add support for MSI injection on Arm

This series adds the base support for MSI injection on Arm. This is needed to streamline virtio-pci interrupt triggering.

With this patches, MSIs can be triggered in guests by issuing the new DM op, inject_msi2. This op is similar to inject_msi, but it allows to specify the source id of the MSI.

We chose the approach of adding a new DM op instead of using the pad field of inject_msi because we have no clear way of distinguishing between set and unset pad fields. New implementations also adds flags field to clearly specify if the SBDF is set.

Patches were tested on QEMU with QEMU virtio-pci backends, with virtio-pci patches and patches for ITS support for DomUs applied.

Branch with all relevant Xen patches:
https://github.com/Deedone/xen/tree/4.20-dev%2Bvirtio

Branch with all relevant QEMU patches:
https://github.com/Deedone/qemu/tree/virtio-msi2

Mykyta Poturai (2):
arm: vgic: Add the ability to trigger MSIs from the Hypervisor xen/dm: arm: Introduce inject_msi2 DM op

 tools/include/xendevicemodel.h               | 14 ++++++++++++++
 tools/libs/devicemodel/core.c                | 20 ++++++++++++++++++++
 tools/libs/devicemodel/libxendevicemodel.map |  5 +++++
 xen/arch/arm/dm.c                            | 17 +++++++++++++++++
 xen/arch/arm/include/asm/vgic.h              | 11 +++++++++++
 xen/arch/arm/vgic-v3-its.c                   | 19 +++++++++++++++++++
 xen/arch/x86/hvm/dm.c                        | 18 ++++++++++++++++++
 xen/include/public/hvm/dm_op.h               | 18 ++++++++++++++++++
 8 files changed, 122 insertions(+)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v2 1/2 arm: vgic: Add the ability to trigger MSIs from the Hypervisor

From: Mykyta Poturai <[email protected]>

Add the vgic_its_trigger_msi() function to the vgic interface. This function allows to inject MSIs from the Hypervisor to the guest. Which is useful for userspace PCI backend drivers.

---
v1->v2:
* replace -1 with -ENOENT
* reduce guest memory access in vgic_its_trigger_msi

---
 xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
 xen/arch/arm/vgic-v3-its.c      | 19 +++++++++++++++++++
 2 files changed, 30 insertions(+)
xen/arch/arm/include/asm/vgic.h +11 -0
index e309dca1ad..3d8e3a8343 100644
@@ -318,6 +318,17 @@ extern bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int ir
 extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
                                              unsigned int rank, uint32_t r);
 
+#ifdef CONFIG_HAS_ITS
+int vgic_its_trigger_msi(struct domain *d, paddr_t doorbell_address,
+                                u32 devid, u32 eventid);
+#else
+static inline int vgic_its_trigger_msi(struct domain *d, paddr_t doorbell_address,
+                                u32 devid, u32 eventid)
+{
+    return -EOPNOTSUPP;
+}
+#endif /* CONFIG_HAS_ITS */
+
 #endif /* !CONFIG_NEW_VGIC */
 
 /*** Common VGIC functions used by Xen arch code ****/
xen/arch/arm/vgic-v3-its.c +19 -0
index c65c1dbf52..be5bfe0d21 100644
@@ -1484,6 +1484,25 @@ static int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr,
     return 0;
 }
 
+int vgic_its_trigger_msi(struct domain *d, paddr_t doorbell_address,
+                                u32 devid, u32 eventid)
+{
+    struct pending_irq *pend;
+    unsigned int vcpu_id;
+
+    pend = gicv3_its_get_event_pending_irq(d,doorbell_address, devid, eventid);
+    if ( !pend )
+        return -ENOENT;
+    
+    vcpu_id = ACCESS_ONCE(pend->lpi_vcpu_id);
+    if ( vcpu_id >= d->max_vcpus )
+          return -ENOENT;
+
+    vgic_inject_irq(d, d->vcpu[vcpu_id], pend->irq, true);
+
+    return 0;
+}
+
 unsigned int vgic_v3_its_count(const struct domain *d)
 {
     struct host_its *hw_its;
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v2 1/2] arm: vgic: Add the ability to trigger MSIs from the Hypervisor

Hi,

Quoted 43 lines · Mykyta Poturai

From: Mykyta Poturai <[email protected]>

Add the vgic_its_trigger_msi() function to the vgic interface. This function allows to inject MSIs from the Hypervisor to the guest. Which is useful for userspace PCI backend drivers.

---
v1->v2:
* replace -1 with -ENOENT
* reduce guest memory access in vgic_its_trigger_msi

---
  xen/arch/arm/include/asm/vgic.h | 11 +++++++++++
  xen/arch/arm/vgic-v3-its.c      | 19 +++++++++++++++++++
  2 files changed, 30 insertions(+)
xen/arch/arm/include/asm/vgic.h +11 -0
index e309dca1ad..3d8e3a8343 100644
@@ -318,6 +318,17 @@ extern bool vgic_migrate_irq(struct vcpu *old, struct vcpu *new, unsigned int ir
  extern void vgic_check_inflight_irqs_pending(struct vcpu *v,
                                               unsigned int rank, uint32_t r);
 
+#ifdef CONFIG_HAS_ITS
+int vgic_its_trigger_msi(struct domain *d, paddr_t doorbell_address,
+                                u32 devid, u32 eventid);
+#else
+static inline int vgic_its_trigger_msi(struct domain *d, paddr_t doorbell_address,
+                                u32 devid, u32 eventid)
+{
+    return -EOPNOTSUPP;
+}
+#endif /* CONFIG_HAS_ITS */
+
  #endif /* !CONFIG_NEW_VGIC */
 
  /*** Common VGIC functions used by Xen arch code ****/
xen/arch/arm/vgic-v3-its.c +0 -0
index c65c1dbf52..be5bfe0d21 100644

xen/arch/arm/vgic-v3-its.c static int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr, · line 1488

Quoted 5 lines
xen/arch/arm/vgic-v3-its.c +2 -0
@@ -1484,6 +1484,25 @@ static int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr,
      return 0;
  }
 
+int vgic_its_trigger_msi(struct domain *d, paddr_t doorbell_address,
+                                u32 devid, u32 eventid)

Looking at how this is used in the next patch, you are assuming that devid == sbdf. However, given there is no support for guest vITS yet, I don't think we took a decision on whether the device ID match the SBDF.

This goes back to what I wrote in the cover letter. It seems that there are more work needed to be merged before this series.

Also coding style: we use uint32_t in newer code.

xen/arch/arm/vgic-v3-its.c static int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr, · line 1493

Quoted 5 lines
xen/arch/arm/vgic-v3-its.c +5 -0
+{
+    struct pending_irq *pend;
+    unsigned int vcpu_id;
+
+    pend = gicv3_its_get_event_pending_irq(d,doorbell_address, devid, eventid);

A couple of questions:

1. What prevents pending_irq to be freed behind our back?
2. Going back to my point about the missing guest ITS support, it is
unclear how virtual device will work and whether we want QEMU to inject interrupts which belongs to a physical device.

coding style: Missing space before the comma.

Quoted 5 lines
xen/arch/arm/vgic-v3-its.c +5 -0
+    if ( !pend )
+        return -ENOENT;
+
+    vcpu_id = ACCESS_ONCE(pend->lpi_vcpu_id);
+    if ( vcpu_id >= d->max_vcpus )

xen/arch/arm/vgic-v3-its.c static int vgic_v3_its_init_virtual(struct domain *d, paddr_t guest_addr, · line 1508

Quoted 10 lines
xen/arch/arm/vgic-v3-its.c +7 -0
+          return -ENOENT;
+
+    vgic_inject_irq(d, d->vcpu[vcpu_id], pend->irq, true);
+
+    return 0;
+}
+
  unsigned int vgic_v3_its_count(const struct domain *d)
  {
      struct host_its *hw_its;

Cheers,

Signature · 1 line
Julien Grall
#3 lore raw

PATCH v2 2/2 xen/dm: arm: Introduce inject_msi2 DM op

In reply to Mykyta Poturai

From: Mykyta Poturai <[email protected]>

Add the second version of inject_msi DM op, which allows to specify the source_id of an MSI interrupt. This is needed for correct MSI injection on ARM.

It would not be safe to include the source_id in the original inject_msi in the pad field, because we have no way to know if it is set or not.

---
v1->v2:
* fix warning for ignored source_id
* rework xen_dm_op_inject_msi2 struct
* rework xendevicemodel_inject_msi2 params

---
 tools/include/xendevicemodel.h               | 14 ++++++++++++++
 tools/libs/devicemodel/core.c                | 20 ++++++++++++++++++++
 tools/libs/devicemodel/libxendevicemodel.map |  5 +++++
 xen/arch/arm/dm.c                            | 17 +++++++++++++++++
 xen/arch/x86/hvm/dm.c                        | 18 ++++++++++++++++++
 xen/include/public/hvm/dm_op.h               | 18 ++++++++++++++++++
 6 files changed, 92 insertions(+)
tools/include/xendevicemodel.h +14 -0
index 227e7fd810..d0847dfdc8 100644
@@ -236,6 +236,20 @@ int xendevicemodel_inject_msi(
     xendevicemodel_handle *dmod, domid_t domid, uint64_t msi_addr,
     uint32_t msi_data);
 
+/**
+ * This function injects an MSI into a guest.
+ *
+ * @parm dmod a handle to an open devicemodel interface.
+ * @parm domid the domain id to be serviced
+ * @parm msi_addr the MSI address
+ * @parm source_id the PCI SBDF of the source device
+ * @parm msi_data the MSI data
+ * @return 0 on success, -1 on failure.
+*/
+int xendevicemodel_inject_msi2(
+    xendevicemodel_handle *dmod, domid_t domid, uint64_t addr, uint32_t source_id,
+    uint32_t data);
+
 /**
  * This function enables tracking of changes in the VRAM area.
  *
tools/libs/devicemodel/core.c +20 -0
index 8e619eeb0a..92db92e89b 100644
@@ -448,6 +448,26 @@ int xendevicemodel_set_irq_level(
     return xendevicemodel_op(dmod, domid, 1, &op, sizeof(op));
 }
 
+int xendevicemodel_inject_msi2(
+    xendevicemodel_handle *dmod, domid_t domid, uint64_t addr, uint32_t source_id,
+    uint32_t data)
+{
+    uint16_t segment = source_id >> 16;
+    uint16_t bdf = source_id & 0xffff;
+    struct xen_dm_op op = {
+        .op = XEN_DMOP_inject_msi2,
+        .u.inject_msi2 = {
+            .addr = addr,
+            .data = data,
+            .segment = segment,
+            .source_id = bdf,
+            .flags = XEN_DMOP_MSI_SOURCE_ID_VALID,
+        },
+    };
+
+    return xendevicemodel_op(dmod, domid, 1, &op, sizeof(op));
+}
+
 int xendevicemodel_set_pci_link_route(
     xendevicemodel_handle *dmod, domid_t domid, uint8_t link, uint8_t irq)
 {
tools/libs/devicemodel/libxendevicemodel.map +5 -0
index f7f9e3d932..aa05768642 100644
@@ -44,3 +44,8 @@ VERS_1.4 {
 		xendevicemodel_set_irq_level;
 		xendevicemodel_nr_vcpus;
 } VERS_1.3;
+
+VERS_1.5 {
+	global:
+		xendevicemodel_inject_msi2;
+} VERS_1.4;
xen/arch/arm/dm.c +17 -0
index 773a0a2592..a1340b45a3 100644
@@ -27,6 +27,7 @@ int dm_op(const struct dmop_args *op_args)
         [XEN_DMOP_set_ioreq_server_state]           = sizeof(struct xen_dm_op_set_ioreq_server_state),
         [XEN_DMOP_destroy_ioreq_server]             = sizeof(struct xen_dm_op_destroy_ioreq_server),
         [XEN_DMOP_set_irq_level]                    = sizeof(struct xen_dm_op_set_irq_level),
+        [XEN_DMOP_inject_msi2]                      = sizeof(struct xen_dm_op_inject_msi2),
         [XEN_DMOP_nr_vcpus]                         = sizeof(struct xen_dm_op_nr_vcpus),
     };
 
@@ -112,6 +113,22 @@ int dm_op(const struct dmop_args *op_args)
         break;
     }
 
+    case XEN_DMOP_inject_msi2:
+    {
+        const struct xen_dm_op_inject_msi2 *data = &op.u.inject_msi2;
+        uint32_t pci_sbdf = data->segment << 16 | data->source_id;
+
+        if ( !(data->flags & XEN_DMOP_MSI_SOURCE_ID_VALID) || data->pad || 
+             data->flags & ~XEN_DMOP_MSI_SOURCE_ID_VALID )
+        {
+            rc = -EINVAL;
+            break;
+        }
+
+        rc = vgic_its_trigger_msi(d, data->addr, pci_sbdf, data->data);
+        break;
+
+    }
     case XEN_DMOP_nr_vcpus:
     {
         struct xen_dm_op_nr_vcpus *data = &op.u.nr_vcpus;
xen/arch/x86/hvm/dm.c +18 -0
index 462691f91d..da0a00844b 100644
@@ -344,6 +344,7 @@ int dm_op(const struct dmop_args *op_args)
         [XEN_DMOP_set_mem_type]                     = sizeof(struct xen_dm_op_set_mem_type),
         [XEN_DMOP_inject_event]                     = sizeof(struct xen_dm_op_inject_event),
         [XEN_DMOP_inject_msi]                       = sizeof(struct xen_dm_op_inject_msi),
+        [XEN_DMOP_inject_msi2]                      = sizeof(struct xen_dm_op_inject_msi2),
         [XEN_DMOP_map_mem_type_to_ioreq_server]     = sizeof(struct xen_dm_op_map_mem_type_to_ioreq_server),
         [XEN_DMOP_remote_shutdown]                  = sizeof(struct xen_dm_op_remote_shutdown),
         [XEN_DMOP_relocate_memory]                  = sizeof(struct xen_dm_op_relocate_memory),
@@ -539,6 +540,23 @@ int dm_op(const struct dmop_args *op_args)
         break;
     }
 
+    case XEN_DMOP_inject_msi2:
+    {
+        const struct xen_dm_op_inject_msi2 *data = &op.u.inject_msi2;
+
+        if ( data->pad || data->flags & ~XEN_DMOP_MSI_SOURCE_ID_VALID )
+        {
+            rc = -EINVAL;
+            break;
+        }
+
+        if ( data->flags & XEN_DMOP_MSI_SOURCE_ID_VALID )
+            gprintk(XENLOG_WARNING "XEN_DMOP_inject_msi2: source_id is ignored\n");
+
+        rc = hvm_inject_msi(d, data->addr, data->data);
+        break;
+    }
+
     case XEN_DMOP_remote_shutdown:
     {
         const struct xen_dm_op_remote_shutdown *data =
xen/include/public/hvm/dm_op.h +18 -0
index 2bf0fdc1ae..4141af4300 100644
@@ -444,6 +444,23 @@ struct xen_dm_op_nr_vcpus {
 };
 typedef struct xen_dm_op_nr_vcpus xen_dm_op_nr_vcpus_t;
 
+#define XEN_DMOP_inject_msi2 21
+
+struct xen_dm_op_inject_msi2 {
+      /* IN - MSI data */
+      uint32_t data;
+      /* IN - next two fields form an ID of the device triggering the MSI */
+      uint16_t segment; /* The segment number */
+      uint16_t source_id; /* The source ID that is local to segment (PCI BDF) */
+      /* IN - types of source ID */
+      uint32_t flags;
+#define XEN_DMOP_MSI_SOURCE_ID_VALID (1u << 0)
+      uint32_t pad;
+      /* IN - MSI address */
+      uint64_aligned_t addr;
+};
+typedef struct xen_dm_op_inject_msi2 xen_dm_op_inject_msi2_t;
+
 struct xen_dm_op {
     uint32_t op;
     uint32_t pad;
@@ -463,6 +480,7 @@ struct xen_dm_op {
         xen_dm_op_set_mem_type_t set_mem_type;
         xen_dm_op_inject_event_t inject_event;
         xen_dm_op_inject_msi_t inject_msi;
+        xen_dm_op_inject_msi2_t inject_msi2;
         xen_dm_op_map_mem_type_to_ioreq_server_t map_mem_type_to_ioreq_server;
         xen_dm_op_remote_shutdown_t remote_shutdown;
         xen_dm_op_relocate_memory_t relocate_memory;
Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH v2 2/2] xen/dm: arm: Introduce inject_msi2 DM op

Quoted 5 lines · Mykyta Poturai

From: Mykyta Poturai <[email protected]>

Add the second version of inject_msi DM op, which allows to specify the source_id of an MSI interrupt. This is needed for correct MSI injection on ARM.

If this is about Arm, why does x86 also gain (incomplete) handling?

xen/arch/x86/hvm/dm.c int dm_op(const struct dmop_args *op_args) · line 547

Quoted 9 lines
xen/arch/x86/hvm/dm.c +5 -0
@@ -539,6 +540,23 @@ int dm_op(const struct dmop_args *op_args)
         break;
     }
 
+    case XEN_DMOP_inject_msi2:
+    {
+        const struct xen_dm_op_inject_msi2 *data = &op.u.inject_msi2;
+
+        if ( data->pad || data->flags & ~XEN_DMOP_MSI_SOURCE_ID_VALID )

Nit: If the x86 code is to stay in the first place, parentheses please around the & expression. Even if not, similar issues appear to exist in the Arm code.

xen/arch/x86/hvm/dm.c int dm_op(const struct dmop_args *op_args) · line 554

Quoted 7 lines
xen/arch/x86/hvm/dm.c +7 -0
+        {
+            rc = -EINVAL;
+            break;
+        }
+
+        if ( data->flags & XEN_DMOP_MSI_SOURCE_ID_VALID )
+            gprintk(XENLOG_WARNING "XEN_DMOP_inject_msi2: source_id is ignored\n");

Does this compile? It looks to me as if there was a missing comma.

Quoted 7 lines
xen/include/public/hvm/dm_op.h +1 -0
@@ -444,6 +444,23 @@ struct xen_dm_op_nr_vcpus {
 };
 typedef struct xen_dm_op_nr_vcpus xen_dm_op_nr_vcpus_t;
 
+#define XEN_DMOP_inject_msi2 21

xen/include/public/hvm/dm_op.h struct xen_dm_op_nr_vcpus { · line 457

Quoted 10 lines
xen/include/public/hvm/dm_op.h +10 -0
+
+struct xen_dm_op_inject_msi2 {
+      /* IN - MSI data */
+      uint32_t data;
+      /* IN - next two fields form an ID of the device triggering the MSI */
+      uint16_t segment; /* The segment number */
+      uint16_t source_id; /* The source ID that is local to segment (PCI BDF) */
+      /* IN - types of source ID */
+      uint32_t flags;
+#define XEN_DMOP_MSI_SOURCE_ID_VALID (1u << 0)

What purpose does this flag serve? IOW what's the value of using this sub-op when one doesn't want to specify a source ID?

Jan

#5 lore raw
In reply to Mykyta Poturai

Hi Mykyta,

Quoted 13 lines · Mykyta Poturai

This series adds the base support for MSI injection on Arm. This is needed to streamline virtio-pci interrupt triggering.

With this patches, MSIs can be triggered in guests by issuing the new DM op, inject_msi2. This op is similar to inject_msi, but it allows to specify the source id of the MSI.

We chose the approach of adding a new DM op instead of using the pad field of inject_msi because we have no clear way of distinguishing between set and unset pad fields. New implementations also adds flags field to clearly specify if the SBDF is set.

Patches were tested on QEMU with

[...]

Quoted 1 line

patches for ITS support for DomUs applied.

This means this series is unusable without external patches. Given this is adding a new DM operations, I think it would be more sensible to have the vITS support merged first. Then we can look at merging this series.

Cheers,

Signature · 1 line
Julien Grall
#6 lore raw
Quoted 17 lines · Julien Grall

Hi Mykyta,

Quoted 7 lines · Mykyta Poturai

This series adds the base support for MSI injection on Arm. This is needed to streamline virtio-pci interrupt triggering.

With this patches, MSIs can be triggered in guests by issuing the new DM op, inject_msi2. This op is similar to inject_msi, but it allows to specify the source id of the MSI.

Quoted 6 lines

We chose the approach of adding a new DM op instead of using the pad field of inject_msi because we have no clear way of distinguishing between set and unset pad fields. New implementations also adds flags field to clearly specify if the SBDF is set.

Patches were tested on QEMU with

Quoted 7 lines

[...]

Quoted 1 line

patches for ITS support for DomUs applied.

This means this series is unusable without external patches. Given this is adding a new DM operations, I think it would be more sensible to have the vITS support merged first. Then we can look at merging this series.

Hi Mykyta, just checking explicitly with you whether this series requires vgic-v3-its.c for DomUs?

If yes, how far are you from sending the relevant patches to xen-devel? How many are they?

#7 lore raw
Quoted 26 lines · Stefano Stabellini
Quoted 17 lines · Julien Grall

Hi Mykyta,

Quoted 7 lines · Mykyta Poturai

This series adds the base support for MSI injection on Arm. This is needed to streamline virtio-pci interrupt triggering.

With this patches, MSIs can be triggered in guests by issuing the new DM op, inject_msi2. This op is similar to inject_msi, but it allows to specify the source id of the MSI.

Quoted 6 lines

We chose the approach of adding a new DM op instead of using the pad field of inject_msi because we have no clear way of distinguishing between set and unset pad fields. New implementations also adds flags field to clearly specify if the SBDF is set.

Patches were tested on QEMU with

Quoted 7 lines

[...]

Quoted 1 line

patches for ITS support for DomUs applied.

This means this series is unusable without external patches. Given this is adding a new DM operations, I think it would be more sensible to have the vITS support merged first. Then we can look at merging this series.

Quoted 5 lines

Hi Mykyta, just checking explicitly with you whether this series requires vgic-v3-its.c for DomUs?

If yes, how far are you from sending the relevant patches to xen-devel? How many are they?

Hi Stefano,
Yes, I am planning to send them together with VPCI MSI support after SMMU patches are merged.

Although the DomU vits itself is only two patches. 1.
https://github.com/Deedone/xen/commit/c7fe54e776e4205fbb993e47a6b826f5154fad76
2.
https://github.com/Deedone/xen/commit/b6b4ffadb368f540a0ae8ef6b4572f0cfd2eccaa

--
Mykyta

#8 lore raw

(+ Rahul)

Quoted 30 lines · Mykyta Poturai
Quoted 29 lines · Stefano Stabellini
Quoted 17 lines · Julien Grall

Hi Mykyta,

Quoted 7 lines · Mykyta Poturai

This series adds the base support for MSI injection on Arm. This is needed to streamline virtio-pci interrupt triggering.

With this patches, MSIs can be triggered in guests by issuing the new DM op, inject_msi2. This op is similar to inject_msi, but it allows to specify the source id of the MSI.

Quoted 6 lines

We chose the approach of adding a new DM op instead of using the pad field of inject_msi because we have no clear way of distinguishing between set and unset pad fields. New implementations also adds flags field to clearly specify if the SBDF is set.

Patches were tested on QEMU with

Quoted 7 lines

[...]

Quoted 1 line

patches for ITS support for DomUs applied.

This means this series is unusable without external patches. Given this is adding a new DM operations, I think it would be more sensible to have the vITS support merged first. Then we can look at merging this series.

Hi Mykyta, just checking explicitly with you whether this series requires vgic-v3-its.c for DomUs?

Quoted 8 lines
Quoted 2 lines

If yes, how far are you from sending the relevant patches to xen-devel? How many are they?

Hi Stefano,
Yes, I am planning to send them together with VPCI MSI support after SMMU patches are merged.

Although the DomU vits itself is only two patches.

I am assuming this is work in progress patches rather than the one you plan to send, correct?

Asking, because currently there are a few ASSERT(is_hardware_domain(its->d)) to indicate where changes are likely for the guests. You seem to remove them without explaining why or any associated code.

While I will not ask to have a security support guest vITS from the start. I would like some confidence that we are going in the right direction. IOW, I would like to see a design document that would explain how we can achieve it. Some of the part to consider:
* Command queue
* LPIs
* 1:1 pITS <-> vITS vs one vITS to many pITS
* The page-tables are shared between the device and CPU. Are we ok to
keep the doorbell writable by the CPU?

There was some discussion in the past about it (I have added Rahul because IIRC he was driving the discussion). So most likely, we would need the design to be respinned and agreed first.

Lastly, I see you seem to go down the route of exposing one vITS only. But I don't think your patch is sufficient to support multiple pITS (the guest doorbell will be mapped to a different host doorbell depending on the guest).

Cheers,

Signature · 1 line
Julien Grall
#9 lore raw
Quoted 61 lines · Julien Grall

(+ Rahul)

Quoted 32 lines · Mykyta Poturai
Quoted 31 lines · Stefano Stabellini
Quoted 19 lines · Julien Grall

Hi Mykyta,

Quoted 13 lines · Mykyta Poturai

This series adds the base support for MSI injection on Arm. This is needed to streamline virtio-pci interrupt triggering.

With this patches, MSIs can be triggered in guests by issuing the new DM op, inject_msi2. This op is similar to inject_msi, but it allows to specify the source id of the MSI.

We chose the approach of adding a new DM op instead of using the pad field of inject_msi because we have no clear way of distinguishing between set and unset pad fields. New implementations also adds flags field to clearly specify if the SBDF is set.

Patches were tested on QEMU with

[...]

Quoted 7 lines
Quoted 1 line

patches for ITS support for DomUs applied.

This means this series is unusable without external patches. Given this is
adding a new DM operations, I think it would be more sensible to have the vITS
support merged first. Then we can look at merging this series.

Hi Mykyta, just checking explicitly with you whether this series requires vgic-v3-its.c for DomUs?

Quoted 8 lines
Quoted 2 lines

If yes, how far are you from sending the relevant patches to xen-devel? How many are they?

Hi Stefano,
Yes, I am planning to send them together with VPCI MSI support after SMMU patches are merged.

Although the DomU vits itself is only two patches.

I am assuming this is work in progress patches rather than the one you plan to send, correct?

Asking, because currently there are a few ASSERT(is_hardware_domain(its->d)) to indicate where changes are likely for the guests. You seem to remove them without explaining why or any associated code.

While I will not ask to have a security support guest vITS from the start. I would like some confidence that we are going in the right direction. IOW, I would like to see a design document that would explain how we can achieve it. Some of the part to consider:
* Command queue
* LPIs
* 1:1 pITS <-> vITS vs one vITS to many pITS
* The page-tables are shared between the device and CPU. Are we ok to keep
the doorbell writable by the CPU?

Quoted 7 lines

There was some discussion in the past about it (I have added Rahul because IIRC he was driving the discussion). So most likely, we would need the design to be respinned and agreed first.

Lastly, I see you seem to go down the route of exposing one vITS only. But I don't think your patch is sufficient to support multiple pITS (the guest doorbell will be mapped to a different host doorbell depending on the guest).

To add to what Julien wrote, one thing is enabling a component with some limitations, such as lacking a given feature. That might be OK.

A different thing is lack of isolation between VMs, or introducing a new vehicle for interference, e.g. one VM using vITS causing troubles to another VM using vITS.

Lack of isolation and interference are not OK. We need to be very careful and be very aware if there any constraints or limitations we are hitting against (e.g. size of the command queue) where one VM could cause a denial of service to another VM because it is exhausting a given physical resource.