Skip to contentSkip to conversation

x86/ioapic: fix edge triggered interrupt migration

PATCH In review Roger Pau Monne 3 participants 13 messages Jul 26, 2023 – Aug 1, 2023

lore
Share & tools
lore
#0 lore raw

v3 cover x86/ioapic: fix edge triggered interrupt migration

Hello,

Following series attempts to solve the issue with IO-APIC edge triggered interrupts seeing an inconsistent IRTE when injected while being migrated.

It's been simplified a bit from the original RFC, and does make the logic in the IOMMU RTE update handlers cleaner, as those get passed the full RTE.

When not using interrupt remapping the logic is left as-is. One further change would be to mask edge triggered interrupts, as destination and vector cannot be changed atomically when dealing with plain IO-APIC RTEs.

The previously final patch to switch AMD IOMMU to use atomic RTE updates has been drop pending feedback from AMD on whether that approach is suitable.

Thanks, Roger.

Roger Pau Monne (4):
x86/ioapic: add a raw field to RTE struct x86/ioapic: RTE modifications must use ioapic_write_entry iommu/vtd: rename io_apic_read_remap_rte() local variable x86/iommu: pass full IO-APIC RTE for remapping table update

 xen/arch/x86/include/asm/io_apic.h       |  65 ++++++-----
 xen/arch/x86/include/asm/iommu.h         |   3 +-
 xen/arch/x86/io_apic.c                   |  46 ++++----
 xen/drivers/passthrough/amd/iommu.h      |   2 +-
 xen/drivers/passthrough/amd/iommu_intr.c | 106 ++---------------
 xen/drivers/passthrough/vtd/extern.h     |   2 +-
 xen/drivers/passthrough/vtd/intremap.c   | 140 +++++++++++------------
 xen/drivers/passthrough/x86/iommu.c      |   4 +-
 xen/include/xen/iommu.h                  |   3 +-
 9 files changed, 142 insertions(+), 229 deletions(-)
Signature · 1 line
2.41.0
#1 lore raw

PATCH v3 1/4 x86/ioapic: add a raw field to RTE struct

Further changes will require access to the full RTE as a single value in order to pass it to IOMMU interrupt remapping handlers.

No functional change intended.

---
Changes since v1:
- Fix initializers.
---
Tested on gitlab CI, builds on all compilers used there.

---
 xen/arch/x86/include/asm/io_apic.h       | 57 +++++++++++++-----------
 xen/arch/x86/io_apic.c                   |  2 +-
 xen/drivers/passthrough/amd/iommu_intr.c |  4 +-
 xen/drivers/passthrough/vtd/intremap.c   |  4 +-
 4 files changed, 35 insertions(+), 32 deletions(-)
xen/arch/x86/include/asm/io_apic.h +30 -27
index bd8cb95f46b1..4c4777b68a51 100644
@@ -90,35 +90,38 @@ enum ioapic_irq_destination_types {
 };
 
 struct IO_APIC_route_entry {
-    unsigned int vector:8;
-    unsigned int delivery_mode:3; /*
-                                   * 000: FIXED
-                                   * 001: lowest prio
-                                   * 111: ExtINT
-                                   */
-    unsigned int dest_mode:1;     /* 0: physical, 1: logical */
-    unsigned int delivery_status:1;
-    unsigned int polarity:1;      /* 0: low, 1: high */
-    unsigned int irr:1;
-    unsigned int trigger:1;       /* 0: edge, 1: level */
-    unsigned int mask:1;          /* 0: enabled, 1: disabled */
-    unsigned int __reserved_2:15;
-
     union {
         struct {
-            unsigned int __reserved_1:24;
-            unsigned int physical_dest:4;
-            unsigned int __reserved_2:4;
-        } physical;
-
-        struct {
-            unsigned int __reserved_1:24;
-            unsigned int logical_dest:8;
-        } logical;
-
-        /* used when Interrupt Remapping with EIM is enabled */
-        unsigned int dest32;
-    } dest;
+            unsigned int vector:8;
+            unsigned int delivery_mode:3; /*
+                                           * 000: FIXED
+                                           * 001: lowest prio
+                                           * 111: ExtINT
+                                           */
+            unsigned int dest_mode:1;     /* 0: physical, 1: logical */
+            unsigned int delivery_status:1;
+            unsigned int polarity:1;      /* 0: low, 1: high */
+            unsigned int irr:1;
+            unsigned int trigger:1;       /* 0: edge, 1: level */
+            unsigned int mask:1;          /* 0: enabled, 1: disabled */
+            unsigned int __reserved_2:15;
+
+            union {
+                struct {
+                    unsigned int __reserved_1:24;
+                    unsigned int physical_dest:4;
+                    unsigned int __reserved_2:4;
+                } physical;
+
+                struct {
+                    unsigned int __reserved_1:24;
+                    unsigned int logical_dest:8;
+                } logical;
+                unsigned int dest32;
+            } dest;
+        };
+        uint64_t raw;
+    };
 };
 
 /*
xen/arch/x86/io_apic.c +1 -1
index 25a08b1ea6c6..aada2ef96c62 100644
@@ -2360,7 +2360,7 @@ int ioapic_guest_read(unsigned long physbase, unsigned int reg, u32 *pval)
 int ioapic_guest_write(unsigned long physbase, unsigned int reg, u32 val)
 {
     int apic, pin, irq, ret, pirq;
-    struct IO_APIC_route_entry rte = { 0 };
+    struct IO_APIC_route_entry rte = { };
     unsigned long flags;
     struct irq_desc *desc;
 
xen/drivers/passthrough/amd/iommu_intr.c +2 -2
index f4de09f43180..9e6be3be3515 100644
@@ -352,8 +352,8 @@ static int update_intremap_entry_from_ioapic(
 void cf_check amd_iommu_ioapic_update_ire(
     unsigned int apic, unsigned int reg, unsigned int value)
 {
-    struct IO_APIC_route_entry old_rte = { 0 };
-    struct IO_APIC_route_entry new_rte = { 0 };
+    struct IO_APIC_route_entry old_rte = { };
+    struct IO_APIC_route_entry new_rte = { };
     unsigned int rte_lo = (reg & 1) ? reg - 1 : reg;
     unsigned int pin = (reg - 0x10) / 2;
     int seg, bdf, rc;
xen/drivers/passthrough/vtd/intremap.c +2 -2
index b39bc832821a..786388b4d9c7 100644
@@ -419,7 +419,7 @@ unsigned int cf_check io_apic_read_remap_rte(
 {
     unsigned int ioapic_pin = (reg - 0x10) / 2;
     int index;
-    struct IO_xAPIC_route_entry old_rte = { 0 };
+    struct IO_xAPIC_route_entry old_rte = { };
     int rte_upper = (reg & 1) ? 1 : 0;
     struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic));
 
@@ -442,7 +442,7 @@ void cf_check io_apic_write_remap_rte(
     unsigned int apic, unsigned int reg, unsigned int value)
 {
     unsigned int ioapic_pin = (reg - 0x10) / 2;
-    struct IO_xAPIC_route_entry old_rte = { 0 };
+    struct IO_xAPIC_route_entry old_rte = { };
     struct IO_APIC_route_remap_entry *remap_rte;
     unsigned int rte_upper = (reg & 1) ? 1 : 0;
     struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic));
Signature · 1 line
2.41.0
#2 lore raw

PATCH v3 2/4 x86/ioapic: RTE modifications must use ioapic_write_entry

In reply to Roger Pau Monne

Do not allow to write to RTE registers using io_apic_write and instead require changes to RTE to be performed using ioapic_write_entry.

This is in preparation for passing the full contents of the RTE to the IOMMU interrupt remapping handlers, so remapping entries for IO-APIC RTEs can be updated atomically when possible.

While immediately this commit might expand the number of MMIO accesses in order to update an IO-APIC RTE, further changes will benefit from getting the full RTE value passed to the IOMMU handlers, as the logic is greatly simplified when the IOMMU handlers can get the complete RTE value in one go.

---
Changes since v2:
- Also adjust __ioapic_read_entry().

Changes since v1:
- Reinstate io_apic_modify().
- Expand commit message.

---
 xen/arch/x86/include/asm/io_apic.h       |  8 ++---
 xen/arch/x86/io_apic.c                   | 43 ++++++++++++------------
 xen/drivers/passthrough/amd/iommu_intr.c |  6 ----
 3 files changed, 25 insertions(+), 32 deletions(-)
xen/arch/x86/include/asm/io_apic.h +4 -4
index 4c4777b68a51..9165da2281ae 100644
@@ -162,8 +162,8 @@ static inline void __io_apic_write(unsigned int apic, unsigned int reg, unsigned
 
 static inline void io_apic_write(unsigned int apic, unsigned int reg, unsigned int value)
 {
-    if ( ioapic_reg_remapped(reg) )
-        return iommu_update_ire_from_apic(apic, reg, value);
+    /* RTE writes must use ioapic_write_entry. */
+    BUG_ON(reg >= 0x10);
     __io_apic_write(apic, reg, value);
 }
 
@@ -173,8 +173,8 @@ static inline void io_apic_write(unsigned int apic, unsigned int reg, unsigned i
  */
 static inline void io_apic_modify(unsigned int apic, unsigned int reg, unsigned int value)
 {
-    if ( ioapic_reg_remapped(reg) )
-        return iommu_update_ire_from_apic(apic, reg, value);
+    /* RTE writes must use ioapic_write_entry. */
+    BUG_ON(reg >= 0x10);
     *(IO_APIC_BASE(apic) + 4) = value;
 }
 
xen/arch/x86/io_apic.c +21 -22

93 lines not on this page · raw message

xen/drivers/passthrough/amd/iommu_intr.c +0 -6
index 9e6be3be3515..f32c418a7e49 100644
@@ -361,12 +361,6 @@ void cf_check amd_iommu_ioapic_update_ire(
     struct amd_iommu *iommu;
     unsigned int idx;
 
-    if ( !iommu_intremap )
-    {
-        __io_apic_write(apic, reg, value);
-        return;
-    }
-
     idx = ioapic_id_to_index(IO_APIC_ID(apic));
     if ( idx == MAX_IO_APICS )
         return;
Signature · 1 line
2.41.0
Reviewed-by on v3 2/4
#3 lore raw

Re: [PATCH v3 2/4] x86/ioapic: RTE modifications must use ioapic_write_entry

Quoted 7 lines · Roger Pau Monne

Do not allow to write to RTE registers using io_apic_write and instead require changes to RTE to be performed using ioapic_write_entry.

This is in preparation for passing the full contents of the RTE to the IOMMU interrupt remapping handlers, so remapping entries for IO-APIC RTEs can be updated atomically when possible.

Quoted 7 lines

While immediately this commit might expand the number of MMIO accesses in order to update an IO-APIC RTE, further changes will benefit from getting the full RTE value passed to the IOMMU handlers, as the logic is greatly simplified when the IOMMU handlers can get the complete RTE value in one go.

#4 lore raw

PATCH v3 3/4 iommu/vtd: rename io_apic_read_remap_rte() local variable

In reply to Roger Pau Monne

Preparatory change to unify the IO-APIC pin variable name between io_apic_read_remap_rte() and amd_iommu_ioapic_update_ire(), so that the local variable can be made a function parameter with the same name across vendors.

---
Changes since v1:
- New in this version.

---
 xen/drivers/passthrough/vtd/intremap.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
xen/drivers/passthrough/vtd/intremap.c +4 -4
index 786388b4d9c7..05df6d5759b1 100644
@@ -441,14 +441,14 @@ unsigned int cf_check io_apic_read_remap_rte(
 void cf_check io_apic_write_remap_rte(
     unsigned int apic, unsigned int reg, unsigned int value)
 {
-    unsigned int ioapic_pin = (reg - 0x10) / 2;
+    unsigned int pin = (reg - 0x10) / 2;
     struct IO_xAPIC_route_entry old_rte = { };
     struct IO_APIC_route_remap_entry *remap_rte;
     unsigned int rte_upper = (reg & 1) ? 1 : 0;
     struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic));
     int saved_mask;
 
-    old_rte = __ioapic_read_entry(apic, ioapic_pin, true);
+    old_rte = __ioapic_read_entry(apic, pin, true);
 
     remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte;
 
@@ -458,7 +458,7 @@ void cf_check io_apic_write_remap_rte(
     __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
     remap_rte->mask = saved_mask;
 
-    if ( ioapic_rte_to_remap_entry(iommu, apic, ioapic_pin,
+    if ( ioapic_rte_to_remap_entry(iommu, apic, pin,
                                    &old_rte, rte_upper, value) )
     {
         __io_apic_write(apic, reg, value);
@@ -468,7 +468,7 @@ void cf_check io_apic_write_remap_rte(
             __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
     }
     else
-        __ioapic_write_entry(apic, ioapic_pin, true, old_rte);
+        __ioapic_write_entry(apic, pin, true, old_rte);
 }
 
 static void set_msi_source_id(struct pci_dev *pdev, struct iremap_entry *ire)
Signature · 1 line
2.41.0
Reviewed-by on v3 3/4
#5 lore raw

RE: [PATCH v3 3/4] iommu/vtd: rename io_apic_read_remap_rte() local variable

Quoted 9 lines

From: Roger Pau Monne <[email protected]>
Sent: Wednesday, July 26, 2023 8:55 PM

Preparatory change to unify the IO-APIC pin variable name between io_apic_read_remap_rte() and amd_iommu_ioapic_update_ire(), so that the local variable can be made a function parameter with the same name across vendors.

#6 lore raw

PATCH v3 4/4 x86/iommu: pass full IO-APIC RTE for remapping table update

In reply to Roger Pau Monne

So that the remapping entry can be updated atomically when possible.

Doing such update atomically will avoid Xen having to mask the IO-APIC pin prior to performing any interrupt movements (ie: changing the destination and vector fields), as the interrupt remapping entry is always consistent.

This also simplifies some of the logic on both VT-d and AMD-Vi implementations, as having the full RTE available instead of half of it avoids to possibly read and update the missing other half from hardware.

While there remove the explicit zeroing of new_ire fields in ioapic_rte_to_remap_entry() and initialize the variable at definition so all fields are zeroed. Note fields could be also initialized with final values at definition, but I found that likely too much to be done at this time.

---
Changes since v2:
- Remove unneeded initialization.
- Use 'rte' as parameter name for update_ire_from_apic() implementations.
- Fix comment style in ioapic_rte_to_remap_entry().
- Fix requirement for atomic write in update_irte() call from ioapic_rte_to_remap_entry().
- Remove remap_rte from io_apic_write_remap_rte().
---
Note that certain combination of changes to the RTE are impossible to handle atomically. For example changing the vector and/or destination fields together with the triggering mode is impossible to be performed atomically (as the destination and vector is set in the IRTE, but the triggering mode is set in the RTE). Xen doesn't attempt to perform such changes in a single update to the RTE anyway, so it's fine.

Naming the iommu_update_ire_from_apic() parameter RTE is not really correct, as the format of the passed value expands the destination field to be 32bits (in order to fit an x2APIC ID). Passing an IO_APIC_route_entry struct is not possible due to the circular dependency that would create between io_apic.h and iommu.h. It might be possible to move IO_APIC_route_entry declaration to a different header, but I haven't looked into it.

---
 xen/arch/x86/include/asm/iommu.h         |   3 +-
 xen/arch/x86/io_apic.c                   |   5 +-
 xen/drivers/passthrough/amd/iommu.h      |   2 +-
 xen/drivers/passthrough/amd/iommu_intr.c | 100 ++---------------
 xen/drivers/passthrough/vtd/extern.h     |   2 +-
 xen/drivers/passthrough/vtd/intremap.c   | 136 +++++++++++------------
 xen/drivers/passthrough/x86/iommu.c      |   4 +-
 xen/include/xen/iommu.h                  |   3 +-
 8 files changed, 86 insertions(+), 169 deletions(-)
xen/arch/x86/include/asm/iommu.h +2 -1
index 0540cd9faa87..eb720205e25e 100644
@@ -84,7 +84,8 @@ struct iommu_init_ops {
 
 extern const struct iommu_init_ops *iommu_init_ops;
 
-void iommu_update_ire_from_apic(unsigned int apic, unsigned int reg, unsigned int value);
+void iommu_update_ire_from_apic(unsigned int apic, unsigned int pin,
+                                uint64_t rte);
 unsigned int iommu_read_apic_from_ire(unsigned int apic, unsigned int reg);
 int iommu_setup_hpet_msi(struct msi_desc *);
 
xen/arch/x86/io_apic.c +1 -4
index 041233b9b706..b3afef8933d7 100644
@@ -275,10 +275,7 @@ void __ioapic_write_entry(
         __io_apic_write(apic, 0x10 + 2 * pin, eu.w1);
     }
     else
-    {
-        iommu_update_ire_from_apic(apic, 0x11 + 2 * pin, eu.w2);
-        iommu_update_ire_from_apic(apic, 0x10 + 2 * pin, eu.w1);
-    }
+        iommu_update_ire_from_apic(apic, pin, e.raw);
 }
 
 static void ioapic_write_entry(
xen/drivers/passthrough/amd/iommu.h +1 -1
index 8bc3c35b1bb1..5429ada58ef5 100644
@@ -300,7 +300,7 @@ int cf_check amd_iommu_free_intremap_table(
 unsigned int amd_iommu_intremap_table_order(
     const void *irt, const struct amd_iommu *iommu);
 void cf_check amd_iommu_ioapic_update_ire(
-    unsigned int apic, unsigned int reg, unsigned int value);
+    unsigned int apic, unsigned int pin, uint64_t rte);
 unsigned int cf_check amd_iommu_read_ioapic_from_ire(
     unsigned int apic, unsigned int reg);
 int cf_check amd_iommu_msi_msg_update_ire(
xen/drivers/passthrough/amd/iommu_intr.c +9 -91

150 lines not on this page · raw message

xen/drivers/passthrough/vtd/extern.h +1 -1
index 39602d1f88f8..b684e9393d1a 100644
@@ -92,7 +92,7 @@ int cf_check intel_iommu_get_reserved_device_memory(
 unsigned int cf_check io_apic_read_remap_rte(
     unsigned int apic, unsigned int reg);
 void cf_check io_apic_write_remap_rte(
-    unsigned int apic, unsigned int reg, unsigned int value);
+    unsigned int apic, unsigned int ioapic_pin, uint64_t rte);
 
 struct msi_desc;
 struct msi_msg;
xen/drivers/passthrough/vtd/intremap.c +68 -68

167 lines not on this page · raw message

xen/drivers/passthrough/x86/iommu.c +2 -2
index be71a4c4641c..d290855959f2 100644
@@ -158,9 +158,9 @@ int iommu_enable_x2apic(void)
 }
 
 void iommu_update_ire_from_apic(
-    unsigned int apic, unsigned int reg, unsigned int value)
+    unsigned int apic, unsigned int pin, uint64_t rte)
 {
-    iommu_vcall(&iommu_ops, update_ire_from_apic, apic, reg, value);
+    iommu_vcall(&iommu_ops, update_ire_from_apic, apic, pin, rte);
 }
 
 unsigned int iommu_read_apic_from_ire(unsigned int apic, unsigned int reg)
xen/include/xen/iommu.h +2 -1
index 405db59971c5..9335cd074cff 100644
@@ -278,7 +278,8 @@ struct iommu_ops {
     int (*enable_x2apic)(void);
     void (*disable_x2apic)(void);
 
-    void (*update_ire_from_apic)(unsigned int apic, unsigned int reg, unsigned int value);
+    void (*update_ire_from_apic)(unsigned int apic, unsigned int pin,
+                                 uint64_t rte);
     unsigned int (*read_apic_from_ire)(unsigned int apic, unsigned int reg);
 
     int (*setup_hpet_msi)(struct msi_desc *);
Signature · 1 line
2.41.0
#7 lore raw

Re: [PATCH v3 4/4] x86/iommu: pass full IO-APIC RTE for remapping table update

xen/drivers/passthrough/vtd/extern.h int cf_check intel_iommu_get_reserved_device_memory( · line 95

Quoted 8 lines · Roger Pau Monne
xen/drivers/passthrough/vtd/extern.h +1 -1
@@ -92,7 +92,7 @@ int cf_check intel_iommu_get_reserved_device_memory(
 unsigned int cf_check io_apic_read_remap_rte(
     unsigned int apic, unsigned int reg);
 void cf_check io_apic_write_remap_rte(
-    unsigned int apic, unsigned int reg, unsigned int value);
+    unsigned int apic, unsigned int ioapic_pin, uint64_t rte);

Forgot to rename the middle parameter to "pin"?

Quoted 39 lines
xen/drivers/passthrough/vtd/extern.h +2 -32
@@ -364,48 +363,40 @@ static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu,
 
     new_ire = *iremap_entry;
 
-    if ( rte_upper )
-    {
-        if ( x2apic_enabled )
-            new_ire.remap.dst = value;
-        else
-            new_ire.remap.dst = (value >> 24) << 8;
-    }
+    if ( x2apic_enabled )
+        new_ire.remap.dst = new_rte.dest.dest32;
     else
-    {
-        *(((u32 *)&new_rte) + 0) = value;
-        new_ire.remap.fpd = 0;
-        new_ire.remap.dm = new_rte.dest_mode;
-        new_ire.remap.tm = new_rte.trigger;
-        new_ire.remap.dlm = new_rte.delivery_mode;
-        /* Hardware require RH = 1 for LPR delivery mode */
-        new_ire.remap.rh = (new_ire.remap.dlm == dest_LowestPrio);
-        new_ire.remap.avail = 0;
-        new_ire.remap.res_1 = 0;
-        new_ire.remap.vector = new_rte.vector;
-        new_ire.remap.res_2 = 0;
-
-        set_ioapic_source_id(IO_APIC_ID(apic), &new_ire);
-        new_ire.remap.res_3 = 0;
-        new_ire.remap.res_4 = 0;
-        new_ire.remap.p = 1;     /* finally, set present bit */
-
-        /* now construct new ioapic rte entry */
-        remap_rte->vector = new_rte.vector;
-        remap_rte->delivery_mode = 0;    /* has to be 0 for remap format */
-        remap_rte->index_15 = (index >> 15) & 0x1;
-        remap_rte->index_0_14 = index & 0x7fff;
-
-        remap_rte->delivery_status = new_rte.delivery_status;

xen/drivers/passthrough/vtd/extern.h static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu, · line 369

Quoted 10 lines
xen/drivers/passthrough/vtd/extern.h +1 -9
-        remap_rte->polarity = new_rte.polarity;
-        remap_rte->irr = new_rte.irr;
-        remap_rte->trigger = new_rte.trigger;
-        remap_rte->mask = new_rte.mask;
-        remap_rte->reserved = 0;
-        remap_rte->format = 1;    /* indicate remap format */
-    }
-
-    update_irte(iommu, iremap_entry, &new_ire, !init);
+        new_ire.remap.dst = (new_rte.dest.dest32 >> 24) << 8;

I realize this was this way before, but I wonder if we couldn't use GET_xAPIC_ID() here to reduce the open-coding at least a bit.

Quoted 50 lines
xen/drivers/passthrough/vtd/extern.h +19 -21
@@ -439,36 +430,45 @@ unsigned int cf_check io_apic_read_remap_rte(
 }
 
 void cf_check io_apic_write_remap_rte(
-    unsigned int apic, unsigned int reg, unsigned int value)
+    unsigned int apic, unsigned int pin, uint64_t rte)
 {
-    unsigned int pin = (reg - 0x10) / 2;
+    struct IO_xAPIC_route_entry new_rte = { .raw = rte };
     struct IO_xAPIC_route_entry old_rte = { };
-    struct IO_APIC_route_remap_entry *remap_rte;
-    unsigned int rte_upper = (reg & 1) ? 1 : 0;
     struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic));
-    int saved_mask;
-
-    old_rte = __ioapic_read_entry(apic, pin, true);
+    bool masked = true;
+    int rc;
 
-    remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte;
-
-    /* mask the interrupt while we change the intremap table */
-    saved_mask = remap_rte->mask;
-    remap_rte->mask = 1;
-    __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
-    remap_rte->mask = saved_mask;
-
-    if ( ioapic_rte_to_remap_entry(iommu, apic, pin,
-                                   &old_rte, rte_upper, value) )
+    if ( !cpu_has_cx16 )
     {
-        __io_apic_write(apic, reg, value);
+       /*
+        * Cannot atomically update the IRTE entry: mask the IO-APIC pin to
+        * avoid interrupts seeing an inconsistent IRTE entry.
+        */
+        old_rte = __ioapic_read_entry(apic, pin, true);
+        if ( !old_rte.mask )
+        {
+            masked = false;
+            old_rte.mask = 1;
+            __ioapic_write_entry(apic, pin, true, old_rte);
+        }
+    }
 
-        /* Recover the original value of 'mask' bit */
-        if ( rte_upper )
-            __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
+    rc = ioapic_rte_to_remap_entry(iommu, apic, pin, &old_rte, new_rte);
+    if ( rc )

xen/drivers/passthrough/vtd/extern.h unsigned int cf_check io_apic_read_remap_rte( · line 467

Quoted 10 lines
xen/drivers/passthrough/vtd/extern.h +10 -0
+    {
+        if ( !masked )
+        {
+            /* Recover the original value of 'mask' bit */
+            old_rte.mask = 0;
+            __ioapic_write_entry(apic, pin, true, old_rte);
+        }
+        dprintk(XENLOG_ERR VTDPREFIX,
+                "failed to update IRTE for IO-APIC#%u pin %u: %d\n",
+                apic, pin, rc);

Afaics you don't alter the error behavior of ioapic_rte_to_remap_entry(), and in the sole (pre-existing) case of an error a debug log message is already being issued. Why the addition?

Jan

#8 lore raw

Re: [PATCH v3 4/4] x86/iommu: pass full IO-APIC RTE for remapping table update

Quoted 57 lines · Jan Beulich
Quoted 56 lines · Roger Pau Monne
xen/drivers/passthrough/vtd/intremap.c +25 -21
@@ -439,36 +430,45 @@ unsigned int cf_check io_apic_read_remap_rte(
 }
 
 void cf_check io_apic_write_remap_rte(
-    unsigned int apic, unsigned int reg, unsigned int value)
+    unsigned int apic, unsigned int pin, uint64_t rte)
 {
-    unsigned int pin = (reg - 0x10) / 2;
+    struct IO_xAPIC_route_entry new_rte = { .raw = rte };
     struct IO_xAPIC_route_entry old_rte = { };
-    struct IO_APIC_route_remap_entry *remap_rte;
-    unsigned int rte_upper = (reg & 1) ? 1 : 0;
     struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic));
-    int saved_mask;
-
-    old_rte = __ioapic_read_entry(apic, pin, true);
+    bool masked = true;
+    int rc;
 
-    remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte;
-
-    /* mask the interrupt while we change the intremap table */
-    saved_mask = remap_rte->mask;
-    remap_rte->mask = 1;
-    __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
-    remap_rte->mask = saved_mask;
-
-    if ( ioapic_rte_to_remap_entry(iommu, apic, pin,
-                                   &old_rte, rte_upper, value) )
+    if ( !cpu_has_cx16 )
     {
-        __io_apic_write(apic, reg, value);
+       /*
+        * Cannot atomically update the IRTE entry: mask the IO-APIC pin to
+        * avoid interrupts seeing an inconsistent IRTE entry.
+        */
+        old_rte = __ioapic_read_entry(apic, pin, true);
+        if ( !old_rte.mask )
+        {
+            masked = false;
+            old_rte.mask = 1;
+            __ioapic_write_entry(apic, pin, true, old_rte);
+        }
+    }
 
-        /* Recover the original value of 'mask' bit */
-        if ( rte_upper )
-            __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
+    rc = ioapic_rte_to_remap_entry(iommu, apic, pin, &old_rte, new_rte);
+    if ( rc )
+    {
+        if ( !masked )
+        {
+            /* Recover the original value of 'mask' bit */
+            old_rte.mask = 0;
+            __ioapic_write_entry(apic, pin, true, old_rte);
Quoted 8 lines

xen/drivers/passthrough/vtd/intremap.c unsigned int cf_check io_apic_read_remap_rte( · line 467

Quoted 4 lines
xen/drivers/passthrough/vtd/intremap.c +4 -0
+        }
+        dprintk(XENLOG_ERR VTDPREFIX,
+                "failed to update IRTE for IO-APIC#%u pin %u: %d\n",
+                apic, pin, rc);

Afaics you don't alter the error behavior of ioapic_rte_to_remap_entry(), and in the sole (pre-existing) case of an error a debug log message is already being issued. Why the addition?

I think I was trying to mimic the behavior of amd_iommu_ioapic_update_ire(), which does print the errors as opposed to doing so in update_intremap_entry_from_ioapic().

I've now removed it, and adjusted the code to match the rest of your comments. Will post v4 of 4/4 only as a reply to v3, I don't think there's a need to resend the rest unless you prefer it that way.

Thanks, Roger.

#9 lore raw

Re: [PATCH v3 4/4] x86/iommu: pass full IO-APIC RTE for remapping table update

Quoted 67 lines · Roger Pau Monné
Quoted 57 lines · Jan Beulich
Quoted 56 lines · Roger Pau Monne
xen/drivers/passthrough/vtd/intremap.c +25 -21
@@ -439,36 +430,45 @@ unsigned int cf_check io_apic_read_remap_rte(
 }
 
 void cf_check io_apic_write_remap_rte(
-    unsigned int apic, unsigned int reg, unsigned int value)
+    unsigned int apic, unsigned int pin, uint64_t rte)
 {
-    unsigned int pin = (reg - 0x10) / 2;
+    struct IO_xAPIC_route_entry new_rte = { .raw = rte };
     struct IO_xAPIC_route_entry old_rte = { };
-    struct IO_APIC_route_remap_entry *remap_rte;
-    unsigned int rte_upper = (reg & 1) ? 1 : 0;
     struct vtd_iommu *iommu = ioapic_to_iommu(IO_APIC_ID(apic));
-    int saved_mask;
-
-    old_rte = __ioapic_read_entry(apic, pin, true);
+    bool masked = true;
+    int rc;
 
-    remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte;
-
-    /* mask the interrupt while we change the intremap table */
-    saved_mask = remap_rte->mask;
-    remap_rte->mask = 1;
-    __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
-    remap_rte->mask = saved_mask;
-
-    if ( ioapic_rte_to_remap_entry(iommu, apic, pin,
-                                   &old_rte, rte_upper, value) )
+    if ( !cpu_has_cx16 )
     {
-        __io_apic_write(apic, reg, value);
+       /*
+        * Cannot atomically update the IRTE entry: mask the IO-APIC pin to
+        * avoid interrupts seeing an inconsistent IRTE entry.
+        */
+        old_rte = __ioapic_read_entry(apic, pin, true);
+        if ( !old_rte.mask )
+        {
+            masked = false;
+            old_rte.mask = 1;
+            __ioapic_write_entry(apic, pin, true, old_rte);
+        }
+    }
 
-        /* Recover the original value of 'mask' bit */
-        if ( rte_upper )
-            __io_apic_write(apic, reg & ~1, *(u32 *)&old_rte);
+    rc = ioapic_rte_to_remap_entry(iommu, apic, pin, &old_rte, new_rte);
+    if ( rc )
+    {
+        if ( !masked )
+        {
+            /* Recover the original value of 'mask' bit */
+            old_rte.mask = 0;
+            __ioapic_write_entry(apic, pin, true, old_rte);
Quoted 8 lines

xen/drivers/passthrough/vtd/intremap.c unsigned int cf_check io_apic_read_remap_rte( · line 467

Quoted 4 lines
xen/drivers/passthrough/vtd/intremap.c +4 -0
+        }
+        dprintk(XENLOG_ERR VTDPREFIX,
+                "failed to update IRTE for IO-APIC#%u pin %u: %d\n",
+                apic, pin, rc);

Afaics you don't alter the error behavior of ioapic_rte_to_remap_entry(), and in the sole (pre-existing) case of an error a debug log message is already being issued. Why the addition?

Quoted 7 lines

I think I was trying to mimic the behavior of amd_iommu_ioapic_update_ire(), which does print the errors as opposed to doing so in update_intremap_entry_from_ioapic().

I've now removed it, and adjusted the code to match the rest of your comments. Will post v4 of 4/4 only as a reply to v3, I don't think there's a need to resend the rest unless you prefer it that way.

Just this patch is going to be fine (maybe even as v3.1).

Jan

#10 lore raw

PATCH v4 4/4 x86/iommu: pass full IO-APIC RTE for remapping table update

In reply to Roger Pau Monne

So that the remapping entry can be updated atomically when possible.

Doing such update atomically will avoid Xen having to mask the IO-APIC pin prior to performing any interrupt movements (ie: changing the destination and vector fields), as the interrupt remapping entry is always consistent.

This also simplifies some of the logic on both VT-d and AMD-Vi implementations, as having the full RTE available instead of half of it avoids to possibly read and update the missing other half from hardware.

While there remove the explicit zeroing of new_ire fields in ioapic_rte_to_remap_entry() and initialize the variable at definition so all fields are zeroed. Note fields could be also initialized with final values at definition, but I found that likely too much to be done at this time.

---
Changes since v3:
- Fix prototype parameter name to match definition.
- Use GET_xAPIC_ID().
- Remove duplicated error message in io_apic_write_remap_rte().

Changes since v2:
- Remove unneeded initialization.
- Use 'rte' as parameter name for update_ire_from_apic() implementations.
- Fix comment style in ioapic_rte_to_remap_entry().
- Fix requirement for atomic write in update_irte() call from ioapic_rte_to_remap_entry().
- Remove remap_rte from io_apic_write_remap_rte().
---
Note that certain combination of changes to the RTE are impossible to handle atomically. For example changing the vector and/or destination fields together with the triggering mode is impossible to be performed atomically (as the destination and vector is set in the IRTE, but the triggering mode is set in the RTE). Xen doesn't attempt to perform such changes in a single update to the RTE anyway, so it's fine.

Naming the iommu_update_ire_from_apic() parameter RTE is not really correct, as the format of the passed value expands the destination field to be 32bits (in order to fit an x2APIC ID). Passing an IO_APIC_route_entry struct is not possible due to the circular dependency that would create between io_apic.h and iommu.h. It might be possible to move IO_APIC_route_entry declaration to a different header, but I haven't looked into it.

---
 xen/arch/x86/include/asm/iommu.h         |   3 +-
 xen/arch/x86/io_apic.c                   |   5 +-
 xen/drivers/passthrough/amd/iommu.h      |   2 +-
 xen/drivers/passthrough/amd/iommu_intr.c | 100 ++---------------
 xen/drivers/passthrough/vtd/extern.h     |   2 +-
 xen/drivers/passthrough/vtd/intremap.c   | 131 +++++++++++------------
 xen/drivers/passthrough/x86/iommu.c      |   4 +-
 xen/include/xen/iommu.h                  |   3 +-
 8 files changed, 82 insertions(+), 168 deletions(-)
xen/arch/x86/include/asm/iommu.h +2 -1
index 0540cd9faa87..eb720205e25e 100644
@@ -84,7 +84,8 @@ struct iommu_init_ops {
 
 extern const struct iommu_init_ops *iommu_init_ops;
 
-void iommu_update_ire_from_apic(unsigned int apic, unsigned int reg, unsigned int value);
+void iommu_update_ire_from_apic(unsigned int apic, unsigned int pin,
+                                uint64_t rte);
 unsigned int iommu_read_apic_from_ire(unsigned int apic, unsigned int reg);
 int iommu_setup_hpet_msi(struct msi_desc *);
 
xen/arch/x86/io_apic.c +1 -4
index 041233b9b706..b3afef8933d7 100644
@@ -275,10 +275,7 @@ void __ioapic_write_entry(
         __io_apic_write(apic, 0x10 + 2 * pin, eu.w1);
     }
     else
-    {
-        iommu_update_ire_from_apic(apic, 0x11 + 2 * pin, eu.w2);
-        iommu_update_ire_from_apic(apic, 0x10 + 2 * pin, eu.w1);
-    }
+        iommu_update_ire_from_apic(apic, pin, e.raw);
 }
 
 static void ioapic_write_entry(
xen/drivers/passthrough/amd/iommu.h +1 -1
index 8bc3c35b1bb1..5429ada58ef5 100644
@@ -300,7 +300,7 @@ int cf_check amd_iommu_free_intremap_table(
 unsigned int amd_iommu_intremap_table_order(
     const void *irt, const struct amd_iommu *iommu);
 void cf_check amd_iommu_ioapic_update_ire(
-    unsigned int apic, unsigned int reg, unsigned int value);
+    unsigned int apic, unsigned int pin, uint64_t rte);
 unsigned int cf_check amd_iommu_read_ioapic_from_ire(
     unsigned int apic, unsigned int reg);
 int cf_check amd_iommu_msi_msg_update_ire(
xen/drivers/passthrough/amd/iommu_intr.c +9 -91

150 lines not on this page · raw message

xen/drivers/passthrough/vtd/extern.h +1 -1
index 39602d1f88f8..d49e40c5ce7d 100644
@@ -92,7 +92,7 @@ int cf_check intel_iommu_get_reserved_device_memory(
 unsigned int cf_check io_apic_read_remap_rte(
     unsigned int apic, unsigned int reg);
 void cf_check io_apic_write_remap_rte(
-    unsigned int apic, unsigned int reg, unsigned int value);
+    unsigned int apic, unsigned int pin, uint64_t rte);
 
 struct msi_desc;
 struct msi_msg;
xen/drivers/passthrough/vtd/intremap.c +64 -67

163 lines not on this page · raw message

xen/drivers/passthrough/x86/iommu.c +2 -2
index be71a4c4641c..d290855959f2 100644
@@ -158,9 +158,9 @@ int iommu_enable_x2apic(void)
 }
 
 void iommu_update_ire_from_apic(
-    unsigned int apic, unsigned int reg, unsigned int value)
+    unsigned int apic, unsigned int pin, uint64_t rte)
 {
-    iommu_vcall(&iommu_ops, update_ire_from_apic, apic, reg, value);
+    iommu_vcall(&iommu_ops, update_ire_from_apic, apic, pin, rte);
 }
 
 unsigned int iommu_read_apic_from_ire(unsigned int apic, unsigned int reg)
xen/include/xen/iommu.h +2 -1
index 405db59971c5..9335cd074cff 100644
@@ -278,7 +278,8 @@ struct iommu_ops {
     int (*enable_x2apic)(void);
     void (*disable_x2apic)(void);
 
-    void (*update_ire_from_apic)(unsigned int apic, unsigned int reg, unsigned int value);
+    void (*update_ire_from_apic)(unsigned int apic, unsigned int pin,
+                                 uint64_t rte);
     unsigned int (*read_apic_from_ire)(unsigned int apic, unsigned int reg);
 
     int (*setup_hpet_msi)(struct msi_desc *);
Signature · 1 line
2.41.0
Reviewed-by on v4 4/4
#11 lore raw

Re: [PATCH v4 4/4] x86/iommu: pass full IO-APIC RTE for remapping table update

Quoted 12 lines · Roger Pau Monne

So that the remapping entry can be updated atomically when possible.

Doing such update atomically will avoid Xen having to mask the IO-APIC pin prior to performing any interrupt movements (ie: changing the destination and vector fields), as the interrupt remapping entry is always consistent.

This also simplifies some of the logic on both VT-d and AMD-Vi implementations, as having the full RTE available instead of half of it avoids to possibly read and update the missing other half from hardware.

Quoted 7 lines

While there remove the explicit zeroing of new_ire fields in ioapic_rte_to_remap_entry() and initialize the variable at definition so all fields are zeroed. Note fields could be also initialized with final values at definition, but I found that likely too much to be done at this time.

Reviewed-by on v4 4/4
#12 lore raw

RE: [PATCH v4 4/4] x86/iommu: pass full IO-APIC RTE for remapping table update

In reply to Roger Pau Monne
Quoted 15 lines

From: Roger Pau Monne <[email protected]>
Sent: Friday, July 28, 2023 5:57 PM

So that the remapping entry can be updated atomically when possible.

Doing such update atomically will avoid Xen having to mask the IO-APIC pin prior to performing any interrupt movements (ie: changing the destination and vector fields), as the interrupt remapping entry is always consistent.

This also simplifies some of the logic on both VT-d and AMD-Vi implementations, as having the full RTE available instead of half of it avoids to possibly read and update the missing other half from hardware.

Quoted 7 lines

While there remove the explicit zeroing of new_ire fields in ioapic_rte_to_remap_entry() and initialize the variable at definition so all fields are zeroed. Note fields could be also initialized with final values at definition, but I found that likely too much to be done at this time.