x86/ioapic: fix edge triggered interrupt migration
v2 cover x86/ioapic: fix edge triggered interrupt migration
Hello,
Following series attempts to solve the issue with IO-APIC edge triggered interrupts seeing an inconsistent IRTE when injected while being migrated.
It's been simplified a bit from the original RFC, and does make the logic in the IOMMU RTE update handlers cleaner, as those get passed the full RTE.
When not using interrupt remapping the logic is left as-is. One further change would be to mask edge triggered interrupts, as destination and vector cannot be changed atomically when dealing with plain IO-APIC RTEs.
Last patch has also been adjusted in order to always to atomic IRTE updates on AMD.
Thanks, Roger.
Roger Pau Monne (5):
x86/ioapic: add a raw field to RTE struct x86/ioapic: RTE modifications must use ioapic_write_entry iommu/vtd: rename io_apic_read_remap_rte() local variable x86/iommu: pass full IO-APIC RTE for remapping table update amd/iommu: force atomic updates of remapping entries
xen/arch/x86/include/asm/io_apic.h | 65 +++++----- xen/arch/x86/include/asm/iommu.h | 3 +- xen/arch/x86/io_apic.c | 40 +++--- xen/drivers/passthrough/amd/iommu.h | 2 +- xen/drivers/passthrough/amd/iommu_init.c | 10 ++ xen/drivers/passthrough/amd/iommu_intr.c | 155 +++-------------------- xen/drivers/passthrough/vtd/extern.h | 2 +- xen/drivers/passthrough/vtd/intremap.c | 131 ++++++++++--------- xen/drivers/passthrough/x86/iommu.c | 4 +- xen/include/xen/iommu.h | 3 +- 10 files changed, 156 insertions(+), 259 deletions(-)
Signature · 1 line
2.41.0
PATCH v2 1/5 x86/ioapic: add a raw field to RTE struct
Further changes will require access to the full RTE as a single value in order to pass it to IOMMU interrupt remapping handlers.
No functional change intended.
---
Changes since v1:
- Fix initializers.
---
Tested on gitlab CI, builds on all compilers used there.
--- xen/arch/x86/include/asm/io_apic.h | 57 +++++++++++++----------- xen/arch/x86/io_apic.c | 2 +- xen/drivers/passthrough/amd/iommu_intr.c | 4 +- xen/drivers/passthrough/vtd/intremap.c | 4 +- 4 files changed, 35 insertions(+), 32 deletions(-)
xen/arch/x86/include/asm/io_apic.h +30 -27
@@ -90,35 +90,38 @@ enum ioapic_irq_destination_types { };struct IO_APIC_route_entry { -unsigned int vector:8 ; -unsigned int delivery_mode:3 ;/* -* 000: FIXED -* 001: lowest prio -* 111: ExtINT -*/ -unsigned int dest_mode:1 ;/* 0: physical, 1: logical */ -unsigned int delivery_status:1 ; -unsigned int polarity:1 ;/* 0: low, 1: high */ -unsigned int irr:1 ; -unsigned int trigger:1 ;/* 0: edge, 1: level */ -unsigned int mask:1 ;/* 0: enabled, 1: disabled */ -unsigned int __reserved_2:15 ; -union {struct { -unsigned int __reserved_1:24 ; -unsigned int physical_dest:4 ; -unsigned int __reserved_2:4 ; - } physical; - -struct { -unsigned int __reserved_1:24 ; -unsigned int logical_dest:8 ; - } logical; - -/* used when Interrupt Remapping with EIM is enabled */ -unsigned int dest32; - } dest; +unsigned int vector:8 ; +unsigned int delivery_mode:3 ;/* +* 000: FIXED +* 001: lowest prio +* 111: ExtINT +*/ +unsigned int dest_mode:1 ;/* 0: physical, 1: logical */ +unsigned int delivery_status:1 ; +unsigned int polarity:1 ;/* 0: low, 1: high */ +unsigned int irr:1 ; +unsigned int trigger:1 ;/* 0: edge, 1: level */ +unsigned int mask:1 ;/* 0: enabled, 1: disabled */ +unsigned int __reserved_2:15 ; + +union { +struct { +unsigned int __reserved_1:24 ; +unsigned int physical_dest:4 ; +unsigned int __reserved_2:4 ; + } physical; + +struct { +unsigned int __reserved_1:24 ; +unsigned int logical_dest:8 ; + } logical; +unsigned int dest32; + } dest; + }; +uint64_t raw; + }; };/*
xen/arch/x86/io_apic.c +1 -1
@@ -2360,7 +2360,7 @@ int ioapic_guest_read(unsigned long physbase, unsigned int reg, u32 *pval)int ioapic_guest_write (unsigned long physbase,unsigned int reg,u32 val) {int apic, pin, irq, ret, pirq; -struct IO_APIC_route_entry rte = {0 }; +struct IO_APIC_route_entry rte = { };unsigned long flags;struct irq_desc *desc;
xen/drivers/passthrough/amd/iommu_intr.c +2 -2
@@ -352,8 +352,8 @@ static int update_intremap_entry_from_ioapic( void cf_check amd_iommu_ioapic_update_ire(unsigned int apic,unsigned int reg,unsigned int value) { -struct IO_APIC_route_entry old_rte = {0 }; -struct IO_APIC_route_entry new_rte = {0 }; +struct IO_APIC_route_entry old_rte = { }; +struct IO_APIC_route_entry new_rte = { };unsigned int rte_lo = (reg &1 ) ? reg -1 : reg;unsigned int pin = (reg -0x10 ) /2 ;int seg, bdf, rc;
xen/drivers/passthrough/vtd/intremap.c +2 -2
@@ -419,7 +419,7 @@ unsigned int cf_check io_apic_read_remap_rte( {unsigned int ioapic_pin = (reg -0x10 ) /2 ;int index; -struct IO_xAPIC_route_entry old_rte = {0 }; +struct IO_xAPIC_route_entry old_rte = { };int rte_upper = (reg &1 ) ?1 :0 ;struct vtd_iommu *iommu =ioapic_to_iommu (IO_APIC_ID (apic));
@@ -442,7 +442,7 @@ void cf_check io_apic_write_remap_rte( unsigned int apic, unsigned int reg, unsigned int value) { unsigned int ioapic_pin = (reg - 0x10) / 2; -struct IO_xAPIC_route_entry old_rte = {0 }; +struct IO_xAPIC_route_entry old_rte = { };struct IO_APIC_route_remap_entry *remap_rte;unsigned int rte_upper = (reg &1 ) ?1 :0 ;struct vtd_iommu *iommu =ioapic_to_iommu (IO_APIC_ID (apic));
Signature · 1 line
2.41.0
Re: [PATCH v2 1/5] x86/ioapic: add a raw field to RTE struct
Quoted 6 lines · Roger Pau Monne
Further changes will require access to the full RTE as a single value in order to pass it to IOMMU interrupt remapping handlers.
No functional change intended.
Signed-off-by Roger Pau Monné <[email protected]>
As indicated for v1, I'm fine in principle, so
but I don't see this (or really any of the prereq changes) go in ahead of the actual meat of this series (patch 4 and 5, aiui).
Jan
PATCH v2 2/5 x86/ioapic: RTE modifications must use ioapic_write_entry
In reply to Roger Pau MonneDo not allow to write to RTE registers using io_apic_write and instead require changes to RTE to be performed using ioapic_write_entry.
This is in preparation for passing the full contents of the RTE to the IOMMU interrupt remapping handlers, so remapping entries for IO-APIC RTEs can be updated atomically when possible.
While immediately this commit might expand the number of MMIO accesses in order to update an IO-APIC RTE, further changes will benefit from getting the full RTE value passed to the IOMMU handlers, as the logic is greatly simplified when the IOMMU handlers can get the complete RTE value in one go.
---
Changes since v1:
- Reinstate io_apic_modify().
- Expand commit message.
--- xen/arch/x86/include/asm/io_apic.h | 8 ++--- xen/arch/x86/io_apic.c | 37 ++++++++++++------------ xen/drivers/passthrough/amd/iommu_intr.c | 6 ---- 3 files changed, 22 insertions(+), 29 deletions(-)
xen/arch/x86/include/asm/io_apic.h +4 -4
@@ -162,8 +162,8 @@ static inline void __io_apic_write(unsigned int apic, unsigned int reg, unsignedstatic inline void io_apic_write (unsigned int apic,unsigned int reg,unsigned int value) { -if (ioapic_reg_remapped (reg) ) -return iommu_update_ire_from_apic (apic, reg, value); +/* RTE writes must use ioapic_write_entry. */ +BUG_ON (reg >=0x10 );__io_apic_write (apic, reg, value); }
@@ -173,8 +173,8 @@ static inline void io_apic_write(unsigned int apic, unsigned int reg, unsigned i*/ static inline void io_apic_modify (unsigned int apic,unsigned int reg,unsigned int value) { -if (ioapic_reg_remapped (reg) ) -return iommu_update_ire_from_apic (apic, reg, value); +/* RTE writes must use ioapic_write_entry. */ +BUG_ON (reg >=0x10 ); *(IO_APIC_BASE (apic) +4 ) = value; }
xen/arch/x86/io_apic.c +18 -19
@@ -269,15 +269,15 @@ void __ioapic_write_entry( {union entry_union eu = { .entry = e }; -if ( raw ) +if ( raw || !iommu_intremap ) {__io_apic_write (apic,0x11 +2 * pin, eu.w2);__io_apic_write (apic,0x10 +2 * pin, eu.w1); }else { -io_apic_write (apic,0x11 +2 * pin, eu.w2); -io_apic_write (apic,0x10 +2 * pin, eu.w1); +iommu_update_ire_from_apic (apic,0x11 +2 * pin, eu.w2); +iommu_update_ire_from_apic (apic,0x10 +2 * pin, eu.w1); } }
@@ -433,16 +433,17 @@ static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, unsigned int disable) {struct irq_pin_list *entry = irq_2_pin + irq; -unsigned int pin, reg;for (;;) { - pin = entry->pin; +unsigned int pin = entry->pin; +struct IO_APIC_route_entry rte; +if (pin ==-1 )break ; - reg =io_apic_read (entry->apic,0x10 + pin*2 ); - reg &= ~disable; - reg |= enable; -io_apic_modify (entry->apic,0x10 + pin*2 , reg); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.raw &= ~(uint64_t )disable; + rte.raw |= enable; +__ioapic_write_entry (entry->apic, pin, false, rte);if (!entry->next)break ; entry = irq_2_pin + entry->next;
@@ -584,16 +585,16 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest =SET_APIC_LOGICAL_ID (dest); entry = irq_2_pin + irq;for (;;) { -unsigned int data; +struct IO_APIC_route_entry rte; + pin = entry->pin;if (pin ==-1 )break ; -io_apic_write (entry->apic,0x10 +1 + pin*2 , dest); - data =io_apic_read (entry->apic,0x10 + pin*2 ); - data &= ~IO_APIC_REDIR_VECTOR_MASK ; - data |=MASK_INSR (desc->arch.vector,IO_APIC_REDIR_VECTOR_MASK ); -io_apic_modify (entry->apic,0x10 + pin*2 , data); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.dest.dest32 = dest; + rte.vector = desc->arch.vector; +__ioapic_write_entry (entry->apic, pin, false, rte);if (!entry->next)break ;
@@ -2127,10 +2128,8 @@ void ioapic_resume(void) reg_00.bits.ID = mp_ioapics[apic].mpc_apicid;__io_apic_write (apic,0 , reg_00.raw); } -for (i =0 ; i < nr_ioapic_entries[apic]; i++, entry++) { -__io_apic_write (apic,0x11 +2 *i, *(((int *)entry)+1 )); -__io_apic_write (apic,0x10 +2 *i, *(((int *)entry)+0 )); - } +for (i =0 ; i < nr_ioapic_entries[apic]; i++, entry++) +__ioapic_write_entry (apic, i, true, *entry); }spin_unlock_irqrestore (&ioapic_lock, flags); }
xen/drivers/passthrough/amd/iommu_intr.c +0 -6
@@ -361,12 +361,6 @@ void cf_check amd_iommu_ioapic_update_ire(struct amd_iommu *iommu;unsigned int idx; -if ( !iommu_intremap ) - { -__io_apic_write (apic, reg, value); -return ; - } - idx =ioapic_id_to_index (IO_APIC_ID (apic));if ( idx ==MAX_IO_APICS )return ;
Signature · 1 line
2.41.0
Re: [PATCH v2 2/5] x86/ioapic: RTE modifications must use ioapic_write_entry
Quoted 10 lines · Roger Pau Monne
xen/arch/x86/io_apic.c +1
-1@@ -269,15 +269,15 @@ void __ioapic_write_entry( {union entry_union eu = { .entry = e }; -if ( raw ) +if ( raw || !iommu_intremap ) {__io_apic_write (apic,0x11 +2 * pin, eu.w2);
xen/arch/x86/io_apic.c void __ioapic_write_entry( · line 282
Quoted 10 lines
xen/arch/x86/io_apic.c +2
-2__io_apic_write (apic,0x10 +2 * pin, eu.w1); }else { -io_apic_write (apic,0x11 +2 * pin, eu.w2); -io_apic_write (apic,0x10 +2 * pin, eu.w1); +iommu_update_ire_from_apic (apic,0x11 +2 * pin, eu.w2); +iommu_update_ire_from_apic (apic,0x10 +2 * pin, eu.w1); } }
I think __ioapic_read_entry() wants updating similarly, so that both remain consistent.
Quoted 11 lines
xen/arch/x86/io_apic.c +3
-2@@ -433,16 +433,17 @@ static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, unsigned int disable) {struct irq_pin_list *entry = irq_2_pin + irq; -unsigned int pin, reg;for (;;) { - pin = entry->pin; +unsigned int pin = entry->pin; +struct IO_APIC_route_entry rte; +
xen/arch/x86/io_apic.c static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, · line 446
Quoted 10 lines
xen/arch/x86/io_apic.c +4
-4if (pin ==-1 )break ; - reg =io_apic_read (entry->apic,0x10 + pin*2 ); - reg &= ~disable; - reg |= enable; -io_apic_modify (entry->apic,0x10 + pin*2 , reg); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.raw &= ~(uint64_t )disable; + rte.raw |= enable; +__ioapic_write_entry (entry->apic, pin, false, rte);
While this isn't going to happen overly often, ...
Quoted 10 lines
xen/arch/x86/io_apic.c +2
-1@@ -584,16 +585,16 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest =SET_APIC_LOGICAL_ID (dest); entry = irq_2_pin + irq;for (;;) { -unsigned int data; +struct IO_APIC_route_entry rte; + pin = entry->pin;if (pin ==-1 )break ;
xen/arch/x86/io_apic.c set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) · line 597
Quoted 10 lines
xen/arch/x86/io_apic.c +4
-5-io_apic_write (entry->apic,0x10 +1 + pin*2 , dest); - data =io_apic_read (entry->apic,0x10 + pin*2 ); - data &= ~IO_APIC_REDIR_VECTOR_MASK ; - data |=MASK_INSR (desc->arch.vector,IO_APIC_REDIR_VECTOR_MASK ); -io_apic_modify (entry->apic,0x10 + pin*2 , data); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.dest.dest32 = dest; + rte.vector = desc->arch.vector; +__ioapic_write_entry (entry->apic, pin, false, rte);
... this makes me wonder whether there shouldn't be an __ioapic_modify_entry() capable of suppressing one of the two writes (but still being handed the full RTE).
Jan
Re: [PATCH v2 2/5] x86/ioapic: RTE modifications must use ioapic_write_entry
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monne
xen/arch/x86/io_apic.c +1
-2@@ -269,15 +269,15 @@ void __ioapic_write_entry( {union entry_union eu = { .entry = e }; -if ( raw ) +if ( raw || !iommu_intremap ) {__io_apic_write (apic,0x11 +2 * pin, eu.w2);__io_apic_write (apic,0x10 +2 * pin, eu.w1); }else { -io_apic_write (apic,0x11 +2 * pin, eu.w2);
Quoted 8 lines
xen/arch/x86/io_apic.c void __ioapic_write_entry( · line 282
Quoted 5 lines
xen/arch/x86/io_apic.c +2
-1-io_apic_write (apic,0x10 +2 * pin, eu.w1); +iommu_update_ire_from_apic (apic,0x11 +2 * pin, eu.w2); +iommu_update_ire_from_apic (apic,0x10 +2 * pin, eu.w1); } }I think
__ioapic_read_entry()wants updating similarly, so that both remain consistent.
My plan was to deal with __ioapic_read_entry() separately, as I would also like to convert iommu_read_apic_from_ire() to get passed a pin instead of a register, but I could make your requested adjustment here for consistency with __ioapic_write_entry().
Quoted 35 lines
Quoted 11 lines
xen/arch/x86/io_apic.c +3
-2@@ -433,16 +433,17 @@ static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, unsigned int disable) {struct irq_pin_list *entry = irq_2_pin + irq; -unsigned int pin, reg;for (;;) { - pin = entry->pin; +unsigned int pin = entry->pin; +struct IO_APIC_route_entry rte; +xen/arch/x86/io_apic.c static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, · line 446
Quoted 10 lines
xen/arch/x86/io_apic.c +4
-4if (pin ==-1 )break ; - reg =io_apic_read (entry->apic,0x10 + pin*2 ); - reg &= ~disable; - reg |= enable; -io_apic_modify (entry->apic,0x10 + pin*2 , reg); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.raw &= ~(uint64_t )disable; + rte.raw |= enable; +__ioapic_write_entry (entry->apic, pin, false, rte);While this isn't going to happen overly often, ...
Quoted 11 lines
xen/arch/x86/io_apic.c +2
-1@@ -584,16 +585,16 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest =SET_APIC_LOGICAL_ID (dest); entry = irq_2_pin + irq;for (;;) { -unsigned int data; +struct IO_APIC_route_entry rte; + pin = entry->pin;if (pin ==-1 )break ;
Quoted 13 lines
xen/arch/x86/io_apic.c set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) · line 597
Quoted 9 lines
xen/arch/x86/io_apic.c +4
-5-io_apic_write (entry->apic,0x10 +1 + pin*2 , dest); - data =io_apic_read (entry->apic,0x10 + pin*2 ); - data &= ~IO_APIC_REDIR_VECTOR_MASK ; - data |=MASK_INSR (desc->arch.vector,IO_APIC_REDIR_VECTOR_MASK ); -io_apic_modify (entry->apic,0x10 + pin*2 , data); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.dest.dest32 = dest; + rte.vector = desc->arch.vector; +__ioapic_write_entry (entry->apic, pin, false, rte);... this makes me wonder whether there shouldn't be an
__ioapic_modify_entry()capable of suppressing one of the two writes (but still being handed the full RTE).
We would then need the original RTE to be passed to __ioapic_modify_entry() in order for it to decide whether one of the accesses can be eliminated (as I don't think we want to read the RTE to check for differences, as we would then perform even more accesses).
This would only be relevant for systems without IOMMU IRTEs, as otherwise the IO-APIC RTE gets modified by the IOMMU handlers.
Thanks, Roger.
Re: [PATCH v2 2/5] x86/ioapic: RTE modifications must use ioapic_write_entry
Quoted 26 lines · Roger Pau Monné
Quoted 16 lines · Jan Beulich
Quoted 15 lines · Roger Pau Monne
xen/arch/x86/io_apic.c +1
-2@@ -269,15 +269,15 @@ void __ioapic_write_entry( {union entry_union eu = { .entry = e }; -if ( raw ) +if ( raw || !iommu_intremap ) {__io_apic_write (apic,0x11 +2 * pin, eu.w2);__io_apic_write (apic,0x10 +2 * pin, eu.w1); }else { -io_apic_write (apic,0x11 +2 * pin, eu.w2);Quoted 8 lines
xen/arch/x86/io_apic.c void __ioapic_write_entry( · line 282
Quoted 5 lines
xen/arch/x86/io_apic.c +2
-1-io_apic_write (apic,0x10 +2 * pin, eu.w1); +iommu_update_ire_from_apic (apic,0x11 +2 * pin, eu.w2); +iommu_update_ire_from_apic (apic,0x10 +2 * pin, eu.w1); } }I think
__ioapic_read_entry()wants updating similarly, so that both remain consistent.
Quoted 4 lines
My plan was to deal with
__ioapic_read_entry()separately, as I would also like to convertiommu_read_apic_from_ire()to get passed a pin instead of a register, but I could make your requested adjustment here for consistency with__ioapic_write_entry().
I would indeed prefer if you did, to avoid the functions going out of sync.
Quoted 49 lines
Quoted 35 lines
Quoted 11 lines
xen/arch/x86/io_apic.c +3
-2@@ -433,16 +433,17 @@ static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, unsigned int disable) {struct irq_pin_list *entry = irq_2_pin + irq; -unsigned int pin, reg;for (;;) { - pin = entry->pin; +unsigned int pin = entry->pin; +struct IO_APIC_route_entry rte; +xen/arch/x86/io_apic.c static void modify_IO_APIC_irq(unsigned int irq, unsigned int enable, · line 446
Quoted 10 lines
xen/arch/x86/io_apic.c +4
-4if (pin ==-1 )break ; - reg =io_apic_read (entry->apic,0x10 + pin*2 ); - reg &= ~disable; - reg |= enable; -io_apic_modify (entry->apic,0x10 + pin*2 , reg); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.raw &= ~(uint64_t )disable; + rte.raw |= enable; +__ioapic_write_entry (entry->apic, pin, false, rte);While this isn't going to happen overly often, ...
Quoted 11 lines
xen/arch/x86/io_apic.c +2
-1@@ -584,16 +585,16 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest =SET_APIC_LOGICAL_ID (dest); entry = irq_2_pin + irq;for (;;) { -unsigned int data; +struct IO_APIC_route_entry rte; + pin = entry->pin;if (pin ==-1 )break ;Quoted 13 lines
xen/arch/x86/io_apic.c set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) · line 597
Quoted 9 lines
xen/arch/x86/io_apic.c +4
-5-io_apic_write (entry->apic,0x10 +1 + pin*2 , dest); - data =io_apic_read (entry->apic,0x10 + pin*2 ); - data &= ~IO_APIC_REDIR_VECTOR_MASK ; - data |=MASK_INSR (desc->arch.vector,IO_APIC_REDIR_VECTOR_MASK ); -io_apic_modify (entry->apic,0x10 + pin*2 , data); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.dest.dest32 = dest; + rte.vector = desc->arch.vector; +__ioapic_write_entry (entry->apic, pin, false, rte);... this makes me wonder whether there shouldn't be an
__ioapic_modify_entry()capable of suppressing one of the two writes (but still being handed the full RTE).
Quoted 5 lines
We would then need the original RTE to be passed to
__ioapic_modify_entry()in order for it to decide whether one of the accesses can be eliminated (as I don't think we want to read the RTE to check for differences, as we would then perform even more accesses).
I was actually thinking of a 2-bit hint that the caller would pass:
Low half unmodified and high half unmodified.
Quoted 2 lines
This would only be relevant for systems without IOMMU IRTEs, as otherwise the IO-APIC RTE gets modified by the IOMMU handlers.
Initially yes. I think there's room there to avoid one half of the write as well.
Jan
Re: [PATCH v2 2/5] x86/ioapic: RTE modifications must use ioapic_write_entry
In reply to Jan BeulichQuoted 14 lines · Jan Beulich
Quoted 13 lines · Roger Pau Monne
xen/arch/x86/io_apic.c +2
-1@@ -584,16 +585,16 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest =SET_APIC_LOGICAL_ID (dest); entry = irq_2_pin + irq;for (;;) { -unsigned int data; +struct IO_APIC_route_entry rte; + pin = entry->pin;if (pin ==-1 )break ;
Quoted 13 lines
xen/arch/x86/io_apic.c set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) · line 597
Quoted 9 lines
xen/arch/x86/io_apic.c +4
-5-io_apic_write (entry->apic,0x10 +1 + pin*2 , dest); - data =io_apic_read (entry->apic,0x10 + pin*2 ); - data &= ~IO_APIC_REDIR_VECTOR_MASK ; - data |=MASK_INSR (desc->arch.vector,IO_APIC_REDIR_VECTOR_MASK ); -io_apic_modify (entry->apic,0x10 + pin*2 , data); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.dest.dest32 = dest; + rte.vector = desc->arch.vector; +__ioapic_write_entry (entry->apic, pin, false, rte);... this makes me wonder whether there shouldn't be an
__ioapic_modify_entry()capable of suppressing one of the two writes (but still being handed the full RTE).
I've wondered about this, and I'm not sure how often can one of the two writes be suppressed here, as we modify both the low (for the vector) and the high part of the RTE (for the destination). It's unlikely that the same vector could be used on both destinations, and IMO such case doesn't warrant the introduction of the extra logic required in order to suppress one of the writes.
Am I overlooking something?
Thanks, Roger.
Re: [PATCH v2 2/5] x86/ioapic: RTE modifications must use ioapic_write_entry
Quoted 29 lines · Roger Pau Monné
Quoted 14 lines · Jan Beulich
Quoted 13 lines · Roger Pau Monne
xen/arch/x86/io_apic.c +2
-1@@ -584,16 +585,16 @@ set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) dest =SET_APIC_LOGICAL_ID (dest); entry = irq_2_pin + irq;for (;;) { -unsigned int data; +struct IO_APIC_route_entry rte; + pin = entry->pin;if (pin ==-1 )break ;Quoted 13 lines
xen/arch/x86/io_apic.c set_ioapic_affinity_irq(struct irq_desc *desc, const cpumask_t *mask) · line 597
Quoted 9 lines
xen/arch/x86/io_apic.c +4
-5-io_apic_write (entry->apic,0x10 +1 + pin*2 , dest); - data =io_apic_read (entry->apic,0x10 + pin*2 ); - data &= ~IO_APIC_REDIR_VECTOR_MASK ; - data |=MASK_INSR (desc->arch.vector,IO_APIC_REDIR_VECTOR_MASK ); -io_apic_modify (entry->apic,0x10 + pin*2 , data); + rte =__ioapic_read_entry (entry->apic, pin, false); + rte.dest.dest32 = dest; + rte.vector = desc->arch.vector; +__ioapic_write_entry (entry->apic, pin, false, rte);... this makes me wonder whether there shouldn't be an
__ioapic_modify_entry()capable of suppressing one of the two writes (but still being handed the full RTE).
Quoted 8 lines
I've wondered about this, and I'm not sure how often can one of the two writes be suppressed here, as we modify both the low (for the vector) and the high part of the RTE (for the destination). It's unlikely that the same vector could be used on both destinations, and IMO such case doesn't warrant the introduction of the extra logic required in order to suppress one of the writes.
Am I overlooking something?
Oh, no, that was me seeing the io_apic_modify() without paying attention to the earlier io_apic_write() (both in the code you replace).
Jan
PATCH v2 3/5 iommu/vtd: rename io_apic_read_remap_rte() local variable
In reply to Roger Pau MonnePreparatory change to unify the IO-APIC pin variable name between io_apic_read_remap_rte() and amd_iommu_ioapic_update_ire(), so that the local variable can be made a function parameter with the same name across vendors.
---
Changes since v1:
- New in this version.
--- xen/drivers/passthrough/vtd/intremap.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
xen/drivers/passthrough/vtd/intremap.c +4 -4
@@ -441,14 +441,14 @@ unsigned int cf_check io_apic_read_remap_rte( void cf_check io_apic_write_remap_rte(unsigned int apic,unsigned int reg,unsigned int value) { -unsigned int ioapic_pin = (reg -0x10 ) /2 ; +unsigned int pin = (reg -0x10 ) /2 ;struct IO_xAPIC_route_entry old_rte = { };struct IO_APIC_route_remap_entry *remap_rte;unsigned int rte_upper = (reg &1 ) ?1 :0 ;struct vtd_iommu *iommu =ioapic_to_iommu (IO_APIC_ID (apic));int saved_mask; - old_rte =__ioapic_read_entry (apic, ioapic_pin, true); + old_rte =__ioapic_read_entry (apic, pin, true); remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte;
@@ -458,7 +458,7 @@ void cf_check io_apic_write_remap_rte(__io_apic_write (apic, reg & ~1 , *(u32 *)&old_rte); remap_rte->mask = saved_mask; -if (ioapic_rte_to_remap_entry (iommu, apic, ioapic_pin, +if (ioapic_rte_to_remap_entry (iommu, apic, pin, &old_rte, rte_upper, value) ) {__io_apic_write (apic, reg, value);
@@ -468,7 +468,7 @@ void cf_check io_apic_write_remap_rte(__io_apic_write (apic, reg & ~1 , *(u32 *)&old_rte); }else -__ioapic_write_entry (apic, ioapic_pin, true, old_rte); +__ioapic_write_entry (apic, pin, true, old_rte); }static void set_msi_source_id (struct pci_dev *pdev,struct iremap_entry *ire)
Signature · 1 line
2.41.0
PATCH v2 4/5 x86/iommu: pass full IO-APIC RTE for remapping table update
In reply to Roger Pau MonneSo that the remapping entry can be updated atomically when possible.
Doing such update atomically will avoid Xen having to mask the IO-APIC pin prior to performing any interrupt movements (ie: changing the destination and vector fields), as the interrupt remapping entry is always consistent.
This also simplifies some of the logic on both VT-d and AMD-Vi implementations, as having the full RTE available instead of half of it avoids to possibly read and update the missing other half from hardware.
While there remove the explicit zeroing of new_ire fields in ioapic_rte_to_remap_entry() and initialize the variable at definition so all fields are zeroed. Note fields could be also initialized with final values at definition, but I found that likely too much to be done at this time.
---
Note that certain combination of changes to the RTE are impossible to handle atomically. For example changing the vector and/or destination fields together with the triggering mode is impossible to be performed atomically (as the destination and vector is set in the IRTE, but the triggering mode is set in the RTE). Xen doesn't attempt to perform such changes in a single update to the RTE anyway, so it's fine.
Naming the iommu_update_ire_from_apic() parameter RTE is not really correct, as the format of the passed value expands the destination field to be 32bits (in order to fit an x2APIC ID). Passing an IO_APIC_route_entry struct is not possible due to the circular dependency that would create between io_apic.h and iommu.h. It might be possible to move IO_APIC_route_entry declaration to a different header, but I haven't looked into it.
--- xen/arch/x86/include/asm/iommu.h | 3 +- xen/arch/x86/io_apic.c | 5 +- xen/drivers/passthrough/amd/iommu.h | 2 +- xen/drivers/passthrough/amd/iommu_intr.c | 98 ++--------------- xen/drivers/passthrough/vtd/extern.h | 2 +- xen/drivers/passthrough/vtd/intremap.c | 127 +++++++++++------------ xen/drivers/passthrough/x86/iommu.c | 4 +- xen/include/xen/iommu.h | 3 +- 8 files changed, 80 insertions(+), 164 deletions(-)
xen/arch/x86/include/asm/iommu.h +2 -1
@@ -84,7 +84,8 @@ struct iommu_init_ops {extern const struct iommu_init_ops *iommu_init_ops; -void iommu_update_ire_from_apic (unsigned int apic,unsigned int reg,unsigned int value); +void iommu_update_ire_from_apic (unsigned int apic,unsigned int pin, +uint64_t rte);unsigned int iommu_read_apic_from_ire (unsigned int apic,unsigned int reg);int iommu_setup_hpet_msi (struct msi_desc *);
xen/arch/x86/io_apic.c +1 -4
@@ -275,10 +275,7 @@ void __ioapic_write_entry(__io_apic_write (apic,0x10 +2 * pin, eu.w1); }else - { -iommu_update_ire_from_apic (apic,0x11 +2 * pin, eu.w2); -iommu_update_ire_from_apic (apic,0x10 +2 * pin, eu.w1); - } +iommu_update_ire_from_apic (apic, pin, e.raw); } static void ioapic_write_entry(
xen/drivers/passthrough/amd/iommu.h +1 -1
@@ -300,7 +300,7 @@ int cf_check amd_iommu_free_intremap_table(unsigned int amd_iommu_intremap_table_order (const void *irt,const struct amd_iommu *iommu); void cf_check amd_iommu_ioapic_update_ire( -unsigned int apic,unsigned int reg,unsigned int value); +unsigned int apic,unsigned int pin,uint64_t raw); unsigned int cf_check amd_iommu_read_ioapic_from_ire(unsigned int apic,unsigned int reg); int cf_check amd_iommu_msi_msg_update_ire(
xen/drivers/passthrough/amd/iommu_intr.c +8 -90
149 lines not on this page · raw message
xen/drivers/passthrough/vtd/extern.h +1 -1
@@ -92,7 +92,7 @@ int cf_check intel_iommu_get_reserved_device_memory( unsigned int cf_check io_apic_read_remap_rte(unsigned int apic,unsigned int reg); void cf_check io_apic_write_remap_rte( -unsigned int apic,unsigned int reg,unsigned int value); +unsigned int apic,unsigned int ioapic_pin,uint64_t raw);struct msi_desc ;struct msi_msg ;
xen/drivers/passthrough/vtd/intremap.c +63 -64
159 lines not on this page · raw message
xen/drivers/passthrough/x86/iommu.c +2 -2
@@ -158,9 +158,9 @@ int iommu_enable_x2apic(void) }void iommu_update_ire_from_apic ( -unsigned int apic,unsigned int reg,unsigned int value) +unsigned int apic,unsigned int pin,uint64_t rte) { -iommu_vcall (&iommu_ops, update_ire_from_apic, apic, reg, value); +iommu_vcall (&iommu_ops, update_ire_from_apic, apic, pin, rte); }unsigned int iommu_read_apic_from_ire (unsigned int apic,unsigned int reg)
xen/include/xen/iommu.h +2 -1
@@ -278,7 +278,8 @@ struct iommu_ops {int (*enable_x2apic)(void );void (*disable_x2apic)(void ); -void (*update_ire_from_apic)(unsigned int apic,unsigned int reg,unsigned int value); +void (*update_ire_from_apic)(unsigned int apic,unsigned int pin, +uint64_t rte);unsigned int (*read_apic_from_ire)(unsigned int apic,unsigned int reg);int (*setup_hpet_msi)(struct msi_desc *);
Signature · 1 line
2.41.0
Re: [PATCH v2 4/5] x86/iommu: pass full IO-APIC RTE for remapping table update
Quoted 34 lines · Roger Pau Monne
So that the remapping entry can be updated atomically when possible.
Doing such update atomically will avoid Xen having to mask the IO-APIC pin prior to performing any interrupt movements (ie: changing the destination and vector fields), as the interrupt remapping entry is always consistent.
This also simplifies some of the logic on both VT-d and AMD-Vi implementations, as having the full RTE available instead of half of it avoids to possibly read and update the missing other half from hardware.
While there remove the explicit zeroing of
new_irefields inioapic_rte_to_remap_entry()and initialize the variable at definition so all fields are zeroed. Note fields could be also initialized with final values at definition, but I found that likely too much to be done at this time.Signed-off-by Roger Pau Monné <[email protected]>---
Note that certain combination of changes to the RTE are impossible to handle atomically. For example changing the vector and/or destination fields together with the triggering mode is impossible to be performed atomically (as the destination and vector is set in the IRTE, but the triggering mode is set in the RTE). Xen doesn't attempt to perform such changes in a single update to the RTE anyway, so it's fine.Naming the
iommu_update_ire_from_apic()parameter RTE is not really correct, as the format of the passed value expands the destination field to be 32bits (in order to fit an x2APIC ID). Passing anIO_APIC_route_entrystruct is not possible due to the circular dependency that would create betweenio_apic.handiommu.h. It might be possible to moveIO_APIC_route_entrydeclaration to a different header, but I haven't looked into it.
Quoted 10 lines
--- xen/arch/x86/include/asm/iommu.h | 3 +- xen/arch/x86/io_apic.c | 5 +- xen/drivers/passthrough/amd/iommu.h | 2 +- xen/drivers/passthrough/amd/iommu_intr.c | 98 ++--------------- xen/drivers/passthrough/vtd/extern.h | 2 +- xen/drivers/passthrough/vtd/intremap.c | 127 +++++++++++------------ xen/drivers/passthrough/x86/iommu.c | 4 +- xen/include/xen/iommu.h | 3 +- 8 files changed, 80 insertions(+), 164 deletions(-)
Nice diffstat.
xen/arch/x86/include/asm/iommu.h struct iommu_init_ops { · line 88
Quoted 9 lines
xen/arch/x86/include/asm/iommu.h +2
-1@@ -84,7 +84,8 @@ struct iommu_init_ops {extern const struct iommu_init_ops *iommu_init_ops; -void iommu_update_ire_from_apic (unsigned int apic,unsigned int reg,unsigned int value); +void iommu_update_ire_from_apic (unsigned int apic,unsigned int pin, +uint64_t rte);
Much like you have it here, ...
xen/drivers/passthrough/amd/iommu.h int cf_check amd_iommu_free_intremap_table( · line 303
Quoted 8 lines
xen/drivers/passthrough/amd/iommu.h +1
-1@@ -300,7 +300,7 @@ int cf_check amd_iommu_free_intremap_table(unsigned int amd_iommu_intremap_table_order (const void *irt,const struct amd_iommu *iommu); void cf_check amd_iommu_ioapic_update_ire( -unsigned int apic,unsigned int reg,unsigned int value); +unsigned int apic,unsigned int pin,uint64_t raw);
... could the changed parameter also have "rte" in its name? I don't mind if it's "raw_rte", to avoid colliding with existing variable names (in at least the VT-d counterpart).
xen/drivers/passthrough/amd/iommu.h static int update_intremap_entry_from_ioapic( · line 324
Quoted 8 lines
xen/drivers/passthrough/amd/iommu.h +1
-1@@ -350,14 +319,11 @@ static int update_intremap_entry_from_ioapic( } void cf_check amd_iommu_ioapic_update_ire( -unsigned int apic,unsigned int reg,unsigned int value) +unsigned int apic,unsigned int pin,uint64_t raw) {struct IO_APIC_route_entry old_rte = { };
Looks like the initializer here now isn't needed anymore?
Quoted 51 lines
xen/drivers/passthrough/amd/iommu.h +5
-41@@ -364,48 +363,37 @@ static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu, new_ire = *iremap_entry; -if ( rte_upper ) - { -if ( x2apic_enabled ) - new_ire.remap.dst = value; -else - new_ire.remap.dst = (value >>24 ) <<8 ; - } +if ( x2apic_enabled ) + new_ire.remap.dst = new_rte.dest.dest32;else - { - *(((u32 *)&new_rte) +0 ) = value; - new_ire.remap.fpd =0 ; - new_ire.remap.dm = new_rte.dest_mode; - new_ire.remap.tm = new_rte.trigger; - new_ire.remap.dlm = new_rte.delivery_mode; -/* Hardware require RH = 1 for LPR delivery mode */ - new_ire.remap.rh = (new_ire.remap.dlm == dest_LowestPrio); - new_ire.remap.avail =0 ; - new_ire.remap.res_1 =0 ; - new_ire.remap.vector = new_rte.vector; - new_ire.remap.res_2 =0 ; - -set_ioapic_source_id (IO_APIC_ID (apic), &new_ire); - new_ire.remap.res_3 =0 ; - new_ire.remap.res_4 =0 ; - new_ire.remap.p =1 ;/* finally, set present bit */ - -/* now construct new ioapic rte entry */ - remap_rte->vector = new_rte.vector; - remap_rte->delivery_mode =0 ;/* has to be 0 for remap format */ - remap_rte->index_15 = (index >>15 ) &0x1 ; - remap_rte->index_0_14 = index &0x7fff ; - - remap_rte->delivery_status = new_rte.delivery_status; - remap_rte->polarity = new_rte.polarity; - remap_rte->irr = new_rte.irr; - remap_rte->trigger = new_rte.trigger; - remap_rte->mask = new_rte.mask; - remap_rte->reserved =0 ; - remap_rte->format =1 ;/* indicate remap format */ - } - -update_irte (iommu, iremap_entry, &new_ire, !init); + new_ire.remap.dst = (new_rte.dest.dest32 >>24 ) <<8 ; + + new_ire.remap.dm = new_rte.dest_mode;
xen/drivers/passthrough/amd/iommu.h static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu, · line 381
Quoted 10 lines
xen/drivers/passthrough/amd/iommu.h +10
-0+ new_ire.remap.tm = new_rte.trigger; + new_ire.remap.dlm = new_rte.delivery_mode; +/* Hardware require RH = 1 for LPR delivery mode */ + new_ire.remap.rh = (new_ire.remap.dlm == dest_LowestPrio); + new_ire.remap.vector = new_rte.vector; + +set_ioapic_source_id (IO_APIC_ID (apic), &new_ire); + new_ire.remap.p =1 ;/* finally, set present bit */ + +/* now construct new ioapic rte entry */
Nit: Would you mind correcting comment style here as you touch (unindent) the line?
Quoted 5 lines
xen/drivers/passthrough/amd/iommu.h +5
-0+ remap_rte->vector = new_rte.vector; + remap_rte->delivery_mode =0 ;/* has to be 0 for remap format */ + remap_rte->index_15 = (index >>15 ) &0x1 ; + remap_rte->index_0_14 = index &0x7fff ; +
xen/drivers/passthrough/amd/iommu.h static int ioapic_rte_to_remap_entry(struct vtd_iommu *iommu, · line 396
Quoted 10 lines
xen/drivers/passthrough/amd/iommu.h +10
-0+ remap_rte->delivery_status = new_rte.delivery_status; + remap_rte->polarity = new_rte.polarity; + remap_rte->irr = new_rte.irr; + remap_rte->trigger = new_rte.trigger; + remap_rte->mask = new_rte.mask; + remap_rte->reserved =0 ; + remap_rte->format =1 ;/* indicate remap format */ + +/* If cmpxchg16b is not available the caller must mask the IO-APIC pin. */ +update_irte (iommu, iremap_entry, &new_ire, !init && cpu_has_cx16);
The comment suggests to me that you mean to strengthen the logic, yet the code change weakens it (the called function BUG()s when "atomic" is true). Wouldn't this better be "!init && !<old RTE was/is masked>" anyway?
Quoted 39 lines
xen/drivers/passthrough/amd/iommu.h +12
-15@@ -439,36 +427,47 @@ unsigned int cf_check io_apic_read_remap_rte( } void cf_check io_apic_write_remap_rte( -unsigned int apic,unsigned int reg,unsigned int value) +unsigned int apic,unsigned int pin,uint64_t raw) { -unsigned int pin = (reg -0x10 ) /2 ; +struct IO_xAPIC_route_entry rte = { .raw = raw };struct IO_xAPIC_route_entry old_rte = { };struct IO_APIC_route_remap_entry *remap_rte; -unsigned int rte_upper = (reg &1 ) ?1 :0 ;struct vtd_iommu *iommu =ioapic_to_iommu (IO_APIC_ID (apic)); -int saved_mask; - - old_rte =__ioapic_read_entry (apic, pin, true); +bool masked = true; +int rc; remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte; -/* mask the interrupt while we change the intremap table */ - saved_mask = remap_rte->mask; - remap_rte->mask =1 ; -__io_apic_write (apic, reg & ~1 , *(u32 *)&old_rte); - remap_rte->mask = saved_mask; - -if (ioapic_rte_to_remap_entry (iommu, apic, pin, - &old_rte, rte_upper, value) ) +if ( !cpu_has_cx16 ) { -__io_apic_write (apic, reg, value); +/* +* Cannot atomically update the IRTE entry: mask the IO-APIC pin to +* avoid interrupts seeing an inconsistent IRTE entry. +*/ + old_rte =__ioapic_read_entry (apic, pin, true); +if ( !old_rte.mask ) + {
xen/drivers/passthrough/amd/iommu.h unsigned int cf_check io_apic_read_remap_rte( · line 456
Quoted 10 lines
xen/drivers/passthrough/amd/iommu.h +6
-3+ masked = false; + old_rte.mask =1 ; +__ioapic_write_entry (apic, pin, true, old_rte); + } + } -/* Recover the original value of 'mask' bit */ -if ( rte_upper ) -__io_apic_write (apic, reg & ~1 , *(u32 *)&old_rte); + rc =ioapic_rte_to_remap_entry (iommu, apic, pin, &old_rte, rte);
I realize it has been like this before, but passing &old_rte here is odd. We already have its properly typed alias: remap_rte. All the called function does is do the same type cast again. Question is whether ...
Quoted 6 lines
xen/drivers/passthrough/amd/iommu.h +6
-0+if ( rc ) + { +if ( !masked ) + { +/* Recover the original value of 'mask' bit */ + old_rte.mask =0 ;
xen/drivers/passthrough/amd/iommu.h unsigned int cf_check io_apic_read_remap_rte( · line 470
Quoted 10 lines
xen/drivers/passthrough/amd/iommu.h +7
-2+__ioapic_write_entry (apic, pin, true, old_rte); + } + dprintk(XENLOG_ERR VTDPREFIX, +"failed to update IRTE for IO-APIC#%u pin %u: %d\n" , + apic, pin, rc); +return ; } -else -__ioapic_write_entry (apic, pin, true, old_rte); +__ioapic_write_entry (apic, pin, true, old_rte);
... the further uses of old_rte then won't end up yet more confusing than they already are (first and foremost again because of "old" not being applicable here).
Jan
Re: [PATCH v2 4/5] x86/iommu: pass full IO-APIC RTE for remapping table update
Quoted 73 lines · Jan Beulich
Quoted 39 lines · Roger Pau Monne
xen/drivers/passthrough/vtd/intremap.c +12
-15@@ -439,36 +427,47 @@ unsigned int cf_check io_apic_read_remap_rte( } void cf_check io_apic_write_remap_rte( -unsigned int apic,unsigned int reg,unsigned int value) +unsigned int apic,unsigned int pin,uint64_t raw) { -unsigned int pin = (reg -0x10 ) /2 ; +struct IO_xAPIC_route_entry rte = { .raw = raw };struct IO_xAPIC_route_entry old_rte = { };struct IO_APIC_route_remap_entry *remap_rte; -unsigned int rte_upper = (reg &1 ) ?1 :0 ;struct vtd_iommu *iommu =ioapic_to_iommu (IO_APIC_ID (apic)); -int saved_mask; - - old_rte =__ioapic_read_entry (apic, pin, true); +bool masked = true; +int rc; remap_rte = (struct IO_APIC_route_remap_entry *) &old_rte; -/* mask the interrupt while we change the intremap table */ - saved_mask = remap_rte->mask; - remap_rte->mask =1 ; -__io_apic_write (apic, reg & ~1 , *(u32 *)&old_rte); - remap_rte->mask = saved_mask; - -if (ioapic_rte_to_remap_entry (iommu, apic, pin, - &old_rte, rte_upper, value) ) +if ( !cpu_has_cx16 ) { -__io_apic_write (apic, reg, value); +/* +* Cannot atomically update the IRTE entry: mask the IO-APIC pin to +* avoid interrupts seeing an inconsistent IRTE entry. +*/ + old_rte =__ioapic_read_entry (apic, pin, true); +if ( !old_rte.mask ) + {xen/drivers/passthrough/vtd/intremap.c unsigned int cf_check io_apic_read_remap_rte( · line 456
Quoted 10 lines
xen/drivers/passthrough/vtd/intremap.c +6
-3+ masked = false; + old_rte.mask =1 ; +__ioapic_write_entry (apic, pin, true, old_rte); + } + } -/* Recover the original value of 'mask' bit */ -if ( rte_upper ) -__io_apic_write (apic, reg & ~1 , *(u32 *)&old_rte); + rc =ioapic_rte_to_remap_entry (iommu, apic, pin, &old_rte, rte);I realize it has been like this before, but passing
&old_rtehere is odd. We already have its properly typed alias:remap_rte. All the called function does is do the same type cast again. Question is whether ...Quoted 6 lines
xen/drivers/passthrough/vtd/intremap.c +6
-0+if ( rc ) + { +if ( !masked ) + { +/* Recover the original value of 'mask' bit */ + old_rte.mask =0 ;xen/drivers/passthrough/vtd/intremap.c unsigned int cf_check io_apic_read_remap_rte( · line 470
Quoted 10 lines
xen/drivers/passthrough/vtd/intremap.c +7
-2+__ioapic_write_entry (apic, pin, true, old_rte); + } + dprintk(XENLOG_ERR VTDPREFIX, +"failed to update IRTE for IO-APIC#%u pin %u: %d\n" , + apic, pin, rc); +return ; } -else -__ioapic_write_entry (apic, pin, true, old_rte); +__ioapic_write_entry (apic, pin, true, old_rte);
Quoted 3 lines
... the further uses of
old_rtethen won't end up yet more confusing than they already are (first and foremost again because of "old" not being applicable here).
I've instead opted to remove remap_rte from io_apic_write_remap_rte(), as it was unused. I've also added a comment to clarify the usage of old_rte when ioapic_rte_to_remap_entry() returns success.
Thanks, Roger.
PATCH v2 5/5 amd/iommu: force atomic updates of remapping entries
In reply to Roger Pau MonneDisable XT (x2APIC) support and thus 128 IRTE entries if cmpxchg16b is not available, do so in order to avoid having to disable the IRTE (and possibly loose interrupts) when updating the entry. Note this also removes the usage of a while loop in order to disable the entry, since RemapEn is no longer toggled when updating.
---
x2APIC support was added late enough on AMD that CPUs that I believe all models that have x2APIC must also have CX16.
The AMD-Vi manual contains the following advice in the "2.3.2 Making Guest Interrupt Virtualization Changes" section:
"""
If RemapEn=1 before the change, the following steps may be followed to change interrupt virtualization information:
* Set RemapEn=0 in the entry to disable interrupt virtualization; device-initiated interrupt requests for the DeviceID and vector are processed as IO_PAGE_FAULT events.
* Update the fields in the IRTE and invalidate the interrupt table (see Section 2.4.5 [INVALIDATE_INTERRUPT_TABLE]).
* Set RemapEn=1 to virtualize interrupts from the device.
"""
However if the update of the IRTE is done atomically I assume that setting RemapEn=0 is not longer necessary, and that the INVALIDATE_INTERRUPT_TABLE command can be executed after the entry has been (atomically) updated.
Note that on VT-d IRTEs always have a size of 128b, so it's not possible to use a smaller entry size if CX16 is not available in order to do atomic updates.
--- xen/drivers/passthrough/amd/iommu_init.c | 10 +++++ xen/drivers/passthrough/amd/iommu_intr.c | 57 +++++++----------------- 2 files changed, 26 insertions(+), 41 deletions(-)
xen/drivers/passthrough/amd/iommu_init.c +10 -0
@@ -1417,6 +1417,16 @@ int __init amd_iommu_prepare(bool xt) has_xt = false; } +/* +* Prevent using 128bit IRTE format if there's no support for cmpxchg16b +* to update the entry atomically. +*/ +if ( xt && has_xt && !cpu_has_cx16 ) + { + has_xt = false; +printk (XENLOG_WARNING "AMD-Vi: x2APIC not supported without CX16\n" ); + } +if ( ivhd_type !=ACPI_IVRS_TYPE_HARDWARE ) iommuv2_enabled = true;
xen/drivers/passthrough/amd/iommu_intr.c +16 -41
101 lines not on this page · raw message
Signature · 1 line
2.41.0
Re: [PATCH v2 5/5] amd/iommu: force atomic updates of remapping entries
Quoted 25 lines · Roger Pau Monne
Disable XT (x2APIC) support and thus 128 IRTE entries if cmpxchg16b is not available, do so in order to avoid having to disable the IRTE (and possibly loose interrupts) when updating the entry. Note this also removes the usage of a while loop in order to disable the entry, since RemapEn is no longer toggled when updating.
Signed-off-by Roger Pau Monné <[email protected]>---
x2APIC support was added late enough on AMD that CPUs that I believe all models that have x2APIC must also have CX16.The AMD-Vi manual contains the following advice in the "2.3.2 Making Guest Interrupt Virtualization Changes" section:
"""
If RemapEn=1 before the change, the following steps may be followed to change interrupt virtualization information:
* Set RemapEn=0 in the entry to disable interrupt virtualization; device-initiated interrupt requests for the DeviceID and vector are processed asIO_PAGE_FAULTevents.
* Update the fields in the IRTE and invalidate the interrupt table (see Section 2.4.5 [INVALIDATE_INTERRUPT_TABLE]).
* Set RemapEn=1 to virtualize interrupts from the device.
"""
Quoted 4 lines
However if the update of the IRTE is done atomically I assume that setting RemapEn=0 is not longer necessary, and that the
INVALIDATE_INTERRUPT_TABLEcommand can be executed after the entry has been (atomically) updated.
There's one additional prereq to this: The IOMMU also needs to read 128-bit IRTEs atomically. I'm afraid we can't take this for given without it being said explicitly in the spec (I've just gone through and looked at all occurrences of "atomic", without finding anything applicable to IRTEs). If this can be resolved, I think I'm fine with the patch.
Jan
Re: [PATCH v2 5/5] amd/iommu: force atomic updates of remapping entries
Quoted 31 lines · Jan Beulich
Quoted 25 lines · Roger Pau Monne
Disable XT (x2APIC) support and thus 128 IRTE entries if cmpxchg16b is not available, do so in order to avoid having to disable the IRTE (and possibly loose interrupts) when updating the entry. Note this also removes the usage of a while loop in order to disable the entry, since RemapEn is no longer toggled when updating.
Signed-off-by Roger Pau Monné <[email protected]>---
x2APIC support was added late enough on AMD that CPUs that I believe all models that have x2APIC must also have CX16.The AMD-Vi manual contains the following advice in the "2.3.2 Making Guest Interrupt Virtualization Changes" section:
"""
If RemapEn=1 before the change, the following steps may be followed to change interrupt virtualization information:
* Set RemapEn=0 in the entry to disable interrupt virtualization; device-initiated interrupt requests for the DeviceID and vector are processed asIO_PAGE_FAULTevents.
* Update the fields in the IRTE and invalidate the interrupt table (see Section 2.4.5 [INVALIDATE_INTERRUPT_TABLE]).
* Set RemapEn=1 to virtualize interrupts from the device.
"""Quoted 4 lines
However if the update of the IRTE is done atomically I assume that setting RemapEn=0 is not longer necessary, and that the
INVALIDATE_INTERRUPT_TABLEcommand can be executed after the entry has been (atomically) updated.
Quoted 6 lines
There's one additional prereq to this: The IOMMU also needs to read 128-bit IRTEs atomically. I'm afraid we can't take this for given without it being said explicitly in the spec (I've just gone through and looked at all occurrences of "atomic", without finding anything applicable to IRTEs). If this can be resolved, I think I'm fine with the patch.
Hm, indeed I was taking for granted that the IOMMU will read the entry atomically.
I was also worried about the IOMMU caching only parts of the IRTE, and thus even when doing an atomic read of the full entry it could still get inconsistent data if using previously cached parts of the entry. The text in INVALIDATE_INTERRUPT_TABLE seems to suggest that IRTE fields could be cached on a field by field basis.
I will raise a question with AMD in order to try to figure out whether the IOMMU will do atomic reads, and also cache the full entry.
Thanks, Roger.