iommu/vt-d: Use cmpxchg16b to update posted format IRTE atomically
PATCH 1/1 iommu/vt-d: Use cmpxchg16b to update posted format IRTE atomically
If IRTE is in posted format, the 'pda' field goes across the 64-bit boundary, we need use cmpxchg16b to atomically update it. We only expose posted-interrupt when X86_FEATURE_CX16 is supported and use to update it atomically.
--- drivers/iommu/intel_irq_remapping.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)
drivers/iommu/intel_irq_remapping.c +30 -3
@@ -169,8 +169,26 @@ static int modify_irte(struct irq_2_iommu *irq_iommu, index = irq_iommu->irte_index + irq_iommu->sub_handle; irte = &iommu->ir_table->base[index]; -set_64bit (&irte->low, irte_modified->low); -set_64bit (&irte->high, irte_modified->high); +#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE ) +if ((irte->pst ==1 ) || (irte_modified->pst ==1 )) { +bool ret; + + ret =cmpxchg_double (&irte->low, &irte->high, + irte->low, irte->high, + irte_modified->low, irte_modified->high); +/* +* We use cmpxchg16 to atomically update the 128-bit IRTE, +* and it cannot be updated by the hardware or other processors +* behind us, so the return value of cmpxchg16 should be the +* same as the old value. +*/ +BUG_ON (!ret); + } else +#endif + { +set_64bit (&irte->low, irte_modified->low); +set_64bit (&irte->high, irte_modified->high); + }__iommu_flush_cache (iommu, irte,sizeof (*irte)); rc =qi_flush_iec (iommu, index,0 );
@@ -725,7 +743,16 @@ static inline void set_irq_posting_cap(void)struct intel_iommu *iommu;if (!disable_irq_post) { - intel_irq_remap_ops.capability |=1 <<IRQ_POSTING_CAP ; +/* +* If IRTE is in posted format, the 'pda' field goes across the +* 64-bit boundary, we need use cmpxchg16b to atomically update +* it. We only expose posted-interrupt when X86_FEATURE_CX16 +* is supported. Actually, hardware platforms supporting PI +* should have X86_FEATURE_CX16 support, this has been confirmed +* with Intel hardware guys. +*/ +if ( cpu_has_cx16 ) + intel_irq_remap_ops.capability |=1 <<IRQ_POSTING_CAP ;for_each_iommu (iommu, drhd)if (!cap_pi_support (iommu->cap)) {
Signature · 1 line
2.1.0
Joerg Roedel
maintainerQuoted 24 lines · Feng Wu
If IRTE is in posted format, the 'pda' field goes across the 64-bit boundary, we need use cmpxchg16b to atomically update it. We only expose posted-interrupt when
X86_FEATURE_CX16is supported and use to update it atomically.Signed-off-by Feng Wu <[email protected]>--- drivers/iommu/intel_irq_remapping.c | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-)drivers/iommu/intel_irq_remapping.c +4
-2@@ -169,8 +169,26 @@ static int modify_irte(struct irq_2_iommu *irq_iommu, index = irq_iommu->irte_index + irq_iommu->sub_handle; irte = &iommu->ir_table->base[index]; -set_64bit (&irte->low, irte_modified->low); -set_64bit (&irte->high, irte_modified->high); +#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE ) +if ((irte->pst ==1 ) || (irte_modified->pst ==1 )) { +bool ret; +
drivers/iommu/intel_irq_remapping.c static int modify_irte(struct irq_2_iommu *irq_iommu, · line 185
Quoted 10 lines
drivers/iommu/intel_irq_remapping.c +10
-0+ ret =cmpxchg_double (&irte->low, &irte->high, + irte->low, irte->high, + irte_modified->low, irte_modified->high); +/* +* We use cmpxchg16 to atomically update the 128-bit IRTE, +* and it cannot be updated by the hardware or other processors +* behind us, so the return value of cmpxchg16 should be the +* same as the old value. +*/ +BUG_ON (!ret);
Changed this to a WARN_ON, otherwise applied.
Quoted 24 lines
-----Original Message-----
From: [email protected] [mailto:linux-kernel- [email protected]] On Behalf Of Joerg Roedel
Sent: Thursday, October 15, 2015 10:24 PM
To: Wu, Feng <[email protected]>
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: [PATCH] iommu/vt-d: Use cmpxchg16b to update posted format
IRTE atomicallydrivers/iommu/intel_irq_remapping.c static int modify_irte(struct irq_2_iommu *irq_iommu,
Quoted 11 lines · Feng Wu
If IRTE is in posted format, the 'pda' field goes across the 64-bit boundary, we need use cmpxchg16b to atomically update it. We only expose posted-interrupt when
X86_FEATURE_CX16is supported and use to update it atomically.drivers/iommu/intel_irq_remapping.c +7
-0+ + ret =cmpxchg_double (&irte->low, &irte->high, + irte->low, irte->high, + irte_modified->low, irte_modified->high); +/* +* We use cmpxchg16 to atomically update the 128-bit IRTE, +* and it cannot be updated by the hardware or other processors
Quoted 6 lines
drivers/iommu/intel_irq_remapping.c static int modify_irte(struct irq_2_iommu *irq_iommu, · line 185
Quoted 4 lines
drivers/iommu/intel_irq_remapping.c +4
-0+* behind us, so the return value of cmpxchg16 should be the +* same as the old value. +*/ +BUG_ON (!ret);Changed this to a
WARN_ON, otherwise applied.
Thanks a lot, Joerg!
Thanks,
Feng