Skip to contentSkip to conversation

iommu/vt-d: Use cmpxchg16b to update posted format IRTE atomically

PATCH In review Feng Wu 2 participants 3 messages Oct 15, 2015 – Oct 16, 2015 IOMMU

lore
Share & tools
lore

Feng Wu

author
#0 lore raw

PATCH 1/1 iommu/vt-d: Use cmpxchg16b to update posted format IRTE atomically

If IRTE is in posted format, the 'pda' field goes across the 64-bit boundary, we need use cmpxchg16b to atomically update it. We only expose posted-interrupt when X86_FEATURE_CX16 is supported and use to update it atomically.

---
 drivers/iommu/intel_irq_remapping.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)
drivers/iommu/intel_irq_remapping.c +30 -3
index f15692a..b4f7569 100644
@@ -169,8 +169,26 @@ static int modify_irte(struct irq_2_iommu *irq_iommu,
 	index = irq_iommu->irte_index + irq_iommu->sub_handle;
 	irte = &iommu->ir_table->base[index];
 
-	set_64bit(&irte->low, irte_modified->low);
-	set_64bit(&irte->high, irte_modified->high);
+#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE)
+	if ((irte->pst == 1) || (irte_modified->pst == 1)) {
+		bool ret;
+
+		ret = cmpxchg_double(&irte->low, &irte->high,
+				     irte->low, irte->high,
+				     irte_modified->low, irte_modified->high);
+		/*
+		 * We use cmpxchg16 to atomically update the 128-bit IRTE,
+		 * and it cannot be updated by the hardware or other processors
+		 * behind us, so the return value of cmpxchg16 should be the
+		 * same as the old value.
+		 */
+		BUG_ON(!ret);
+	} else
+#endif
+	{
+		set_64bit(&irte->low, irte_modified->low);
+		set_64bit(&irte->high, irte_modified->high);
+	}
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -725,7 +743,16 @@ static inline void set_irq_posting_cap(void)
 	struct intel_iommu *iommu;
 
 	if (!disable_irq_post) {
-		intel_irq_remap_ops.capability |= 1 << IRQ_POSTING_CAP;
+		/*
+		 * If IRTE is in posted format, the 'pda' field goes across the
+		 * 64-bit boundary, we need use cmpxchg16b to atomically update
+		 * it. We only expose posted-interrupt when X86_FEATURE_CX16
+		 * is supported. Actually, hardware platforms supporting PI
+		 * should have X86_FEATURE_CX16 support, this has been confirmed
+		 * with Intel hardware guys.
+		 */
+		if ( cpu_has_cx16 )
+			intel_irq_remap_ops.capability |= 1 << IRQ_POSTING_CAP;
 
 		for_each_iommu(iommu, drhd)
 			if (!cap_pi_support(iommu->cap)) {
Signature · 1 line
2.1.0
maintainer
#1 lore raw
Quoted 24 lines · Feng Wu

If IRTE is in posted format, the 'pda' field goes across the 64-bit boundary, we need use cmpxchg16b to atomically update it. We only expose posted-interrupt when X86_FEATURE_CX16 is supported and use to update it atomically.

---
 drivers/iommu/intel_irq_remapping.c | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)
drivers/iommu/intel_irq_remapping.c +4 -2
index f15692a..b4f7569 100644
@@ -169,8 +169,26 @@ static int modify_irte(struct irq_2_iommu *irq_iommu,
 	index = irq_iommu->irte_index + irq_iommu->sub_handle;
 	irte = &iommu->ir_table->base[index];
 
-	set_64bit(&irte->low, irte_modified->low);
-	set_64bit(&irte->high, irte_modified->high);
+#if defined(CONFIG_HAVE_CMPXCHG_DOUBLE)
+	if ((irte->pst == 1) || (irte_modified->pst == 1)) {
+		bool ret;
+

drivers/iommu/intel_irq_remapping.c static int modify_irte(struct irq_2_iommu *irq_iommu, · line 185

Quoted 10 lines
drivers/iommu/intel_irq_remapping.c +10 -0
+		ret = cmpxchg_double(&irte->low, &irte->high,
+				     irte->low, irte->high,
+				     irte_modified->low, irte_modified->high);
+		/*
+		 * We use cmpxchg16 to atomically update the 128-bit IRTE,
+		 * and it cannot be updated by the hardware or other processors
+		 * behind us, so the return value of cmpxchg16 should be the
+		 * same as the old value.
+		 */
+		BUG_ON(!ret);

Changed this to a WARN_ON, otherwise applied.

author
#2 lore raw
Quoted 24 lines

-----Original Message-----
From: [email protected] [mailto:linux-kernel- [email protected]] On Behalf Of Joerg Roedel
Sent: Thursday, October 15, 2015 10:24 PM
To: Wu, Feng <[email protected]>
Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: [PATCH] iommu/vt-d: Use cmpxchg16b to update posted format
IRTE atomically

drivers/iommu/intel_irq_remapping.c static int modify_irte(struct irq_2_iommu *irq_iommu,

Quoted 11 lines · Feng Wu

If IRTE is in posted format, the 'pda' field goes across the 64-bit boundary, we need use cmpxchg16b to atomically update it. We only expose posted-interrupt when X86_FEATURE_CX16 is supported and use to update it atomically.

drivers/iommu/intel_irq_remapping.c +7 -0
+
+		ret = cmpxchg_double(&irte->low, &irte->high,
+				     irte->low, irte->high,
+				     irte_modified->low, irte_modified->high);
+		/*
+		 * We use cmpxchg16 to atomically update the 128-bit IRTE,
+		 * and it cannot be updated by the hardware or other

processors

Quoted 6 lines

drivers/iommu/intel_irq_remapping.c static int modify_irte(struct irq_2_iommu *irq_iommu, · line 185

Quoted 4 lines
drivers/iommu/intel_irq_remapping.c +4 -0
+		 * behind us, so the return value of cmpxchg16 should be the
+		 * same as the old value.
+		 */
+		BUG_ON(!ret);

Changed this to a WARN_ON, otherwise applied.

Thanks a lot, Joerg!

Thanks,
Feng