Skip to contentSkip to conversation

iommu/vt-d: Refactor PRI enable/disable steps

PATCH In review Lu Baolu 4 participants 16 messages Jul 1, 2024 – Jul 3, 2024

Subsystems (2)
lore
Share & tools
lore

Lu Baolu

author maintainer
#0 lore raw

v3 cover iommu/vt-d: Refactor PRI enable/disable steps

The page fault handling framework within the iommu core has defined the PRI enable and disable flows in the comments for the iopf_queue_remove_device() interface. This series aims to refactor the PRI enable/disable steps in the Intel iommu driver to align with these definitions.

Change log:
v3:
- Refine the lock requirement. Only assert the lock when it is required.

v2:
- https://lore.kernel.org/linux-iommu/[email protected]/
- The cache invalidation for a context entry change should not affect the devices not related to the entry. Fix this by always using device-selective cache invalidation.
v1:
- https://lore.kernel.org/linux-iommu/[email protected]/

Lu Baolu (2):
iommu/vt-d: Add helper to flush caches for context change iommu/vt-d: Refactor PCI PRI enabling/disabling callbacks

 drivers/iommu/intel/iommu.h |  13 +++++
 drivers/iommu/intel/iommu.c |  89 +++++++++++++++++------------
 drivers/iommu/intel/pasid.c | 108 +++++++++++++++++++++++++++++-------
 3 files changed, 153 insertions(+), 57 deletions(-)
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v3 1/2 iommu/vt-d: Add helper to flush caches for context change

This helper is used to flush the related caches following a change in a context table entry that was previously present. The VT-d specification provides guidance for such invalidations in section 6.5.3.3.

This helper replaces the existing open code in the code paths where a present context entry is being torn down.

---
 drivers/iommu/intel/iommu.h |   4 ++
 drivers/iommu/intel/iommu.c |  32 +----------
 drivers/iommu/intel/pasid.c | 106 +++++++++++++++++++++++++++++-------
 3 files changed, 92 insertions(+), 50 deletions(-)
drivers/iommu/intel/iommu.h +4 -0
index 9a3b064126de..63eb3306c025 100644
@@ -1143,6 +1143,10 @@ void cache_tag_flush_all(struct dmar_domain *domain);
 void cache_tag_flush_range_np(struct dmar_domain *domain, unsigned long start,
 			      unsigned long end);
 
+void intel_context_flush_present(struct device_domain_info *info,
+				 struct context_entry *context,
+				 bool affect_domains);
+
 #ifdef CONFIG_INTEL_IOMMU_SVM
 void intel_svm_check(struct intel_iommu *iommu);
 int intel_svm_enable_prq(struct intel_iommu *iommu);
drivers/iommu/intel/iommu.c +1 -31
index 1f0d6892a0b6..e84b0fdca107 100644
@@ -1359,21 +1359,6 @@ static void iommu_disable_pci_caps(struct device_domain_info *info)
 	}
 }
 
-static void __iommu_flush_dev_iotlb(struct device_domain_info *info,
-				    u64 addr, unsigned int mask)
-{
-	u16 sid, qdep;
-
-	if (!info || !info->ats_enabled)
-		return;
-
-	sid = info->bus << 8 | info->devfn;
-	qdep = info->ats_qdep;
-	qi_flush_dev_iotlb(info->iommu, sid, info->pfsid,
-			   qdep, addr, mask);
-	quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep);
-}
-
 static void intel_flush_iotlb_all(struct iommu_domain *domain)
 {
 	cache_tag_flush_all(to_dmar_domain(domain));
@@ -1959,7 +1944,6 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
 {
 	struct intel_iommu *iommu = info->iommu;
 	struct context_entry *context;
-	u16 did_old;
 
 	spin_lock(&iommu->lock);
 	context = iommu_context_addr(iommu, bus, devfn, 0);
@@ -1968,24 +1952,10 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
 		return;
 	}
 
-	did_old = context_domain_id(context);
-
 	context_clear_entry(context);
 	__iommu_flush_cache(iommu, context, sizeof(*context));
 	spin_unlock(&iommu->lock);
-	iommu->flush.flush_context(iommu,
-				   did_old,
-				   (((u16)bus) << 8) | devfn,
-				   DMA_CCMD_MASK_NOBIT,
-				   DMA_CCMD_DEVICE_INVL);
-
-	iommu->flush.flush_iotlb(iommu,
-				 did_old,
-				 0,
-				 0,
-				 DMA_TLB_DSI_FLUSH);
-
-	__iommu_flush_dev_iotlb(info, 0, MAX_AGAW_PFN_WIDTH);
+	intel_context_flush_present(info, context, true);
 }
 
 static int domain_setup_first_level(struct intel_iommu *iommu,
drivers/iommu/intel/pasid.c +87 -19

115 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#14 lore raw

PATCH v3 2/2 iommu/vt-d: Refactor PCI PRI enabling/disabling callbacks

In reply to Lu Baolu

Commit 0095bf83554f8 ("iommu: Improve iopf_queue_remove_device()") specified the flow for disabling the PRI on a device. Refactor the PRI callbacks in the intel iommu driver to better manage PRI enabling and disabling and align it with the device queue interfaces in the iommu core.

---
 drivers/iommu/intel/iommu.h |  9 ++++++
 drivers/iommu/intel/iommu.c | 57 +++++++++++++++++++++++++++++++++----
 drivers/iommu/intel/pasid.c |  2 --
 3 files changed, 61 insertions(+), 7 deletions(-)
drivers/iommu/intel/iommu.h +9 -0
index 63eb3306c025..b67c14da1240 100644
@@ -1045,6 +1045,15 @@ static inline void context_set_sm_pre(struct context_entry *context)
 	context->lo |= BIT_ULL(4);
 }
 
+/*
+ * Clear the PRE(Page Request Enable) field of a scalable mode context
+ * entry.
+ */
+static inline void context_clear_sm_pre(struct context_entry *context)
+{
+	context->lo &= ~BIT_ULL(4);
+}
+
 /* Returns a number of VTD pages, but aligned to MM page size */
 static inline unsigned long aligned_nrpages(unsigned long host_addr, size_t size)
 {
drivers/iommu/intel/iommu.c +52 -5

86 lines not on this page · raw message

drivers/iommu/intel/pasid.c +0 -2
index d6623d2c2050..9a7b5668c723 100644
@@ -749,8 +749,6 @@ static int context_entry_set_pasid_table(struct context_entry *context,
 
 	if (info->ats_supported)
 		context_set_sm_dte(context);
-	if (info->pri_supported)
-		context_set_sm_pre(context);
 	if (info->pasid_supported)
 		context_set_pasid(context);
 
Signature · 1 line
2.34.1
Reviewed-by on v3 1/2
#2 lore raw

RE: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

In reply to Lu Baolu
Quoted 27 lines

From: Lu Baolu <[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

drivers/iommu/intel/pasid.c +22 -0
+
+	/*
+	 * For scalable mode:
+	 * - Domain-selective PASID-cache invalidation to affected domains
+	 * - Domain-selective IOTLB invalidation to affected domains
+	 * - Global Device-TLB invalidation to affected functions
+	 */
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and there might
+		 * be potential PASID translations, the caller should hold
+		 * the lock to ensure that context changes and cache flushes
+		 * are atomic.
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;
+
+			did = pasid_get_domain_id(pte);
+			qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);

(quoted diff) +1 -0
+			iommu->flush.flush_iotlb(iommu, did, 0, 0,

DMA_TLB_DSI_FLUSH);

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 941

Quoted 5 lines
drivers/iommu/intel/pasid.c +5 -0
+		}
+	}
+
+	__context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.

otherwise this looks good:

Reviewed-by on v3 2/2
#15 lore raw

RE: [PATCH v3 2/2] iommu/vt-d: Refactor PCI PRI enabling/disabling callbacks

In reply to Lu Baolu
Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

Commit 0095bf83554f8 ("iommu: Improve iopf_queue_remove_device()") specified the flow for disabling the PRI on a device. Refactor the PRI callbacks in the intel iommu driver to better manage PRI enabling and disabling and align it with the device queue interfaces in the iommu core.

author maintainer
#3 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

In reply to Tian, Kevin
Quoted 26 lines · Kevin
Quoted 26 lines

From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

drivers/iommu/intel/pasid.c +22 -0
+
+	/*
+	 * For scalable mode:
+	 * - Domain-selective PASID-cache invalidation to affected domains
+	 * - Domain-selective IOTLB invalidation to affected domains
+	 * - Global Device-TLB invalidation to affected functions
+	 */
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and there might
+		 * be potential PASID translations, the caller should hold
+		 * the lock to ensure that context changes and cache flushes
+		 * are atomic.
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;
+
+			did = pasid_get_domain_id(pte);
+			qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);

(quoted diff) +1 -0
+			iommu->flush.flush_iotlb(iommu, did, 0, 0,
Quoted 8 lines

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 941

Quoted 6 lines

DMA_TLB_DSI_FLUSH);

drivers/iommu/intel/pasid.c +5 -0
+		}
+	}
+
+	__context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.

I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.

If so, perhaps we need a separated fix to address this.

Best regards,
baolu

author maintainer
#4 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 36 lines · Baolu Lu
Quoted 26 lines · Kevin
Quoted 26 lines

From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

(quoted diff) +22 -0
+
+    /*
+     * For scalable mode:
+     * - Domain-selective PASID-cache invalidation to affected domains
+     * - Domain-selective IOTLB invalidation to affected domains
+     * - Global Device-TLB invalidation to affected functions
+     */
+    if (flush_domains) {
+        /*
+         * If the IOMMU is running in scalable mode and there might
+         * be potential PASID translations, the caller should hold
+         * the lock to ensure that context changes and cache flushes
+         * are atomic.
+         */
+        assert_spin_locked(&iommu->lock);
+        for (i = 0; i < info->pasid_table->max_pasid; i++) {
+            pte = intel_pasid_get_entry(info->dev, i);
+            if (!pte || !pasid_pte_is_present(pte))
+                continue;
+
+            did = pasid_get_domain_id(pte);
+            qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);
+            iommu->flush.flush_iotlb(iommu, did, 0, 0,

Quoted 8 lines

Review

Quoted 6 lines

DMA_TLB_DSI_FLUSH);
+        }
+    }

(quoted diff) +3 -0
+
+    __context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.

Quoted 3 lines

I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.

I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.

I will add below line to address this.

drivers/iommu/intel/pasid.c +0 -0
index 9a7b5668c723..91db0876682e 100644
@@ -932,6 +932,7 @@ void intel_context_flush_present(struct device_domain_info *info,
                        did = pasid_get_domain_id(pte);
                        qi_flush_pasid_cache(iommu, did, 
QI_PC_ALL_PASIDS, 0);
                        iommu->flush.flush_iotlb(iommu, did, 0, 0, 
DMA_TLB_DSI_FLUSH);
+                       pasid_cache_invalidation_with_pasid(iommu, did, i);
                }
        }

Thanks!

Best regards,
baolu

#9 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

In reply to Lu Baolu
Quoted 8 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3 -0
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and there

might

(quoted diff) +1 -0
+		 * be potential PASID translations, the caller should

hold

(quoted diff) +1 -0
+		 * the lock to ensure that context changes and cache

flushes

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932

Quoted 7 lines
drivers/iommu/intel/pasid.c +7 -0
+		 * are atomic.
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;

Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.

Review

Quoted 7 lines
(quoted diff) +2 -0
+			did = pasid_get_domain_id(pte);
+			qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);

(quoted diff) +1 -0
+			iommu->flush.flush_iotlb(iommu, did, 0, 0,

DMA_TLB_DSI_FLUSH);

(quoted diff) +2 -0
+		}
+	}

Thanks,

Jacob

author maintainer
#10 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3 -0
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and there

might

(quoted diff) +1 -0
+		 * be potential PASID translations, the caller should

hold

(quoted diff) +1 -0
+		 * the lock to ensure that context changes and cache

flushes

drivers/iommu/intel/pasid.c +1 -0
+		 * are atomic.
Quoted 8 lines

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932

Quoted 6 lines
drivers/iommu/intel/pasid.c +6 -0
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;

Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.

Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.

Best regards,
baolu

author maintainer
#5 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

In reply to Baolu Lu
Quoted 53 lines · Baolu Lu
Quoted 36 lines · Baolu Lu
Quoted 26 lines · Kevin
Quoted 26 lines

From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

(quoted diff) +22 -0
+
+    /*
+     * For scalable mode:
+     * - Domain-selective PASID-cache invalidation to affected domains
+     * - Domain-selective IOTLB invalidation to affected domains
+     * - Global Device-TLB invalidation to affected functions
+     */
+    if (flush_domains) {
+        /*
+         * If the IOMMU is running in scalable mode and there might
+         * be potential PASID translations, the caller should hold
+         * the lock to ensure that context changes and cache flushes
+         * are atomic.
+         */
+        assert_spin_locked(&iommu->lock);
+        for (i = 0; i < info->pasid_table->max_pasid; i++) {
+            pte = intel_pasid_get_entry(info->dev, i);
+            if (!pte || !pasid_pte_is_present(pte))
+                continue;
+
+            did = pasid_get_domain_id(pte);
+            qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);
+            iommu->flush.flush_iotlb(iommu, did, 0, 0,

Quoted 8 lines

Review

Quoted 6 lines

DMA_TLB_DSI_FLUSH);
+        }
+    }

(quoted diff) +3 -0
+
+    __context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.

Quoted 3 lines

I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.

I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.

I will add below line to address this.

drivers/iommu/intel/pasid.c +0 -0
index 9a7b5668c723..91db0876682e 100644

drivers/iommu/intel/pasid.c void intel_context_flush_present(struct · line 938

Quoted 7 lines
drivers/iommu/intel/pasid.c +1 -0
@@ -932,6 +932,7 @@ void intel_context_flush_present(struct 
 device_domain_info *info,
                         did = pasid_get_domain_id(pte);
                         qi_flush_pasid_cache(iommu, did, 
 QI_PC_ALL_PASIDS, 0);
                         iommu->flush.flush_iotlb(iommu, did, 0, 0, 
 DMA_TLB_DSI_FLUSH);
+                       pasid_cache_invalidation_with_pasid(iommu, did, i);

Should be

	devtlb_invalidation_with_pasid(iommu, info->dev, i);

Sorry for the typo.

Best regards,
baolu

#6 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 42 lines · Baolu Lu
Quoted 41 lines · Baolu Lu
Quoted 39 lines · Baolu Lu
Quoted 26 lines · Kevin
Quoted 26 lines

From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

(quoted diff) +22 -0
+
+    /*
+     * For scalable mode:
+     * - Domain-selective PASID-cache invalidation to affected domains
+     * - Domain-selective IOTLB invalidation to affected domains
+     * - Global Device-TLB invalidation to affected functions
+     */
+    if (flush_domains) {
+        /*
+         * If the IOMMU is running in scalable mode and there might
+         * be potential PASID translations, the caller should hold
+         * the lock to ensure that context changes and cache flushes
+         * are atomic.
+         */
+        assert_spin_locked(&iommu->lock);
+        for (i = 0; i < info->pasid_table->max_pasid; i++) {
+            pte = intel_pasid_get_entry(info->dev, i);
+            if (!pte || !pasid_pte_is_present(pte))
+                continue;
+
+            did = pasid_get_domain_id(pte);
+            qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);
+            iommu->flush.flush_iotlb(iommu, did, 0, 0,

Quoted 8 lines

Review

Quoted 6 lines

DMA_TLB_DSI_FLUSH);
+        }
+    }

(quoted diff) +3 -0
+
+    __context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.

I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.

Quoted 5 lines
Quoted 5 lines

I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.

hmmm. "Table 25. Guidance to Software for Invalidations" only mentions global devTLB invalidation. 10.3.8 PASID and Global Invalidate of PCIe 6.2 spec has below definition.

For Invalidation Requests that do not have a PASID, the ATC shall:
• Invalidate ATC entries within the indicate memory range that were requested without a PASID value.
• Invalidate ATC entries at all addresses that were requested with any PASID value.

AFAIK. A devTLB invalidate descriptor submitted to VT-d should be converted
to be a PCIe ATC invalidation request without PASID prefix. So it may be
subjected to the above definition. If so, no need to have a PASID-based devTLB invalidation descriptor.

Quoted 22 lines

drivers/iommu/intel/pasid.c void intel_context_flush_present(struct · line 936

Quoted 12 lines

I will add below line to address this.

drivers/iommu/intel/pasid.c +1 -0
index 9a7b5668c723..91db0876682e 100644
@@ -932,6 +932,7 @@ void intel_context_flush_present(struct 
 device_domain_info *info,
                          did = pasid_get_domain_id(pte);
                          qi_flush_pasid_cache(iommu, did, QI_PC_ALL_PASIDS, 0);
                          iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH);
+                       pasid_cache_invalidation_with_pasid(iommu, did, i);

Should be

    devtlb_invalidation_with_pasid(iommu, info->dev, i);

Sorry for the typo.

Best regards,
baolu

Signature · 2 lines
Regards,
Yi Liu
#7 lore raw

RE: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 63 lines

From: Liu, Yi L <[email protected]>
Sent: Tuesday, July 2, 2024 2:25 PM

Quoted 43 lines · Baolu Lu
Quoted 42 lines · Baolu Lu
Quoted 40 lines · Baolu Lu
Quoted 27 lines · Kevin
Quoted 27 lines

From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

(quoted diff) +22 -0
+
+    /*
+     * For scalable mode:
+     * - Domain-selective PASID-cache invalidation to affected domains
+     * - Domain-selective IOTLB invalidation to affected domains
+     * - Global Device-TLB invalidation to affected functions
+     */
+    if (flush_domains) {
+        /*
+         * If the IOMMU is running in scalable mode and there might
+         * be potential PASID translations, the caller should hold
+         * the lock to ensure that context changes and cache flushes
+         * are atomic.
+         */
+        assert_spin_locked(&iommu->lock);
+        for (i = 0; i < info->pasid_table->max_pasid; i++) {
+            pte = intel_pasid_get_entry(info->dev, i);
+            if (!pte || !pasid_pte_is_present(pte))
+                continue;
+
+            did = pasid_get_domain_id(pte);
+            qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);
+            iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH);

Quoted 8 lines

Review

Quoted 5 lines

+        }
+    }

(quoted diff) +3 -0
+
+    __context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation
with global bit set.

I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.

Quoted 5 lines
Quoted 5 lines

I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.

hmmm. "Table 25. Guidance to Software for Invalidations" only mentions global devTLB invalidation. 10.3.8 PASID and Global Invalidate of PCIe 6.2 spec has below definition.

For Invalidation Requests that do not have a PASID, the ATC shall:
• Invalidate ATC entries within the indicate memory range that were requested without a PASID value.
• Invalidate ATC entries at all addresses that were requested with any PASID value.

Quoted 5 lines

AFAIK. A devTLB invalidate descriptor submitted to VT-d should be converted to be a PCIe ATC invalidation request without PASID prefix. So it may be subjected to the above definition. If so, no need to have a PASID-based devTLB invalidation descriptor.

You are correct. The wording in VT-d spec is a bit confusing on saying that devtlb invalidation descriptor is only for request w/o PASID. 😉

author maintainer
#8 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 63 lines · Kevin
Quoted 63 lines

From: Liu, Yi L <[email protected]>
Sent: Tuesday, July 2, 2024 2:25 PM

Quoted 43 lines · Baolu Lu
Quoted 42 lines · Baolu Lu
Quoted 40 lines · Baolu Lu
Quoted 27 lines · Kevin
Quoted 27 lines

From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM

(quoted diff) +22 -0
+
+    /*
+     * For scalable mode:
+     * - Domain-selective PASID-cache invalidation to affected domains
+     * - Domain-selective IOTLB invalidation to affected domains
+     * - Global Device-TLB invalidation to affected functions
+     */
+    if (flush_domains) {
+        /*
+         * If the IOMMU is running in scalable mode and there might
+         * be potential PASID translations, the caller should hold
+         * the lock to ensure that context changes and cache flushes
+         * are atomic.
+         */
+        assert_spin_locked(&iommu->lock);
+        for (i = 0; i < info->pasid_table->max_pasid; i++) {
+            pte = intel_pasid_get_entry(info->dev, i);
+            if (!pte || !pasid_pte_is_present(pte))
+                continue;
+
+            did = pasid_get_domain_id(pte);
+            qi_flush_pasid_cache(iommu, did,

QI_PC_ALL_PASIDS, 0);
+            iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH);

Quoted 8 lines

Review

Quoted 5 lines

+        }
+    }

(quoted diff) +3 -0
+
+    __context_flush_dev_iotlb(info);
+}

this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation
with global bit set.

I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.

Quoted 5 lines
Quoted 5 lines

I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.

hmmm. "Table 25. Guidance to Software for Invalidations" only mentions global devTLB invalidation. 10.3.8 PASID and Global Invalidate of PCIe 6.2 spec has below definition.

For Invalidation Requests that do not have a PASID, the ATC shall:
• Invalidate ATC entries within the indicate memory range that were requested without a PASID value.
• Invalidate ATC entries at all addresses that were requested with any PASID value.

Quoted 8 lines
Quoted 5 lines

AFAIK. A devTLB invalidate descriptor submitted to VT-d should be converted to be a PCIe ATC invalidation request without PASID prefix. So it may be subjected to the above definition. If so, no need to have a PASID-based devTLB invalidation descriptor.

You are correct. The wording in VT-d spec is a bit confusing on saying that devtlb invalidation descriptor is only for request w/o PASID. 😉

Okay, so I will remove the line of pasid-based-dev-iotlb invalidation.

Thank you Yi, for the clarification.

Best regards,
baolu

#11 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

In reply to Baolu Lu
Quoted 22 lines · Baolu Lu
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3 -0
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and there

might

(quoted diff) +1 -0
+		 * be potential PASID translations, the caller should

hold

(quoted diff) +1 -0
+		 * the lock to ensure that context changes and cache

flushes

drivers/iommu/intel/pasid.c +1 -0
+		 * are atomic.
Quoted 8 lines

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932

Quoted 6 lines
drivers/iommu/intel/pasid.c +6 -0
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;  

Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.

Quoted 2 lines

Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.

I think it is necessary, because without tracking domain IDs, the code above would have duplicated invalidations.
For example: a device PASID table has the following entries
PASID DomainID
-------------------------

	100	1
	200	1
	300	2

-------------------------
When a present context entry changes, we need to do:
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);

With this code, we do
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);//duplicated! qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);

Thanks,

Jacob

author maintainer
#12 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 39 lines · Jacob Pan
Quoted 21 lines · Baolu Lu
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3 -0
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and there

might

(quoted diff) +1 -0
+		 * be potential PASID translations, the caller should

hold

(quoted diff) +1 -0
+		 * the lock to ensure that context changes and cache

flushes

drivers/iommu/intel/pasid.c +1 -0
+		 * are atomic.
Quoted 8 lines

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932

Quoted 6 lines
drivers/iommu/intel/pasid.c +6 -0
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;

Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.

Quoted 2 lines

Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.

I think it is necessary, because without tracking domain IDs, the code above would have duplicated invalidations.
For example: a device PASID table has the following entries
PASID DomainID
-------------------------

	100	1
	200	1
	300	2

-------------------------
When a present context entry changes, we need to do:
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);

Quoted 4 lines

With this code, we do
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);//duplicated! qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);

Yes, this is likely. But currently enabling and disabling PRI happens in driver's probe and release paths. Therefore such duplicate is not so critical.

For long term, I have a plan to abstract the domain id into an object so that domains attached to different PASIDs of a device could share a domain id. With that done, we could improve this code by iterating the domain id objects for a device and performing cache invalidation directly.

Thanks,
baolu

Reviewed-by
#13 lore raw

Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change

Quoted 49 lines · Baolu Lu
Quoted 39 lines · Jacob Pan
Quoted 21 lines · Baolu Lu
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3 -0
+	if (flush_domains) {
+		/*
+		 * If the IOMMU is running in scalable mode and

there might

(quoted diff) +1 -0
+		 * be potential PASID translations, the caller

should hold

(quoted diff) +1 -0
+		 * the lock to ensure that context changes and cache

flushes

drivers/iommu/intel/pasid.c +1 -0
+		 * are atomic.
Quoted 8 lines

drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932

Quoted 6 lines
drivers/iommu/intel/pasid.c +6 -0
+		 */
+		assert_spin_locked(&iommu->lock);
+		for (i = 0; i < info->pasid_table->max_pasid; i++) {
+			pte = intel_pasid_get_entry(info->dev, i);
+			if (!pte || !pasid_pte_is_present(pte))
+				continue;  

Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.

Quoted 2 lines

Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.

I think it is necessary, because without tracking domain IDs, the code above would have duplicated invalidations.
For example: a device PASID table has the following entries
PASID DomainID
-------------------------

	100	1
	200	1
	300	2

-------------------------
When a present context entry changes, we need to do:
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);

Quoted 4 lines

With this code, we do
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);//duplicated! qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);

Yes, this is likely. But currently enabling and disabling PRI happens in driver's probe and release paths. Therefore such duplicate is not so critical.

Quoted 5 lines

For long term, I have a plan to abstract the domain id into an object so that domains attached to different PASIDs of a device could share a domain id. With that done, we could improve this code by iterating the domain id objects for a device and performing cache invalidation directly.

Sounds good. It might be helpful to add a comment to clarify for others who might wonder about the duplicates.

Thanks,

Jacob