iommu/vt-d: Refactor PRI enable/disable steps
Subsystems (2)
Lu Baolu
maintainerv3 cover iommu/vt-d: Refactor PRI enable/disable steps
The page fault handling framework within the iommu core has defined the PRI enable and disable flows in the comments for the iopf_queue_remove_device() interface. This series aims to refactor the PRI enable/disable steps in the Intel iommu driver to align with these definitions.
Change log:
v3:
- Refine the lock requirement. Only assert the lock when it is required.
v2:
- https://lore.kernel.org/linux-iommu/[email protected]/
- The cache invalidation for a context entry change should not affect the devices not related to the entry. Fix this by always using device-selective cache invalidation.
v1:
- https://lore.kernel.org/linux-iommu/[email protected]/
Lu Baolu (2):
iommu/vt-d: Add helper to flush caches for context change iommu/vt-d: Refactor PCI PRI enabling/disabling callbacks
drivers/iommu/intel/iommu.h | 13 +++++ drivers/iommu/intel/iommu.c | 89 +++++++++++++++++------------ drivers/iommu/intel/pasid.c | 108 +++++++++++++++++++++++++++++------- 3 files changed, 153 insertions(+), 57 deletions(-)
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v3 1/2 iommu/vt-d: Add helper to flush caches for context change
This helper is used to flush the related caches following a change in a context table entry that was previously present. The VT-d specification provides guidance for such invalidations in section 6.5.3.3.
This helper replaces the existing open code in the code paths where a present context entry is being torn down.
--- drivers/iommu/intel/iommu.h | 4 ++ drivers/iommu/intel/iommu.c | 32 +---------- drivers/iommu/intel/pasid.c | 106 +++++++++++++++++++++++++++++------- 3 files changed, 92 insertions(+), 50 deletions(-)
drivers/iommu/intel/iommu.h +4 -0
@@ -1143,6 +1143,10 @@ void cache_tag_flush_all(struct dmar_domain *domain);void cache_tag_flush_range_np (struct dmar_domain *domain,unsigned long start,unsigned long end); +void intel_context_flush_present (struct device_domain_info *info, +struct context_entry *context, +bool affect_domains); +#ifdef CONFIG_INTEL_IOMMU_SVM void intel_svm_check (struct intel_iommu *iommu);int intel_svm_enable_prq (struct intel_iommu *iommu);
drivers/iommu/intel/iommu.c +1 -31
@@ -1359,21 +1359,6 @@ static void iommu_disable_pci_caps(struct device_domain_info *info) } } -static void __iommu_flush_dev_iotlb (struct device_domain_info *info, -u64 addr,unsigned int mask) -{ -u16 sid, qdep; - -if (!info || !info->ats_enabled) -return ; - - sid = info->bus <<8 | info->devfn; - qdep = info->ats_qdep; -qi_flush_dev_iotlb (info->iommu, sid, info->pfsid, - qdep, addr, mask); -quirk_extra_dev_tlb_flush (info, addr, mask,IOMMU_NO_PASID , qdep); -} -static void intel_flush_iotlb_all (struct iommu_domain *domain) {cache_tag_flush_all (to_dmar_domain (domain));
@@ -1959,7 +1944,6 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8 {struct intel_iommu *iommu = info->iommu;struct context_entry *context; -u16 did_old;spin_lock (&iommu->lock); context =iommu_context_addr (iommu, bus, devfn,0 );
@@ -1968,24 +1952,10 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8return ; } - did_old =context_domain_id (context); -context_clear_entry (context);__iommu_flush_cache (iommu, context,sizeof (*context));spin_unlock (&iommu->lock); - iommu->flush.flush_context (iommu, - did_old, - (((u16 )bus) <<8 ) | devfn, -DMA_CCMD_MASK_NOBIT , -DMA_CCMD_DEVICE_INVL ); - - iommu->flush.flush_iotlb (iommu, - did_old, -0 , -0 , -DMA_TLB_DSI_FLUSH ); - -__iommu_flush_dev_iotlb (info,0 ,MAX_AGAW_PFN_WIDTH ); +intel_context_flush_present (info, context, true); } static int domain_setup_first_level(struct intel_iommu *iommu,
drivers/iommu/intel/pasid.c +87 -19
115 lines not on this page · raw message
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v3 2/2 iommu/vt-d: Refactor PCI PRI enabling/disabling callbacks
In reply to Lu BaoluCommit 0095bf83554f8 ("iommu: Improve iopf_queue_remove_device()") specified the flow for disabling the PRI on a device. Refactor the PRI callbacks in the intel iommu driver to better manage PRI enabling and disabling and align it with the device queue interfaces in the iommu core.
--- drivers/iommu/intel/iommu.h | 9 ++++++ drivers/iommu/intel/iommu.c | 57 +++++++++++++++++++++++++++++++++---- drivers/iommu/intel/pasid.c | 2 -- 3 files changed, 61 insertions(+), 7 deletions(-)
drivers/iommu/intel/iommu.h +9 -0
@@ -1045,6 +1045,15 @@ static inline void context_set_sm_pre(struct context_entry *context) context->lo |=BIT_ULL (4 ); } +/* +* Clear the PRE(Page Request Enable) field of a scalable mode context +* entry. +*/ +static inline void context_clear_sm_pre (struct context_entry *context) +{ + context->lo &= ~BIT_ULL (4 ); +} +/* Returns a number of VTD pages, but aligned to MM page size */ static inline unsigned long aligned_nrpages (unsigned long host_addr,size_t size) {
drivers/iommu/intel/iommu.c +52 -5
86 lines not on this page · raw message
drivers/iommu/intel/pasid.c +0 -2
@@ -749,8 +749,6 @@ static int context_entry_set_pasid_table(struct context_entry *context,if (info->ats_supported)context_set_sm_dte (context); -if (info->pri_supported) -context_set_sm_pre (context);if (info->pasid_supported)context_set_pasid (context);
Signature · 1 line
2.34.1
RE: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
In reply to Lu BaoluQuoted 27 lines
From: Lu Baolu <[email protected]>
Sent: Monday, July 1, 2024 7:23 PMdrivers/iommu/intel/pasid.c +22
-0+ +/* +* For scalable mode: +* - Domain-selective PASID-cache invalidation to affected domains +* - Domain-selective IOTLB invalidation to affected domains +* - Global Device-TLB invalidation to affected functions +*/ +if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might +* be potential PASID translations, the caller should hold +* the lock to ensure that context changes and cache flushes +* are atomic. +*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ; + + did =pasid_get_domain_id (pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);(quoted diff) +1
-0+ iommu->flush.flush_iotlb(iommu, did, 0, 0,
DMA_TLB_DSI_FLUSH);
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 941
Quoted 5 lines
drivers/iommu/intel/pasid.c +5
-0+ } + } + +__context_flush_dev_iotlb (info); +}
this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.
otherwise this looks good:
RE: [PATCH v3 2/2] iommu/vt-d: Refactor PCI PRI enabling/disabling callbacks
In reply to Lu BaoluQuoted 10 lines
From: Lu Baolu <[email protected]>
Sent: Monday, July 1, 2024 7:23 PMCommit
0095bf83554f8("iommu: Improveiopf_queue_remove_device()") specified the flow for disabling the PRI on a device. Refactor the PRI callbacks in the intel iommu driver to better manage PRI enabling and disabling and align it with the device queue interfaces in the iommu core.Signed-off-by Lu Baolu <[email protected]>
Baolu Lu
maintainerRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
In reply to Tian, KevinQuoted 26 lines · Kevin
Quoted 26 lines
From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PMdrivers/iommu/intel/pasid.c +22
-0+ +/* +* For scalable mode: +* - Domain-selective PASID-cache invalidation to affected domains +* - Domain-selective IOTLB invalidation to affected domains +* - Global Device-TLB invalidation to affected functions +*/ +if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might +* be potential PASID translations, the caller should hold +* the lock to ensure that context changes and cache flushes +* are atomic. +*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ; + + did =pasid_get_domain_id (pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);(quoted diff) +1
-0+ iommu->flush.flush_iotlb(iommu, did, 0, 0,
Quoted 8 lines
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 941
Quoted 6 lines
DMA_TLB_DSI_FLUSH);drivers/iommu/intel/pasid.c +5
-0+ } + } + +__context_flush_dev_iotlb (info); +}this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.
I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.
If so, perhaps we need a separated fix to address this.
Best regards,
baolu
Baolu Lu
maintainerRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 36 lines · Baolu Lu
Quoted 26 lines · Kevin
Quoted 26 lines
From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM(quoted diff) +22
-0+ + /* + * For scalable mode: + * - Domain-selective PASID-cache invalidation to affected domains + * - Domain-selective IOTLB invalidation to affected domains + * - Global Device-TLB invalidation to affected functions + */ + if (flush_domains) { + /* + * If the IOMMU is running in scalable mode and there might + * be potential PASID translations, the caller should hold + * the lock to ensure that context changes and cache flushes + * are atomic. + */ + assert_spin_locked(&iommu->lock); + for (i = 0; i < info->pasid_table->max_pasid; i++) { + pte = intel_pasid_get_entry(info->dev, i); + if (!pte || !pasid_pte_is_present(pte)) + continue; + + did = pasid_get_domain_id(pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);
+ iommu->flush.flush_iotlb(iommu, did, 0, 0,Quoted 8 lines
Review
Quoted 6 lines
DMA_TLB_DSI_FLUSH);
+ }
+ }(quoted diff) +3
-0+ + __context_flush_dev_iotlb(info); +}this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.
Quoted 3 lines
I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.
I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.
I will add below line to address this.
drivers/iommu/intel/pasid.c +0 -0
@@ -932,6 +932,7 @@ void intel_context_flush_present(struct device_domain_info *info, did =pasid_get_domain_id (pte); qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);
iommu->flush.flush_iotlb(iommu, did, 0, 0,
DMA_TLB_DSI_FLUSH);
+ pasid_cache_invalidation_with_pasid(iommu, did, i);
}
}Thanks!
Best regards,
baolu
Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
In reply to Lu BaoluQuoted 8 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3
-0+if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might
(quoted diff) +1
-0+ * be potential PASID translations, the caller shouldhold
(quoted diff) +1
-0+ * the lock to ensure that context changes and cacheflushes
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932
Quoted 7 lines
drivers/iommu/intel/pasid.c +7
-0+* are atomic. +*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ;
Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.
Review
Quoted 7 lines
(quoted diff) +2
-0+ did = pasid_get_domain_id(pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);(quoted diff) +1
-0+ iommu->flush.flush_iotlb(iommu, did, 0, 0,
DMA_TLB_DSI_FLUSH);(quoted diff) +2
-0+ } + }
Thanks,
Jacob
Baolu Lu
maintainerRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3
-0+if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might
(quoted diff) +1
-0+ * be potential PASID translations, the caller shouldhold
(quoted diff) +1
-0+ * the lock to ensure that context changes and cacheflushes
drivers/iommu/intel/pasid.c +1
-0+* are atomic.
Quoted 8 lines
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932
Quoted 6 lines
drivers/iommu/intel/pasid.c +6
-0+*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ;Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.
Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.
Best regards,
baolu
Baolu Lu
maintainerRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
In reply to Baolu LuQuoted 53 lines · Baolu Lu
Quoted 36 lines · Baolu Lu
Quoted 26 lines · Kevin
Quoted 26 lines
From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM(quoted diff) +22
-0+ + /* + * For scalable mode: + * - Domain-selective PASID-cache invalidation to affected domains + * - Domain-selective IOTLB invalidation to affected domains + * - Global Device-TLB invalidation to affected functions + */ + if (flush_domains) { + /* + * If the IOMMU is running in scalable mode and there might + * be potential PASID translations, the caller should hold + * the lock to ensure that context changes and cache flushes + * are atomic. + */ + assert_spin_locked(&iommu->lock); + for (i = 0; i < info->pasid_table->max_pasid; i++) { + pte = intel_pasid_get_entry(info->dev, i); + if (!pte || !pasid_pte_is_present(pte)) + continue; + + did = pasid_get_domain_id(pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);
+ iommu->flush.flush_iotlb(iommu, did, 0, 0,Quoted 8 lines
Review
Quoted 6 lines
DMA_TLB_DSI_FLUSH);
+ }
+ }(quoted diff) +3
-0+ + __context_flush_dev_iotlb(info); +}this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.
Quoted 3 lines
I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.
I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.
I will add below line to address this.
drivers/iommu/intel/pasid.c +0
-0
drivers/iommu/intel/pasid.c void intel_context_flush_present(struct · line 938
Quoted 7 lines
drivers/iommu/intel/pasid.c +1
-0@@ -932,6 +932,7 @@ void intel_context_flush_present(structdevice_domain_info *info, did = pasid_get_domain_id(pte); qi_flush_pasid_cache(iommu, did,QI_PC_ALL_PASIDS ,0 ); iommu->flush.flush_iotlb(iommu, did, 0, 0,DMA_TLB_DSI_FLUSH ); + pasid_cache_invalidation_with_pasid(iommu, did, i);
Should be
devtlb_invalidation_with_pasid(iommu, info->dev, i);
Sorry for the typo.
Best regards,
baolu
Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 42 lines · Baolu Lu
Quoted 41 lines · Baolu Lu
Quoted 39 lines · Baolu Lu
Quoted 26 lines · Kevin
Quoted 26 lines
From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM(quoted diff) +22
-0+ + /* + * For scalable mode: + * - Domain-selective PASID-cache invalidation to affected domains + * - Domain-selective IOTLB invalidation to affected domains + * - Global Device-TLB invalidation to affected functions + */ + if (flush_domains) { + /* + * If the IOMMU is running in scalable mode and there might + * be potential PASID translations, the caller should hold + * the lock to ensure that context changes and cache flushes + * are atomic. + */ + assert_spin_locked(&iommu->lock); + for (i = 0; i < info->pasid_table->max_pasid; i++) { + pte = intel_pasid_get_entry(info->dev, i); + if (!pte || !pasid_pte_is_present(pte)) + continue; + + did = pasid_get_domain_id(pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);
+ iommu->flush.flush_iotlb(iommu, did, 0, 0,Quoted 8 lines
Review
Quoted 6 lines
DMA_TLB_DSI_FLUSH);
+ }
+ }(quoted diff) +3
-0+ + __context_flush_dev_iotlb(info); +}this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation with global bit set.
I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.
Quoted 5 lines
Quoted 5 lines
I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.
hmmm. "Table 25. Guidance to Software for Invalidations" only mentions global devTLB invalidation. 10.3.8 PASID and Global Invalidate of PCIe 6.2 spec has below definition.
For Invalidation Requests that do not have a PASID, the ATC shall:
• Invalidate ATC entries within the indicate memory range that were requested without a PASID value.
• Invalidate ATC entries at all addresses that were requested with any PASID value.
AFAIK. A devTLB invalidate descriptor submitted to VT-d should be converted
to be a PCIe ATC invalidation request without PASID prefix. So it may be
subjected to the above definition. If so, no need to have a PASID-based devTLB invalidation descriptor.
Quoted 22 lines
drivers/iommu/intel/pasid.c void intel_context_flush_present(struct · line 936
Quoted 12 lines
I will add below line to address this.
drivers/iommu/intel/pasid.c +1
-0@@ -932,6 +932,7 @@ void intel_context_flush_present(structdevice_domain_info *info, did = pasid_get_domain_id(pte); qi_flush_pasid_cache(iommu, did, QI_PC_ALL_PASIDS, 0); iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); + pasid_cache_invalidation_with_pasid(iommu, did, i);Should be
devtlb_invalidation_with_pasid(iommu,
info->dev, i);Sorry for the typo.
Best regards,
baolu
Signature · 2 lines
Regards, Yi Liu
RE: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 63 lines
From: Liu, Yi L <[email protected]>
Sent: Tuesday, July 2, 2024 2:25 PMQuoted 43 lines · Baolu Lu
Quoted 42 lines · Baolu Lu
Quoted 40 lines · Baolu Lu
Quoted 27 lines · Kevin
Quoted 27 lines
From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM(quoted diff) +22
-0+ + /* + * For scalable mode: + * - Domain-selective PASID-cache invalidation to affected domains + * - Domain-selective IOTLB invalidation to affected domains + * - Global Device-TLB invalidation to affected functions + */ + if (flush_domains) { + /* + * If the IOMMU is running in scalable mode and there might + * be potential PASID translations, the caller should hold + * the lock to ensure that context changes and cache flushes + * are atomic. + */ + assert_spin_locked(&iommu->lock); + for (i = 0; i < info->pasid_table->max_pasid; i++) { + pte = intel_pasid_get_entry(info->dev, i); + if (!pte || !pasid_pte_is_present(pte)) + continue; + + did = pasid_get_domain_id(pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);
+iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH);Quoted 8 lines
Review
Quoted 5 lines
+ }
+ }(quoted diff) +3
-0+ + __context_flush_dev_iotlb(info); +}this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation
with global bit set.I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.
Quoted 5 lines
Quoted 5 lines
I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.
hmmm. "Table 25. Guidance to Software for Invalidations" only mentions global devTLB invalidation. 10.3.8 PASID and Global Invalidate of PCIe 6.2 spec has below definition.
For Invalidation Requests that do not have a PASID, the ATC shall:
• Invalidate ATC entries within the indicate memory range that were requested without a PASID value.
• Invalidate ATC entries at all addresses that were requested with any PASID value.
Quoted 5 lines
AFAIK. A devTLB invalidate descriptor submitted to VT-d should be converted to be a PCIe ATC invalidation request without PASID prefix. So it may be subjected to the above definition. If so, no need to have a PASID-based devTLB invalidation descriptor.
You are correct. The wording in VT-d spec is a bit confusing on saying that devtlb invalidation descriptor is only for request w/o PASID. 😉
Baolu Lu
maintainerRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 63 lines · Kevin
Quoted 63 lines
From: Liu, Yi L <[email protected]>
Sent: Tuesday, July 2, 2024 2:25 PMQuoted 43 lines · Baolu Lu
Quoted 42 lines · Baolu Lu
Quoted 40 lines · Baolu Lu
Quoted 27 lines · Kevin
Quoted 27 lines
From: Lu Baolu<[email protected]>
Sent: Monday, July 1, 2024 7:23 PM(quoted diff) +22
-0+ + /* + * For scalable mode: + * - Domain-selective PASID-cache invalidation to affected domains + * - Domain-selective IOTLB invalidation to affected domains + * - Global Device-TLB invalidation to affected functions + */ + if (flush_domains) { + /* + * If the IOMMU is running in scalable mode and there might + * be potential PASID translations, the caller should hold + * the lock to ensure that context changes and cache flushes + * are atomic. + */ + assert_spin_locked(&iommu->lock); + for (i = 0; i < info->pasid_table->max_pasid; i++) { + pte = intel_pasid_get_entry(info->dev, i); + if (!pte || !pasid_pte_is_present(pte)) + continue; + + did = pasid_get_domain_id(pte); + qi_flush_pasid_cache(iommu, did,
QI_PC_ALL_PASIDS, 0);
+iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH);Quoted 8 lines
Review
Quoted 5 lines
+ }
+ }(quoted diff) +3
-0+ + __context_flush_dev_iotlb(info); +}this only invalidates devtlb w/o PASID. We miss a pasid devtlb invalidation
with global bit set.I am not sure about this. The spec says "Global Device-TLB invalidation to affected functions", I am not sure whether this implies any PASID- based-Device-TLB invalidation.
Quoted 5 lines
Quoted 5 lines
I just revisited the spec, Device-TLB invalidation only covers caches for requests-without-PASID. If pasid translation is affected while updating the context entry, we should also take care of the caches for requests-with-pasid.
hmmm. "Table 25. Guidance to Software for Invalidations" only mentions global devTLB invalidation. 10.3.8 PASID and Global Invalidate of PCIe 6.2 spec has below definition.
For Invalidation Requests that do not have a PASID, the ATC shall:
• Invalidate ATC entries within the indicate memory range that were requested without a PASID value.
• Invalidate ATC entries at all addresses that were requested with any PASID value.
Quoted 8 lines
Quoted 5 lines
AFAIK. A devTLB invalidate descriptor submitted to VT-d should be converted to be a PCIe ATC invalidation request without PASID prefix. So it may be subjected to the above definition. If so, no need to have a PASID-based devTLB invalidation descriptor.
You are correct. The wording in VT-d spec is a bit confusing on saying that devtlb invalidation descriptor is only for request w/o PASID. 😉
Okay, so I will remove the line of pasid-based-dev-iotlb invalidation.
Thank you Yi, for the clarification.
Best regards,
baolu
Re: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
In reply to Baolu LuQuoted 22 lines · Baolu Lu
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3
-0+if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might
(quoted diff) +1
-0+ * be potential PASID translations, the caller shouldhold
(quoted diff) +1
-0+ * the lock to ensure that context changes and cacheflushes
drivers/iommu/intel/pasid.c +1
-0+* are atomic. Quoted 8 lines
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932
Quoted 6 lines
drivers/iommu/intel/pasid.c +6
-0+*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ;Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.
Quoted 2 lines
Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.
I think it is necessary, because without tracking domain IDs, the code above would have duplicated invalidations.
For example: a device PASID table has the following entries
PASID DomainID
-------------------------
100 1 200 1 300 2
-------------------------
When a present context entry changes, we need to do:qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);
With this code, we doqi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0); qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);//duplicated! qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);
Thanks,
Jacob
Baolu Lu
maintainerRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 39 lines · Jacob Pan
Quoted 21 lines · Baolu Lu
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3
-0+if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might
(quoted diff) +1
-0+ * be potential PASID translations, the caller shouldhold
(quoted diff) +1
-0+ * the lock to ensure that context changes and cacheflushes
drivers/iommu/intel/pasid.c +1
-0+* are atomic. Quoted 8 lines
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932
Quoted 6 lines
drivers/iommu/intel/pasid.c +6
-0+*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ;Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.
Quoted 2 lines
Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.
I think it is necessary, because without tracking domain IDs, the code above would have duplicated invalidations.
For example: a device PASID table has the following entries
PASID DomainID
-------------------------100 1 200 1 300 2-------------------------
When a present context entry changes, we need to do:qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);
Quoted 4 lines
With this code, we do
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);//duplicated!qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);
Yes, this is likely. But currently enabling and disabling PRI happens in driver's probe and release paths. Therefore such duplicate is not so critical.
For long term, I have a plan to abstract the domain id into an object so that domains attached to different PASIDs of a device could share a domain id. With that done, we could improve this code by iterating the domain id objects for a device and performing cache invalidation directly.
Thanks,
baolu
Jacob Pan
Reviewed-byRe: [PATCH v3 1/2] iommu/vt-d: Add helper to flush caches for context change
Quoted 49 lines · Baolu Lu
Quoted 39 lines · Jacob Pan
Quoted 21 lines · Baolu Lu
Quoted 12 lines · Jacob Pan
Quoted 9 lines · Lu Baolu
drivers/iommu/intel/pasid.c +3
-0+if (flush_domains) { +/* +* If the IOMMU is running in scalable mode and there might
(quoted diff) +1
-0+ * be potential PASID translations, the callershould hold
(quoted diff) +1
-0+ * the lock to ensure that context changes and cacheflushes
drivers/iommu/intel/pasid.c +1
-0+* are atomic. Quoted 8 lines
drivers/iommu/intel/pasid.c int intel_pasid_setup_sm_context(struct device *dev) · line 932
Quoted 6 lines
drivers/iommu/intel/pasid.c +6
-0+*/ +assert_spin_locked (&iommu->lock); +for (i =0 ; i < info->pasid_table->max_pasid; i++) { + pte =intel_pasid_get_entry (info->dev, i); +if (!pte || !pasid_pte_is_present (pte)) +continue ;Is it worth going through 1M PASIDs just to skip the PASID cache invalidation? Or just do the flush on all used DIDs unconditionally.
Quoted 2 lines
Currently we don't track all domains attached to a device. If such optimization is necessary, perhaps we can add it later.
I think it is necessary, because without tracking domain IDs, the code above would have duplicated invalidations.
For example: a device PASID table has the following entries
PASID DomainID
-------------------------100 1 200 1 300 2-------------------------
When a present context entry changes, we need to do:qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);Quoted 4 lines
With this code, we do
qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);qi_flush_pasid_cache(iommu, 1, QI_PC_ALL_PASIDS, 0);//duplicated!qi_flush_pasid_cache(iommu, 2, QI_PC_ALL_PASIDS, 0);Yes, this is likely. But currently enabling and disabling PRI happens in driver's probe and release paths. Therefore such duplicate is not so critical.
Quoted 5 lines
For long term, I have a plan to abstract the domain id into an object so that domains attached to different PASIDs of a device could share a domain id. With that done, we could improve this code by iterating the domain id objects for a device and performing cache invalidation directly.
Sounds good. It might be helpful to add a comment to clarify for others who might wonder about the duplicates.
Thanks,
Jacob