Intel IOMMU updates for v7.2
Subsystems (2)
Lu Baolu
maintainerHi Joerg,
The following changes are queued for v7.2-rc1. This update introduces a few quick fixes and several refactorings, including:
- Add the PCI segment number to DMA fault messages.
- Improve support for non-PRI mode SVA.
- Ensure atomicity during context entry teardown.
- Fix RB-tree corruption in the probe error path.
These patches are based on v7.1-rc6. Please consider pulling them for the iommu/vt-d branch.
Best regards,
baolu
Change log:
v2:
- Dropped "iommu/vt-d: Fix Use-After-Free in probe error path".
v1: https://lore.kernel.org/linux-iommu/[email protected]/
Guanghui Feng (1):
iommu/vt-d: Improve IOMMU fault information
Lu Baolu (1):
iommu/vt-d: Avoid WARNING in sva unbind path
Michael Bommarito (1):
iommu/vt-d: Clear Present bit before tearing down scalable-mode
context entry
Michał Grzelak (1):
iommu/vt-d: Remove typo from pasid_pte_config_nested()
Pranjal Shrivastava (1):
iommu/vt-d: Fix RB-tree corruption in probe error path
drivers/iommu/intel/iommu.h | 11 +++++++++++ drivers/iommu/intel/dmar.c | 9 ++++++--- drivers/iommu/intel/iommu.c | 6 +++++- drivers/iommu/intel/pasid.c | 16 +++++++++------- drivers/iommu/intel/svm.c | 12 ++++-------- 5 files changed, 35 insertions(+), 19 deletions(-)
Signature · 1 line
2.43.0
Lu Baolu
maintainerPATCH v2 1/5 iommu/vt-d: Avoid WARNING in sva unbind path
The Intel IOMMU driver allows SVA on devices even if they do not support PCI/PRI. Commit 39c20c4e83b9 ("iommu/vt-d: Only handle IOPF for SVA when PRI is supported") modified the SVA bind path to allow this configuration by skipping IOPF enablement when PRI is missing. However, it failed to update the unbind path.
This creates an imbalance: the unbind path attempts to disable IOPF for a device that never had it enabled, triggering a WARNING in intel_iommu_disable_iopf():
WARNING: drivers/iommu/intel/iommu.c:3475 at intel_iommu_disable_iopf+0x4f/0x90d Call Trace: <TASK> blocking_domain_set_dev_pasid+0x50/0x70 iommu_detach_device_pasid+0x89/0xc0 iommu_sva_unbind_device+0x73/0x150 xe_vm_close_and_put+0x4d2/0x1200 [xe]
Fix this by bypassing IOPF operations for SVA domains on non-PRI hardware in both the bind and unbind paths.
--- drivers/iommu/intel/iommu.h | 11 +++++++++++ drivers/iommu/intel/svm.c | 12 ++++-------- 2 files changed, 15 insertions(+), 8 deletions(-)
drivers/iommu/intel/iommu.h +11 -0
@@ -1254,18 +1254,29 @@ void intel_iommu_disable_iopf(struct device *dev);static inline int iopf_for_domain_set (struct iommu_domain *domain,struct device *dev) { +struct device_domain_info *info =dev_iommu_priv_get (dev); +if (!domain || !domain->iopf_handler)return 0 ; +/* SVA with non-IOMMU/PRI IOPF handling is allowed. */ +if (domain->type ==IOMMU_DOMAIN_SVA && !info->pri_supported) +return 0 ; +return intel_iommu_enable_iopf (dev); }static inline void iopf_for_domain_remove (struct iommu_domain *domain,struct device *dev) { +struct device_domain_info *info =dev_iommu_priv_get (dev); +if (!domain || !domain->iopf_handler)return ; +if (domain->type ==IOMMU_DOMAIN_SVA && !info->pri_supported) +return ; +intel_iommu_disable_iopf (dev); }
drivers/iommu/intel/svm.c +4 -8
@@ -164,12 +164,9 @@ static int intel_svm_set_dev_pasid(struct iommu_domain *domain,if (IS_ERR (dev_pasid))return PTR_ERR (dev_pasid); -/* SVA with non-IOMMU/PRI IOPF handling is allowed. */ -if (info->pri_supported) { - ret =iopf_for_domain_replace (domain, old, dev); -if (ret) - goto out_remove_dev_pasid; - } + ret =iopf_for_domain_replace (domain, old, dev); +if (ret) + goto out_remove_dev_pasid;/* Setup the pasid table: */ sflags =cpu_feature_enabled (X86_FEATURE_LA57 ) ?PASID_FLAG_FL5LP :0 ;
@@ -184,8 +181,7 @@ static int intel_svm_set_dev_pasid(struct iommu_domain *domain,return 0 ; out_unwind_iopf: -if (info->pri_supported) -iopf_for_domain_replace (old, domain, dev); +iopf_for_domain_replace (old, domain, dev); out_remove_dev_pasid:domain_remove_dev_pasid (domain, dev, pasid);return ret;
Signature · 1 line
2.43.0
Lu Baolu
maintainerPATCH v2 2/5 iommu/vt-d: Clear Present bit before tearing down scalable-mode context entry
In reply to Lu BaoluFrom: Michael Bommarito <[email protected]>
device_pasid_table_teardown() zeroes the 128-bit scalable-mode context entry with context_clear_entry() while the Present bit is still set. This creates a window where the hardware can fetch a torn entry, with some fields already zeroed while Present is still set, leading to unpredictable behavior or spurious faults. The context-cache invalidation is issued only after the entry has been zeroed, and intel_pasid_free_table() then frees the PASID directory pages, so the IOMMU can keep walking a stale Present=1 entry that points at freed memory.
While x86 provides strong write ordering, the compiler may reorder the two 64-bit writes to the entry, and the hardware fetch is not guaranteed to be atomic with respect to multiple CPU writes.
Commit c1e4f1dccbe9d ("iommu/vt-d: Clear Present bit before tearing down context entry") fixed this exact pattern in domain_context_clear_one() and the copied-context path, but device_pasid_table_teardown() was not converted.
Align it with the "Guidance to Software for Invalidations" in the VT-d spec, Section 6.5.3.3, using the same ownership handshake as the sibling
fix: clear only the Present bit, flush it to the IOMMU, perform the context-cache invalidation, and only then zero the rest of the entry.
--- drivers/iommu/intel/pasid.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/iommu/intel/pasid.c +3 -1
@@ -748,10 +748,12 @@ static void device_pasid_table_teardown(struct device *dev, u8 bus, u8 devfn) } did =context_domain_id (context); -context_clear_entry (context); +context_clear_present (context);__iommu_flush_cache (iommu, context,sizeof (*context));spin_unlock (&iommu->lock);intel_context_flush_no_pasid (info, context, did); +context_clear_entry (context); +__iommu_flush_cache (iommu, context,sizeof (*context)); }static int pci_pasid_table_teardown (struct pci_dev *pdev,u16 alias,void *data)
Signature · 1 line
2.43.0
Lu Baolu
maintainerPATCH v2 3/5 iommu/vt-d: Remove typo from pasid_pte_config_nested()
In reply to Lu BaoluFrom: Michał Grzelak <[email protected]>
Rename pasid_pte_config_nestd() into pasid_pte_config_nested(). Do it to match other function names ending with _nested().
--- drivers/iommu/intel/pasid.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/iommu/intel/pasid.c +6 -6
@@ -618,11 +618,11 @@ void intel_pasid_setup_page_snoop_control(struct intel_iommu *iommu,intel_pasid_flush_present (iommu, dev, pasid, did, pte); } -static void pasid_pte_config_nestd (struct intel_iommu *iommu, -struct pasid_entry *pte, -struct iommu_hwpt_vtd_s1 *s1_cfg, -struct dmar_domain *s2_domain, -u16 did) +static void pasid_pte_config_nested (struct intel_iommu *iommu, +struct pasid_entry *pte, +struct iommu_hwpt_vtd_s1 *s1_cfg, +struct dmar_domain *s2_domain, +u16 did) {struct pt_iommu_vtdss_hw_info pt_info;
@@ -720,7 +720,7 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu, struct device *dev,return -EBUSY ; } -pasid_pte_config_nestd (iommu, pte, s1_cfg, s2_domain, did); +pasid_pte_config_nested (iommu, pte, s1_cfg, s2_domain, did);spin_unlock (&iommu->lock);pasid_flush_caches (iommu, pte, pasid, did);
Signature · 1 line
2.43.0
Lu Baolu
maintainerPATCH v2 4/5 iommu/vt-d: Improve IOMMU fault information
In reply to Lu BaoluFrom: Guanghui Feng <[email protected]>
In some environments, multiple PCIe segments exist, and PCIe device information needs to be differentiated and identified based on the segment. When an IOMMU fault event occurs, the IOMMU and device segment information should be output in detail in dmar_fault_do_one.
--- drivers/iommu/intel/dmar.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
drivers/iommu/intel/dmar.c +6 -3
@@ -1894,7 +1894,8 @@ static int dmar_fault_do_one(struct intel_iommu *iommu, int type, reason =dmar_get_fault_reason (fault_reason, &fault_type);if (fault_type ==INTR_REMAP ) { -pr_err ("[INTR-REMAP] Request device [%02x:%02x.%d] fault index 0x%llx [fault reason 0x%02x] %s\n" , +pr_err ("[INTR-REMAP] Request device [%04x:%02x:%02x.%d] fault index 0x%llx [fault reason 0x%02x] %s\n" , + iommu->segment, source_id >>8 ,PCI_SLOT (source_id &0xFF ),PCI_FUNC (source_id &0xFF ), addr >>48 , fault_reason, reason);
@@ -1903,14 +1904,16 @@ static int dmar_fault_do_one(struct intel_iommu *iommu, int type, }if (pasid ==IOMMU_PASID_INVALID ) -pr_err ("[%s NO_PASID] Request device [%02x:%02x.%d] fault addr 0x%llx [fault reason 0x%02x] %s\n" , +pr_err ("[%s NO_PASID] Request device [%04x:%02x:%02x.%d] fault addr 0x%llx [fault reason 0x%02x] %s\n" , type ?"DMA Read" :"DMA Write" , + iommu->segment, source_id >>8 ,PCI_SLOT (source_id &0xFF ),PCI_FUNC (source_id &0xFF ), addr, fault_reason, reason);else -pr_err ("[%s PASID 0x%x] Request device [%02x:%02x.%d] fault addr 0x%llx [fault reason 0x%02x] %s\n" , +pr_err ("[%s PASID 0x%x] Request device [%04x:%02x:%02x.%d] fault addr 0x%llx [fault reason 0x%02x] %s\n" , type ?"DMA Read" :"DMA Write" , pasid, + iommu->segment, source_id >>8 ,PCI_SLOT (source_id &0xFF ),PCI_FUNC (source_id &0xFF ), addr, fault_reason, reason);
Signature · 1 line
2.43.0
Lu Baolu
maintainerPATCH v2 5/5 iommu/vt-d: Fix RB-tree corruption in probe error path
In reply to Lu BaoluFrom: Pranjal Shrivastava <[email protected]>
The info->node RB-tree member is zero-initialized via kzalloc. If a device does not support ATS, the device_rbtree_insert() call is skipped. If a subsequent probe step fails, the error path jumps to device_rbtree_remove(), which misinterprets the zeroed node as a tree root and corrupts the device RB-tree.
Fix this by explicitly initializing the RB-node as empty using RB_CLEAR_NODE() during initialization and guarding the removal with RB_EMPTY_NODE().
--- drivers/iommu/intel/iommu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/iommu/intel/iommu.c +5 -1
@@ -157,7 +157,10 @@ static void device_rbtree_remove(struct device_domain_info *info)unsigned long flags;spin_lock_irqsave (&iommu->device_rbtree_lock, flags); -rb_erase (&info->node, &iommu->device_rbtree); +if (!RB_EMPTY_NODE (&info->node)) { +rb_erase (&info->node, &iommu->device_rbtree); +RB_CLEAR_NODE (&info->node); + }spin_unlock_irqrestore (&iommu->device_rbtree_lock, flags); }
@@ -3254,6 +3257,7 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev) info->dev = dev; info->iommu = iommu; +RB_CLEAR_NODE (&info->node);if (dev_is_pci (dev)) { if (ecap_dev_iotlb_support(iommu->ecap) && pci_ats_supported(pdev) &&
Signature · 1 line
2.43.0
Quoted 16 lines · Lu Baolu
Guanghui Feng (1):
iommu/vt-d: Improve IOMMU fault informationLu Baolu (1):
iommu/vt-d: Avoid WARNING in sva unbind pathMichael Bommarito (1):
iommu/vt-d: Clear Present bit before tearing down scalable-mode
context entryMichał Grzelak (1):
iommu/vt-d: Remove typo frompasid_pte_config_nested()Pranjal Shrivastava (1):
iommu/vt-d: Fix RB-tree corruption in probe error path
Quoted 6 lines
drivers/iommu/intel/iommu.h | 11 +++++++++++ drivers/iommu/intel/dmar.c | 9 ++++++--- drivers/iommu/intel/iommu.c | 6 +++++- drivers/iommu/intel/pasid.c | 16 +++++++++------- drivers/iommu/intel/svm.c | 12 ++++-------- 5 files changed, 35 insertions(+), 19 deletions(-)
Applied, thanks.