Skip to contentSkip to conversation

VT-d Native Shared virtual memory cleanup and fixes

PATCH Superseded Jacob Pan 2 participants 23 messages Nov 15, 2019 – Nov 18, 2019 IOMMU

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Jacob Pan

author
#0 lore raw

cover VT-d Native Shared virtual memory cleanup and fixes

Mostly extracted from nested SVA/SVM series based on review comments of v7.
https://lkml.org/lkml/2019/10/24/852

This series also adds a few important fixes for native use of SVA. Nested SVA new code will be submitted separately as a smaller set. Based on the core branch of IOMMU tree staged for v5.5, where common APIs for vSVA were applied.
git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git core

Thanks,

Jacob

Jacob Pan (10):
iommu/vt-d: Introduce native SVM capable flag iommu/vt-d: Fix CPU and IOMMU SVM feature matching checks iommu/vt-d: Reject SVM bind for failed capability check iommu/vt-d: Match CPU and IOMMU paging mode iommu/vt-d: Avoid duplicated code for PASID setup iommu/vt-d: Fix off-by-one in PASID allocation iommu/vt-d: Replace Intel specific PASID allocator with IOASID iommu/vt-d: Fix PASID cache flush
iommu/vt-d: Avoid sending invalid page response iommu/vt-d: Misc macro clean up for SVM

 drivers/iommu/Kconfig       |   1 +
 drivers/iommu/intel-iommu.c |  22 +++---
 drivers/iommu/intel-pasid.c |  99 ++++++++------------------
 drivers/iommu/intel-svm.c   | 164 +++++++++++++++++++++++++-------------------
 include/linux/intel-iommu.h |   5 +-
 5 files changed, 137 insertions(+), 154 deletions(-)
Signature · 1 line
2.7.4
author
#1 lore raw

PATCH 1/10 iommu/vt-d: Introduce native SVM capable flag

Shared Virtual Memory(SVM) is based on a collective set of hardware features detected at runtime. There are requirements for matching CPU and IOMMU capabilities.

This patch introduces a flag which will be used to mark and test the capability of SVM.

---
 include/linux/intel-iommu.h | 1 +
 1 file changed, 1 insertion(+)
include/linux/intel-iommu.h +1 -0
index ed11ef594378..63118991824c 100644
@@ -433,6 +433,7 @@ enum {
 
 #define VTD_FLAG_TRANS_PRE_ENABLED	(1 << 0)
 #define VTD_FLAG_IRQ_REMAP_PRE_ENABLED	(1 << 1)
+#define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
 extern int intel_iommu_sm;
 
Signature · 1 line
2.7.4
author
#3 lore raw

PATCH 2/10 iommu/vt-d: Fix CPU and IOMMU SVM feature matching checks

In reply to Jacob Pan

The current code checks CPU and IOMMU feature set for SVM support but the result is never stored nor used. Therefore, SVM can still be used even when these checks failed.

This patch consolidates code for checking PASID, CPU vs. IOMMU paging mode compatibility, as well as provides specific error messages for each failed checks.

---
 drivers/iommu/intel-iommu.c | 10 ++--------
 drivers/iommu/intel-svm.c   | 40 +++++++++++++++++++++++++++-------------
 include/linux/intel-iommu.h |  4 +++-
 3 files changed, 32 insertions(+), 22 deletions(-)
drivers/iommu/intel-iommu.c +2 -8
index 3f974919d3bd..d598168e410d 100644
@@ -3289,10 +3289,7 @@ static int __init init_dmars(void)
 
 		if (!ecap_pass_through(iommu->ecap))
 			hw_pass_through = 0;
-#ifdef CONFIG_INTEL_IOMMU_SVM
-		if (pasid_supported(iommu))
-			intel_svm_init(iommu);
-#endif
+		intel_svm_check(iommu);
 	}
 
 	/*
@@ -4471,10 +4468,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
 	if (ret)
 		goto out;
 
-#ifdef CONFIG_INTEL_IOMMU_SVM
-	if (pasid_supported(iommu))
-		intel_svm_init(iommu);
-#endif
+	intel_svm_check(iommu);
 
 	if (dmaru->ignored) {
 		/*
drivers/iommu/intel-svm.c +27 -13
index 9b159132405d..e9773d714862 100644
@@ -23,19 +23,6 @@
 
 static irqreturn_t prq_event_thread(int irq, void *d);
 
-int intel_svm_init(struct intel_iommu *iommu)
-{
-	if (cpu_feature_enabled(X86_FEATURE_GBPAGES) &&
-			!cap_fl1gp_support(iommu->cap))
-		return -EINVAL;
-
-	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
-			!cap_5lp_support(iommu->cap))
-		return -EINVAL;
-
-	return 0;
-}
-
 #define PRQ_ORDER 0
 
 int intel_svm_enable_prq(struct intel_iommu *iommu)
@@ -99,6 +86,33 @@ int intel_svm_finish_prq(struct intel_iommu *iommu)
 	return 0;
 }
 
+static inline bool intel_svm_capable(struct intel_iommu *iommu)
+{
+	return iommu->flags & VTD_FLAG_SVM_CAPABLE;
+}
+
+void intel_svm_check(struct intel_iommu *iommu)
+{
+	if (!pasid_supported(iommu))
+		return;
+
+	if (cpu_feature_enabled(X86_FEATURE_GBPAGES) &&
+	    !cap_fl1gp_support(iommu->cap)) {
+		pr_err("%s SVM disabled, incompatible 1GB page capability\n",
+			iommu->name);
+		return;
+	}
+
+	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("%s SVM disabled, incompatible paging mode\n",
+			iommu->name);
+		return;
+	}
+
+	iommu->flags |= VTD_FLAG_SVM_CAPABLE;
+}
+
 static void intel_flush_svm_range_dev (struct intel_svm *svm, struct intel_svm_dev *sdev,
 				unsigned long address, unsigned long pages, int ih)
 {
include/linux/intel-iommu.h +3 -1
index 63118991824c..7dcfa1c4a844 100644
@@ -657,7 +657,7 @@ void iommu_flush_write_buffer(struct intel_iommu *iommu);
 int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev);
 
 #ifdef CONFIG_INTEL_IOMMU_SVM
-int intel_svm_init(struct intel_iommu *iommu);
+extern void intel_svm_check(struct intel_iommu *iommu);
 extern int intel_svm_enable_prq(struct intel_iommu *iommu);
 extern int intel_svm_finish_prq(struct intel_iommu *iommu);
 
@@ -685,6 +685,8 @@ struct intel_svm {
 };
 
 extern struct intel_iommu *intel_svm_device_to_iommu(struct device *dev);
+#else
+static inline void intel_svm_check(struct intel_iommu *iommu) {}
 #endif
 
 #ifdef CONFIG_INTEL_IOMMU_DEBUGFS
Signature · 1 line
2.7.4
author
#7 lore raw

PATCH 4/10 iommu/vt-d: Match CPU and IOMMU paging mode

In reply to Jacob Pan

When setting up first level page tables for sharing with CPU, we need to ensure IOMMU can support no less than the levels supported by the CPU.
It is not adequate, as in the current code, to set up 5-level paging in PASID entry First Level Paging Mode(FLPM) solely based on CPU.

Fixes 437f35e1cd4c8 ("iommu/vt-d: Add first level page table

interface")

---
 drivers/iommu/intel-pasid.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
drivers/iommu/intel-pasid.c +10 -2
index 040a445be300..e7cb0b8a7332 100644
@@ -499,8 +499,16 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
 	}
 
 #ifdef CONFIG_X86
-	if (cpu_feature_enabled(X86_FEATURE_LA57))
-		pasid_set_flpm(pte, 1);
+	/* Both CPU and IOMMU paging mode need to match */
+	if (cpu_feature_enabled(X86_FEATURE_LA57)) {
+		if (cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);
+		} else {
+			pr_err("VT-d has no 5-level paging support for CPU\n");
+			pasid_clear_entry(pte);
+			return -EINVAL;
+		}
+	}
 #endif /* CONFIG_X86 */
 
 	pasid_set_domain_id(pte, did);
Signature · 1 line
2.7.4
author
#5 lore raw

PATCH 3/10 iommu/vt-d: Reject SVM bind for failed capability check

In reply to Jacob Pan

Add a check during SVM bind to ensure CPU and IOMMU hardware capabilities are met.

---
 drivers/iommu/intel-svm.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/iommu/intel-svm.c +3 -0
index e9773d714862..924a4de84be1 100644
@@ -238,6 +238,9 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 	if (!iommu || dmar_disabled)
 		return -EINVAL;
 
+	if (!intel_svm_capable(iommu))
+		return -ENOTSUPP;
+
 	if (dev_is_pci(dev)) {
 		pasid_max = pci_max_pasids(to_pci_dev(dev));
 		if (pasid_max < 0)
Signature · 1 line
2.7.4
author
#9 lore raw

PATCH 5/10 iommu/vt-d: Avoid duplicated code for PASID setup

In reply to Jacob Pan

After each setup for PASID entry, related translation caches must be flushed. We can combine duplicated code into one function which is less error prone.

---
 drivers/iommu/intel-pasid.c | 48 +++++++++++++++++----------------------------
 1 file changed, 18 insertions(+), 30 deletions(-)
drivers/iommu/intel-pasid.c +18 -30
index e7cb0b8a7332..732bfee228df 100644
@@ -465,6 +465,21 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 		devtlb_invalidation_with_pasid(iommu, dev, pasid);
 }
 
+static void pasid_flush_caches(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				int pasid, u16 did)
+{
+	if (!ecap_coherent(iommu->ecap))
+		clflush_cache_range(pte, sizeof(*pte));
+
+	if (cap_caching_mode(iommu->cap)) {
+		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
+		iotlb_invalidation_with_pasid(iommu, did, pasid);
+	} else {
+		iommu_flush_write_buffer(iommu);
+	}
+}
+
 /*
  * Set up the scalable mode pasid table entry for first only
  * translation type.
@@ -518,16 +533,7 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
 	/* Setup Present and PASID Granular Transfer Type: */
 	pasid_set_translation_type(pte, 1);
 	pasid_set_present(pte);
-
-	if (!ecap_coherent(iommu->ecap))
-		clflush_cache_range(pte, sizeof(*pte));
-
-	if (cap_caching_mode(iommu->cap)) {
-		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
-		iotlb_invalidation_with_pasid(iommu, did, pasid);
-	} else {
-		iommu_flush_write_buffer(iommu);
-	}
+	pasid_flush_caches(iommu, pte, pasid, did);
 
 	return 0;
 }
@@ -591,16 +597,7 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
 	 */
 	pasid_set_sre(pte);
 	pasid_set_present(pte);
-
-	if (!ecap_coherent(iommu->ecap))
-		clflush_cache_range(pte, sizeof(*pte));
-
-	if (cap_caching_mode(iommu->cap)) {
-		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
-		iotlb_invalidation_with_pasid(iommu, did, pasid);
-	} else {
-		iommu_flush_write_buffer(iommu);
-	}
+	pasid_flush_caches(iommu, pte, pasid, did);
 
 	return 0;
 }
@@ -634,16 +631,7 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 	 */
 	pasid_set_sre(pte);
 	pasid_set_present(pte);
-
-	if (!ecap_coherent(iommu->ecap))
-		clflush_cache_range(pte, sizeof(*pte));
-
-	if (cap_caching_mode(iommu->cap)) {
-		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
-		iotlb_invalidation_with_pasid(iommu, did, pasid);
-	} else {
-		iommu_flush_write_buffer(iommu);
-	}
+	pasid_flush_caches(iommu, pte, pasid, did);
 
 	return 0;
 }
Signature · 1 line
2.7.4
author
#11 lore raw

PATCH 6/10 iommu/vt-d: Fix off-by-one in PASID allocation

In reply to Jacob Pan

PASID allocator uses IDR which is exclusive for the end of the allocation range. There is no need to decrement pasid_max.

Fixes af39507305fb ("iommu/vt-d: Apply global PASID in SVA")
---
 drivers/iommu/intel-svm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel-svm.c +1 -1
index 924a4de84be1..b5537f27592f 100644
@@ -338,7 +338,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 		/* Do not use PASID 0 in caching mode (virtualised IOMMU) */
 		ret = intel_pasid_alloc_id(svm,
 					   !!cap_caching_mode(iommu->cap),
-					   pasid_max - 1, GFP_KERNEL);
+					   pasid_max, GFP_KERNEL);
 		if (ret < 0) {
 			kfree(svm);
 			kfree(sdev);
Signature · 1 line
2.7.4
author
#20 lore raw

PATCH 10/10 iommu/vt-d: Misc macro clean up for SVM

In reply to Jacob Pan

Use combined macros for_each_svm_dev() to simplify SVM device iteration and error checking.

---
 drivers/iommu/intel-svm.c | 85 ++++++++++++++++++++++-------------------------
 1 file changed, 40 insertions(+), 45 deletions(-)
drivers/iommu/intel-svm.c +40 -45
index 189865501411..a7f67a9da3fc 100644
@@ -226,6 +226,10 @@ static const struct mmu_notifier_ops intel_mmuops = {
 static DEFINE_MUTEX(pasid_mutex);
 static LIST_HEAD(global_svm_list);
 
+#define for_each_svm_dev(sdev, svm, dev)		\
+	list_for_each_entry(sdev, &svm->devs, list)	\
+		if (dev != sdev->dev) {} else
+
 int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ops *ops)
 {
 	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
@@ -274,15 +278,13 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
 				goto out;
 			}
 
-			list_for_each_entry(sdev, &svm->devs, list) {
-				if (dev == sdev->dev) {
-					if (sdev->ops != ops) {
-						ret = -EBUSY;
-						goto out;
-					}
-					sdev->users++;
-					goto success;
+			for_each_svm_dev(sdev, svm, dev) {
+				if (sdev->ops != ops) {
+					ret = -EBUSY;
+					goto out;
 				}
+				sdev->users++;
+				goto success;
 			}
 
 			break;
@@ -427,45 +429,38 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
 		goto out;
 	}
 
-	if (!svm)
-		goto out;
-
-	list_for_each_entry(sdev, &svm->devs, list) {
-		if (dev == sdev->dev) {
-			ret = 0;
-			sdev->users--;
-			if (!sdev->users) {
-				list_del_rcu(&sdev->list);
-				/* Flush the PASID cache and IOTLB for this device.
-				 * Note that we do depend on the hardware *not* using
-				 * the PASID any more. Just as we depend on other
-				 * devices never using PASIDs that they have no right
-				 * to use. We have a *shared* PASID table, because it's
-				 * large and has to be physically contiguous. So it's
-				 * hard to be as defensive as we might like. */
-				intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
-				intel_flush_svm_range_dev(svm, sdev, 0, -1, 0);
-				kfree_rcu(sdev, rcu);
-
-				if (list_empty(&svm->devs)) {
-					/* Clear private data so that free pass check */
-					ioasid_set_data(svm->pasid, NULL);
-					ioasid_free(svm->pasid);
-					if (svm->mm)
-						mmu_notifier_unregister(&svm->notifier, svm->mm);
-
-					list_del(&svm->list);
-
-					/* We mandate that no page faults may be outstanding
-					 * for the PASID when intel_svm_unbind_mm() is called.
-					 * If that is not obeyed, subtle errors will happen.
-					 * Let's make them less subtle... */
-					memset(svm, 0x6b, sizeof(*svm));
-					kfree(svm);
-				}
+	for_each_svm_dev(sdev, svm, dev) {
+		ret = 0;
+		sdev->users--;
+		if (!sdev->users) {
+			list_del_rcu(&sdev->list);
+			/* Flush the PASID cache and IOTLB for this device.
+			 * Note that we do depend on the hardware *not* using
+			 * the PASID any more. Just as we depend on other
+			 * devices never using PASIDs that they have no right
+			 * to use. We have a *shared* PASID table, because it's
+			 * large and has to be physically contiguous. So it's
+			 * hard to be as defensive as we might like. */
+			intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
+			intel_flush_svm_range_dev(svm, sdev, 0, -1, 0);
+			kfree_rcu(sdev, rcu);
+
+			if (list_empty(&svm->devs)) {
+				/* Clear private data so that free pass check */
+				ioasid_set_data(svm->pasid, NULL);
+				ioasid_free(svm->pasid);
+				if (svm->mm)
+					mmu_notifier_unregister(&svm->notifier, svm->mm);
+				list_del(&svm->list);
+				/* We mandate that no page faults may be outstanding
+				 * for the PASID when intel_svm_unbind_mm() is called.
+				 * If that is not obeyed, subtle errors will happen.
+				 * Let's make them less subtle... */
+				memset(svm, 0x6b, sizeof(*svm));
+				kfree(svm);
 			}
-			break;
 		}
+		break;
 	}
  out:
 	mutex_unlock(&pasid_mutex);
Signature · 1 line
2.7.4
author
#13 lore raw

PATCH 7/10 iommu/vt-d: Replace Intel specific PASID allocator with IOASID

In reply to Jacob Pan

Make use of generic IOASID code to manage PASID allocation, free, and lookup. Replace Intel specific code.
IOASID allocator is inclusive for both start and end of the allocation range. The current code is based on IDR, which is exclusive for the end of the allocation range. This patch fixes the off-by-one error in intel_svm_bind_mm, where pasid_max - 1 is used for the end of allocation range.

---
 drivers/iommu/Kconfig       |  1 +
 drivers/iommu/intel-iommu.c | 12 ++++++------
 drivers/iommu/intel-pasid.c | 36 ------------------------------------
 drivers/iommu/intel-svm.c   | 41 +++++++++++++++++++++++++++--------------
 4 files changed, 34 insertions(+), 56 deletions(-)
drivers/iommu/Kconfig +1 -0
index fd50ddffffbf..43ce450a40d3 100644
@@ -212,6 +212,7 @@ config INTEL_IOMMU_SVM
 	depends on INTEL_IOMMU && X86
 	select PCI_PASID
 	select MMU_NOTIFIER
+	select IOASID
 	help
 	  Shared Virtual Memory (SVM) provides a facility for devices
 	  to access DMA resources through process address space by
drivers/iommu/intel-iommu.c +6 -6
index d598168e410d..844dcf8cbc61 100644
@@ -5238,7 +5238,7 @@ static void auxiliary_unlink_device(struct dmar_domain *domain,
 	domain->auxd_refcnt--;
 
 	if (!domain->auxd_refcnt && domain->default_pasid > 0)
-		intel_pasid_free_id(domain->default_pasid);
+		ioasid_free(domain->default_pasid);
 }
 
 static int aux_domain_add_dev(struct dmar_domain *domain,
@@ -5256,10 +5256,10 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
 	if (domain->default_pasid <= 0) {
 		int pasid;
 
-		pasid = intel_pasid_alloc_id(domain, PASID_MIN,
-					     pci_max_pasids(to_pci_dev(dev)),
-					     GFP_KERNEL);
-		if (pasid <= 0) {
+		/* No private data needed for the default pasid */
+		pasid = ioasid_alloc(NULL, PASID_MIN, pci_max_pasids(to_pci_dev(dev)) - 1,
+				NULL);
+		if (pasid == INVALID_IOASID) {
 			pr_err("Can't allocate default pasid\n");
 			return -ENODEV;
 		}
@@ -5295,7 +5295,7 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
 	spin_unlock(&iommu->lock);
 	spin_unlock_irqrestore(&device_domain_lock, flags);
 	if (!domain->auxd_refcnt && domain->default_pasid > 0)
-		intel_pasid_free_id(domain->default_pasid);
+		ioasid_free(domain->default_pasid);
 
 	return ret;
 }
drivers/iommu/intel-pasid.c +0 -36
index 732bfee228df..3cb569e76642 100644
@@ -26,42 +26,6 @@
  */
 static DEFINE_SPINLOCK(pasid_lock);
 u32 intel_pasid_max_id = PASID_MAX;
-static DEFINE_IDR(pasid_idr);
-
-int intel_pasid_alloc_id(void *ptr, int start, int end, gfp_t gfp)
-{
-	int ret, min, max;
-
-	min = max_t(int, start, PASID_MIN);
-	max = min_t(int, end, intel_pasid_max_id);
-
-	WARN_ON(in_interrupt());
-	idr_preload(gfp);
-	spin_lock(&pasid_lock);
-	ret = idr_alloc(&pasid_idr, ptr, min, max, GFP_ATOMIC);
-	spin_unlock(&pasid_lock);
-	idr_preload_end();
-
-	return ret;
-}
-
-void intel_pasid_free_id(int pasid)
-{
-	spin_lock(&pasid_lock);
-	idr_remove(&pasid_idr, pasid);
-	spin_unlock(&pasid_lock);
-}
-
-void *intel_pasid_lookup_id(int pasid)
-{
-	void *p;
-
-	spin_lock(&pasid_lock);
-	p = idr_find(&pasid_idr, pasid);
-	spin_unlock(&pasid_lock);
-
-	return p;
-}
 
 /*
  * Per device pasid table management:
drivers/iommu/intel-svm.c +27 -14

100 lines not on this page · raw message

Signature · 1 line
2.7.4
author
#18 lore raw

PATCH 9/10 iommu/vt-d: Avoid sending invalid page response

In reply to Jacob Pan

Page responses should only be sent when last page in group (LPIG) or private data is present in the page request. This patch avoids sending invalid descriptors.

Fixes 5d308fc1ecf53 ("iommu/vt-d: Add 256-bit invalidation descriptor

support")

---
 drivers/iommu/intel-svm.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/iommu/intel-svm.c +3 -4
index a223ae93b269..189865501411 100644
@@ -688,11 +688,10 @@ static irqreturn_t prq_event_thread(int irq, void *d)
 			if (req->priv_data_present)
 				memcpy(&resp.qw2, req->priv_data,
 				       sizeof(req->priv_data));
+			resp.qw2 = 0;
+			resp.qw3 = 0;
+			qi_submit_sync(&resp, iommu);
 		}
-		resp.qw2 = 0;
-		resp.qw3 = 0;
-		qi_submit_sync(&resp, iommu);
-
 		head = (head + sizeof(*req)) & PRQ_RING_MASK;
 	}
 
Signature · 1 line
2.7.4
author
#16 lore raw

PATCH 8/10 iommu/vt-d: Fix PASID cache flush

In reply to Jacob Pan

Use the correct invalidation descriptor type and granularity.

---
 drivers/iommu/intel-pasid.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/iommu/intel-pasid.c +2 -1
index 3cb569e76642..ee6ea1bbd917 100644
@@ -365,7 +365,8 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 {
 	struct qi_desc desc;
 
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL | QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;
 	desc.qw1 = 0;
 	desc.qw2 = 0;
 	desc.qw3 = 0;
Signature · 1 line
2.7.4
Acked-by on v1 1/10
#2 lore raw

Re: [PATCH 01/10] iommu/vt-d: Introduce native SVM capable flag

In reply to Jacob Pan

Hi,

Quoted 8 lines · Jacob Pan

Shared Virtual Memory(SVM) is based on a collective set of hardware features detected at runtime. There are requirements for matching CPU and IOMMU capabilities.

This patch introduces a flag which will be used to mark and test the capability of SVM.

Best regards,
baolu

Quoted 17 lines
---
  include/linux/intel-iommu.h | 1 +
  1 file changed, 1 insertion(+)
include/linux/intel-iommu.h +1 -0
index ed11ef594378..63118991824c 100644
@@ -433,6 +433,7 @@ enum {
 
  #define VTD_FLAG_TRANS_PRE_ENABLED	(1 << 0)
  #define VTD_FLAG_IRQ_REMAP_PRE_ENABLED	(1 << 1)
+#define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
  extern int intel_iommu_sm;
 
Acked-by on v1 2/10
#4 lore raw

Re: [PATCH 02/10] iommu/vt-d: Fix CPU and IOMMU SVM feature matching checks

In reply to Jacob Pan

Hi,

Quoted 9 lines · Jacob Pan

The current code checks CPU and IOMMU feature set for SVM support but the result is never stored nor used. Therefore, SVM can still be used even when these checks failed.

This patch consolidates code for checking PASID, CPU vs. IOMMU paging mode compatibility, as well as provides specific error messages for each failed checks.

Best regards,
baolu

Quoted 115 lines
---
  drivers/iommu/intel-iommu.c | 10 ++--------
  drivers/iommu/intel-svm.c   | 40 +++++++++++++++++++++++++++-------------
  include/linux/intel-iommu.h |  4 +++-
  3 files changed, 32 insertions(+), 22 deletions(-)
drivers/iommu/intel-iommu.c +2 -8
index 3f974919d3bd..d598168e410d 100644
@@ -3289,10 +3289,7 @@ static int __init init_dmars(void)
 
  		if (!ecap_pass_through(iommu->ecap))
  			hw_pass_through = 0;
-#ifdef CONFIG_INTEL_IOMMU_SVM
-		if (pasid_supported(iommu))
-			intel_svm_init(iommu);
-#endif
+		intel_svm_check(iommu);
  	}
 
  	/*
@@ -4471,10 +4468,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
  	if (ret)
  		goto out;
 
-#ifdef CONFIG_INTEL_IOMMU_SVM
-	if (pasid_supported(iommu))
-		intel_svm_init(iommu);
-#endif
+	intel_svm_check(iommu);
 
  	if (dmaru->ignored) {
  		/*
drivers/iommu/intel-svm.c +27 -13
index 9b159132405d..e9773d714862 100644
@@ -23,19 +23,6 @@
 
  static irqreturn_t prq_event_thread(int irq, void *d);
 
-int intel_svm_init(struct intel_iommu *iommu)
-{
-	if (cpu_feature_enabled(X86_FEATURE_GBPAGES) &&
-			!cap_fl1gp_support(iommu->cap))
-		return -EINVAL;
-
-	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
-			!cap_5lp_support(iommu->cap))
-		return -EINVAL;
-
-	return 0;
-}
-
  #define PRQ_ORDER 0
 
  int intel_svm_enable_prq(struct intel_iommu *iommu)
@@ -99,6 +86,33 @@ int intel_svm_finish_prq(struct intel_iommu *iommu)
  	return 0;
  }
 
+static inline bool intel_svm_capable(struct intel_iommu *iommu)
+{
+	return iommu->flags & VTD_FLAG_SVM_CAPABLE;
+}
+
+void intel_svm_check(struct intel_iommu *iommu)
+{
+	if (!pasid_supported(iommu))
+		return;
+
+	if (cpu_feature_enabled(X86_FEATURE_GBPAGES) &&
+	    !cap_fl1gp_support(iommu->cap)) {
+		pr_err("%s SVM disabled, incompatible 1GB page capability\n",
+			iommu->name);
+		return;
+	}
+
+	if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("%s SVM disabled, incompatible paging mode\n",
+			iommu->name);
+		return;
+	}
+
+	iommu->flags |= VTD_FLAG_SVM_CAPABLE;
+}
+
  static void intel_flush_svm_range_dev (struct intel_svm *svm, struct intel_svm_dev *sdev,
  				unsigned long address, unsigned long pages, int ih)
  {
include/linux/intel-iommu.h +3 -1
index 63118991824c..7dcfa1c4a844 100644
@@ -657,7 +657,7 @@ void iommu_flush_write_buffer(struct intel_iommu *iommu);
  int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev);
 
  #ifdef CONFIG_INTEL_IOMMU_SVM
-int intel_svm_init(struct intel_iommu *iommu);
+extern void intel_svm_check(struct intel_iommu *iommu);
  extern int intel_svm_enable_prq(struct intel_iommu *iommu);
  extern int intel_svm_finish_prq(struct intel_iommu *iommu);
 
@@ -685,6 +685,8 @@ struct intel_svm {
  };
 
  extern struct intel_iommu *intel_svm_device_to_iommu(struct device *dev);
+#else
+static inline void intel_svm_check(struct intel_iommu *iommu) {}
  #endif
 
  #ifdef CONFIG_INTEL_IOMMU_DEBUGFS
Acked-by on v1 3/10
#6 lore raw

Re: [PATCH 03/10] iommu/vt-d: Reject SVM bind for failed capability check

In reply to Jacob Pan

Hi,

Quoted 4 lines · Jacob Pan

Add a check during SVM bind to ensure CPU and IOMMU hardware capabilities are met.

Best regards,
baolu

Quoted 19 lines
---
  drivers/iommu/intel-svm.c | 3 +++
  1 file changed, 3 insertions(+)
drivers/iommu/intel-svm.c +3 -0
index e9773d714862..924a4de84be1 100644
@@ -238,6 +238,9 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
  	if (!iommu || dmar_disabled)
  		return -EINVAL;
 
+	if (!intel_svm_capable(iommu))
+		return -ENOTSUPP;
+
  	if (dev_is_pci(dev)) {
  		pasid_max = pci_max_pasids(to_pci_dev(dev));
  		if (pasid_max < 0)
Acked-by on v1 4/10
#8 lore raw

Re: [PATCH 04/10] iommu/vt-d: Match CPU and IOMMU paging mode

In reply to Jacob Pan

Hi,

Quoted 9 lines · Jacob Pan

When setting up first level page tables for sharing with CPU, we need to ensure IOMMU can support no less than the levels supported by the CPU.
It is not adequate, as in the current code, to set up 5-level paging in PASID entry First Level Paging Mode(FLPM) solely based on CPU.

Fixes 437f35e1cd4c8 ("iommu/vt-d: Add first level page table

interface")

Best regards,
baolu

Quoted 28 lines
---
  drivers/iommu/intel-pasid.c | 12 ++++++++++--
  1 file changed, 10 insertions(+), 2 deletions(-)
drivers/iommu/intel-pasid.c +10 -2
index 040a445be300..e7cb0b8a7332 100644
@@ -499,8 +499,16 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
  	}
 
  #ifdef CONFIG_X86
-	if (cpu_feature_enabled(X86_FEATURE_LA57))
-		pasid_set_flpm(pte, 1);
+	/* Both CPU and IOMMU paging mode need to match */
+	if (cpu_feature_enabled(X86_FEATURE_LA57)) {
+		if (cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);
+		} else {
+			pr_err("VT-d has no 5-level paging support for CPU\n");
+			pasid_clear_entry(pte);
+			return -EINVAL;
+		}
+	}
  #endif /* CONFIG_X86 */
 
  	pasid_set_domain_id(pte, did);
Acked-by on v1 5/10
#10 lore raw

Re: [PATCH 05/10] iommu/vt-d: Avoid duplicated code for PASID setup

In reply to Jacob Pan

Hi,

Quoted 1 line · Jacob Pan

After each setup for PASID entry, related translation caches must be flushed.

nit - maximum 75 chars per line

Others look good to me.

Best regards,
baolu

Quoted 89 lines

We can combine duplicated code into one function which is less error prone.

---
  drivers/iommu/intel-pasid.c | 48 +++++++++++++++++----------------------------
  1 file changed, 18 insertions(+), 30 deletions(-)
drivers/iommu/intel-pasid.c +18 -30
index e7cb0b8a7332..732bfee228df 100644
@@ -465,6 +465,21 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
  		devtlb_invalidation_with_pasid(iommu, dev, pasid);
  }
 
+static void pasid_flush_caches(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				int pasid, u16 did)
+{
+	if (!ecap_coherent(iommu->ecap))
+		clflush_cache_range(pte, sizeof(*pte));
+
+	if (cap_caching_mode(iommu->cap)) {
+		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
+		iotlb_invalidation_with_pasid(iommu, did, pasid);
+	} else {
+		iommu_flush_write_buffer(iommu);
+	}
+}
+
  /*
   * Set up the scalable mode pasid table entry for first only
   * translation type.
@@ -518,16 +533,7 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
  	/* Setup Present and PASID Granular Transfer Type: */
  	pasid_set_translation_type(pte, 1);
  	pasid_set_present(pte);
-
-	if (!ecap_coherent(iommu->ecap))
-		clflush_cache_range(pte, sizeof(*pte));
-
-	if (cap_caching_mode(iommu->cap)) {
-		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
-		iotlb_invalidation_with_pasid(iommu, did, pasid);
-	} else {
-		iommu_flush_write_buffer(iommu);
-	}
+	pasid_flush_caches(iommu, pte, pasid, did);
 
  	return 0;
  }
@@ -591,16 +597,7 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
  	 */
  	pasid_set_sre(pte);
  	pasid_set_present(pte);
-
-	if (!ecap_coherent(iommu->ecap))
-		clflush_cache_range(pte, sizeof(*pte));
-
-	if (cap_caching_mode(iommu->cap)) {
-		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
-		iotlb_invalidation_with_pasid(iommu, did, pasid);
-	} else {
-		iommu_flush_write_buffer(iommu);
-	}
+	pasid_flush_caches(iommu, pte, pasid, did);
 
  	return 0;
  }
@@ -634,16 +631,7 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
  	 */
  	pasid_set_sre(pte);
  	pasid_set_present(pte);
-
-	if (!ecap_coherent(iommu->ecap))
-		clflush_cache_range(pte, sizeof(*pte));
-
-	if (cap_caching_mode(iommu->cap)) {
-		pasid_cache_invalidation_with_pasid(iommu, did, pasid);
-		iotlb_invalidation_with_pasid(iommu, did, pasid);
-	} else {
-		iommu_flush_write_buffer(iommu);
-	}
+	pasid_flush_caches(iommu, pte, pasid, did);
 
  	return 0;
  }
Acked-by on v1 6/10
#12 lore raw

Re: [PATCH 06/10] iommu/vt-d: Fix off-by-one in PASID allocation

In reply to Jacob Pan

Hi,

Quoted 6 lines · Jacob Pan

PASID allocator uses IDR which is exclusive for the end of the allocation range. There is no need to decrement pasid_max.

Fixes af39507305fb ("iommu/vt-d: Apply global PASID in SVA")

Best regards,
baolu

Quoted 18 lines
---
  drivers/iommu/intel-svm.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel-svm.c +1 -1
index 924a4de84be1..b5537f27592f 100644
@@ -338,7 +338,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
  		/* Do not use PASID 0 in caching mode (virtualised IOMMU) */
  		ret = intel_pasid_alloc_id(svm,
  					   !!cap_caching_mode(iommu->cap),
-					   pasid_max - 1, GFP_KERNEL);
+					   pasid_max, GFP_KERNEL);
  		if (ret < 0) {
  			kfree(svm);
  			kfree(sdev);
Acked-by on v1 7/10
#14 lore raw

Re: [PATCH 07/10] iommu/vt-d: Replace Intel specific PASID allocator with IOASID

In reply to Jacob Pan

Hi,

Quoted 10 lines · Jacob Pan

Make use of generic IOASID code to manage PASID allocation, free, and lookup. Replace Intel specific code.
IOASID allocator is inclusive for both start and end of the allocation range. The current code is based on IDR, which is exclusive for the end of the allocation range. This patch fixes the off-by-one error in intel_svm_bind_mm, where pasid_max - 1 is used for the end of allocation range.

Two nit comments in line. With that fixed,

Best regards,
baolu

Quoted 33 lines
---
  drivers/iommu/Kconfig       |  1 +
  drivers/iommu/intel-iommu.c | 12 ++++++------
  drivers/iommu/intel-pasid.c | 36 ------------------------------------
  drivers/iommu/intel-svm.c   | 41 +++++++++++++++++++++++++++--------------
  4 files changed, 34 insertions(+), 56 deletions(-)
drivers/iommu/Kconfig +1 -0
index fd50ddffffbf..43ce450a40d3 100644
@@ -212,6 +212,7 @@ config INTEL_IOMMU_SVM
  	depends on INTEL_IOMMU && X86
  	select PCI_PASID
  	select MMU_NOTIFIER
+	select IOASID
  	help
  	  Shared Virtual Memory (SVM) provides a facility for devices
  	  to access DMA resources through process address space by
drivers/iommu/intel-iommu.c +1 -1
index d598168e410d..844dcf8cbc61 100644
@@ -5238,7 +5238,7 @@ static void auxiliary_unlink_device(struct dmar_domain *domain,
  	domain->auxd_refcnt--;
 
  	if (!domain->auxd_refcnt && domain->default_pasid > 0)
-		intel_pasid_free_id(domain->default_pasid);
+		ioasid_free(domain->default_pasid);
  }
 
  static int aux_domain_add_dev(struct dmar_domain *domain,

drivers/iommu/intel-iommu.c static int aux_domain_add_dev(struct dmar_domain *domain, · line 5261

Quoted 10 lines
drivers/iommu/intel-iommu.c +3 -4
@@ -5256,10 +5256,10 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
  	if (domain->default_pasid <= 0) {
  		int pasid;
 
-		pasid = intel_pasid_alloc_id(domain, PASID_MIN,
-					     pci_max_pasids(to_pci_dev(dev)),
-					     GFP_KERNEL);
-		if (pasid <= 0) {
+		/* No private data needed for the default pasid */
+		pasid = ioasid_alloc(NULL, PASID_MIN, pci_max_pasids(to_pci_dev(dev)) - 1,
+				NULL);

Nit - short line within 80 characters and align the new line with open parenthesis.

Quoted 74 lines
drivers/iommu/intel-iommu.c +2 -1
+		if (pasid == INVALID_IOASID) {
  			pr_err("Can't allocate default pasid\n");
  			return -ENODEV;
  		}
@@ -5295,7 +5295,7 @@ static int aux_domain_add_dev(struct dmar_domain *domain,
  	spin_unlock(&iommu->lock);
  	spin_unlock_irqrestore(&device_domain_lock, flags);
  	if (!domain->auxd_refcnt && domain->default_pasid > 0)
-		intel_pasid_free_id(domain->default_pasid);
+		ioasid_free(domain->default_pasid);
 
  	return ret;
  }
drivers/iommu/intel-pasid.c +0 -36
index 732bfee228df..3cb569e76642 100644
@@ -26,42 +26,6 @@
   */
  static DEFINE_SPINLOCK(pasid_lock);
  u32 intel_pasid_max_id = PASID_MAX;
-static DEFINE_IDR(pasid_idr);
-
-int intel_pasid_alloc_id(void *ptr, int start, int end, gfp_t gfp)
-{
-	int ret, min, max;
-
-	min = max_t(int, start, PASID_MIN);
-	max = min_t(int, end, intel_pasid_max_id);
-
-	WARN_ON(in_interrupt());
-	idr_preload(gfp);
-	spin_lock(&pasid_lock);
-	ret = idr_alloc(&pasid_idr, ptr, min, max, GFP_ATOMIC);
-	spin_unlock(&pasid_lock);
-	idr_preload_end();
-
-	return ret;
-}
-
-void intel_pasid_free_id(int pasid)
-{
-	spin_lock(&pasid_lock);
-	idr_remove(&pasid_idr, pasid);
-	spin_unlock(&pasid_lock);
-}
-
-void *intel_pasid_lookup_id(int pasid)
-{
-	void *p;
-
-	spin_lock(&pasid_lock);
-	p = idr_find(&pasid_idr, pasid);
-	spin_unlock(&pasid_lock);
-
-	return p;
-}
 
  /*
   * Per device pasid table management:
drivers/iommu/intel-svm.c +1 -0
index b5537f27592f..a223ae93b269 100644
@@ -17,6 +17,7 @@
  #include <linux/dmar.h>
  #include <linux/interrupt.h>
  #include <linux/mm_types.h>
+#include <linux/ioasid.h>
  #include <asm/page.h>
 
  #include "intel-pasid.h"
@@ -335,16 +336,15 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
  		if (pasid_max > intel_pasid_max_id)

drivers/iommu/intel-svm.c int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ · line 341

Quoted 10 lines
drivers/iommu/intel-svm.c +3 -5
  			pasid_max = intel_pasid_max_id;
 
-		/* Do not use PASID 0 in caching mode (virtualised IOMMU) */
-		ret = intel_pasid_alloc_id(svm,
-					   !!cap_caching_mode(iommu->cap),
-					   pasid_max, GFP_KERNEL);
-		if (ret < 0) {
+		/* Do not use PASID 0, reserved for RID to PASID */
+		svm->pasid = ioasid_alloc(NULL, PASID_MIN,
+					pasid_max - 1, svm);

Nit - open parenthesis alignment.

Quoted 89 lines
drivers/iommu/intel-svm.c +23 -9
+		if (svm->pasid == INVALID_IOASID) {
  			kfree(svm);
  			kfree(sdev);
+			ret = -ENOSPC;
  			goto out;
  		}
-		svm->pasid = ret;
  		svm->notifier.ops = &intel_mmuops;
  		svm->mm = mm;
  		svm->flags = flags;
@@ -354,7 +354,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
  		if (mm) {
  			ret = mmu_notifier_register(&svm->notifier, mm);
  			if (ret) {
-				intel_pasid_free_id(svm->pasid);
+				ioasid_free(svm->pasid);
  				kfree(svm);
  				kfree(sdev);
  				goto out;
@@ -370,7 +370,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
  		if (ret) {
  			if (mm)
  				mmu_notifier_unregister(&svm->notifier, mm);
-			intel_pasid_free_id(svm->pasid);
+			ioasid_free(svm->pasid);
  			kfree(svm);
  			kfree(sdev);
  			goto out;
@@ -418,7 +418,15 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
  	if (!iommu)
  		goto out;
 
-	svm = intel_pasid_lookup_id(pasid);
+	svm = ioasid_find(NULL, pasid, NULL);
+	if (!svm)
+		goto out;
+
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
  	if (!svm)
  		goto out;
 
@@ -440,7 +448,9 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
  				kfree_rcu(sdev, rcu);
 
  				if (list_empty(&svm->devs)) {
-					intel_pasid_free_id(svm->pasid);
+					/* Clear private data so that free pass check */
+					ioasid_set_data(svm->pasid, NULL);
+					ioasid_free(svm->pasid);
  					if (svm->mm)
  						mmu_notifier_unregister(&svm->notifier, svm->mm);
 
@@ -475,10 +485,14 @@ int intel_svm_is_pasid_valid(struct device *dev, int pasid)
  	if (!iommu)
  		goto out;
 
-	svm = intel_pasid_lookup_id(pasid);
+	svm = ioasid_find(NULL, pasid, NULL);
  	if (!svm)
  		goto out;
 
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
  	/* init_mm is used in this case */
  	if (!svm->mm)
  		ret = 1;
@@ -585,13 +599,12 @@ static irqreturn_t prq_event_thread(int irq, void *d)
 
  		if (!svm || svm->pasid != req->pasid) {
  			rcu_read_lock();
-			svm = intel_pasid_lookup_id(req->pasid);
+			svm = ioasid_find(NULL, req->pasid, NULL);
  			/* It *can't* go away, because the driver is not permitted
  			 * to unbind the mm while any page faults are outstanding.
  			 * So we only need RCU to protect the internal idr code. */
  			rcu_read_unlock();
-
-			if (!svm) {
+			if (IS_ERR_OR_NULL(svm)) {
  				pr_err("%s: Page request for invalid PASID %d: %08llx %08llx\n",
  				       iommu->name, req->pasid, ((unsigned long long *)req)[0],
  				       ((unsigned long long *)req)[1]);
Acked-by on v1 8/10
#17 lore raw

Re: [PATCH 08/10] iommu/vt-d: Fix PASID cache flush

In reply to Jacob Pan

Hi,

Quoted 3 lines · Jacob Pan

Use the correct invalidation descriptor type and granularity.

Best regards,
baolu

Quoted 19 lines
---
  drivers/iommu/intel-pasid.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
drivers/iommu/intel-pasid.c +2 -1
index 3cb569e76642..ee6ea1bbd917 100644
@@ -365,7 +365,8 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
  {
  	struct qi_desc desc;
 
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL | QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;
  	desc.qw1 = 0;
  	desc.qw2 = 0;
  	desc.qw3 = 0;
Acked-by on v1 9/10
#19 lore raw

Re: [PATCH 09/10] iommu/vt-d: Avoid sending invalid page response

In reply to Jacob Pan

Hi,

Quoted 7 lines · Jacob Pan

Page responses should only be sent when last page in group (LPIG) or private data is present in the page request. This patch avoids sending invalid descriptors.

Fixes 5d308fc1ecf53 ("iommu/vt-d: Add 256-bit invalidation descriptor

support")

Best regards,
baolu

Quoted 24 lines
---
  drivers/iommu/intel-svm.c | 7 +++----
  1 file changed, 3 insertions(+), 4 deletions(-)
drivers/iommu/intel-svm.c +3 -4
index a223ae93b269..189865501411 100644
@@ -688,11 +688,10 @@ static irqreturn_t prq_event_thread(int irq, void *d)
  			if (req->priv_data_present)
  				memcpy(&resp.qw2, req->priv_data,
  				       sizeof(req->priv_data));
+			resp.qw2 = 0;
+			resp.qw3 = 0;
+			qi_submit_sync(&resp, iommu);
  		}
-		resp.qw2 = 0;
-		resp.qw3 = 0;
-		qi_submit_sync(&resp, iommu);
-
  		head = (head + sizeof(*req)) & PRQ_RING_MASK;
  	}
 
#21 lore raw

Re: [PATCH 10/10] iommu/vt-d: Misc macro clean up for SVM

In reply to Jacob Pan

Hi,

Quoted 15 lines · Jacob Pan

Use combined macros for_each_svm_dev() to simplify SVM device iteration and error checking.

---
  drivers/iommu/intel-svm.c | 85 ++++++++++++++++++++++-------------------------
  1 file changed, 40 insertions(+), 45 deletions(-)
drivers/iommu/intel-svm.c +0 -0
index 189865501411..a7f67a9da3fc 100644

drivers/iommu/intel-svm.c static const struct mmu_notifier_ops intel_mmuops = { · line 231

Quoted 6 lines
drivers/iommu/intel-svm.c +3 -0
@@ -226,6 +226,10 @@ static const struct mmu_notifier_ops intel_mmuops = {
  static DEFINE_MUTEX(pasid_mutex);
  static LIST_HEAD(global_svm_list);
 
+#define for_each_svm_dev(sdev, svm, dev)		\
+	list_for_each_entry(sdev, &svm->devs, list)	\
+		if (dev != sdev->dev) {} else

"dev" has been reused in "sdev->dev", how about below?

#define for_each_svm_dev(sdev, svm, d)                  \
        list_for_each_entry((sdev), &(svm)->devs, list) \
                if ((d) != (sdev)->dev) {} else

Best regards,
baolu

Quoted 103 lines
drivers/iommu/intel-svm.c +37 -45
+
  int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ops *ops)
  {
  	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
@@ -274,15 +278,13 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_
  				goto out;
  			}
 
-			list_for_each_entry(sdev, &svm->devs, list) {
-				if (dev == sdev->dev) {
-					if (sdev->ops != ops) {
-						ret = -EBUSY;
-						goto out;
-					}
-					sdev->users++;
-					goto success;
+			for_each_svm_dev(sdev, svm, dev) {
+				if (sdev->ops != ops) {
+					ret = -EBUSY;
+					goto out;
  				}
+				sdev->users++;
+				goto success;
  			}
 
  			break;
@@ -427,45 +429,38 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)
  		goto out;
  	}
 
-	if (!svm)
-		goto out;
-
-	list_for_each_entry(sdev, &svm->devs, list) {
-		if (dev == sdev->dev) {
-			ret = 0;
-			sdev->users--;
-			if (!sdev->users) {
-				list_del_rcu(&sdev->list);
-				/* Flush the PASID cache and IOTLB for this device.
-				 * Note that we do depend on the hardware *not* using
-				 * the PASID any more. Just as we depend on other
-				 * devices never using PASIDs that they have no right
-				 * to use. We have a *shared* PASID table, because it's
-				 * large and has to be physically contiguous. So it's
-				 * hard to be as defensive as we might like. */
-				intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
-				intel_flush_svm_range_dev(svm, sdev, 0, -1, 0);
-				kfree_rcu(sdev, rcu);
-
-				if (list_empty(&svm->devs)) {
-					/* Clear private data so that free pass check */
-					ioasid_set_data(svm->pasid, NULL);
-					ioasid_free(svm->pasid);
-					if (svm->mm)
-						mmu_notifier_unregister(&svm->notifier, svm->mm);
-
-					list_del(&svm->list);
-
-					/* We mandate that no page faults may be outstanding
-					 * for the PASID when intel_svm_unbind_mm() is called.
-					 * If that is not obeyed, subtle errors will happen.
-					 * Let's make them less subtle... */
-					memset(svm, 0x6b, sizeof(*svm));
-					kfree(svm);
-				}
+	for_each_svm_dev(sdev, svm, dev) {
+		ret = 0;
+		sdev->users--;
+		if (!sdev->users) {
+			list_del_rcu(&sdev->list);
+			/* Flush the PASID cache and IOTLB for this device.
+			 * Note that we do depend on the hardware *not* using
+			 * the PASID any more. Just as we depend on other
+			 * devices never using PASIDs that they have no right
+			 * to use. We have a *shared* PASID table, because it's
+			 * large and has to be physically contiguous. So it's
+			 * hard to be as defensive as we might like. */
+			intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
+			intel_flush_svm_range_dev(svm, sdev, 0, -1, 0);
+			kfree_rcu(sdev, rcu);
+
+			if (list_empty(&svm->devs)) {
+				/* Clear private data so that free pass check */
+				ioasid_set_data(svm->pasid, NULL);
+				ioasid_free(svm->pasid);
+				if (svm->mm)
+					mmu_notifier_unregister(&svm->notifier, svm->mm);
+				list_del(&svm->list);
+				/* We mandate that no page faults may be outstanding
+				 * for the PASID when intel_svm_unbind_mm() is called.
+				 * If that is not obeyed, subtle errors will happen.
+				 * Let's make them less subtle... */
+				memset(svm, 0x6b, sizeof(*svm));
+				kfree(svm);
  			}
-			break;
  		}
+		break;
  	}
   out:
  	mutex_unlock(&pasid_mutex);
author
#22 lore raw

Re: [PATCH 10/10] iommu/vt-d: Misc macro clean up for SVM

Quoted 28 lines · On Mon, 18 Nov 2019 10:23:34 +0800 Lu Baolu <[email protected]> wrote:

Hi,

Quoted 16 lines · Jacob Pan

Use combined macros for_each_svm_dev() to simplify SVM device iteration and error checking.

---
  drivers/iommu/intel-svm.c | 85

++++++++++++++++++++++------------------------- 1 file changed, 40 insertions(+), 45 deletions(-)

drivers/iommu/intel-svm.c +0 -0
index 189865501411..a7f67a9da3fc 100644

drivers/iommu/intel-svm.c static const struct mmu_notifier_ops · line 231

Quoted 6 lines
drivers/iommu/intel-svm.c +3 -0
@@ -226,6 +226,10 @@ static const struct mmu_notifier_ops
 intel_mmuops = { static DEFINE_MUTEX(pasid_mutex);
  static LIST_HEAD(global_svm_list);
 
+#define for_each_svm_dev(sdev, svm, dev)		\
+	list_for_each_entry(sdev, &svm->devs, list)	\
+		if (dev != sdev->dev) {} else  

"dev" has been reused in "sdev->dev", how about below?

Quoted 4 lines
#define for_each_svm_dev(sdev, svm, d)                  \
         list_for_each_entry((sdev), &(svm)->devs, list) \
                 if ((d) != (sdev)->dev) {} else

sounds good. will do.

Quoted 5 lines

Best regards,
baolu

[...]

[Jacob Pan]

author
#15 lore raw

Re: [PATCH 07/10] iommu/vt-d: Replace Intel specific PASID allocator with IOASID

In reply to Lu Baolu
Quoted 11 lines · On Mon, 18 Nov 2019 09:43:00 +0800 Lu Baolu <[email protected]> wrote:

Hi,

Quoted 8 lines · Jacob Pan

Make use of generic IOASID code to manage PASID allocation, free, and lookup. Replace Intel specific code.
IOASID allocator is inclusive for both start and end of the allocation range. The current code is based on IDR, which is exclusive for the end of the allocation range. This patch fixes the off-by-one error in intel_svm_bind_mm, where pasid_max - 1 is used for the end of allocation range.

Quoted 7 lines
Quoted 2 lines

Two nit comments in line. With that fixed,

Thanks, will fix below.

Quoted 5 lines

Best regards,
baolu

[...]

Quoted 8 lines

Nit - short line within 80 characters and align the new line with open parenthesis.

[...]

Nit - open parenthesis alignment.

[...]

[Jacob Pan]