Skip to contentSkip to conversation

Nested Shared Virtual Address (SVA) VT-d support

PATCH Superseded Jacob Pan 6 participants 68 messages Mar 20, 2020 – Apr 2, 2020 IOMMU

v10 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Jacob Pan

author
#0 lore raw

v10 cover Nested Shared Virtual Address (SVA) VT-d support

Shared virtual address (SVA), a.k.a, Shared virtual memory (SVM) on Intel platforms allow address space sharing between device DMA and applications. SVA can reduce programming complexity and enhance security.
This series is intended to enable SVA virtualization, i.e. enable use of SVA within a guest user application.

This is the remaining portion of the original patchset that is based on Joerg's x86/vt-d branch. The preparatory and cleanup patches are merged here.
(git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git)

Only IOMMU portion of the changes are included in this series. Additional support is needed in VFIO and QEMU (will be submitted separately) to complete this functionality.

To make incremental changes and reduce the size of each patchset. This series does not inlcude support for page request services.

In VT-d implementation, PASID table is per device and maintained in the host. Guest PASID table is shadowed in VMM where virtual IOMMU is emulated.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

This is the remaining VT-d only portion of V5 since the uAPIs and IOASID common code have been applied to Joerg's IOMMU core branch.
(https://lkml.org/lkml/2019/10/2/833)

The complete set with VFIO patches are here:
https://github.com/jacobpan/linux.git:siov_sva

The complete nested SVA upstream patches are divided into three phases:
1. Common APIs and PCI device direct assignment
2. Page Request Services (PRS) support
3. Mediated device assignment

With this set and the accompanied VFIO code, we will achieve phase #1.

Thanks,

Jacob

ChangeLog:
- v10
- Addressed Eric's review in v7 and v9. Most fixes are in 3/10 and 6/10. Extra condition checks and consolidation of duplicated codes.

- v9
- Addressed Baolu's comments for v8 for IOTLB flush consolidation, bug fixes
- Removed IOASID notifier code which will be submitted separately to address PASID life cycle management with multiple users.

- v8
- Extracted cleanup patches from V7 and accepted into maintainer's tree (https://lkml.org/lkml/2019/12/2/514).
- Added IOASID notifier and VT-d handler for termination of PASID IOMMU context upon free. This will ensure success of VFIO IOASID free API regardless PASID is in use.
(https://lore.kernel.org/linux-iommu/[email protected]/)

- V7
- Respect vIOMMU PASID range in virtual command PASID/IOASID allocator
- Caching virtual command capabilities to avoid runtime checks that could cause vmexits.

- V6
- Rebased on top of Joerg's core branch
(git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git core)
- Adapt to new uAPIs and IOASID allocators

- V5
Rebased on v5.3-rc4 which has some of the IOMMU fault APIs merged. Addressed v4 review comments from Eric Auger, Baolu Lu, and
Jonathan Cameron. Specific changes are as follows:
- Refined custom IOASID allocator to support multiple vIOMMU, hotplug cases.
- Extracted vendor data from IOMMU guest PASID bind data, for VT-d will support all necessary guest PASID entry fields for PASID bind.
- Support non-identity host-guest PASID mapping
- Exception handling in various cases

- V4
- Redesigned IOASID allocator such that it can support custom allocators with shared helper functions. Use separate XArray to store IOASIDs per allocator. Took advice from Eric Auger to have default allocator use the generic allocator structure. Combined into one patch in that the default allocator is just "another" allocator now. Can be built as a module in case of driver use without IOMMU.
- Extended bind guest PASID data to support SMMU and non-identity guest to host PASID mapping https://lkml.org/lkml/2019/5/21/802
- Rebased on Jean's sva/api common tree, new patches starts with [PATCH v4 10/22]

- V3
- Addressed thorough review comments from Eric Auger (Thank you!)
- Moved IOASID allocator from driver core to IOMMU code per suggestion by Christoph Hellwig
(https://lkml.org/lkml/2019/4/26/462)
- Rebased on top of Jean's SVA API branch and Eric's v7[1]
(git://linux-arm.org/linux-jpb.git sva/api)
- All IOMMU APIs are unmodified (except the new bind guest PASID call in patch 9/16)

- V2
- Rebased on Joerg's IOMMU x86/vt-d branch v5.1-rc4
- Integrated with Eric Auger's new v7 series for common APIs
(https://github.com/eauger/linux/tree/v5.1-rc3-2stage-v7)
- Addressed review comments from Andy Shevchenko and Alex Williamson on IOASID custom allocator.
- Support multiple custom IOASID allocators (vIOMMUs) and dynamic registration.

Jacob Pan (10):
iommu/vt-d: Move domain helper to header iommu/uapi: Define a mask for bind data iommu/vt-d: Add a helper function to skip agaw iommu/vt-d: Use helper function to skip agaw for SL iommu/vt-d: Add nested translation helper function iommu/vt-d: Add bind guest PASID support iommu/vt-d: Support flushing more translation cache types iommu/vt-d: Add svm/sva invalidate function iommu/vt-d: Cache virtual command capability register iommu/vt-d: Add custom allocator for IOASID

Lu Baolu (1):
iommu/vt-d: Enlightened PASID allocation

 drivers/iommu/dmar.c        |  37 +++++
 drivers/iommu/intel-iommu.c | 276 +++++++++++++++++++++++++++++++++++-
 drivers/iommu/intel-pasid.c | 336 ++++++++++++++++++++++++++++++++++++++++++--
 drivers/iommu/intel-pasid.h |  25 +++-
 drivers/iommu/intel-svm.c   | 224 +++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  45 +++++-
 include/linux/intel-svm.h   |  17 +++
 include/uapi/linux/iommu.h  |   5 +-
 8 files changed, 938 insertions(+), 27 deletions(-)
Signature · 1 line
2.7.4
author
#1 lore raw

PATCH v10 1/11 iommu/vt-d: Move domain helper to header

Move domain helper to header to be used by SVA code.

---
 drivers/iommu/intel-iommu.c | 6 ------
 include/linux/intel-iommu.h | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/iommu/intel-iommu.c +0 -6
index 4be549478691..e599b2537b1c 100644
@@ -446,12 +446,6 @@ static void init_translation_status(struct intel_iommu *iommu)
 		iommu->flags |= VTD_FLAG_TRANS_PRE_ENABLED;
 }
 
-/* Convert generic 'struct iommu_domain to private struct dmar_domain */
-static struct dmar_domain *to_dmar_domain(struct iommu_domain *dom)
-{
-	return container_of(dom, struct dmar_domain, domain);
-}
-
 static int __init intel_iommu_setup(char *str)
 {
 	if (!str)
include/linux/intel-iommu.h +6 -0
index 980234ae0312..ed7171d2ae1f 100644
@@ -595,6 +595,12 @@ static inline void __iommu_flush_cache(
 		clflush_cache_range(addr, size);
 }
 
+/* Convert generic struct iommu_domain to private struct dmar_domain */
+static inline struct dmar_domain *to_dmar_domain(struct iommu_domain *dom)
+{
+	return container_of(dom, struct dmar_domain, domain);
+}
+
 /*
  * 0: readable
  * 1: writable
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Reviewed-by on v10 1/11
#2 lore raw

RE: [PATCH V10 01/11] iommu/vt-d: Move domain helper to header

Quoted 47 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Move domain helper to header to be used by SVA code.

---
 drivers/iommu/intel-iommu.c | 6 ------
 include/linux/intel-iommu.h | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/iommu/intel-iommu.c +0 -6
index 4be549478691..e599b2537b1c 100644
@@ -446,12 +446,6 @@ static void init_translation_status(struct
 intel_iommu *iommu)
 		iommu->flags |= VTD_FLAG_TRANS_PRE_ENABLED;
 }
 
-/* Convert generic 'struct iommu_domain to private struct dmar_domain */
-static struct dmar_domain *to_dmar_domain(struct iommu_domain *dom)
-{
-	return container_of(dom, struct dmar_domain, domain);
-}
-
 static int __init intel_iommu_setup(char *str)
 {
drivers/iommu/intel-iommu.c +0 -0
 	if (!str)
include/linux/intel-iommu.h +6 -0
index 980234ae0312..ed7171d2ae1f 100644
@@ -595,6 +595,12 @@ static inline void __iommu_flush_cache(
 		clflush_cache_range(addr, size);
 }
 
+/* Convert generic struct iommu_domain to private struct dmar_domain */
+static inline struct dmar_domain *to_dmar_domain(struct iommu_domain
 *dom)
+{
+	return container_of(dom, struct dmar_domain, domain);
+}
+
 /*
  * 0: readable
Quoted 3 lines

* 1: writable
--
2.7.4

author
#3 lore raw

PATCH v10 2/11 iommu/uapi: Define a mask for bind data

In reply to Jacob Pan

Memory type related flags can be grouped together for one simple check.

---
v9 renamed from EMT to MTS since these are memory type support flags.
---

---
 include/uapi/linux/iommu.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
include/uapi/linux/iommu.h +4 -1
index 4ad3496e5c43..d7bcbc5f79b0 100644
@@ -284,7 +284,10 @@ struct iommu_gpasid_bind_data_vtd {
 	__u32 pat;
 	__u32 emt;
 };
-
+#define IOMMU_SVA_VTD_GPASID_MTS_MASK	(IOMMU_SVA_VTD_GPASID_CD | \
+					 IOMMU_SVA_VTD_GPASID_EMTE | \
+					 IOMMU_SVA_VTD_GPASID_PCD |  \
+					 IOMMU_SVA_VTD_GPASID_PWT)
 /**
  * struct iommu_gpasid_bind_data - Information about device and guest PASID binding
  * @version:	Version of this data structure
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#4 lore raw

Re: [PATCH V10 02/11] iommu/uapi: Define a mask for bind data

Quoted 16 lines · Jacob Pan

Memory type related flags can be grouped together for one simple check.

---
v9 renamed from EMT to MTS since these are memory type support flags.
---

---
  include/uapi/linux/iommu.h | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
include/uapi/linux/iommu.h +0 -0
index 4ad3496e5c43..d7bcbc5f79b0 100644

include/uapi/linux/iommu.h struct iommu_gpasid_bind_data_vtd { · line 290

Quoted 8 lines
include/uapi/linux/iommu.h +4 -1
@@ -284,7 +284,10 @@ struct iommu_gpasid_bind_data_vtd {
  	__u32 pat;
  	__u32 emt;
  };
-
+#define IOMMU_SVA_VTD_GPASID_MTS_MASK	(IOMMU_SVA_VTD_GPASID_CD | \
+					 IOMMU_SVA_VTD_GPASID_EMTE | \
+					 IOMMU_SVA_VTD_GPASID_PCD |  \
+					 IOMMU_SVA_VTD_GPASID_PWT)

As name implies, can this move to intel-iommu.h?

Best regards,
baolu

Quoted 4 lines
include/uapi/linux/iommu.h +0 -0
  /**
   * struct iommu_gpasid_bind_data - Information about device and guest PASID binding
   * @version:	Version of this data structure
author
#5 lore raw

Re: [PATCH V10 02/11] iommu/uapi: Define a mask for bind data

Quoted 24 lines · On Sun, 22 Mar 2020 09:29:32 +0800 Lu Baolu <[email protected]> wrote:
Quoted 23 lines · Jacob Pan

Memory type related flags can be grouped together for one simple check.

---
v9 renamed from EMT to MTS since these are memory type support flags. ---

---
  include/uapi/linux/iommu.h | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
include/uapi/linux/iommu.h +1 -1
index 4ad3496e5c43..d7bcbc5f79b0 100644
@@ -284,7 +284,10 @@ struct iommu_gpasid_bind_data_vtd {
  	__u32 pat;
  	__u32 emt;
  };
-
+#define IOMMU_SVA_VTD_GPASID_MTS_MASK
 (IOMMU_SVA_VTD_GPASID_CD | \
Quoted 9 lines
Quoted 6 lines
include/uapi/linux/iommu.h +3 -0
+					 IOMMU_SVA_VTD_GPASID_EMTE
 | \
+					 IOMMU_SVA_VTD_GPASID_PCD
 |  \
+

IOMMU_SVA_VTD_GPASID_PWT)

As name implies, can this move to intel-iommu.h?

I also thought about this but the masks are in vendor specific part of the UAPI.

Quoted 8 lines

Best regards,
baolu

Quoted 5 lines

/**
* struct iommu_gpasid_bind_data - Information about device and
guest PASID binding
* @version: Version of this data structure

[Jacob Pan]

#6 lore raw

Re: [PATCH V10 02/11] iommu/uapi: Define a mask for bind data

Quoted 34 lines · Jacob Pan
Quoted 23 lines · On Sun, 22 Mar 2020 09:29:32 +0800> Lu Baolu<[email protected]> wrote:
Quoted 22 lines · Jacob Pan

Memory type related flags can be grouped together for one simple check.

---
v9 renamed from EMT to MTS since these are memory type support flags. ---

---
   include/uapi/linux/iommu.h | 5 ++++-
   1 file changed, 4 insertions(+), 1 deletion(-)
include/uapi/linux/iommu.h +1 -1
index 4ad3496e5c43..d7bcbc5f79b0 100644
@@ -284,7 +284,10 @@ struct iommu_gpasid_bind_data_vtd {
   	__u32 pat;
   	__u32 emt;
   };
-
+#define IOMMU_SVA_VTD_GPASID_MTS_MASK
Quoted 9 lines
Quoted 7 lines
include/uapi/linux/iommu.h +3 -0
 (IOMMU_SVA_VTD_GPASID_CD | \
+					 IOMMU_SVA_VTD_GPASID_EMTE
 | \
+					 IOMMU_SVA_VTD_GPASID_PCD
 |  \
+

IOMMU_SVA_VTD_GPASID_PWT)

As name implies, can this move to intel-iommu.h?

Quoted 3 lines

I also thought about this but the masks are in vendor specific part of the UAPI.

I looked through this patch series. It looks good to me. I will do some code style cleanup and take it to v5.7. I am not the right person to decide whether include/uapi/linux/iommu.h is the right place for this, so I will move it to Intel IOMMU driver for now.

Best regards,
baolu

Reviewed-by on v10 2/11
#7 lore raw

RE: [PATCH V10 02/11] iommu/uapi: Define a mask for bind data

In reply to Jacob Pan
Quoted 30 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Memory type related flags can be grouped together for one simple check.

---
v9 renamed from EMT to MTS since these are memory type support flags.
---

---
 include/uapi/linux/iommu.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
include/uapi/linux/iommu.h +4 -1
index 4ad3496e5c43..d7bcbc5f79b0 100644
@@ -284,7 +284,10 @@ struct iommu_gpasid_bind_data_vtd {
 	__u32 pat;
 	__u32 emt;
 };
-
+#define IOMMU_SVA_VTD_GPASID_MTS_MASK
 	(IOMMU_SVA_VTD_GPASID_CD | \
+					 IOMMU_SVA_VTD_GPASID_EMTE | \
+					 IOMMU_SVA_VTD_GPASID_PCD |  \
+					 IOMMU_SVA_VTD_GPASID_PWT)
 /**
  * struct iommu_gpasid_bind_data - Information about device and guest
Quoted 4 lines

PASID binding
* @version: Version of this data structure
--
2.7.4

Reviewed-by on v10 2/11
#8 lore raw

Re: [PATCH V10 02/11] iommu/uapi: Define a mask for bind data

In reply to Jacob Pan

Hi Jacob,

Quoted 7 lines · Jacob Pan

Memory type related flags can be grouped together for one simple check.

---
v9 renamed from EMT to MTS since these are memory type support flags.
---

Thanks

Eric

Quoted 21 lines
---
 include/uapi/linux/iommu.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
include/uapi/linux/iommu.h +4 -1
index 4ad3496e5c43..d7bcbc5f79b0 100644
@@ -284,7 +284,10 @@ struct iommu_gpasid_bind_data_vtd {
 	__u32 pat;
 	__u32 emt;
 };
-
+#define IOMMU_SVA_VTD_GPASID_MTS_MASK	(IOMMU_SVA_VTD_GPASID_CD | \
+					 IOMMU_SVA_VTD_GPASID_EMTE | \
+					 IOMMU_SVA_VTD_GPASID_PCD |  \
+					 IOMMU_SVA_VTD_GPASID_PWT)
 /**
  * struct iommu_gpasid_bind_data - Information about device and guest PASID binding
  * @version:	Version of this data structure
author
#9 lore raw

PATCH v10 3/11 iommu/vt-d: Add a helper function to skip agaw

In reply to Jacob Pan
---
 drivers/iommu/intel-pasid.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
drivers/iommu/intel-pasid.c +22 -0
index 22b30f10b396..191508c7c03e 100644
@@ -500,6 +500,28 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
 }
 
 /*
+ * Skip top levels of page tables for iommu which has less agaw
+ * than default. Unnecessary for PT mode.
+ */
+static inline int iommu_skip_agaw(struct dmar_domain *domain,
+				  struct intel_iommu *iommu,
+				  struct dma_pte **pgd)
+{
+	int agaw;
+
+	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
+		*pgd = phys_to_virt(dma_pte_addr(*pgd));
+		if (!dma_pte_present(*pgd)) {
+			return -EINVAL;
+		}
+	}
+	pr_debug_ratelimited("%s: pgd: %llx, agaw %d d_agaw %d\n", __func__, (u64)*pgd,
+		iommu->agaw, domain->agaw);
+
+	return agaw;
+}
+
+/*
  * Set up the scalable mode pasid entry for second only translation type.
  */
 int intel_pasid_setup_second_level(struct intel_iommu *iommu,
Signature · 1 line
2.7.4
#10 lore raw

RE: [PATCH V10 03/11] iommu/vt-d: Add a helper function to skip agaw

Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

could you elaborate in which scenario this helper function is required?

Quoted 41 lines
---
 drivers/iommu/intel-pasid.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
drivers/iommu/intel-pasid.c +22 -0
index 22b30f10b396..191508c7c03e 100644
@@ -500,6 +500,28 @@ int intel_pasid_setup_first_level(struct intel_iommu
 *iommu,
 }
 
 /*
+ * Skip top levels of page tables for iommu which has less agaw
+ * than default. Unnecessary for PT mode.
+ */
+static inline int iommu_skip_agaw(struct dmar_domain *domain,
+				  struct intel_iommu *iommu,
+				  struct dma_pte **pgd)
+{
+	int agaw;
+
+	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
+		*pgd = phys_to_virt(dma_pte_addr(*pgd));
+		if (!dma_pte_present(*pgd)) {
+			return -EINVAL;
+		}
+	}
+	pr_debug_ratelimited("%s: pgd: %llx, agaw %d d_agaw %d\n",
 __func__, (u64)*pgd,
+		iommu->agaw, domain->agaw);
+
+	return agaw;
+}
+
+/*
  * Set up the scalable mode pasid entry for second only translation type.

*/
int intel_pasid_setup_second_level(struct intel_iommu *iommu,
--
2.7.4

#11 lore raw

Re: [PATCH V10 03/11] iommu/vt-d: Add a helper function to skip agaw

Quoted 6 lines · Kevin
Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

could you elaborate in which scenario this helper function is required?

I added below commit message:

An Intel iommu domain uses 5-level page table by default. If the iommu that the domain tries to attach supports less page levels, the top level page tables should be skipped. Add a helper to do this so that it could be used in other places.

Best regards,
baolu

Quoted 34 lines
Quoted 34 lines
  
---
  drivers/iommu/intel-pasid.c | 22 ++++++++++++++++++++++
  1 file changed, 22 insertions(+)
drivers/iommu/intel-pasid.c +19 -0
index 22b30f10b396..191508c7c03e 100644
@@ -500,6 +500,28 @@ int intel_pasid_setup_first_level(struct intel_iommu
 *iommu,
  }
 
  /*
+ * Skip top levels of page tables for iommu which has less agaw
+ * than default. Unnecessary for PT mode.
+ */
+static inline int iommu_skip_agaw(struct dmar_domain *domain,
+				  struct intel_iommu *iommu,
+				  struct dma_pte **pgd)
+{
+	int agaw;
+
+	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
+		*pgd = phys_to_virt(dma_pte_addr(*pgd));
+		if (!dma_pte_present(*pgd)) {
+			return -EINVAL;
+		}
+	}
+	pr_debug_ratelimited("%s: pgd: %llx, agaw %d d_agaw %d\n",
 __func__, (u64)*pgd,
+		iommu->agaw, domain->agaw);
+
+	return agaw;
Quoted 9 lines
Quoted 8 lines
drivers/iommu/intel-pasid.c +3 -0
+}
+
+/*
   * Set up the scalable mode pasid entry for second only translation type.
   */

int intel_pasid_setup_second_level(struct intel_iommu *iommu,
--
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#12 lore raw

Re: [PATCH V10 03/11] iommu/vt-d: Add a helper function to skip agaw

Quoted 9 lines · On Sun, 29 Mar 2020 15:20:55 +0800 Lu Baolu <[email protected]> wrote:
Quoted 7 lines · Kevin
Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

could you elaborate in which scenario this helper function is required?

Quoted 7 lines

I added below commit message:

An Intel iommu domain uses 5-level page table by default. If the iommu that the domain tries to attach supports less page levels, the top level page tables should be skipped. Add a helper to do this so that it could be used in other places.

Thanks Baolu,
I will also add this to my v11, it might save you some time :)

Quoted 46 lines

Best regards,
baolu

Quoted 43 lines
Quoted 42 lines
    
---
  drivers/iommu/intel-pasid.c | 22 ++++++++++++++++++++++
  1 file changed, 22 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 22b30f10b396..191508c7c03e 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +22 -0
@@ -500,6 +500,28 @@ int intel_pasid_setup_first_level(struct
 intel_iommu *iommu,
  }
 
  /*
+ * Skip top levels of page tables for iommu which has less agaw
+ * than default. Unnecessary for PT mode.
+ */
+static inline int iommu_skip_agaw(struct dmar_domain *domain,
+				  struct intel_iommu *iommu,
+				  struct dma_pte **pgd)
+{
+	int agaw;
+
+	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
+		*pgd = phys_to_virt(dma_pte_addr(*pgd));
+		if (!dma_pte_present(*pgd)) {
+			return -EINVAL;
+		}
+	}
+	pr_debug_ratelimited("%s: pgd: %llx, agaw %d d_agaw %d\n",
 __func__, (u64)*pgd,
+		iommu->agaw, domain->agaw);
+
+	return agaw;
+}
+
+/*
   * Set up the scalable mode pasid entry for second only

translation type. */
int intel_pasid_setup_second_level(struct intel_iommu *iommu,
--
2.7.4

[Jacob Pan]

author
#13 lore raw

PATCH v10 4/11 iommu/vt-d: Use helper function to skip agaw for SL

In reply to Jacob Pan
---
 drivers/iommu/intel-pasid.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
drivers/iommu/intel-pasid.c +4 -10
index 191508c7c03e..9bdb7ee228b6 100644
@@ -544,17 +544,11 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
 		return -EINVAL;
 	}
 
-	/*
-	 * Skip top levels of page tables for iommu which has less agaw
-	 * than default. Unnecessary for PT mode.
-	 */
 	pgd = domain->pgd;
-	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
-		pgd = phys_to_virt(dma_pte_addr(pgd));
-		if (!dma_pte_present(pgd)) {
-			dev_err(dev, "Invalid domain page table\n");
-			return -EINVAL;
-		}
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
 	}
 
 	pgd_val = virt_to_phys(pgd);
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#14 lore raw

RE: [PATCH V10 04/11] iommu/vt-d: Use helper function to skip agaw for SL

Quoted 23 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

---
 drivers/iommu/intel-pasid.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
drivers/iommu/intel-pasid.c +0 -5
index 191508c7c03e..9bdb7ee228b6 100644
@@ -544,17 +544,11 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
 		return -EINVAL;
 	}
 
-	/*
-	 * Skip top levels of page tables for iommu which has less agaw
-	 * than default. Unnecessary for PT mode.
-	 */
 	pgd = domain->pgd;
-	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {

drivers/iommu/intel-pasid.c int intel_pasid_setup_second_level(struct · line 553

Quoted 10 lines
drivers/iommu/intel-pasid.c +4 -5
-		pgd = phys_to_virt(dma_pte_addr(pgd));
-		if (!dma_pte_present(pgd)) {
-			dev_err(dev, "Invalid domain page table\n");
-			return -EINVAL;
-		}
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
 	}

ok, I see how it is used. possibly combine last and this one together since it's mostly moving code...

Quoted 4 lines
drivers/iommu/intel-pasid.c +0 -1
 	pgd_val = virt_to_phys(pgd);
--

2.7.4

#15 lore raw

Re: [PATCH V10 04/11] iommu/vt-d: Use helper function to skip agaw for SL

Hi Jacob,

Quoted 17 lines · Kevin
Quoted 17 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

---
 drivers/iommu/intel-pasid.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)
drivers/iommu/intel-pasid.c +0 -0
index 191508c7c03e..9bdb7ee228b6 100644
@@ -544,17 +544,11 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
 		return -EINVAL;
 	}
 
Quoted 14 lines
Quoted 4 lines
drivers/iommu/intel-pasid.c +0 -4
-	/*
-	 * Skip top levels of page tables for iommu which has less agaw
-	 * than default. Unnecessary for PT mode.
-	 */

drivers/iommu/intel-pasid.c int intel_pasid_setup_second_level(struct intel_iommu *iommu, · line 550

Quoted 10 lines
drivers/iommu/intel-pasid.c +3 -6
 	pgd = domain->pgd;
-	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
-		pgd = phys_to_virt(dma_pte_addr(pgd));
-		if (!dma_pte_present(pgd)) {
-			dev_err(dev, "Invalid domain page table\n");
-			return -EINVAL;
-		}
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");

is the dev_err() really requested. I see in domain_setup_first_level(), there is none.

Quoted 5 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_second_level(struct intel_iommu *iommu, · line 552

Quoted 2 lines
drivers/iommu/intel-pasid.c +1 -0
+		return -EINVAL;
 	}

ok, I see how it is used. possibly combine last and this one together since it's mostly moving code...

I tend to agree with Kevin. May be better squash the 2 patches. Also not sure the inline of iommu_skip_agaw() is meaningful then. Also Add commit messages on the resulting patch.

Note domain_setup_first_level() also could use the helper while we are it (if declaration moved to common helper). Only the error code differs in case !dma_pte_present(pgd), ie. -ENOMEM. May be good to align.

Otherwise those stuff may be done in a fixup patch.

Thanks

Eric

Quoted 6 lines
Quoted 5 lines
drivers/iommu/intel-pasid.c +0 -1
 	pgd_val = virt_to_phys(pgd);
--

2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#16 lore raw

Re: [PATCH V10 04/11] iommu/vt-d: Use helper function to skip agaw for SL

Quoted 44 lines · Auger Eric

Hi Jacob,

Quoted 17 lines · Kevin
Quoted 17 lines

From: Jacob Pan<[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

---
  drivers/iommu/intel-pasid.c | 14 ++++----------
  1 file changed, 4 insertions(+), 10 deletions(-)
drivers/iommu/intel-pasid.c +0 -0
index 191508c7c03e..9bdb7ee228b6 100644
@@ -544,17 +544,11 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
  		return -EINVAL;
  	}
 
Quoted 14 lines
Quoted 4 lines
drivers/iommu/intel-pasid.c +0 -4
-	/*
-	 * Skip top levels of page tables for iommu which has less agaw
-	 * than default. Unnecessary for PT mode.
-	 */

drivers/iommu/intel-pasid.c int intel_pasid_setup_second_level(struct intel_iommu *iommu, · line 550

Quoted 10 lines
drivers/iommu/intel-pasid.c +3 -6
  	pgd = domain->pgd;
-	for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) {
-		pgd = phys_to_virt(dma_pte_addr(pgd));
-		if (!dma_pte_present(pgd)) {
-			dev_err(dev, "Invalid domain page table\n");
-			return -EINVAL;
-		}
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");

is the dev_err() really requested. I see in domain_setup_first_level(), there is none.

Quoted 4 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_second_level(struct intel_iommu *iommu, · line 552

Quoted 2 lines
drivers/iommu/intel-pasid.c +1 -0
+		return -EINVAL;
  	}

ok, I see how it is used. possibly combine last and this one together since it's mostly moving code...

I tend to agree with Kevin. May be better squash the 2 patches. Also not sure the inline of iommu_skip_agaw() is meaningful then. Also Add commit messages on the resulting patch.

Quoted 5 lines

Note domain_setup_first_level() also could use the helper while we are it (if declaration moved to common helper). Only the error code differs in case !dma_pte_present(pgd), ie. -ENOMEM. May be good to align.

Otherwise those stuff may be done in a fixup patch.

Agreed. Will squash these 2 patches with a meaningful commit message. As for using this helper in other files, like domain_setup_first_level(), we need more review and test efforts, hence it's better to put it in a followup patch.

Best regards,
baolu

author
#17 lore raw

PATCH v10 5/11 iommu/vt-d: Add nested translation helper function

In reply to Jacob Pan

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8. With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
 drivers/iommu/intel-pasid.c | 240 +++++++++++++++++++++++++++++++++++++++++++-
 drivers/iommu/intel-pasid.h |  12 +++
 include/linux/intel-iommu.h |   3 +
 3 files changed, 252 insertions(+), 3 deletions(-)
drivers/iommu/intel-pasid.c +237 -3
index 9bdb7ee228b6..10c7856afc6b 100644
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64 value)
 	pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
 }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value << 27);
+}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value << 32);
+}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
 static void
 pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
 	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
 	pasid_set_present(pte);
 	pasid_flush_caches(iommu, pte, pasid, did);
 
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
 	pasid_set_domain_id(pte, did);
 	pasid_set_slptr(pte, pgd_val);
 	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 	pasid_clear_entry(pte);
 	pasid_set_domain_id(pte, did);
 	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 
 	return 0;
 }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd *pasid_data)
+{
+	/*
+	 * Not all guest PASID table entry fields are passed down during bind,
+	 * here we only set up the ones that are dependent on guest settings.
+	 * Execution related bits such as NXE, SMEP are not meaningful to IOMMU,
+	 * therefore not set. Other fields, such as snoop related, are set based
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on %s\n",
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support on %s\n",
+				iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
+	 * Memory type is only applicable to devices inside processor coherent
+	 * domain. PCIe devices are not included. We can skip the rest of the
+	 * flags if IOMMU does not support MTS.
+	 */
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK) {
+		pr_err("No memory type support for bind guest PASID on %s\n",
+			iommu->name);
+		return -EINVAL;
+	}
+
+	return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested translation.
+ * This could be used for guest shared virtual address. In this case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.
+ *
+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in GPA
+ * @pasid:      PASID to be programmed in the device PASID table
+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain:     Domain info for setting up second level page tables
+ * @addr_width: Address width of the first level (guest)
+ */
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct iommu_gpasid_bind_data_vtd *pasid_data,
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation support\n",
+		       iommu->name);
+		return -EINVAL;
+	}
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;
+
+	/*
+	 * Caller must ensure PASID entry is not in use, i.e. not bind the
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;
+
+	pasid_clear_entry(pte);
+
+	/* Sanity checking performed by caller to make sure address
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);
+		} else {
+			dev_err(dev, "5-level paging not supported\n");
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n", addr_width);
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the second level */
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max %llx\n",
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not supported\n");
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
drivers/iommu/intel-pasid.h +12 -0
index 92de6df24ccb..698015ee3f04 100644
@@ -36,6 +36,7 @@
  * to vmalloc or even module mappings.
  */
 #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
 /*
  * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for first-
@@ -51,6 +52,11 @@ struct pasid_entry {
 	u64 val[8];
 };
 
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
 /* The representative of a PASID table */
 struct pasid_table {
 	void			*table;		/* pasid table pointer */
@@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
 int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 				   struct dmar_domain *domain,
 				   struct device *dev, int pasid);
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *pgd,
+			int pasid,
+			struct iommu_gpasid_bind_data_vtd *pasid_data,
+			struct dmar_domain *domain,
+			int addr_width);
 void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 				 struct device *dev, int pasid);
 
include/linux/intel-iommu.h +3 -0
index ed7171d2ae1f..eda1d6687144 100644
@@ -42,6 +42,9 @@
 #define DMA_FL_PTE_PRESENT	BIT_ULL(0)
 #define DMA_FL_PTE_XD		BIT_ULL(63)
 
+#define ADDR_WIDTH_5LEVEL	(57)
+#define ADDR_WIDTH_4LEVEL	(48)
+
 #define CONTEXT_TT_MULTI_LEVEL	0
 #define CONTEXT_TT_DEV_IOTLB	1
 #define CONTEXT_TT_PASS_THROUGH 2
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#18 lore raw

Re: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

I love your patch! Perhaps something to improve:

Hi Jacob,

I love your patch! Perhaps something to improve:

[auto build test WARNING on iommu/next]
[also build test WARNING on linux/master linus/master v5.6-rc7 next-20200325] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system. BTW, we also suggest to use '--base' option to specify the base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Jacob-Pan/Nested-Shared-Virtual-Address-SVA-VT-d-support/20200321-073024
base:   https://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git next
config: i386-randconfig-c001-20200326 (attached as .config)
compiler: gcc-7 (Debian 7.5.0-5) 7.5.0
reproduce:
        # save the attached .config to linux build tree
        make ARCH=i386 

If you fix the issue, kindly add following tag

All warnings (new ones prefixed by >>):

In file included from include/linux/kernel.h:15:0,

                    from include/asm-generic/bug.h:19,
                    from arch/x86/include/asm/bug.h:83,
                    from include/linux/bug.h:5,
                    from include/linux/jump_label.h:250,
                    from include/linux/static_key.h:1,
                    from arch/x86/include/asm/nospec-branch.h:6,
                    from arch/x86/include/asm/paravirt_types.h:46,
                    from arch/x86/include/asm/ptrace.h:94,
                    from arch/x86/include/asm/math_emu.h:5,
                    from arch/x86/include/asm/processor.h:13,
                    from arch/x86/include/asm/cpufeature.h:5,
                    from include/linux/cpufeature.h:13,
                    from drivers/iommu/intel-pasid.c:13:
   drivers/iommu/intel-pasid.c: In function 'iommu_skip_agaw':
   drivers/iommu/intel-pasid.c:588:71: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     pr_debug_ratelimited("%s: pgd: %llx, agaw %d d_agaw %d\n", __func__, (u64)*pgd,
                                                                          ^
   include/linux/printk.h:137:17: note: in definition of macro 'no_printk'
      printk(fmt, ##__VA_ARGS__);  \
                    ^~~~~~~~~~~
   drivers/iommu/intel-pasid.c:588:2: note: in expansion of macro 'pr_debug_ratelimited'
     pr_debug_ratelimited("%s: pgd: %llx, agaw %d d_agaw %d\n", __func__, (u64)*pgd,
     ^~~~~~~~~~~~~~~~~~~~
   In file included from arch/x86/include/asm/bitops.h:16:0,
                    from include/linux/bitops.h:29,
                    from drivers/iommu/intel-pasid.c:12:
   drivers/iommu/intel-pasid.c: In function 'intel_pasid_setup_nested':
   drivers/iommu/intel-pasid.c:812:6: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     if ((u64)gpgd > domain->max_addr) {
         ^
   include/linux/compiler.h:58:52: note: in definition of macro '__trace_if_var'
    #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
                                                       ^~~~
>> drivers/iommu/intel-pasid.c:812:2: note: in expansion of macro 'if'
     if ((u64)gpgd > domain->max_addr) {
     ^~
   drivers/iommu/intel-pasid.c:812:6: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     if ((u64)gpgd > domain->max_addr) {
         ^
   include/linux/compiler.h:58:61: note: in definition of macro '__trace_if_var'
    #define __trace_if_var(cond) (__builtin_constant_p(cond) ? (cond) : __trace_if_value(cond))
                                                                ^~~~
>> drivers/iommu/intel-pasid.c:812:2: note: in expansion of macro 'if'
     if ((u64)gpgd > domain->max_addr) {
     ^~
   drivers/iommu/intel-pasid.c:812:6: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     if ((u64)gpgd > domain->max_addr) {
         ^
   include/linux/compiler.h:69:3: note: in definition of macro '__trace_if_value'
     (cond) ?     \
      ^~~~
   include/linux/compiler.h:56:28: note: in expansion of macro '__trace_if_var'
    #define if(cond, ...) if ( __trace_if_var( !!(cond , ## __VA_ARGS__) ) )
                               ^~~~~~~~~~~~~~
>> drivers/iommu/intel-pasid.c:812:2: note: in expansion of macro 'if'
     if ((u64)gpgd > domain->max_addr) {
     ^~
   In file included from include/linux/device.h:15:0,
                    from include/linux/acpi.h:15,
                    from include/linux/dmar.h:12,
                    from drivers/iommu/intel-pasid.c:14:
   drivers/iommu/intel-pasid.c:814:4: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
       (u64)gpgd, domain->max_addr);
       ^
   include/linux/dev_printk.h:104:32: note: in definition of macro 'dev_err'
     _dev_err(dev, dev_fmt(fmt), ##__VA_ARGS__)
                                   ^~~~~~~~~~~
   drivers/iommu/intel-pasid.c:817:23: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
     pasid_set_flptr(pte, (u64)gpgd);
                          ^

vim +/if +812 drivers/iommu/intel-pasid.c

741
742 /**
743 * intel_pasid_setup_nested() - Set up PASID entry for nested translation.
744 * This could be used for guest shared virtual address. In this case, the
745 * first level page tables are used for GVA-GPA translation in the guest,
746 * second level page tables are used for GPA-HPA translation.
747 *

   748	 * @iommu:      IOMMU which the device belong to
   749	 * @dev:        Device to be set up for translation
   750	 * @gpgd:       FLPTPTR: First Level Page translation pointer in GPA
   751	 * @pasid:      PASID to be programmed in the device PASID table
   752	 * @pasid_data: Additional PASID info from the guest bind request
   753	 * @domain:     Domain info for setting up second level page tables
   754	 * @addr_width: Address width of the first level (guest)
   755	 */
   756	int intel_pasid_setup_nested(struct intel_iommu *iommu,
   757				struct device *dev, pgd_t *gpgd,
   758				int pasid, struct iommu_gpasid_bind_data_vtd *pasid_data,
   759				struct dmar_domain *domain,
   760				int addr_width)
   761	{
   762		struct pasid_entry *pte;
   763		struct dma_pte *pgd;
   764		int ret = 0;
   765		u64 pgd_val;
   766		int agaw;
   767		u16 did;
   768	
   769		if (!ecap_nest(iommu->ecap)) {
   770			pr_err("IOMMU: %s: No nested translation support\n",
   771			       iommu->name);
   772			return -EINVAL;
   773		}
   774	
   775		pte = intel_pasid_get_entry(dev, pasid);
   776		if (WARN_ON(!pte))
   777			return -EINVAL;
   778	
   779		/*
   780		 * Caller must ensure PASID entry is not in use, i.e. not bind the
   781		 * same PASID to the same device twice.
   782		 */
   783		if (pasid_pte_is_present(pte))
   784			return -EBUSY;
   785	
   786		pasid_clear_entry(pte);
   787	
   788		/* Sanity checking performed by caller to make sure address
   789		 * width matching in two dimensions:
   790		 * 1. CPU vs. IOMMU
   791		 * 2. Guest vs. Host.
   792		 */
   793		switch (addr_width) {
   794		case ADDR_WIDTH_5LEVEL:
   795			if (cpu_feature_enabled(X86_FEATURE_LA57) &&
   796				cap_5lp_support(iommu->cap)) {
   797				pasid_set_flpm(pte, 1);
   798			} else {
   799				dev_err(dev, "5-level paging not supported\n");
   800				return -EINVAL;
   801			}
   802			break;
   803		case ADDR_WIDTH_4LEVEL:
   804			pasid_set_flpm(pte, 0);
   805			break;
   806		default:
   807			dev_err(dev, "Invalid guest address width %d\n", addr_width);
   808			return -EINVAL;
   809		}
   810	
   811		/* First level PGD is in GPA, must be supported by the second level */
 > 812		if ((u64)gpgd > domain->max_addr) {

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#19 lore raw

RE: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

In reply to Jacob Pan
Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8.

now the spec is already at rev3.1 😊

Quoted 171 lines

With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
 drivers/iommu/intel-pasid.c | 240
+++++++++++++++++++++++++++++++++++++++++++-
 drivers/iommu/intel-pasid.h |  12 +++
 include/linux/intel-iommu.h |   3 +
 3 files changed, 252 insertions(+), 3 deletions(-)
drivers/iommu/intel-pasid.c +77 -3
index 9bdb7ee228b6..10c7856afc6b 100644
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64 value)
 	pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
 }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value << 27);
+}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value << 32);
+}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
 static void
 pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct intel_iommu
 *iommu,
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
 	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
 	pasid_set_present(pte);
 	pasid_flush_caches(iommu, pte, pasid, did);
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
 	pasid_set_domain_id(pte, did);
 	pasid_set_slptr(pte, pgd_val);
 	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
 	pasid_clear_entry(pte);
 	pasid_set_domain_id(pte, did);
 	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
 
 	return 0;
 }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd

*pasid_data)

drivers/iommu/intel-pasid.c +5 -0
+{
+	/*
+	 * Not all guest PASID table entry fields are passed down during bind,
+	 * here we only set up the ones that are dependent on guest settings.
+	 * Execution related bits such as NXE, SMEP are not meaningful to

IOMMU,

drivers/iommu/intel-pasid.c +1 -0
+	 * therefore not set. Other fields, such as snoop related, are set

based

drivers/iommu/intel-pasid.c +22 -0
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on %s\n",
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support on %s\n",
+				iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
+	 * Memory type is only applicable to devices inside processor

coherent

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 718

Quoted 2 lines
drivers/iommu/intel-pasid.c +2 -0
+	 * domain. PCIe devices are not included. We can skip the rest of the
+	 * flags if IOMMU does not support MTS.

when you say that PCI devices are not included, is it simple for information or should we impose some check to make sure below path not applied to them?

Quoted 20 lines
drivers/iommu/intel-pasid.c +14 -0
+	 */
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK)

{

drivers/iommu/intel-pasid.c +1 -0
+		pr_err("No memory type support for bind guest PASID

on %s\n",

drivers/iommu/intel-pasid.c +3 -0
+			iommu->name);
+		return -EINVAL;
+	}

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 746

Quoted 10 lines
drivers/iommu/intel-pasid.c +10 -0
+
+	return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested translation.
+ * This could be used for guest shared virtual address. In this case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.

GVA->GPA is just one example. It could be gIOVA->GPA too. Here the point is that the first level is the translation table managed by the guest.

Quoted 22 lines

+ *

+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in GPA
+ * @pasid:      PASID to be programmed in the device PASID table

+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain: Domain info for setting up second level page tables
+ * @addr_width: Address width of the first level (guest)
+ */

drivers/iommu/intel-pasid.c +3 -0
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct iommu_gpasid_bind_data_vtd

*pasid_data,

drivers/iommu/intel-pasid.c +9 -0
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 777

Quoted 10 lines
drivers/iommu/intel-pasid.c +10 -0
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation support\n",
+		       iommu->name);
+		return -EINVAL;
+	}
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;

should we have intel_pasid_get_entry to return error which is then carried here? Looking at that function there could be error conditions both being invalid parameter and no memory...

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 784

Quoted 7 lines
drivers/iommu/intel-pasid.c +7 -0
+
+	/*
+	 * Caller must ensure PASID entry is not in use, i.e. not bind the
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;

is any lock held outside of this function? curious whether any race condition may happen in between.

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 797

Quoted 13 lines
drivers/iommu/intel-pasid.c +13 -0
+
+	pasid_clear_entry(pte);
+
+	/* Sanity checking performed by caller to make sure address
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);

define a macro for 4lvl and 5lvl

Quoted 105 lines
drivers/iommu/intel-pasid.c +10 -0
+		} else {
+			dev_err(dev, "5-level paging not supported\n");
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n",

addr_width);

drivers/iommu/intel-pasid.c +41 -0
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the second level */
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max %llx\n",
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not supported\n");
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
drivers/iommu/intel-pasid.h +12 -0
index 92de6df24ccb..698015ee3f04 100644
@@ -36,6 +36,7 @@
  * to vmalloc or even module mappings.
  */
 #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
 /*
  * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for first-
@@ -51,6 +52,11 @@ struct pasid_entry {
 	u64 val[8];
 };
 
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
 /* The representative of a PASID table */
 struct pasid_table {
 	void			*table;		/* pasid table pointer */
@@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
 int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 				   struct dmar_domain *domain,
 				   struct device *dev, int pasid);
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *pgd,
+			int pasid,
+			struct iommu_gpasid_bind_data_vtd *pasid_data,
+			struct dmar_domain *domain,
+			int addr_width);
 void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 				 struct device *dev, int pasid);
include/linux/intel-iommu.h +3 -1
index ed7171d2ae1f..eda1d6687144 100644
@@ -42,6 +42,9 @@
 #define DMA_FL_PTE_PRESENT	BIT_ULL(0)
 #define DMA_FL_PTE_XD		BIT_ULL(63)
 
+#define ADDR_WIDTH_5LEVEL	(57)
+#define ADDR_WIDTH_4LEVEL	(48)
+
 #define CONTEXT_TT_MULTI_LEVEL	0
 #define CONTEXT_TT_DEV_IOTLB	1
 #define CONTEXT_TT_PASS_THROUGH 2
--
Quoted 1 line

2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#20 lore raw

Re: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

Quoted 6 lines · Kevin
Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8.

now the spec is already at rev3.1 😊

Updated.

Quoted 170 lines
Quoted 170 lines

With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
  drivers/iommu/intel-pasid.c | 240
+++++++++++++++++++++++++++++++++++++++++++-
  drivers/iommu/intel-pasid.h |  12 +++
  include/linux/intel-iommu.h |   3 +
  3 files changed, 252 insertions(+), 3 deletions(-)
drivers/iommu/intel-pasid.c +77 -3
index 9bdb7ee228b6..10c7856afc6b 100644
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64 value)
  	pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
  }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value << 27);
+}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value << 32);
+}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
  static void
  pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
  				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct intel_iommu
 *iommu,
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
  	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
  	pasid_set_present(pte);
  	pasid_flush_caches(iommu, pte, pasid, did);
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
  	pasid_set_domain_id(pte, did);
  	pasid_set_slptr(pte, pgd_val);
  	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
  	pasid_set_fault_enable(pte);
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
  	pasid_clear_entry(pte);
  	pasid_set_domain_id(pte, did);
  	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
  	pasid_set_fault_enable(pte);
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
 
  	return 0;
  }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd

*pasid_data)

drivers/iommu/intel-pasid.c +5 -0
+{
+	/*
+	 * Not all guest PASID table entry fields are passed down during bind,
+	 * here we only set up the ones that are dependent on guest settings.
+	 * Execution related bits such as NXE, SMEP are not meaningful to

IOMMU,

drivers/iommu/intel-pasid.c +1 -0
+	 * therefore not set. Other fields, such as snoop related, are set

based

drivers/iommu/intel-pasid.c +21 -0
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on %s\n",
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support on %s\n",
+				iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 718

Quoted 4 lines
drivers/iommu/intel-pasid.c +1 -0
+	 * Memory type is only applicable to devices inside processor

coherent

drivers/iommu/intel-pasid.c +2 -0
+	 * domain. PCIe devices are not included. We can skip the rest of the
+	 * flags if IOMMU does not support MTS.

when you say that PCI devices are not included, is it simple for information or should we impose some check to make sure below path not applied to them?

Jacob, does it work for you if I add below check?

	if (ecap_mts(iommu->ecap) && !dev_is_pci(dev))

Or, we need to remove this comment line?

Quoted 26 lines
Quoted 26 lines
drivers/iommu/intel-pasid.c +14 -0
+	 */
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK)

{

drivers/iommu/intel-pasid.c +1 -0
+		pr_err("No memory type support for bind guest PASID

on %s\n",

drivers/iommu/intel-pasid.c +8 -0
+			iommu->name);
+		return -EINVAL;
+	}
+
+	return 0;
+
+}
+
Quoted 8 lines
Quoted 5 lines

+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested translation.
+ * This could be used for guest shared virtual address. In this case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.

GVA->GPA is just one example. It could be gIOVA->GPA too. Here the point is that the first level is the translation table managed by the guest.

Agreed.

Quoted 29 lines
Quoted 29 lines

+ *

+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in GPA
+ * @pasid:      PASID to be programmed in the device PASID table

+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain: Domain info for setting up second level page tables
+ * @addr_width: Address width of the first level (guest)
+ */

drivers/iommu/intel-pasid.c +3 -0
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct iommu_gpasid_bind_data_vtd

*pasid_data,

drivers/iommu/intel-pasid.c +15 -0
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation support\n",
+		       iommu->name);
+		return -EINVAL;
+	}
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 777

Quoted 4 lines
drivers/iommu/intel-pasid.c +4 -0
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;

should we have intel_pasid_get_entry to return error which is then carried here? Looking at that function there could be error conditions both being invalid parameter and no memory...

Agreed. Will do this in a followup patch.

Quoted 11 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 784

Quoted 8 lines
drivers/iommu/intel-pasid.c +7 -0
+
+	/*
+	 * Caller must ensure PASID entry is not in use, i.e. not bind the
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;

is any lock held outside of this function? curious whether any race condition may happen in between.

The pasid entry change should always be protected by iommu->lock.

Quoted 115 lines
Quoted 4 lines
drivers/iommu/intel-pasid.c +3 -0
+
+	pasid_clear_entry(pte);
+

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 797

Quoted 10 lines
drivers/iommu/intel-pasid.c +10 -0
+	/* Sanity checking performed by caller to make sure address
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);

define a macro for 4lvl and 5lvl

Quoted 98 lines
drivers/iommu/intel-pasid.c +10 -0
+		} else {
+			dev_err(dev, "5-level paging not supported\n");
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n",

addr_width);

drivers/iommu/intel-pasid.c +41 -0
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the second level */
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max %llx\n",
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not supported\n");
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
drivers/iommu/intel-pasid.h +12 -0
index 92de6df24ccb..698015ee3f04 100644
@@ -36,6 +36,7 @@
   * to vmalloc or even module mappings.
   */
  #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
  /*
   * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for first-
@@ -51,6 +52,11 @@ struct pasid_entry {
  	u64 val[8];
  };
 
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
  /* The representative of a PASID table */
  struct pasid_table {
  	void			*table;		/* pasid table pointer */
@@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
  int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
  				   struct dmar_domain *domain,
  				   struct device *dev, int pasid);
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *pgd,
+			int pasid,
+			struct iommu_gpasid_bind_data_vtd *pasid_data,
+			struct dmar_domain *domain,
+			int addr_width);
  void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
  				 struct device *dev, int pasid);
include/linux/intel-iommu.h +0 -0
index ed7171d2ae1f..eda1d6687144 100644
@@ -42,6 +42,9 @@
  #define DMA_FL_PTE_PRESENT	BIT_ULL(0)
  #define DMA_FL_PTE_XD		BIT_ULL(63)
 
Quoted 9 lines
Quoted 8 lines
include/linux/intel-iommu.h +3 -1
+#define ADDR_WIDTH_5LEVEL	(57)
+#define ADDR_WIDTH_4LEVEL	(48)
+
  #define CONTEXT_TT_MULTI_LEVEL	0
  #define CONTEXT_TT_DEV_IOTLB	1
  #define CONTEXT_TT_PASS_THROUGH 2
--

2.7.4

Best regards,
baolu

author
#21 lore raw

Re: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

Quoted 240 lines · On Sun, 29 Mar 2020 16:03:36 +0800 Lu Baolu <[email protected]> wrote:
Quoted 6 lines · Kevin
Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8.

now the spec is already at rev3.1 😊

Updated.

Quoted 175 lines
Quoted 175 lines

With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
  drivers/iommu/intel-pasid.c | 240
+++++++++++++++++++++++++++++++++++++++++++-
  drivers/iommu/intel-pasid.h |  12 +++
  include/linux/intel-iommu.h |   3 +
  3 files changed, 252 insertions(+), 3 deletions(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 9bdb7ee228b6..10c7856afc6b 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +75 -3
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64
 value) pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
  }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value <<
 27); +}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value <<
 32); +}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
  static void
  pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
  				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct
 intel_iommu *iommu,
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
  	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
  	pasid_set_present(pte);
  	pasid_flush_caches(iommu, pte, pasid, did);
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
  	pasid_set_domain_id(pte, did);
  	pasid_set_slptr(pte, pgd_val);
  	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
  	pasid_set_fault_enable(pte);
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
  	pasid_clear_entry(pte);
  	pasid_set_domain_id(pte, did);
  	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
  	pasid_set_fault_enable(pte);
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
 
  	return 0;
  }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd

*pasid_data)

drivers/iommu/intel-pasid.c +3 -0
+{
+	/*
+	 * Not all guest PASID table entry fields are passed down

during bind,

(quoted diff) +1 -0
+	 * here we only set up the ones that are dependent on

guest settings.

(quoted diff) +1 -0
+	 * Execution related bits such as NXE, SMEP are not

meaningful to IOMMU,

(quoted diff) +1 -0
+	 * therefore not set. Other fields, such as snoop

related, are set based

drivers/iommu/intel-pasid.c +5 -0
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on

%s\n",

drivers/iommu/intel-pasid.c +9 -0
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support

on %s\n",

drivers/iommu/intel-pasid.c +8 -0
+				iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
+	 * Memory type is only applicable to devices inside
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 718

Quoted 4 lines

processor coherent

(quoted diff) +1 -0
+	 * domain. PCIe devices are not included. We can skip the

rest of the

drivers/iommu/intel-pasid.c +1 -0
+	 * flags if IOMMU does not support MTS.  

when you say that PCI devices are not included, is it simple for information or should we impose some check to make sure below path not applied to them?

Jacob, does it work for you if I add below check?

	if (ecap_mts(iommu->ecap) && !dev_is_pci(dev))

Or, we need to remove this comment line?

Quoted 31 lines
Quoted 31 lines
drivers/iommu/intel-pasid.c +3 -0
+	 */
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags &

IOMMU_SVA_VTD_GPASID_EMTE) {

drivers/iommu/intel-pasid.c +11 -0
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags &

IOMMU_SVA_VTD_GPASID_MTS_MASK) {

(quoted diff) +1 -0
+		pr_err("No memory type support for bind guest

PASID on %s\n",

drivers/iommu/intel-pasid.c +10 -0
+			iommu->name);
+		return -EINVAL;
+	}
+
+	return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested

translation.
+ * This could be used for guest shared virtual address. In this

Quoted 8 lines
Quoted 4 lines

case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.

GVA->GPA is just one example. It could be gIOVA->GPA too. Here the point is that the first level is the translation table managed by the guest.

Quoted 2 lines

Agreed.

Yes, that is why I chose the word "could be" :), but mention both cases are good.

Quoted 56 lines
Quoted 32 lines
Quoted 32 lines

+ *

+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in

GPA
+ * @pasid: PASID to be programmed in the device PASID table
+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain: Domain info for setting up second level page
tables
+ * @addr_width: Address width of the first level (guest)
+ */

drivers/iommu/intel-pasid.c +3 -0
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct

iommu_gpasid_bind_data_vtd *pasid_data,

drivers/iommu/intel-pasid.c +12 -0
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation

support\n",

drivers/iommu/intel-pasid.c +3 -0
+		       iommu->name);
+		return -EINVAL;
+	}
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 777

Quoted 4 lines
drivers/iommu/intel-pasid.c +4 -0
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;  

should we have intel_pasid_get_entry to return error which is then carried here? Looking at that function there could be error conditions both being invalid parameter and no memory...

Agreed. Will do this in a followup patch.

Quoted 12 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 784

Quoted 9 lines
drivers/iommu/intel-pasid.c +3 -0
+
+	/*
+	 * Caller must ensure PASID entry is not in use, i.e. not

bind the

drivers/iommu/intel-pasid.c +4 -0
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;  

is any lock held outside of this function? curious whether any race condition may happen in between.

Quoted 2 lines

The pasid entry change should always be protected by iommu->lock.

Agreed.

Quoted 131 lines
Quoted 121 lines
Quoted 6 lines
drivers/iommu/intel-pasid.c +4 -0
+
+	pasid_clear_entry(pte);
+
+	/* Sanity checking performed by caller to make sure

address

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 797

Quoted 9 lines
drivers/iommu/intel-pasid.c +9 -0
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);  

define a macro for 4lvl and 5lvl

Quoted 103 lines
(quoted diff) +2 -0
+		} else {
+			dev_err(dev, "5-level paging not

supported\n");

drivers/iommu/intel-pasid.c +8 -0
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n",

addr_width);

drivers/iommu/intel-pasid.c +4 -0
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the

second level */

(quoted diff) +2 -0
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max

%llx\n",

drivers/iommu/intel-pasid.c +8 -0
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not

supported\n");

drivers/iommu/intel-pasid.c +27 -0
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.h index 92de6df24ccb..698015ee3f04 100644 --- a/drivers/iommu/intel-pasid.h
+++ b/drivers/iommu/intel-pasid.h

drivers/iommu/intel-pasid.h +1 -0
@@ -36,6 +36,7 @@
   * to vmalloc or even module mappings.
   */
  #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
  /*
   * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for

first- @@ -51,6 +52,11 @@ struct pasid_entry {
u64 val[8];
};

drivers/iommu/intel-pasid.h +5 -0
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
  /* The representative of a PASID table */
  struct pasid_table {
  	void			*table;		/*

pasid table pointer */ @@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
struct dmar_domain *domain, struct device *dev, int
pasid); +int intel_pasid_setup_nested(struct intel_iommu *iommu,

drivers/iommu/intel-pasid.h +3 -0
+			struct device *dev, pgd_t *pgd,
+			int pasid,
+			struct iommu_gpasid_bind_data_vtd

*pasid_data,

drivers/iommu/intel-pasid.h +2 -0
+			struct dmar_domain *domain,
+			int addr_width);
  void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
  				 struct device *dev, int pasid);
(quoted diff) +0 -0

b/include/linux/intel-iommu.h index ed7171d2ae1f..eda1d6687144 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

include/linux/intel-iommu.h +0 -0
@@ -42,6 +42,9 @@
  #define DMA_FL_PTE_PRESENT	BIT_ULL(0)
  #define DMA_FL_PTE_XD		BIT_ULL(63)
 
Quoted 9 lines
Quoted 8 lines
include/linux/intel-iommu.h +3 -1
+#define ADDR_WIDTH_5LEVEL	(57)
+#define ADDR_WIDTH_4LEVEL	(48)
+
  #define CONTEXT_TT_MULTI_LEVEL	0
  #define CONTEXT_TT_DEV_IOTLB	1
  #define CONTEXT_TT_PASS_THROUGH 2
--

2.7.4

Quoted 2 lines

Best regards,
baolu

[Jacob Pan]

#22 lore raw

RE: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

Quoted 246 lines

From: Jacob Pan <[email protected]>
Sent: Tuesday, March 31, 2020 2:22 AM

Quoted 240 lines · On Sun, 29 Mar 2020 16:03:36 +0800 Lu Baolu <[email protected]> wrote:
Quoted 6 lines · Kevin
Quoted 4 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8.

now the spec is already at rev3.1 😊

Updated.

Quoted 175 lines
Quoted 175 lines

With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
  drivers/iommu/intel-pasid.c | 240
+++++++++++++++++++++++++++++++++++++++++++-
  drivers/iommu/intel-pasid.h |  12 +++
  include/linux/intel-iommu.h |   3 +
  3 files changed, 252 insertions(+), 3 deletions(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 9bdb7ee228b6..10c7856afc6b 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +75 -3
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64
 value) pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
  }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value <<
 27); +}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value <<
 32); +}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
  static void
  pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
  				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct
 intel_iommu *iommu,
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
  	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
  	pasid_set_present(pte);
  	pasid_flush_caches(iommu, pte, pasid, did);
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu,
  	pasid_set_domain_id(pte, did);
  	pasid_set_slptr(pte, pgd_val);
  	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
  	pasid_set_fault_enable(pte);
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
  	pasid_clear_entry(pte);
  	pasid_set_domain_id(pte, did);
  	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
  	pasid_set_fault_enable(pte);
  	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu,
 
  	return 0;
  }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd

*pasid_data)

drivers/iommu/intel-pasid.c +3 -0
+{
+	/*
+	 * Not all guest PASID table entry fields are passed down

during bind,

(quoted diff) +1 -0
+	 * here we only set up the ones that are dependent on

guest settings.

(quoted diff) +1 -0
+	 * Execution related bits such as NXE, SMEP are not

meaningful to IOMMU,

(quoted diff) +1 -0
+	 * therefore not set. Other fields, such as snoop

related, are set based

drivers/iommu/intel-pasid.c +5 -0
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on

%s\n",

drivers/iommu/intel-pasid.c +9 -0
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support

on %s\n",

drivers/iommu/intel-pasid.c +8 -0
+				iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
+	 * Memory type is only applicable to devices inside
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 718

Quoted 4 lines

processor coherent

(quoted diff) +1 -0
+	 * domain. PCIe devices are not included. We can skip the

rest of the

drivers/iommu/intel-pasid.c +1 -0
+	 * flags if IOMMU does not support MTS.

when you say that PCI devices are not included, is it simple for information or should we impose some check to make sure below path not applied to them?

Jacob, does it work for you if I add below check?

	if (ecap_mts(iommu->ecap) && !dev_is_pci(dev))

Or, we need to remove this comment line?

Quoted 39 lines
Quoted 29 lines
drivers/iommu/intel-pasid.c +3 -0
+	 */
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags &

IOMMU_SVA_VTD_GPASID_EMTE) {

drivers/iommu/intel-pasid.c +11 -0
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags &

IOMMU_SVA_VTD_GPASID_MTS_MASK) {

(quoted diff) +1 -0
+		pr_err("No memory type support for bind guest

PASID on %s\n",

drivers/iommu/intel-pasid.c +10 -0
+			iommu->name);
+		return -EINVAL;
+	}
+
+	return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested
Quoted 6 lines

translation.
+ * This could be used for guest shared virtual address. In this case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.

GVA->GPA is just one example. It could be gIOVA->GPA too. Here the point is that the first level is the translation table managed by the guest.

Quoted 4 lines
Quoted 2 lines

Agreed.

Yes, that is why I chose the word "could be" :), but mention both cases are good.

"could be" is fine. I read too fast.

Quoted 57 lines
Quoted 57 lines
Quoted 33 lines
Quoted 33 lines

+ *

+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in

GPA
+ * @pasid: PASID to be programmed in the device PASID table
+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain: Domain info for setting up second level page
tables
+ * @addr_width: Address width of the first level (guest)
+ */

drivers/iommu/intel-pasid.c +3 -0
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct

iommu_gpasid_bind_data_vtd *pasid_data,

drivers/iommu/intel-pasid.c +12 -0
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation

support\n",

drivers/iommu/intel-pasid.c +3 -0
+		       iommu->name);
+		return -EINVAL;
+	}
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 777

Quoted 4 lines
drivers/iommu/intel-pasid.c +4 -0
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;

should we have intel_pasid_get_entry to return error which is then carried here? Looking at that function there could be error conditions both being invalid parameter and no memory...

Agreed. Will do this in a followup patch.

Quoted 12 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 784

Quoted 9 lines
drivers/iommu/intel-pasid.c +3 -0
+
+	/*
+	 * Caller must ensure PASID entry is not in use, i.e. not

bind the

drivers/iommu/intel-pasid.c +4 -0
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;

is any lock held outside of this function? curious whether any race condition may happen in between.

Quoted 3 lines
Quoted 2 lines

The pasid entry change should always be protected by iommu->lock.

Agreed.

yes, I realized it after completing the review of whole series. 😊

Quoted 136 lines
Quoted 132 lines
Quoted 122 lines
Quoted 7 lines
drivers/iommu/intel-pasid.c +4 -0
+
+	pasid_clear_entry(pte);
+
+	/* Sanity checking performed by caller to make sure

address

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 797

Quoted 9 lines
drivers/iommu/intel-pasid.c +9 -0
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);

define a macro for 4lvl and 5lvl

Quoted 103 lines
(quoted diff) +2 -0
+		} else {
+			dev_err(dev, "5-level paging not

supported\n");

drivers/iommu/intel-pasid.c +8 -0
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n",

addr_width);

drivers/iommu/intel-pasid.c +4 -0
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the

second level */

(quoted diff) +2 -0
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max

%llx\n",

drivers/iommu/intel-pasid.c +8 -0
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not

supported\n");

drivers/iommu/intel-pasid.c +27 -0
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.h index 92de6df24ccb..698015ee3f04 100644 --- a/drivers/iommu/intel-pasid.h
+++ b/drivers/iommu/intel-pasid.h

drivers/iommu/intel-pasid.h +1 -0
@@ -36,6 +36,7 @@
   * to vmalloc or even module mappings.
   */
  #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
  /*
   * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for

first- @@ -51,6 +52,11 @@ struct pasid_entry {
u64 val[8];
};

drivers/iommu/intel-pasid.h +5 -0
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

pasid table pointer */ @@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
struct dmar_domain *domain, struct device *dev, int
pasid); +int intel_pasid_setup_nested(struct intel_iommu *iommu,

drivers/iommu/intel-pasid.h +3 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

*pasid_data,

drivers/iommu/intel-pasid.h +2 -0

Lines of this file are not shown. View the raw message.

(quoted diff) +0 -0

The rest of this diff is not shown. View the raw message.

3 lines not shown · raw message

include/linux/intel-iommu.h +0 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Quoted 9 lines

8 lines not shown · raw message

Quoted 2 lines

2 lines not shown · raw message

1 line not shown · raw message

Parts of this message are not shown here. View the raw message.

#23 lore raw

Re: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

In reply to Jacob Pan

Hi Jacob,

Quoted 119 lines · Jacob Pan

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8. With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
 drivers/iommu/intel-pasid.c | 240 +++++++++++++++++++++++++++++++++++++++++++-
 drivers/iommu/intel-pasid.h |  12 +++
 include/linux/intel-iommu.h |   3 +
 3 files changed, 252 insertions(+), 3 deletions(-)
drivers/iommu/intel-pasid.c +72 -2
index 9bdb7ee228b6..10c7856afc6b 100644
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64 value)
 	pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
 }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value << 27);
+}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value << 32);
+}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
 static void
 pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 
 	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
 	pasid_set_present(pte);
 	pasid_flush_caches(iommu, pte, pasid, did);
 
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
 	pasid_set_domain_id(pte, did);
 	pasid_set_slptr(pte, pgd_val);
 	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 673

Quoted 7 lines
drivers/iommu/intel-pasid.c +1 -1
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 	pasid_clear_entry(pte);
 	pasid_set_domain_id(pte, did);
 	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));

All above looks good to me

Quoted 30 lines
drivers/iommu/intel-pasid.c +25 -0
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 
 	return 0;
 }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd *pasid_data)
+{
+	/*
+	 * Not all guest PASID table entry fields are passed down during bind,
+	 * here we only set up the ones that are dependent on guest settings.
+	 * Execution related bits such as NXE, SMEP are not meaningful to IOMMU,
+	 * therefore not set. Other fields, such as snoop related, are set based
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on %s\n",
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support on %s\n",
+				iommu->name);

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 719

Quoted 10 lines
drivers/iommu/intel-pasid.c +10 -0
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
+	 * Memory type is only applicable to devices inside processor coherent
+	 * domain. PCIe devices are not included. We can skip the rest of the
+	 * flags if IOMMU does not support MTS.
+	 */

nit:

	if (!pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK)
		return 0;

	if (!ecap_mts(iommu->ecap) {
		pr_err("No memory type support for bind guest PASID on %s\n",
		iommu->name);
		return -EINVAL;
	}

	settings ../..
Quoted 44 lines
drivers/iommu/intel-pasid.c +44 -0
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK) {
+		pr_err("No memory type support for bind guest PASID on %s\n",
+			iommu->name);
+		return -EINVAL;
+	}
+
+	return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested translation.
+ * This could be used for guest shared virtual address. In this case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.
+ *
+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in GPA
+ * @pasid:      PASID to be programmed in the device PASID table
+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain:     Domain info for setting up second level page tables
+ * @addr_width: Address width of the first level (guest)
+ */
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct iommu_gpasid_bind_data_vtd *pasid_data,
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 773

Quoted 10 lines
drivers/iommu/intel-pasid.c +10 -0
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation support\n",
+		       iommu->name);
+		return -EINVAL;
+	}

I am surprised you don't check that the dmar_domain has the DOMAIN_FLAG_NESTED_MODE flag (ir I missed it). Don't you have any risk that the userspace overwrites the PTE of a device attached to an usual domain, ie. fulled handled by the host?

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 784

Quoted 11 lines
drivers/iommu/intel-pasid.c +11 -0
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;
+
+	/*
+	 * Caller must ensure PASID entry is not in use, i.e. not bind the
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;

Here you check the PTE is not valid, is it sufficient to guarantee the above? Also refering to the race potential issue pointed out by Kevin.

Quoted 27 lines
drivers/iommu/intel-pasid.c +27 -0
+
+	pasid_clear_entry(pte);
+
+	/* Sanity checking performed by caller to make sure address
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);
+		} else {
+			dev_err(dev, "5-level paging not supported\n");
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n", addr_width);
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the second level */

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 821

Quoted 10 lines
drivers/iommu/intel-pasid.c +10 -0
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max %llx\n",
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not supported\n");

Shall we output all those traces without limit? They are triggered by userspace, meaning this latter can trigger a storm of those.

Quoted 70 lines
drivers/iommu/intel-pasid.c +27 -0
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
drivers/iommu/intel-pasid.h +12 -0
index 92de6df24ccb..698015ee3f04 100644
@@ -36,6 +36,7 @@
  * to vmalloc or even module mappings.
  */
 #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
 /*
  * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for first-
@@ -51,6 +52,11 @@ struct pasid_entry {
 	u64 val[8];
 };
 
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
 /* The representative of a PASID table */
 struct pasid_table {
 	void			*table;		/* pasid table pointer */
@@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
 int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
 				   struct dmar_domain *domain,
 				   struct device *dev, int pasid);
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *pgd,
+			int pasid,
+			struct iommu_gpasid_bind_data_vtd *pasid_data,
+			struct dmar_domain *domain,
+			int addr_width);
 void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 				 struct device *dev, int pasid);
 
include/linux/intel-iommu.h +0 -0
index ed7171d2ae1f..eda1d6687144 100644

include/linux/intel-iommu.h line 50

Quoted 9 lines
include/linux/intel-iommu.h +3 -0
@@ -42,6 +42,9 @@
 #define DMA_FL_PTE_PRESENT	BIT_ULL(0)
 #define DMA_FL_PTE_XD		BIT_ULL(63)
 
+#define ADDR_WIDTH_5LEVEL	(57)
+#define ADDR_WIDTH_4LEVEL	(48)
+
 #define CONTEXT_TT_MULTI_LEVEL	0
 #define CONTEXT_TT_DEV_IOTLB	1
 #define CONTEXT_TT_PASS_THROUGH 2

Thanks

Eric

author
#24 lore raw

Re: [PATCH V10 05/11] iommu/vt-d: Add nested translation helper function

Quoted 180 lines · On Sun, 29 Mar 2020 13:35:15 +0200 Auger Eric <[email protected]> wrote:

Hi Jacob,

Quoted 119 lines · Jacob Pan

Nested translation mode is supported in VT-d 3.0 Spec.CH 3.8. With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.

This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.

---
 drivers/iommu/intel-pasid.c | 240

+++++++++++++++++++++++++++++++++++++++++++-
drivers/iommu/intel-pasid.h | 12 +++ include/linux/intel-iommu.h
| 3 + 3 files changed, 252 insertions(+), 3 deletions(-)

(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 9bdb7ee228b6..10c7856afc6b 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +70 -2
@@ -359,6 +359,76 @@ pasid_set_flpm(struct pasid_entry *pe, u64
 value) pasid_set_bits(&pe->val[2], GENMASK_ULL(3, 2), value << 2);
 }
 
+/*
+ * Setup the Extended Memory Type(EMT) field (Bits 91-93)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emt(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(29, 27), value <<
 27); +}
+
+/*
+ * Setup the Page Attribute Table (PAT) field (Bits 96-127)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pat(struct pasid_entry *pe, u64 value)
+{
+	pasid_set_bits(&pe->val[1], GENMASK_ULL(63, 32), value <<
 32); +}
+
+/*
+ * Setup the Cache Disable (CD) field (Bit 89)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_cd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 25, 1 << 25);
+}
+
+/*
+ * Setup the Extended Memory Type Enable (EMTE) field (Bit 90)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_emte(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 26, 1 << 26);
+}
+
+/*
+ * Setup the Extended Access Flag Enable (EAFE) field (Bit 135)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_eafe(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[2], 1 << 7, 1 << 7);
+}
+
+/*
+ * Setup the Page-level Cache Disable (PCD) field (Bit 95)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pcd(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 31, 1 << 31);
+}
+
+/*
+ * Setup the Page-level Write-Through (PWT)) field (Bit 94)
+ * of a scalable mode PASID entry.
+ */
+static inline void
+pasid_set_pwt(struct pasid_entry *pe)
+{
+	pasid_set_bits(&pe->val[1], 1 << 30, 1 << 30);
+}
+
 static void
 pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 				    u16 did, int pasid)
@@ -492,7 +562,7 @@ int intel_pasid_setup_first_level(struct
 intel_iommu *iommu,
 pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap)); 
 	/* Setup Present and PASID Granular Transfer Type: */
-	pasid_set_translation_type(pte, 1);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_FL_ONLY);
 	pasid_set_present(pte);
 	pasid_flush_caches(iommu, pte, pasid, did);
 
@@ -564,7 +634,7 @@ int intel_pasid_setup_second_level(struct
 intel_iommu *iommu, pasid_set_domain_id(pte, did);
 	pasid_set_slptr(pte, pgd_val);
 	pasid_set_address_width(pte, agaw);
-	pasid_set_translation_type(pte, 2);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
 

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct · line 673

Quoted 7 lines
drivers/iommu/intel-pasid.c +1 -1
@@ -598,7 +668,7 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu, pasid_clear_entry(pte);
 	pasid_set_domain_id(pte, did);
 	pasid_set_address_width(pte, iommu->agaw);
-	pasid_set_translation_type(pte, 4);
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_PT);
 	pasid_set_fault_enable(pte);
 	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));  

All above looks good to me

Quoted 46 lines
drivers/iommu/intel-pasid.c +4 -0
@@ -612,3 +682,167 @@ int intel_pasid_setup_pass_through(struct
 intel_iommu *iommu, 
 	return 0;
 }
+
+static int intel_pasid_setup_bind_data(struct intel_iommu *iommu,
+				struct pasid_entry *pte,
+				struct iommu_gpasid_bind_data_vtd

*pasid_data) +{

(quoted diff) +2 -0
+	/*
+	 * Not all guest PASID table entry fields are passed down

during bind,

(quoted diff) +1 -0
+	 * here we only set up the ones that are dependent on

guest settings.

(quoted diff) +1 -0
+	 * Execution related bits such as NXE, SMEP are not

meaningful to IOMMU,

(quoted diff) +1 -0
+	 * therefore not set. Other fields, such as snoop related,

are set based

drivers/iommu/intel-pasid.c +5 -0
+	 * on host needs regardless of guest settings.
+	 */
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_SRE) {
+		if (!ecap_srs(iommu->ecap)) {
+			pr_err("No supervisor request support on

%s\n",

drivers/iommu/intel-pasid.c +9 -0
+			       iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_sre(pte);
+	}
+
+	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EAFE) {
+		if (!ecap_eafs(iommu->ecap)) {
+			pr_err("No extended access flag support on

%s\n",

drivers/iommu/intel-pasid.c +8 -0
+				iommu->name);
+			return -EINVAL;
+		}
+		pasid_set_eafe(pte);
+	}
+
+	/*
+	 * Memory type is only applicable to devices inside

processor coherent

(quoted diff) +1 -0
+	 * domain. PCIe devices are not included. We can skip the

rest of the

Review

Quoted 2 lines
(quoted diff) +2 -0
+	 * flags if IOMMU does not support MTS.
+	 */  

nit:

Quoted 10 lines
	if (!pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK)
		return 0;

	if (!ecap_mts(iommu->ecap) {
		pr_err("No memory type support for bind guest PASID
on %s\n", iommu->name);
		return -EINVAL;
	}

	settings ../..

Looks better in flow. Will change to:

	if (!(pasid_data->flags & IOMMU_SVA_VTD_GPASID_MTS_MASK))
		return 0;

	if (!ecap_mts(iommu->ecap)) {
		pr_err("No memory type support for bind guest PASID on %s\n", iommu->name);
		return -EINVAL;
	}

	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE) {
		pasid_set_emte(pte);
		pasid_set_emt(pte, pasid_data->emt);
	}
	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
		pasid_set_pcd(pte);
	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
		pasid_set_pwt(pte);
	if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
		pasid_set_cd(pte);
	pasid_set_pat(pte, pasid_data->pat);
Quoted 59 lines
Quoted 59 lines
(quoted diff) +2 -0
+	if (ecap_mts(iommu->ecap)) {
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_EMTE)

{

drivers/iommu/intel-pasid.c +11 -0
+			pasid_set_emte(pte);
+			pasid_set_emt(pte, pasid_data->emt);
+		}
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PCD)
+			pasid_set_pcd(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_PWT)
+			pasid_set_pwt(pte);
+		if (pasid_data->flags & IOMMU_SVA_VTD_GPASID_CD)
+			pasid_set_cd(pte);
+		pasid_set_pat(pte, pasid_data->pat);
+	} else if (pasid_data->flags &

IOMMU_SVA_VTD_GPASID_MTS_MASK) {

(quoted diff) +1 -0
+		pr_err("No memory type support for bind guest

PASID on %s\n",

drivers/iommu/intel-pasid.c +10 -0
+			iommu->name);
+		return -EINVAL;
+	}
+
+	return 0;
+
+}
+
+/**
+ * intel_pasid_setup_nested() - Set up PASID entry for nested

translation.
+ * This could be used for guest shared virtual address. In this case, the
+ * first level page tables are used for GVA-GPA translation in the guest,
+ * second level page tables are used for GPA-HPA translation.
+ *

+ * @iommu:      IOMMU which the device belong to
+ * @dev:        Device to be set up for translation
+ * @gpgd:       FLPTPTR: First Level Page translation pointer in

GPA
+ * @pasid: PASID to be programmed in the device PASID table
+ * @pasid_data: Additional PASID info from the guest bind request
+ * @domain: Domain info for setting up second level page tables
+ * @addr_width: Address width of the first level (guest)
+ */

drivers/iommu/intel-pasid.c +3 -0
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
+			struct device *dev, pgd_t *gpgd,
+			int pasid, struct

iommu_gpasid_bind_data_vtd *pasid_data,

drivers/iommu/intel-pasid.c +12 -0
+			struct dmar_domain *domain,
+			int addr_width)
+{
+	struct pasid_entry *pte;
+	struct dma_pte *pgd;
+	int ret = 0;
+	u64 pgd_val;
+	int agaw;
+	u16 did;
+
+	if (!ecap_nest(iommu->ecap)) {
+		pr_err("IOMMU: %s: No nested translation
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 702

Quoted 4 lines

support\n",

drivers/iommu/intel-pasid.c +3 -0
+		       iommu->name);
+		return -EINVAL;
+	}  

I am surprised you don't check that the dmar_domain has the DOMAIN_FLAG_NESTED_MODE flag (ir I missed it). Don't you have any risk that the userspace overwrites the PTE of a device attached to an usual domain, ie. fulled handled by the host?

Good point, this domain attr is recently added to vt-d. So I will add the check and move the flags to intel-iommu.h

	if (!(domain->flags & DOMAIN_FLAG_NESTING_MODE)) {
		pr_err("Domain is not in nesting mode, %x\n", domain->flags);
		return -EINVAL;
	}
Quoted 6 lines
Quoted 6 lines
drivers/iommu/intel-pasid.c +6 -0
+
+	pte = intel_pasid_get_entry(dev, pasid);
+	if (WARN_ON(!pte))
+		return -EINVAL;
+
+	/*
Quoted 8 lines

drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 784

Quoted 6 lines
(quoted diff) +1 -0
+	 * Caller must ensure PASID entry is not in use, i.e. not

bind the

drivers/iommu/intel-pasid.c +4 -0
+	 * same PASID to the same device twice.
+	 */
+	if (pasid_pte_is_present(pte))
+		return -EBUSY;  

Here you check the PTE is not valid, is it sufficient to guarantee the above?

The caller has already checked the condition above. Here is just additional sanity checking.

Quoted 1 line

Also refering to the race potential issue pointed out by Kevin.

Will add spin_lock(&iommu->lock) around nested setup.

Quoted 36 lines
Quoted 36 lines
drivers/iommu/intel-pasid.c +15 -0
+
+	pasid_clear_entry(pte);
+
+	/* Sanity checking performed by caller to make sure address
+	 * width matching in two dimensions:
+	 * 1. CPU vs. IOMMU
+	 * 2. Guest vs. Host.
+	 */
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
+			pasid_set_flpm(pte, 1);
+		} else {
+			dev_err(dev, "5-level paging not

supported\n");

drivers/iommu/intel-pasid.c +8 -0
+			return -EINVAL;
+		}
+		break;
+	case ADDR_WIDTH_4LEVEL:
+		pasid_set_flpm(pte, 0);
+		break;
+	default:
+		dev_err(dev, "Invalid guest address width %d\n",

addr_width);

drivers/iommu/intel-pasid.c +4 -0
+		return -EINVAL;
+	}
+
+	/* First level PGD is in GPA, must be supported by the

second level */

(quoted diff) +2 -0
+	if ((u64)gpgd > domain->max_addr) {
+		dev_err(dev, "Guest PGD %llx not supported, max

%llx\n",

drivers/iommu/intel-pasid.c +3 -0
+			(u64)gpgd, domain->max_addr);
+		return -EINVAL;
+	}
Quoted 8 lines
Quoted 6 lines
drivers/iommu/intel-pasid.c +5 -0
+	pasid_set_flptr(pte, (u64)gpgd);
+
+	ret = intel_pasid_setup_bind_data(iommu, pte, pasid_data);
+	if (ret) {
+		dev_err(dev, "Guest PASID bind data not

supported\n");

Shall we output all those traces without limit? They are triggered by userspace, meaning this latter can trigger a storm of those.

Good point, will change all guest invoked calls to _ratelimited version.

Thanks!

Quoted 82 lines

include/linux/intel-iommu.h

Quoted 80 lines
drivers/iommu/intel-pasid.c +27 -0
+		return ret;
+	}
+
+	/* Setup the second level based on the given domain */
+	pgd = domain->pgd;
+
+	agaw = iommu_skip_agaw(domain, iommu, &pgd);
+	if (agaw < 0) {
+		dev_err(dev, "Invalid domain page table\n");
+		return -EINVAL;
+	}
+	pgd_val = virt_to_phys(pgd);
+	pasid_set_slptr(pte, pgd_val);
+	pasid_set_fault_enable(pte);
+
+	did = domain->iommu_did[iommu->seq_id];
+	pasid_set_domain_id(pte, did);
+
+	pasid_set_address_width(pte, agaw);
+	pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap));
+
+	pasid_set_translation_type(pte, PASID_ENTRY_PGTT_NESTED);
+	pasid_set_present(pte);
+	pasid_flush_caches(iommu, pte, pasid, did);
+
+	return ret;
+}
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.h index 92de6df24ccb..698015ee3f04 100644 --- a/drivers/iommu/intel-pasid.h
+++ b/drivers/iommu/intel-pasid.h

drivers/iommu/intel-pasid.h +1 -0
@@ -36,6 +36,7 @@
  * to vmalloc or even module mappings.
  */
 #define PASID_FLAG_SUPERVISOR_MODE	BIT(0)
+#define PASID_FLAG_NESTED		BIT(1)
 
 /*
  * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for

first- @@ -51,6 +52,11 @@ struct pasid_entry {

drivers/iommu/intel-pasid.h +5 -0
 	u64 val[8];
 };
 
+#define PASID_ENTRY_PGTT_FL_ONLY	(1)
+#define PASID_ENTRY_PGTT_SL_ONLY	(2)
+#define PASID_ENTRY_PGTT_NESTED		(3)
+#define PASID_ENTRY_PGTT_PT		(4)
+
 /* The representative of a PASID table */
 struct pasid_table {
 	void			*table;		/*

pasid table pointer */ @@ -99,6 +105,12 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu, int intel_pasid_setup_pass_through(struct intel_iommu *iommu, struct dmar_domain *domain, struct device *dev, int pasid);

drivers/iommu/intel-pasid.h +4 -0
+int intel_pasid_setup_nested(struct intel_iommu *iommu,
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

*pasid_data,

drivers/iommu/intel-pasid.h +2 -0

Lines of this file are not shown. View the raw message.

(quoted diff) +0 -0

The rest of this diff is not shown. View the raw message.

3 lines not shown · raw message

include/linux/intel-iommu.h +3 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

Quoted 2 lines

1 line not shown · raw message

1 line not shown · raw message

Parts of this message are not shown here. View the raw message.

author
#25 lore raw

PATCH v10 6/11 iommu/vt-d: Add bind guest PASID support

In reply to Jacob Pan

When supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.

For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

---
 drivers/iommu/intel-iommu.c |   4 +
 drivers/iommu/intel-svm.c   | 224 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   8 +-
 include/linux/intel-svm.h   |  17 ++++
 4 files changed, 252 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +4 -0
index e599b2537b1c..b1477cd423dd 100644
@@ -6203,6 +6203,10 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
+#ifdef CONFIG_INTEL_IOMMU_SVM
+	.sva_bind_gpasid	= intel_svm_bind_gpasid,
+	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
+#endif
 };
 
 static void quirk_iommu_igfx(struct pci_dev *dev)
drivers/iommu/intel-svm.c +224 -0

230 lines not on this page · raw message

include/linux/intel-iommu.h +7 -1
index eda1d6687144..85b05120940e 100644
@@ -681,7 +681,9 @@ struct dmar_domain *find_domain(struct device *dev);
 extern void intel_svm_check(struct intel_iommu *iommu);
 extern int intel_svm_enable_prq(struct intel_iommu *iommu);
 extern int intel_svm_finish_prq(struct intel_iommu *iommu);
-
+extern int intel_svm_bind_gpasid(struct iommu_domain *domain,
+		struct device *dev, struct iommu_gpasid_bind_data *data);
+extern int intel_svm_unbind_gpasid(struct device *dev, int pasid);
 struct svm_dev_ops;
 
 struct intel_svm_dev {
@@ -698,9 +700,13 @@ struct intel_svm_dev {
 struct intel_svm {
 	struct mmu_notifier notifier;
 	struct mm_struct *mm;
+
 	struct intel_iommu *iommu;
 	int flags;
 	int pasid;
+	int gpasid; /* Guest PASID in case of vSVA bind with non-identity host
+		     * to guest PASID mapping.
+		     */
 	struct list_head devs;
 	struct list_head list;
 };
include/linux/intel-svm.h +17 -0
index d7c403d0dd27..c19690937540 100644
@@ -44,6 +44,23 @@ struct svm_dev_ops {
  * do such IOTLB flushes automatically.
  */
 #define SVM_FLAG_SUPERVISOR_MODE	(1<<1)
+/*
+ * The SVM_FLAG_GUEST_MODE flag is used when a guest process bind to a device.
+ * In this case the mm_struct is in the guest kernel or userspace, its life
+ * cycle is managed by VMM and VFIO layer. For IOMMU driver, this API provides
+ * means to bind/unbind guest CR3 with PASIDs allocated for a device.
+ */
+#define SVM_FLAG_GUEST_MODE	(1<<2)
+/*
+ * The SVM_FLAG_GUEST_PASID flag is used when a guest has its own PASID space,
+ * which requires guest and host PASID translation at both directions. We keep
+ * track of guest PASID in order to provide lookup service to device drivers.
+ * One such example is a physical function (PF) driver that supports mediated
+ * device (mdev) assignment. Guest programming of mdev configuration space can
+ * only be done with guest PASID, therefore PF driver needs to find the matching
+ * host PASID to program the real hardware.
+ */
+#define SVM_FLAG_GUEST_PASID	(1<<3)
 
 #ifdef CONFIG_INTEL_IOMMU_SVM
 
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#26 lore raw

RE: [PATCH V10 06/11] iommu/vt-d: Add bind guest PASID support

Quoted 68 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.

For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

---
 drivers/iommu/intel-iommu.c |   4 +
 drivers/iommu/intel-svm.c   | 224
++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   8 +-
 include/linux/intel-svm.h   |  17 ++++
 4 files changed, 252 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +4 -0
index e599b2537b1c..b1477cd423dd 100644
@@ -6203,6 +6203,10 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
+#ifdef CONFIG_INTEL_IOMMU_SVM
+	.sva_bind_gpasid	= intel_svm_bind_gpasid,
+	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
+#endif
 };
 
 static void quirk_iommu_igfx(struct pci_dev *dev)
drivers/iommu/intel-svm.c +0 -0
index d7f2a5358900..47c0deb5ae56 100644

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 234

Quoted 9 lines
drivers/iommu/intel-svm.c +6 -0
@@ -226,6 +226,230 @@ static LIST_HEAD(global_svm_list);
 	list_for_each_entry((sdev), &(svm)->devs, list)	\
 		if ((d) != (sdev)->dev) {} else
 
+int intel_svm_bind_gpasid(struct iommu_domain *domain,
+			struct device *dev,
+			struct iommu_gpasid_bind_data *data)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct dmar_domain *ddomain;

what about the full name e.g. dmar_domain? though a bit longer but clearer than ddomain.

Quoted 26 lines
drivers/iommu/intel-svm.c +26 -0
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = 0;
+
+	if (WARN_ON(!iommu) || !data)
+		return -EINVAL;
+
+	if (data->version != IOMMU_GPASID_BIND_VERSION_1 ||
+	    data->format != IOMMU_PASID_FORMAT_INTEL_VTD)
+		return -EINVAL;
+
+	if (dev_is_pci(dev)) {
+		/* VT-d supports devices with full 20 bit PASIDs only */
+		if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
+			return -EINVAL;
+	} else {
+		return -ENOTSUPP;
+	}
+
+	/*
+	 * We only check host PASID range, we have no knowledge to check
+	 * guest PASID range nor do we use the guest PASID.
+	 */
+	if (data->hpasid <= 0 || data->hpasid >= PASID_MAX)
+		return -EINVAL;
+

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 268

Quoted 10 lines
drivers/iommu/intel-svm.c +8 -0
+	ddomain = to_dmar_domain(domain);
+
+	/* Sanity check paging mode support match between host and guest
 */
+	if (data->addr_width == ADDR_WIDTH_5LEVEL &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("Cannot support 5 level paging requested by
 guest!\n");
+		return -EINVAL;
+	}

-ENOTSUPP?

Quoted 13 lines
drivers/iommu/intel-svm.c +13 -0
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, data->hpasid, NULL);
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	if (svm) {
+		/*
+		 * If we found svm for the PASID, there must be at
+		 * least one device bond, otherwise svm should be freed.
+		 */

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list);

Quoted 10 lines
drivers/iommu/intel-svm.c +9 -0
+		if (WARN_ON(list_empty(&svm->devs))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (svm->mm == get_task_mm(current) &&
+		    data->hpasid == svm->pasid &&
+		    data->gpasid == svm->gpasid) {
+			pr_warn("Cannot bind the same guest-host PASID for
 the same process\n");

Sorry I didn’t get the rationale here. Isn't this branch is for binding the same PASID to multiple devices? In that case definitely it is binding the same guest-host PASID for the same process. otherwise if hpasid is different then you'll hit a different intel_svm, while if gpasid is different how you can use one intel_svm to hold multiple gpasids?

I feel the error condition should be the opposite. and suppose SVM_FLAG_ GUEST_PASID should be verified before checking gpasid.

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 300

Quoted 10 lines
drivers/iommu/intel-svm.c +10 -0
+			mmput(svm->mm);
+			ret = -EINVAL;
+			goto out;
+		}
+		mmput(current->mm);
+
+		for_each_svm_dev(sdev, svm, dev) {
+			/* In case of multiple sub-devices of the same pdev
+			 * assigned, we should allow multiple bind calls with
+			 * the same PASID and pdev.

Does sub-device mean mdev? I didn't find such notation in current iommu directory.

and to make it clearer, "In case of multiple mdevs of the same pdev assigned to the same guest process".

Quoted 16 lines
drivers/iommu/intel-svm.c +6 -0
+			 */
+			sdev->users++;
+			goto out;
+		}
+	} else {
+		/* We come here when PASID has never been bond to a

device. */

drivers/iommu/intel-svm.c +6 -0
+		svm = kzalloc(sizeof(*svm), GFP_KERNEL);
+		if (!svm) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		/* REVISIT: upper layer/VFIO can track host process that bind

the PASID.

drivers/iommu/intel-svm.c +1 -0
+		 * ioasid_set = mm might be sufficient for vfio to check pasid

VMM

drivers/iommu/intel-svm.c

Quoted 2 lines
drivers/iommu/intel-svm.c +2 -0
+		 * ownership.
+		 */

Above message is unclear about what should be revisited. Does it describe the current implementation or the expected revision in the future?

Quoted 22 lines
drivers/iommu/intel-svm.c +22 -0
+		svm->mm = get_task_mm(current);
+		svm->pasid = data->hpasid;
+		if (data->flags & IOMMU_SVA_GPASID_VAL) {
+			svm->gpasid = data->gpasid;
+			svm->flags |= SVM_FLAG_GUEST_PASID;
+		}
+		ioasid_set_data(data->hpasid, svm);
+		INIT_LIST_HEAD_RCU(&svm->devs);
+		mmput(svm->mm);
+	}
+	sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
+	if (!sdev) {
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		ret = -ENOMEM;
+		goto out;
+	}
+	sdev->dev = dev;
+	sdev->users = 1;
+

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 347

Quoted 10 lines
drivers/iommu/intel-svm.c +10 -0
+	/* Set up device context entry for PASID if not enabled already */
+	ret = intel_iommu_enable_pasid(iommu, sdev->dev);
+	if (ret) {
+		dev_err(dev, "Failed to enable PASID capability\n");
+		kfree(sdev);
+		/*
+		 * If this this a new PASID that never bond to a device, then
+		 * the device list must be empty which indicates struct svm
+		 * was allocated in this function.
+		 */

the comment better move to the 1st occurrence when sdev allocation fails. or even better put it in out label...

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 360

Quoted 13 lines
drivers/iommu/intel-svm.c +13 -0
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+
+	/*
+	 * For guest bind, we need to set up PASID table entry as follows:
+	 * - FLPM matches guest paging mode
+	 * - turn on nested mode
+	 * - SL guest address width matching
+	 */

looks above just explains the internal detail of intel_pasid_setup_nested, which is not necessary to be here.

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 367

Quoted 7 lines
drivers/iommu/intel-svm.c +7 -0
+	ret = intel_pasid_setup_nested(iommu,
+				       dev,
+				       (pgd_t *)data->gpgd,
+				       data->hpasid,
+				       &data->vtd,
+				       ddomain,
+				       data->addr_width);

It's worthy of an explanation here that setup_nested is required for every device (even when they are sharing same intel_svm) because we allocate pasid table per device. Otherwise I made a mistake to think that only the 1st device bound to a new hpasid requires this step. 😊

Quoted 76 lines
drivers/iommu/intel-svm.c +2 -0
+	if (ret) {
+		dev_err(dev, "Failed to set up PASID %llu in nested mode,

Err %d\n",

drivers/iommu/intel-svm.c +50 -0
+			data->hpasid, ret);
+		/*
+		 * PASID entry should be in cleared state if nested mode
+		 * set up failed. So we only need to clear IOASID tracking
+		 * data such that free call will succeed.
+		 */
+		kfree(sdev);
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+	svm->flags |= SVM_FLAG_GUEST_MODE;
+
+	init_rcu_head(&sdev->rcu);
+	list_add_rcu(&sdev->list, &svm->devs);
+ out:
+	mutex_unlock(&pasid_mutex);
+	return ret;
+}
+
+int intel_svm_unbind_gpasid(struct device *dev, int pasid)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = -EINVAL;
+
+	if (WARN_ON(!iommu))
+		return -EINVAL;
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, pasid, NULL);
+	if (!svm) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	for_each_svm_dev(sdev, svm, dev) {
+		ret = 0;
+		sdev->users--;
+		if (!sdev->users) {
+			list_del_rcu(&sdev->list);
+			intel_pasid_tear_down_entry(iommu, dev, svm-
Quoted 1 line

pasid);

drivers/iommu/intel-svm.c +12 -0
+			/* TODO: Drain in flight PRQ for the PASID since it
+			 * may get reused soon, we don't want to
+			 * confuse with its previous life.
+			 * intel_svm_drain_prq(dev, pasid);
+			 */
+			kfree_rcu(sdev, rcu);
+
+			if (list_empty(&svm->devs)) {
+				/*
+				 * We do not free PASID here until explicit call
+				 * from VFIO to free. The PASID life cycle
+				 * management is largely tied to VFIO

management

drivers/iommu/intel-svm.c +6 -0
+				 * of assigned device life cycles. In case of
+				 * guest exit without a explicit free PASID call,
+				 * the responsibility lies in VFIO layer to free
+				 * the PASIDs allocated for the guest.
+				 * For security reasons, VFIO has to track the
+				 * PASID ownership per guest anyway to

ensure

drivers/iommu/intel-svm.c +1 -0
+				 * that PASID allocated by one guest cannot

be

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 439

Quoted 1 line
drivers/iommu/intel-svm.c +1 -0
+				 * used by another.

As commented in other patches, VFIO is only one example user of this API...

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 446

Quoted 7 lines
drivers/iommu/intel-svm.c +7 -0
+				 */
+				ioasid_set_data(pasid, NULL);
+				kfree(svm);
+			}
+		}
+		break;
+	}

what about no dev match? an -EINVAL is also required then.

Quoted 27 lines
drivers/iommu/intel-svm.c +6 -0
+out:
+	mutex_unlock(&pasid_mutex);
+
+	return ret;
+}
+
 int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct

svm_dev_ops *ops)
{

drivers/iommu/intel-svm.c +0 -0
 	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
include/linux/intel-iommu.h +3 -1
index eda1d6687144..85b05120940e 100644
@@ -681,7 +681,9 @@ struct dmar_domain *find_domain(struct device
 *dev);
 extern void intel_svm_check(struct intel_iommu *iommu);
 extern int intel_svm_enable_prq(struct intel_iommu *iommu);
 extern int intel_svm_finish_prq(struct intel_iommu *iommu);
-
+extern int intel_svm_bind_gpasid(struct iommu_domain *domain,
+		struct device *dev, struct iommu_gpasid_bind_data *data);
+extern int intel_svm_unbind_gpasid(struct device *dev, int pasid);
 struct svm_dev_ops;
 

struct intel_svm_dev {

include/linux/intel-iommu.h struct intel_svm_dev { · line 709

Quoted 10 lines
include/linux/intel-iommu.h +4 -0
@@ -698,9 +700,13 @@ struct intel_svm_dev {
 struct intel_svm {
 	struct mmu_notifier notifier;
 	struct mm_struct *mm;
+
 	struct intel_iommu *iommu;
 	int flags;
 	int pasid;
+	int gpasid; /* Guest PASID in case of vSVA bind with non-identity host
+		     * to guest PASID mapping.
+		     */

we don't need to highlight identity or non-identity thing, since either way shares the same infrastructure here and it is not the knowledge that the kernel driver should assume

Quoted 40 lines
include/linux/intel-iommu.h +0 -0
 	struct list_head devs;
 	struct list_head list;
 };
include/linux/intel-svm.h +2 -0
index d7c403d0dd27..c19690937540 100644
@@ -44,6 +44,23 @@ struct svm_dev_ops {
  * do such IOTLB flushes automatically.
  */
 #define SVM_FLAG_SUPERVISOR_MODE	(1<<1)
+/*
+ * The SVM_FLAG_GUEST_MODE flag is used when a guest process bind to a

device.
+ * In this case the mm_struct is in the guest kernel or userspace, its life
+ * cycle is managed by VMM and VFIO layer. For IOMMU driver, this API provides
+ * means to bind/unbind guest CR3 with PASIDs allocated for a device.
+ */

include/linux/intel-svm.h +3 -0
+#define SVM_FLAG_GUEST_MODE	(1<<2)
+/*
+ * The SVM_FLAG_GUEST_PASID flag is used when a guest has its own PASID

space,
+ * which requires guest and host PASID translation at both directions. We keep
+ * track of guest PASID in order to provide lookup service to device drivers.
+ * One such example is a physical function (PF) driver that supports mediated
+ * device (mdev) assignment. Guest programming of mdev configuration space can
+ * only be done with guest PASID, therefore PF driver needs to find the matching
+ * host PASID to program the real hardware.
+ */
+#define SVM_FLAG_GUEST_PASID (1<<3)

#ifdef CONFIG_INTEL_IOMMU_SVM

--
2.7.4

author
#27 lore raw

Re: [PATCH V10 06/11] iommu/vt-d: Add bind guest PASID support

Quoted 71 lines · Kevin
Quoted 71 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.

For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

---
 drivers/iommu/intel-iommu.c |   4 +
 drivers/iommu/intel-svm.c   | 224
++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   8 +-
 include/linux/intel-svm.h   |  17 ++++
 4 files changed, 252 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index e599b2537b1c..b1477cd423dd 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +0 -0
@@ -6203,6 +6203,10 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	=

intel_iommu_is_attach_deferred, .pgsize_bitmap =
INTEL_IOMMU_PGSIZES, +#ifdef CONFIG_INTEL_IOMMU_SVM

drivers/iommu/intel-iommu.c +3 -0
+	.sva_bind_gpasid	= intel_svm_bind_gpasid,
+	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
+#endif
 };
 
 static void quirk_iommu_igfx(struct pci_dev *dev)
drivers/iommu/intel-svm.c +0 -0
index d7f2a5358900..47c0deb5ae56 100644
@@ -226,6 +226,230 @@ static LIST_HEAD(global_svm_list);
 	list_for_each_entry((sdev), &(svm)->devs, list)	\
 		if ((d) != (sdev)->dev) {} else
 
Quoted 10 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 234

Quoted 6 lines
drivers/iommu/intel-svm.c +6 -0
+int intel_svm_bind_gpasid(struct iommu_domain *domain,
+			struct device *dev,
+			struct iommu_gpasid_bind_data *data)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct dmar_domain *ddomain;  

what about the full name e.g. dmar_domain? though a bit longer but clearer than ddomain.

Sure, I don't have preference.

Quoted 32 lines
Quoted 32 lines
drivers/iommu/intel-svm.c +13 -0
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = 0;
+
+	if (WARN_ON(!iommu) || !data)
+		return -EINVAL;
+
+	if (data->version != IOMMU_GPASID_BIND_VERSION_1 ||
+	    data->format != IOMMU_PASID_FORMAT_INTEL_VTD)
+		return -EINVAL;
+
+	if (dev_is_pci(dev)) {
+		/* VT-d supports devices with full 20 bit PASIDs

only */

drivers/iommu/intel-svm.c +8 -0
+		if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
+			return -EINVAL;
+	} else {
+		return -ENOTSUPP;
+	}
+
+	/*
+	 * We only check host PASID range, we have no knowledge to

check

drivers/iommu/intel-svm.c +8 -0
+	 * guest PASID range nor do we use the guest PASID.
+	 */
+	if (data->hpasid <= 0 || data->hpasid >= PASID_MAX)
+		return -EINVAL;
+
+	ddomain = to_dmar_domain(domain);
+
+	/* Sanity check paging mode support match between host and

guest */

Quoted 8 lines

drivers/iommu/intel-svm.c

Quoted 6 lines
drivers/iommu/intel-svm.c +3 -0
+	if (data->addr_width == ADDR_WIDTH_5LEVEL &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("Cannot support 5 level paging requested by

guest!\n");

drivers/iommu/intel-svm.c +2 -0
+		return -EINVAL;
+	}  

-ENOTSUPP?

I was thinking from this API p.o.v, the input is invalid. Since both cap and addr_width are derived from input arguments.

Quoted 33 lines
Quoted 24 lines
drivers/iommu/intel-svm.c +12 -0
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, data->hpasid, NULL);
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	if (svm) {
+		/*
+		 * If we found svm for the PASID, there must be at
+		 * least one device bond, otherwise svm should be

freed.

drivers/iommu/intel-svm.c +10 -0
+		 */
+		if (WARN_ON(list_empty(&svm->devs))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (svm->mm == get_task_mm(current) &&
+		    data->hpasid == svm->pasid &&
+		    data->gpasid == svm->gpasid) {
+			pr_warn("Cannot bind the same guest-host
Quoted 1 line

PASID for the same process\n");

Sorry I didn’t get the rationale here. Isn't this branch is for binding the same PASID to multiple devices? In that case definitely it is binding the same guest-host PASID for the same process. otherwise if hpasid is different then you'll hit a different intel_svm, while if gpasid is different how you can use one intel_svm to hold multiple gpasids?

Quoted 3 lines

I feel the error condition should be the opposite. and suppose SVM_FLAG_ GUEST_PASID should be verified before checking gpasid.

You are right, actually we don't need the check here. The scenario for multiple devices bind to the same PASID is checked in for_each_svm_dev()
I will remove this code.

Quoted 7 lines
Quoted 7 lines
drivers/iommu/intel-svm.c +7 -0
+			mmput(svm->mm);
+			ret = -EINVAL;
+			goto out;
+		}
+		mmput(current->mm);
+
+		for_each_svm_dev(sdev, svm, dev) {
Quoted 9 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 300

Quoted 5 lines
drivers/iommu/intel-svm.c +1 -0
+			/* In case of multiple sub-devices of the

same pdev

drivers/iommu/intel-svm.c +1 -0
+			 * assigned, we should allow multiple bind

calls with

drivers/iommu/intel-svm.c +1 -0
+			 * the same PASID and pdev.  

Does sub-device mean mdev? I didn't find such notation in current iommu directory.

yes it is intended for mdev.

Quoted 3 lines

and to make it clearer, "In case of multiple mdevs of the same pdev assigned to the same guest process".

I am avoiding mdev on purpose since it is not a concept in iommu driver. sub-device is more generic.

Quoted 14 lines
Quoted 14 lines
drivers/iommu/intel-svm.c +6 -0
+			 */
+			sdev->users++;
+			goto out;
+		}
+	} else {
+		/* We come here when PASID has never been bond to a

device. */

drivers/iommu/intel-svm.c +6 -0
+		svm = kzalloc(sizeof(*svm), GFP_KERNEL);
+		if (!svm) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		/* REVISIT: upper layer/VFIO can track host

process that bind the PASID.

Quoted 9 lines

drivers/iommu/intel-svm.c

Quoted 4 lines
drivers/iommu/intel-svm.c +1 -0
+		 * ioasid_set = mm might be sufficient for vfio to

check pasid VMM

drivers/iommu/intel-svm.c +2 -0
+		 * ownership.
+		 */  

Above message is unclear about what should be revisited. Does it describe the current implementation or the expected revision in the future?

What I meant was if VFIO can check PASID-mm ownership by itself, then we don;t have to store svm->mm here. Will drop the line below. I will add this comment to clarify.

Quoted 31 lines
Quoted 31 lines
drivers/iommu/intel-svm.c +23 -0
+		svm->mm = get_task_mm(current);
+		svm->pasid = data->hpasid;
+		if (data->flags & IOMMU_SVA_GPASID_VAL) {
+			svm->gpasid = data->gpasid;
+			svm->flags |= SVM_FLAG_GUEST_PASID;
+		}
+		ioasid_set_data(data->hpasid, svm);
+		INIT_LIST_HEAD_RCU(&svm->devs);
+		mmput(svm->mm);
+	}
+	sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
+	if (!sdev) {
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		ret = -ENOMEM;
+		goto out;
+	}
+	sdev->dev = dev;
+	sdev->users = 1;
+
+	/* Set up device context entry for PASID if not enabled

already */

drivers/iommu/intel-svm.c +3 -0
+	ret = intel_iommu_enable_pasid(iommu, sdev->dev);
+	if (ret) {
+		dev_err(dev, "Failed to enable PASID

capability\n");

drivers/iommu/intel-svm.c +3 -0
+		kfree(sdev);
+		/*
+		 * If this this a new PASID that never bond to a
Quoted 9 lines

drivers/iommu/intel-svm.c

Quoted 5 lines

device, then

drivers/iommu/intel-svm.c +1 -0
+		 * the device list must be empty which indicates

struct svm

drivers/iommu/intel-svm.c +2 -0
+		 * was allocated in this function.
+		 */  

the comment better move to the 1st occurrence when sdev allocation fails. or even better put it in out label...

Sounds good.

Quoted 9 lines
Quoted 9 lines
drivers/iommu/intel-svm.c +9 -0
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+
+	/*
+	 * For guest bind, we need to set up PASID table entry as
Quoted 9 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 360

Quoted 5 lines

follows:

drivers/iommu/intel-svm.c +4 -0
+	 * - FLPM matches guest paging mode
+	 * - turn on nested mode
+	 * - SL guest address width matching
+	 */  

looks above just explains the internal detail of intel_pasid_setup_nested, which is not necessary to be here.

Right, will remove the comments.

Quoted 5 lines
Quoted 5 lines
drivers/iommu/intel-svm.c +5 -0
+	ret = intel_pasid_setup_nested(iommu,
+				       dev,
+				       (pgd_t *)data->gpgd,
+				       data->hpasid,
+				       &data->vtd,
Quoted 9 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 367

Quoted 2 lines
drivers/iommu/intel-svm.c +2 -0
+				       ddomain,
+				       data->addr_width);  

It's worthy of an explanation here that setup_nested is required for every device (even when they are sharing same intel_svm) because we allocate pasid table per device. Otherwise I made a mistake to think that only the 1st device bound to a new hpasid requires this step. 😊

Good suggestion, I will add the comments as:
/*
* PASID table is per device for better security. Therefore, for
* each bind of a new device even with an existing PASID, we need to
* call the nested mode setup function here.
*/

Quoted 83 lines
Quoted 83 lines
drivers/iommu/intel-svm.c +2 -0
+	if (ret) {
+		dev_err(dev, "Failed to set up PASID %llu in

nested mode, Err %d\n",

drivers/iommu/intel-svm.c +3 -0
+			data->hpasid, ret);
+		/*
+		 * PASID entry should be in cleared state if

nested mode

drivers/iommu/intel-svm.c +1 -0
+		 * set up failed. So we only need to clear IOASID

tracking

drivers/iommu/intel-svm.c +46 -0
+		 * data such that free call will succeed.
+		 */
+		kfree(sdev);
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+	svm->flags |= SVM_FLAG_GUEST_MODE;
+
+	init_rcu_head(&sdev->rcu);
+	list_add_rcu(&sdev->list, &svm->devs);
+ out:
+	mutex_unlock(&pasid_mutex);
+	return ret;
+}
+
+int intel_svm_unbind_gpasid(struct device *dev, int pasid)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = -EINVAL;
+
+	if (WARN_ON(!iommu))
+		return -EINVAL;
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, pasid, NULL);
+	if (!svm) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	for_each_svm_dev(sdev, svm, dev) {
+		ret = 0;
+		sdev->users--;
+		if (!sdev->users) {
+			list_del_rcu(&sdev->list);
+			intel_pasid_tear_down_entry(iommu, dev,

svm-

Quoted 1 line

pasid);

drivers/iommu/intel-svm.c +1 -0
+			/* TODO: Drain in flight PRQ for the PASID

since it

drivers/iommu/intel-svm.c +9 -0
+			 * may get reused soon, we don't want to
+			 * confuse with its previous life.
+			 * intel_svm_drain_prq(dev, pasid);
+			 */
+			kfree_rcu(sdev, rcu);
+
+			if (list_empty(&svm->devs)) {
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

explicit call

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

life cycle

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

VFIO management

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

In case of

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

free PASID call,

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

layer to free

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

guest.

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

to track the

Quoted 9 lines

drivers/iommu/intel-svm.c

Quoted 5 lines
drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

anyway to ensure

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

guest cannot be

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

As commented in other patches, VFIO is only one example user of this API...

Right, how about this:
/*
* We do not free the IOASID here in that
* IOMMU driver did not allocate it.
* Unlike native SVM, IOASID for guest use was
* allocated prior to the bind call.
* In any case, if the free call comes before
* the unbind, IOMMU driver will get notified
* and perform cleanup.
*/

Quoted 10 lines

drivers/iommu/intel-svm.c

Quoted 7 lines
drivers/iommu/intel-svm.c +7 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

what about no dev match? an -EINVAL is also required then.

Yes, ret is initialized as -EINVAL

91 lines not shown · raw message

Parts of this message are not shown here. View the raw message.

#28 lore raw

RE: [PATCH V10 06/11] iommu/vt-d: Add bind guest PASID support

Quoted 129 lines

From: Jacob Pan <[email protected]>
Sent: Tuesday, March 31, 2020 4:52 AM

Quoted 71 lines · Kevin
Quoted 71 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.

For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

---
 drivers/iommu/intel-iommu.c |   4 +
 drivers/iommu/intel-svm.c   | 224
++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   8 +-
 include/linux/intel-svm.h   |  17 ++++
 4 files changed, 252 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index e599b2537b1c..b1477cd423dd 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +0 -0
@@ -6203,6 +6203,10 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	=

intel_iommu_is_attach_deferred, .pgsize_bitmap =
INTEL_IOMMU_PGSIZES, +#ifdef CONFIG_INTEL_IOMMU_SVM

drivers/iommu/intel-iommu.c +3 -0
+	.sva_bind_gpasid	= intel_svm_bind_gpasid,
+	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
+#endif
 };
 
 static void quirk_iommu_igfx(struct pci_dev *dev)
drivers/iommu/intel-svm.c +0 -0
index d7f2a5358900..47c0deb5ae56 100644
@@ -226,6 +226,230 @@ static LIST_HEAD(global_svm_list);
 	list_for_each_entry((sdev), &(svm)->devs, list)	\
 		if ((d) != (sdev)->dev) {} else
 
Quoted 10 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 234

Quoted 6 lines
drivers/iommu/intel-svm.c +6 -0
+int intel_svm_bind_gpasid(struct iommu_domain *domain,
+			struct device *dev,
+			struct iommu_gpasid_bind_data *data)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct dmar_domain *ddomain;

what about the full name e.g. dmar_domain? though a bit longer but clearer than ddomain.

Sure, I don't have preference.

Quoted 32 lines
Quoted 32 lines
drivers/iommu/intel-svm.c +13 -0
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = 0;
+
+	if (WARN_ON(!iommu) || !data)
+		return -EINVAL;
+
+	if (data->version != IOMMU_GPASID_BIND_VERSION_1 ||
+	    data->format != IOMMU_PASID_FORMAT_INTEL_VTD)
+		return -EINVAL;
+
+	if (dev_is_pci(dev)) {
+		/* VT-d supports devices with full 20 bit PASIDs

only */

drivers/iommu/intel-svm.c +8 -0
+		if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
+			return -EINVAL;
+	} else {
+		return -ENOTSUPP;
+	}
+
+	/*
+	 * We only check host PASID range, we have no knowledge to

check

drivers/iommu/intel-svm.c +8 -0
+	 * guest PASID range nor do we use the guest PASID.
+	 */
+	if (data->hpasid <= 0 || data->hpasid >= PASID_MAX)
+		return -EINVAL;
+
+	ddomain = to_dmar_domain(domain);
+
+	/* Sanity check paging mode support match between host and

guest */

Quoted 8 lines

drivers/iommu/intel-svm.c

Quoted 6 lines
drivers/iommu/intel-svm.c +3 -0
+	if (data->addr_width == ADDR_WIDTH_5LEVEL &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("Cannot support 5 level paging requested by

guest!\n");

drivers/iommu/intel-svm.c +2 -0
+		return -EINVAL;
+	}

-ENOTSUPP?

Quoted 2 lines

I was thinking from this API p.o.v, the input is invalid. Since both cap and addr_width are derived from input arguments.

ok, suppose the userspace already enumerates the capabilities before making this call.

Quoted 58 lines
Quoted 34 lines
Quoted 25 lines
drivers/iommu/intel-svm.c +12 -0
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, data->hpasid, NULL);
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	if (svm) {
+		/*
+		 * If we found svm for the PASID, there must be at
+		 * least one device bond, otherwise svm should be

freed.

drivers/iommu/intel-svm.c +10 -0
+		 */
+		if (WARN_ON(list_empty(&svm->devs))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (svm->mm == get_task_mm(current) &&
+		    data->hpasid == svm->pasid &&
+		    data->gpasid == svm->gpasid) {
+			pr_warn("Cannot bind the same guest-host
Quoted 1 line

PASID for the same process\n");

Sorry I didn’t get the rationale here. Isn't this branch is for binding the same PASID to multiple devices? In that case definitely it is binding the same guest-host PASID for the same process. otherwise if hpasid is different then you'll hit a different intel_svm, while if gpasid is different how you can use one intel_svm to hold multiple gpasids?

Quoted 3 lines

I feel the error condition should be the opposite. and suppose SVM_FLAG_ GUEST_PASID should be verified before checking gpasid.

You are right, actually we don't need the check here. The scenario for multiple devices bind to the same PASID is checked in for_each_svm_dev()
I will remove this code.

Quoted 7 lines
Quoted 7 lines
drivers/iommu/intel-svm.c +7 -0
+			mmput(svm->mm);
+			ret = -EINVAL;
+			goto out;
+		}
+		mmput(current->mm);
+
+		for_each_svm_dev(sdev, svm, dev) {
Quoted 9 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 300

Quoted 5 lines
drivers/iommu/intel-svm.c +1 -0
+			/* In case of multiple sub-devices of the

same pdev

drivers/iommu/intel-svm.c +1 -0
+			 * assigned, we should allow multiple bind

calls with

drivers/iommu/intel-svm.c +1 -0
+			 * the same PASID and pdev.

Does sub-device mean mdev? I didn't find such notation in current iommu directory.

Quoted 6 lines

yes it is intended for mdev.

Quoted 3 lines

and to make it clearer, "In case of multiple mdevs of the same pdev assigned to the same guest process".

I am avoiding mdev on purpose since it is not a concept in iommu driver. sub-device is more generic.

ok, fine to me.

Quoted 203 lines
Quoted 15 lines
Quoted 15 lines
drivers/iommu/intel-svm.c +6 -0
+			 */
+			sdev->users++;
+			goto out;
+		}
+	} else {
+		/* We come here when PASID has never been bond to a

device. */

drivers/iommu/intel-svm.c +6 -0
+		svm = kzalloc(sizeof(*svm), GFP_KERNEL);
+		if (!svm) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		/* REVISIT: upper layer/VFIO can track host

process that bind the PASID.

Quoted 9 lines

drivers/iommu/intel-svm.c

Quoted 4 lines
drivers/iommu/intel-svm.c +1 -0
+		 * ioasid_set = mm might be sufficient for vfio to

check pasid VMM

drivers/iommu/intel-svm.c +2 -0
+		 * ownership.
+		 */

Above message is unclear about what should be revisited. Does it describe the current implementation or the expected revision in the future?

What I meant was if VFIO can check PASID-mm ownership by itself, then we don;t have to store svm->mm here. Will drop the line below. I will add this comment to clarify.

Quoted 31 lines
Quoted 31 lines
drivers/iommu/intel-svm.c +23 -0
+		svm->mm = get_task_mm(current);
+		svm->pasid = data->hpasid;
+		if (data->flags & IOMMU_SVA_GPASID_VAL) {
+			svm->gpasid = data->gpasid;
+			svm->flags |= SVM_FLAG_GUEST_PASID;
+		}
+		ioasid_set_data(data->hpasid, svm);
+		INIT_LIST_HEAD_RCU(&svm->devs);
+		mmput(svm->mm);
+	}
+	sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
+	if (!sdev) {
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		ret = -ENOMEM;
+		goto out;
+	}
+	sdev->dev = dev;
+	sdev->users = 1;
+
+	/* Set up device context entry for PASID if not enabled

already */

drivers/iommu/intel-svm.c +3 -0
+	ret = intel_iommu_enable_pasid(iommu, sdev->dev);
+	if (ret) {
+		dev_err(dev, "Failed to enable PASID

capability\n");

drivers/iommu/intel-svm.c +3 -0
+		kfree(sdev);
+		/*
+		 * If this this a new PASID that never bond to a
Quoted 9 lines

drivers/iommu/intel-svm.c

Quoted 5 lines

device, then

drivers/iommu/intel-svm.c +1 -0
+		 * the device list must be empty which indicates

struct svm

drivers/iommu/intel-svm.c +2 -0
+		 * was allocated in this function.
+		 */

the comment better move to the 1st occurrence when sdev allocation fails. or even better put it in out label...

Sounds good.

Quoted 9 lines
Quoted 9 lines
drivers/iommu/intel-svm.c +9 -0
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+
+	/*
+	 * For guest bind, we need to set up PASID table entry as
Quoted 9 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 360

Quoted 5 lines

follows:

drivers/iommu/intel-svm.c +4 -0
+	 * - FLPM matches guest paging mode
+	 * - turn on nested mode
+	 * - SL guest address width matching
+	 */

looks above just explains the internal detail of intel_pasid_setup_nested, which is not necessary to be here.

Right, will remove the comments.

Quoted 5 lines
Quoted 5 lines
drivers/iommu/intel-svm.c +5 -0
+	ret = intel_pasid_setup_nested(iommu,
+				       dev,
+				       (pgd_t *)data->gpgd,
+				       data->hpasid,
+				       &data->vtd,
Quoted 9 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 367

Quoted 2 lines
drivers/iommu/intel-svm.c +2 -0
+				       ddomain,
+				       data->addr_width);

It's worthy of an explanation here that setup_nested is required for every device (even when they are sharing same intel_svm) because we allocate pasid table per device. Otherwise I made a mistake to think that only the 1st device bound to a new hpasid requires this step. 😊

Good suggestion, I will add the comments as:
/*
* PASID table is per device for better security. Therefore, for
* each bind of a new device even with an existing PASID, we need to
* call the nested mode setup function here.
*/

Quoted 83 lines
Quoted 83 lines
drivers/iommu/intel-svm.c +2 -0
+	if (ret) {
+		dev_err(dev, "Failed to set up PASID %llu in

nested mode, Err %d\n",

drivers/iommu/intel-svm.c +3 -0
+			data->hpasid, ret);
+		/*
+		 * PASID entry should be in cleared state if

nested mode

drivers/iommu/intel-svm.c +1 -0
+		 * set up failed. So we only need to clear IOASID

tracking

drivers/iommu/intel-svm.c +46 -0
+		 * data such that free call will succeed.
+		 */
+		kfree(sdev);
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+	svm->flags |= SVM_FLAG_GUEST_MODE;
+
+	init_rcu_head(&sdev->rcu);
+	list_add_rcu(&sdev->list, &svm->devs);
+ out:
+	mutex_unlock(&pasid_mutex);
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

svm-

Quoted 1 line

pasid);

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

since it

drivers/iommu/intel-svm.c +9 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

explicit call

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

drivers/iommu/intel-svm.c +1 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

1 line not shown · raw message

Quoted 9 lines

7 lines not shown · raw message

Quoted 10 lines

10 lines not shown · raw message

114 lines not shown · raw message

Parts of this message are not shown here. View the raw message.

author
#29 lore raw

Re: [PATCH V10 06/11] iommu/vt-d: Add bind guest PASID support

Quoted 24 lines · Kevin
Quoted 18 lines
Quoted 9 lines
Quoted 9 lines

struct intel_svm_dev {

include/linux/intel-iommu.h +1 -0
@@ -698,9 +700,13 @@ struct intel_svm_dev {
 struct intel_svm {
 	struct mmu_notifier notifier;
 	struct mm_struct *mm;
+
 	struct intel_iommu *iommu;
 	int flags;
 	int pasid;
Quoted 9 lines

include/linux/intel-iommu.h struct intel_svm_dev { · line 709

Quoted 4 lines
(quoted diff) +1 -0
+	int gpasid; /* Guest PASID in case of vSVA bind with

non-identity host

include/linux/intel-iommu.h +2 -0
+		     * to guest PASID mapping.
+		     */  

we don't need to highlight identity or non-identity thing, since either way shares the same infrastructure here and it is not the knowledge that the kernel driver should assume

Quoted 5 lines

Sorry, I don't get your point.

What I meant was that this field "gpasid" is only used for non-identity case. For identity case, we don't have SVM_FLAG_GUEST_PASID.

Quoted 6 lines

what's the problem if a guest tries to set gpasid even in identity case? do you want to add check to reject it? Also I remember we discussed before that we want to provide a consistent interface to other consumer e.g. KVM to setup VMCS PASID translation table. In that case, regardless of identity or non-identity, we need provide such mapping info.

The solution is in flux a little. For KVM to set up VMCS, we are planning to use IOASID set private ID as guest PASID. So this part of the code will go away, i.e. G-H PASID mapping will no longer stored in IOMMU driver. Perhaps we can address this after the transition?

#30 lore raw

Re: [PATCH V10 06/11] iommu/vt-d: Add bind guest PASID support

In reply to Jacob Pan

Hi,

Quoted 85 lines · Jacob Pan

When supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.

For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

---
 drivers/iommu/intel-iommu.c |   4 +
 drivers/iommu/intel-svm.c   | 224 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |   8 +-
 include/linux/intel-svm.h   |  17 ++++
 4 files changed, 252 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +4 -0
index e599b2537b1c..b1477cd423dd 100644
@@ -6203,6 +6203,10 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
+#ifdef CONFIG_INTEL_IOMMU_SVM
+	.sva_bind_gpasid	= intel_svm_bind_gpasid,
+	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
+#endif
 };
 
 static void quirk_iommu_igfx(struct pci_dev *dev)
drivers/iommu/intel-svm.c +18 -0
index d7f2a5358900..47c0deb5ae56 100644
@@ -226,6 +226,230 @@ static LIST_HEAD(global_svm_list);
 	list_for_each_entry((sdev), &(svm)->devs, list)	\
 		if ((d) != (sdev)->dev) {} else
 
+int intel_svm_bind_gpasid(struct iommu_domain *domain,
+			struct device *dev,
+			struct iommu_gpasid_bind_data *data)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct dmar_domain *ddomain;
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = 0;
+
+	if (WARN_ON(!iommu) || !data)
+		return -EINVAL;
+
+	if (data->version != IOMMU_GPASID_BIND_VERSION_1 ||
+	    data->format != IOMMU_PASID_FORMAT_INTEL_VTD)
+		return -EINVAL;
+
+	if (dev_is_pci(dev)) {

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 256

Quoted 10 lines
drivers/iommu/intel-svm.c +10 -0
+		/* VT-d supports devices with full 20 bit PASIDs only */
+		if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
+			return -EINVAL;
+	} else {
+		return -ENOTSUPP;
+	}
+
+	/*
+	 * We only check host PASID range, we have no knowledge to check
+	 * guest PASID range nor do we use the guest PASID.

nit : "nor do we use the guest PASID". Well the guest PASID FLAG is checked below and if set, svm->gpasid is set ;-)

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 267

Quoted 11 lines
drivers/iommu/intel-svm.c +11 -0
+	 */
+	if (data->hpasid <= 0 || data->hpasid >= PASID_MAX)
+		return -EINVAL;
+
+	ddomain = to_dmar_domain(domain);
+
+	/* Sanity check paging mode support match between host and guest */
+	if (data->addr_width == ADDR_WIDTH_5LEVEL &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("Cannot support 5 level paging requested by guest!\n");
+		return -EINVAL;

nit: This check also is done in intel_pasid_setup_nested with an extra check:
+ switch (addr_width) {
+ case ADDR_WIDTH_5LEVEL:
+ if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+ cap_5lp_support(iommu->cap)) {

Quoted 54 lines
drivers/iommu/intel-svm.c +54 -0
+	}
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, data->hpasid, NULL);
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	if (svm) {
+		/*
+		 * If we found svm for the PASID, there must be at
+		 * least one device bond, otherwise svm should be freed.
+		 */
+		if (WARN_ON(list_empty(&svm->devs))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (svm->mm == get_task_mm(current) &&
+		    data->hpasid == svm->pasid &&
+		    data->gpasid == svm->gpasid) {
+			pr_warn("Cannot bind the same guest-host PASID for the same process\n");
+			mmput(svm->mm);
+			ret = -EINVAL;
+			goto out;
+		}
+		mmput(current->mm);
+
+		for_each_svm_dev(sdev, svm, dev) {
+			/* In case of multiple sub-devices of the same pdev
+			 * assigned, we should allow multiple bind calls with
+			 * the same PASID and pdev.
+			 */
+			sdev->users++;
+			goto out;
+		}
+	} else {
+		/* We come here when PASID has never been bond to a device. */
+		svm = kzalloc(sizeof(*svm), GFP_KERNEL);
+		if (!svm) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		/* REVISIT: upper layer/VFIO can track host process that bind the PASID.
+		 * ioasid_set = mm might be sufficient for vfio to check pasid VMM
+		 * ownership.
+		 */
+		svm->mm = get_task_mm(current);
+		svm->pasid = data->hpasid;
+		if (data->flags & IOMMU_SVA_GPASID_VAL) {
+			svm->gpasid = data->gpasid;
+			svm->flags |= SVM_FLAG_GUEST_PASID;
+		}

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 331

Quoted 10 lines
drivers/iommu/intel-svm.c +10 -0
+		ioasid_set_data(data->hpasid, svm);
+		INIT_LIST_HEAD_RCU(&svm->devs);
+		mmput(svm->mm);
+	}
+	sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
+	if (!sdev) {
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}

nit: the above 4 lines are duplicated 3 times. Might be worth a helper.

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 341

Quoted 10 lines
drivers/iommu/intel-svm.c +10 -0
+		ret = -ENOMEM;
+		goto out;
+	}
+	sdev->dev = dev;
+	sdev->users = 1;
+
+	/* Set up device context entry for PASID if not enabled already */
+	ret = intel_iommu_enable_pasid(iommu, sdev->dev);
+	if (ret) {
+		dev_err(dev, "Failed to enable PASID capability\n");

unlimited tracing upon userspace call? Don't know what is the best policy.

Quoted 163 lines
drivers/iommu/intel-svm.c +111 -0
+		kfree(sdev);
+		/*
+		 * If this this a new PASID that never bond to a device, then
+		 * the device list must be empty which indicates struct svm
+		 * was allocated in this function.
+		 */
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+
+	/*
+	 * For guest bind, we need to set up PASID table entry as follows:
+	 * - FLPM matches guest paging mode
+	 * - turn on nested mode
+	 * - SL guest address width matching
+	 */
+	ret = intel_pasid_setup_nested(iommu,
+				       dev,
+				       (pgd_t *)data->gpgd,
+				       data->hpasid,
+				       &data->vtd,
+				       ddomain,
+				       data->addr_width);
+	if (ret) {
+		dev_err(dev, "Failed to set up PASID %llu in nested mode, Err %d\n",
+			data->hpasid, ret);
+		/*
+		 * PASID entry should be in cleared state if nested mode
+		 * set up failed. So we only need to clear IOASID tracking
+		 * data such that free call will succeed.
+		 */
+		kfree(sdev);
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
 
+		goto out;
+	}
+	svm->flags |= SVM_FLAG_GUEST_MODE;
+
+	init_rcu_head(&sdev->rcu);
+	list_add_rcu(&sdev->list, &svm->devs);
+ out:
+	mutex_unlock(&pasid_mutex);
+	return ret;
+}
+
+int intel_svm_unbind_gpasid(struct device *dev, int pasid)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = -EINVAL;
+
+	if (WARN_ON(!iommu))
+		return -EINVAL;
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, pasid, NULL);
+	if (!svm) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	for_each_svm_dev(sdev, svm, dev) {
+		ret = 0;
+		sdev->users--;
+		if (!sdev->users) {
+			list_del_rcu(&sdev->list);
+			intel_pasid_tear_down_entry(iommu, dev, svm->pasid);
+			/* TODO: Drain in flight PRQ for the PASID since it
+			 * may get reused soon, we don't want to
+			 * confuse with its previous life.
+			 * intel_svm_drain_prq(dev, pasid);
+			 */
+			kfree_rcu(sdev, rcu);
+
+			if (list_empty(&svm->devs)) {
+				/*
+				 * We do not free PASID here until explicit call
+				 * from VFIO to free. The PASID life cycle
+				 * management is largely tied to VFIO management
+				 * of assigned device life cycles. In case of
+				 * guest exit without a explicit free PASID call,
+				 * the responsibility lies in VFIO layer to free
+				 * the PASIDs allocated for the guest.
+				 * For security reasons, VFIO has to track the
+				 * PASID ownership per guest anyway to ensure
+				 * that PASID allocated by one guest cannot be
+				 * used by another.
+				 */
+				ioasid_set_data(pasid, NULL);
+				kfree(svm);
+			}
+		}
+		break;
+	}
+out:
+	mutex_unlock(&pasid_mutex);
+
+	return ret;
+}
+
 int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ops *ops)
 {
drivers/iommu/intel-svm.c +0 -0
 	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
include/linux/intel-iommu.h +7 -1
index eda1d6687144..85b05120940e 100644
@@ -681,7 +681,9 @@ struct dmar_domain *find_domain(struct device *dev);
 extern void intel_svm_check(struct intel_iommu *iommu);
 extern int intel_svm_enable_prq(struct intel_iommu *iommu);
 extern int intel_svm_finish_prq(struct intel_iommu *iommu);
-
+extern int intel_svm_bind_gpasid(struct iommu_domain *domain,
+		struct device *dev, struct iommu_gpasid_bind_data *data);
+extern int intel_svm_unbind_gpasid(struct device *dev, int pasid);
 struct svm_dev_ops;
 
 struct intel_svm_dev {
@@ -698,9 +700,13 @@ struct intel_svm_dev {
 struct intel_svm {
 	struct mmu_notifier notifier;
 	struct mm_struct *mm;
+
 	struct intel_iommu *iommu;
 	int flags;
 	int pasid;
+	int gpasid; /* Guest PASID in case of vSVA bind with non-identity host
+		     * to guest PASID mapping.
+		     */
 	struct list_head devs;
 	struct list_head list;
 };
include/linux/intel-svm.h +10 -0
index d7c403d0dd27..c19690937540 100644
@@ -44,6 +44,23 @@ struct svm_dev_ops {
  * do such IOTLB flushes automatically.
  */
 #define SVM_FLAG_SUPERVISOR_MODE	(1<<1)
+/*
+ * The SVM_FLAG_GUEST_MODE flag is used when a guest process bind to a device.
+ * In this case the mm_struct is in the guest kernel or userspace, its life
+ * cycle is managed by VMM and VFIO layer. For IOMMU driver, this API provides
+ * means to bind/unbind guest CR3 with PASIDs allocated for a device.
+ */
+#define SVM_FLAG_GUEST_MODE	(1<<2)
+/*
+ * The SVM_FLAG_GUEST_PASID flag is used when a guest has its own PASID space,
+ * which requires guest and host PASID translation at both directions. We keep

include/linux/intel-svm.h struct svm_dev_ops { · line 66

Quoted 10 lines
include/linux/intel-svm.h +7 -0
+ * track of guest PASID in order to provide lookup service to device drivers.
+ * One such example is a physical function (PF) driver that supports mediated
+ * device (mdev) assignment. Guest programming of mdev configuration space can
+ * only be done with guest PASID, therefore PF driver needs to find the matching
+ * host PASID to program the real hardware.
+ */
+#define SVM_FLAG_GUEST_PASID	(1<<3)
 
 #ifdef CONFIG_INTEL_IOMMU_SVM
 

Thanks

Eric

author
#31 lore raw

Re: [PATCH V10 06/11] iommu/vt-d: Add bind guest PASID support

Quoted 94 lines · On Sun, 29 Mar 2020 15:40:22 +0200 Auger Eric <[email protected]> wrote:

Hi,

Quoted 91 lines · Jacob Pan

When supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.

For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.

    .-------------.  .---------------------------.
    |   vIOMMU    |  | Guest process CR3, FL only|
    |             |  '---------------------------'

.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest

------| Shadow |--------------------------|--------
      v        v                          v
Host
    .-------------.  .----------------------.
    |   pIOMMU    |  | Bind FL for GVA-GPA  |
    |             |  '----------------------'
    .----------------/  |
    | PASID Entry |     V (Nested xlate)
    '----------------\.------------------------------.
    |             |   |SL for GPA-HPA, default domain|
    |             |   '------------------------------'
    '-------------'

Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables

---
 drivers/iommu/intel-iommu.c |   4 +
 drivers/iommu/intel-svm.c   | 224

++++++++++++++++++++++++++++++++++++++++++++
include/linux/intel-iommu.h | 8 +- include/linux/intel-svm.h |
17 ++++ 4 files changed, 252 insertions(+), 1 deletion(-)

(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index e599b2537b1c..b1477cd423dd 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +0 -0
@@ -6203,6 +6203,10 @@ const struct iommu_ops intel_iommu_ops = {
 	.dev_disable_feat	= intel_iommu_dev_disable_feat,
 	.is_attach_deferred	=

intel_iommu_is_attach_deferred, .pgsize_bitmap =
INTEL_IOMMU_PGSIZES, +#ifdef CONFIG_INTEL_IOMMU_SVM

drivers/iommu/intel-iommu.c +3 -0
+	.sva_bind_gpasid	= intel_svm_bind_gpasid,
+	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
+#endif
 };
 
 static void quirk_iommu_igfx(struct pci_dev *dev)
drivers/iommu/intel-svm.c +19 -0
index d7f2a5358900..47c0deb5ae56 100644
@@ -226,6 +226,230 @@ static LIST_HEAD(global_svm_list);
 	list_for_each_entry((sdev), &(svm)->devs, list)	\
 		if ((d) != (sdev)->dev) {} else
 
+int intel_svm_bind_gpasid(struct iommu_domain *domain,
+			struct device *dev,
+			struct iommu_gpasid_bind_data *data)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct dmar_domain *ddomain;
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = 0;
+
+	if (WARN_ON(!iommu) || !data)
+		return -EINVAL;
+
+	if (data->version != IOMMU_GPASID_BIND_VERSION_1 ||
+	    data->format != IOMMU_PASID_FORMAT_INTEL_VTD)
+		return -EINVAL;
+
+	if (dev_is_pci(dev)) {
+		/* VT-d supports devices with full 20 bit PASIDs

only */

drivers/iommu/intel-svm.c +4 -0
+		if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
+			return -EINVAL;
+	} else {
+		return -ENOTSUPP;
Quoted 8 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 256

Quoted 6 lines
drivers/iommu/intel-svm.c +4 -0
+	}
+
+	/*
+	 * We only check host PASID range, we have no knowledge to

check

drivers/iommu/intel-svm.c +1 -0
+	 * guest PASID range nor do we use the guest PASID.  

nit : "nor do we use the guest PASID". Well the guest PASID FLAG is checked below and if set, svm->gpasid is set ;-)

Yes, it is a little contradictory, I will remove the use.

I meant we don;t really use the gpasid for real work in host driver :)

Quoted 7 lines
Quoted 7 lines
drivers/iommu/intel-svm.c +7 -0
+	 */
+	if (data->hpasid <= 0 || data->hpasid >= PASID_MAX)
+		return -EINVAL;
+
+	ddomain = to_dmar_domain(domain);
+
+	/* Sanity check paging mode support match between host and
Quoted 8 lines

drivers/iommu/intel-svm.c

Quoted 6 lines

guest */

drivers/iommu/intel-svm.c +3 -0
+	if (data->addr_width == ADDR_WIDTH_5LEVEL &&
+	    !cap_5lp_support(iommu->cap)) {
+		pr_err("Cannot support 5 level paging requested by

guest!\n");

drivers/iommu/intel-svm.c +1 -0
+		return -EINVAL;  

nit: This check also is done in intel_pasid_setup_nested with an extra check:

Good catch, I will remove this.

Quoted 70 lines
(quoted diff) +4 -0
+	switch (addr_width) {
+	case ADDR_WIDTH_5LEVEL:
+		if (cpu_feature_enabled(X86_FEATURE_LA57) &&
+			cap_5lp_support(iommu->cap)) {
Quoted 66 lines
drivers/iommu/intel-svm.c +13 -0
+	}
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, data->hpasid, NULL);
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	if (svm) {
+		/*
+		 * If we found svm for the PASID, there must be at
+		 * least one device bond, otherwise svm should be

freed.

drivers/iommu/intel-svm.c +10 -0
+		 */
+		if (WARN_ON(list_empty(&svm->devs))) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		if (svm->mm == get_task_mm(current) &&
+		    data->hpasid == svm->pasid &&
+		    data->gpasid == svm->gpasid) {
+			pr_warn("Cannot bind the same guest-host

PASID for the same process\n");

drivers/iommu/intel-svm.c +8 -0
+			mmput(svm->mm);
+			ret = -EINVAL;
+			goto out;
+		}
+		mmput(current->mm);
+
+		for_each_svm_dev(sdev, svm, dev) {
+			/* In case of multiple sub-devices of the

same pdev

drivers/iommu/intel-svm.c +1 -0
+			 * assigned, we should allow multiple bind

calls with

drivers/iommu/intel-svm.c +7 -0
+			 * the same PASID and pdev.
+			 */
+			sdev->users++;
+			goto out;
+		}
+	} else {
+		/* We come here when PASID has never been bond to

a device. */

drivers/iommu/intel-svm.c +6 -0
+		svm = kzalloc(sizeof(*svm), GFP_KERNEL);
+		if (!svm) {
+			ret = -ENOMEM;
+			goto out;
+		}
+		/* REVISIT: upper layer/VFIO can track host

process that bind the PASID.

drivers/iommu/intel-svm.c +1 -0
+		 * ioasid_set = mm might be sufficient for vfio to

check pasid VMM

drivers/iommu/intel-svm.c +12 -0
+		 * ownership.
+		 */
+		svm->mm = get_task_mm(current);
+		svm->pasid = data->hpasid;
+		if (data->flags & IOMMU_SVA_GPASID_VAL) {
+			svm->gpasid = data->gpasid;
+			svm->flags |= SVM_FLAG_GUEST_PASID;
+		}
+		ioasid_set_data(data->hpasid, svm);
+		INIT_LIST_HEAD_RCU(&svm->devs);
+		mmput(svm->mm);
+	}
Quoted 8 lines

drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 331

Quoted 6 lines
drivers/iommu/intel-svm.c +6 -0
+	sdev = kzalloc(sizeof(*sdev), GFP_KERNEL);
+	if (!sdev) {
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}  

nit: the above 4 lines are duplicated 3 times. Might be worth a helper.

Good point, I will add a helper like this

static inline void intel_svm_free_if_empty(struct intel_svm *svm, u64 pasid)

{
	if (list_empty(&svm->devs)) {
		ioasid_attach_data(pasid, NULL);
		kfree(svm);
	}
}
Quoted 6 lines
Quoted 6 lines
drivers/iommu/intel-svm.c +6 -0
+		ret = -ENOMEM;
+		goto out;
+	}
+	sdev->dev = dev;
+	sdev->users = 1;
+
Quoted 8 lines
Quoted 6 lines
drivers/iommu/intel-svm.c +1 -0
+	/* Set up device context entry for PASID if not enabled

already */

drivers/iommu/intel-svm.c +3 -0
+	ret = intel_iommu_enable_pasid(iommu, sdev->dev);
+	if (ret) {
+		dev_err(dev, "Failed to enable PASID

capability\n");

unlimited tracing upon userspace call? Don't know what is the best policy.

Good point. Perhaps just use dev_err_ratelimited for all user calls?

Quoted 199 lines
Quoted 199 lines
drivers/iommu/intel-svm.c +3 -0
+		kfree(sdev);
+		/*
+		 * If this this a new PASID that never bond to a

device, then

drivers/iommu/intel-svm.c +1 -0
+		 * the device list must be empty which indicates

struct svm

drivers/iommu/intel-svm.c +11 -0
+		 * was allocated in this function.
+		 */
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}
+		goto out;
+	}
+
+	/*
+	 * For guest bind, we need to set up PASID table entry as

follows:

drivers/iommu/intel-svm.c +13 -0
+	 * - FLPM matches guest paging mode
+	 * - turn on nested mode
+	 * - SL guest address width matching
+	 */
+	ret = intel_pasid_setup_nested(iommu,
+				       dev,
+				       (pgd_t *)data->gpgd,
+				       data->hpasid,
+				       &data->vtd,
+				       ddomain,
+				       data->addr_width);
+	if (ret) {
+		dev_err(dev, "Failed to set up PASID %llu in

nested mode, Err %d\n",

drivers/iommu/intel-svm.c +3 -0
+			data->hpasid, ret);
+		/*
+		 * PASID entry should be in cleared state if

nested mode

drivers/iommu/intel-svm.c +1 -0
+		 * set up failed. So we only need to clear IOASID

tracking

drivers/iommu/intel-svm.c +46 -0
+		 * data such that free call will succeed.
+		 */
+		kfree(sdev);
+		if (list_empty(&svm->devs)) {
+			ioasid_set_data(data->hpasid, NULL);
+			kfree(svm);
+		}  
 
+		goto out;
+	}
+	svm->flags |= SVM_FLAG_GUEST_MODE;
+
+	init_rcu_head(&sdev->rcu);
+	list_add_rcu(&sdev->list, &svm->devs);
+ out:
+	mutex_unlock(&pasid_mutex);
+	return ret;
+}
+
+int intel_svm_unbind_gpasid(struct device *dev, int pasid)
+{
+	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
+	struct intel_svm_dev *sdev;
+	struct intel_svm *svm;
+	int ret = -EINVAL;
+
+	if (WARN_ON(!iommu))
+		return -EINVAL;
+
+	mutex_lock(&pasid_mutex);
+	svm = ioasid_find(NULL, pasid, NULL);
+	if (!svm) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (IS_ERR(svm)) {
+		ret = PTR_ERR(svm);
+		goto out;
+	}
+
+	for_each_svm_dev(sdev, svm, dev) {
+		ret = 0;
+		sdev->users--;
+		if (!sdev->users) {
+			list_del_rcu(&sdev->list);
+			intel_pasid_tear_down_entry(iommu, dev,

svm->pasid);

drivers/iommu/intel-svm.c +1 -0
+			/* TODO: Drain in flight PRQ for the PASID

since it

drivers/iommu/intel-svm.c +9 -0
+			 * may get reused soon, we don't want to
+			 * confuse with its previous life.
+			 * intel_svm_drain_prq(dev, pasid);
+			 */
+			kfree_rcu(sdev, rcu);
+
+			if (list_empty(&svm->devs)) {
+				/*
+				 * We do not free PASID here until

explicit call

drivers/iommu/intel-svm.c +1 -0
+				 * from VFIO to free. The PASID

life cycle

drivers/iommu/intel-svm.c +1 -0
+				 * management is largely tied to

VFIO management

drivers/iommu/intel-svm.c +1 -0
+				 * of assigned device life cycles.

In case of

drivers/iommu/intel-svm.c +1 -0
+				 * guest exit without a explicit

free PASID call,

drivers/iommu/intel-svm.c +1 -0
+				 * the responsibility lies in VFIO

layer to free

drivers/iommu/intel-svm.c +1 -0
+				 * the PASIDs allocated for the

guest.

drivers/iommu/intel-svm.c +1 -0
+				 * For security reasons, VFIO has

to track the

drivers/iommu/intel-svm.c +1 -0
+				 * PASID ownership per guest

anyway to ensure

drivers/iommu/intel-svm.c +1 -0
+				 * that PASID allocated by one

guest cannot be

drivers/iommu/intel-svm.c +14 -0
+				 * used by another.
+				 */
+				ioasid_set_data(pasid, NULL);
+				kfree(svm);
+			}
+		}
+		break;
+	}
+out:
+	mutex_unlock(&pasid_mutex);
+
+	return ret;
+}
+
 int intel_svm_bind_mm(struct device *dev, int *pasid, int flags,

struct svm_dev_ops *ops) {

drivers/iommu/intel-svm.c +0 -0
 	struct intel_iommu *iommu = intel_svm_device_to_iommu(dev);
(quoted diff) +0 -0

b/include/linux/intel-iommu.h index eda1d6687144..85b05120940e 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

include/linux/intel-iommu.h +4 -1
@@ -681,7 +681,9 @@ struct dmar_domain *find_domain(struct device
 *dev); extern void intel_svm_check(struct intel_iommu *iommu);
 extern int intel_svm_enable_prq(struct intel_iommu *iommu);
 extern int intel_svm_finish_prq(struct intel_iommu *iommu);
-
+extern int intel_svm_bind_gpasid(struct iommu_domain *domain,
+		struct device *dev, struct iommu_gpasid_bind_data
 *data); +extern int intel_svm_unbind_gpasid(struct device *dev, int
 pasid); struct svm_dev_ops;
 
 struct intel_svm_dev {
@@ -698,9 +700,13 @@ struct intel_svm_dev {
 struct intel_svm {
 	struct mmu_notifier notifier;
 	struct mm_struct *mm;
+
 	struct intel_iommu *iommu;
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

non-identity host

include/linux/intel-iommu.h +2 -0

Lines of this file are not shown. View the raw message.

include/linux/intel-svm.h +2 -0
index d7c403d0dd27..c19690937540 100644

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

to a device.
+ * In this case the mm_struct is in the guest kernel or userspace, its life
+ * cycle is managed by VMM and VFIO layer. For IOMMU driver, this API provides
+ * means to bind/unbind guest CR3 with PASIDs allocated for a device.
+ */

include/linux/intel-svm.h +3 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

14 lines not shown · raw message

Quoted 7 lines

5 lines not shown · raw message

1 line not shown · raw message

Parts of this message are not shown here. View the raw message.

author
#32 lore raw

PATCH v10 7/11 iommu/vt-d: Support flushing more translation cache types

In reply to Jacob Pan

When Shared Virtual Memory is exposed to a guest via vIOMMU, scalable IOTLB invalidation may be passed down from outside IOMMU subsystems. This patch adds invalidation functions that can be used for additional translation cache types.

---
v9 -> v10:
Fix off by 1 in pasid device iotlb flush

Address v7 missed review from Eric

---

---
 drivers/iommu/dmar.c        | 36 ++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-pasid.c |  3 ++-
 include/linux/intel-iommu.h | 20 ++++++++++++++++----
 3 files changed, 54 insertions(+), 5 deletions(-)
drivers/iommu/dmar.c +36 -0
index f77dae7ba7d4..4d6b7b5b37ee 100644
@@ -1421,6 +1421,42 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr,
 	qi_submit_sync(&desc, iommu);
 }
 
+/* PASID-based device IOTLB Invalidate */
+void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid,
+		u32 pasid,  u16 qdep, u64 addr, unsigned size_order, u64 granu)
+{
+	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
+	struct qi_desc desc = {.qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_DEV_EIOTLB_PASID(pasid) | QI_DEV_EIOTLB_SID(sid) |
+		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
+		QI_DEV_IOTLB_PFSID(pfsid);
+	desc.qw1 = QI_DEV_EIOTLB_GLOB(granu);
+
+	/*
+	 * If S bit is 0, we only flush a single page. If S bit is set,
+	 * The least significant zero bit indicates the invalidation address
+	 * range. VT-d spec 6.5.2.6.
+	 * e.g. address bit 12[0] indicates 8KB, 13[0] indicates 16KB.
+	 * size order = 0 is PAGE_SIZE 4KB
+	 * Max Invs Pending (MIP) is set to 0 for now until we have DIT in
+	 * ECAP.
+	 */
+	desc.qw1 |= addr & ~mask;
+	if (size_order)
+		desc.qw1 |= QI_DEV_EIOTLB_SIZE;
+
+	qi_submit_sync(&desc, iommu);
+}
+
+void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid)
+{
+	struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) | QI_PC_GRAN(granu) | QI_PC_TYPE;
+	qi_submit_sync(&desc, iommu);
+}
+
 /*
  * Disable Queued Invalidation interface.
  */
drivers/iommu/intel-pasid.c +2 -1
index 10c7856afc6b..9f6d07410722 100644
@@ -435,7 +435,8 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 {
 	struct qi_desc desc;
 
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL | QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;
 	desc.qw1 = 0;
 	desc.qw2 = 0;
 	desc.qw3 = 0;
include/linux/intel-iommu.h +16 -4
index 85b05120940e..43539713b3b3 100644
@@ -334,7 +334,7 @@ enum {
 #define QI_IOTLB_GRAN(gran) 	(((u64)gran) >> (DMA_TLB_FLUSH_GRANU_OFFSET-4))
 #define QI_IOTLB_ADDR(addr)	(((u64)addr) & VTD_PAGE_MASK)
 #define QI_IOTLB_IH(ih)		(((u64)ih) << 6)
-#define QI_IOTLB_AM(am)		(((u8)am))
+#define QI_IOTLB_AM(am)		(((u8)am) & 0x3f)
 
 #define QI_CC_FM(fm)		(((u64)fm) << 48)
 #define QI_CC_SID(sid)		(((u64)sid) << 32)
@@ -353,16 +353,21 @@ enum {
 #define QI_PC_DID(did)		(((u64)did) << 16)
 #define QI_PC_GRAN(gran)	(((u64)gran) << 4)
 
-#define QI_PC_ALL_PASIDS	(QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL		(QI_PC_TYPE | QI_PC_GRAN(1))
+/* PASID cache invalidation granu */
+#define QI_PC_ALL_PASIDS	0
+#define QI_PC_PASID_SEL		1
 
 #define QI_EIOTLB_ADDR(addr)	((u64)(addr) & VTD_PAGE_MASK)
 #define QI_EIOTLB_IH(ih)	(((u64)ih) << 6)
-#define QI_EIOTLB_AM(am)	(((u64)am))
+#define QI_EIOTLB_AM(am)	(((u64)am) & 0x3f)
 #define QI_EIOTLB_PASID(pasid) 	(((u64)pasid) << 32)
 #define QI_EIOTLB_DID(did)	(((u64)did) << 16)
 #define QI_EIOTLB_GRAN(gran) 	(((u64)gran) << 4)
 
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0
+
 #define QI_DEV_EIOTLB_ADDR(a)	((u64)(a) & VTD_PAGE_MASK)
 #define QI_DEV_EIOTLB_SIZE	(((u64)1) << 11)
 #define QI_DEV_EIOTLB_GLOB(g)	((u64)g)
@@ -662,8 +667,15 @@ extern void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 			  unsigned int size_order, u64 type);
 extern void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
 			u16 qdep, u64 addr, unsigned mask);
+
 void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr,
 		     unsigned long npages, bool ih);
+
+extern void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid,
+			u32 pasid, u16 qdep, u64 addr, unsigned size_order, u64 granu);
+
+extern void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid);
+
 extern int qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
 extern int dmar_ir_support(void);
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Reviewed-by on v10 7/11
#33 lore raw

Re: [PATCH V10 07/11] iommu/vt-d: Support flushing more translation cache types

Hi Jacob,

Quoted 72 lines · Jacob Pan

When Shared Virtual Memory is exposed to a guest via vIOMMU, scalable IOTLB invalidation may be passed down from outside IOMMU subsystems. This patch adds invalidation functions that can be used for additional translation cache types.

---
v9 -> v10:
Fix off by 1 in pasid device iotlb flush

Address v7 missed review from Eric

---

---
 drivers/iommu/dmar.c        | 36 ++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-pasid.c |  3 ++-
 include/linux/intel-iommu.h | 20 ++++++++++++++++----
 3 files changed, 54 insertions(+), 5 deletions(-)
drivers/iommu/dmar.c +36 -0
index f77dae7ba7d4..4d6b7b5b37ee 100644
@@ -1421,6 +1421,42 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr,
 	qi_submit_sync(&desc, iommu);
 }
 
+/* PASID-based device IOTLB Invalidate */
+void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid,
+		u32 pasid,  u16 qdep, u64 addr, unsigned size_order, u64 granu)
+{
+	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
+	struct qi_desc desc = {.qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_DEV_EIOTLB_PASID(pasid) | QI_DEV_EIOTLB_SID(sid) |
+		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
+		QI_DEV_IOTLB_PFSID(pfsid);
+	desc.qw1 = QI_DEV_EIOTLB_GLOB(granu);
+
+	/*
+	 * If S bit is 0, we only flush a single page. If S bit is set,
+	 * The least significant zero bit indicates the invalidation address
+	 * range. VT-d spec 6.5.2.6.
+	 * e.g. address bit 12[0] indicates 8KB, 13[0] indicates 16KB.
+	 * size order = 0 is PAGE_SIZE 4KB
+	 * Max Invs Pending (MIP) is set to 0 for now until we have DIT in
+	 * ECAP.
+	 */
+	desc.qw1 |= addr & ~mask;
+	if (size_order)
+		desc.qw1 |= QI_DEV_EIOTLB_SIZE;
+
+	qi_submit_sync(&desc, iommu);
+}
+
+void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid)
+{
+	struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) | QI_PC_GRAN(granu) | QI_PC_TYPE;
+	qi_submit_sync(&desc, iommu);
+}
+
 /*
  * Disable Queued Invalidation interface.
  */
drivers/iommu/intel-pasid.c +0 -0
index 10c7856afc6b..9f6d07410722 100644

drivers/iommu/intel-pasid.c pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, · line 439

Quoted 6 lines
drivers/iommu/intel-pasid.c +2 -1
@@ -435,7 +435,8 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu,
 {
 	struct qi_desc desc;
 
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL | QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;

Just a nit, this fix is not documented in the commit message.

Besides

Thanks

Eric

Quoted 47 lines
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw1 = 0;
 	desc.qw2 = 0;
 	desc.qw3 = 0;
include/linux/intel-iommu.h +9 -4
index 85b05120940e..43539713b3b3 100644
@@ -334,7 +334,7 @@ enum {
 #define QI_IOTLB_GRAN(gran) 	(((u64)gran) >> (DMA_TLB_FLUSH_GRANU_OFFSET-4))
 #define QI_IOTLB_ADDR(addr)	(((u64)addr) & VTD_PAGE_MASK)
 #define QI_IOTLB_IH(ih)		(((u64)ih) << 6)
-#define QI_IOTLB_AM(am)		(((u8)am))
+#define QI_IOTLB_AM(am)		(((u8)am) & 0x3f)
 
 #define QI_CC_FM(fm)		(((u64)fm) << 48)
 #define QI_CC_SID(sid)		(((u64)sid) << 32)
@@ -353,16 +353,21 @@ enum {
 #define QI_PC_DID(did)		(((u64)did) << 16)
 #define QI_PC_GRAN(gran)	(((u64)gran) << 4)
 
-#define QI_PC_ALL_PASIDS	(QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL		(QI_PC_TYPE | QI_PC_GRAN(1))
+/* PASID cache invalidation granu */
+#define QI_PC_ALL_PASIDS	0
+#define QI_PC_PASID_SEL		1
 
 #define QI_EIOTLB_ADDR(addr)	((u64)(addr) & VTD_PAGE_MASK)
 #define QI_EIOTLB_IH(ih)	(((u64)ih) << 6)
-#define QI_EIOTLB_AM(am)	(((u64)am))
+#define QI_EIOTLB_AM(am)	(((u64)am) & 0x3f)
 #define QI_EIOTLB_PASID(pasid) 	(((u64)pasid) << 32)
 #define QI_EIOTLB_DID(did)	(((u64)did) << 16)
 #define QI_EIOTLB_GRAN(gran) 	(((u64)gran) << 4)
 
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0> +
 #define QI_DEV_EIOTLB_ADDR(a)	((u64)(a) & VTD_PAGE_MASK)
 #define QI_DEV_EIOTLB_SIZE	(((u64)1) << 11)
 #define QI_DEV_EIOTLB_GLOB(g)	((u64)g)
@@ -662,8 +667,15 @@ extern void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
 			  unsigned int size_order, u64 type);
 extern void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid, u16 pfsid,
 			u16 qdep, u64 addr, unsigned mask);
+
 void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr,

include/linux/intel-iommu.h extern void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, · line 681

Quoted 10 lines
include/linux/intel-iommu.h +6 -0
 		     unsigned long npages, bool ih);
+
+extern void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid, u16 pfsid,
+			u32 pasid, u16 qdep, u64 addr, unsigned size_order, u64 granu);
+
+extern void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid);
+
 extern int qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
 
 extern int dmar_ir_support(void);

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#34 lore raw

Re: [PATCH V10 07/11] iommu/vt-d: Support flushing more translation cache types

Quoted 86 lines · On Fri, 27 Mar 2020 15:46:23 +0100 Auger Eric <[email protected]> wrote:

Hi Jacob,

Quoted 83 lines · Jacob Pan

When Shared Virtual Memory is exposed to a guest via vIOMMU, scalable IOTLB invalidation may be passed down from outside IOMMU subsystems. This patch adds invalidation functions that can be used for additional translation cache types.

---
v9 -> v10:
Fix off by 1 in pasid device iotlb flush

Address v7 missed review from Eric

---

---
 drivers/iommu/dmar.c        | 36
++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.c |
3 ++- include/linux/intel-iommu.h | 20 ++++++++++++++++----
 3 files changed, 54 insertions(+), 5 deletions(-)
drivers/iommu/dmar.c +7 -0
index f77dae7ba7d4..4d6b7b5b37ee 100644
@@ -1421,6 +1421,42 @@ void qi_flush_piotlb(struct intel_iommu
 *iommu, u16 did, u32 pasid, u64 addr, qi_submit_sync(&desc, iommu);
 }
 
+/* PASID-based device IOTLB Invalidate */
+void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid,
 u16 pfsid,
+		u32 pasid,  u16 qdep, u64 addr, unsigned
 size_order, u64 granu) +{
+	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order -
 1);
+	struct qi_desc desc = {.qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_DEV_EIOTLB_PASID(pasid) |

QI_DEV_EIOTLB_SID(sid) |

drivers/iommu/dmar.c +6 -0
+		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
+		QI_DEV_IOTLB_PFSID(pfsid);
+	desc.qw1 = QI_DEV_EIOTLB_GLOB(granu);
+
+	/*
+	 * If S bit is 0, we only flush a single page. If S bit is

set,

drivers/iommu/dmar.c +1 -0
+	 * The least significant zero bit indicates the

invalidation address

drivers/iommu/dmar.c +2 -0
+	 * range. VT-d spec 6.5.2.6.
+	 * e.g. address bit 12[0] indicates 8KB, 13[0] indicates

16KB.

drivers/iommu/dmar.c +2 -0
+	 * size order = 0 is PAGE_SIZE 4KB
+	 * Max Invs Pending (MIP) is set to 0 for now until we

have DIT in

drivers/iommu/dmar.c +10 -0
+	 * ECAP.
+	 */
+	desc.qw1 |= addr & ~mask;
+	if (size_order)
+		desc.qw1 |= QI_DEV_EIOTLB_SIZE;
+
+	qi_submit_sync(&desc, iommu);
+}
+
+void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64

granu, int pasid) +{

drivers/iommu/dmar.c +3 -0
+	struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) |

QI_PC_GRAN(granu) | QI_PC_TYPE;

drivers/iommu/dmar.c +3 -0
+	qi_submit_sync(&desc, iommu);
+}
+
 /*
  * Disable Queued Invalidation interface.
  */
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 10c7856afc6b..9f6d07410722 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +0 -0
@@ -435,7 +435,8 @@ pasid_cache_invalidation_with_pasid(struct
 intel_iommu *iommu, {
 	struct qi_desc desc;
 
Quoted 6 lines

drivers/iommu/intel-pasid.c pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, · line 439

Quoted 4 lines
drivers/iommu/intel-pasid.c +2 -1
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL |
 QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;  

Just a nit, this fix is not documented in the commit message.

Thanks, I just sent out this fix separately. Will remove this from the set.
https://lkml.org/lkml/2020/3/30/1065

Quoted 57 lines

Besides

Thanks

Eric

Quoted 50 lines
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw1 = 0;
 	desc.qw2 = 0;
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw3 = 0;
(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 85b05120940e..43539713b3b3 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

(quoted diff) +0 -0
@@ -334,7 +334,7 @@ enum {
 #define QI_IOTLB_GRAN(gran) 	(((u64)gran) >>
 (DMA_TLB_FLUSH_GRANU_OFFSET-4)) #define QI_IOTLB_ADDR(addr)

(((u64)addr) & VTD_PAGE_MASK) #define
QI_IOTLB_IH(ih) (((u64)ih) << 6) -#define
QI_IOTLB_AM(am) (((u8)am)) +#define
QI_IOTLB_AM(am) (((u8)am) & 0x3f)
#define QI_CC_FM(fm) (((u64)fm) << 48)
#define QI_CC_SID(sid) (((u64)sid) << 32)

include/linux/intel-iommu.h +10 -3
@@ -353,16 +353,21 @@ enum {
 #define QI_PC_DID(did)		(((u64)did) << 16)
 #define QI_PC_GRAN(gran)	(((u64)gran) << 4)
 
-#define QI_PC_ALL_PASIDS	(QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL		(QI_PC_TYPE | QI_PC_GRAN(1))
+/* PASID cache invalidation granu */
+#define QI_PC_ALL_PASIDS	0
+#define QI_PC_PASID_SEL		1
 
 #define QI_EIOTLB_ADDR(addr)	((u64)(addr) & VTD_PAGE_MASK)
 #define QI_EIOTLB_IH(ih)	(((u64)ih) << 6)
-#define QI_EIOTLB_AM(am)	(((u64)am))
+#define QI_EIOTLB_AM(am)	(((u64)am) & 0x3f)
 #define QI_EIOTLB_PASID(pasid) 	(((u64)pasid) << 32)
 #define QI_EIOTLB_DID(did)	(((u64)did) << 16)
 #define QI_EIOTLB_GRAN(gran) 	(((u64)gran) << 4)
 
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0> +
 #define QI_DEV_EIOTLB_ADDR(a)	((u64)(a) & VTD_PAGE_MASK)
 #define QI_DEV_EIOTLB_SIZE	(((u64)1) << 11)
 #define QI_DEV_EIOTLB_GLOB(g)	((u64)g)
@@ -662,8 +667,15 @@ extern void qi_flush_iotlb(struct intel_iommu
 *iommu, u16 did, u64 addr, unsigned int size_order, u64 type);
 extern void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16 sid,
 u16 pfsid, u16 qdep, u64 addr, unsigned mask);
+
 void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32
 pasid, u64 addr, unsigned long npages, bool ih);
+
+extern void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu,
 u16 sid, u16 pfsid,
Quoted 9 lines
Quoted 8 lines
(quoted diff) +1 -0
+			u32 pasid, u16 qdep, u64 addr, unsigned

size_order, u64 granu); +
+extern void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid); +
extern int qi_submit_sync(struct qi_desc *desc, struct intel_iommu
*iommu);
extern int dmar_ir_support(void);

[Jacob Pan]

author
#35 lore raw

Re: [PATCH V10 07/11] iommu/vt-d: Support flushing more translation cache types

Quoted 95 lines · On Mon, 30 Mar 2020 16:28:34 -0700 Jacob Pan <[email protected]>…
Quoted 86 lines · On Fri, 27 Mar 2020 15:46:23 +0100 Auger Eric <[email protected]> wrote:

Hi Jacob,

Quoted 83 lines · Jacob Pan

When Shared Virtual Memory is exposed to a guest via vIOMMU, scalable IOTLB invalidation may be passed down from outside IOMMU subsystems. This patch adds invalidation functions that can be used for additional translation cache types.

---
v9 -> v10:
Fix off by 1 in pasid device iotlb flush

Address v7 missed review from Eric

---

---
 drivers/iommu/dmar.c        | 36
++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.c |
3 ++- include/linux/intel-iommu.h | 20 ++++++++++++++++----
 3 files changed, 54 insertions(+), 5 deletions(-)
drivers/iommu/dmar.c +7 -1
index f77dae7ba7d4..4d6b7b5b37ee 100644
@@ -1421,6 +1421,42 @@ void qi_flush_piotlb(struct intel_iommu
 *iommu, u16 did, u32 pasid, u64 addr, qi_submit_sync(&desc,
 iommu); }
 
+/* PASID-based device IOTLB Invalidate */
+void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid,
 u16 pfsid,
+		u32 pasid,  u16 qdep, u64 addr, unsigned
 size_order, u64 granu) +{
+	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order
- 1);
+	struct qi_desc desc = {.qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_DEV_EIOTLB_PASID(pasid) |

QI_DEV_EIOTLB_SID(sid) |

drivers/iommu/dmar.c +6 -0
+		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
+		QI_DEV_IOTLB_PFSID(pfsid);
+	desc.qw1 = QI_DEV_EIOTLB_GLOB(granu);
+
+	/*
+	 * If S bit is 0, we only flush a single page. If S bit

is set,

drivers/iommu/dmar.c +1 -0
+	 * The least significant zero bit indicates the

invalidation address

drivers/iommu/dmar.c +2 -0
+	 * range. VT-d spec 6.5.2.6.
+	 * e.g. address bit 12[0] indicates 8KB, 13[0] indicates

16KB.

drivers/iommu/dmar.c +2 -0
+	 * size order = 0 is PAGE_SIZE 4KB
+	 * Max Invs Pending (MIP) is set to 0 for now until we

have DIT in

drivers/iommu/dmar.c +10 -0
+	 * ECAP.
+	 */
+	desc.qw1 |= addr & ~mask;
+	if (size_order)
+		desc.qw1 |= QI_DEV_EIOTLB_SIZE;
+
+	qi_submit_sync(&desc, iommu);
+}
+
+void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64

granu, int pasid) +{

drivers/iommu/dmar.c +3 -0
+	struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) |

QI_PC_GRAN(granu) | QI_PC_TYPE;

drivers/iommu/dmar.c +3 -0
+	qi_submit_sync(&desc, iommu);
+}
+
 /*
  * Disable Queued Invalidation interface.
  */
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 10c7856afc6b..9f6d07410722 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +0 -0
@@ -435,7 +435,8 @@ pasid_cache_invalidation_with_pasid(struct
 intel_iommu *iommu, {
 	struct qi_desc desc;
 
Quoted 6 lines

drivers/iommu/intel-pasid.c pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, · line 439

Quoted 4 lines
drivers/iommu/intel-pasid.c +2 -1
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL |
 QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;    

Just a nit, this fix is not documented in the commit message.

Quoted 4 lines

Thanks, I just sent out this fix separately. Will remove this from the set.
https://lkml.org/lkml/2020/3/30/1065

I just realized this is not a fix. since I redefined below macros such that I could use them for granularity lookup without the QI_PC_TYPE.

-#define QI_PC_ALL_PASIDS (QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL (QI_PC_TYPE | QI_PC_GRAN(1))
+/* PASID cache invalidation granu */
+#define QI_PC_ALL_PASIDS 0
+#define QI_PC_PASID_SEL 1

Thanks,

Jacob

Quoted 67 lines
Quoted 57 lines

Besides

Thanks

Eric

Quoted 50 lines
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw1 = 0;
 	desc.qw2 = 0;
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw3 = 0;
(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 85b05120940e..43539713b3b3 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

(quoted diff) +0 -0
@@ -334,7 +334,7 @@ enum {
 #define QI_IOTLB_GRAN(gran) 	(((u64)gran) >>
 (DMA_TLB_FLUSH_GRANU_OFFSET-4)) #define QI_IOTLB_ADDR(addr)

(((u64)addr) & VTD_PAGE_MASK) #define
QI_IOTLB_IH(ih) (((u64)ih) << 6) -#define
QI_IOTLB_AM(am) (((u8)am)) +#define
QI_IOTLB_AM(am) (((u8)am) & 0x3f)
#define QI_CC_FM(fm) (((u64)fm) << 48)
#define QI_CC_SID(sid) (((u64)sid) << 32)

include/linux/intel-iommu.h +0 -2
@@ -353,16 +353,21 @@ enum {
 #define QI_PC_DID(did)		(((u64)did) << 16)
 #define QI_PC_GRAN(gran)	(((u64)gran) << 4)
 
-#define QI_PC_ALL_PASIDS	(QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL		(QI_PC_TYPE |

QI_PC_GRAN(1)) +/* PASID cache invalidation granu */

include/linux/intel-iommu.h +9 -1
+#define QI_PC_ALL_PASIDS	0
+#define QI_PC_PASID_SEL		1
 
 #define QI_EIOTLB_ADDR(addr)	((u64)(addr) & VTD_PAGE_MASK)
 #define QI_EIOTLB_IH(ih)	(((u64)ih) << 6)
-#define QI_EIOTLB_AM(am)	(((u64)am))
+#define QI_EIOTLB_AM(am)	(((u64)am) & 0x3f)
 #define QI_EIOTLB_PASID(pasid) 	(((u64)pasid) << 32)
 #define QI_EIOTLB_DID(did)	(((u64)did) << 16)
 #define QI_EIOTLB_GRAN(gran) 	(((u64)gran) << 4)
 
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0> +
 #define QI_DEV_EIOTLB_ADDR(a)	((u64)(a) & VTD_PAGE_MASK)
 #define QI_DEV_EIOTLB_SIZE	(((u64)1) << 11)
 #define QI_DEV_EIOTLB_GLOB(g)	((u64)g)
@@ -662,8 +667,15 @@ extern void qi_flush_iotlb(struct intel_iommu
 *iommu, u16 did, u64 addr, unsigned int size_order, u64 type);
 extern void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16
 sid, u16 pfsid, u16 qdep, u64 addr, unsigned mask);
+
 void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32
 pasid, u64 addr, unsigned long npages, bool ih);
+
+extern void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu,
 u16 sid, u16 pfsid,
Quoted 9 lines
Quoted 8 lines
(quoted diff) +1 -0
+			u32 pasid, u16 qdep, u64 addr, unsigned

size_order, u64 granu); +
+extern void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid); +
extern int qi_submit_sync(struct qi_desc *desc, struct
intel_iommu *iommu);
extern int dmar_ir_support(void);

Quoted 1 line

[Jacob Pan]

[Jacob Pan]

#36 lore raw

Re: [PATCH V10 07/11] iommu/vt-d: Support flushing more translation cache types

Hi Jacob,

Quoted 102 lines · Jacob Pan
Quoted 95 lines · On Mon, 30 Mar 2020 16:28:34 -0700 Jacob Pan <[email protected]>…
Quoted 86 lines · On Fri, 27 Mar 2020 15:46:23 +0100 Auger Eric <[email protected]> wrote:

Hi Jacob,

Quoted 83 lines · Jacob Pan

When Shared Virtual Memory is exposed to a guest via vIOMMU, scalable IOTLB invalidation may be passed down from outside IOMMU subsystems. This patch adds invalidation functions that can be used for additional translation cache types.

---
v9 -> v10:
Fix off by 1 in pasid device iotlb flush

Address v7 missed review from Eric

---

---
 drivers/iommu/dmar.c        | 36
++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.c |
3 ++- include/linux/intel-iommu.h | 20 ++++++++++++++++----
 3 files changed, 54 insertions(+), 5 deletions(-)
drivers/iommu/dmar.c +7 -1
index f77dae7ba7d4..4d6b7b5b37ee 100644
@@ -1421,6 +1421,42 @@ void qi_flush_piotlb(struct intel_iommu
 *iommu, u16 did, u32 pasid, u64 addr, qi_submit_sync(&desc,
 iommu); }
 
+/* PASID-based device IOTLB Invalidate */
+void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu, u16 sid,
 u16 pfsid,
+		u32 pasid,  u16 qdep, u64 addr, unsigned
 size_order, u64 granu) +{
+	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order
- 1);
+	struct qi_desc desc = {.qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_DEV_EIOTLB_PASID(pasid) |

QI_DEV_EIOTLB_SID(sid) |

drivers/iommu/dmar.c +6 -0
+		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
+		QI_DEV_IOTLB_PFSID(pfsid);
+	desc.qw1 = QI_DEV_EIOTLB_GLOB(granu);
+
+	/*
+	 * If S bit is 0, we only flush a single page. If S bit

is set,

drivers/iommu/dmar.c +1 -0
+	 * The least significant zero bit indicates the

invalidation address

drivers/iommu/dmar.c +2 -0
+	 * range. VT-d spec 6.5.2.6.
+	 * e.g. address bit 12[0] indicates 8KB, 13[0] indicates

16KB.

drivers/iommu/dmar.c +2 -0
+	 * size order = 0 is PAGE_SIZE 4KB
+	 * Max Invs Pending (MIP) is set to 0 for now until we

have DIT in

drivers/iommu/dmar.c +10 -0
+	 * ECAP.
+	 */
+	desc.qw1 |= addr & ~mask;
+	if (size_order)
+		desc.qw1 |= QI_DEV_EIOTLB_SIZE;
+
+	qi_submit_sync(&desc, iommu);
+}
+
+void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64

granu, int pasid) +{

drivers/iommu/dmar.c +3 -0
+	struct qi_desc desc = {.qw1 = 0, .qw2 = 0, .qw3 = 0};
+
+	desc.qw0 = QI_PC_PASID(pasid) | QI_PC_DID(did) |

QI_PC_GRAN(granu) | QI_PC_TYPE;

drivers/iommu/dmar.c +3 -0
+	qi_submit_sync(&desc, iommu);
+}
+
 /*
  * Disable Queued Invalidation interface.
  */
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 10c7856afc6b..9f6d07410722 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

drivers/iommu/intel-pasid.c +0 -0
@@ -435,7 +435,8 @@ pasid_cache_invalidation_with_pasid(struct
 intel_iommu *iommu, {
 	struct qi_desc desc;
 
Quoted 6 lines

drivers/iommu/intel-pasid.c pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, · line 439

Quoted 4 lines
drivers/iommu/intel-pasid.c +2 -1
-	desc.qw0 = QI_PC_DID(did) | QI_PC_PASID_SEL |
 QI_PC_PASID(pasid);
+	desc.qw0 = QI_PC_DID(did) | QI_PC_GRAN(QI_PC_PASID_SEL) |
+		QI_PC_PASID(pasid) | QI_PC_TYPE;    

Just a nit, this fix is not documented in the commit message.

Quoted 4 lines

Thanks, I just sent out this fix separately. Will remove this from the set.
https://lkml.org/lkml/2020/3/30/1065

Quoted 8 lines

I just realized this is not a fix. since I redefined below macros such that I could use them for granularity lookup without the QI_PC_TYPE.

-#define QI_PC_ALL_PASIDS (QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL (QI_PC_TYPE | QI_PC_GRAN(1))
+/* PASID cache invalidation granu */
+#define QI_PC_ALL_PASIDS 0
+#define QI_PC_PASID_SEL 1

ouah ok. So it improves code reading/consistency at least ;-)

Thanks

Eric

Quoted 76 lines

Thanks,

Jacob

Quoted 67 lines
Quoted 57 lines

Besides

Thanks

Eric

Quoted 50 lines
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw1 = 0;
 	desc.qw2 = 0;
drivers/iommu/intel-pasid.c +0 -0
 	desc.qw3 = 0;
(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 85b05120940e..43539713b3b3 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

(quoted diff) +0 -0
@@ -334,7 +334,7 @@ enum {
 #define QI_IOTLB_GRAN(gran) 	(((u64)gran) >>
 (DMA_TLB_FLUSH_GRANU_OFFSET-4)) #define QI_IOTLB_ADDR(addr)

(((u64)addr) & VTD_PAGE_MASK) #define
QI_IOTLB_IH(ih) (((u64)ih) << 6) -#define
QI_IOTLB_AM(am) (((u8)am)) +#define
QI_IOTLB_AM(am) (((u8)am) & 0x3f)
#define QI_CC_FM(fm) (((u64)fm) << 48)
#define QI_CC_SID(sid) (((u64)sid) << 32)

include/linux/intel-iommu.h +0 -2
@@ -353,16 +353,21 @@ enum {
 #define QI_PC_DID(did)		(((u64)did) << 16)
 #define QI_PC_GRAN(gran)	(((u64)gran) << 4)
 
-#define QI_PC_ALL_PASIDS	(QI_PC_TYPE | QI_PC_GRAN(0))
-#define QI_PC_PASID_SEL		(QI_PC_TYPE |

QI_PC_GRAN(1)) +/* PASID cache invalidation granu */

include/linux/intel-iommu.h +9 -1
+#define QI_PC_ALL_PASIDS	0
+#define QI_PC_PASID_SEL		1
 
 #define QI_EIOTLB_ADDR(addr)	((u64)(addr) & VTD_PAGE_MASK)
 #define QI_EIOTLB_IH(ih)	(((u64)ih) << 6)
-#define QI_EIOTLB_AM(am)	(((u64)am))
+#define QI_EIOTLB_AM(am)	(((u64)am) & 0x3f)
 #define QI_EIOTLB_PASID(pasid) 	(((u64)pasid) << 32)
 #define QI_EIOTLB_DID(did)	(((u64)did) << 16)
 #define QI_EIOTLB_GRAN(gran) 	(((u64)gran) << 4)
 
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0> +
 #define QI_DEV_EIOTLB_ADDR(a)	((u64)(a) & VTD_PAGE_MASK)
 #define QI_DEV_EIOTLB_SIZE	(((u64)1) << 11)
 #define QI_DEV_EIOTLB_GLOB(g)	((u64)g)
@@ -662,8 +667,15 @@ extern void qi_flush_iotlb(struct intel_iommu
 *iommu, u16 did, u64 addr, unsigned int size_order, u64 type);
 extern void qi_flush_dev_iotlb(struct intel_iommu *iommu, u16
 sid, u16 pfsid, u16 qdep, u64 addr, unsigned mask);
+
 void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32
 pasid, u64 addr, unsigned long npages, bool ih);
+
+extern void qi_flush_dev_iotlb_pasid(struct intel_iommu *iommu,
 u16 sid, u16 pfsid,
Quoted 9 lines
Quoted 8 lines
(quoted diff) +1 -0
+			u32 pasid, u16 qdep, u64 addr, unsigned

size_order, u64 granu); +
+extern void qi_flush_pasid_cache(struct intel_iommu *iommu, u16 did, u64 granu, int pasid); +
extern int qi_submit_sync(struct qi_desc *desc, struct
intel_iommu *iommu);
extern int dmar_ir_support(void);

Quoted 1 line

[Jacob Pan]

[Jacob Pan]

author
#37 lore raw

PATCH v10 8/11 iommu/vt-d: Add svm/sva invalidate function

In reply to Jacob Pan

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +182 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second level).
+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},
+	/* PASID cache */
+	{1, 1, 0}
+};
+
+const static int inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >= IOMMU_INV_GRANU_NR ||
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and number of
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info *inv_info)
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version != IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity, &granu);
+		if (ret) {
+			pr_err("Invalid cache type and granu combination %d/%d\n", cache_type,
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on granularity */
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags & IOMMU_INV_PASID_FLAGS_PASID)
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR &&
+			inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_PASID)
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and granularity\n");
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity != IOMMU_INV_GRANU_PASID) &&
+				size && (inv_info->addr_info.addr & ((BIT(VTD_PAGE_SHIFT + size)) - 1))) {
+				pr_err("Address out of range, 0x%llx, size order %llu\n",
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info->addr_info.addr),
+					(granu == QI_GRAN_NONG_PASID) ? -1 : 1 << size,
+					inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_LEAF);
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since guest
+			 * vIOMMU exposes CM = 1, no device IOTLB flush will be passed
+			 * down.
+			 */
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info->pfsid,
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr, size,
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info->pfsid,
+						inv_info->addr_info.pasid, info->ats_qdep,
+						inv_info->addr_info.addr, size,
+						granu);
+			} else
+				pr_warn("Passdown device IOTLB flush w/o ATS!\n");
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info->pasid_info.pasid);
+
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation type %d\n",
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#38 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 10 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 38 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +9 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second
Quoted 1 line

level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 13 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_ NR] = {

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 7 lines
drivers/iommu/intel-iommu.c +7 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

drivers/iommu/intel-iommu.c

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

Quoted 18 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU _NR] = {

drivers/iommu/intel-iommu.c +11 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Quoted 38 lines
drivers/iommu/intel-iommu.c +9 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and

number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info

*inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +5 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

Quoted 43 lines
drivers/iommu/intel-iommu.c +10 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache,

IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity,

&granu);

drivers/iommu/intel-iommu.c +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on granularity

*/

drivers/iommu/intel-iommu.c +2 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

drivers/iommu/intel-iommu.c +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR

&&

drivers/iommu/intel-iommu.c +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and

granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 23 lines
drivers/iommu/intel-iommu.c +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

drivers/iommu/intel-iommu.c +1 -0
+				pr_err("Address out of range, 0x%llx, size

order %llu\n",

drivers/iommu/intel-iommu.c +8 -0
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-
Quoted 1 line

addr_info.addr),

drivers/iommu/intel-iommu.c +1 -0
+					(granu == QI_GRAN_NONG_PASID) ? -

1 : 1 << size,

drivers/iommu/intel-iommu.c +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since

guest

drivers/iommu/intel-iommu.c +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB flush

will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 17 lines
drivers/iommu/intel-iommu.c +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +2 -0
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.pasid,

info->ats_qdep,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

Quoted 9 lines
drivers/iommu/intel-iommu.c +2 -0
+			} else
+				pr_warn("Passdown device IOTLB flush w/o

ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info-
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

Quoted 28 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation

type %d\n",

drivers/iommu/intel-iommu.c +13 -1
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--
Quoted 1 line

2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#39 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Hi,

Quoted 76 lines · Kevin
Quoted 10 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 38 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +9 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second
Quoted 1 line

level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 16 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_ NR] = {

drivers/iommu/intel-iommu.c +3 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
Quoted 8 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Isn't it the same when G=1? Still the addr/size is used. Doesn't it correspond to IOMMU_INV_GRANU_ADDR with IOMMU_INV_ADDR_FLAGS_PASID flag unset?

so {0, 0, 1}?

Thanks

Eric

Quoted 227 lines

drivers/iommu/intel-iommu.c

Quoted 2 lines
drivers/iommu/intel-iommu.c +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

Quoted 18 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU _NR] = {

drivers/iommu/intel-iommu.c +11 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Quoted 38 lines
drivers/iommu/intel-iommu.c +9 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and

number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info

*inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +5 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

Quoted 43 lines
drivers/iommu/intel-iommu.c +10 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache,

IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity,

&granu);

drivers/iommu/intel-iommu.c +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on granularity

*/

drivers/iommu/intel-iommu.c +2 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

drivers/iommu/intel-iommu.c +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR

&&

drivers/iommu/intel-iommu.c +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and

granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 23 lines
drivers/iommu/intel-iommu.c +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

drivers/iommu/intel-iommu.c +1 -0
+				pr_err("Address out of range, 0x%llx, size

order %llu\n",

drivers/iommu/intel-iommu.c +8 -0
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-
Quoted 1 line

addr_info.addr),

drivers/iommu/intel-iommu.c +1 -0
+					(granu == QI_GRAN_NONG_PASID) ? -

1 : 1 << size,

drivers/iommu/intel-iommu.c +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since

guest

drivers/iommu/intel-iommu.c +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB flush

will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 17 lines
drivers/iommu/intel-iommu.c +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +2 -0
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.pasid,

info->ats_qdep,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

Quoted 9 lines
drivers/iommu/intel-iommu.c +2 -0
+			} else
+				pr_warn("Passdown device IOTLB flush w/o

ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info-
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

Quoted 29 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation

type %d\n",

drivers/iommu/intel-iommu.c +13 -1
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--

2.7.4

#40 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 96 lines

From: Auger Eric <[email protected]>
Sent: Sunday, March 29, 2020 11:34 PM

Hi,

Quoted 5 lines · Kevin
Quoted 5 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 38 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 29 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +0 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
Quoted 2 lines

drivers/iommu/intel-iommu.c static void · line 5624

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity

to

Quoted 1 line

drivers/iommu/intel-iommu.c static void · line 5625

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 1 line

drivers/iommu/intel-iommu.c static void · line 5626

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 1 line

drivers/iommu/intel-iommu.c static void · line 5627

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+ * owns the first level page tables. Invalidations of translation caches in

the

Quoted 6 lines
Quoted 5 lines
drivers/iommu/intel-iommu.c +4 -0
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second

level).

Quoted 18 lines

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 12 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 4 lines
Quoted 4 lines

NR] = {

drivers/iommu/intel-iommu.c +3 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
Quoted 8 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 1 line

Isn't it the same when G=1? Still the addr/size is used. Doesn't it

I thought addr/size is not used when G=1, but it might be wrong. I'm checking with our vt-d spec owner.

Quoted 5 lines

correspond to IOMMU_INV_GRANU_ADDR with
IOMMU_INV_ADDR_FLAGS_PASID flag
unset?

so {0, 0, 1}?

I have one more open:

How does userspace know which invalidation type/gran is supported? I didn't see such capability reporting in Yi's VFIO vSVA patch set. Do we want the user/kernel assume the same capability set if they are architectural? However the kernel could also do some optimization e.g. hide devtlb invalidation capability given that the kernel already invalidate devtlb automatically when serving iotlb invalidation...

Thanks
Kevin

Quoted 233 lines

Thanks

Eric

Quoted 7 lines

drivers/iommu/intel-iommu.c

Quoted 2 lines
drivers/iommu/intel-iommu.c +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

Quoted 5 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain,

Quoted 5 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

Quoted 215 lines
Quoted 13 lines

_NR] = {

drivers/iommu/intel-iommu.c +11 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Quoted 38 lines
drivers/iommu/intel-iommu.c +9 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and

number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info

*inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +5 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

Quoted 43 lines
drivers/iommu/intel-iommu.c +10 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache,

IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity,

&granu);

drivers/iommu/intel-iommu.c +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on granularity

*/

drivers/iommu/intel-iommu.c +2 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

drivers/iommu/intel-iommu.c +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR

&&

drivers/iommu/intel-iommu.c +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and

granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 23 lines
drivers/iommu/intel-iommu.c +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

drivers/iommu/intel-iommu.c +1 -0
+				pr_err("Address out of range, 0x%llx, size

order %llu\n",

drivers/iommu/intel-iommu.c +8 -0
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-
Quoted 1 line

addr_info.addr),

drivers/iommu/intel-iommu.c +1 -0
+					(granu == QI_GRAN_NONG_PASID) ? -

1 : 1 << size,

drivers/iommu/intel-iommu.c +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since

guest

drivers/iommu/intel-iommu.c +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB flush

will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 17 lines
drivers/iommu/intel-iommu.c +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +2 -0
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.pasid,

info->ats_qdep,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

Quoted 9 lines
drivers/iommu/intel-iommu.c +2 -0
+			} else
+				pr_warn("Passdown device IOTLB flush w/o

ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info-
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

Quoted 29 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation

type %d\n",

drivers/iommu/intel-iommu.c +13 -1
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--

2.7.4

author
#41 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 48 lines · Kevin
Quoted 18 lines

From: Auger Eric <[email protected]>
Sent: Sunday, March 29, 2020 11:34 PM

Hi,

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 5 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

[...]
[...]

Quoted 10 lines

to

Quoted 2 lines
Quoted 2 lines

+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 2 lines
Quoted 2 lines

+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 2 lines
Quoted 2 lines

+ * owns the first level page tables. Invalidations of translation caches in

the

[...]
[...]
[...]

Quoted 15 lines

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 14 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 10 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

drivers/iommu/intel-iommu.c +4 -0
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},  

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 13 lines
Quoted 2 lines

Isn't it the same when G=1? Still the addr/size is used. Doesn't it

I thought addr/size is not used when G=1, but it might be wrong. I'm checking with our vt-d spec owner.

Quoted 5 lines

correspond to IOMMU_INV_GRANU_ADDR with
IOMMU_INV_ADDR_FLAGS_PASID flag
unset?

so {0, 0, 1}?

I am not sure I got your logic. The three fields correspond to
IOMMU_INV_GRANU_DOMAIN, /* domain-selective invalidation */
IOMMU_INV_GRANU_PASID, /* PASID-selective invalidation */
IOMMU_INV_GRANU_ADDR, /* page-selective invalidation *

For devTLB, we use domain as global since there is no domain. Then I came up with {1, 1, 0}, which means we could have global and pasid granu invalidation for PASID based devTLB.

If the caller also provide addr and S bit, the flush routine will put that into QI descriptor. I know this is a little odd, but from the granu translation p.o.v. VT-d spec has no G bit for page selective invalidation.

Quoted 10 lines

I have one more open:

How does userspace know which invalidation type/gran is supported? I didn't see such capability reporting in Yi's VFIO vSVA patch set. Do we want the user/kernel assume the same capability set if they are architectural? However the kernel could also do some optimization e.g. hide devtlb invalidation capability given that the kernel already invalidate devtlb automatically when serving iotlb invalidation...

In general, we are trending to use VFIO capability chain to expose iommu capabilities.

But for architectural features such as type/granu, we have to assume the same capability between host & guest. Granu and types are not enumerated on the host IOMMU either.

For devTLB optimization, I agree we need to expose a capability to the guest stating that implicit devtlb invalidation is supported. Otherwise, if Linux guest runs on other OSes may not support implicit devtlb invalidation.

Right Yi?

Quoted 49 lines

Thanks
Kevin

Quoted 12 lines

Thanks

Eric

Quoted 7 lines

Review

Quoted 2 lines
(quoted diff) +1 -0
+	/* PASID cache */  

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

[...]

Quoted 1 line

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

[...]

Quoted 5 lines
Quoted 5 lines

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

[...]

Quoted 3 lines
Quoted 3 lines

-ENOTSUPP?

[...]

Quoted 4 lines
Quoted 4 lines

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

[...]

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

addr_info.addr),

[...]
[...]

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
(quoted diff) +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

[...]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line

pfsid,

+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

(quoted diff) +1 -0
+						granu);  

[...]
[...]

Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

[...]

Quoted 1 line
Quoted 1 line

[Jacob Pan]

#42 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 74 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 4:58 AM

Quoted 26 lines · Kevin
Quoted 18 lines

From: Auger Eric <[email protected]>
Sent: Sunday, March 29, 2020 11:34 PM

Hi,

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 5 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

[...]
[...]

Quoted 6 lines

to

Quoted 2 lines
Quoted 2 lines

+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 2 lines
Quoted 2 lines

+ * as a result of DMA or VFIO unmap. However, for assigned devices

Quoted 8 lines
Quoted 4 lines

guest

Quoted 2 lines
Quoted 2 lines

+ * owns the first level page tables. Invalidations of translation caches in

the

[...]
[...]
[...]

Quoted 1 line

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 10 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

drivers/iommu/intel-iommu.c +4 -0
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 13 lines
Quoted 2 lines

Isn't it the same when G=1? Still the addr/size is used. Doesn't it

I thought addr/size is not used when G=1, but it might be wrong. I'm checking with our vt-d spec owner.

Quoted 5 lines

correspond to IOMMU_INV_GRANU_ADDR with
IOMMU_INV_ADDR_FLAGS_PASID flag
unset?

so {0, 0, 1}?

I am not sure I got your logic. The three fields correspond to
IOMMU_INV_GRANU_DOMAIN, /* domain-selective
invalidation */
IOMMU_INV_GRANU_PASID, /* PASID-selective invalidation */
IOMMU_INV_GRANU_ADDR, /* page-selective invalidation *

Quoted 5 lines

For devTLB, we use domain as global since there is no domain. Then I came up with {1, 1, 0}, which means we could have global and pasid granu invalidation for PASID based devTLB.

If the caller also provide addr and S bit, the flush routine will put

"also" -> "must", because vt-d requires addr/size must be provided in devtlb descriptor, that is why Eric suggests {0, 0, 1}.

Quoted 3 lines

that into QI descriptor. I know this is a little odd, but from the granu translation p.o.v. VT-d spec has no G bit for page selective invalidation.

We don't need such odd way if can do it properly. 😊

Quoted 18 lines
Quoted 11 lines

I have one more open:

How does userspace know which invalidation type/gran is supported? I didn't see such capability reporting in Yi's VFIO vSVA patch set. Do we want the user/kernel assume the same capability set if they are architectural? However the kernel could also do some optimization e.g. hide devtlb invalidation capability given that the kernel already invalidate devtlb automatically when serving iotlb invalidation...

In general, we are trending to use VFIO capability chain to expose iommu capabilities.

But for architectural features such as type/granu, we have to assume the same capability between host & guest. Granu and types are not enumerated on the host IOMMU either.

Quoted 6 lines

For devTLB optimization, I agree we need to expose a capability to the guest stating that implicit devtlb invalidation is supported. Otherwise, if Linux guest runs on other OSes may not support implicit devtlb invalidation.

Right Yi?

Thanks for explanation. So we are assumed to support all operations defined in spec, so no need to expose them one-by-one. For optimization, I'm fine to do it later.

Quoted 54 lines
Quoted 10 lines

Thanks
Kevin

Quoted 6 lines

Thanks

Eric

Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Quoted 6 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

[...]

Quoted 1 line

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

Quoted 35 lines

[...]

Quoted 5 lines
Quoted 5 lines

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

[...]

Quoted 3 lines
Quoted 3 lines

-ENOTSUPP?

[...]

Quoted 4 lines
Quoted 4 lines

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

[...]

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

addr_info.addr),

[...]
[...]

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
(quoted diff) +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

[...]

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line

pfsid,

+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

(quoted diff) +1 -0
+						granu);

[...]
[...]

Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Quoted 1 line

pasid_info.pasid);

+

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

[...]

Quoted 1 line
Quoted 1 line

[Jacob Pan]

Thanks
Kevin
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#43 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 83 lines

From: Tian, Kevin <[email protected]>
Sent: Wednesday, April 1, 2020 2:30 PM
To: Jacob Pan <[email protected]>
Subject: RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 33 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 4:58 AM

Quoted 27 lines · Kevin
Quoted 18 lines

From: Auger Eric <[email protected]>
Sent: Sunday, March 29, 2020 11:34 PM

Hi,

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 5 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

[...]
[...]

Quoted 7 lines

to

Quoted 2 lines
Quoted 2 lines

+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 2 lines
Quoted 2 lines

+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 30 lines
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines

+ * owns the first level page tables. Invalidations of translation caches in

the

[...]
[...]
[...]

Quoted 1 line

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 22 lines
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

(quoted diff) +2 -0
+	 * page selective (address granularity)
+	 */
Quoted 8 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 4 lines
(quoted diff) +2 -0
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 2 lines

Isn't it the same when G=1? Still the addr/size is used. Doesn't it

I thought addr/size is not used when G=1, but it might be wrong. I'm checking with our vt-d spec owner.

Quoted 1 line

correspond to IOMMU_INV_GRANU_ADDR with

IOMMU_INV_ADDR_FLAGS_PASID

Quoted 14 lines
Quoted 4 lines
Quoted 3 lines

flag unset?

so {0, 0, 1}?

I am not sure I got your logic. The three fields correspond to
IOMMU_INV_GRANU_DOMAIN, /* domain-selective
invalidation */
IOMMU_INV_GRANU_PASID, /* PASID-selective invalidation */
IOMMU_INV_GRANU_ADDR, /* page-selective invalidation *

For devTLB, we use domain as global since there is no domain. Then I came up with {1, 1, 0}, which means we could have global and pasid granu invalidation for PASID based devTLB.

Quoted 5 lines
Quoted 1 line

If the caller also provide addr and S bit, the flush routine will put

"also" -> "must", because vt-d requires addr/size must be provided in devtlb
descriptor, that is why Eric suggests {0, 0, 1}.

I think it should be {0, 0, 1} :-) addr field and S field are must, pasid field depends on G bit.

I didn’t read through all comments. Here is a concern with this 2-D table, the iommu cache type is defined as below. I suppose there is a problem here. If I'm using IOMMU_CACHE_INV_TYPE_PASID, it will beyond the 2-D table.

/* IOMMU paging structure cache */

#define IOMMU_CACHE_INV_TYPE_IOTLB      (1 << 0) /* IOMMU IOTLB */
#define IOMMU_CACHE_INV_TYPE_DEV_IOTLB  (1 << 1) /* Device IOTLB */
#define IOMMU_CACHE_INV_TYPE_PASID      (1 << 2) /* PASID cache */
#define IOMMU_CACHE_INV_TYPE_NR         (3)
Quoted 23 lines
Quoted 23 lines
Quoted 11 lines

I have one more open:

How does userspace know which invalidation type/gran is supported? I didn't see such capability reporting in Yi's VFIO vSVA patch set. Do we want the user/kernel assume the same capability set if they are architectural? However the kernel could also do some optimization e.g. hide devtlb invalidation capability given that the kernel already invalidate devtlb automatically when serving iotlb invalidation...

In general, we are trending to use VFIO capability chain to expose iommu capabilities.

But for architectural features such as type/granu, we have to assume the same capability between host & guest. Granu and types are not enumerated on the host IOMMU either.

For devTLB optimization, I agree we need to expose a capability to the guest stating that implicit devtlb invalidation is supported. Otherwise, if Linux guest runs on other OSes may not support implicit devtlb invalidation.

Quoted 5 lines
Quoted 1 line

Right Yi?

Thanks for explanation. So we are assumed to support all operations defined in spec, so no need to expose them one-by-one. For optimization, I'm fine to do it later.

yes. :-)

Regards,
Yi Liu

#44 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Hi,

Quoted 83 lines · Yi L
Quoted 83 lines

From: Tian, Kevin <[email protected]>
Sent: Wednesday, April 1, 2020 2:30 PM
To: Jacob Pan <[email protected]>
Subject: RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 33 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 4:58 AM

Quoted 27 lines · Kevin
Quoted 18 lines

From: Auger Eric <[email protected]>
Sent: Sunday, March 29, 2020 11:34 PM

Hi,

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 5 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

[...]
[...]

Quoted 7 lines

to

Quoted 2 lines
Quoted 2 lines

+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 2 lines
Quoted 2 lines

+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 30 lines
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
Quoted 2 lines

+ * owns the first level page tables. Invalidations of translation caches in

the

[...]
[...]
[...]

Quoted 1 line

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 22 lines
Quoted 14 lines
Quoted 6 lines
Quoted 6 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

(quoted diff) +2 -0
+	 * page selective (address granularity)
+	 */
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

(quoted diff) +1 -0
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 2 lines

Isn't it the same when G=1? Still the addr/size is used. Doesn't it

I thought addr/size is not used when G=1, but it might be wrong. I'm checking with our vt-d spec owner.

Quoted 1 line

correspond to IOMMU_INV_GRANU_ADDR with

IOMMU_INV_ADDR_FLAGS_PASID

Quoted 14 lines
Quoted 4 lines
Quoted 3 lines

flag unset?

so {0, 0, 1}?

I am not sure I got your logic. The three fields correspond to
IOMMU_INV_GRANU_DOMAIN, /* domain-selective
invalidation */
IOMMU_INV_GRANU_PASID, /* PASID-selective invalidation */
IOMMU_INV_GRANU_ADDR, /* page-selective invalidation *

For devTLB, we use domain as global since there is no domain. Then I came up with {1, 1, 0}, which means we could have global and pasid granu invalidation for PASID based devTLB.

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

If the caller also provide addr and S bit, the flush routine will put

"also" -> "must", because vt-d requires addr/size must be provided in devtlb
descriptor, that is why Eric suggests {0, 0, 1}.

I think it should be {0, 0, 1} :-) addr field and S field are must, pasid field depends on G bit.

On my side, I understood from the spec that addr/S are always used whatever the granularity, hence the above suggestion.

As a comparison, for PASID based IOTLB invalidation, it is clearly stated that if G matches PASID selective invalidation, address field is ignored. This is not written that way for PASID-based device TLB inv.

Quoted 10 lines

I didn’t read through all comments. Here is a concern with this 2-D table, the iommu cache type is defined as below. I suppose there is a problem here. If I'm using IOMMU_CACHE_INV_TYPE_PASID, it will beyond the 2-D table.

/* IOMMU paging structure cache */

#define IOMMU_CACHE_INV_TYPE_IOTLB      (1 << 0) /* IOMMU IOTLB */
#define IOMMU_CACHE_INV_TYPE_DEV_IOTLB  (1 << 1) /* Device IOTLB */
#define IOMMU_CACHE_INV_TYPE_PASID      (1 << 2) /* PASID cache */
#define IOMMU_CACHE_INV_TYPE_NR         (3)

oups indeed

Thanks

Eric

Quoted 35 lines
Quoted 24 lines
Quoted 24 lines
Quoted 4 lines

I have one more open:

Quoted 8 lines

How does userspace know which invalidation type/gran is supported? I didn't see such capability reporting in Yi's VFIO vSVA patch set. Do we want the user/kernel assume the same capability set if they are architectural? However the kernel could also do some optimization e.g. hide devtlb invalidation capability given that the kernel already invalidate devtlb automatically when serving iotlb invalidation...

In general, we are trending to use VFIO capability chain to expose iommu capabilities.

But for architectural features such as type/granu, we have to assume the same capability between host & guest. Granu and types are not enumerated on the host IOMMU either.

For devTLB optimization, I agree we need to expose a capability to the guest stating that implicit devtlb invalidation is supported. Otherwise, if Linux guest runs on other OSes may not support implicit devtlb invalidation.

Quoted 5 lines
Quoted 1 line

Right Yi?

Thanks for explanation. So we are assumed to support all operations defined in spec, so no need to expose them one-by-one. For optimization, I'm fine to do it later.

yes. :-)

Regards,
Yi Liu

author
#45 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 6 lines · On Wed, 1 Apr 2020 09:32:37 +0200 Auger Eric <[email protected]> wrote:
Quoted 5 lines
Quoted 2 lines

devtlb
descriptor, that is why Eric suggests {0, 0, 1}.

I think it should be {0, 0, 1} :-) addr field and S field are must, pasid field depends on G bit.

Quoted 8 lines

On my side, I understood from the spec that addr/S are always used whatever the granularity, hence the above suggestion.

As a comparison, for PASID based IOTLB invalidation, it is clearly stated that if G matches PASID selective invalidation, address field is ignored. This is not written that way for PASID-based device TLB inv.

Quoted 1 line

I misread the S bit. It all makes sense now. Thanks for the proposal and explanation.

Jacob

author
#46 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

In reply to Auger Eric
Quoted 11 lines · On Wed, 1 Apr 2020 09:32:37 +0200 Auger Eric <[email protected]> wrote:
Quoted 10 lines

I didn’t read through all comments. Here is a concern with this 2-D table, the iommu cache type is defined as below. I suppose there is a problem here. If I'm using IOMMU_CACHE_INV_TYPE_PASID, it will beyond the 2-D table.

/* IOMMU paging structure cache */

#define IOMMU_CACHE_INV_TYPE_IOTLB      (1 << 0) /* IOMMU IOTLB */
#define IOMMU_CACHE_INV_TYPE_DEV_IOTLB  (1 << 1) /* Device IOTLB */
#define IOMMU_CACHE_INV_TYPE_PASID      (1 << 2) /* PASID cache */
#define IOMMU_CACHE_INV_TYPE_NR         (3)  

oups indeed

I think it is not an issue, since we use bit position not the raw cache type as index into the 2D array. Right?

for_each_set_bit(cache_type,

ret = to_vtd_granularity(cache_type, inv_info->granularity, &

static inline int to_vtd_granularity(int type, int granu, int *vtd_granu) {

	*vtd_granu = inv_type_granu_table[type][granu];
#47 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

In reply to Tian, Kevin
Quoted 85 lines · Kevin
Quoted 10 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 38 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +9 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second
Quoted 1 line

level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 13 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_ NR] = {

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 7 lines
drivers/iommu/intel-iommu.c +7 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 5 lines

drivers/iommu/intel-iommu.c

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

I tend to agree here.

Quoted 20 lines
Quoted 20 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU _NR] = {

drivers/iommu/intel-iommu.c +11 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c +1 -0
+		!inv_type_granu_map[type][granu])
Quoted 8 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

I asked the same question some time ago. If I remember correctly the issue is while a granu can be supported in inv_type_granu_map, the associated value in inv_type_granu_table can be 0. This typically matches both values of G field (0 or 1) in the invalidation cmd. See other comment below.

Quoted 141 lines
Quoted 39 lines
drivers/iommu/intel-iommu.c +9 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and

number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info

*inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +5 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

Quoted 43 lines
drivers/iommu/intel-iommu.c +10 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache,

IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity,

&granu);

drivers/iommu/intel-iommu.c +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on granularity

*/

drivers/iommu/intel-iommu.c +2 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

drivers/iommu/intel-iommu.c +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR

&&

drivers/iommu/intel-iommu.c +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and

granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 23 lines
drivers/iommu/intel-iommu.c +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

drivers/iommu/intel-iommu.c +1 -0
+				pr_err("Address out of range, 0x%llx, size

order %llu\n",

drivers/iommu/intel-iommu.c +8 -0
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-
Quoted 1 line

addr_info.addr),

drivers/iommu/intel-iommu.c +1 -0
+					(granu == QI_GRAN_NONG_PASID) ? -

1 : 1 << size,

drivers/iommu/intel-iommu.c +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since

guest

drivers/iommu/intel-iommu.c +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB flush

will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 13 lines
drivers/iommu/intel-iommu.c +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +2 -0
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

Quoted 8 lines

drivers/iommu/intel-iommu.c

Quoted 5 lines
drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.pasid,

info->ats_qdep,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

granu is the result of to_vtd_granularity() so it can take either of the 2 values.

Thanks

Eric

Quoted 44 lines
Quoted 10 lines
drivers/iommu/intel-iommu.c +2 -0
+			} else
+				pr_warn("Passdown device IOTLB flush w/o

ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info-
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

Quoted 29 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation

type %d\n",

drivers/iommu/intel-iommu.c +13 -1
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--

2.7.4

#48 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 130 lines

From: Auger Eric <[email protected]>
Sent: Monday, March 30, 2020 12:05 AM

Quoted 5 lines · Kevin
Quoted 5 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 38 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 29 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +0 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
Quoted 2 lines

drivers/iommu/intel-iommu.c static void · line 5624

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity

to

Quoted 1 line

drivers/iommu/intel-iommu.c static void · line 5625

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 1 line

drivers/iommu/intel-iommu.c static void · line 5626

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 1 line

drivers/iommu/intel-iommu.c static void · line 5627

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+ * owns the first level page tables. Invalidations of translation caches in

the

Quoted 6 lines
Quoted 5 lines
drivers/iommu/intel-iommu.c +4 -0
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second

level).

Quoted 18 lines

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 12 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 13 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 8 lines

NR] = {

drivers/iommu/intel-iommu.c +7 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 5 lines

drivers/iommu/intel-iommu.c

Quoted 1 line
drivers/iommu/intel-iommu.c +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

I tend to agree here.

Quoted 6 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain,

Quoted 6 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

Quoted 14 lines
Quoted 14 lines

_NR] = {

drivers/iommu/intel-iommu.c +11 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c +1 -0
+		!inv_type_granu_map[type][granu])
Quoted 8 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Quoted 5 lines

I asked the same question some time ago. If I remember correctly the issue is while a granu can be supported in inv_type_granu_map, the associated value in inv_type_granu_table can be 0. This typically matches both values of G field (0 or 1) in the invalidation cmd. See other comment below.

I didn't fully understand it. Also what does a value '0' imply? also it's interesting to see below in [PATCH 07/11]:

+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL 1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL 0
+

Quoted 149 lines
Quoted 141 lines
Quoted 39 lines
drivers/iommu/intel-iommu.c +9 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and

number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info

*inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +5 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

Quoted 43 lines
drivers/iommu/intel-iommu.c +10 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache,

IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity,

&granu);

drivers/iommu/intel-iommu.c +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on granularity

*/

drivers/iommu/intel-iommu.c +2 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

drivers/iommu/intel-iommu.c +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR

&&

drivers/iommu/intel-iommu.c +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and

granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 23 lines
drivers/iommu/intel-iommu.c +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

drivers/iommu/intel-iommu.c +1 -0
+				pr_err("Address out of range, 0x%llx, size

order %llu\n",

drivers/iommu/intel-iommu.c +8 -0
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-
Quoted 1 line

addr_info.addr),

drivers/iommu/intel-iommu.c +1 -0
+					(granu == QI_GRAN_NONG_PASID) ? -

1 : 1 << size,

drivers/iommu/intel-iommu.c +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since

guest

drivers/iommu/intel-iommu.c +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB flush

will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 13 lines
drivers/iommu/intel-iommu.c +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +2 -0
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu, sid, info-
Quoted 1 line

pfsid,

Quoted 8 lines

drivers/iommu/intel-iommu.c

Quoted 5 lines
drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.pasid,

info->ats_qdep,

drivers/iommu/intel-iommu.c +1 -0
+						inv_info->addr_info.addr,

size,

drivers/iommu/intel-iommu.c +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

Quoted 2 lines

granu is the result of to_vtd_granularity() so it can take either of the 2 values.

yes, you're right.

Quoted 48 lines

Thanks

Eric

Quoted 44 lines
Quoted 10 lines
drivers/iommu/intel-iommu.c +2 -0
+			} else
+				pr_warn("Passdown device IOTLB flush w/o

ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info-
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

Quoted 29 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation

type %d\n",

drivers/iommu/intel-iommu.c +13 -1
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--

2.7.4

author
#49 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 140 lines · Kevin
Quoted 140 lines

From: Auger Eric <[email protected]>
Sent: Monday, March 30, 2020 12:05 AM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 38 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 29 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

drivers/iommu/intel-iommu.c +0 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
Quoted 3 lines
Quoted 3 lines
(quoted diff) +2 -0
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity

to

Quoted 2 lines
Quoted 2 lines

+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 2 lines
Quoted 2 lines

+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 2 lines
Quoted 2 lines

+ * owns the first level page tables. Invalidations of translation caches in

the

Quoted 7 lines
Quoted 6 lines

+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second level).

Quoted 19 lines

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 13 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 15 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 10 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

drivers/iommu/intel-iommu.c +4 -0
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},  

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 6 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */  

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

I tend to agree here.

Quoted 6 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain,

Quoted 6 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

Quoted 22 lines
Quoted 13 lines

_NR] = {

drivers/iommu/intel-iommu.c +9 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int

*vtd_granu) +{

(quoted diff) +1 -0
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+  

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Quoted 9 lines
Quoted 5 lines

I asked the same question some time ago. If I remember correctly the issue is while a granu can be supported in inv_type_granu_map, the associated value in inv_type_granu_table can be 0. This typically matches both values of G field (0 or 1) in the invalidation cmd. See other comment below.

I didn't fully understand it. Also what does a value '0' imply? also it's interesting to see below in [PATCH 07/11]:

0 in 2D map array means invalid.
0 in granu table can be either valid or invalid
That is why we need the map table to tell the difference. I will add following comments since this causes lots of confusion.

* Granu_map array indicates validity of the table. 1: valid, 0: invalid
* This is useful when the entry in the granu table has a value of 0,
* which can be a valid or invalid value.

Review

Quoted 5 lines
(quoted diff) +4 -0
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0
+

Sorry I didn't get the point? These are the valid vt-d granu values. Per Spec CH 6.5.2.6

Quoted 162 lines
Quoted 162 lines
Quoted 162 lines
Quoted 50 lines
drivers/iommu/intel-iommu.c +6 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >>

VTD_PAGE_SHIFT; +

(quoted diff) +1 -0
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for

4k, 9 for 2MB, etc.

(quoted diff) +1 -0
+	 * IOMMU cache invalidate API passes granu_size in

bytes, and number of

drivers/iommu/intel-iommu.c +7 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain

*domain,

(quoted diff) +1 -0
+		struct device *dev, struct

iommu_cache_invalidate_info *inv_info)

(quoted diff) +2 -0
+{
+	struct dmar_domain *dmar_domain =

to_dmar_domain(domain);

drivers/iommu/intel-iommu.c +11 -0
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +12 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus,

devfn);

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 3 lines
drivers/iommu/intel-iommu.c +3 -0
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;  

-ENOTSUPP?

Quoted 46 lines
drivers/iommu/intel-iommu.c +5 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective

invalidation */

(quoted diff) +2 -0
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size =

to_vtd_size(inv_info->addr_info.granule_size,
+
inv_info->addr_info.nb_granules); +

(quoted diff) +1 -0
+	for_each_set_bit(cache_type, (unsigned long

*)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type,

inv_info->granularity, &granu);

(quoted diff) +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based

on granularity */

(quoted diff) +1 -0
+		if (inv_info->granularity ==

IOMMU_INV_GRANU_PASID &&

(quoted diff) +1 -0
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

(quoted diff) +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity ==

IOMMU_INV_GRANU_ADDR &&

(quoted diff) +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given

cache type and granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 26 lines
(quoted diff) +1 -0
+				size &&

(inv_info->addr_info.addr & ((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

(quoted diff) +1 -0
+				pr_err("Address out of range,

0x%llx, size order %llu\n",
+
inv_info->addr_info.addr, size);

drivers/iommu/intel-iommu.c +7 -0
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+

mm_to_dma_pfn(inv_info-

Quoted 1 line

addr_info.addr),

(quoted diff) +1 -0
+					(granu ==

QI_GRAN_NONG_PASID) ? - 1 : 1 << size,
+
inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is

enabled since guest

(quoted diff) +1 -0
+			 * vIOMMU exposes CM = 1, no device

IOTLB flush will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */  

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 20 lines
(quoted diff) +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						pasid,

info->ats_qdep,
+
inv_info->addr_info.addr, size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

Review

Quoted 1 line
(quoted diff) +1 -0
+						granu);  

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

Quoted 55 lines
Quoted 2 lines

granu is the result of to_vtd_granularity() so it can take either of the 2 values.

yes, you're right.

Quoted 49 lines

Thanks

Eric

Quoted 45 lines
Quoted 11 lines
(quoted diff) +2 -0
+			} else
+				pr_warn("Passdown device IOTLB

flush w/o ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu,

inv_info-

Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

Quoted 29 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU

invalidation type %d\n",

drivers/iommu/intel-iommu.c +12 -0
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t

gfp) @@ -6204,6 +6385,7 @@ const struct iommu_ops
intel_iommu_ops = { .is_attach_deferred =
intel_iommu_is_attach_deferred, .pgsize_bitmap =
INTEL_IOMMU_PGSIZES, #ifdef CONFIG_INTEL_IOMMU_SVM

drivers/iommu/intel-iommu.c +1 -1
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--

2.7.4

[Jacob Pan]

#50 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 167 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 5:08 AM

Quoted 75 lines · Kevin
Quoted 75 lines

From: Auger Eric <[email protected]>
Sent: Monday, March 30, 2020 12:05 AM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and

driver

Quoted 38 lines
Quoted 5 lines

level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

Quoted 29 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

drivers/iommu/intel-iommu.c +0 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
Quoted 3 lines
Quoted 3 lines
(quoted diff) +2 -0
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity

to

Quoted 2 lines
Quoted 2 lines

+ * VT-d granularity. Invalidation is typically included in the unmap

operation

Quoted 2 lines
Quoted 2 lines

+ * as a result of DMA or VFIO unmap. However, for assigned devices

guest

Quoted 2 lines
Quoted 2 lines

+ * owns the first level page tables. Invalidations of translation caches in

the

Quoted 13 lines
Quoted 6 lines

+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity

Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 10 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

drivers/iommu/intel-iommu.c +4 -0
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

Quoted 8 lines
Quoted 8 lines

I tend to agree here.

Quoted 6 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain,

Quoted 6 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

Quoted 22 lines
Quoted 22 lines
Quoted 22 lines
Quoted 13 lines

_NR] = {

drivers/iommu/intel-iommu.c +9 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int

*vtd_granu) +{

(quoted diff) +1 -0
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Quoted 9 lines
Quoted 5 lines

I asked the same question some time ago. If I remember correctly the issue is while a granu can be supported in inv_type_granu_map, the associated value in inv_type_granu_table can be 0. This typically matches both values of G field (0 or 1) in the invalidation cmd. See other comment below.

I didn't fully understand it. Also what does a value '0' imply? also it's interesting to see below in [PATCH 07/11]:

0 in 2D map array means invalid.
0 in granu table can be either valid or invalid
That is why we need the map table to tell the difference. I will add following comments since this causes lots of confusion.

* Granu_map array indicates validity of the table. 1: valid, 0: invalid
* This is useful when the entry in the granu table has a value of 0,
* which can be a valid or invalid value.

Quoted 7 lines

Review

Quoted 5 lines
(quoted diff) +4 -0
+/* QI Dev-IOTLB inv granu */
+#define QI_DEV_IOTLB_GRAN_ALL		1
+#define QI_DEV_IOTLB_GRAN_PASID_SEL	0
+

Sorry I didn't get the point? These are the valid vt-d granu values. Per Spec CH 6.5.2.6

well, I thought '0' means invalid then why we define a valid granu as invalid. But with your latest explanation I think I get the rationale behind now.

just one more thought, since the element type is int in the granular table, why not using -1 to represent invalid then you can still use one table?

Quoted 219 lines
Quoted 163 lines
Quoted 163 lines
Quoted 163 lines
Quoted 51 lines
drivers/iommu/intel-iommu.c +6 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >>

VTD_PAGE_SHIFT; +

(quoted diff) +1 -0
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for

4k, 9 for 2MB, etc.

(quoted diff) +1 -0
+	 * IOMMU cache invalidate API passes granu_size in

bytes, and number of

drivers/iommu/intel-iommu.c +7 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain

*domain,

(quoted diff) +1 -0
+		struct device *dev, struct

iommu_cache_invalidate_info *inv_info)

(quoted diff) +2 -0
+{
+	struct dmar_domain *dmar_domain =

to_dmar_domain(domain);

drivers/iommu/intel-iommu.c +11 -0
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +12 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus,

devfn);

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 3 lines
drivers/iommu/intel-iommu.c +3 -0
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

Quoted 46 lines
drivers/iommu/intel-iommu.c +5 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective

invalidation */

(quoted diff) +2 -0
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size =

to_vtd_size(inv_info->addr_info.granule_size,
+
inv_info->addr_info.nb_granules); +

(quoted diff) +1 -0
+	for_each_set_bit(cache_type, (unsigned long

*)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type,

inv_info->granularity, &granu);

(quoted diff) +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based

on granularity */

(quoted diff) +1 -0
+		if (inv_info->granularity ==

IOMMU_INV_GRANU_PASID &&

(quoted diff) +1 -0
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

(quoted diff) +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity ==

IOMMU_INV_GRANU_ADDR &&

(quoted diff) +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given

cache type and granularity\n");

drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=
Quoted 1 line

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Quoted 26 lines
(quoted diff) +1 -0
+				size &&

(inv_info->addr_info.addr & ((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

(quoted diff) +1 -0
+				pr_err("Address out of range,

0x%llx, size order %llu\n",
+
inv_info->addr_info.addr, size);

drivers/iommu/intel-iommu.c +7 -0
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+

mm_to_dma_pfn(inv_info-

Quoted 1 line

addr_info.addr),

(quoted diff) +1 -0
+					(granu ==

QI_GRAN_NONG_PASID) ? - 1 : 1 << size,
+
inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is

enabled since guest

(quoted diff) +1 -0
+			 * vIOMMU exposes CM = 1, no device

IOTLB flush will be passed

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 2 lines
drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 20 lines
(quoted diff) +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						pasid,

info->ats_qdep,
+
inv_info->addr_info.addr, size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

Review

Quoted 1 line
(quoted diff) +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

Quoted 55 lines
Quoted 2 lines

granu is the result of to_vtd_granularity() so it can take either of the 2 values.

yes, you're right.

Quoted 49 lines

Thanks

Eric

Quoted 45 lines
Quoted 11 lines
(quoted diff) +2 -0
+			} else
+				pr_warn("Passdown device IOTLB

flush w/o ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu,

inv_info-

Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

29 lines not shown · raw message

Quoted 1 line

1 line not shown · raw message

2 lines not shown · raw message

Part of this message is not shown here. View the raw message.

author
#51 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

In reply to Tian, Kevin
Quoted 8 lines · Kevin
Quoted 8 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

Quoted 6 lines
Quoted 2 lines

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

True, but it does not invalidate this statement about emulated IOMMU. I will add another statement saying "the same interface can be used for virtio-IOMMU as well". OK?

Quoted 42 lines
Quoted 42 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

drivers/iommu/intel-iommu.c +2 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore

Quoted 9 lines
Quoted 2 lines

+ * we do not include IOTLB granularity for request without PASID (second level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Sounds good.

Quoted 19 lines
Quoted 19 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0:
invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_ NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

(quoted diff) +2 -0
+	 * page selective (address granularity)
+	 */
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 4 lines
(quoted diff) +2 -0
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single

PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},  

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

This is for translation between generic UAPI granu to VT-d granu, it has nothing to do with address and size.
e.g.
If user passes IOMMU_INV_GRANU_PASID for the single PASID case as you mentioned, this map table shows it is valid.

Then the lookup result will get VT-d granu:
QI_DEV_IOTLB_GRAN_PASID_SEL, which means G=0.

Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */  

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

True for vIOMMU, this is here for completeness. Can be used by virtio IOMMU, since PC flush is inclusive (IOTLB, devTLB), it is more efficient.

Quoted 19 lines
Quoted 19 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU _NR] = {

drivers/iommu/intel-iommu.c +9 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int

*vtd_granu) +{

(quoted diff) +1 -0
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c +1 -0
+		!inv_type_granu_map[type][granu])
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+  

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Yes, for value = 1. e.g. G=0 but still valid.

Quoted 44 lines
Quoted 44 lines
drivers/iommu/intel-iommu.c +6 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >>

VTD_PAGE_SHIFT; +

(quoted diff) +1 -0
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9

for 2MB, etc.

(quoted diff) +1 -0
+	 * IOMMU cache invalidate API passes granu_size in bytes,

and number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct

iommu_cache_invalidate_info *inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +10 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 6 lines
(quoted diff) +2 -0
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus,

devfn);

drivers/iommu/intel-iommu.c +3 -0
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;  

-ENOTSUPP?

I guess it can go either way in that the error is based on invalid inputs.

Quoted 42 lines
Quoted 42 lines
drivers/iommu/intel-iommu.c +5 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation

*/

(quoted diff) +2 -0
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size =

to_vtd_size(inv_info->addr_info.granule_size,
+
inv_info->addr_info.nb_granules); +

(quoted diff) +1 -0
+	for_each_set_bit(cache_type, (unsigned long

*)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type,

inv_info->granularity, &granu);

(quoted diff) +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on

granularity */

(quoted diff) +1 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID

&&

(quoted diff) +1 -0
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

(quoted diff) +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity ==

IOMMU_INV_GRANU_ADDR &&

(quoted diff) +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache

type and granularity\n");

(quoted diff) +2 -0
+			break;
+		}
Quoted 9 lines
Quoted 5 lines
drivers/iommu/intel-iommu.c +4 -0
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Good point! will fix.

Quoted 14 lines
Quoted 14 lines
(quoted diff) +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

(quoted diff) +1 -0
+				pr_err("Address out of range,

0x%llx, size order %llu\n",

(quoted diff) +1 -0
+					inv_info->addr_info.addr,

size);

drivers/iommu/intel-iommu.c +7 -0
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-  
Quoted 1 line

addr_info.addr),

Quoted 16 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 12 lines
(quoted diff) +1 -0
+					(granu ==

QI_GRAN_NONG_PASID) ? - 1 : 1 << size,

(quoted diff) +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is

enabled since guest

(quoted diff) +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB

flush will be passed

drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */  

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Not explicitly. Just following the guideline in CH6.1 for efficient virtualization. Early on, we also had discussion on supporting virtio where IOTLB flush is inclusive.
Let me rephrase the comment:
/*
* Always flush device IOTLB if ATS is enabled. vIOMMU
* in the guest may assume IOTLB flush is inclusive,
* which is more efficient.
*/

Quoted 16 lines
Quoted 16 lines
(quoted diff) +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						pasid,

info->ats_qdep,
+
inv_info->addr_info.addr, size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

Quoted 9 lines

Review

Quoted 5 lines

+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

(quoted diff) +1 -0
+						granu);  

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

granu is passed into the flush function, which can be 1 or 0.

Quoted 8 lines
Quoted 8 lines
(quoted diff) +2 -0
+			} else
+				pr_warn("Passdown device IOTLB

flush w/o ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu,

inv_info-

Quoted 6 lines
Quoted 2 lines
Quoted 1 line

pasid_info.pasid);

+

as earlier comment, we shouldn't allow userspace or guest to invalidate PASID cache

same explanation :)

Quoted 21 lines
Quoted 21 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU

invalidation type %d\n",

drivers/iommu/intel-iommu.c +12 -0
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	=
Quoted 9 lines
Quoted 8 lines

intel_iommu_is_attach_deferred, .pgsize_bitmap =
INTEL_IOMMU_PGSIZES, #ifdef CONFIG_INTEL_IOMMU_SVM

drivers/iommu/intel-iommu.c +1 -1
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif
--

2.7.4

[Jacob Pan]

#52 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 17 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 2:14 AM

Quoted 11 lines · Kevin
Quoted 10 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

Quoted 6 lines
Quoted 3 lines

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

True, but it does not invalidate this statement about emulated IOMMU. I will add another statement saying "the same interface can be used for virtio-IOMMU as well". OK?

sure

Quoted 83 lines
Quoted 43 lines
Quoted 43 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

drivers/iommu/intel-iommu.c +2 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore

Quoted 9 lines
Quoted 2 lines

+ * we do not include IOTLB granularity for request without PASID (second level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Sounds good.

Quoted 5 lines
Quoted 5 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity

Quoted 8 lines
Quoted 8 lines

+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0:
invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 6 lines
Quoted 6 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

(quoted diff) +2 -0
+	 * page selective (address granularity)
+	 */
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 4 lines
(quoted diff) +2 -0
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single

PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 2 lines

This is for translation between generic UAPI granu to VT-d granu, it has nothing to do with address and size.

Generic UAPI defines three granularities: domain, pasid and addr. from the definition domain applies all entries related to did, pasid applies to all entries related to pasid, while addr is specific for a range.

from what we just confirmed internally with VT-d spec owner, our PASID based dev TLB invalidation always requires addr and size, while current uAPI doesn't support multiple PASIDs based range invaliation. It sounds to me that you want to use domain to replace multiple PASIDs case (G=1), but it then changes the meaning of the domain granularity and easily lead to confusion.

I feel Eric's proposal makes more sense. Here we'd better use {0, 0, 1} to indicate only addr range invalidation is allowed, matching the spec definition. We may use a special flag in iommu_inv_addr_info to indicate G=1 case, if necessary.

Quoted 16 lines

e.g.
If user passes IOMMU_INV_GRANU_PASID for the single PASID case as you mentioned, this map table shows it is valid.

Then the lookup result will get VT-d granu:
QI_DEV_IOTLB_GRAN_PASID_SEL, which means G=0.

Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

Quoted 3 lines

True for vIOMMU, this is here for completeness. Can be used by virtio IOMMU, since PC flush is inclusive (IOTLB, devTLB), it is more efficient.

I think it is not correct in concept. We should not allow the userspace or guest to request an operation which is beyond its privilege (just because doing so may bring some performance benefit). You can always introduce new cmd for such purpose.

Quoted 171 lines
Quoted 6 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain,

Quoted 6 lines
drivers/iommu/intel-iommu.c +4 -0
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU

Quoted 14 lines
Quoted 14 lines

_NR] = {

drivers/iommu/intel-iommu.c +9 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int

*vtd_granu) +{

(quoted diff) +1 -0
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c +1 -0
+		!inv_type_granu_map[type][granu])
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5669

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+

btw do we really need both map and table here? Can't we just use one table with unsupported granularity marked as a special value?

Yes, for value = 1. e.g. G=0 but still valid.

Quoted 44 lines
Quoted 44 lines
drivers/iommu/intel-iommu.c +6 -0
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >>

VTD_PAGE_SHIFT; +

(quoted diff) +1 -0
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9

for 2MB, etc.

(quoted diff) +1 -0
+	 * IOMMU cache invalidate API passes granu_size in bytes,

and number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct

iommu_cache_invalidate_info *inv_info)

drivers/iommu/intel-iommu.c +13 -0
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +10 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5714

Quoted 6 lines
(quoted diff) +2 -0
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus,

devfn);

drivers/iommu/intel-iommu.c +3 -0
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;

-ENOTSUPP?

I guess it can go either way in that the error is based on invalid inputs.

Quoted 42 lines
Quoted 42 lines
drivers/iommu/intel-iommu.c +5 -0
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation

*/

(quoted diff) +2 -0
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size =

to_vtd_size(inv_info->addr_info.granule_size,
+
inv_info->addr_info.nb_granules); +

(quoted diff) +1 -0
+	for_each_set_bit(cache_type, (unsigned long

*)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type,

inv_info->granularity, &granu);

(quoted diff) +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on

granularity */

(quoted diff) +1 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID

&&

(quoted diff) +1 -0
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

(quoted diff) +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity ==

IOMMU_INV_GRANU_ADDR &&

(quoted diff) +1 -0
+			inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache

type and granularity\n");

(quoted diff) +2 -0
+			break;
+		}
Quoted 9 lines
Quoted 5 lines
drivers/iommu/intel-iommu.c +4 -0
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=

IOMMU_INV_GRANU_PASID) &&

granularity == IOMMU_INV_GRANU_ADDR? otherwise it's unclear why IOMMU_INV_GRANU_DOMAIN also needs size check.

Good point! will fix.

Quoted 14 lines
Quoted 14 lines
(quoted diff) +1 -0
+				size && (inv_info->addr_info.addr &

((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

(quoted diff) +1 -0
+				pr_err("Address out of range,

0x%llx, size order %llu\n",

(quoted diff) +1 -0
+					inv_info->addr_info.addr,

size);

drivers/iommu/intel-iommu.c +7 -0
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+					mm_to_dma_pfn(inv_info-
Quoted 1 line

addr_info.addr),

Quoted 16 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5767

Quoted 12 lines
(quoted diff) +1 -0
+					(granu ==

QI_GRAN_NONG_PASID) ? - 1 : 1 << size,

(quoted diff) +1 -0
+					inv_info->addr_info.flags &

IOMMU_INV_ADDR_FLAGS_LEAF);

drivers/iommu/intel-iommu.c +3 -0
+
+			/*
+			 * Always flush device IOTLB if ATS is

enabled since guest

(quoted diff) +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB

flush will be passed

drivers/iommu/intel-iommu.c +2 -0
+			 * down.
+			 */

Does VT-d spec mention that no device IOTLB flush is required when CM=1?

Quoted 9 lines

Not explicitly. Just following the guideline in CH6.1 for efficient virtualization. Early on, we also had discussion on supporting virtio where IOTLB flush is inclusive.
Let me rephrase the comment:
/*
* Always flush device IOTLB if ATS is enabled. vIOMMU
* in the guest may assume IOTLB flush is inclusive,
* which is more efficient.
*/

this looks better.

Quoted 77 lines
Quoted 18 lines
Quoted 18 lines
(quoted diff) +2 -0
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						pasid,

info->ats_qdep,
+
inv_info->addr_info.addr, size,

drivers/iommu/intel-iommu.c +6 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info-

Quoted 1 line

pfsid,

Quoted 9 lines

Review

Quoted 5 lines

+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

(quoted diff) +1 -0
+						granu);

I'm confused here. There are two granularities allowed for devtlb, but here you only handle one of them?

granu is passed into the flush function, which can be 1 or 0.

Quoted 8 lines
Quoted 8 lines
(quoted diff) +2 -0
+			} else
+				pr_warn("Passdown device IOTLB

flush w/o ATS!\n");

drivers/iommu/intel-iommu.c +4 -0
+
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu,

inv_info-

Quoted 6 lines

4 lines not shown · raw message

32 lines not shown · raw message

Part of this message is not shown here. View the raw message.

#53 lore raw

RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 129 lines

From: Tian, Kevin <[email protected]>
Sent: Wednesday, April 1, 2020 2:24 PM
To: Jacob Pan <[email protected]>
Subject: RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 17 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 2:14 AM

Quoted 11 lines · Kevin
Quoted 10 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

Quoted 6 lines
Quoted 3 lines

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

True, but it does not invalidate this statement about emulated IOMMU. I will add another statement saying "the same interface can be used for virtio-IOMMU as well". OK?

sure

Quoted 83 lines
Quoted 43 lines
Quoted 43 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

drivers/iommu/intel-iommu.c +2 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore

Quoted 9 lines
Quoted 2 lines

+ * we do not include IOTLB granularity for request without PASID (second level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Sounds good.

Quoted 5 lines
Quoted 5 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity

Quoted 8 lines
Quoted 8 lines

+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0:
invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 6 lines
Quoted 6 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

(quoted diff) +2 -0
+	 * page selective (address granularity)
+	 */
Quoted 9 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5648

Quoted 4 lines
(quoted diff) +2 -0
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single

PASID */

drivers/iommu/intel-iommu.c +1 -0
+	{1, 1, 0},

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 2 lines

This is for translation between generic UAPI granu to VT-d granu, it has nothing to do with address and size.

Generic UAPI defines three granularities: domain, pasid and addr. from the definition domain applies all entries related to did, pasid applies to all entries related to pasid, while addr is specific for a range.

from what we just confirmed internally with VT-d spec owner, our PASID based dev TLB invalidation always requires addr and size, while current uAPI doesn't support multiple PASIDs based range invaliation. It sounds to me that you want to use domain to replace multiple PASIDs case (G=1), but it then changes the meaning of the domain granularity and easily lead to confusion.

Quoted 4 lines

I feel Eric's proposal makes more sense. Here we'd better use {0, 0, 1} to indicate only addr range invalidation is allowed, matching the spec definition. We may use a special flag in iommu_inv_addr_info to indicate G=1 case, if necessary.

I agree. G=1 case should be supported. I think we had a flag for global as there is GL bit in p_iotlb_inv_dsc (a.k.a ext_iotlb_inv_dsc), but it was dropped as 3.0 spec dropped GL bit. Let's add it back as for DevTLB flush case.

Quoted 16 lines
Quoted 16 lines

e.g.
If user passes IOMMU_INV_GRANU_PASID for the single PASID case as you mentioned, this map table shows it is valid.

Then the lookup result will get VT-d granu:
QI_DEV_IOTLB_GRAN_PASID_SEL, which means G=0.

Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

Quoted 8 lines
Quoted 3 lines

True for vIOMMU, this is here for completeness. Can be used by virtio IOMMU, since PC flush is inclusive (IOTLB, devTLB), it is more efficient.

I think it is not correct in concept. We should not allow the userspace or guest to request an operation which is beyond its privilege (just because doing so may bring some performance benefit). You can always introduce new cmd for such purpose.

I guess it was added for the pasid table binding case? Now, our platform doesn't support it. So I guess we can just make it as unsupported in the 2D table.

Regards,
Yi Liu

author
#54 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 137 lines · Yi L
Quoted 132 lines

From: Tian, Kevin <[email protected]>
Sent: Wednesday, April 1, 2020 2:24 PM
To: Jacob Pan <[email protected]>
Subject: RE: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 19 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 2:14 AM

Quoted 13 lines · Kevin
Quoted 12 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue

Quoted 6 lines
Quoted 3 lines

emulated IOMMU -> vIOMMU, since virito-iommu could use the interface as well.

True, but it does not invalidate this statement about emulated IOMMU. I will add another statement saying "the same interface can be used for virtio-IOMMU as well". OK?

sure

Quoted 84 lines
Quoted 43 lines
Quoted 43 lines

invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182
++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +2 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore

Quoted 9 lines
Quoted 2 lines

+ * we do not include IOTLB granularity for request without PASID (second level).

I would revise above as "We do not support IOTLB granularity for request without PASID (second level), therefore any vIOMMU implementation that exposes the SVA capability to the guest should only expose the first level page tables, implying all invalidation requests from the guest will include a valid PASID"

Sounds good.

Quoted 6 lines
Quoted 6 lines

+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity

Quoted 8 lines
Quoted 8 lines

+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1:
valid, 0: invalid
+ *
+ */
+const static int

inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_

Quoted 6 lines
Quoted 6 lines

NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective

(per PASID),

(quoted diff) +2 -0
+	 * page selective (address granularity)
+	 */
Quoted 9 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or

single PASID */

(quoted diff) +1 -0
+	{1, 1, 0},  

Is this combination correct? when single PASID is being specified, it is essentially a page-selective invalidation since you need provide Address and Size.

Quoted 2 lines

This is for translation between generic UAPI granu to VT-d granu, it has nothing to do with address and size.

Generic UAPI defines three granularities: domain, pasid and addr. from the definition domain applies all entries related to did, pasid applies to all entries related to pasid, while addr is specific for a range.

from what we just confirmed internally with VT-d spec owner, our PASID based dev TLB invalidation always requires addr and size, while current uAPI doesn't support multiple PASIDs based range invaliation. It sounds to me that you want to use domain to replace multiple PASIDs case (G=1), but it then changes the meaning of the domain granularity and easily lead to confusion.

Quoted 4 lines

I feel Eric's proposal makes more sense. Here we'd better use {0, 0, 1} to indicate only addr range invalidation is allowed, matching the spec definition. We may use a special flag in iommu_inv_addr_info to indicate G=1 case, if necessary.

Quoted 5 lines

I agree. G=1 case should be supported. I think we had a flag for global as there is GL bit in p_iotlb_inv_dsc (a.k.a ext_iotlb_inv_dsc), but it was dropped as 3.0 spec dropped GL bit. Let's add it back as for DevTLB flush case.

Make sense. I will change that to

drivers/iommu/intel-iommu.c +2 -2
@@ -5741,7 +5741,7 @@ const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] 
         */                                                                                            
        {0, 1, 1},                                                                                     
        /* PASID based dev TLBs, only support all PASIDs or single PASID */                            
-       {1, 1, 0},                                                                                     
+       {0, 0, 1},                                                                                     
        /* PASID cache */                                                                              
        {1, 1, 0}                                                                                      
 };                                                                                                    
@@ -5750,7 +5750,7 @@ const static int inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_N 
        /* PASID based IOTLB */                                                                        
        {0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},                                                    
        /* PASID based dev TLBs */                                                                     
-       {QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},                                       
+       {0, 0, QI_DEV_IOTLB_GRAN_PASID_SEL},                                                           
        /* PASID cache */                                                                              
        {QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},                                                        
 };                                                                                                    
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

e.g.
If user passes IOMMU_INV_GRANU_PASID for the single PASID case as you mentioned, this map table shows it is valid.

Then the lookup result will get VT-d granu:
QI_DEV_IOTLB_GRAN_PASID_SEL, which means G=0.

Quoted 7 lines

Review

Quoted 1 line
(quoted diff) +1 -0
+	/* PASID cache */  

PASID cache is fully managed by the host. Guest PASID cache invalidation is interpreted by vIOMMU for bind and unbind operations. I don't think we should accept any PASID cache invalidation from userspace or guest.

True for vIOMMU, this is here for completeness. Can be used by virtio IOMMU, since PC flush is inclusive (IOTLB, devTLB), it is more efficient.

Quoted 8 lines
Quoted 4 lines

I think it is not correct in concept. We should not allow the userspace or guest to request an operation which is beyond its privilege (just because doing so may bring some performance benefit). You can always introduce new cmd for such purpose.

I guess it was added for the pasid table binding case? Now, our platform doesn't support it. So I guess we can just make it as unsupported in the 2D table.

Sounds good.

#55 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

In reply to Jacob Pan

Hi Jacob,

Quoted 146 lines · Jacob Pan

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 182 insertions(+)
drivers/iommu/intel-iommu.c +112 -0
index b1477cd423dd..a76afb0fd51a 100644
@@ -5619,6 +5619,187 @@ static void intel_iommu_aux_detach_device(struct iommu_domain *domain,
 	aux_domain_remove_dev(to_dmar_domain(domain), dev);
 }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second level).
+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0: invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per PASID),
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single PASID */
+	{1, 1, 0},
+	/* PASID cache */
+	{1, 1, 0}
+};
+
+const static int inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int *vtd_granu)
+{
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >= IOMMU_INV_GRANU_NR ||
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >> VTD_PAGE_SHIFT;
+
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc.
+	 * IOMMU cache invalidate API passes granu_size in bytes, and number of
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct iommu_cache_invalidate_info *inv_info)
+{
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version != IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus, devfn);
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation */
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size = to_vtd_size(inv_info->addr_info.granule_size,
+				   inv_info->addr_info.nb_granules);
+
+	for_each_set_bit(cache_type, (unsigned long *)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type, inv_info->granularity, &granu);
+		if (ret) {
+			pr_err("Invalid cache type and granu combination %d/%d\n", cache_type,
+				inv_info->granularity);
+			break;
+		}

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5743

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+
+		/* PASID is stored in different locations based on granularity */
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID &&
+			inv_info->pasid_info.flags & IOMMU_INV_PASID_FLAGS_PASID)
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR &&
+			inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_PASID)
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache type and granularity\n");

I don't get this error msg. In case of domain-selective invalidation, PASID is not used so if I am not wrong you will end up here while there is no issue.

Quoted 15 lines
drivers/iommu/intel-iommu.c +15 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity != IOMMU_INV_GRANU_PASID) &&
+				size && (inv_info->addr_info.addr & ((BIT(VTD_PAGE_SHIFT + size)) - 1))) {
+				pr_err("Address out of range, 0x%llx, size order %llu\n",
+					inv_info->addr_info.addr, size);
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5768

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+					mm_to_dma_pfn(inv_info->addr_info.addr),
+					(granu == QI_GRAN_NONG_PASID) ? -1 : 1 << size,
+					inv_info->addr_info.flags & IOMMU_INV_ADDR_FLAGS_LEAF);
+
+			/*
+			 * Always flush device IOTLB if ATS is enabled since guest
+			 * vIOMMU exposes CM = 1, no device IOTLB flush will be passed
+			 * down.
+			 */
+			if (info->ats_enabled) {

nit {} not requested

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5776

Quoted 8 lines
drivers/iommu/intel-iommu.c +8 -0
+				qi_flush_dev_iotlb_pasid(iommu, sid, info->pfsid,
+						pasid, info->ats_qdep,
+						inv_info->addr_info.addr, size,
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {

nit {} not requested

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5783

Quoted 7 lines
drivers/iommu/intel-iommu.c +7 -0
+				qi_flush_dev_iotlb_pasid(iommu, sid, info->pfsid,
+						inv_info->addr_info.pasid, info->ats_qdep,
+						inv_info->addr_info.addr, size,
+						granu);
+			} else
+				pr_warn("Passdown device IOTLB flush w/o ATS!\n");
+

nit: extra line

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5787

Quoted 4 lines
drivers/iommu/intel-iommu.c +4 -0
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu, inv_info->pasid_info.pasid);
+

nit: extra line

Quoted 20 lines
drivers/iommu/intel-iommu.c +15 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU invalidation type %d\n",
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)

drivers/iommu/intel-iommu.c const struct iommu_ops intel_iommu_ops = { · line 6391

Quoted 7 lines
drivers/iommu/intel-iommu.c +1 -0
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	= intel_iommu_is_attach_deferred,
 	.pgsize_bitmap		= INTEL_IOMMU_PGSIZES,
 #ifdef CONFIG_INTEL_IOMMU_SVM
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif

Thanks

Eric

author
#56 lore raw

Re: [PATCH V10 08/11] iommu/vt-d: Add svm/sva invalidate function

Quoted 183 lines · On Sun, 29 Mar 2020 18:05:47 +0200 Auger Eric <[email protected]> wrote:

Hi Jacob,

Quoted 180 lines · Jacob Pan

When Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.

The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue invalidation capability and passes down all descriptors from the guest to the physical IOMMU.

The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.

---
v7 review fixed in v10
---

---
 drivers/iommu/intel-iommu.c | 182

++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 182 insertions(+)

(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index b1477cd423dd..a76afb0fd51a 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +2 -0
@@ -5619,6 +5619,187 @@ static void
 intel_iommu_aux_detach_device(struct iommu_domain *domain,
 aux_domain_remove_dev(to_dmar_domain(domain), dev); }
 
+/*
+ * 2D array for converting and sanitizing IOMMU generic TLB

granularity to
+ * VT-d granularity. Invalidation is typically included in the unmap operation
+ * as a result of DMA or VFIO unmap. However, for assigned devices guest
+ * owns the first level page tables. Invalidations of translation caches in the
+ * guest are trapped and passed down to the host.
+ *
+ * vIOMMU in the guest will only expose first level page tables, therefore
+ * we do not include IOTLB granularity for request without PASID (second level).
+ *
+ * For example, to find the VT-d granularity encoding for IOTLB
+ * type and page selective granularity within PASID:
+ * X: indexed by iommu cache type
+ * Y: indexed by enum iommu_inv_granularity
+ * [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR]
+ *
+ * Granu_map array indicates validity of the table. 1: valid, 0:
invalid
+ *
+ */
+const static int inv_type_granu_map[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {

(quoted diff) +2 -0
+	/*
+	 * PASID based IOTLB invalidation: PASID selective (per

PASID),

drivers/iommu/intel-iommu.c +4 -0
+	 * page selective (address granularity)
+	 */
+	{0, 1, 1},
+	/* PASID based dev TLBs, only support all PASIDs or single

PASID */

drivers/iommu/intel-iommu.c +6 -0
+	{1, 1, 0},
+	/* PASID cache */
+	{1, 1, 0}
+};
+
+const static int

inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR][IOMMU_INV_GRANU_NR] = {

drivers/iommu/intel-iommu.c +9 -0
+	/* PASID based IOTLB */
+	{0, QI_GRAN_NONG_PASID, QI_GRAN_PSI_PASID},
+	/* PASID based dev TLBs */
+	{QI_DEV_IOTLB_GRAN_ALL, QI_DEV_IOTLB_GRAN_PASID_SEL, 0},
+	/* PASID cache */
+	{QI_PC_ALL_PASIDS, QI_PC_PASID_SEL, 0},
+};
+
+static inline int to_vtd_granularity(int type, int granu, int

*vtd_granu) +{

(quoted diff) +1 -0
+	if (type >= IOMMU_CACHE_INV_TYPE_NR || granu >=

IOMMU_INV_GRANU_NR ||

drivers/iommu/intel-iommu.c +11 -0
+		!inv_type_granu_map[type][granu])
+		return -EINVAL;
+
+	*vtd_granu = inv_type_granu_table[type][granu];
+
+	return 0;
+}
+
+static inline u64 to_vtd_size(u64 granu_size, u64 nr_granules)
+{
+	u64 nr_pages = (granu_size * nr_granules) >>

VTD_PAGE_SHIFT; +

(quoted diff) +1 -0
+	/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9

for 2MB, etc.

(quoted diff) +1 -0
+	 * IOMMU cache invalidate API passes granu_size in bytes,

and number of

drivers/iommu/intel-iommu.c +8 -0
+	 * granu size in contiguous memory.
+	 */
+	return order_base_2(nr_pages);
+}
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static int intel_iommu_sva_invalidate(struct iommu_domain *domain,
+		struct device *dev, struct

iommu_cache_invalidate_info *inv_info) +{

drivers/iommu/intel-iommu.c +12 -0
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct device_domain_info *info;
+	struct intel_iommu *iommu;
+	unsigned long flags;
+	int cache_type;
+	u8 bus, devfn;
+	u16 did, sid;
+	int ret = 0;
+	u64 size = 0;
+
+	if (!inv_info || !dmar_domain ||
+		inv_info->version !=

IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)

drivers/iommu/intel-iommu.c +12 -0
+		return -EINVAL;
+
+	if (!dev || !dev_is_pci(dev))
+		return -ENODEV;
+
+	iommu = device_to_iommu(dev, &bus, &devfn);
+	if (!iommu)
+		return -ENODEV;
+
+	spin_lock_irqsave(&device_domain_lock, flags);
+	spin_lock(&iommu->lock);
+	info = iommu_support_dev_iotlb(dmar_domain, iommu, bus,

devfn);

drivers/iommu/intel-iommu.c +8 -0
+	if (!info) {
+		ret = -EINVAL;
+		goto out_unlock;
+	}
+	did = dmar_domain->iommu_did[iommu->seq_id];
+	sid = PCI_DEVID(bus, devfn);
+
+	/* Size is only valid in non-PASID selective invalidation

*/

(quoted diff) +2 -0
+	if (inv_info->granularity != IOMMU_INV_GRANU_PASID)
+		size =

to_vtd_size(inv_info->addr_info.granule_size,
+
inv_info->addr_info.nb_granules); +

(quoted diff) +1 -0
+	for_each_set_bit(cache_type, (unsigned long

*)&inv_info->cache, IOMMU_CACHE_INV_TYPE_NR) {

drivers/iommu/intel-iommu.c +4 -0
+		int granu = 0;
+		u64 pasid = 0;
+
+		ret = to_vtd_granularity(cache_type,

inv_info->granularity, &granu);

(quoted diff) +2 -0
+		if (ret) {
+			pr_err("Invalid cache type and granu

combination %d/%d\n", cache_type,

drivers/iommu/intel-iommu.c +5 -0
+				inv_info->granularity);
+			break;
+		}
+
+		/* PASID is stored in different locations based on

granularity */

(quoted diff) +1 -0
+		if (inv_info->granularity == IOMMU_INV_GRANU_PASID

&&

(quoted diff) +1 -0
+			inv_info->pasid_info.flags &

IOMMU_INV_PASID_FLAGS_PASID)

(quoted diff) +2 -0
+			pasid = inv_info->pasid_info.pasid;
+		else if (inv_info->granularity ==

IOMMU_INV_GRANU_ADDR &&

(quoted diff) +1 -0
+			inv_info->addr_info.flags &
Quoted 8 lines
Quoted 5 lines

IOMMU_INV_ADDR_FLAGS_PASID)

drivers/iommu/intel-iommu.c +3 -0
+			pasid = inv_info->addr_info.pasid;
+		else {
+			pr_err("Cannot find PASID for given cache

type and granularity\n");

I don't get this error msg. In case of domain-selective invalidation, PASID is not used so if I am not wrong you will end up here while there is no issue.

Right, I will remove the else.

Quoted 26 lines
Quoted 26 lines
drivers/iommu/intel-iommu.c +6 -0
+			break;
+		}
+
+		switch (BIT(cache_type)) {
+		case IOMMU_CACHE_INV_TYPE_IOTLB:
+			if ((inv_info->granularity !=

IOMMU_INV_GRANU_PASID) &&

(quoted diff) +1 -0
+				size && (inv_info->addr_info.addr

& ((BIT(VTD_PAGE_SHIFT + size)) - 1))) {

(quoted diff) +1 -0
+				pr_err("Address out of range,

0x%llx, size order %llu\n",

(quoted diff) +1 -0
+					inv_info->addr_info.addr,

size);

drivers/iommu/intel-iommu.c +7 -0
+				ret = -ERANGE;
+				goto out_unlock;
+			}
+
+			qi_flush_piotlb(iommu, did,
+					pasid,
+

mm_to_dma_pfn(inv_info->addr_info.addr),

(quoted diff) +1 -0
+					(granu ==

QI_GRAN_NONG_PASID) ? -1 : 1 << size,

(quoted diff) +1 -0
+					inv_info->addr_info.flags

& IOMMU_INV_ADDR_FLAGS_LEAF); +

(quoted diff) +1 -0
+			/*
Quoted 8 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5768

Quoted 7 lines
(quoted diff) +1 -0
+			 * Always flush device IOTLB if ATS is

enabled since guest

(quoted diff) +1 -0
+			 * vIOMMU exposes CM = 1, no device IOTLB

flush will be passed

drivers/iommu/intel-iommu.c +3 -0
+			 * down.
+			 */
+			if (info->ats_enabled) {  

nit {} not requested

Will fix. same for the remaining.

Thanks!

Jacob

Quoted 60 lines

drivers/iommu/intel-iommu.c static void intel_iommu_aux_detach_device(struct iommu_domain *domain, · line 5776

Quoted 11 lines
(quoted diff) +1 -0
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info->pfsid,

drivers/iommu/intel-iommu.c +1 -0
+						pasid,

info->ats_qdep,
+
inv_info->addr_info.addr, size,

drivers/iommu/intel-iommu.c +5 -0
+						granu);
+			}
+			break;
+		case IOMMU_CACHE_INV_TYPE_DEV_IOTLB:
+			if (info->ats_enabled) {  

nit {} not requested

Quoted 11 lines
(quoted diff) +1 -0
+				qi_flush_dev_iotlb_pasid(iommu,

sid, info->pfsid,
+
inv_info->addr_info.pasid, info->ats_qdep,
+
inv_info->addr_info.addr, size,

drivers/iommu/intel-iommu.c +3 -0
+						granu);
+			} else
+				pr_warn("Passdown device IOTLB

flush w/o ATS!\n");
+

nit: extra line

Quoted 5 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		case IOMMU_CACHE_INV_TYPE_PASID:
+			qi_flush_pasid_cache(iommu, did, granu,

inv_info->pasid_info.pasid);
+

nit: extra line

Quoted 24 lines
drivers/iommu/intel-iommu.c +3 -0
+			break;
+		default:
+			dev_err(dev, "Unsupported IOMMU

invalidation type %d\n",

drivers/iommu/intel-iommu.c +12 -0
+				cache_type);
+			ret = -EINVAL;
+		}
+	}
+out_unlock:
+	spin_unlock(&iommu->lock);
+	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	return ret;
+}
+#endif
+
 static int intel_iommu_map(struct iommu_domain *domain,
 			   unsigned long iova, phys_addr_t hpa,
 			   size_t size, int iommu_prot, gfp_t gfp)
@@ -6204,6 +6385,7 @@ const struct iommu_ops intel_iommu_ops = {
 	.is_attach_deferred	=

intel_iommu_is_attach_deferred, .pgsize_bitmap =
INTEL_IOMMU_PGSIZES, #ifdef CONFIG_INTEL_IOMMU_SVM

drivers/iommu/intel-iommu.c const struct iommu_ops intel_iommu_ops = { · line 6391

Quoted 4 lines
drivers/iommu/intel-iommu.c +1 -0
+	.cache_invalidate	= intel_iommu_sva_invalidate,
 	.sva_bind_gpasid	= intel_svm_bind_gpasid,
 	.sva_unbind_gpasid	= intel_svm_unbind_gpasid,
 #endif

Thanks

Quoted 2 lines

Eric

[Jacob Pan]

author
#57 lore raw

PATCH v10 9/11 iommu/vt-d: Cache virtual command capability register

In reply to Jacob Pan

Virtual command registers are used in the guest only, to prevent vmexit cost, we cache the capability and store it during initialization.

---
v7 Reviewed by Eric & Baolu
---

---
 drivers/iommu/dmar.c        | 1 +
 include/linux/intel-iommu.h | 5 +++++
 2 files changed, 6 insertions(+)
drivers/iommu/dmar.c +1 -0
index 4d6b7b5b37ee..3b36491c8bbb 100644
@@ -963,6 +963,7 @@ static int map_iommu(struct intel_iommu *iommu, u64 phys_addr)
 		warn_invalid_dmar(phys_addr, " returns all ones");
 		goto unmap;
 	}
+	iommu->vccap = dmar_readq(iommu->reg + DMAR_VCCAP_REG);
 
 	/* the registers might be more than one page */
 	map_size = max_t(int, ecap_max_iotlb_offset(iommu->ecap),
include/linux/intel-iommu.h +5 -0
index 43539713b3b3..ccbf164fb711 100644
@@ -194,6 +194,9 @@
 #define ecap_max_handle_mask(e) ((e >> 20) & 0xf)
 #define ecap_sc_support(e)	((e >> 7) & 0x1) /* Snooping Control */
 
+/* Virtual command interface capabilities */
+#define vccap_pasid(v)		((v & DMA_VCS_PAS)) /* PASID allocation */
+
 /* IOTLB_REG */
 #define DMA_TLB_FLUSH_GRANU_OFFSET  60
 #define DMA_TLB_GLOBAL_FLUSH (((u64)1) << 60)
@@ -287,6 +290,7 @@
 
 /* PRS_REG */
 #define DMA_PRS_PPR	((u32)1)
+#define DMA_VCS_PAS	((u64)1)
 
 #define IOMMU_WAIT_OP(iommu, offset, op, cond, sts)			\
 do {									\
@@ -537,6 +541,7 @@ struct intel_iommu {
 	u64		reg_size; /* size of hw register set */
 	u64		cap;
 	u64		ecap;
+	u64		vccap;
 	u32		gcmd; /* Holds TE, EAFL. Don't need SRTP, SFL, WBF */
 	raw_spinlock_t	register_lock; /* protect register handling */
 	int		seq_id;	/* sequence id of the iommu */
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Reviewed-by on v10 9/11
#58 lore raw

RE: [PATCH V10 09/11] iommu/vt-d: Cache virtual command capability register

Quoted 36 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Virtual command registers are used in the guest only, to prevent vmexit cost, we cache the capability and store it during initialization.

---
v7 Reviewed by Eric & Baolu
---

---
 drivers/iommu/dmar.c        | 1 +
 include/linux/intel-iommu.h | 5 +++++
 2 files changed, 6 insertions(+)
drivers/iommu/dmar.c +1 -0
index 4d6b7b5b37ee..3b36491c8bbb 100644
@@ -963,6 +963,7 @@ static int map_iommu(struct intel_iommu *iommu,
 u64 phys_addr)
 		warn_invalid_dmar(phys_addr, " returns all ones");
 		goto unmap;
 	}
+	iommu->vccap = dmar_readq(iommu->reg + DMAR_VCCAP_REG);
 
 	/* the registers might be more than one page */
drivers/iommu/dmar.c +0 -0
 	map_size = max_t(int, ecap_max_iotlb_offset(iommu->ecap),
include/linux/intel-iommu.h +0 -0
index 43539713b3b3..ccbf164fb711 100644

include/linux/intel-iommu.h line 197

Quoted 4 lines
include/linux/intel-iommu.h +1 -0
@@ -194,6 +194,9 @@
 #define ecap_max_handle_mask(e) ((e >> 20) & 0xf)
 #define ecap_sc_support(e)	((e >> 7) & 0x1) /* Snooping Control */
 
+/* Virtual command interface capabilities */

capabilities -> capability

Quoted 23 lines
include/linux/intel-iommu.h +1 -0
+#define vccap_pasid(v)		((v & DMA_VCS_PAS)) /* PASID

allocation */

include/linux/intel-iommu.h +2 -0
+
 /* IOTLB_REG */
 #define DMA_TLB_FLUSH_GRANU_OFFSET  60
 #define DMA_TLB_GLOBAL_FLUSH (((u64)1) << 60)
@@ -287,6 +290,7 @@
 
 /* PRS_REG */
 #define DMA_PRS_PPR	((u32)1)
+#define DMA_VCS_PAS	((u64)1)
 
 #define IOMMU_WAIT_OP(iommu, offset, op, cond, sts)
 	\

do { \

include/linux/intel-iommu.h +1 -0
@@ -537,6 +541,7 @@ struct intel_iommu {
 	u64		reg_size; /* size of hw register set */
 	u64		cap;
 	u64		ecap;
+	u64		vccap;
 	u32		gcmd; /* Holds TE, EAFL. Don't need SRTP, SFL, WBF
 */
 	raw_spinlock_t	register_lock; /* protect register handling */
Quoted 3 lines

int seq_id; /* sequence id of the iommu */
--
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#59 lore raw

Re: [PATCH V10 09/11] iommu/vt-d: Cache virtual command capability register

Quoted 41 lines · Kevin
Quoted 41 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

Virtual command registers are used in the guest only, to prevent vmexit cost, we cache the capability and store it during initialization.

---
v7 Reviewed by Eric & Baolu
---

---
 drivers/iommu/dmar.c        | 1 +
 include/linux/intel-iommu.h | 5 +++++
 2 files changed, 6 insertions(+)
drivers/iommu/dmar.c +1 -0
index 4d6b7b5b37ee..3b36491c8bbb 100644
@@ -963,6 +963,7 @@ static int map_iommu(struct intel_iommu *iommu,
 u64 phys_addr)
 		warn_invalid_dmar(phys_addr, " returns all ones");
 		goto unmap;
 	}
+	iommu->vccap = dmar_readq(iommu->reg + DMAR_VCCAP_REG);
 
 	/* the registers might be more than one page */
drivers/iommu/dmar.c +0 -0
 	map_size = max_t(int, ecap_max_iotlb_offset(iommu->ecap),
(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 43539713b3b3..ccbf164fb711 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

(quoted diff) +0 -0
@@ -194,6 +194,9 @@
 #define ecap_max_handle_mask(e) ((e >> 20) & 0xf)
 #define ecap_sc_support(e)	((e >> 7) & 0x1) /* Snooping

Control */

Quoted 3 lines
Quoted 1 line

+/* Virtual command interface capabilities */

capabilities -> capability

Will do, I was thinking the future :)

Thanks,

Jacob

Quoted 15 lines
Quoted 12 lines

+#define vccap_pasid(v) ((v & DMA_VCS_PAS)) /* PASID
allocation */

include/linux/intel-iommu.h +2 -0
+
 /* IOTLB_REG */
 #define DMA_TLB_FLUSH_GRANU_OFFSET  60
 #define DMA_TLB_GLOBAL_FLUSH (((u64)1) << 60)
@@ -287,6 +290,7 @@
 
 /* PRS_REG */
 #define DMA_PRS_PPR	((u32)1)
+#define DMA_VCS_PAS	((u64)1)
 
Quoted 15 lines
Quoted 9 lines
(quoted diff) +0 -0
 #define IOMMU_WAIT_OP(iommu, offset, op, cond, sts)
 	\

do
{
\ @@ -537,6 +541,7 @@ struct intel_iommu { u64
reg_size; /* size of hw register set */ u64 cap;

include/linux/intel-iommu.h +1 -0
 	u64		ecap;
+	u64		vccap;
 	u32		gcmd; /* Holds TE, EAFL. Don't need
Quoted 5 lines

SRTP, SFL, WBF */
raw_spinlock_t register_lock; /* protect register
handling */ int seq_id; /* sequence id of the
iommu */ --
2.7.4

[Jacob Pan]

author
#60 lore raw

PATCH v10 10/11 iommu/vt-d: Enlightened PASID allocation

In reply to Jacob Pan

From: Lu Baolu <[email protected]>

Enabling IOMMU in a guest requires communication with the host driver for certain aspects. Use of PASID ID to enable Shared Virtual Addressing (SVA) requires managing PASID's in the host. VT-d 3.0 spec provides a Virtual Command Register (VCMD) to facilitate this. Writes to this register in the guest are trapped by QEMU which proxies the call to the host driver.

This virtual command interface consists of a capability register, a virtual command register, and a virtual response register. Refer to section 10.4.42, 10.4.43, 10.4.44 for more information.

This patch adds the enlightened PASID allocation/free interfaces via the virtual command interface.

---
 drivers/iommu/intel-pasid.c | 57 +++++++++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-pasid.h | 13 ++++++++++-
 include/linux/intel-iommu.h |  1 +
 3 files changed, 70 insertions(+), 1 deletion(-)
drivers/iommu/intel-pasid.c +57 -0
index 9f6d07410722..e87ad67aad36 100644
@@ -27,6 +27,63 @@
 static DEFINE_SPINLOCK(pasid_lock);
 u32 intel_pasid_max_id = PASID_MAX;
 
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid)
+{
+	unsigned long flags;
+	u8 status_code;
+	int ret = 0;
+	u64 res;
+
+	raw_spin_lock_irqsave(&iommu->register_lock, flags);
+	dmar_writeq(iommu->reg + DMAR_VCMD_REG, VCMD_CMD_ALLOC);
+	IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+		      !(res & VCMD_VRSP_IP), res);
+	raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+	status_code = VCMD_VRSP_SC(res);
+	switch (status_code) {
+	case VCMD_VRSP_SC_SUCCESS:
+		*pasid = VCMD_VRSP_RESULT_PASID(res);
+		break;
+	case VCMD_VRSP_SC_NO_PASID_AVAIL:
+		pr_info("IOMMU: %s: No PASID available\n", iommu->name);
+		ret = -ENOSPC;
+		break;
+	default:
+		ret = -ENODEV;
+		pr_warn("IOMMU: %s: Unexpected error code %d\n",
+			iommu->name, status_code);
+	}
+
+	return ret;
+}
+
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid)
+{
+	unsigned long flags;
+	u8 status_code;
+	u64 res;
+
+	raw_spin_lock_irqsave(&iommu->register_lock, flags);
+	dmar_writeq(iommu->reg + DMAR_VCMD_REG,
+		    VCMD_CMD_OPERAND(pasid) | VCMD_CMD_FREE);
+	IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+		      !(res & VCMD_VRSP_IP), res);
+	raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+	status_code = VCMD_VRSP_SC(res);
+	switch (status_code) {
+	case VCMD_VRSP_SC_SUCCESS:
+		break;
+	case VCMD_VRSP_SC_INVALID_PASID:
+		pr_info("IOMMU: %s: Invalid PASID\n", iommu->name);
+		break;
+	default:
+		pr_warn("IOMMU: %s: Unexpected error code %d\n",
+			iommu->name, status_code);
+	}
+}
+
 /*
  * Per device pasid table management:
  */
drivers/iommu/intel-pasid.h +12 -1
index 698015ee3f04..cd3d63f3e936 100644
@@ -23,6 +23,16 @@
 #define is_pasid_enabled(entry)		(((entry)->lo >> 3) & 0x1)
 #define get_pasid_dir_size(entry)	(1 << ((((entry)->lo >> 9) & 0x7) + 7))
 
+/* Virtual command interface for enlightened pasid management. */
+#define VCMD_CMD_ALLOC			0x1
+#define VCMD_CMD_FREE			0x2
+#define VCMD_VRSP_IP			0x1
+#define VCMD_VRSP_SC(e)			(((e) >> 1) & 0x3)
+#define VCMD_VRSP_SC_SUCCESS		0
+#define VCMD_VRSP_SC_NO_PASID_AVAIL	1
+#define VCMD_VRSP_SC_INVALID_PASID	1
+#define VCMD_VRSP_RESULT_PASID(e)	(((e) >> 8) & 0xfffff)
+#define VCMD_CMD_OPERAND(e)		((e) << 8)
 /*
  * Domain ID reserved for pasid entries programmed for first-level
  * only and pass-through transfer modes.
@@ -113,5 +123,6 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu,
 			int addr_width);
 void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 				 struct device *dev, int pasid);
-
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid);
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid);
 #endif /* __INTEL_PASID_H */
include/linux/intel-iommu.h +1 -0
index ccbf164fb711..9cbf5357138b 100644
@@ -169,6 +169,7 @@
 #define ecap_smpwc(e)		(((e) >> 48) & 0x1)
 #define ecap_flts(e)		(((e) >> 47) & 0x1)
 #define ecap_slts(e)		(((e) >> 46) & 0x1)
+#define ecap_vcs(e)		(((e) >> 44) & 0x1)
 #define ecap_smts(e)		(((e) >> 43) & 0x1)
 #define ecap_dit(e)		((e >> 41) & 0x1)
 #define ecap_pasid(e)		((e >> 40) & 0x1)
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Reviewed-by on v10 10/11
#61 lore raw

RE: [PATCH V10 10/11] iommu/vt-d: Enlightened PASID allocation

Quoted 11 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

From: Lu Baolu <[email protected]>

Enabling IOMMU in a guest requires communication with the host driver for certain aspects. Use of PASID ID to enable Shared Virtual Addressing (SVA) requires managing PASID's in the host. VT-d 3.0 spec provides a Virtual Command Register (VCMD) to facilitate this. Writes to this register in the guest are trapped by QEMU which proxies the call to the host driver.

Qemu -> vIOMMU

Quoted 134 lines

This virtual command interface consists of a capability register, a virtual command register, and a virtual response register. Refer to section 10.4.42, 10.4.43, 10.4.44 for more information.

This patch adds the enlightened PASID allocation/free interfaces via the virtual command interface.

---
 drivers/iommu/intel-pasid.c | 57
+++++++++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-pasid.h | 13 ++++++++++-
 include/linux/intel-iommu.h |  1 +
 3 files changed, 70 insertions(+), 1 deletion(-)
drivers/iommu/intel-pasid.c +57 -0
index 9f6d07410722..e87ad67aad36 100644
@@ -27,6 +27,63 @@
 static DEFINE_SPINLOCK(pasid_lock);
 u32 intel_pasid_max_id = PASID_MAX;
 
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid)
+{
+	unsigned long flags;
+	u8 status_code;
+	int ret = 0;
+	u64 res;
+
+	raw_spin_lock_irqsave(&iommu->register_lock, flags);
+	dmar_writeq(iommu->reg + DMAR_VCMD_REG,
 VCMD_CMD_ALLOC);
+	IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+		      !(res & VCMD_VRSP_IP), res);
+	raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+	status_code = VCMD_VRSP_SC(res);
+	switch (status_code) {
+	case VCMD_VRSP_SC_SUCCESS:
+		*pasid = VCMD_VRSP_RESULT_PASID(res);
+		break;
+	case VCMD_VRSP_SC_NO_PASID_AVAIL:
+		pr_info("IOMMU: %s: No PASID available\n", iommu->name);
+		ret = -ENOSPC;
+		break;
+	default:
+		ret = -ENODEV;
+		pr_warn("IOMMU: %s: Unexpected error code %d\n",
+			iommu->name, status_code);
+	}
+
+	return ret;
+}
+
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid)
+{
+	unsigned long flags;
+	u8 status_code;
+	u64 res;
+
+	raw_spin_lock_irqsave(&iommu->register_lock, flags);
+	dmar_writeq(iommu->reg + DMAR_VCMD_REG,
+		    VCMD_CMD_OPERAND(pasid) | VCMD_CMD_FREE);
+	IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+		      !(res & VCMD_VRSP_IP), res);
+	raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+	status_code = VCMD_VRSP_SC(res);
+	switch (status_code) {
+	case VCMD_VRSP_SC_SUCCESS:
+		break;
+	case VCMD_VRSP_SC_INVALID_PASID:
+		pr_info("IOMMU: %s: Invalid PASID\n", iommu->name);
+		break;
+	default:
+		pr_warn("IOMMU: %s: Unexpected error code %d\n",
+			iommu->name, status_code);
+	}
+}
+
 /*
  * Per device pasid table management:

*/

drivers/iommu/intel-pasid.h +12 -1
index 698015ee3f04..cd3d63f3e936 100644
@@ -23,6 +23,16 @@
 #define is_pasid_enabled(entry)		(((entry)->lo >> 3) & 0x1)
 #define get_pasid_dir_size(entry)	(1 << ((((entry)->lo >> 9) & 0x7) + 7))
 
+/* Virtual command interface for enlightened pasid management. */
+#define VCMD_CMD_ALLOC			0x1
+#define VCMD_CMD_FREE			0x2
+#define VCMD_VRSP_IP			0x1
+#define VCMD_VRSP_SC(e)			(((e) >> 1) & 0x3)
+#define VCMD_VRSP_SC_SUCCESS		0
+#define VCMD_VRSP_SC_NO_PASID_AVAIL	1
+#define VCMD_VRSP_SC_INVALID_PASID	1
+#define VCMD_VRSP_RESULT_PASID(e)	(((e) >> 8) & 0xfffff)
+#define VCMD_CMD_OPERAND(e)		((e) << 8)
 /*
  * Domain ID reserved for pasid entries programmed for first-level
  * only and pass-through transfer modes.
@@ -113,5 +123,6 @@ int intel_pasid_setup_nested(struct intel_iommu
 *iommu,
 			int addr_width);
 void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 				 struct device *dev, int pasid);
-
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int *pasid);
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid);

#endif /* __INTEL_PASID_H */

include/linux/intel-iommu.h +1 -1
index ccbf164fb711..9cbf5357138b 100644
@@ -169,6 +169,7 @@
 #define ecap_smpwc(e)		(((e) >> 48) & 0x1)
 #define ecap_flts(e)		(((e) >> 47) & 0x1)
 #define ecap_slts(e)		(((e) >> 46) & 0x1)
+#define ecap_vcs(e)		(((e) >> 44) & 0x1)
 #define ecap_smts(e)		(((e) >> 43) & 0x1)
 #define ecap_dit(e)		((e >> 41) & 0x1)
 #define ecap_pasid(e)		((e >> 40) & 0x1)
--
Quoted 1 line

2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#62 lore raw

Re: [PATCH V10 10/11] iommu/vt-d: Enlightened PASID allocation

Quoted 5 lines · Kevin
Quoted 5 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

From: Lu Baolu <[email protected]>

Quoted 9 lines
Quoted 6 lines

Enabling IOMMU in a guest requires communication with the host driver for certain aspects. Use of PASID ID to enable Shared Virtual Addressing (SVA) requires managing PASID's in the host. VT-d 3.0 spec provides a Virtual Command Register (VCMD) to facilitate this. Writes to this register in the guest are trapped by QEMU which proxies the call to the host driver.

Qemu -> vIOMMU

Sounds good. Thanks!

Quoted 131 lines
Quoted 131 lines

This virtual command interface consists of a capability register, a virtual command register, and a virtual response register. Refer to section 10.4.42, 10.4.43, 10.4.44 for more information.

This patch adds the enlightened PASID allocation/free interfaces via the virtual command interface.

---
 drivers/iommu/intel-pasid.c | 57
+++++++++++++++++++++++++++++++++++++++++++++
 drivers/iommu/intel-pasid.h | 13 ++++++++++-
 include/linux/intel-iommu.h |  1 +
 3 files changed, 70 insertions(+), 1 deletion(-)
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.c index 9f6d07410722..e87ad67aad36 100644 --- a/drivers/iommu/intel-pasid.c
+++ b/drivers/iommu/intel-pasid.c

(quoted diff) +56 -0
@@ -27,6 +27,63 @@
 static DEFINE_SPINLOCK(pasid_lock);
 u32 intel_pasid_max_id = PASID_MAX;
 
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int
 *pasid) +{
+	unsigned long flags;
+	u8 status_code;
+	int ret = 0;
+	u64 res;
+
+	raw_spin_lock_irqsave(&iommu->register_lock, flags);
+	dmar_writeq(iommu->reg + DMAR_VCMD_REG,
 VCMD_CMD_ALLOC);
+	IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+		      !(res & VCMD_VRSP_IP), res);
+	raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+	status_code = VCMD_VRSP_SC(res);
+	switch (status_code) {
+	case VCMD_VRSP_SC_SUCCESS:
+		*pasid = VCMD_VRSP_RESULT_PASID(res);
+		break;
+	case VCMD_VRSP_SC_NO_PASID_AVAIL:
+		pr_info("IOMMU: %s: No PASID available\n",
 iommu->name);
+		ret = -ENOSPC;
+		break;
+	default:
+		ret = -ENODEV;
+		pr_warn("IOMMU: %s: Unexpected error code %d\n",
+			iommu->name, status_code);
+	}
+
+	return ret;
+}
+
+void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid)
+{
+	unsigned long flags;
+	u8 status_code;
+	u64 res;
+
+	raw_spin_lock_irqsave(&iommu->register_lock, flags);
+	dmar_writeq(iommu->reg + DMAR_VCMD_REG,
+		    VCMD_CMD_OPERAND(pasid) | VCMD_CMD_FREE);
+	IOMMU_WAIT_OP(iommu, DMAR_VCRSP_REG, dmar_readq,
+		      !(res & VCMD_VRSP_IP), res);
+	raw_spin_unlock_irqrestore(&iommu->register_lock, flags);
+
+	status_code = VCMD_VRSP_SC(res);
+	switch (status_code) {
+	case VCMD_VRSP_SC_SUCCESS:
+		break;
+	case VCMD_VRSP_SC_INVALID_PASID:
+		pr_info("IOMMU: %s: Invalid PASID\n", iommu->name);
+		break;
+	default:
+		pr_warn("IOMMU: %s: Unexpected error code %d\n",
+			iommu->name, status_code);
+	}
+}
+
 /*
  * Per device pasid table management:
  */
(quoted diff) +0 -0

b/drivers/iommu/intel-pasid.h index 698015ee3f04..cd3d63f3e936 100644 --- a/drivers/iommu/intel-pasid.h
+++ b/drivers/iommu/intel-pasid.h

(quoted diff) +0 -0
@@ -23,6 +23,16 @@
 #define is_pasid_enabled(entry)		(((entry)->lo >> 3)

& 0x1) #define get_pasid_dir_size(entry) (1 <<
((((entry)->lo >> 9) & 0x7) + 7))

drivers/iommu/intel-pasid.h +11 -1
+/* Virtual command interface for enlightened pasid management. */
+#define VCMD_CMD_ALLOC			0x1
+#define VCMD_CMD_FREE			0x2
+#define VCMD_VRSP_IP			0x1
+#define VCMD_VRSP_SC(e)			(((e) >> 1) & 0x3)
+#define VCMD_VRSP_SC_SUCCESS		0
+#define VCMD_VRSP_SC_NO_PASID_AVAIL	1
+#define VCMD_VRSP_SC_INVALID_PASID	1
+#define VCMD_VRSP_RESULT_PASID(e)	(((e) >> 8) & 0xfffff)
+#define VCMD_CMD_OPERAND(e)		((e) << 8)
 /*
  * Domain ID reserved for pasid entries programmed for first-level
  * only and pass-through transfer modes.
@@ -113,5 +123,6 @@ int intel_pasid_setup_nested(struct intel_iommu
 *iommu,
 			int addr_width);
 void intel_pasid_tear_down_entry(struct intel_iommu *iommu,
 				 struct device *dev, int pasid);
-
+int vcmd_alloc_pasid(struct intel_iommu *iommu, unsigned int

*pasid); +void vcmd_free_pasid(struct intel_iommu *iommu, unsigned int pasid); #endif /* __INTEL_PASID_H */

(quoted diff) +0 -0

b/include/linux/intel-iommu.h index ccbf164fb711..9cbf5357138b 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

include/linux/intel-iommu.h +0 -0
@@ -169,6 +169,7 @@
 #define ecap_smpwc(e)		(((e) >> 48) & 0x1)
 #define ecap_flts(e)		(((e) >> 47) & 0x1)
 #define ecap_slts(e)		(((e) >> 46) & 0x1)
Quoted 9 lines
Quoted 6 lines
include/linux/intel-iommu.h +1 -1
+#define ecap_vcs(e)		(((e) >> 44) & 0x1)
 #define ecap_smts(e)		(((e) >> 43) & 0x1)
 #define ecap_dit(e)		((e >> 41) & 0x1)
 #define ecap_pasid(e)		((e >> 40) & 0x1)
--

2.7.4

[Jacob Pan]

author
#63 lore raw

PATCH v10 11/11 iommu/vt-d: Add custom allocator for IOASID

In reply to Jacob Pan

When VT-d driver runs in the guest, PASID allocation must be performed via virtual command interface. This patch registers a custom IOASID allocator which takes precedence over the default XArray based allocator. The resulting IOASID allocation will always come from the host. This ensures that PASID namespace is system- wide.

---
 drivers/iommu/intel-iommu.c | 84 +++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  2 ++
 2 files changed, 86 insertions(+)
drivers/iommu/intel-iommu.c +84 -0
index a76afb0fd51a..c1c0b0fb93c3 100644
@@ -1757,6 +1757,9 @@ static void free_dmar_iommu(struct intel_iommu *iommu)
 		if (ecap_prs(iommu->ecap))
 			intel_svm_finish_prq(iommu);
 	}
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap))
+		ioasid_unregister_allocator(&iommu->pasid_allocator);
+
 #endif
 }
 
@@ -3291,6 +3294,84 @@ static int copy_translation_tables(struct intel_iommu *iommu)
 	return ret;
 }
 
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max, void *data)
+{
+	struct intel_iommu *iommu = data;
+	ioasid_t ioasid;
+
+	if (!iommu)
+		return INVALID_IOASID;
+	/*
+	 * VT-d virtual command interface always uses the full 20 bit
+	 * PASID range. Host can partition guest PASID range based on
+	 * policies but it is out of guest's control.
+	 */
+	if (min < PASID_MIN || max > intel_pasid_max_id)
+		return INVALID_IOASID;
+
+	if (vcmd_alloc_pasid(iommu, &ioasid))
+		return INVALID_IOASID;
+
+	return ioasid;
+}
+
+static void intel_ioasid_free(ioasid_t ioasid, void *data)
+{
+	struct intel_iommu *iommu = data;
+
+	if (!iommu)
+		return;
+	/*
+	 * Sanity check the ioasid owner is done at upper layer, e.g. VFIO
+	 * We can only free the PASID when all the devices are unbound.
+	 */
+	if (ioasid_find(NULL, ioasid, NULL)) {
+		pr_alert("Cannot free active IOASID %d\n", ioasid);
+		return;
+	}
+	vcmd_free_pasid(iommu, ioasid);
+}
+
+static void register_pasid_allocator(struct intel_iommu *iommu)
+{
+	/*
+	 * If we are running in the host, no need for custom allocator
+	 * in that PASIDs are allocated from the host system-wide.
+	 */
+	if (!cap_caching_mode(iommu->cap))
+		return;
+
+	if (!sm_supported(iommu)) {
+		pr_warn("VT-d Scalable Mode not enabled, no PASID allocation\n");
+		return;
+	}
+
+	/*
+	 * Register a custom PASID allocator if we are running in a guest,
+	 * guest PASID must be obtained via virtual command interface.
+	 * There can be multiple vIOMMUs in each guest but only one allocator
+	 * is active. All vIOMMU allocators will eventually be calling the same
+	 * host allocator.
+	 */
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap)) {
+		pr_info("Register custom PASID allocator\n");
+		iommu->pasid_allocator.alloc = intel_ioasid_alloc;
+		iommu->pasid_allocator.free = intel_ioasid_free;
+		iommu->pasid_allocator.pdata = (void *)iommu;
+		if (ioasid_register_allocator(&iommu->pasid_allocator)) {
+			pr_warn("Custom PASID allocator failed, scalable mode disabled\n");
+			/*
+			 * Disable scalable mode on this IOMMU if there
+			 * is no custom allocator. Mixing SM capable vIOMMU
+			 * and non-SM vIOMMU are not supported.
+			 */
+			intel_iommu_sm = 0;
+		}
+	}
+}
+#endif
+
 static int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -3408,6 +3489,9 @@ static int __init init_dmars(void)
 	 */
 	for_each_active_iommu(iommu, drhd) {
 		iommu_flush_write_buffer(iommu);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+		register_pasid_allocator(iommu);
+#endif
 		iommu_set_root_entry(iommu);
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
include/linux/intel-iommu.h +2 -0
index 9cbf5357138b..9c357a325c72 100644
@@ -19,6 +19,7 @@
 #include <linux/iommu.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/dmar.h>
+#include <linux/ioasid.h>
 
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -563,6 +564,7 @@ struct intel_iommu {
 #ifdef CONFIG_INTEL_IOMMU_SVM
 	struct page_req_dsc *prq;
 	unsigned char prq_name[16];    /* Name for PRQ interrupt */
+	struct ioasid_allocator_ops pasid_allocator; /* Custom allocator for PASIDs */
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
 	u32 *iommu_state; /* Store iommu states between suspend and resume.*/
Signature · 6 lines
2.7.4

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#64 lore raw

RE: [PATCH V10 11/11] iommu/vt-d: Add custom allocator for IOASID

Quoted 35 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When VT-d driver runs in the guest, PASID allocation must be performed via virtual command interface. This patch registers a custom IOASID allocator which takes precedence over the default XArray based allocator. The resulting IOASID allocation will always come from the host. This ensures that PASID namespace is system- wide.

---
 drivers/iommu/intel-iommu.c | 84
+++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  2 ++
 2 files changed, 86 insertions(+)
drivers/iommu/intel-iommu.c +3 -0
index a76afb0fd51a..c1c0b0fb93c3 100644
@@ -1757,6 +1757,9 @@ static void free_dmar_iommu(struct intel_iommu
 *iommu)
 		if (ecap_prs(iommu->ecap))
 			intel_svm_finish_prq(iommu);
 	}
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap))
+		ioasid_unregister_allocator(&iommu->pasid_allocator);
+
 #endif
 }

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct · line 3299

Quoted 6 lines
drivers/iommu/intel-iommu.c +2 -0
@@ -3291,6 +3294,84 @@ static int copy_translation_tables(struct
 intel_iommu *iommu)
 	return ret;
 }
 
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max, void *data)

the name is too generic... can we add vcmd in the name to clarify its purpose, e.g. intel_vcmd_ioasid_alloc?

Quoted 24 lines
drivers/iommu/intel-iommu.c +24 -0
+{
+	struct intel_iommu *iommu = data;
+	ioasid_t ioasid;
+
+	if (!iommu)
+		return INVALID_IOASID;
+	/*
+	 * VT-d virtual command interface always uses the full 20 bit
+	 * PASID range. Host can partition guest PASID range based on
+	 * policies but it is out of guest's control.
+	 */
+	if (min < PASID_MIN || max > intel_pasid_max_id)
+		return INVALID_IOASID;
+
+	if (vcmd_alloc_pasid(iommu, &ioasid))
+		return INVALID_IOASID;
+
+	return ioasid;
+}
+
+static void intel_ioasid_free(ioasid_t ioasid, void *data)
+{
+	struct intel_iommu *iommu = data;
+

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3332

Quoted 10 lines
drivers/iommu/intel-iommu.c +10 -0
+	if (!iommu)
+		return;
+	/*
+	 * Sanity check the ioasid owner is done at upper layer, e.g. VFIO
+	 * We can only free the PASID when all the devices are unbound.
+	 */
+	if (ioasid_find(NULL, ioasid, NULL)) {
+		pr_alert("Cannot free active IOASID %d\n", ioasid);
+		return;
+	}

However the sanity check is not done in default_free. Is there a reason why using vcmd adds such new requirement?

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3343

Quoted 11 lines
drivers/iommu/intel-iommu.c +11 -0
+	vcmd_free_pasid(iommu, ioasid);
+}
+
+static void register_pasid_allocator(struct intel_iommu *iommu)
+{
+	/*
+	 * If we are running in the host, no need for custom allocator
+	 * in that PASIDs are allocated from the host system-wide.
+	 */
+	if (!cap_caching_mode(iommu->cap))
+		return;

is it more accurate to check against vcmd capability?

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3354

Quoted 13 lines
drivers/iommu/intel-iommu.c +10 -0
+
+	if (!sm_supported(iommu)) {
+		pr_warn("VT-d Scalable Mode not enabled, no PASID
 allocation\n");
+		return;
+	}
+
+	/*
+	 * Register a custom PASID allocator if we are running in a guest,
+	 * guest PASID must be obtained via virtual command interface.
+	 * There can be multiple vIOMMUs in each guest but only one

allocator

drivers/iommu/intel-iommu.c +1 -0
+	 * is active. All vIOMMU allocators will eventually be calling the same

which one? the first or last?

Quoted 14 lines
drivers/iommu/intel-iommu.c +9 -0
+	 * host allocator.
+	 */
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap)) {
+		pr_info("Register custom PASID allocator\n");
+		iommu->pasid_allocator.alloc = intel_ioasid_alloc;
+		iommu->pasid_allocator.free = intel_ioasid_free;
+		iommu->pasid_allocator.pdata = (void *)iommu;
+		if (ioasid_register_allocator(&iommu->pasid_allocator)) {
+			pr_warn("Custom PASID allocator failed, scalable

mode disabled\n");

drivers/iommu/intel-iommu.c +3 -0
+			/*
+			 * Disable scalable mode on this IOMMU if there
+			 * is no custom allocator. Mixing SM capable

vIOMMU

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3369

Quoted 3 lines
drivers/iommu/intel-iommu.c +3 -0
+			 * and non-SM vIOMMU are not supported.
+			 */
+			intel_iommu_sm = 0;

since you register an allocator for every vIOMMU, means previously registered allocators should also be unregistered here?

Quoted 44 lines
drivers/iommu/intel-iommu.c +8 -0
+		}
+	}
+}
+#endif
+
 static int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -3408,6 +3489,9 @@ static int __init init_dmars(void)
 	 */
 	for_each_active_iommu(iommu, drhd) {
 		iommu_flush_write_buffer(iommu);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+		register_pasid_allocator(iommu);
+#endif
 		iommu_set_root_entry(iommu);
 		iommu->flush.flush_context(iommu, 0, 0, 0,
 DMA_CCMD_GLOBAL_INVL);

iommu->flush.flush_iotlb(iommu, 0, 0, 0,
DMA_TLB_GLOBAL_FLUSH);

include/linux/intel-iommu.h +2 -0
index 9cbf5357138b..9c357a325c72 100644
@@ -19,6 +19,7 @@
 #include <linux/iommu.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/dmar.h>
+#include <linux/ioasid.h>
 
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -563,6 +564,7 @@ struct intel_iommu {
 #ifdef CONFIG_INTEL_IOMMU_SVM
 	struct page_req_dsc *prq;
 	unsigned char prq_name[16];    /* Name for PRQ interrupt */
+	struct ioasid_allocator_ops pasid_allocator; /* Custom allocator for

PASIDs */
#endif

include/linux/intel-iommu.h +0 -0
 	struct q_inval  *qi;            /* Queued invalidation info */
 	u32 *iommu_state; /* Store iommu states between suspend and

resume.*/
--
2.7.4

author
#65 lore raw

Re: [PATCH V10 11/11] iommu/vt-d: Add custom allocator for IOASID

Quoted 39 lines · Kevin
Quoted 39 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When VT-d driver runs in the guest, PASID allocation must be performed via virtual command interface. This patch registers a custom IOASID allocator which takes precedence over the default XArray based allocator. The resulting IOASID allocation will always come from the host. This ensures that PASID namespace is system- wide.

---
 drivers/iommu/intel-iommu.c | 84
+++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  2 ++
 2 files changed, 86 insertions(+)
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index a76afb0fd51a..c1c0b0fb93c3 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

drivers/iommu/intel-iommu.c +2 -0
@@ -1757,6 +1757,9 @@ static void free_dmar_iommu(struct intel_iommu
 *iommu)
 		if (ecap_prs(iommu->ecap))
 			intel_svm_finish_prq(iommu);
 	}
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap))
+

ioasid_unregister_allocator(&iommu->pasid_allocator); +
#endif
}

drivers/iommu/intel-iommu.c +0 -0
@@ -3291,6 +3294,84 @@ static int copy_translation_tables(struct
 intel_iommu *iommu)
 	return ret;
 }
 
Quoted 7 lines

Review static int copy_translation_tables(struct · line 3300

Quoted 3 lines
(quoted diff) +2 -0
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max,
 void *data)  

the name is too generic... can we add vcmd in the name to clarify its purpose, e.g. intel_vcmd_ioasid_alloc?

I feel the intel_ prefix is a natural extension of a generic API, we do that for other IOMMU APIs, right?

Quoted 33 lines
Quoted 33 lines
drivers/iommu/intel-iommu.c +8 -0
+{
+	struct intel_iommu *iommu = data;
+	ioasid_t ioasid;
+
+	if (!iommu)
+		return INVALID_IOASID;
+	/*
+	 * VT-d virtual command interface always uses the full 20

bit

(quoted diff) +1 -0
+	 * PASID range. Host can partition guest PASID range based

on

drivers/iommu/intel-iommu.c +19 -0
+	 * policies but it is out of guest's control.
+	 */
+	if (min < PASID_MIN || max > intel_pasid_max_id)
+		return INVALID_IOASID;
+
+	if (vcmd_alloc_pasid(iommu, &ioasid))
+		return INVALID_IOASID;
+
+	return ioasid;
+}
+
+static void intel_ioasid_free(ioasid_t ioasid, void *data)
+{
+	struct intel_iommu *iommu = data;
+
+	if (!iommu)
+		return;
+	/*
+	 * Sanity check the ioasid owner is done at upper layer,

e.g. VFIO

(quoted diff) +1 -0
+	 * We can only free the PASID when all the devices are

unbound.

Quoted 9 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3332

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+	 */
+	if (ioasid_find(NULL, ioasid, NULL)) {
+		pr_alert("Cannot free active IOASID %d\n", ioasid);
+		return;
+	}  

However the sanity check is not done in default_free. Is there a reason why using vcmd adds such new requirement?

Since we don't support nested guest. This vcmd allocator is only used by the guest IOMMU driver not VFIO. We expect IOMMU driver to have control of the free()/unbind() ordering.

For default_free, it can come from user space and host VFIO which can be out of order. But we will solve that issue with the blocking notifier.

Quoted 6 lines
Quoted 6 lines
drivers/iommu/intel-iommu.c +6 -0
+	vcmd_free_pasid(iommu, ioasid);
+}
+
+static void register_pasid_allocator(struct intel_iommu *iommu)
+{
+	/*
Quoted 9 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3343

Quoted 6 lines
(quoted diff) +1 -0
+	 * If we are running in the host, no need for custom

allocator

drivers/iommu/intel-iommu.c +4 -0
+	 * in that PASIDs are allocated from the host system-wide.
+	 */
+	if (!cap_caching_mode(iommu->cap))
+		return;  

is it more accurate to check against vcmd capability?

I think this is sufficient. The spec says if vcmd is present, we must use it but not the other way.

Quoted 10 lines
Quoted 10 lines
drivers/iommu/intel-iommu.c +3 -0
+
+	if (!sm_supported(iommu)) {
+		pr_warn("VT-d Scalable Mode not enabled, no PASID

allocation\n");

drivers/iommu/intel-iommu.c +5 -0
+		return;
+	}
+
+	/*
+	 * Register a custom PASID allocator if we are running in

a guest,

Quoted 9 lines
Quoted 6 lines
(quoted diff) +1 -0
+	 * guest PASID must be obtained via virtual command

interface.

(quoted diff) +1 -0
+	 * There can be multiple vIOMMUs in each guest but only one

allocator

(quoted diff) +1 -0
+	 * is active. All vIOMMU allocators will eventually be

calling the same

which one? the first or last?

All allocators share the same ops, so first=last. IOASID code will inspect the ops function and see if they are shared with others then use the same ops.

Quoted 14 lines
Quoted 14 lines
drivers/iommu/intel-iommu.c +8 -0
+	 * host allocator.
+	 */
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap)) {
+		pr_info("Register custom PASID allocator\n");
+		iommu->pasid_allocator.alloc = intel_ioasid_alloc;
+		iommu->pasid_allocator.free = intel_ioasid_free;
+		iommu->pasid_allocator.pdata = (void *)iommu;
+		if

(ioasid_register_allocator(&iommu->pasid_allocator)) {

(quoted diff) +1 -0
+			pr_warn("Custom PASID allocator failed,

scalable mode disabled\n");

(quoted diff) +2 -0
+			/*
+			 * Disable scalable mode on this IOMMU if

there

Quoted 9 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3369

Quoted 5 lines
(quoted diff) +1 -0
+			 * is no custom allocator. Mixing SM

capable vIOMMU

drivers/iommu/intel-iommu.c +3 -0
+			 * and non-SM vIOMMU are not supported.
+			 */
+			intel_iommu_sm = 0;  

since you register an allocator for every vIOMMU, means previously registered allocators should also be unregistered here?

True, but it is not necessary for two reasons:
1. This should not happen unless something went seriously wrong. All vIOMMU shares the same alloc/free function, so they are put under the same bucket by IOASID. So the case for the first vIOMMU to succeed then fail in later vIOMMU registration should not happen. Unless kernel run out of memory etc.

2. Once SM is disabled, there is no user of ioasid allocator.

Quoted 30 lines
Quoted 30 lines
drivers/iommu/intel-iommu.c +8 -0
+		}
+	}
+}
+#endif
+
 static int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -3408,6 +3489,9 @@ static int __init init_dmars(void)
 	 */
 	for_each_active_iommu(iommu, drhd) {
 		iommu_flush_write_buffer(iommu);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+		register_pasid_allocator(iommu);
+#endif
 		iommu_set_root_entry(iommu);
 		iommu->flush.flush_context(iommu, 0, 0, 0,
 DMA_CCMD_GLOBAL_INVL);

iommu->flush.flush_iotlb(iommu, 0, 0, 0,
DMA_TLB_GLOBAL_FLUSH);

(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 9cbf5357138b..9c357a325c72 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

include/linux/intel-iommu.h +1 -0
@@ -19,6 +19,7 @@
 #include <linux/iommu.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/dmar.h>
+#include <linux/ioasid.h>
 
Quoted 15 lines
Quoted 7 lines
include/linux/intel-iommu.h +1 -0
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -563,6 +564,7 @@ struct intel_iommu {
 #ifdef CONFIG_INTEL_IOMMU_SVM
 	struct page_req_dsc *prq;
 	unsigned char prq_name[16];    /* Name for PRQ interrupt */
+	struct ioasid_allocator_ops pasid_allocator; /* Custom
Quoted 7 lines

allocator for PASIDs */
#endif
struct q_inval *qi; /* Queued invalidation
info */ u32 *iommu_state; /* Store iommu states between suspend and resume.*/
--
2.7.4

[Jacob Pan]

#66 lore raw

RE: [PATCH V10 11/11] iommu/vt-d: Add custom allocator for IOASID

Quoted 53 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 11:48 PM

Quoted 19 lines · Kevin
Quoted 19 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When VT-d driver runs in the guest, PASID allocation must be performed via virtual command interface. This patch registers a custom IOASID allocator which takes precedence over the default XArray based allocator. The resulting IOASID allocation will always come from the host. This ensures that PASID namespace is system- wide.

---
 drivers/iommu/intel-iommu.c | 84
+++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  2 ++
 2 files changed, 86 insertions(+)
Quoted 5 lines

Review static void free_dmar_iommu(struct

Quoted 5 lines
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index a76afb0fd51a..c1c0b0fb93c3 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +0 -0
@@ -1757,6 +1757,9 @@ static void free_dmar_iommu(struct

intel_iommu

Quoted 15 lines
Quoted 15 lines
drivers/iommu/intel-iommu.c +2 -0
 *iommu)
 		if (ecap_prs(iommu->ecap))
 			intel_svm_finish_prq(iommu);
 	}
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap))
+

ioasid_unregister_allocator(&iommu->pasid_allocator); +
#endif
}

drivers/iommu/intel-iommu.c +0 -0
@@ -3291,6 +3294,84 @@ static int copy_translation_tables(struct
 intel_iommu *iommu)
 	return ret;
 }
 
Quoted 7 lines

Review static int copy_translation_tables(struct · line 3300

Quoted 3 lines
(quoted diff) +2 -0
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max,
 void *data)

the name is too generic... can we add vcmd in the name to clarify its purpose, e.g. intel_vcmd_ioasid_alloc?

Quoted 2 lines

I feel the intel_ prefix is a natural extension of a generic API, we do that for other IOMMU APIs, right?

other IOMMU APIs have no difference between host and guest, but this one only applies to guest with vcmd interface.

Quoted 66 lines
Quoted 34 lines
Quoted 34 lines
drivers/iommu/intel-iommu.c +8 -0
+{
+	struct intel_iommu *iommu = data;
+	ioasid_t ioasid;
+
+	if (!iommu)
+		return INVALID_IOASID;
+	/*
+	 * VT-d virtual command interface always uses the full 20

bit

(quoted diff) +1 -0
+	 * PASID range. Host can partition guest PASID range based

on

drivers/iommu/intel-iommu.c +19 -0
+	 * policies but it is out of guest's control.
+	 */
+	if (min < PASID_MIN || max > intel_pasid_max_id)
+		return INVALID_IOASID;
+
+	if (vcmd_alloc_pasid(iommu, &ioasid))
+		return INVALID_IOASID;
+
+	return ioasid;
+}
+
+static void intel_ioasid_free(ioasid_t ioasid, void *data)
+{
+	struct intel_iommu *iommu = data;
+
+	if (!iommu)
+		return;
+	/*
+	 * Sanity check the ioasid owner is done at upper layer,

e.g. VFIO

(quoted diff) +1 -0
+	 * We can only free the PASID when all the devices are

unbound.

Quoted 9 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3332

Quoted 5 lines
drivers/iommu/intel-iommu.c +5 -0
+	 */
+	if (ioasid_find(NULL, ioasid, NULL)) {
+		pr_alert("Cannot free active IOASID %d\n", ioasid);
+		return;
+	}

However the sanity check is not done in default_free. Is there a reason why using vcmd adds such new requirement?

Since we don't support nested guest. This vcmd allocator is only used by the guest IOMMU driver not VFIO. We expect IOMMU driver to have control of the free()/unbind() ordering.

For default_free, it can come from user space and host VFIO which can be out of order. But we will solve that issue with the blocking notifier.

Quoted 6 lines
Quoted 6 lines
drivers/iommu/intel-iommu.c +6 -0
+	vcmd_free_pasid(iommu, ioasid);
+}
+
+static void register_pasid_allocator(struct intel_iommu *iommu)
+{
+	/*
Quoted 9 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3343

Quoted 6 lines
(quoted diff) +1 -0
+	 * If we are running in the host, no need for custom

allocator

drivers/iommu/intel-iommu.c +4 -0
+	 * in that PASIDs are allocated from the host system-wide.
+	 */
+	if (!cap_caching_mode(iommu->cap))
+		return;

is it more accurate to check against vcmd capability?

Quoted 2 lines

I think this is sufficient. The spec says if vcmd is present, we must use it but not the other way.

No, what about an vIOMMU implementation reports CM but not VCMD? I didn't get the rationale why we check an indirect capability when there is already one well defined for the purpose.

Quoted 20 lines
Quoted 11 lines
Quoted 11 lines
drivers/iommu/intel-iommu.c +3 -0
+
+	if (!sm_supported(iommu)) {
+		pr_warn("VT-d Scalable Mode not enabled, no PASID

allocation\n");

drivers/iommu/intel-iommu.c +5 -0
+		return;
+	}
+
+	/*
+	 * Register a custom PASID allocator if we are running in

a guest,

Quoted 9 lines
Quoted 6 lines
(quoted diff) +1 -0
+	 * guest PASID must be obtained via virtual command

interface.

(quoted diff) +1 -0
+	 * There can be multiple vIOMMUs in each guest but only one

allocator

(quoted diff) +1 -0
+	 * is active. All vIOMMU allocators will eventually be

calling the same

which one? the first or last?

Quoted 3 lines

All allocators share the same ops, so first=last. IOASID code will inspect the ops function and see if they are shared with others then use the same ops.

ok, got you.

Quoted 80 lines
Quoted 15 lines
Quoted 15 lines
drivers/iommu/intel-iommu.c +8 -0
+	 * host allocator.
+	 */
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap)) {
+		pr_info("Register custom PASID allocator\n");
+		iommu->pasid_allocator.alloc = intel_ioasid_alloc;
+		iommu->pasid_allocator.free = intel_ioasid_free;
+		iommu->pasid_allocator.pdata = (void *)iommu;
+		if

(ioasid_register_allocator(&iommu->pasid_allocator)) {

(quoted diff) +1 -0
+			pr_warn("Custom PASID allocator failed,

scalable mode disabled\n");

(quoted diff) +2 -0
+			/*
+			 * Disable scalable mode on this IOMMU if

there

Quoted 9 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3369

Quoted 5 lines
(quoted diff) +1 -0
+			 * is no custom allocator. Mixing SM

capable vIOMMU

drivers/iommu/intel-iommu.c +3 -0
+			 * and non-SM vIOMMU are not supported.
+			 */
+			intel_iommu_sm = 0;

since you register an allocator for every vIOMMU, means previously registered allocators should also be unregistered here?

True, but it is not necessary for two reasons:
1. This should not happen unless something went seriously wrong. All vIOMMU shares the same alloc/free function, so they are put under the same bucket by IOASID. So the case for the first vIOMMU to succeed then fail in later vIOMMU registration should not happen. Unless kernel run out of memory etc.

2. Once SM is disabled, there is no user of ioasid allocator.

Quoted 30 lines
Quoted 30 lines
drivers/iommu/intel-iommu.c +8 -0
+		}
+	}
+}
+#endif
+
 static int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -3408,6 +3489,9 @@ static int __init init_dmars(void)
 	 */
 	for_each_active_iommu(iommu, drhd) {
 		iommu_flush_write_buffer(iommu);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+		register_pasid_allocator(iommu);
+#endif
 		iommu_set_root_entry(iommu);
 		iommu->flush.flush_context(iommu, 0, 0, 0,
 DMA_CCMD_GLOBAL_INVL);

iommu->flush.flush_iotlb(iommu, 0, 0, 0,
DMA_TLB_GLOBAL_FLUSH);

(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 9cbf5357138b..9c357a325c72 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

include/linux/intel-iommu.h +1 -0
@@ -19,6 +19,7 @@
 #include <linux/iommu.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/dmar.h>
+#include <linux/ioasid.h>
 
Quoted 15 lines
Quoted 7 lines
include/linux/intel-iommu.h +1 -0
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -563,6 +564,7 @@ struct intel_iommu {
 #ifdef CONFIG_INTEL_IOMMU_SVM
 	struct page_req_dsc *prq;
 	unsigned char prq_name[16];    /* Name for PRQ interrupt */
+	struct ioasid_allocator_ops pasid_allocator; /* Custom
Quoted 7 lines

allocator for PASIDs */
#endif
struct q_inval *qi; /* Queued invalidation
info */ u32 *iommu_state; /* Store iommu states between suspend and resume.*/
--
2.7.4

[Jacob Pan]

author
#67 lore raw

Re: [PATCH V10 11/11] iommu/vt-d: Add custom allocator for IOASID

Quoted 53 lines · Kevin
Quoted 53 lines

From: Jacob Pan <[email protected]>
Sent: Wednesday, April 1, 2020 11:48 PM

Quoted 19 lines · Kevin
Quoted 19 lines

From: Jacob Pan <[email protected]>
Sent: Saturday, March 21, 2020 7:28 AM

When VT-d driver runs in the guest, PASID allocation must be performed via virtual command interface. This patch registers a custom IOASID allocator which takes precedence over the default XArray based allocator. The resulting IOASID allocation will always come from the host. This ensures that PASID namespace is system- wide.

---
 drivers/iommu/intel-iommu.c | 84
+++++++++++++++++++++++++++++++++++++++++++++
 include/linux/intel-iommu.h |  2 ++
 2 files changed, 86 insertions(+)
Quoted 5 lines

Review static void free_dmar_iommu(struct

Quoted 5 lines
(quoted diff) +0 -0

b/drivers/iommu/intel-iommu.c index a76afb0fd51a..c1c0b0fb93c3 100644 --- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c

(quoted diff) +0 -0
@@ -1757,6 +1757,9 @@ static void free_dmar_iommu(struct  

intel_iommu

Quoted 22 lines
Quoted 11 lines
drivers/iommu/intel-iommu.c +2 -0
 *iommu)
 		if (ecap_prs(iommu->ecap))
 			intel_svm_finish_prq(iommu);
 	}
+	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap))
+

ioasid_unregister_allocator(&iommu->pasid_allocator); +
#endif
}

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct · line 3300

Quoted 7 lines
drivers/iommu/intel-iommu.c +2 -0
@@ -3291,6 +3294,84 @@ static int copy_translation_tables(struct
 intel_iommu *iommu)
 	return ret;
 }
 
+#ifdef CONFIG_INTEL_IOMMU_SVM
+static ioasid_t intel_ioasid_alloc(ioasid_t min, ioasid_t max,
 void *data)  

the name is too generic... can we add vcmd in the name to clarify its purpose, e.g. intel_vcmd_ioasid_alloc?

Quoted 6 lines
Quoted 2 lines

I feel the intel_ prefix is a natural extension of a generic API, we do that for other IOMMU APIs, right?

other IOMMU APIs have no difference between host and guest, but this one only applies to guest with vcmd interface.

OK, sounds good. It is more explicit, improves readability.

Quoted 68 lines
Quoted 68 lines
Quoted 35 lines
Quoted 35 lines
drivers/iommu/intel-iommu.c +8 -0
+{
+	struct intel_iommu *iommu = data;
+	ioasid_t ioasid;
+
+	if (!iommu)
+		return INVALID_IOASID;
+	/*
+	 * VT-d virtual command interface always uses the full

20 bit

(quoted diff) +1 -0
+	 * PASID range. Host can partition guest PASID range

based on

drivers/iommu/intel-iommu.c +19 -0
+	 * policies but it is out of guest's control.
+	 */
+	if (min < PASID_MIN || max > intel_pasid_max_id)
+		return INVALID_IOASID;
+
+	if (vcmd_alloc_pasid(iommu, &ioasid))
+		return INVALID_IOASID;
+
+	return ioasid;
+}
+
+static void intel_ioasid_free(ioasid_t ioasid, void *data)
+{
+	struct intel_iommu *iommu = data;
+
+	if (!iommu)
+		return;
+	/*
+	 * Sanity check the ioasid owner is done at upper

layer, e.g. VFIO

(quoted diff) +1 -0
+	 * We can only free the PASID when all the devices are

unbound.

(quoted diff) +1 -0
+	 */
Quoted 9 lines

Review

Quoted 5 lines
(quoted diff) +2 -0
+	if (ioasid_find(NULL, ioasid, NULL)) {
+		pr_alert("Cannot free active IOASID %d\n",

ioasid);

(quoted diff) +2 -0
+		return;
+	}  

However the sanity check is not done in default_free. Is there a reason why using vcmd adds such new requirement?

Since we don't support nested guest. This vcmd allocator is only used by the guest IOMMU driver not VFIO. We expect IOMMU driver to have control of the free()/unbind() ordering.

For default_free, it can come from user space and host VFIO which can be out of order. But we will solve that issue with the blocking notifier.

Quoted 16 lines

drivers/iommu/intel-iommu.c static int copy_translation_tables(struct intel_iommu *iommu) · line 3343

Quoted 13 lines
drivers/iommu/intel-iommu.c +7 -0
+	vcmd_free_pasid(iommu, ioasid);
+}
+
+static void register_pasid_allocator(struct intel_iommu *iommu)
+{
+	/*
+	 * If we are running in the host, no need for custom

allocator

(quoted diff) +1 -0
+	 * in that PASIDs are allocated from the host

system-wide.

drivers/iommu/intel-iommu.c +3 -0
+	 */
+	if (!cap_caching_mode(iommu->cap))
+		return;  

is it more accurate to check against vcmd capability?

Quoted 8 lines
Quoted 2 lines

I think this is sufficient. The spec says if vcmd is present, we must use it but not the other way.

No, what about an vIOMMU implementation reports CM but not VCMD?
I didn't get the rationale why we check an indirect capability when there is already one well defined for the purpose.

We _do_ check ecap_vcs() later on. Just an ordering thing, my thinking was a quick check if we are running in a host.
...

	if (ecap_vcs(iommu->ecap) && vccap_pasid(iommu->vccap)) {
Quoted 12 lines
Quoted 1 line

[...]

Quoted 6 lines
Quoted 3 lines

which one? the first or last?

All allocators share the same ops, so first=last. IOASID code will inspect the ops function and see if they are shared with others then use the same ops.

ok, got you.

Quoted 1 line
Quoted 54 lines

[...]

Quoted 4 lines
Quoted 4 lines

since you register an allocator for every vIOMMU, means previously registered allocators should also be unregistered here?

[...]

Quoted 48 lines
Quoted 30 lines
Quoted 30 lines
drivers/iommu/intel-iommu.c +8 -0
+		}
+	}
+}
+#endif
+
 static int __init init_dmars(void)
 {
 	struct dmar_drhd_unit *drhd;
@@ -3408,6 +3489,9 @@ static int __init init_dmars(void)
 	 */
 	for_each_active_iommu(iommu, drhd) {
 		iommu_flush_write_buffer(iommu);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+		register_pasid_allocator(iommu);
+#endif
 		iommu_set_root_entry(iommu);
 		iommu->flush.flush_context(iommu, 0, 0, 0,
 DMA_CCMD_GLOBAL_INVL);

iommu->flush.flush_iotlb(iommu, 0, 0, 0,
DMA_TLB_GLOBAL_FLUSH);

(quoted diff) +0 -0

b/include/linux/intel-iommu.h index 9cbf5357138b..9c357a325c72 100644 --- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h

include/linux/intel-iommu.h +1 -0
@@ -19,6 +19,7 @@
 #include <linux/iommu.h>
 #include <linux/io-64-nonatomic-lo-hi.h>
 #include <linux/dmar.h>
+#include <linux/ioasid.h>
 
Quoted 16 lines
Quoted 8 lines
include/linux/intel-iommu.h +0 -0
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -563,6 +564,7 @@ struct intel_iommu {
 #ifdef CONFIG_INTEL_IOMMU_SVM
 	struct page_req_dsc *prq;
 	unsigned char prq_name[16];    /* Name for PRQ

interrupt */

(quoted diff) +1 -0
+	struct ioasid_allocator_ops pasid_allocator; /* Custom
Quoted 7 lines

allocator for PASIDs */
#endif
struct q_inval *qi; /* Queued invalidation
info */ u32 *iommu_state; /* Store iommu states between suspend and resume.*/
--
2.7.4

[Jacob Pan]

[Jacob Pan]