Nested Shared Virtual Address (SVA) VT-d support
v13 cover Nested Shared Virtual Address (SVA) VT-d support
Shared virtual address (SVA), a.k.a, Shared virtual memory (SVM) on Intel platforms allow address space sharing between device DMA and applications. SVA can reduce programming complexity and enhance security.
This series is intended to enable SVA virtualization, i.e. enable use of SVA within a guest user application.
This is the remaining portion of the original patchset that is based on Joerg's x86/vt-d branch. The preparatory and cleanup patches are merged here.
(git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git)
Only IOMMU portion of the changes are included in this series. Additional support is needed in VFIO and QEMU (will be submitted separately) to complete this functionality.
To make incremental changes and reduce the size of each patchset. This series does not inlcude support for page request services.
In VT-d implementation, PASID table is per device and maintained in the host. Guest PASID table is shadowed in VMM where virtual IOMMU is emulated.
.-------------. .---------------------------.
| vIOMMU | | Guest process CR3, FL only|
| | '---------------------------' .----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest
------| Shadow |--------------------------|--------
v v v
Host
.-------------. .----------------------.
| pIOMMU | | Bind FL for GVA-GPA |
| | '----------------------'
.----------------/ |
| PASID Entry | V (Nested xlate)
'----------------\.------------------------------.
| | |SL for GPA-HPA, default domain|
| | '------------------------------'
'-------------'Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables
This is the remaining VT-d only portion of V5 since the uAPIs and IOASID common code have been applied to Joerg's IOMMU core branch.
(https://lkml.org/lkml/2019/10/2/833)
The complete set with VFIO patches are here:
https://github.com/jacobpan/linux.git:siov_sva
The complete nested SVA upstream patches are divided into three phases:
1. Common APIs and PCI device direct assignment
2. Page Request Services (PRS) support
3. Mediated device assignment
With this set and the accompanied VFIO code, we will achieve phase #1.
Thanks,
Jacob
ChangeLog:
- v13
- Dropped memory type support (MTS) in guest PASID bind
- Do not support multiple bind gpasid if device has no aux domain
- Removed extra error msgs in pasid_setup_bind_data()
- Replaced SVM device list free function with combined out label
- v12
- Fixed IA64 cross compile error
- Squashed two patches that add macros with its users
- Use ratelimited prints for all user called APIs
- Check domain nesting attr for vSVA APIs.
- Misc style improvements
- v11 Misc fixes and improvements based on review by Kevin & Eric
- Fixed devTLB granularity conversion
- Simplified VT-d granulairy lookup by replacing 2D map array with invalid entries.
- Fixed locking in bind guest PASID
- Added nesting domain attr check
- Squashed agaw checking patch with user
- Use rate limitted error message for all user originated calls
- v10
- Addressed Eric's review in v7 and v9. Most fixes are in 3/10 and 6/10. Extra condition checks and consolidation of duplicated codes.
- v9
- Addressed Baolu's comments for v8 for IOTLB flush consolidation, bug fixes
- Removed IOASID notifier code which will be submitted separately to address PASID life cycle management with multiple users.
- v8
- Extracted cleanup patches from V7 and accepted into maintainer's tree (https://lkml.org/lkml/2019/12/2/514).
- Added IOASID notifier and VT-d handler for termination of PASID IOMMU context upon free. This will ensure success of VFIO IOASID free API regardless PASID is in use.
(https://lore.kernel.org/linux-iommu/[email protected]/)
- V7
- Respect vIOMMU PASID range in virtual command PASID/IOASID allocator
- Caching virtual command capabilities to avoid runtime checks that could cause vmexits.
- V6
- Rebased on top of Joerg's core branch
(git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git core)
- Adapt to new uAPIs and IOASID allocators
- V5
Rebased on v5.3-rc4 which has some of the IOMMU fault APIs merged. Addressed v4 review comments from Eric Auger, Baolu Lu, and
Jonathan Cameron. Specific changes are as follows:
- Refined custom IOASID allocator to support multiple vIOMMU, hotplug cases.
- Extracted vendor data from IOMMU guest PASID bind data, for VT-d will support all necessary guest PASID entry fields for PASID bind.
- Support non-identity host-guest PASID mapping
- Exception handling in various cases
- V4
- Redesigned IOASID allocator such that it can support custom allocators with shared helper functions. Use separate XArray to store IOASIDs per allocator. Took advice from Eric Auger to have default allocator use the generic allocator structure. Combined into one patch in that the default allocator is just "another" allocator now. Can be built as a module in case of driver use without IOMMU.
- Extended bind guest PASID data to support SMMU and non-identity guest to host PASID mapping https://lkml.org/lkml/2019/5/21/802
- Rebased on Jean's sva/api common tree, new patches starts with [PATCH v4 10/22]
- V3
- Addressed thorough review comments from Eric Auger (Thank you!)
- Moved IOASID allocator from driver core to IOMMU code per suggestion by Christoph Hellwig
(https://lkml.org/lkml/2019/4/26/462)
- Rebased on top of Jean's SVA API branch and Eric's v7[1]
(git://linux-arm.org/linux-jpb.git sva/api)
- All IOMMU APIs are unmodified (except the new bind guest PASID call in patch 9/16)
- V2
- Rebased on Joerg's IOMMU x86/vt-d branch v5.1-rc4
- Integrated with Eric Auger's new v7 series for common APIs
(https://github.com/eauger/linux/tree/v5.1-rc3-2stage-v7)
- Addressed review comments from Andy Shevchenko and Alex Williamson on IOASID custom allocator.
- Support multiple custom IOASID allocators (vIOMMUs) and dynamic registration.
Jacob Pan (7):
iommu/vt-d: Move domain helper to header iommu/vt-d: Use a helper function to skip agaw for SL iommu/vt-d: Add nested translation helper function iommu/vt-d: Add bind guest PASID support iommu/vt-d: Support flushing more translation cache types iommu/vt-d: Add svm/sva invalidate function iommu/vt-d: Add custom allocator for IOASID
Lu Baolu (1):
iommu/vt-d: Enlightened PASID allocation
drivers/iommu/dmar.c | 40 ++++++ drivers/iommu/intel-iommu.c | 291 +++++++++++++++++++++++++++++++++++++++----- drivers/iommu/intel-pasid.c | 266 +++++++++++++++++++++++++++++++++++++--- drivers/iommu/intel-pasid.h | 23 +++- drivers/iommu/intel-svm.c | 203 ++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 69 ++++++++++- include/linux/intel-svm.h | 12 ++ include/uapi/linux/iommu.h | 5 + 8 files changed, 858 insertions(+), 51 deletions(-)
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v13 1/8 iommu/vt-d: Move domain helper to header
Move domain helper to header to be used by SVA code.
--- drivers/iommu/intel-iommu.c | 6 ------ include/linux/intel-iommu.h | 6 ++++++ 2 files changed, 6 insertions(+), 6 deletions(-)
drivers/iommu/intel-iommu.c +0 -6
@@ -446,12 +446,6 @@ static void init_translation_status(struct intel_iommu *iommu) iommu->flags |=VTD_FLAG_TRANS_PRE_ENABLED ; } -/* Convert generic 'struct iommu_domain to private struct dmar_domain */ -static struct dmar_domain *to_dmar_domain (struct iommu_domain *dom) -{ - return container_of(dom, struct dmar_domain, domain); -} -static int __init intel_iommu_setup (char *str) {if (!str)
include/linux/intel-iommu.h +6 -0
@@ -595,6 +595,12 @@ static inline void __iommu_flush_cache(clflush_cache_range (addr, size); } +/* Convert generic struct iommu_domain to private struct dmar_domain */ +static inline struct dmar_domain *to_dmar_domain (struct iommu_domain *dom) +{ + return container_of(dom, struct dmar_domain, domain); +} +/* * 0: readable * 1: writable
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v13 2/8 iommu/vt-d: Use a helper function to skip agaw for SL
In reply to Jacob PanAn Intel iommu domain uses 5-level page table by default. If the iommu that the domain tries to attach supports less page levels, the top level page tables should be skipped. Add a helper to do this so that it could be used in other places.
--- drivers/iommu/intel-pasid.c | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-)
drivers/iommu/intel-pasid.c +23 -10
@@ -500,6 +500,25 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu, }/* +* Skip top levels of page tables for iommu which has less agaw +* than default. Unnecessary for PT mode. +*/ +static inline int iommu_skip_agaw (struct dmar_domain *domain, +struct intel_iommu *iommu, +struct dma_pte **pgd) +{ +int agaw; + +for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { + *pgd =phys_to_virt (dma_pte_addr (*pgd)); +if (!dma_pte_present (*pgd)) +return -EINVAL ; + } + +return agaw; +} + +/* * Set up the scalable mode pasid entry for second only translation type. */ int intel_pasid_setup_second_level(struct intel_iommu *iommu,
@@ -522,17 +541,11 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,return -EINVAL ; } -/* -* Skip top levels of page tables for iommu which has less agaw -* than default. Unnecessary for PT mode. -*/ pgd = domain->pgd; -for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { - pgd =phys_to_virt (dma_pte_addr (pgd)); -if (!dma_pte_present (pgd)) { -dev_err (dev,"Invalid domain page table\n" ); -return -EINVAL ; - } + agaw =iommu_skip_agaw (domain, iommu, &pgd); +if (agaw <0 ) { +dev_err (dev,"Invalid domain page table\n" ); +return -EINVAL ; } pgd_val =virt_to_phys (pgd);
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v13 2/8] iommu/vt-d: Use a helper function to skip agaw for SL
Quoted 4 lines · Jacob Pan
An Intel iommu domain uses 5-level page table by default. If the iommu that the domain tries to attach supports less page levels, the top level page tables should be skipped. Add a helper to do this so that it could be used in other places.
Please use up all 73 chars for the commit log.
Re: [PATCH v13 2/8] iommu/vt-d: Use a helper function to skip agaw for SL
Quoted 7 lines · On Wed, 13 May 2020 22:51:32 -0700 Christoph Hellwig <[email protected]> wrote:
Quoted 4 lines · Jacob Pan
An Intel iommu domain uses 5-level page table by default. If the iommu that the domain tries to attach supports less page levels, the top level page tables should be skipped. Add a helper to do this so that it could be used in other places.
Please use up all 73 chars for the commit log.
Will do. thanks.
PATCH v13 3/8 iommu/vt-d: Add nested translation helper function
In reply to Jacob PanNested translation mode is supported in VT-d 3.0 Spec.CH 3.8. With PASID granular translation type set to 0x11b, translation result from the first level(FL) also subject to a second level(SL) page table translation. This mode is used for SVA virtualization, where FL performs guest virtual to guest physical translation and SL performs guest physical to host physical translation.
This patch adds a helper function for setting up nested translation where second level comes from a domain and first level comes from a guest PGD.
---
v13 Dropped MTS support
---
--- drivers/iommu/intel-iommu.c | 25 ------- drivers/iommu/intel-pasid.c | 173 +++++++++++++++++++++++++++++++++++++++++++- drivers/iommu/intel-pasid.h | 10 +++ include/linux/intel-iommu.h | 28 +++++++ include/uapi/linux/iommu.h | 5 ++ 5 files changed, 213 insertions(+), 28 deletions(-)
drivers/iommu/intel-iommu.c +0 -25
@@ -296,31 +296,6 @@ static inline void context_clear_entry(struct context_entry *context)static struct dmar_domain *si_domain;static int hw_pass_through =1 ; -/* si_domain contains mulitple devices */ -#define DOMAIN_FLAG_STATIC_IDENTITY BIT(0) - -/* -* This is a DMA domain allocated through the iommu domain allocation -* interface. But one or more devices belonging to this domain have -* been chosen to use a private domain. We should avoid to use the -* map/unmap/iova_to_phys APIs on it. -*/ -#define DOMAIN_FLAG_LOSE_CHILDREN BIT(1) - -/* -* When VT-d works in the scalable mode, it allows DMA translation to -* happen through either first level or second level page table. This -* bit marks that the DMA translation for the domain goes through the -* first level page table, otherwise, it goes through the second level. -*/ -#define DOMAIN_FLAG_USE_FIRST_LEVEL BIT(2) - -/* -* Domain represents a virtual machine which demands iommu nested -* translation mode support. -*/ -#define DOMAIN_FLAG_NESTING_MODE BIT(3) -#define for_each_domain_iommu (idx, domain) \ for (idx = 0; idx < g_num_of_iommus; idx++) \ if (domain->iommu_refcnt[idx])
drivers/iommu/intel-pasid.c +170 -3
200 lines not on this page · raw message
drivers/iommu/intel-pasid.h +10 -0
@@ -36,6 +36,7 @@* to vmalloc or even module mappings. */ #define PASID_FLAG_SUPERVISOR_MODE BIT(0) +#define PASID_FLAG_NESTED BIT(1)/* * The PASID_FLAG_FL5LP flag Indicates using 5-level paging for first-
@@ -51,6 +52,11 @@ struct pasid_entry {u64 val[8 ]; }; +#define PASID_ENTRY_PGTT_FL_ONLY (1) +#define PASID_ENTRY_PGTT_SL_ONLY (2) +#define PASID_ENTRY_PGTT_NESTED (3) +#define PASID_ENTRY_PGTT_PT (4) +/* The representative of a PASID table */ struct pasid_table {void *table;/* pasid table pointer */
@@ -99,6 +105,10 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,int intel_pasid_setup_pass_through (struct intel_iommu *iommu,struct dmar_domain *domain,struct device *dev,int pasid); +int intel_pasid_setup_nested (struct intel_iommu *iommu, +struct device *dev,pgd_t *pgd,int pasid, +struct iommu_gpasid_bind_data_vtd *pasid_data, +struct dmar_domain *domain,int addr_width);void intel_pasid_tear_down_entry (struct intel_iommu *iommu,struct device *dev,int pasid);
include/linux/intel-iommu.h +28 -0
@@ -42,6 +42,9 @@#define DMA_FL_PTE_PRESENT BIT_ULL(0)#define DMA_FL_PTE_XD BIT_ULL(63) +#define ADDR_WIDTH_5LEVEL (57) +#define ADDR_WIDTH_4LEVEL (48) +#define CONTEXT_TT_MULTI_LEVEL 0#define CONTEXT_TT_DEV_IOTLB 1#define CONTEXT_TT_PASS_THROUGH 2
@@ -480,6 +483,31 @@ struct context_entry {u64 hi; }; +/* si_domain contains mulitple devices */ +#define DOMAIN_FLAG_STATIC_IDENTITY BIT(0) + +/* +* This is a DMA domain allocated through the iommu domain allocation +* interface. But one or more devices belonging to this domain have +* been chosen to use a private domain. We should avoid to use the +* map/unmap/iova_to_phys APIs on it. +*/ +#define DOMAIN_FLAG_LOSE_CHILDREN BIT(1) + +/* +* When VT-d works in the scalable mode, it allows DMA translation to +* happen through either first level or second level page table. This +* bit marks that the DMA translation for the domain goes through the +* first level page table, otherwise, it goes through the second level. +*/ +#define DOMAIN_FLAG_USE_FIRST_LEVEL BIT(2) + +/* +* Domain represents a virtual machine which demands iommu nested +* translation mode support. +*/ +#define DOMAIN_FLAG_NESTING_MODE BIT(3) +struct dmar_domain {int nid;/* node id */
include/uapi/linux/iommu.h +5 -0
@@ -285,6 +285,11 @@ struct iommu_gpasid_bind_data_vtd {__u32 emt; }; +#define IOMMU_SVA_VTD_GPASID_MTS_MASK (IOMMU_SVA_VTD_GPASID_CD | \ + IOMMU_SVA_VTD_GPASID_EMTE | \ + IOMMU_SVA_VTD_GPASID_PCD | \ + IOMMU_SVA_VTD_GPASID_PWT) +/** * struct iommu_gpasid_bind_data - Information about device and guest PASID binding * @version: Version of this data structure
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v13 3/8] iommu/vt-d: Add nested translation helper function
Quoted 4 lines
drivers/iommu/intel-pasid.c +4
-0+* 1. CPU vs. IOMMU +* 2. Guest vs. Host. +*/ +switch (addr_width) {
drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 731
Quoted 10 lines
drivers/iommu/intel-pasid.c +10
-0+#ifdef CONFIG_X86 +case ADDR_WIDTH_5LEVEL : +if (cpu_feature_enabled (X86_FEATURE_LA57 ) && +cap_5lp_support (iommu->cap)) { +pasid_set_flpm (pte,1 ); + }else { +dev_err_ratelimited (dev,"5-level paging not supported\n" ); +return -EINVAL ; + } +break ;
The normal style would be to handle the early error return first:
if (!cpu_feature_enabled(X86_FEATURE_LA57) ||
!cap_5lp_support(iommu->cap)) {
dev_err_ratelimited(dev,
"5-level paging not supported\n");
return -EINVAL;
}
pasid_set_flpm(pte, 1);
break;Re: [PATCH v13 3/8] iommu/vt-d: Add nested translation helper function
Quoted 18 lines · On Wed, 13 May 2020 22:54:24 -0700 Christoph Hellwig <[email protected]> wrote:
Quoted 12 lines
drivers/iommu/intel-pasid.c +11
-0+* 1. CPU vs. IOMMU +* 2. Guest vs. Host. +*/ + switch (addr_width) { +#ifdef CONFIG_X86 + case ADDR_WIDTH_5LEVEL: + if (cpu_feature_enabled(X86_FEATURE_LA57) && + cap_5lp_support(iommu->cap)) { + pasid_set_flpm(pte, 1); + } else { + dev_err_ratelimited(dev, "5-level pagingnot supported\n");
drivers/iommu/intel-pasid.c int intel_pasid_setup_pass_through(struct intel_iommu *iommu, · line 731
Quoted 3 lines
drivers/iommu/intel-pasid.c +3
-0+return -EINVAL ; + } +break ;The normal style would be to handle the early error return first:
Quoted 10 lines
if (!cpu_feature_enabled(X86_FEATURE_LA57) ||
!cap_5lp_support(iommu->cap)) {
dev_err_ratelimited(dev,
"5-level paging not supported\n");
return -EINVAL;
}
pasid_set_flpm(pte, 1);
break;Sounds good. I was thinking in positive logic.
thanks
PATCH v13 4/8 iommu/vt-d: Add bind guest PASID support
In reply to Jacob PanWhen supporting guest SVA with emulated IOMMU, the guest PASID table is shadowed in VMM. Updates to guest vIOMMU PASID table will result in PASID cache flush which will be passed down to the host as bind guest PASID calls.
For the SL page tables, it will be harvested from device's default domain (request w/o PASID), or aux domain in case of mediated device.
.-------------. .---------------------------.
| vIOMMU | | Guest process CR3, FL only|
| | '---------------------------' .----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest
------| Shadow |--------------------------|--------
v v v
Host
.-------------. .----------------------.
| pIOMMU | | Bind FL for GVA-GPA |
| | '----------------------'
.----------------/ |
| PASID Entry | V (Nested xlate)
'----------------\.------------------------------.
| | |SL for GPA-HPA, default domain|
| | '------------------------------'
'-------------'Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tables
---
v13 Only allow multiple bind if device has aux domain
---
--- drivers/iommu/intel-iommu.c | 4 + drivers/iommu/intel-svm.c | 203 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 6 +- include/linux/intel-svm.h | 12 +++ 4 files changed, 224 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +4 -0
@@ -6179,6 +6179,10 @@ const struct iommu_ops intel_iommu_ops = { .dev_disable_feat = intel_iommu_dev_disable_feat, .is_attach_deferred = intel_iommu_is_attach_deferred, .pgsize_bitmap =INTEL_IOMMU_PGSIZES , +#ifdef CONFIG_INTEL_IOMMU_SVM + .sva_bind_gpasid = intel_svm_bind_gpasid, + .sva_unbind_gpasid = intel_svm_unbind_gpasid, +#endif };static void quirk_iommu_igfx (struct pci_dev *dev)
drivers/iommu/intel-svm.c +203 -0
209 lines not on this page · raw message
include/linux/intel-iommu.h +5 -1
@@ -706,7 +706,9 @@ struct dmar_domain *find_domain(struct device *dev);extern void intel_svm_check (struct intel_iommu *iommu);extern int intel_svm_enable_prq (struct intel_iommu *iommu);extern int intel_svm_finish_prq (struct intel_iommu *iommu); - +int intel_svm_bind_gpasid (struct iommu_domain *domain,struct device *dev, +struct iommu_gpasid_bind_data *data); +int intel_svm_unbind_gpasid (struct device *dev,int pasid);struct svm_dev_ops ;struct intel_svm_dev {
@@ -723,9 +725,11 @@ struct intel_svm_dev {struct intel_svm {struct mmu_notifier notifier;struct mm_struct *mm; +struct intel_iommu *iommu;int flags;int pasid; +int gpasid;/* In case that guest PASID is different from host PASID */ struct list_head devs;struct list_head list; };
include/linux/intel-svm.h +12 -0
@@ -44,6 +44,18 @@ struct svm_dev_ops {* do such IOTLB flushes automatically. */ #define SVM_FLAG_SUPERVISOR_MODE (1<<1) +/* +* The SVM_FLAG_GUEST_MODE flag is used when a PASID bind is for guest +* processes. Compared to the host bind, the primary differences are: +* 1. mm life cycle management +* 2. fault reporting +*/ +#define SVM_FLAG_GUEST_MODE (1<<2) +/* +* The SVM_FLAG_GUEST_PASID flag is used when a guest has its own PASID space, +* which requires guest and host PASID translation at both directions. +*/ +#define SVM_FLAG_GUEST_PASID (1<<3)#ifdef CONFIG_INTEL_IOMMU_SVM
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v13 4/8] iommu/vt-d: Add bind guest PASID support
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 251
Quoted 7 lines
drivers/iommu/intel-svm.c +7
-0+if (dev_is_pci (dev)) { +/* VT-d supports devices with full 20 bit PASIDs only */ +if (pci_max_pasids (to_pci_dev (dev)) !=PASID_MAX ) +return -EINVAL ; + }else { +return -ENOTSUPP ; + }
This looks strange. Why not:
if (!dev_is_pci(dev)) {
return -ENOTSUPP;
/* VT-d supports devices with full 20 bit PASIDs only */
if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
return -EINVAL;drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 291
Quoted 13 lines
drivers/iommu/intel-svm.c +13
-0+for_each_svm_dev (sdev, svm, dev) { +/* +* For devices with aux domains, we should allow multiple +* bind calls with the same PASID and pdev. +*/ +if (iommu_dev_feature_enabled (dev,IOMMU_DEV_FEAT_AUX )) { + sdev->users++; + }else { +dev_warn_ratelimited (dev,"Already bound with PASID %u\n" , + svm->pasid); + ret = -EBUSY ; + } + goto out;
Is this intentionally a for loop that jumps out of the loop after the first device?
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 346
Quoted 13 lines
drivers/iommu/intel-svm.c +13
-0+/* +* PASID table is per device for better security. Therefore, for +* each bind of a new device even with an existing PASID, we need to +* call the nested mode setup function here. +*/ +spin_lock (&iommu->lock); + ret =intel_pasid_setup_nested (iommu, + dev, + (pgd_t *)data->gpgd, + data->hpasid, + &data->vtd, + dmar_domain, + data->addr_width);
Why not:
et = intel_pasid_setup_nested(iommu, dev, (pgd_t *)data->gpgd, data->hpasid, &data->vtd, dmar_domain, data->addr_width);
?
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 397
Quoted 2 lines
drivers/iommu/intel-svm.c +2
-0+for_each_svm_dev (sdev, svm, dev) { + ret =0 ;
...
Review
Quoted 2 lines
(quoted diff) +2
-0+ break; + }
Same only looks at the first device style. Why dos it only care about the first device? That needs at least a comment, and probably a first_svm_dev or so heper to make it explicit.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v13 4/8] iommu/vt-d: Add bind guest PASID support
Hi Christoph,
Thanks a lot for the reviews, comments below.
Jacob
Quoted 11 lines · On Wed, 13 May 2020 22:59:30 -0700 Christoph Hellwig <[email protected]> wrote:
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 251
Quoted 8 lines
(quoted diff) +2
-0+ if (dev_is_pci(dev)) { + /* VT-d supports devices with full 20 bit PASIDsonly */
drivers/iommu/intel-svm.c +5
-0+if (pci_max_pasids (to_pci_dev (dev)) !=PASID_MAX ) +return -EINVAL ; + }else { +return -ENOTSUPP ; + }This looks strange. Why not:
Quoted 7 lines
if (!dev_is_pci(dev)) {
return -ENOTSUPP;
/* VT-d supports devices with full 20 bit PASIDs only */
if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX)
return -EINVAL;That is better, will do.
Quoted 11 lines
Quoted 11 lines
drivers/iommu/intel-svm.c +3
-0+for_each_svm_dev (sdev, svm, dev) { +/* +* For devices with aux domains, we should allow multiple
drivers/iommu/intel-svm.c +3
-0+* bind calls with the same PASID and pdev. +*/ + if (iommu_dev_feature_enabled(dev,
IOMMU_DEV_FEAT_AUX)) {drivers/iommu/intel-svm.c +3
-0+ sdev->users++; + }else { + dev_warn_ratelimited(dev, "Already
Quoted 9 lines
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 291
Quoted 5 lines
bound with PASID %u\n",
drivers/iommu/intel-svm.c +4
-0+ svm->pasid); + ret = -EBUSY ; + } + goto out;Is this intentionally a for loop that jumps out of the loop after the first device?
The name is confusing, it is not a loop. I will change it to find_svm_dev() and comments like this?
/*
* Find the matching device in a given SVM. The bind code ensures that
* each device can only be added to the SVM list once.
*/
#define find_svm_dev(sdev, svm, d) \
list_for_each_entry((sdev), &(svm)->devs, list) \
if ((d) != (sdev)->dev) {} elseQuoted 18 lines
Quoted 5 lines
(quoted diff) +2
-0+ /* + * PASID table is per device for better security.Therefore, for
(quoted diff) +1
-0+ * each bind of a new device even with an existing PASID,we need to
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 346
Quoted 10 lines
drivers/iommu/intel-svm.c +10
-0+* call the nested mode setup function here. +*/ +spin_lock (&iommu->lock); + ret =intel_pasid_setup_nested (iommu, + dev, + (pgd_t *)data->gpgd, + data->hpasid, + &data->vtd, + dmar_domain, + data->addr_width);Why not:
Quoted 6 lines
et = intel_pasid_setup_nested(iommu, dev, (pgd_t *)data->gpgd, data->hpasid, &data->vtd, dmar_domain, data->addr_width);?
I thought we want to align the parentheses? Either way is fine. Baolu?
Quoted 11 lines
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 397
Quoted 2 lines
drivers/iommu/intel-svm.c +2
-0+for_each_svm_dev (sdev, svm, dev) { + ret =0 ;...Review
Quoted 2 lines
(quoted diff) +2
-0+ break; + }Same only looks at the first device style. Why dos it only care about the first device? That needs at least a comment, and probably a
first_svm_devor so heper to make it explicit.
Yes, same as above. change to find_svm_dev() since there should be at most one matching device in the svm list.
Re: [PATCH v13 4/8] iommu/vt-d: Add bind guest PASID support
Hi Jacob,
Quoted 17 lines · Jacob Pan
Quoted 17 lines
Quoted 5 lines
(quoted diff) +2
-0+ /* + * PASID table is per device for better security.Therefore, for
(quoted diff) +1
-0+ * each bind of a new device even with an existing PASID,we need to
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 346
Quoted 10 lines
drivers/iommu/intel-svm.c +10
-0+* call the nested mode setup function here. +*/ +spin_lock (&iommu->lock); + ret =intel_pasid_setup_nested (iommu, + dev, + (pgd_t *)data->gpgd, + data->hpasid, + &data->vtd, + dmar_domain, + data->addr_width);Why not:
Quoted 9 lines
Quoted 6 lines
et = intel_pasid_setup_nested(iommu, dev, (pgd_t *)data->gpgd, data->hpasid, &data->vtd, dmar_domain, data->addr_width);?
I thought we want to align the parentheses? Either way is fine. Baolu?
Let's keep the code style consistent in this file.
Best regards,
baolu
Re: [PATCH v13 4/8] iommu/vt-d: Add bind guest PASID support
In reply to Jacob PanHi Jacob,
Quoted 50 lines · Jacob Pan
Hi Christoph,
Thanks a lot for the reviews, comments below.
Jacob
Quoted 10 lines · On Wed, 13 May 2020 22:59:30 -0700 Christoph Hellwig<[email protected]> wrote:
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 251
Quoted 8 lines
(quoted diff) +2
-0+ if (dev_is_pci(dev)) { + /* VT-d supports devices with full 20 bit PASIDsonly */
drivers/iommu/intel-svm.c +5
-0+if (pci_max_pasids (to_pci_dev (dev)) !=PASID_MAX ) +return -EINVAL ; + }else { +return -ENOTSUPP ; + }This looks strange. Why not:
Quoted 7 lines
if (!dev_is_pci(dev)) { return -ENOTSUPP; /* VT-d supports devices with full 20 bit PASIDs only */ if (pci_max_pasids(to_pci_dev(dev)) != PASID_MAX) return -EINVAL;That is better, will do.
Quoted 10 lines
Quoted 10 lines
drivers/iommu/intel-svm.c +3
-0+for_each_svm_dev (sdev, svm, dev) { +/* +* For devices with aux domains, we should allow multiple
drivers/iommu/intel-svm.c +3
-0+* bind calls with the same PASID and pdev. +*/ + if (iommu_dev_feature_enabled(dev,
IOMMU_DEV_FEAT_AUX)) {(quoted diff) +2
-0+ sdev->users++; + } else {Quoted 9 lines
drivers/iommu/intel-svm.c static LIST_HEAD(global_svm_list); · line 291
Quoted 6 lines
(quoted diff) +1
-0+ dev_warn_ratelimited(dev, "Alreadybound with PASID %u\n",
drivers/iommu/intel-svm.c +4
-0+ svm->pasid); + ret = -EBUSY ; + } + goto out;Is this intentionally a for loop that jumps out of the loop after the first device?
The name is confusing, it is not a loop. I will change it to
find_svm_dev()and comments like this?
Quoted 8 lines
/*
* Find the matching device in a given SVM. The bind code ensures that
* each device can only be added to the SVM list once.
*/#define find_svm_dev(sdev, svm, d) \ list_for_each_entry((sdev), &(svm)->devs, list) \ if ((d) != (sdev)->dev) {} else
The for_each_svm_dev() is not added by this series and is also used by other functions. How about changing it in a separated patch?
Best regards,
baolu
PATCH v13 5/8 iommu/vt-d: Support flushing more translation cache types
In reply to Jacob PanWhen Shared Virtual Memory is exposed to a guest via vIOMMU, scalable IOTLB invalidation may be passed down from outside IOMMU subsystems. This patch adds invalidation functions that can be used for additional translation cache types.
--- drivers/iommu/dmar.c | 39 +++++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.c | 3 ++- include/linux/intel-iommu.h | 21 +++++++++++++++++---- 3 files changed, 58 insertions(+), 5 deletions(-)
drivers/iommu/dmar.c +39 -0
@@ -1421,6 +1421,45 @@ void qi_flush_piotlb(struct intel_iommu *iommu, u16 did, u32 pasid, u64 addr,qi_submit_sync (&desc, iommu); } +/* PASID-based device IOTLB Invalidate */ +void qi_flush_dev_iotlb_pasid (struct intel_iommu *iommu,u16 sid,u16 pfsid, +u32 pasid,u16 qdep,u64 addr, +unsigned int size_order,u64 granu) +{ +unsigned long mask =1UL << (VTD_PAGE_SHIFT + size_order -1 ); +struct qi_desc desc = {.qw1 =0 , .qw2 =0 , .qw3 =0 }; + + desc.qw0 =QI_DEV_EIOTLB_PASID (pasid) |QI_DEV_EIOTLB_SID (sid) | +QI_DEV_EIOTLB_QDEP (qdep) |QI_DEIOTLB_TYPE | +QI_DEV_IOTLB_PFSID (pfsid); + desc.qw1 =QI_DEV_EIOTLB_GLOB (granu); + +/* +* If S bit is 0, we only flush a single page. If S bit is set, +* The least significant zero bit indicates the invalidation address +* range. VT-d spec 6.5.2.6. +* e.g. address bit 12[0] indicates 8KB, 13[0] indicates 16KB. +* size order = 0 is PAGE_SIZE 4KB +* Max Invs Pending (MIP) is set to 0 for now until we have DIT in +* ECAP. +*/ + desc.qw1 |= addr & ~mask; +if (size_order) + desc.qw1 |=QI_DEV_EIOTLB_SIZE ; + +qi_submit_sync (&desc, iommu); +} + +void qi_flush_pasid_cache (struct intel_iommu *iommu,u16 did, +u64 granu,int pasid) +{ +struct qi_desc desc = {.qw1 =0 , .qw2 =0 , .qw3 =0 }; + + desc.qw0 =QI_PC_PASID (pasid) |QI_PC_DID (did) | +QI_PC_GRAN (granu) |QI_PC_TYPE ; +qi_submit_sync (&desc, iommu); +} +/* * Disable Queued Invalidation interface. */
drivers/iommu/intel-pasid.c +2 -1
@@ -375,7 +375,8 @@ pasid_cache_invalidation_with_pasid(struct intel_iommu *iommu, {struct qi_desc desc; - desc.qw0 =QI_PC_DID (did) |QI_PC_PASID_SEL |QI_PC_PASID (pasid); + desc.qw0 =QI_PC_DID (did) |QI_PC_GRAN (QI_PC_PASID_SEL ) | +QI_PC_PASID (pasid) |QI_PC_TYPE ; desc.qw1 =0 ; desc.qw2 =0 ; desc.qw3 =0 ;
include/linux/intel-iommu.h +17 -4
@@ -334,7 +334,7 @@ enum {#define QI_IOTLB_GRAN (gran) (((u64)gran) >> (DMA_TLB_FLUSH_GRANU_OFFSET-4))#define QI_IOTLB_ADDR (addr) (((u64)addr) & VTD_PAGE_MASK)#define QI_IOTLB_IH (ih) (((u64)ih) << 6) -#define QI_IOTLB_AM (am) (((u8)am)) +#define QI_IOTLB_AM (am) (((u8)am) & 0x3f)#define QI_CC_FM (fm) (((u64)fm) << 48)#define QI_CC_SID (sid) (((u64)sid) << 32)
@@ -353,16 +353,21 @@ enum {#define QI_PC_DID (did) (((u64)did) << 16)#define QI_PC_GRAN (gran) (((u64)gran) << 4) -#define QI_PC_ALL_PASIDS (QI_PC_TYPE | QI_PC_GRAN(0)) -#define QI_PC_PASID_SEL (QI_PC_TYPE | QI_PC_GRAN(1)) +/* PASID cache invalidation granu */ +#define QI_PC_ALL_PASIDS 0 +#define QI_PC_PASID_SEL 1#define QI_EIOTLB_ADDR (addr) ((u64)(addr) & VTD_PAGE_MASK)#define QI_EIOTLB_IH (ih) (((u64)ih) << 6) -#define QI_EIOTLB_AM (am) (((u64)am)) +#define QI_EIOTLB_AM (am) (((u64)am) & 0x3f)#define QI_EIOTLB_PASID (pasid) (((u64)pasid) << 32)#define QI_EIOTLB_DID (did) (((u64)did) << 16)#define QI_EIOTLB_GRAN (gran) (((u64)gran) << 4) +/* QI Dev-IOTLB inv granu */ +#define QI_DEV_IOTLB_GRAN_ALL 1 +#define QI_DEV_IOTLB_GRAN_PASID_SEL 0 +#define QI_DEV_EIOTLB_ADDR (a) ((u64)(a) & VTD_PAGE_MASK)#define QI_DEV_EIOTLB_SIZE (((u64)1) << 11)#define QI_DEV_EIOTLB_GLOB (g) ((u64)g)
@@ -687,8 +692,16 @@ extern void qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr, unsigned int size_order, u64 type);extern void qi_flush_dev_iotlb (struct intel_iommu *iommu,u16 sid,u16 pfsid,u16 qdep,u64 addr,unsigned mask); +void qi_flush_piotlb (struct intel_iommu *iommu,u16 did,u32 pasid,u64 addr,unsigned long npages,bool ih); + +void qi_flush_dev_iotlb_pasid (struct intel_iommu *iommu,u16 sid,u16 pfsid, +u32 pasid,u16 qdep,u64 addr, +unsigned int size_order,u64 granu); +void qi_flush_pasid_cache (struct intel_iommu *iommu,u16 did,u64 granu, +int pasid); +extern int qi_submit_sync (struct qi_desc *desc,struct intel_iommu *iommu);extern int dmar_ir_support (void );
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v13 6/8 iommu/vt-d: Add svm/sva invalidate function
In reply to Jacob PanWhen Shared Virtual Address (SVA) is enabled for a guest OS via vIOMMU, we need to provide invalidation support at IOMMU API and driver level. This patch adds Intel VT-d specific function to implement iommu passdown invalidate API for shared virtual address.
The use case is for supporting caching structure invalidation of assigned SVM capable devices. Emulated IOMMU exposes queue invalidation capability and passes down all descriptors from the guest to the physical IOMMU.
The assumption is that guest to host device ID mapping should be resolved prior to calling IOMMU driver. Based on the device handle, host IOMMU driver can replace certain fields before submit to the invalidation queue.
--- drivers/iommu/intel-iommu.c | 171 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 171 insertions(+)
drivers/iommu/intel-iommu.c +171 -0
@@ -5595,6 +5595,176 @@ static void intel_iommu_aux_detach_device(struct iommu_domain *domain,aux_domain_remove_dev (to_dmar_domain (domain), dev); } +/* +* 2D array for converting and sanitizing IOMMU generic TLB granularity to +* VT-d granularity. Invalidation is typically included in the unmap operation +* as a result of DMA or VFIO unmap. However, for assigned devices guest +* owns the first level page tables. Invalidations of translation caches in the +* guest are trapped and passed down to the host. +* +* vIOMMU in the guest will only expose first level page tables, therefore +* we do not support IOTLB granularity for request without PASID (second level). +* +* For example, to find the VT-d granularity encoding for IOTLB +* type and page selective granularity within PASID: +* X: indexed by iommu cache type +* Y: indexed by enum iommu_inv_granularity +* [IOMMU_CACHE_INV_TYPE_IOTLB][IOMMU_INV_GRANU_ADDR] +*/ + +const static int +inv_type_granu_table[IOMMU_CACHE_INV_TYPE_NR ][IOMMU_INV_GRANU_NR ] = { +/* +* PASID based IOTLB invalidation: PASID selective (per PASID), +* page selective (address granularity) +*/ + {-EINVAL ,QI_GRAN_NONG_PASID ,QI_GRAN_PSI_PASID }, +/* PASID based dev TLBs */ + {-EINVAL , -EINVAL ,QI_DEV_IOTLB_GRAN_PASID_SEL }, +/* PASID cache */ + {-EINVAL , -EINVAL , -EINVAL } +}; + +static inline int to_vtd_granularity (int type,int granu) +{ +return inv_type_granu_table[type][granu]; +} + +static inline u64 to_vtd_size (u64 granu_size,u64 nr_granules) +{ +u64 nr_pages = (granu_size * nr_granules) >>VTD_PAGE_SHIFT ; + +/* VT-d size is encoded as 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc. +* IOMMU cache invalidate API passes granu_size in bytes, and number of +* granu size in contiguous memory. +*/ +return order_base_2 (nr_pages); +} + +#ifdef CONFIG_INTEL_IOMMU_SVM +static int +intel_iommu_sva_invalidate (struct iommu_domain *domain,struct device *dev, +struct iommu_cache_invalidate_info *inv_info) +{ +struct dmar_domain *dmar_domain =to_dmar_domain (domain); +struct device_domain_info *info; +struct intel_iommu *iommu; +unsigned long flags; +int cache_type; +u8 bus, devfn; +u16 did, sid; +int ret =0 ; +u64 size =0 ; + +if (!inv_info || !dmar_domain || + inv_info->version !=IOMMU_CACHE_INVALIDATE_INFO_VERSION_1 ) +return -EINVAL ; + +if (!dev || !dev_is_pci (dev)) +return -ENODEV ; + + iommu =device_to_iommu (dev, &bus, &devfn); +if (!iommu) +return -ENODEV ; + +if (!(dmar_domain->flags &DOMAIN_FLAG_NESTING_MODE )) +return -EINVAL ; + +spin_lock_irqsave (&device_domain_lock, flags); +spin_lock (&iommu->lock); + info = dev->archdata.iommu; +if (!info) { + ret = -EINVAL ; + goto out_unlock; + } + did = dmar_domain->iommu_did[iommu->seq_id]; + sid =PCI_DEVID (bus, devfn); + +/* Size is only valid in address selective invalidation */ +if (inv_info->granularity !=IOMMU_INV_GRANU_PASID ) + size =to_vtd_size (inv_info->addr_info.granule_size, + inv_info->addr_info.nb_granules); + +for_each_set_bit (cache_type, + (unsigned long *)&inv_info->cache, +IOMMU_CACHE_INV_TYPE_NR ) { +int granu =0 ; +u64 pasid =0 ; + + granu =to_vtd_granularity (cache_type, inv_info->granularity); +if (granu == -EINVAL ) { +pr_err_ratelimited ("Invalid cache type and granu combination %d/%d\n" , + cache_type, inv_info->granularity); +break ; + } + +/* +* PASID is stored in different locations based on the +* granularity. +*/ +if (inv_info->granularity ==IOMMU_INV_GRANU_PASID && + (inv_info->pasid_info.flags &IOMMU_INV_PASID_FLAGS_PASID )) + pasid = inv_info->pasid_info.pasid; +else if (inv_info->granularity ==IOMMU_INV_GRANU_ADDR && + (inv_info->addr_info.flags &IOMMU_INV_ADDR_FLAGS_PASID )) + pasid = inv_info->addr_info.pasid; + +switch (BIT (cache_type)) { +case IOMMU_CACHE_INV_TYPE_IOTLB : +if (inv_info->granularity ==IOMMU_INV_GRANU_ADDR && + size && + (inv_info->addr_info.addr & ((BIT (VTD_PAGE_SHIFT + size)) -1 ))) { +pr_err_ratelimited ("Address out of range, 0x%llx, size order %llu\n" , + inv_info->addr_info.addr, size); + ret = -ERANGE ; + goto out_unlock; + } + +/* +* If granu is PASID-selective, address is ignored. +* We use npages = -1 to indicate that. +*/ +qi_flush_piotlb (iommu, did, pasid, +mm_to_dma_pfn (inv_info->addr_info.addr), + (granu ==QI_GRAN_NONG_PASID ) ?-1 :1 << size, + inv_info->addr_info.flags &IOMMU_INV_ADDR_FLAGS_LEAF ); + +/* +* Always flush device IOTLB if ATS is enabled. vIOMMU +* in the guest may assume IOTLB flush is inclusive, +* which is more efficient. +*/ +if (info->ats_enabled) +qi_flush_dev_iotlb_pasid (iommu, sid, + info->pfsid, pasid, + info->ats_qdep, + inv_info->addr_info.addr, + size, granu); +break ; +case IOMMU_CACHE_INV_TYPE_DEV_IOTLB : +if (info->ats_enabled) +qi_flush_dev_iotlb_pasid (iommu, sid, + info->pfsid, pasid, + info->ats_qdep, + inv_info->addr_info.addr, + size, granu); +else +pr_warn_ratelimited ("Passdown device IOTLB flush w/o ATS!\n" ); +break ; +default : +dev_err_ratelimited (dev,"Unsupported IOMMU invalidation type %d\n" , + cache_type); + ret = -EINVAL ; + } + } +out_unlock: +spin_unlock (&iommu->lock); +spin_unlock_irqrestore (&device_domain_lock, flags); + +return ret; +} +#endif +static int intel_iommu_map (struct iommu_domain *domain,unsigned long iova,phys_addr_t hpa,size_t size,int iommu_prot,gfp_t gfp)
@@ -6180,6 +6350,7 @@ const struct iommu_ops intel_iommu_ops = { .is_attach_deferred = intel_iommu_is_attach_deferred, .pgsize_bitmap =INTEL_IOMMU_PGSIZES , #ifdef CONFIG_INTEL_IOMMU_SVM + .cache_invalidate = intel_iommu_sva_invalidate, .sva_bind_gpasid = intel_svm_bind_gpasid, .sva_unbind_gpasid = intel_svm_unbind_gpasid, #endif
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v13 7/8 iommu/vt-d: Enlightened PASID allocation
In reply to Jacob PanFrom: Lu Baolu <[email protected]>
Enabling IOMMU in a guest requires communication with the host driver for certain aspects. Use of PASID ID to enable Shared Virtual Addressing (SVA) requires managing PASID's in the host. VT-d 3.0 spec provides a Virtual Command Register (VCMD) to facilitate this. Writes to this register in the guest are trapped by vIOMMU which proxies the call to the host driver.
This virtual command interface consists of a capability register, a virtual command register, and a virtual response register. Refer to section 10.4.42, 10.4.43, 10.4.44 for more information.
This patch adds the enlightened PASID allocation/free interfaces via the virtual command interface.
--- drivers/iommu/intel-pasid.c | 57 +++++++++++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.h | 13 ++++++++++- include/linux/intel-iommu.h | 1 + 3 files changed, 70 insertions(+), 1 deletion(-)
drivers/iommu/intel-pasid.c +57 -0
@@ -27,6 +27,63 @@static DEFINE_SPINLOCK (pasid_lock);u32 intel_pasid_max_id =PASID_MAX ; +int vcmd_alloc_pasid (struct intel_iommu *iommu,unsigned int *pasid) +{ +unsigned long flags; +u8 status_code; +int ret =0 ; +u64 res; + +raw_spin_lock_irqsave (&iommu->register_lock, flags); +dmar_writeq (iommu->reg +DMAR_VCMD_REG ,VCMD_CMD_ALLOC ); +IOMMU_WAIT_OP (iommu,DMAR_VCRSP_REG , dmar_readq, + !(res &VCMD_VRSP_IP ), res); +raw_spin_unlock_irqrestore (&iommu->register_lock, flags); + + status_code =VCMD_VRSP_SC (res); +switch (status_code) { +case VCMD_VRSP_SC_SUCCESS : + *pasid =VCMD_VRSP_RESULT_PASID (res); +break ; +case VCMD_VRSP_SC_NO_PASID_AVAIL : +pr_info ("IOMMU: %s: No PASID available\n" , iommu->name); + ret = -ENOSPC ; +break ; +default : + ret = -ENODEV ; +pr_warn ("IOMMU: %s: Unexpected error code %d\n" , + iommu->name, status_code); + } + +return ret; +} + +void vcmd_free_pasid (struct intel_iommu *iommu,unsigned int pasid) +{ +unsigned long flags; +u8 status_code; +u64 res; + +raw_spin_lock_irqsave (&iommu->register_lock, flags); +dmar_writeq (iommu->reg +DMAR_VCMD_REG , +VCMD_CMD_OPERAND (pasid) |VCMD_CMD_FREE ); +IOMMU_WAIT_OP (iommu,DMAR_VCRSP_REG , dmar_readq, + !(res &VCMD_VRSP_IP ), res); +raw_spin_unlock_irqrestore (&iommu->register_lock, flags); + + status_code =VCMD_VRSP_SC (res); +switch (status_code) { +case VCMD_VRSP_SC_SUCCESS : +break ; +case VCMD_VRSP_SC_INVALID_PASID : +pr_info ("IOMMU: %s: Invalid PASID\n" , iommu->name); +break ; +default : +pr_warn ("IOMMU: %s: Unexpected error code %d\n" , + iommu->name, status_code); + } +} +/* * Per device pasid table management: */
drivers/iommu/intel-pasid.h +12 -1
@@ -23,6 +23,16 @@#define is_pasid_enabled (entry) (((entry)->lo >> 3) & 0x1)#define get_pasid_dir_size (entry) (1 << ((((entry)->lo >> 9) & 0x7) + 7)) +/* Virtual command interface for enlightened pasid management. */ +#define VCMD_CMD_ALLOC 0x1 +#define VCMD_CMD_FREE 0x2 +#define VCMD_VRSP_IP 0x1 +#define VCMD_VRSP_SC (e) (((e) >> 1) & 0x3) +#define VCMD_VRSP_SC_SUCCESS 0 +#define VCMD_VRSP_SC_NO_PASID_AVAIL 1 +#define VCMD_VRSP_SC_INVALID_PASID 1 +#define VCMD_VRSP_RESULT_PASID (e) (((e) >> 8) & 0xfffff) +#define VCMD_CMD_OPERAND (e) ((e) << 8)/* * Domain ID reserved for pasid entries programmed for first-level * only and pass-through transfer modes.
@@ -111,5 +121,6 @@ int intel_pasid_setup_nested(struct intel_iommu *iommu, struct dmar_domain *domain, int addr_width);void intel_pasid_tear_down_entry (struct intel_iommu *iommu,struct device *dev,int pasid); - +int vcmd_alloc_pasid (struct intel_iommu *iommu,unsigned int *pasid); +void vcmd_free_pasid (struct intel_iommu *iommu,unsigned int pasid);#endif /* __INTEL_PASID_H */
include/linux/intel-iommu.h +1 -0
@@ -169,6 +169,7 @@#define ecap_smpwc (e) (((e) >> 48) & 0x1)#define ecap_flts (e) (((e) >> 47) & 0x1)#define ecap_slts (e) (((e) >> 46) & 0x1) +#define ecap_vcs (e) (((e) >> 44) & 0x1)#define ecap_smts (e) (((e) >> 43) & 0x1)#define ecap_dit (e) ((e >> 41) & 0x1)#define ecap_pasid (e) ((e >> 40) & 0x1)
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v13 8/8 iommu/vt-d: Add custom allocator for IOASID
In reply to Jacob PanWhen VT-d driver runs in the guest, PASID allocation must be performed via virtual command interface. This patch registers a custom IOASID allocator which takes precedence over the default XArray based allocator. The resulting IOASID allocation will always come from the host. This ensures that PASID namespace is system- wide.
Virtual command registers are used in the guest only, to prevent vmexit cost, we cache the capability and store it during initialization.
---
v12 - squashed virtual command register caching patch with its user
- reduce indentation
---
--- drivers/iommu/dmar.c | 1 + drivers/iommu/intel-iommu.c | 85 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 7 ++++ 3 files changed, 93 insertions(+)
drivers/iommu/dmar.c +1 -0
@@ -963,6 +963,7 @@ static int map_iommu(struct intel_iommu *iommu, u64 phys_addr)warn_invalid_dmar (phys_addr," returns all ones" ); goto unmap; } + iommu->vccap =dmar_readq (iommu->reg +DMAR_VCCAP_REG );/* the registers might be more than one page */ map_size = max_t(int, ecap_max_iotlb_offset(iommu->ecap),
drivers/iommu/intel-iommu.c +85 -0
103 lines not on this page · raw message
include/linux/intel-iommu.h +7 -0
@@ -19,6 +19,7 @@#include <linux/iommu.h> #include <linux/io-64-nonatomic-lo-hi.h> #include <linux/dmar.h> +#include <linux/ioasid.h> #include <asm/cacheflush.h> #include <asm/iommu.h>
@@ -195,6 +196,9 @@#define ecap_max_handle_mask (e) ((e >> 20) & 0xf)#define ecap_sc_support (e) ((e >> 7) & 0x1)/* Snooping Control */ +/* Virtual command interface capability */ +#define vccap_pasid (v) (((v) & DMA_VCS_PAS))/* PASID allocation */ +/* IOTLB_REG */ #define DMA_TLB_FLUSH_GRANU_OFFSET 60#define DMA_TLB_GLOBAL_FLUSH (((u64)1) << 60)
@@ -288,6 +292,7 @@/* PRS_REG */ #define DMA_PRS_PPR ((u32)1) +#define DMA_VCS_PAS ((u64)1)#define IOMMU_WAIT_OP (iommu, offset, op, cond, sts) \ do { \
@@ -563,6 +568,7 @@ struct intel_iommu {u64 reg_size;/* size of hw register set */ u64 cap;u64 ecap; +u64 vccap;u32 gcmd;/* Holds TE, EAFL. Don't need SRTP, SFL, WBF */ raw_spinlock_t register_lock;/* protect register handling */ int seq_id;/* sequence id of the iommu */
@@ -583,6 +589,7 @@ struct intel_iommu {#ifdef CONFIG_INTEL_IOMMU_SVM struct page_req_dsc *prq;unsigned char prq_name[16 ];/* Name for PRQ interrupt */ +struct ioasid_allocator_ops pasid_allocator;/* Custom allocator for PASIDs */ #endif struct q_inval *qi;/* Queued invalidation info */ u32 *iommu_state;/* Store iommu states between suspend and resume.*/
Signature · 6 lines
2.7.4 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 53 lines · Jacob Pan
Shared virtual address (SVA), a.k.a, Shared virtual memory (SVM) on Intel platforms allow address space sharing between device DMA and applications. SVA can reduce programming complexity and enhance security.
This series is intended to enable SVA virtualization, i.e. enable use of SVA within a guest user application.This is the remaining portion of the original patchset that is based on Joerg's x86/vt-d branch. The preparatory and cleanup patches are merged here.
(git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git)Only IOMMU portion of the changes are included in this series. Additional support is needed in VFIO and QEMU (will be submitted separately) to complete this functionality.
To make incremental changes and reduce the size of each patchset. This series does not inlcude support for page request services.
In VT-d implementation, PASID table is per device and maintained in the host. Guest PASID table is shadowed in VMM where virtual IOMMU is emulated.
.-------------. .---------------------------. | vIOMMU | | Guest process CR3, FL only| | | '---------------------------'.----------------/
| PASID Entry |--- PASID cache flush -
'-------------' |
| | V
| | CR3 in GPA
'-------------'
Guest
------| Shadow |--------------------------|--------
v v v
Host.-------------. .----------------------. | pIOMMU | | Bind FL for GVA-GPA | | | '----------------------' .----------------/ | | PASID Entry | V (Nested xlate) '----------------\.------------------------------. | | |SL for GPA-HPA, default domain| | | '------------------------------' '-------------'Where:
- FL = First level/stage one page tables
- SL = Second level/stage two page tablesThis is the remaining VT-d only portion of V5 since the uAPIs and IOASID common code have been applied to Joerg's IOMMU core branch.
(https://lkml.org/lkml/2019/10/2/833)The complete set with VFIO patches are here:
https://github.com/jacobpan/linux.git:siov_sva
Quoted 6 lines
The complete nested SVA upstream patches are divided into three phases:
1. Common APIs and PCI device direct assignment
2. Page Request Services (PRS) support
3. Mediated device assignmentWith this set and the accompanied VFIO code, we will achieve phase #1.
With Christoph's comments covered, this patch set has been queued for v5.8. Thank you, Jacob!
Best regards,
baolu
Quoted 113 lines
Thanks,
Jacob
ChangeLog:
- v13
- Dropped memory type support (MTS) in guest PASID bind
- Do not support multiple bind gpasid if device has no aux domain
- Removed extra error msgs inpasid_setup_bind_data()
- Replaced SVM device list free function with combined out label- v12
- Fixed IA64 cross compile error
- Squashed two patches that add macros with its users
- Use ratelimited prints for all user called APIs
- Check domain nesting attr for vSVA APIs.
- Misc style improvements- v11 Misc fixes and improvements based on review by Kevin & Eric
- Fixed devTLB granularity conversion
- Simplified VT-d granulairy lookup by replacing 2D map array with invalid entries.
- Fixed locking in bind guest PASID
- Added nesting domain attr check
- Squashed agaw checking patch with user
- Use rate limitted error message for all user originated calls- v10
- Addressed Eric's review in v7 and v9. Most fixes are in 3/10 and 6/10. Extra condition checks and consolidation of duplicated codes.- v9
- Addressed Baolu's comments for v8 for IOTLB flush consolidation, bug fixes
- Removed IOASID notifier code which will be submitted separately to address PASID life cycle management with multiple users.- v8
- Extracted cleanup patches from V7 and accepted into maintainer's tree (https://lkml.org/lkml/2019/12/2/514).
- Added IOASID notifier and VT-d handler for termination of PASID IOMMU context upon free. This will ensure success of VFIO IOASID free API regardless PASID is in use.
(https://lore.kernel.org/linux-iommu/[email protected]/)- V7
- Respect vIOMMU PASID range in virtual command PASID/IOASID allocator
- Caching virtual command capabilities to avoid runtime checks that could cause vmexits.- V6
- Rebased on top of Joerg's core branch
(git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.gitcore)
- Adapt to new uAPIs and IOASID allocators- V5
Rebased on v5.3-rc4 which has some of the IOMMU fault APIs merged. Addressed v4 review comments from Eric Auger, Baolu Lu, and
Jonathan Cameron. Specific changes are as follows:
- Refined custom IOASID allocator to support multiple vIOMMU, hotplug cases.
- Extracted vendor data from IOMMU guest PASID bind data, for VT-d will support all necessary guest PASID entry fields for PASID bind.
- Support non-identity host-guest PASID mapping
- Exception handling in various cases- V4
- Redesigned IOASID allocator such that it can support custom allocators with shared helper functions. Use separate XArray to store IOASIDs per allocator. Took advice from Eric Auger to have default allocator use the generic allocator structure. Combined into one patch in that the default allocator is just "another" allocator now. Can be built as a module in case of driver use without IOMMU.
- Extended bind guest PASID data to support SMMU and non-identity guest to host PASID mapping https://lkml.org/lkml/2019/5/21/802
- Rebased on Jean's sva/api common tree, new patches starts with [PATCH v4 10/22]- V3
- Addressed thorough review comments from Eric Auger (Thank you!)
- Moved IOASID allocator from driver core to IOMMU code per suggestion by Christoph Hellwig
(https://lkml.org/lkml/2019/4/26/462)
- Rebased on top of Jean's SVA API branch and Eric's v7[1]
(git://linux-arm.org/linux-jpb.gitsva/api)
- All IOMMU APIs are unmodified (except the new bind guest PASID call in patch 9/16)- V2
- Rebased on Joerg's IOMMU x86/vt-d branch v5.1-rc4
- Integrated with Eric Auger's new v7 series for common APIs
(https://github.com/eauger/linux/tree/v5.1-rc3-2stage-v7)
- Addressed review comments from Andy Shevchenko and Alex Williamson on IOASID custom allocator.
- Support multiple custom IOASID allocators (vIOMMUs) and dynamic registration.Jacob Pan (7):
iommu/vt-d: Move domain helper to header iommu/vt-d: Use a helper function to skip agaw for SL iommu/vt-d: Add nested translation helper function iommu/vt-d: Add bind guest PASID support iommu/vt-d: Support flushing more translation cache types iommu/vt-d: Add svm/sva invalidate function iommu/vt-d: Add custom allocator for IOASIDLu Baolu (1):
iommu/vt-d: Enlightened PASID allocation
Quoted 10 lines
drivers/iommu/dmar.c | 40 ++++++ drivers/iommu/intel-iommu.c | 291 +++++++++++++++++++++++++++++++++++++++----- drivers/iommu/intel-pasid.c | 266 +++++++++++++++++++++++++++++++++++++--- drivers/iommu/intel-pasid.h | 23 +++- drivers/iommu/intel-svm.c | 203 ++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 69 ++++++++++- include/linux/intel-svm.h | 12 ++ include/uapi/linux/iommu.h | 5 + 8 files changed, 858 insertions(+), 51 deletions(-)
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu