iommu/vt-d: Improve PASID id and table management
v3 cover iommu/vt-d: Improve PASID id and table management
Hi,
This patch set improves the PASID id and PASID table management for Intel IOMMU driver.
PATCH 1~3 replace per IOMMU idr name space with a global one. Current per IOMMU idr doesn't work in some cases where one application (associated with a PASID) might talk to two physical devices simultaneously while the two devices could reside behind two different IOMMU units.
PATCH 4~9 implement per device PASID table. Current per IOMMU PASID table implementation is insecure in the cases where multiple devices under one single IOMMU unit support PASID feature. With per domain PASID table, we can achieve finer protection and isolation granularity. This has been discussed at https://lkml.org/lkml/2018/5/16/154.
Best regards,
Lu Baolu
Change log:v1->v2:
- Patches have been reviewed by "Liu Yi L <[email protected]>".
- An error case handling was added in PATCH 6/9.
- Some commit messages are refined to be more accurate.v2->v3:
- Patches rebased on top of v4.18-rc1.
- Replace per-domain with per-device pasid table.
Lu Baolu (9):
iommu/vt-d: Global PASID name space iommu/vt-d: Avoid using idr_for_each_entry() iommu/vt-d: Apply global PASID in SVA iommu/vt-d: Move device_domain_info to header iommu/vt-d: Add for_each_device_domain() helper iommu/vt-d: Per PCI device pasid table interfaces iommu/vt-d: Allocate and free pasid table iommu/vt-d: Apply per pci device pasid table in SVA iommu/vt-d: Remove the obsolete per iommu pasid tables
drivers/iommu/Makefile | 2 +- drivers/iommu/intel-iommu.c | 142 ++++++++++++-------------- drivers/iommu/intel-pasid.c | 238 ++++++++++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.h | 39 ++++++++ drivers/iommu/intel-svm.c | 79 ++++++--------- include/linux/intel-iommu.h | 79 ++++++++++++++- 6 files changed, 447 insertions(+), 132 deletions(-) create mode 100644 drivers/iommu/intel-pasid.c create mode 100644 drivers/iommu/intel-pasid.h
Signature · 1 line
2.7.4
PATCH v3 1/9 iommu/vt-d: Global PASID name space
This adds the system wide PASID name space for the PASID allocation. Currently we are using per IOMMU PASID name spaces which are not suitable for some use cases. For an example, one application (associated with a PASID) might talk to two physical devices simultaneously while the two devices could reside behind two different IOMMU units.
--- drivers/iommu/Makefile | 2 +- drivers/iommu/intel-iommu.c | 13 ++++++++++ drivers/iommu/intel-pasid.c | 60 +++++++++++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.h | 21 ++++++++++++++++ 4 files changed, 95 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/intel-pasid.c create mode 100644 drivers/iommu/intel-pasid.h
drivers/iommu/Makefile +1 -1
@@ -14,7 +14,7 @@ obj-$(CONFIG_AMD_IOMMU_V2) += amd_iommu_v2.o obj-$(CONFIG_ARM_SMMU) += arm-smmu.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += dmar.o -obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o +obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.o obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o obj-$(CONFIG_IRQ_REMAP) += intel_irq_remapping.o irq_remapping.o
drivers/iommu/intel-iommu.c +13 -0
@@ -53,6 +53,7 @@#include <asm/iommu.h> #include "irq_remapping.h" +#include "intel-pasid.h" #define ROOT_SIZE VTD_PAGE_SIZE#define CONTEXT_SIZE VTD_PAGE_SIZE
@@ -3277,6 +3278,18 @@ static int __init init_dmars(void) }for_each_active_iommu (iommu, drhd) { +/* +* Find the max pasid size of all IOMMU's in the system. +* We need to ensure the system pasid table is no bigger +* than the smallest supported. +*/ +if (pasid_enabled (iommu)) { +u32 temp =2 <<ecap_pss (iommu->ecap); + + intel_pasid_max_id =min_t (u32, temp, + intel_pasid_max_id); + } + g_iommus[iommu->seq_id] = iommu;intel_iommu_init_qi (iommu);
drivers/iommu/intel-pasid.c new file +60 -0
60 lines not on this page · raw message
drivers/iommu/intel-pasid.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* intel-pasid.h - PASID idr, table and entry header +* +* Copyright (C) 2018 Intel Corporation +* +* Author: Lu Baolu <[email protected]> +*/ + +#ifndef __INTEL_PASID_H +#define __INTEL_PASID_H + +#define PASID_MIN 0x1 +#define PASID_MAX 0x20000 + +extern u32 intel_pasid_max_id; +int intel_pasid_alloc_id (void *ptr,int start,int end,gfp_t gfp); +void intel_pasid_free_id (int pasid); +void *intel_pasid_lookup_id (int pasid); + +#endif /* __INTEL_PASID_H */
Signature · 1 line
2.7.4
PATCH v3 2/9 iommu/vt-d: Avoid using idr_for_each_entry()
In reply to Lu Baoluidr_for_each_entry() is used to iteratte over idr elements of a given type. It isn't suitable for the globle pasid idr since the pasid idr consumer could specify different types of pointers to bind with a pasid.
--- drivers/iommu/intel-svm.c | 14 ++++++++++---- include/linux/intel-iommu.h | 1 + 2 files changed, 11 insertions(+), 4 deletions(-)
drivers/iommu/intel-svm.c +10 -4
@@ -298,6 +298,7 @@ static const struct mmu_notifier_ops intel_mmuops = { };static DEFINE_MUTEX (pasid_mutex); +static LIST_HEAD (global_svm_list);int intel_svm_bind_mm (struct device *dev,int *pasid,int flags,struct svm_dev_ops *ops) {
@@ -329,13 +330,13 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_mutex_lock (&pasid_mutex);if (pasid && !(flags &SVM_FLAG_PRIVATE_PASID )) { -int i; +struct intel_svm *t; -idr_for_each_entry (&iommu->pasid_idr, svm, i) { -if (svm->mm != mm || - (svm->flags &SVM_FLAG_PRIVATE_PASID )) +list_for_each_entry (t, &global_svm_list, list) { +if (t->mm != mm || (t->flags &SVM_FLAG_PRIVATE_PASID ))continue ; + svm = t;if (svm->pasid >= pasid_max) { dev_warn(dev, "Limited PASID width. Cannot use existing PASID %d\n",
@@ -404,6 +405,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ svm->mm = mm; svm->flags = flags;INIT_LIST_HEAD_RCU (&svm->devs); +INIT_LIST_HEAD (&svm->list); ret = -ENOMEM ;if (mm) { ret =mmu_notifier_register (&svm->notifier, mm);
@@ -430,6 +432,8 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_*/ if (cap_caching_mode (iommu->cap))intel_flush_pasid_dev (svm, sdev, svm->pasid); + +list_add_tail (&svm->list, &global_svm_list); }list_add_rcu (&sdev->list, &svm->devs);
@@ -485,6 +489,8 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)if (svm->mm)mmu_notifier_unregister (&svm->notifier, svm->mm); +list_del (&svm->list); +/* We mandate that no page faults may be outstanding * for the PASID when intel_svm_unbind_mm() is called. * If that is not obeyed, subtle errors will happen.
include/linux/intel-iommu.h +1 -0
@@ -485,6 +485,7 @@ struct intel_svm {int flags;int pasid;struct list_head devs; +struct list_head list; };extern int intel_iommu_enable_pasid (struct intel_iommu *iommu,struct intel_svm_dev *sdev);
Signature · 1 line
2.7.4
PATCH v3 3/9 iommu/vt-d: Apply global PASID in SVA
In reply to Lu BaoluThis patch applies the global pasid name space in the shared virtual address (SVA) implementation.
--- drivers/iommu/intel-svm.c | 22 +++++++++++----------- include/linux/intel-iommu.h | 1 - 2 files changed, 11 insertions(+), 12 deletions(-)
drivers/iommu/intel-svm.c +11 -11
@@ -26,6 +26,8 @@#include <linux/interrupt.h> #include <asm/page.h> +#include "intel-pasid.h" +#define PASID_ENTRY_P BIT_ULL(0)#define PASID_ENTRY_FLPM_5LP BIT_ULL(9)#define PASID_ENTRY_SRE BIT_ULL(11)
@@ -85,8 +87,6 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu) iommu->name); } -idr_init (&iommu->pasid_idr); -return 0 ; }
@@ -102,7 +102,7 @@ int intel_svm_free_pasid_tables(struct intel_iommu *iommu)free_pages ((unsigned long )iommu->pasid_state_table, order); iommu->pasid_state_table =NULL ; } -idr_destroy (&iommu->pasid_idr); +return 0 ; }
@@ -392,9 +392,9 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ pasid_max = iommu->pasid_max;/* Do not use PASID 0 in caching mode (virtualised IOMMU) */ - ret =idr_alloc (&iommu->pasid_idr, svm, - !!cap_caching_mode (iommu->cap), - pasid_max -1 ,GFP_KERNEL ); + ret =intel_pasid_alloc_id (svm, + !!cap_caching_mode (iommu->cap), + pasid_max -1 ,GFP_KERNEL );if (ret <0 ) {kfree (svm);kfree (sdev);
@@ -410,7 +410,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_if (mm) { ret =mmu_notifier_register (&svm->notifier, mm);if (ret) { -idr_remove (&svm->iommu->pasid_idr, svm->pasid); +intel_pasid_free_id (svm->pasid);kfree (svm);kfree (sdev); goto out;
@@ -460,7 +460,7 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)if (!iommu || !iommu->pasid_table) goto out; - svm =idr_find (&iommu->pasid_idr, pasid); + svm =intel_pasid_lookup_id (pasid);if (!svm) goto out;
@@ -485,7 +485,7 @@ int intel_svm_unbind_mm(struct device *dev, int pasid) svm->iommu->pasid_table[svm->pasid].val =0 ;wmb (); -idr_remove (&svm->iommu->pasid_idr, svm->pasid); +intel_pasid_free_id (svm->pasid);if (svm->mm)mmu_notifier_unregister (&svm->notifier, svm->mm);
@@ -520,7 +520,7 @@ int intel_svm_is_pasid_valid(struct device *dev, int pasid)if (!iommu || !iommu->pasid_table) goto out; - svm =idr_find (&iommu->pasid_idr, pasid); + svm =intel_pasid_lookup_id (pasid);if (!svm) goto out;
@@ -618,7 +618,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)if (!svm || svm->pasid != req->pasid) {rcu_read_lock (); - svm =idr_find (&iommu->pasid_idr, req->pasid); + svm =intel_pasid_lookup_id (req->pasid);/* It *can't* go away, because the driver is not permitted * to unbind the mm while any page faults are outstanding. * So we only need RCU to protect the internal idr code. */
include/linux/intel-iommu.h +0 -1
@@ -417,7 +417,6 @@ struct intel_iommu {struct pasid_state_entry *pasid_state_table;struct page_req_dsc *prq;unsigned char prq_name[16 ];/* Name for PRQ interrupt */ -struct idr pasid_idr;u32 pasid_max;#endif struct q_inval *qi;/* Queued invalidation info */
Signature · 1 line
2.7.4
PATCH v3 4/9 iommu/vt-d: Move device_domain_info to header
In reply to Lu BaoluThis allows the per device iommu data and some helpers to be used in other files.
--- drivers/iommu/intel-iommu.c | 62 +++-------------------------------------- include/linux/intel-iommu.h | 68 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+), 58 deletions(-)
drivers/iommu/intel-iommu.c +4 -58
@@ -381,60 +381,6 @@ static int hw_pass_through = 1;for (idx =0 ; idx < g_num_of_iommus; idx++) \if (domain->iommu_refcnt[idx]) -struct dmar_domain { -int nid;/* node id */ - -unsigned iommu_refcnt[DMAR_UNITS_SUPPORTED ]; -/* Refcount of devices per iommu */ - - -u16 iommu_did[DMAR_UNITS_SUPPORTED ]; -/* Domain ids per IOMMU. Use u16 since -* domain ids are 16 bit wide according -* to VT-d spec, section 9.3 */ - -bool has_iotlb_device; -struct list_head devices;/* all devices' list */ -struct iova_domain iovad;/* iova's that belong to this domain */ - -struct dma_pte *pgd;/* virtual address */ -int gaw;/* max guest address width */ - -/* adjusted guest address width, 0 is level 2 30-bit */ -int agaw; - -int flags;/* flags to find out type of domain */ - -int iommu_coherency;/* indicate coherency of iommu access */ -int iommu_snooping;/* indicate snooping control feature*/ -int iommu_count;/* reference count of iommu */ -int iommu_superpage;/* Level of superpages supported: -0 == 4KiB (no superpages), 1 == 2MiB, -2 == 1GiB, 3 == 512GiB, 4 == 1TiB */ -u64 max_addr;/* maximum mapped address */ - -struct iommu_domain domain;/* generic domain data structure for -iommu core */ -}; - -/* PCI domain-device relationship */ -struct device_domain_info { -struct list_head link;/* link to domain siblings */ -struct list_head global;/* link to global list */ -u8 bus;/* PCI bus number */ -u8 devfn;/* PCI devfn number */ -u8 pasid_supported:3 ; -u8 pasid_enabled:1 ; -u8 pri_supported:1 ; -u8 pri_enabled:1 ; -u8 ats_supported:1 ; -u8 ats_enabled:1 ; -u8 ats_qdep; -struct device *dev;/* it's NULL for PCIe-to-PCI bridge */ -struct intel_iommu *iommu;/* IOMMU used by this device */ -struct dmar_domain *domain;/* pointer to domain */ -}; - struct dmar_rmrr_unit {struct list_head list;/* list of rmrr units */ struct acpi_dmar_header *hdr;/* ACPI header */
@@ -603,7 +549,7 @@ static void set_iommu_domain(struct intel_iommu *iommu, u16 did, domains[did &0xff ] = domain; } -static inline void *alloc_pgtable_page (int node) +void *alloc_pgtable_page (int node) {struct page *page;void *vaddr =NULL ;
@@ -614,7 +560,7 @@ static inline void *alloc_pgtable_page(int node)return vaddr; } -static inline void free_pgtable_page (void *vaddr) +void free_pgtable_page (void *vaddr) {free_page ((unsigned long )vaddr); }
@@ -697,7 +643,7 @@ int iommu_calculate_agaw(struct intel_iommu *iommu) }/* This functionin only returns single iommu in a domain */ -static struct intel_iommu *domain_get_iommu (struct dmar_domain *domain) +struct intel_iommu *domain_get_iommu (struct dmar_domain *domain) {int iommu_id;
@@ -3512,7 +3458,7 @@ static unsigned long intel_alloc_iova(struct device *dev,return iova_pfn; } -static struct dmar_domain *get_valid_domain_for_dev (struct device *dev) +struct dmar_domain *get_valid_domain_for_dev (struct device *dev) {struct dmar_domain *domain, *tmp;struct dmar_rmrr_unit *rmrr;
include/linux/intel-iommu.h +68 -0
92 lines not on this page · raw message
Signature · 1 line
2.7.4
PATCH v3 5/9 iommu/vt-d: Add for_each_device_domain() helper
In reply to Lu BaoluThis adds a helper named for_each_device_domain() to iterate over the elements in device_domain_list and invoke a callback against each element. This allows to search the device_domain list in other source files.
--- drivers/iommu/intel-iommu.c | 21 +++++++++++++++++++++ include/linux/intel-iommu.h | 2 ++ 2 files changed, 23 insertions(+)
drivers/iommu/intel-iommu.c +21 -0
@@ -448,6 +448,27 @@ EXPORT_SYMBOL_GPL(intel_iommu_gfx_mapped);static DEFINE_SPINLOCK (device_domain_lock);static LIST_HEAD (device_domain_list); +/* +* Iterate over elements in device_domain_list and call the specified +* callback @fn against each element. This helper should only be used +* in the context where the device_domain_lock has already been holden. +*/ +int for_each_device_domain (int (*fn)(struct device_domain_info *info, +void *data),void *data) +{ +int ret =0 ; +struct device_domain_info *info; + +assert_spin_locked (&device_domain_lock); +list_for_each_entry (info, &device_domain_list, global) { + ret =fn (info, data); +if (ret) +return ret; + } + +return 0 ; +} +const struct iommu_ops intel_iommu_ops;static bool translation_pre_enabled (struct intel_iommu *iommu)
include/linux/intel-iommu.h +2 -0
@@ -525,6 +525,8 @@ struct dmar_domain *get_valid_domain_for_dev(struct device *dev);void *alloc_pgtable_page (int node);void free_pgtable_page (void *vaddr);struct intel_iommu *domain_get_iommu (struct dmar_domain *domain); +int for_each_device_domain (int (*fn)(struct device_domain_info *info, +void *data),void *data);#ifdef CONFIG_INTEL_IOMMU_SVM extern int intel_svm_alloc_pasid_tables (struct intel_iommu *iommu);
Signature · 1 line
2.7.4
PATCH v3 6/9 iommu/vt-d: Per PCI device pasid table interfaces
In reply to Lu BaoluThis patch adds the interfaces for per PCI device pasid table management. Currently we allocate one pasid table for all PCI devices under the scope of an IOMMU. It's insecure in some cases where multiple devices under one single IOMMU unit support PASID features. With per PCI device pasid table, we can achieve finer protection and isolation granularity.
--- drivers/iommu/intel-iommu.c | 1 + drivers/iommu/intel-pasid.c | 178 ++++++++++++++++++++++++++++++++++++++++++++ drivers/iommu/intel-pasid.h | 18 +++++ drivers/iommu/intel-svm.c | 4 - include/linux/intel-iommu.h | 2 + 5 files changed, 199 insertions(+), 4 deletions(-)
drivers/iommu/intel-iommu.c +1 -0
@@ -2436,6 +2436,7 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu, info->dev = dev; info->domain = domain; info->iommu = iommu; + info->pasid_table =NULL ;if (dev &&dev_is_pci (dev)) {struct pci_dev *pdev =to_pci_dev (info->dev);
drivers/iommu/intel-pasid.c +178 -0
187 lines not on this page · raw message
drivers/iommu/intel-pasid.h +18 -0
@@ -13,9 +13,27 @@#define PASID_MIN 0x1#define PASID_MAX 0x20000 +struct pasid_entry { +u64 val; +}; + +/* The representative of a PASID table */ +struct pasid_table { +void *table;/* pasid table pointer */ +int order;/* page order of pasid table */ +int max_pasid;/* max pasid */ +struct list_head dev;/* device list */ +}; +extern u32 intel_pasid_max_id;int intel_pasid_alloc_id (void *ptr,int start,int end,gfp_t gfp);void intel_pasid_free_id (int pasid);void *intel_pasid_lookup_id (int pasid); +int intel_pasid_alloc_table (struct device *dev); +void intel_pasid_free_table (struct device *dev); +struct pasid_table *intel_pasid_get_table (struct device *dev); +int intel_pasid_get_dev_max_id (struct device *dev); +struct pasid_entry *intel_pasid_get_entry (struct device *dev,int pasid); +void intel_pasid_clear_entry (struct device *dev,int pasid);#endif /* __INTEL_PASID_H */
drivers/iommu/intel-svm.c +0 -4
@@ -34,10 +34,6 @@static irqreturn_t prq_event_thread (int irq,void *d); -struct pasid_entry { -u64 val; -}; -struct pasid_state_entry {u64 val; };
include/linux/intel-iommu.h +2 -0
@@ -481,6 +481,7 @@ struct intel_iommu {struct device_domain_info {struct list_head link;/* link to domain siblings */ struct list_head global;/* link to global list */ +struct list_head table;/* link to pasid table */ u8 bus;/* PCI bus number */ u8 devfn;/* PCI devfn number */ u8 pasid_supported:3 ;
@@ -493,6 +494,7 @@ struct device_domain_info {struct device *dev;/* it's NULL for PCIe-to-PCI bridge */ struct intel_iommu *iommu;/* IOMMU used by this device */ struct dmar_domain *domain;/* pointer to domain */ +struct pasid_table *pasid_table;/* pasid table */ }; static inline void __iommu_flush_cache(
Signature · 1 line
2.7.4
PATCH v3 7/9 iommu/vt-d: Allocate and free pasid table
In reply to Lu BaoluThis patch allocates a PASID table for a PCI device at the time when the dmar dev_info is attached to dev->archdata.iommu, and free it in the opposite case.
--- drivers/iommu/intel-iommu.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/iommu/intel-iommu.c +10 -0
@@ -2494,6 +2494,15 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu,list_add (&info->global, &device_domain_list);if (dev) dev->archdata.iommu = info; + +if (dev &&dev_is_pci (dev) && info->pasid_supported) { + ret =intel_pasid_alloc_table (dev); +if (ret) { +__dmar_remove_one_dev_info (info); +spin_unlock_irqrestore (&device_domain_lock, flags); +return NULL ; + } + }spin_unlock_irqrestore (&device_domain_lock, flags);if (dev &&domain_context_mapping (domain, dev)) {
@@ -4828,6 +4837,7 @@ static void __dmar_remove_one_dev_info(struct device_domain_info *info)if (info->dev) {iommu_disable_dev_iotlb (info);domain_context_clear (iommu, info->dev); +intel_pasid_free_table (info->dev); }unlink_domain_info (info);
Signature · 1 line
2.7.4
PATCH v3 8/9 iommu/vt-d: Apply per pci device pasid table in SVA
In reply to Lu BaoluThis patch applies the per pci device pasid table in the Shared Virtual Address (SVA) implementation.
--- drivers/iommu/intel-iommu.c | 29 +++++++++-------------------- drivers/iommu/intel-svm.c | 22 ++++++++++------------ 2 files changed, 19 insertions(+), 32 deletions(-)
drivers/iommu/intel-iommu.c +9 -20
@@ -5163,22 +5163,16 @@ static void intel_iommu_put_resv_regions(struct device *dev,#ifdef CONFIG_INTEL_IOMMU_SVM #define MAX_NR_PASID_BITS (20) -static inline unsigned long intel_iommu_get_pts (struct intel_iommu *iommu) +static inline unsigned long intel_iommu_get_pts (struct device *dev) { -/* -* Convert ecap_pss to extend context entry pts encoding, also -* respect the soft pasid_max value set by the iommu. -* - number of PASID bits = ecap_pss + 1 -* - number of PASID table entries = 2^(pts + 5) -* Therefore, pts = ecap_pss - 4 -* e.g. KBL ecap_pss = 0x13, PASID has 20 bits, pts = 15 -*/ -if (ecap_pss (iommu->ecap) <5 ) +int pts, max_pasid; + + max_pasid =intel_pasid_get_dev_max_id (dev); + pts =find_first_bit ((unsigned long *)&max_pasid,MAX_NR_PASID_BITS ); +if (pts <5 )return 0 ; -/* pasid_max is encoded as actual number of entries not the bits */ -return find_first_bit ((unsigned long *)&iommu->pasid_max, -MAX_NR_PASID_BITS ) -5 ; +return pts -5 ; }int intel_iommu_enable_pasid (struct intel_iommu *iommu,struct intel_svm_dev *sdev)
@@ -5214,8 +5208,8 @@ int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct intel_svm_dev *sdif (!(ctx_lo &CONTEXT_PASIDE )) {if (iommu->pasid_state_table) context[1 ].hi = (u64 )virt_to_phys (iommu->pasid_state_table); - context[1 ].lo = (u64 )virt_to_phys (iommu->pasid_table) | -intel_iommu_get_pts (iommu); + context[1 ].lo = (u64 )virt_to_phys (info->pasid_table->table) | +intel_iommu_get_pts (sdev->dev);wmb ();/* CONTEXT_TT_MULTI_LEVEL and CONTEXT_TT_DEV_IOTLB are both
@@ -5282,11 +5276,6 @@ struct intel_iommu *intel_svm_device_to_iommu(struct device *dev) return NULL; } -if (!iommu->pasid_table) { -dev_err (dev,"PASID not enabled on IOMMU; cannot enable SVM\n" ); -return NULL ; - } -return iommu; }#endif /* CONFIG_INTEL_IOMMU_SVM */
drivers/iommu/intel-svm.c +10 -12
@@ -274,11 +274,9 @@ static void intel_mm_release(struct mmu_notifier *mn, struct mm_struct *mm)* page) so that we end up taking a fault that the hardware really * *has* to handle gracefully without affecting other processes. */ - svm->iommu->pasid_table[svm->pasid].val =0 ; -wmb (); -rcu_read_lock ();list_for_each_entry_rcu (sdev, &svm->devs, list) { +intel_pasid_clear_entry (sdev->dev, svm->pasid);intel_flush_pasid_dev (svm, sdev, svm->pasid);intel_flush_svm_range_dev (svm, sdev,0 ,-1 ,0 , !svm->mm); }
@@ -299,6 +297,7 @@ static LIST_HEAD(global_svm_list);int intel_svm_bind_mm (struct device *dev,int *pasid,int flags,struct svm_dev_ops *ops) {struct intel_iommu *iommu =intel_svm_device_to_iommu (dev); +struct pasid_entry *entry;struct intel_svm_dev *sdev;struct intel_svm *svm =NULL ;struct mm_struct *mm =NULL ;
@@ -306,7 +305,7 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_int pasid_max;int ret; -if (!iommu || !iommu->pasid_table) +if (!iommu)return -EINVAL ;if (dev_is_pci (dev)) {
@@ -384,8 +383,8 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_ } svm->iommu = iommu; -if (pasid_max > iommu->pasid_max) - pasid_max = iommu->pasid_max; +if (pasid_max > intel_pasid_max_id) + pasid_max = intel_pasid_max_id;/* Do not use PASID 0 in caching mode (virtualised IOMMU) */ ret = intel_pasid_alloc_id(svm,
@@ -418,7 +417,8 @@ int intel_svm_bind_mm(struct device *dev, int *pasid, int flags, struct svm_dev_if (cpu_feature_enabled (X86_FEATURE_LA57 )) pasid_entry_val |=PASID_ENTRY_FLPM_5LP ; - iommu->pasid_table[svm->pasid].val = pasid_entry_val; + entry =intel_pasid_get_entry (dev, svm->pasid); + entry->val = pasid_entry_val;wmb ();
@@ -453,7 +453,7 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)mutex_lock (&pasid_mutex); iommu =intel_svm_device_to_iommu (dev); -if (!iommu || !iommu->pasid_table) +if (!iommu) goto out; svm =intel_pasid_lookup_id (pasid);
@@ -476,11 +476,9 @@ int intel_svm_unbind_mm(struct device *dev, int pasid)intel_flush_pasid_dev (svm, sdev, svm->pasid);intel_flush_svm_range_dev (svm, sdev,0 ,-1 ,0 , !svm->mm);kfree_rcu (sdev, rcu); +intel_pasid_clear_entry (dev, svm->pasid);if (list_empty (&svm->devs)) { - svm->iommu->pasid_table[svm->pasid].val =0 ; -wmb (); -intel_pasid_free_id (svm->pasid);if (svm->mm)mmu_notifier_unregister (&svm->notifier, svm->mm);
@@ -513,7 +511,7 @@ int intel_svm_is_pasid_valid(struct device *dev, int pasid)mutex_lock (&pasid_mutex); iommu =intel_svm_device_to_iommu (dev); -if (!iommu || !iommu->pasid_table) +if (!iommu) goto out; svm =intel_pasid_lookup_id (pasid);
Signature · 1 line
2.7.4
PATCH v3 9/9 iommu/vt-d: Remove the obsolete per iommu pasid tables
In reply to Lu BaoluThe obsolete per iommu pasid tables are no longer used. Hence, clean up them.
--- drivers/iommu/intel-iommu.c | 6 +++--- drivers/iommu/intel-svm.c | 17 ++--------------- include/linux/intel-iommu.h | 5 ++--- 3 files changed, 7 insertions(+), 21 deletions(-)
drivers/iommu/intel-iommu.c +3 -3
@@ -1741,7 +1741,7 @@ static void free_dmar_iommu(struct intel_iommu *iommu)if (pasid_enabled (iommu)) {if (ecap_prs (iommu->ecap))intel_svm_finish_prq (iommu); -intel_svm_free_pasid_tables (iommu); +intel_svm_exit (iommu); }#endif }
@@ -3322,7 +3322,7 @@ static int __init init_dmars(void) hw_pass_through =0 ;#ifdef CONFIG_INTEL_IOMMU_SVM if (pasid_enabled (iommu)) -intel_svm_alloc_pasid_tables (iommu); +intel_svm_init (iommu);#endif }
@@ -4285,7 +4285,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)#ifdef CONFIG_INTEL_IOMMU_SVM if (pasid_enabled (iommu)) -intel_svm_alloc_pasid_tables (iommu); +intel_svm_init (iommu);#endif if (dmaru->ignored) {
drivers/iommu/intel-svm.c +2 -15
@@ -38,7 +38,7 @@ struct pasid_state_entry {u64 val; }; -int intel_svm_alloc_pasid_tables (struct intel_iommu *iommu) +int intel_svm_init (struct intel_iommu *iommu) {struct page *pages;int order;
@@ -63,15 +63,6 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu) iommu->pasid_max =0x20000 ; order =get_order (sizeof (struct pasid_entry ) * iommu->pasid_max); - pages =alloc_pages (GFP_KERNEL | __GFP_ZERO, order); -if (!pages) { -pr_warn ("IOMMU: %s: Failed to allocate PASID table\n" , - iommu->name); -return -ENOMEM ; - } - iommu->pasid_table =page_address (pages); -pr_info ("%s: Allocated order %d PASID table.\n" , iommu->name, order); -if (ecap_dis (iommu->ecap)) {/* Just making it explicit... */ BUILD_BUG_ON (sizeof (struct pasid_entry ) !=sizeof (struct pasid_state_entry ));
@@ -86,14 +77,10 @@ int intel_svm_alloc_pasid_tables(struct intel_iommu *iommu)return 0 ; } -int intel_svm_free_pasid_tables (struct intel_iommu *iommu) +int intel_svm_exit (struct intel_iommu *iommu) {int order =get_order (sizeof (struct pasid_entry ) * iommu->pasid_max); -if (iommu->pasid_table) { -free_pages ((unsigned long )iommu->pasid_table, order); - iommu->pasid_table =NULL ; - }if (iommu->pasid_state_table) {free_pages ((unsigned long )iommu->pasid_state_table, order); iommu->pasid_state_table =NULL ;
include/linux/intel-iommu.h +2 -3
@@ -458,7 +458,6 @@ struct intel_iommu {* devices away to userspace processes (e.g. for DPDK) and don't * want to trust that userspace will use *only* the PASID it was * told to. But while it's all driver-arbitrated, we're fine. */ -struct pasid_entry *pasid_table;struct pasid_state_entry *pasid_state_table;struct page_req_dsc *prq;unsigned char prq_name[16 ];/* Name for PRQ interrupt */
@@ -531,8 +530,8 @@ int for_each_device_domain(int (*fn)(struct device_domain_info *info, void *data), void *data);#ifdef CONFIG_INTEL_IOMMU_SVM -extern int intel_svm_alloc_pasid_tables (struct intel_iommu *iommu); -extern int intel_svm_free_pasid_tables (struct intel_iommu *iommu); +int intel_svm_init (struct intel_iommu *iommu); +int intel_svm_exit (struct intel_iommu *iommu);extern int intel_svm_enable_prq (struct intel_iommu *iommu);extern int intel_svm_finish_prq (struct intel_iommu *iommu);
Signature · 1 line
2.7.4
Joerg Roedel
maintainerHey Lu,
the patches do not apply cleanly to my x86/vt-d branch. Can you rebase against that branch after the weekend when I pushed it out and re-send, please?
Thanks a lot,
Joerg
Hi Joerg,
Quoted 5 lines · Joerg Roedel
Hey Lu,
the patches do not apply cleanly to my x86/vt-d branch. Can you rebase against that branch after the weekend when I pushed it out and re-send, please?
Sure.
Best regards,
Lu Baolu