iommu/amd: SVA Support (Part 4) - SVA and IOPF
Subsystems (2)
Vasant Hegde
maintainercover iommu/amd: SVA Support (Part 4) - SVA and IOPF
This is part 4 of the 4-part series to introduce Share Virtual Address (SVA) support for devices, which can support PCI ATS, PASID and PRI capabilities. These devices are referred to as SVA-capable devices in this series.
It contains the following enhancements:
* Patch 1 - 2:
Rename / add support to enable/disable features
* Patch 3 - 4:
Introduce SVA support
* Patch 5 - 10:
Add IOPF support
This patch series is based on top of SVA Part 3 [1].
[1] https://lore.kernel.org/linux-iommu/[email protected]/
This is also available at github :
https://github.com/AMDESE/linux/tree/iommu_sva_part4_v1_v6.5_rc1
Thank you,
Vasant / Suravee
Suravee Suthikulpanit (5):
iommu/amd: Add support to enable/disable SVA feature iommu/amd: Move PPR-related functions into ppr.c iommu/amd: Define per-IOMMU iopf_queue iommu/amd: Add support for page response iommu/amd: Introduce logic to enable/disable IOPF
Vasant Hegde (2):
iommu/amd: Rename amd_iommu_v2_supported() as amd_iommu_sva_supported() iommu/amd: Initial SVA support for AMD IOMMU
Wei Huang (3):
iommu/amd: Add support for enabling/disabling IOMMU features iommu/amd: Add support for add/remove device for IOPF iommu/amd: Add IO page fault notifier handler
drivers/iommu/amd/Kconfig | 1 + drivers/iommu/amd/Makefile | 2 +- drivers/iommu/amd/amd_iommu.h | 41 ++- drivers/iommu/amd/amd_iommu_types.h | 22 ++ drivers/iommu/amd/init.c | 72 ++---- drivers/iommu/amd/iommu.c | 170 ++++++++----- drivers/iommu/amd/ppr.c | 377 ++++++++++++++++++++++++++++ drivers/iommu/amd/sva.c | 328 ++++++++++++++++++++++++ 8 files changed, 894 insertions(+), 119 deletions(-) create mode 100644 drivers/iommu/amd/ppr.c create mode 100644 drivers/iommu/amd/sva.c
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 1/10 iommu/amd: Rename amd_iommu_v2_supported() as amd_iommu_sva_supported()
To reflect its usage. No functional changes intended.
--- drivers/iommu/amd/amd_iommu.h | 2 +- drivers/iommu/amd/init.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/iommu/amd/amd_iommu.h +1 -1
@@ -41,7 +41,7 @@ extern int amd_iommu_guest_ir;extern enum io_pgtable_fmt amd_iommu_pgtable;extern int amd_iommu_gpt_level; -bool amd_iommu_v2_supported (void ); +bool amd_iommu_sva_supported (void );struct amd_iommu *get_amd_iommu (unsigned int idx);u8 amd_iommu_pc_get_max_banks (unsigned int idx);bool amd_iommu_pc_supported (void );
drivers/iommu/amd/init.c +1 -1
@@ -3652,7 +3652,7 @@ __setup("ivrs_ioapic", parse_ivrs_ioapic);__setup ("ivrs_hpet" , parse_ivrs_hpet);__setup ("ivrs_acpihid" , parse_ivrs_acpihid); -bool amd_iommu_v2_supported (void ) +bool amd_iommu_sva_supported (void ) {/* CPU page table size should match IOMMU guest page table size */ if (cpu_feature_enabled(X86_FEATURE_LA57) &&
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 2/10 iommu/amd: Add support for enabling/disabling IOMMU features
In reply to Vasant HegdeFrom: Wei Huang <[email protected]>
Add support for struct iommu_ops.dev_{enable/disable}_feat. Please note that the empty feature switches will be populated by subsequent patches.
--- drivers/iommu/amd/iommu.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
drivers/iommu/amd/iommu.c +28 -0
@@ -2563,6 +2563,32 @@ static bool amd_iommu_enforce_cache_coherency(struct iommu_domain *domain)return true; } +static int amd_iommu_dev_enable_feature (struct device *dev, +enum iommu_dev_features feat) +{ +int ret; + +switch (feat) { +default : + ret = -EINVAL ; +break ; + } +return ret; +} + +static int amd_iommu_dev_disable_feature (struct device *dev, +enum iommu_dev_features feat) +{ +int ret; + +switch (feat) { +default : + ret = -EINVAL ; +break ; + } +return ret; +} +const struct iommu_ops amd_iommu_ops = { .capable = amd_iommu_capable, .domain_alloc = amd_iommu_domain_alloc,
@@ -2574,6 +2600,8 @@ const struct iommu_ops amd_iommu_ops = { .is_attach_deferred = amd_iommu_is_attach_deferred, .pgsize_bitmap = AMD_IOMMU_PGSIZES, .def_domain_type = amd_iommu_def_domain_type, + .dev_enable_feat = amd_iommu_dev_enable_feature, + .dev_disable_feat = amd_iommu_dev_disable_feature, .default_domain_ops = &(const struct iommu_domain_ops ) { .attach_dev = amd_iommu_attach_device, .map_pages = amd_iommu_map_pages,
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 3/10 iommu/amd: Initial SVA support for AMD IOMMU
In reply to Vasant HegdeThis includes :
- Add support to allocate SVA domain. It also sets 'set_dev_pasid' to bind PASID to device and make necessary updates to DTE.
- Add iommu_ops.remove_dev_pasid support. It will unbind PASID from device and makes necessary update to DTE.
- PASID management
xarray to keep track the PASIDs
struct amd_sva_pasid
It will contain mm struct, mmu notifier, protection domain and list of devices within IOMMU group using this pasid.
struct amd_sva_dev
This contains device list and pointer to iommu_dev_data. So that we can track pasid to device binding.
- mmu notifier for iotlb invalidation
- Add IOMMU_SVA as dependency to AMD_IOMMU driver
--- drivers/iommu/amd/Kconfig | 1 + drivers/iommu/amd/Makefile | 2 +- drivers/iommu/amd/amd_iommu.h | 5 + drivers/iommu/amd/amd_iommu_types.h | 10 + drivers/iommu/amd/iommu.c | 10 + drivers/iommu/amd/sva.c | 299 ++++++++++++++++++++++++++++ 6 files changed, 326 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/amd/sva.c
drivers/iommu/amd/Kconfig +1 -0
@@ -10,6 +10,7 @@ config AMD_IOMMU select IOMMU_API select IOMMU_IOVA select IOMMU_IO_PGTABLE + select IOMMU_SVA depends on X86_64 && PCI && ACPI && HAVE_CMPXCHG_DOUBLE help With this option you can enable support for AMD IOMMU hardware in
drivers/iommu/amd/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o +obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o sva.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += debugfs.o
drivers/iommu/amd/amd_iommu.h +5 -0
@@ -41,7 +41,12 @@ extern int amd_iommu_guest_ir;extern enum io_pgtable_fmt amd_iommu_pgtable;extern int amd_iommu_gpt_level; +/* SVA */ bool amd_iommu_sva_supported (void ); +int amd_iommu_set_dev_pasid (struct iommu_domain *domain, +struct device *dev,ioasid_t pasid); +void amd_iommu_remove_dev_pasid (struct device *dev,ioasid_t pasid); +struct amd_iommu *get_amd_iommu (unsigned int idx);u8 amd_iommu_pc_get_max_banks (unsigned int idx);bool amd_iommu_pc_supported (void );
drivers/iommu/amd/amd_iommu_types.h +10 -0
@@ -922,6 +922,16 @@ static inline int get_hpet_devid(int id)return -EINVAL ; } +static inline struct protection_domain *amd_iommu_get_pdomain (struct device *dev) +{ +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!dev_data || !dev_data->domain) +return NULL ; + +return dev_data->domain; +} +enum amd_iommu_intr_mode_type {AMD_IOMMU_GUEST_IR_LEGACY ,
drivers/iommu/amd/iommu.c +10 -0
@@ -76,6 +76,7 @@ struct iommu_cmd {struct kmem_cache *amd_iommu_irq_cache;static void detach_device (struct device *dev); +static void amd_iommu_domain_free (struct iommu_domain *dom); static void set_dte_entry(struct amd_iommu *iommu, u16 devid, struct gcr3_tbl_info *gcr3_info,
@@ -2187,6 +2188,11 @@ static int protection_domain_init_v2(struct protection_domain *pdom)return 0 ; } +static const struct iommu_domain_ops amd_svm_domain_ops = { + .set_dev_pasid = amd_iommu_set_dev_pasid, + .free = amd_iommu_domain_free +}; +static struct protection_domain *protection_domain_alloc (unsigned int type) {struct io_pgtable_ops *pgtbl_ops;
@@ -2221,6 +2227,9 @@ static struct protection_domain *protection_domain_alloc(unsigned int type)case IOMMU_DOMAIN_UNMANAGED : pgtable =AMD_IOMMU_V1 ;break ; +case IOMMU_DOMAIN_SVA : + domain->domain.ops = &amd_svm_domain_ops; +return domain;default : goto out_err; }
@@ -2602,6 +2611,7 @@ const struct iommu_ops amd_iommu_ops = { .def_domain_type = amd_iommu_def_domain_type, .dev_enable_feat = amd_iommu_dev_enable_feature, .dev_disable_feat = amd_iommu_dev_disable_feature, + .remove_dev_pasid = amd_iommu_remove_dev_pasid, .default_domain_ops = &(const struct iommu_domain_ops ) { .attach_dev = amd_iommu_attach_device, .map_pages = amd_iommu_map_pages,
drivers/iommu/amd/sva.c new file +299 -0
299 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Quoted 44 lines · Vasant Hegde
drivers/iommu/amd/sva.c new file +38
-0@@ -0,0 +1,299 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2023 Advanced Micro Devices, Inc. +*/ + +#define pr_fmt (fmt) "AMD-Vi: " fmt +#define dev_fmt (fmt) pr_fmt(fmt) + +#include <linux/iommu.h> +#include <linux/mm_types.h> +#include <linux/mmu_notifier.h> + +#include "amd_iommu.h" +#include "../iommu-sva.h" + +struct amd_sva_pasid { +u32 pasid;/* PASID index */ +struct mm_struct *mm;/* mm_struct for the faults */ +struct mmu_notifier mn;/* mmu_notifier handle */ +struct list_head dev_list;/* List of devices for this pasid */ +}; + +struct amd_sva_dev { +struct device *dev; +struct iommu_dev_data *dev_data; +struct list_head list; +struct rcu_head rcu; +}; + +static DEFINE_MUTEX (pasid_mutex); +static DEFINE_XARRAY_ALLOC (sva_pasid_array); + + +static int sva_pasid_private_add (u32 pasid,void *priv) +{ +return xa_alloc (&sva_pasid_array, &pasid, priv, +XA_LIMIT (pasid, pasid),GFP_ATOMIC ); +}
drivers/iommu/amd/sva.c line 48
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+ +static void sva_pasid_private_remove (u32 pasid) +{ +xa_erase (&sva_pasid_array, pasid); +} + +static void *sva_pasid_private_find (u32 pasid) +{ +return xa_load (&sva_pasid_array, pasid); +}
No PASID stuff in SVA code at all please, all of this is wrong.
The only PASID comes from here, and it should be the only place PASID shows up:
+int amd_iommu_set_dev_pasid(struct iommu_domain *domain,
+ struct device *dev, ioasid_t pasid)
+{
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+static int sva_bind_mm (struct device *dev,struct mm_struct *mm) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +int ret = -EINVAL ;
drivers/iommu/amd/sva.c line 181
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+ + sva_dev =sva_dev_alloc (dev); +if (!sva_dev) +return ret; + + sva_pasid =sva_pasid_private_find (mm->pasid); +if (!sva_pasid) { + sva_pasid =sva_pasid_alloc (mm); +if (!sva_pasid) + goto out_sva_dev;
No per-PASID struct. AMD enablement should go after this series:
https://lore.kernel.org/linux-iommu/[email protected]/
Put the mmu_notifier directly into the protection_domain. Assume you have a single protection_domain per mm.
Also amd_sva_dev is not appropriate, the list of PASIDs (and masters) a domain is associated with is part of the generic PASID support in the protection_domain itself.
These details would be clearer if you start from enabling PASID support for an UNMANAGED domain.
SVA should be a tiny incremental from that which simply calls the same invalidation and manages the mmu notifier.
Quoted 32 lines
drivers/iommu/amd/sva.c new file +32
-0+static struct amd_sva_dev *sva_dev_alloc (struct device *dev) +{ +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + + sva_dev =kzalloc (sizeof (*sva_dev),GFP_KERNEL ); +if (!sva_dev) +return NULL ; + + sva_dev->dev = dev; + sva_dev->dev_data = dev_data; +init_rcu_head (&sva_dev->rcu); + +return sva_dev; +} + +static inline void sva_dev_free (struct amd_sva_dev *sva_dev) +{ +kfree_rcu (sva_dev, rcu); +} + +static void sva_mn_invalidate_range (struct mmu_notifier *mn, +struct mm_struct *mm, +unsigned long start,unsigned long end) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; + +rcu_read_lock (); + + sva_pasid = container_of(mn, struct amd_sva_pasid, mn); +if (!sva_pasid) {
drivers/iommu/amd/sva.c line 91
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+rcu_read_unlock (); +return ; + } + +list_for_each_entry_rcu (sva_dev, &sva_pasid->dev_list, list) { +if ((start ^ (end -1 )) <PAGE_SIZE ) +amd_iommu_flush_page (sva_dev->dev_data->domain, sva_pasid->pasid, start); +else +amd_iommu_flush_tlb (sva_dev->dev_data->domain, sva_pasid->pasid); + }
SVA invalidation should be the same as normal PASID invalidation. You need to track the list of PASIDs a protection_domain is associated with in the protection domain itself, not in special SVA code. Don't repeat the SMMU mistakes please.
Use container_of(mn) to get back to the SVA protection domain and then you can access the protection domains list of PASIDs & masters.
Quoted 20 lines
drivers/iommu/amd/sva.c new file +20
-0+static int sva_bind_mm (struct device *dev,struct mm_struct *mm) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +int ret = -EINVAL ; + + sva_dev =sva_dev_alloc (dev); +if (!sva_dev) +return ret; + + sva_pasid =sva_pasid_private_find (mm->pasid); +if (!sva_pasid) { + sva_pasid =sva_pasid_alloc (mm); +if (!sva_pasid) + goto out_sva_dev; + + ret =sva_pasid_private_add (sva_pasid->pasid, sva_pasid); +if (ret) + goto out_sva_pasid;
drivers/iommu/amd/sva.c line 195
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+ + ret =amd_iommu_set_gcr3 (dev_data, sva_pasid->pasid, +iommu_virt_to_phys (sva_pasid->mm->pgd)); +if (ret) + goto out_pasid_remove; + + ret =mmu_notifier_register (&sva_pasid->mn, mm); +if (ret) + goto out_clear_gcr3; + }
The mmu_notifier should be setup when the domain is allocated, not during bind.
This is an issue we need to fix in the core code :(
drivers/iommu/amd/sva.c line 278
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+void amd_iommu_remove_dev_pasid (struct device *dev,ioasid_t pasid) +{ +struct iommu_domain *domain; + +if (pasid ==0 || pasid >= dev->iommu->max_pasids) +return ;
We should probably have the core code pass in the old domain to this function, it is looking more like a mistake we didn't do that.
Quoted 4 lines
drivers/iommu/amd/sva.c new file +4
-0+ +/* Get SVA domain */ + domain =iommu_get_domain_for_dev_pasid (dev, pasid,0 ); +if (!domain)
drivers/iommu/amd/sva.c line 292
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+return ; + +switch (domain->type) { +case IOMMU_DOMAIN_SVA : +/* Ensure that all queued faults have been processed */ +iopf_queue_flush_dev (dev); + +mutex_lock (&pasid_mutex); +sva_unbind_mm (dev, pasid); +mutex_unlock (&pasid_mutex);
SVA should not be special for detach, this should all be generic code.
The mmu notifier is freed during SVA domain dealloc.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Jason,
Quoted 49 lines · Jason Gunthorpe
Quoted 48 lines · Vasant Hegde
drivers/iommu/amd/sva.c new file +42
-0@@ -0,0 +1,299 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2023 Advanced Micro Devices, Inc. +*/ + +#define pr_fmt (fmt) "AMD-Vi: " fmt +#define dev_fmt (fmt) pr_fmt(fmt) + +#include <linux/iommu.h> +#include <linux/mm_types.h> +#include <linux/mmu_notifier.h> + +#include "amd_iommu.h" +#include "../iommu-sva.h" + +struct amd_sva_pasid { +u32 pasid;/* PASID index */ +struct mm_struct *mm;/* mm_struct for the faults */ +struct mmu_notifier mn;/* mmu_notifier handle */ +struct list_head dev_list;/* List of devices for this pasid */ +}; + +struct amd_sva_dev { +struct device *dev; +struct iommu_dev_data *dev_data; +struct list_head list; +struct rcu_head rcu; +}; + +static DEFINE_MUTEX (pasid_mutex); +static DEFINE_XARRAY_ALLOC (sva_pasid_array); + + +static int sva_pasid_private_add (u32 pasid,void *priv) +{ +return xa_alloc (&sva_pasid_array, &pasid, priv, +XA_LIMIT (pasid, pasid),GFP_ATOMIC ); +} + +static void sva_pasid_private_remove (u32 pasid) +{ +xa_erase (&sva_pasid_array, pasid);
Quoted 8 lines
drivers/iommu/amd/sva.c line 48
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+} + +static void *sva_pasid_private_find (u32 pasid) +{ +return xa_load (&sva_pasid_array, pasid); +}No PASID stuff in SVA code at all please, all of this is wrong.
This is based on upstream code!
Quoted 27 lines
The only PASID comes from here, and it should be the only place PASID shows up:
+int amd_iommu_set_dev_pasid(struct
iommu_domain*domain,
+ struct device *dev,ioasid_tpasid)
+{Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+static int sva_bind_mm (struct device *dev,struct mm_struct *mm) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +int ret = -EINVAL ;drivers/iommu/amd/sva.c line 181
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+ + sva_dev =sva_dev_alloc (dev); +if (!sva_dev) +return ret; + + sva_pasid =sva_pasid_private_find (mm->pasid); +if (!sva_pasid) { + sva_pasid =sva_pasid_alloc (mm); +if (!sva_pasid) + goto out_sva_dev;No per-PASID struct. AMD enablement should go after this series:
Quoted 4 lines
https://lore.kernel.org/linux-iommu/[email protected]/
Put the
mmu_notifierdirectly into theprotection_domain. Assume you have a singleprotection_domainper mm.
Actually we have protection domain for each IOMMU group (or multiple group in case of VM). Our invalidation needs device protection domain ID (not the SVA domain ID which is created during device/PASID binding).
Invalidation takes three parameter :
Device Domain ID, PASID, Address
Quoted 4 lines
Also
amd_sva_devis not appropriate, the list of PASIDs (and masters) a domain is associated with is part of the generic PASID support in theprotection_domainitself.
You mean to say, we maintain PASID list in device protection_domain and then in invalidation path (somehow) we retrieve protection domain and use it for invalidation? -OR- someway establish link between SVA domain to base device protection domain?
Quoted 3 lines
These details would be clearer if you start from enabling PASID support for an UNMANAGED domain.
We don't support PASID with V1 page table.
Quoted 3 lines
SVA should be a tiny incremental from that which simply calls the same invalidation and manages the mmu notifier.
Yeah. If we can get a way to retrieve device protection domain ID (not SVA protection domain) in notifier path then it makes life easy. But currently I don't see a way to do that.
Quoted 37 lines
Quoted 37 lines
drivers/iommu/amd/sva.c new file +36
-0+static struct amd_sva_dev *sva_dev_alloc (struct device *dev) +{ +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + + sva_dev =kzalloc (sizeof (*sva_dev),GFP_KERNEL ); +if (!sva_dev) +return NULL ; + + sva_dev->dev = dev; + sva_dev->dev_data = dev_data; +init_rcu_head (&sva_dev->rcu); + +return sva_dev; +} + +static inline void sva_dev_free (struct amd_sva_dev *sva_dev) +{ +kfree_rcu (sva_dev, rcu); +} + +static void sva_mn_invalidate_range (struct mmu_notifier *mn, +struct mm_struct *mm, +unsigned long start,unsigned long end) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; + +rcu_read_lock (); + + sva_pasid = container_of(mn, struct amd_sva_pasid, mn); +if (!sva_pasid) { +rcu_read_unlock (); +return ; + } +
Quoted 8 lines
drivers/iommu/amd/sva.c line 91
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+list_for_each_entry_rcu (sva_dev, &sva_pasid->dev_list, list) { +if ((start ^ (end -1 )) <PAGE_SIZE ) +amd_iommu_flush_page (sva_dev->dev_data->domain, sva_pasid->pasid, start); +else +amd_iommu_flush_tlb (sva_dev->dev_data->domain, sva_pasid->pasid); + }SVA invalidation should be the same as normal PASID invalidation. You
Its same (Currently we have two different functions based on PAGE_SIZE. We have a separate series to improve our invalidation logic).
Quoted 6 lines
need to track the list of PASIDs a
protection_domainis associated with in the protection domain itself, not in special SVA code. Don't repeat the SMMU mistakes please.Use
container_of(mn)to get back to the SVA protection domain and then you can access the protection domains list of PASIDs & masters.
I don't think I understood this. Even with Tina's patch series, we can get the SVA domain list. But we don't have a way to get device base protection domain.
Quoted 26 lines
Quoted 26 lines
drivers/iommu/amd/sva.c new file +25
-0+static int sva_bind_mm (struct device *dev,struct mm_struct *mm) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +int ret = -EINVAL ; + + sva_dev =sva_dev_alloc (dev); +if (!sva_dev) +return ret; + + sva_pasid =sva_pasid_private_find (mm->pasid); +if (!sva_pasid) { + sva_pasid =sva_pasid_alloc (mm); +if (!sva_pasid) + goto out_sva_dev; + + ret =sva_pasid_private_add (sva_pasid->pasid, sva_pasid); +if (ret) + goto out_sva_pasid; + + ret =amd_iommu_set_gcr3 (dev_data, sva_pasid->pasid, +iommu_virt_to_phys (sva_pasid->mm->pgd)); +if (ret) + goto out_pasid_remove;
Quoted 8 lines
drivers/iommu/amd/sva.c line 195
Quoted 5 lines
drivers/iommu/amd/sva.c new file +5
-0+ + ret =mmu_notifier_register (&sva_pasid->mn, mm); +if (ret) + goto out_clear_gcr3; + }The
mmu_notifiershould be setup when the domain is allocated, not during bind.
We need to program (at least with AMD IOMMU) device PASID table before it can handle the invalidation notifications. Not sure how it will work if we move mmu notifier setup to domain allocation path.
Quoted 12 lines
This is an issue we need to fix in the core code :(
drivers/iommu/amd/sva.c line 278
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+void amd_iommu_remove_dev_pasid (struct device *dev,ioasid_t pasid) +{ +struct iommu_domain *domain; + +if (pasid ==0 || pasid >= dev->iommu->max_pasids) +return ;We should probably have the core code pass in the old domain to this function, it is looking more like a mistake we didn't do that.
I don't think I understood this. Well, honestly I never understood why remove_dev_pasid() is part of iommu_ops while set_dev_pasid is part of domain ops.
My understanding is in this path we necessary things like in AMD case updating device table entry and finally remove mmu notifier.
-Vasant
Quoted 21 lines
Quoted 5 lines
drivers/iommu/amd/sva.c new file +4
-0+ +/* Get SVA domain */ + domain =iommu_get_domain_for_dev_pasid (dev, pasid,0 ); +if (!domain)drivers/iommu/amd/sva.c line 292
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+return ; + +switch (domain->type) { +case IOMMU_DOMAIN_SVA : +/* Ensure that all queued faults have been processed */ +iopf_queue_flush_dev (dev); + +mutex_lock (&pasid_mutex); +sva_unbind_mm (dev, pasid); +mutex_unlock (&pasid_mutex);SVA should not be special for detach, this should all be generic code.
The mmu notifier is freed during SVA domain dealloc.
Jason
Re: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Quoted 8 lines · Vasant Hegde
Quoted 6 lines
drivers/iommu/amd/sva.c line 48
Quoted 4 lines
drivers/iommu/amd/sva.c new file +4
-0+static void *sva_pasid_private_find (u32 pasid) +{ +return xa_load (&sva_pasid_array, pasid); +}No PASID stuff in SVA code at all please, all of this is wrong.
This is based on upstream code!
Yes, and we are changing all of this because how wrong the drivers went with stuff like this. :(
Quoted 9 lines
Quoted 6 lines
No per-PASID struct. AMD enablement should go after this series:
https://lore.kernel.org/linux-iommu/[email protected]/
Put the
mmu_notifierdirectly into theprotection_domain. Assume you have a singleprotection_domainper mm.Actually we have protection domain for each IOMMU group (or multiple group in case of VM).
Why would you do that for SVA? That is not the direction we are going. One iommu_domain per mm, managed by the core code.
Driver assumes this and optimizes based on it.
If driver needs per device/pasid/whatever data then it keeps only that data in a list hanging off the single iommu_domain.
It should be *exactly the same* data that an unmanaged domain needs to manage its invalidations and ATC.
Quoted 6 lines
Quoted 3 lines
Also
amd_sva_devis not appropriate, the list of PASIDs (and masters) a domain is associated with is part of the generic PASID support in theprotection_domainitself.You mean to say, we maintain PASID list in device
protection_domainand then in
yes, a linked list of devices RID and PASIDs that the domain is attached to.
This is mandatory to issue any form of invalidation, an UNMANAGED domains should use the same list, and same mechanism to invalidate their PASID attachments as well.
Quoted 2 lines
invalidation path (somehow) we retrieve protection domain and use it for
Yes, obtain the domain trivially via container_of(mmu_notifier). Store the notifier in the SVA iommu_domain's driver struct (protection_domain) to do this.
This is why the core code helps the driver by de-duplicating the SVA domains, it can assume the iommu_domain is already minimal and it can then safely place the notifier there. The drivers should not try to de-duplicate the notifier with refcounting/etc.
Quoted 4 lines
Quoted 2 lines
These details would be clearer if you start from enabling PASID support for an UNMANAGED domain.
We don't support PASID with V1 page table.
I didn't say V1 page table, I said enabling PASID support for UNMANGED domains. This means you need a flavour of UNMANAGED domain that is V2 page table. Just like ARM does. You already have this support in the driver.
Quoted 24 lines
Quoted 15 lines
Quoted 15 lines
drivers/iommu/amd/sva.c new file +15
-0+static void sva_mn_invalidate_range (struct mmu_notifier *mn, +struct mm_struct *mm, +unsigned long start,unsigned long end) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; + +rcu_read_lock (); + + sva_pasid = container_of(mn, struct amd_sva_pasid, mn); +if (!sva_pasid) { +rcu_read_unlock (); +return ; + } +Quoted 8 lines
drivers/iommu/amd/sva.c line 91
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+list_for_each_entry_rcu (sva_dev, &sva_pasid->dev_list, list) { +if ((start ^ (end -1 )) <PAGE_SIZE ) +amd_iommu_flush_page (sva_dev->dev_data->domain, sva_pasid->pasid, start); +else +amd_iommu_flush_tlb (sva_dev->dev_data->domain, sva_pasid->pasid); + }SVA invalidation should be the same as normal PASID invalidation. You
Quoted 2 lines
Its same (Currently we have two different functions based on
PAGE_SIZE. We have a separate series to improve our invalidation logic).
It is not the same, you have this weird sva_pasid thing in here. PASID is NOT part of the SVA layer.
The API expects UNMANAGED domains will support PASID attach as well, that is a significant use case.
Quoted 5 lines
Quoted 2 lines
Use
container_of(mn)to get back to the SVA protection domain and then you can access the protection domains list of PASIDs & masters.I don't think I understood this. Even with Tina's patch series, we can get the SVA domain list. But we don't have a way to get device base protection domain.
Tina's patch series allows you to place the mmu_notifier struct directly in the protection_domain struct.
When you get the invalidate() callback you go back to the iommu_domain/protection domain that is affiliated with this MM.
From there you have all the information you can possibly need:
- Global information in the protect_domain shared by all the devices (eg ARM uses this to put the shared cache tag)
- Per attachment information (RID & PASID/etc) for each attachment. Iterate over this list to get the PASID/etc.
Quoted 34 lines
Quoted 25 lines
Quoted 25 lines
drivers/iommu/amd/sva.c new file +25
-0+static int sva_bind_mm (struct device *dev,struct mm_struct *mm) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +int ret = -EINVAL ; + + sva_dev =sva_dev_alloc (dev); +if (!sva_dev) +return ret; + + sva_pasid =sva_pasid_private_find (mm->pasid); +if (!sva_pasid) { + sva_pasid =sva_pasid_alloc (mm); +if (!sva_pasid) + goto out_sva_dev; + + ret =sva_pasid_private_add (sva_pasid->pasid, sva_pasid); +if (ret) + goto out_sva_pasid; + + ret =amd_iommu_set_gcr3 (dev_data, sva_pasid->pasid, +iommu_virt_to_phys (sva_pasid->mm->pgd)); +if (ret) + goto out_pasid_remove;Quoted 8 lines
drivers/iommu/amd/sva.c line 195
Quoted 5 lines
drivers/iommu/amd/sva.c new file +5
-0+ + ret =mmu_notifier_register (&sva_pasid->mn, mm); +if (ret) + goto out_clear_gcr3; + }The
mmu_notifiershould be setup when the domain is allocated, not during bind.
Quoted 3 lines
We need to program (at least with AMD IOMMU) device PASID table before it can handle the invalidation notifications. Not sure how it will work if we move mmu notifier setup to domain allocation path.
The device list will be empty so there will be no PASIDs to invalidate at this point. There will be seperate locking to protect the device list, ensure that locking properly covers populating the device list and setting up the HW to respond to the PASID.
Quoted 10 lines
Quoted 9 lines
drivers/iommu/amd/sva.c line 278
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+void amd_iommu_remove_dev_pasid (struct device *dev,ioasid_t pasid) +{ +struct iommu_domain *domain; + +if (pasid ==0 || pasid >= dev->iommu->max_pasids) +return ;We should probably have the core code pass in the old domain to this function, it is looking more like a mistake we didn't do that.
Quoted 2 lines
I don't think I understood this. Well, honestly I never understood why
remove_dev_pasid()is part ofiommu_opswhileset_dev_pasidis part of domain ops.
Yeah, that is a bit weird, not sure there is a good reason
Regardless, the API should take in the current domain parmeter and drivers shouldn't call the core to look it up in the xarray if that is what drivers need to do.
Quoted 2 lines
My understanding is in this path we necessary things like in AMD case updating device table entry and finally remove mmu notifier.
You should remove the RID&PASID from the device list and flush out the HW caches under a lock.
mmu_notifier should be removed when the domain is deallocated.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Jason,
Quoted 49 lines · Jason Gunthorpe
Quoted 8 lines · Vasant Hegde
Quoted 6 lines
drivers/iommu/amd/sva.c line 48
Quoted 4 lines
drivers/iommu/amd/sva.c new file +4
-0+static void *sva_pasid_private_find (u32 pasid) +{ +return xa_load (&sva_pasid_array, pasid); +}No PASID stuff in SVA code at all please, all of this is wrong.
This is based on upstream code!
Yes, and we are changing all of this because how wrong the drivers went with stuff like this. :(
Quoted 9 lines
Quoted 6 lines
No per-PASID struct. AMD enablement should go after this series:
https://lore.kernel.org/linux-iommu/[email protected]/
Put the
mmu_notifierdirectly into theprotection_domain. Assume you have a singleprotection_domainper mm.Actually we have protection domain for each IOMMU group (or multiple group in case of VM).
Why would you do that for SVA? That is not the direction we are going. One
iommu_domainper mm, managed by the core code.Driver assumes this and optimizes based on it.
If driver needs per device/pasid/whatever data then it keeps only that data in a list hanging off the single
iommu_domain.It should be *exactly the same* data that an unmanaged domain needs to manage its invalidations and ATC.
Quoted 6 lines
Quoted 3 lines
Also
amd_sva_devis not appropriate, the list of PASIDs (and masters) a domain is associated with is part of the generic PASID support in theprotection_domainitself.You mean to say, we maintain PASID list in device
protection_domainand then inyes, a linked list of devices RID and PASIDs that the domain is attached to.
This is mandatory to issue any form of invalidation, an UNMANAGED domains should use the same list, and same mechanism to invalidate their PASID attachments as well.
Quoted 6 lines
Quoted 2 lines
invalidation path (somehow) we retrieve protection domain and use it for
Yes, obtain the domain trivially via
container_of(mmu_notifier). Store the notifier in the SVA iommu_domain's driver struct (protection_domain) to do this.
Reading through the discussion so far again and the other series, my understanding is :
- set_dev_pasid() will check the compatibility and bind device/pasid only if
its compatibility. In AMD case we will check against protection domain. Ex:
If we have two devices (devA and devB) in two different protection domain then:
set_dev_pasid(sva_domain, devA, pasidX) - SUCCESS set_dev_pasid(sva_domain, devB, pasidX) - Compatibility check fail Core will allocate new SVA domain (sva_domain_new) set_dev_pasid(sva_domain_new, devB, pasidX) - SUCCESS
- We will track mmu notifier and other data required for invalidation in SVA
protection domain.
- During invalidation, we will retrieve SVA protection domain using mmu
notifier. Use device protection domain which was tracked in this SVA domain for invalidation.
Does above flow makes sense? I do have a code based on above flow. I will try to post soon.
Quoted 40 lines
This is why the core code helps the driver by de-duplicating the SVA domains, it can assume the
iommu_domainis already minimal and it can then safely place the notifier there. The drivers should not try to de-duplicate the notifier with refcounting/etc.Quoted 4 lines
Quoted 2 lines
These details would be clearer if you start from enabling PASID support for an UNMANAGED domain.
We don't support PASID with V1 page table.
I didn't say V1 page table, I said enabling PASID support for UNMANGED domains. This means you need a flavour of UNMANAGED domain that is V2 page table. Just like ARM does. You already have this support in the driver.
Quoted 24 lines
Quoted 23 lines
Quoted 11 lines
drivers/iommu/amd/sva.c new file +11
-0+static void sva_mn_invalidate_range (struct mmu_notifier *mn, +struct mm_struct *mm, +unsigned long start,unsigned long end) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; + +rcu_read_lock (); + + sva_pasid = container_of(mn, struct amd_sva_pasid, mn); +if (!sva_pasid) {drivers/iommu/amd/sva.c line 91
Quoted 10 lines
drivers/iommu/amd/sva.c new file +10
-0+rcu_read_unlock (); +return ; + } + +list_for_each_entry_rcu (sva_dev, &sva_pasid->dev_list, list) { +if ((start ^ (end -1 )) <PAGE_SIZE ) +amd_iommu_flush_page (sva_dev->dev_data->domain, sva_pasid->pasid, start); +else +amd_iommu_flush_tlb (sva_dev->dev_data->domain, sva_pasid->pasid); + }SVA invalidation should be the same as normal PASID invalidation. You
Quoted 8 lines
Quoted 2 lines
Its same (Currently we have two different functions based on
PAGE_SIZE. We have a separate series to improve our invalidation logic).It is not the same, you have this weird
sva_pasidthing in here. PASID is NOT part of the SVA layer.The API expects UNMANAGED domains will support PASID attach as well, that is a significant use case.
Can you elaborate the use cases you are referring here?
We do have use cases for PASID and PASID+PRI. But I am not aware of any use case for UNMANAGED domain.
Quoted 13 lines
Quoted 6 lines
Quoted 3 lines
Use
container_of(mn)to get back to the SVA protection domain and then you can access the protection domains list of PASIDs & masters.I don't think I understood this. Even with Tina's patch series, we can get the SVA domain list. But we don't have a way to get device base protection domain.
Tina's patch series allows you to place the
mmu_notifierstruct directly in theprotection_domainstruct.When you get the
invalidate()callback you go back to the iommu_domain/protection domain that is affiliated with this MM.
Quoted 7 lines
From there you have all the information you can possibly need:
- Global information in the
protect_domainshared by all the devices (eg ARM uses this to put the shared cache tag)- Per attachment information (RID & PASID/etc) for each attachment. Iterate over this list to get the PASID/etc.
Got it. Thanks.
Quoted 54 lines
Quoted 35 lines
Quoted 26 lines
Quoted 26 lines
drivers/iommu/amd/sva.c new file +25
-0+static int sva_bind_mm (struct device *dev,struct mm_struct *mm) +{ +struct amd_sva_pasid *sva_pasid; +struct amd_sva_dev *sva_dev; +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +int ret = -EINVAL ; + + sva_dev =sva_dev_alloc (dev); +if (!sva_dev) +return ret; + + sva_pasid =sva_pasid_private_find (mm->pasid); +if (!sva_pasid) { + sva_pasid =sva_pasid_alloc (mm); +if (!sva_pasid) + goto out_sva_dev; + + ret =sva_pasid_private_add (sva_pasid->pasid, sva_pasid); +if (ret) + goto out_sva_pasid; + + ret =amd_iommu_set_gcr3 (dev_data, sva_pasid->pasid, +iommu_virt_to_phys (sva_pasid->mm->pgd)); +if (ret) + goto out_pasid_remove;Quoted 8 lines
drivers/iommu/amd/sva.c line 195
Quoted 5 lines
drivers/iommu/amd/sva.c new file +5
-0+ + ret =mmu_notifier_register (&sva_pasid->mn, mm); +if (ret) + goto out_clear_gcr3; + }The
mmu_notifiershould be setup when the domain is allocated, not during bind.Quoted 3 lines
We need to program (at least with AMD IOMMU) device PASID table before it can handle the invalidation notifications. Not sure how it will work if we move mmu notifier setup to domain allocation path.
The device list will be empty so there will be no PASIDs to invalidate at this point. There will be seperate locking to protect the device list, ensure that locking properly covers populating the device list and setting up the HW to respond to the PASID.
Quoted 10 lines
Quoted 9 lines
drivers/iommu/amd/sva.c line 278
Quoted 6 lines
drivers/iommu/amd/sva.c new file +6
-0+void amd_iommu_remove_dev_pasid (struct device *dev,ioasid_t pasid) +{ +struct iommu_domain *domain; + +if (pasid ==0 || pasid >= dev->iommu->max_pasids) +return ;We should probably have the core code pass in the old domain to this function, it is looking more like a mistake we didn't do that.
Quoted 8 lines
Quoted 2 lines
I don't think I understood this. Well, honestly I never understood why
remove_dev_pasid()is part ofiommu_opswhileset_dev_pasidis part of domain ops.Yeah, that is a bit weird, not sure there is a good reason
Regardless, the API should take in the current domain parmeter and drivers shouldn't call the core to look it up in the xarray if that is what drivers need to do.
Yeah. All I care is retrieving SVA domain. Using SVA domain I can get the protection domain and do rest of the stuff.
With all new changes, it makes sense to pass sva domain to remove_dev_pasid.
-Vasant
Re: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Quoted 9 lines · Vasant Hegde
Reading through the discussion so far again and the other series, my understanding is :
-set_dev_pasid()will check the compatibility and bind device/pasid only if
its compatibility. In AMD case we will check against protection domain. Ex:
If we have two devices (devA and devB) in two different protection domain then:
set_dev_pasid(sva_domain, devA, pasidX)- SUCCESSset_dev_pasid(sva_domain, devB, pasidX)- Compatibility check fail Core will allocate new SVA domain (sva_domain_new)set_dev_pasid(sva_domain_new, devB, pasidX)- SUCCESS
I don't expect a compatability check to ever fail on an AMD driver - what condition do you imagine where re-allocating a SVA domain will make it work with a device?
Quoted 2 lines
- We will track mmu notifier and other data required for invalidation in SVA
protection domain.
Yes
Quoted 3 lines
- During invalidation, we will retrieve SVA protection domain using mmu
notifier. Use device protection domain which was tracked in this SVA domain for invalidation.
The iommu_domain/protection_domain must NOT be 1:1 with a device. The driver must maintain a list of devices attached to the domain, and the per-device-attachment parameters like PASID/cache tags/etc.
This is very important.
Quoted 10 lines
Quoted 5 lines
It is not the same, you have this weird
sva_pasidthing in here. PASID is NOT part of the SVA layer.The API expects UNMANAGED domains will support PASID attach as well, that is a significant use case.
Can you elaborate the use cases you are referring here?
We do have use cases for PASID and PASID+PRI. But I am not aware of any use case for UNMANAGED domain.
The iommu API is evolving so there are only a few domain types:
BLOCKED, IDENTITY, PAGING, SVA, OPAQUE
PAGING is what we today call UNMANAGED/DMA/DMA_FQ
PAGING domains need to support PASID+PRI, in today's language that means UNMANAGED domains.
We have many use cases for PRI support with generic PAGING domains.
SVA is a special case of a PAGING domain where there is no map/unmap/invalidate API and the IO page table comes from a mm_struct.
Start by making UNMANAGED (aka PAGING) domains work with PASID + PRI and then SVA is a very small incremental step.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Jason,
Quoted 9 lines · Jason Gunthorpe
Quoted 7 lines · Vasant Hegde
Reading through the discussion so far again and the other series, my understanding is :
-set_dev_pasid()will check the compatibility and bind device/pasid only if
its compatibility. In AMD case we will check against protection domain. Ex:
If we have two devices (devA and devB) in two different protection domain then:
set_dev_pasid(sva_domain, devA, pasidX)- SUCCESSset_dev_pasid(sva_domain, devB, pasidX)- Compatibility check fail
Here I expect it to fails as devB is in different protection domain. From invalidation point of view, devA and devB are not compatible. Hence I think it should fail binding.
Quoted 6 lines
Quoted 2 lines
Core will allocate new SVA domain (
sva_domain_new)set_dev_pasid(sva_domain_new, devB, pasidX)- SUCCESSI don't expect a compatability check to ever fail on an AMD driver - what condition do you imagine where re-allocating a SVA domain will make it work with a device?
In above scenario. Essentially we will have 1 x 1 mapping between SVA domain to device protection domain (Here I am referring to the domain that we attached in attach_device() path).
Quoted 5 lines
Quoted 2 lines
- We will track mmu notifier and other data required for invalidation in SVA
protection domain.Yes
Quoted 7 lines
Quoted 3 lines
- During invalidation, we will retrieve SVA protection domain using mmu
notifier. Use device protection domain which was tracked in this SVA domain for invalidation.The iommu_domain/protection_domain must NOT be 1:1 with a device. The driver must maintain a list of devices attached to the domain, and the per-device-attachment parameters like PASID/cache tags/etc.
We want to track SVA protection domain to device protection domain link. So that invalidation becomes straight.
If we track dev/PASID then I am not sure how we can solve duplicate invalidation issue that we have today (i. e. if we have two devices within same protection domain and if we track dev/pasid combinatin, then will call invalidation twice).
Quoted 11 lines
This is very important.
Quoted 8 lines
Quoted 5 lines
It is not the same, you have this weird
sva_pasidthing in here. PASID is NOT part of the SVA layer.The API expects UNMANAGED domains will support PASID attach as well, that is a significant use case.
Can you elaborate the use cases you are referring here?
Quoted 7 lines
Quoted 2 lines
We do have use cases for PASID and PASID+PRI. But I am not aware of any use case for UNMANAGED domain.
The iommu API is evolving so there are only a few domain types:
BLOCKED, IDENTITY, PAGING, SVA, OPAQUEPAGING is what we today call UNMANAGED/DMA/DMA_FQ
Ok.
Quoted 5 lines
PAGING domains need to support PASID+PRI, in today's language that means UNMANAGED domains.
We have many use cases for PRI support with generic PAGING domains.
What is that usecase? That's what I am trying to understand it better.
Quoted 6 lines
SVA is a special case of a PAGING domain where there is no map/unmap/invalidate API and the IO page table comes from a
mm_struct.Start by making UNMANAGED (aka PAGING) domains work with PASID + PRI and then SVA is a very small incremental step.
If there is a real use case then why not? We can do that. But for now we want the proper SVA to go first.
-Vasant
Re: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Quoted 9 lines · Vasant Hegde
Quoted 5 lines
Quoted 5 lines
-
set_dev_pasid()will check the compatibility and bind device/pasid only if
its compatibility. In AMD case we will check against protection domain. Ex:
If we have two devices (devA and devB) in two different protection domain then:
set_dev_pasid(sva_domain, devA, pasidX)- SUCCESSset_dev_pasid(sva_domain, devB, pasidX)- Compatibility check failHere I expect it to fails as devB is in different protection domain. From invalidation point of view, devA and devB are not compatible. Hence I think it should fail binding.
No, that isn't the best thing to do - you could do this, but it will be more inefficient.
The domain should represent the ioptes not the target of invalidation.
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
- During invalidation, we will retrieve SVA protection domain using mmu
notifier. Use device protection domain which was tracked in this SVA domain for invalidation.The iommu_domain/protection_domain must NOT be 1:1 with a device. The driver must maintain a list of devices attached to the domain, and the per-device-attachment parameters like PASID/cache tags/etc.
Quoted 5 lines
We want to track SVA protection domain to device protection domain link. So that invalidation becomes straight.
If we track dev/PASID then I am not sure how we can solve duplicate invalidation issue that we have today (i. e. if we have two devices within same protection domain and if we track dev/pasid combinatin, then will call invalidation twice).
Look at what Michael is doing for SMMUv3, they have a per-smmu instance cache tag and a per-device ATC invalidation they need to issue. They keep a sorted list of device attachments and simply do one invalidation per-smmu instance and one invalidation per entry.
It is pretty simple logic.
Quoted 6 lines
Quoted 4 lines
PAGING domains need to support PASID+PRI, in today's language that means UNMANAGED domains.
We have many use cases for PRI support with generic PAGING domains.
What is that usecase? That's what I am trying to understand it better.
iommufd will require this for generic vSVA in all paths that don't use nesting.
We have use cases to share a KVM page table with PRI.
Google apparently has some usecase since they are fixing it in ARM.
Besides that, it is the IOMMU API, drivers have to implement it, you don't get to pick and choose.
Quoted 8 lines
Quoted 5 lines
SVA is a special case of a PAGING domain where there is no map/unmap/invalidate API and the IO page table comes from a
mm_struct.Start by making UNMANAGED (aka PAGING) domains work with PASID + PRI and then SVA is a very small incremental step.
If there is a real use case then why not? We can do that. But for now we want the proper SVA to go first.
You are not going to get SVA without also properly doing all infrastructure to enable paging and unmanaged - I won't support another shortcut hackjob for SVA that needs unwinding like ARM has.
PASID and PRI must not be tightly linked to SVA.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Jason,
Quoted 7 lines · Jason Gunthorpe
Quoted 6 lines · Vasant Hegde
Quoted 5 lines
Quoted 5 lines
-
set_dev_pasid()will check the compatibility and bind device/pasid only if
its compatibility. In AMD case we will check against protection domain. Ex:
If we have two devices (devA and devB) in two different protection domain then:
set_dev_pasid(sva_domain, devA, pasidX)- SUCCESSset_dev_pasid(sva_domain, devB, pasidX)- Compatibility check fail
Quoted 6 lines
Quoted 3 lines
Here I expect it to fails as devB is in different protection domain. From invalidation point of view, devA and devB are not compatible. Hence I think it should fail binding.
No, that isn't the best thing to do - you could do this, but it will be more inefficient.
What inefficiency are you referring? Allocating extra SVA domain -OR- mmu notifier getting called for each SVA domain -OR- something else?
Quoted 2 lines
The domain should represent the ioptes not the target of invalidation.
I agree with this and I have done SVA domain implementation for AMD driver in v2 [1].
Then I am wondering why not just create single SVA domain for a PASID and let individual driver handle underneath requirement like target invalidation? Am I missing here?
Quoted 32 lines
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines
- During invalidation, we will retrieve SVA protection domain using mmu
notifier. Use device protection domain which was tracked in this SVA domain for invalidation.The iommu_domain/protection_domain must NOT be 1:1 with a device. The driver must maintain a list of devices attached to the domain, and the per-device-attachment parameters like PASID/cache tags/etc.
Quoted 5 lines
We want to track SVA protection domain to device protection domain link. So that invalidation becomes straight.
If we track dev/PASID then I am not sure how we can solve duplicate invalidation issue that we have today (i. e. if we have two devices within same protection domain and if we track dev/pasid combinatin, then will call invalidation twice).Look at what Michael is doing for SMMUv3, they have a per-smmu instance cache tag and a per-device ATC invalidation they need to issue. They keep a sorted list of device attachments and simply do one invalidation per-smmu instance and one invalidation per entry.
It is pretty simple logic.
Quoted 6 lines
Quoted 4 lines
PAGING domains need to support PASID+PRI, in today's language that means UNMANAGED domains.
We have many use cases for PRI support with generic PAGING domains.
What is that usecase? That's what I am trying to understand it better.
iommufd will require this for generic vSVA in all paths that don't use nesting.
Quoted 6 lines
We have use cases to share a KVM page table with PRI.
Google apparently has some usecase since they are fixing it in ARM.
Besides that, it is the IOMMU API, drivers have to implement it, you don't get to pick and choose.
Is there any documentation on the mentioned use case and the required IOMMU API because this is new to me. I would like to understand things better before making the change in AMD IOMMU driver.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
SVA is a special case of a PAGING domain where there is no map/unmap/invalidate API and the IO page table comes from a
mm_struct.Start by making UNMANAGED (aka PAGING) domains work with PASID + PRI and then SVA is a very small incremental step.
Quoted 6 lines
Quoted 2 lines
If there is a real use case then why not? We can do that. But for now we want the proper SVA to go first.
You are not going to get SVA without also properly doing all infrastructure to enable paging and unmanaged - I won't support another shortcut hackjob for SVA that needs unwinding like ARM has.
The goal of this series is to introduce the SVA support for AMD IOMMU driver.
So far we have been accommodating all the review comments.
- Dropping iommu_v2 module before getting SVA
- Reworking SVA top of Tina's series
- Reworking IOPF on top of Baolu's IOPF enhancement
Adding PASID support for the unmanaged domain is a new requirement and not relevant to this series. It will also require a considerable amount of changes in AMD IOMMU driver that we need to further investigate to better understand. For now, we would like to stay focus on getting SVA support in upstream before moving on to add PASID support for UNMANAGED domain.
I just posted v2 [1]. Please take a look.
[1]
https://lore.kernel.org/linux-iommu/[email protected]/T/#u
Quoted 2 lines
PASID and PRI must not be tightly linked to SVA.
Its not. Patchset already supports PASID and PASID+PRI combination.
-Vasant
Re: [PATCH RESEND 03/10] iommu/amd: Initial SVA support for AMD IOMMU
Quoted 14 lines · Vasant Hegde
Jason,
Quoted 11 lines · Jason Gunthorpe
Quoted 9 lines · Vasant Hegde
Quoted 5 lines
Quoted 5 lines
-
set_dev_pasid()will check the compatibility and bind device/pasid only if
its compatibility. In AMD case we will check against protection domain. Ex:
If we have two devices (devA and devB) in two different protection domain then:
set_dev_pasid(sva_domain, devA, pasidX)- SUCCESSset_dev_pasid(sva_domain, devB, pasidX)- Compatibility check failHere I expect it to fails as devB is in different protection domain. From invalidation point of view, devA and devB are not compatible. Hence I think it should fail binding.
Quoted 5 lines
Quoted 2 lines
No, that isn't the best thing to do - you could do this, but it will be more inefficient.
What inefficiency are you referring? Allocating extra SVA domain -OR- mmu notifier getting called for each SVA domain -OR- something else?
Primarily multiple mmu notifiers is not so great
Quoted 3 lines
Then I am wondering why not just create single SVA domain for a PASID and let individual driver handle underneath requirement like target invalidation? Am I missing here?
domains are not connected to PASIDs - PASID is part of the attachment.
We want drivers to support one domain per IOPTE table and allow all possible combinations of compatible attachments.
Quoted 11 lines
Quoted 6 lines
We have use cases to share a KVM page table with PRI.
Google apparently has some usecase since they are fixing it in ARM.
Besides that, it is the IOMMU API, drivers have to implement it, you don't get to pick and choose.
Is there any documentation on the mentioned use case and the required IOMMU API because this is new to me. I would like to understand things better before making the change in AMD IOMMU driver.
Just the API definitions we have been working toward and the dicussions on the list.
It is not a complex concept, UNMANAGED paging domains need to support PRI and PASID.
Quoted 10 lines
Quoted 3 lines
You are not going to get SVA without also properly doing all infrastructure to enable paging and unmanaged - I won't support another shortcut hackjob for SVA that needs unwinding like ARM has.
The goal of this series is to introduce the SVA support for AMD IOMMU driver.
So far we have been accommodating all the review comments.
- Droppingiommu_v2module before getting SVA
- Reworking SVA top of Tina's series
- Reworking IOPF on top of Baolu's IOPF enhancement
If you would prefer to have a more stable base to work on then wait a few months until ARM and Intel drviers are fully fixed. Otherwise you are going to have to keep pace with the advancing work.
Quoted 2 lines
Adding PASID support for the unmanaged domain is a new requirement and not relevant to this series.
It is not. It is what the iommu API is designed to do, ARM got grandfathered in with their half implementation because it already existed. New drivers must implement the API properly and not take shortcuts.
Quoted 3 lines
It will also require a considerable amount of changes in AMD IOMMU driver that we need to further investigate to better understand.
Exactly. I do not want to see new drivers follow down the path of ARM at exctly the same time we are trying to fix ARM to work properly! This undermines all the progress we are making! The API has been defined, please follow it!
On ARM unwinding the bad design without breaking SVA is proving very hard after the fact. You even say it will be hard on AMD, so NO, do it right to start with.
Every single thing you need to implement to support SVA is also needed to support generic PRI+PASID. The request here is to layer the code correctly so that the PASID and PRI bits are not linked to SVA.
You don't need to *test* PRI+PASID but you do need to layer the code correctly so that it is trivial to implement.
Jason
Vasant Hegde
maintainerPATCH RESEND 4/10 iommu/amd: Add support to enable/disable SVA feature
In reply to Vasant HegdeFrom: Suravee Suthikulpanit <[email protected]>
Add support to enable/disable SVA feature. It will setup v2 page table mode with PASID and enables SVA.
--- drivers/iommu/amd/amd_iommu.h | 4 +++ drivers/iommu/amd/iommu.c | 63 +++++++++++++++++++++++++++++++++++ drivers/iommu/amd/sva.c | 29 ++++++++++++++++ 3 files changed, 96 insertions(+)
drivers/iommu/amd/amd_iommu.h +4 -0
@@ -46,6 +46,10 @@ bool amd_iommu_sva_supported(void);int amd_iommu_set_dev_pasid (struct iommu_domain *domain,struct device *dev,ioasid_t pasid);void amd_iommu_remove_dev_pasid (struct device *dev,ioasid_t pasid); +int amd_iommu_sva_enable (struct device *dev); +int amd_iommu_sva_disable (struct device *dev); +int amd_iommu_sva_gcr3_init (struct iommu_dev_data *dev_data,int pasids); +int amd_iommu_sva_gcr3_uninit (struct iommu_dev_data *dev_data);struct amd_iommu *get_amd_iommu (unsigned int idx);u8 amd_iommu_pc_get_max_banks (unsigned int idx);
drivers/iommu/amd/iommu.c +63 -0
81 lines not on this page · raw message
drivers/iommu/amd/sva.c +29 -0
@@ -297,3 +297,32 @@ void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)break ; } } + +int amd_iommu_sva_enable (struct device *dev) +{ +struct pci_dev *pdev =dev_is_pci (dev) ?to_pci_dev (dev) :NULL ; +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!pdev || !iommu || !dev_data) +return -EINVAL ; + +if (!amd_iommu_sva_supported ()) +return -ENODEV ; + +if (!dev_data->pasid_enabled) +return -EINVAL ; + +return amd_iommu_sva_gcr3_init (dev_data, dev->iommu->max_pasids); +} + +int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}
Signature · 1 line
2.31.1
Re: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Quoted 18 lines · Vasant Hegde
drivers/iommu/amd/sva.c +18
-0+int amd_iommu_sva_enable (struct device *dev) +{ +struct pci_dev *pdev =dev_is_pci (dev) ?to_pci_dev (dev) :NULL ; +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!pdev || !iommu || !dev_data) +return -EINVAL ; + +if (!amd_iommu_sva_supported ()) +return -ENODEV ; + +if (!dev_data->pasid_enabled) +return -EINVAL ; + +return amd_iommu_sva_gcr3_init (dev_data, dev->iommu->max_pasids); +} +
drivers/iommu/amd/sva.c void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 328
Quoted 10 lines
drivers/iommu/amd/sva.c +10
-0+int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}
I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
gcr3 touching and enforcing should be part of generic PASID code and be triggered naturally when the SVA domain is attached to a PASID.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Jason,
Quoted 25 lines · Jason Gunthorpe
Quoted 23 lines · Vasant Hegde
drivers/iommu/amd/sva.c +23
-0+int amd_iommu_sva_enable (struct device *dev) +{ +struct pci_dev *pdev =dev_is_pci (dev) ?to_pci_dev (dev) :NULL ; +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!pdev || !iommu || !dev_data) +return -EINVAL ; + +if (!amd_iommu_sva_supported ()) +return -ENODEV ; + +if (!dev_data->pasid_enabled) +return -EINVAL ; + +return amd_iommu_sva_gcr3_init (dev_data, dev->iommu->max_pasids); +} + +int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +
Quoted 8 lines
drivers/iommu/amd/sva.c void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 328
Quoted 5 lines
drivers/iommu/amd/sva.c +5
-0+if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
We need to configure Device Table Entry when we enable/disable SVA. Also based on base domain type we do adjust the GCR3 table.
-Vasant
Quoted 5 lines
gcr3 touching and enforcing should be part of generic PASID code and be triggered naturally when the SVA domain is attached to a PASID.
Jason
Re: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Quoted 38 lines · Vasant Hegde
Jason,
Quoted 25 lines · Jason Gunthorpe
Quoted 23 lines · Vasant Hegde
drivers/iommu/amd/sva.c +23
-0+int amd_iommu_sva_enable (struct device *dev) +{ +struct pci_dev *pdev =dev_is_pci (dev) ?to_pci_dev (dev) :NULL ; +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!pdev || !iommu || !dev_data) +return -EINVAL ; + +if (!amd_iommu_sva_supported ()) +return -ENODEV ; + +if (!dev_data->pasid_enabled) +return -EINVAL ; + +return amd_iommu_sva_gcr3_init (dev_data, dev->iommu->max_pasids); +} + +int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +Quoted 8 lines
drivers/iommu/amd/sva.c void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 328
Quoted 5 lines
drivers/iommu/amd/sva.c +5
-0+if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
Quoted 1 line
We need to configure Device Table Entry when we enable/disable SVA.
The DTE should only be changed by bind/unbind of RID/PASID - it has nothing to do with these APIs.
Again, start with enabling native PASID support for UNAMANGED v2 domains and your SVA will make alot more sense and be alot cleaner.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Jason,
Quoted 39 lines · Jason Gunthorpe
Quoted 38 lines · Vasant Hegde
Jason,
Quoted 33 lines · Jason Gunthorpe
Quoted 18 lines · Vasant Hegde
drivers/iommu/amd/sva.c +18
-0+int amd_iommu_sva_enable (struct device *dev) +{ +struct pci_dev *pdev =dev_is_pci (dev) ?to_pci_dev (dev) :NULL ; +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!pdev || !iommu || !dev_data) +return -EINVAL ; + +if (!amd_iommu_sva_supported ()) +return -ENODEV ; + +if (!dev_data->pasid_enabled) +return -EINVAL ; + +return amd_iommu_sva_gcr3_init (dev_data, dev->iommu->max_pasids); +} +drivers/iommu/amd/sva.c void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 328
Quoted 10 lines
drivers/iommu/amd/sva.c +10
-0+int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
Quoted 4 lines
Quoted 1 line
We need to configure Device Table Entry when we enable/disable SVA.
The DTE should only be changed by bind/unbind of RID/PASID - it has nothing to do with these APIs.
Above functions are called during enable_feature(SVA) path. Not in PASID bind/unbind path. In PASID bind/unbind path we just update PASID table.
Quoted 3 lines
Again, start with enabling native PASID support for UNAMANGED v2 domains and your SVA will make alot more sense and be alot cleaner.
Currently, the IOMMU_DOMAIN_UNMANAGED is used by VFIO, which sets up the v1 table for GPA->SPA translation.
If we were to force the unmanaged domain to use the v2 table by default, we would not be able to support nested translation, where it requires v1 table to be managed by the host ( GPA->SPA) and v2 in the guest (GVA->GPA).
In this case, how would you handle this issue?
A solution might be to allow specifying which page table (v1 or v2) to use when creating the unmanaged domain.
-Vasant
Re: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Quoted 14 lines · Vasant Hegde
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
drivers/iommu/amd/sva.c void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 328
Quoted 10 lines
drivers/iommu/amd/sva.c +10
-0+int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
We need to configure Device Table Entry when we enable/disable SVA.
The DTE should only be changed by bind/unbind of RID/PASID - it has nothing to do with these APIs.
Above functions are called during
enable_feature(SVA)path. Not in PASID bind/unbind path. In PASID bind/unbind path we just update PASID table.
I know, which is why they can't change anything about the IOMMU HW. Only attach/bind/unbind should change HW state.
I intend to remove these APIs once ARM is fixed, so please don't start to rely on them. The driver is doing something wrong if it is changing HW setups outside of attach/bind/unbind.
These APIs are from an era before we had proper PASID API support and no longer make sense.
Quoted 9 lines
Quoted 2 lines
Again, start with enabling native PASID support for UNAMANGED v2 domains and your SVA will make alot more sense and be alot cleaner.
Currently, the
IOMMU_DOMAIN_UNMANAGEDis used by VFIO, which sets up the v1 table forGPA->SPAtranslation.If we were to force the unmanaged domain to use the v2 table by default, we would not be able to support nested translation, where it requires v1 table to be managed by the host (
GPA->SPA) and v2 in the guest (GVA->GPA).
I've said this so many times now
Upstream kernel does not support nested translation with VFIO. The force v1 hack in the AMD driver *SHOULD NOT* be part of mainline.
I don't care about any unmerged out of tree patch sets that rely on it - especially ones that ignored that the semi-upstream approved hack to do this is VFIO_TYPE1_NESTING_IOMMU (which is also about to be removed), not what you are talking about here.
iommufd has the upstream solution for this problem and it relies on the iommufd user specifically asking for a nesting parent page table which is the only signal that the AMD driver should use to force a v1 format on a PASID capable device.
Otherwise *all* requests for any kind of paging domain should use the v2 if the device has PASID support.
Indeed after my other series is merged the AMD driver should be converted to the alloc_domain_paging() interface that directly prevents this abuse of the API.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Jason,
Quoted 23 lines · Jason Gunthorpe
Quoted 14 lines · Vasant Hegde
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
drivers/iommu/amd/sva.c void amd_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 328
Quoted 10 lines
drivers/iommu/amd/sva.c +10
-0+int amd_iommu_sva_disable (struct device *dev) +{ +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!iommu || !dev_data) +return -EINVAL ; + +return amd_iommu_sva_gcr3_uninit (dev_data); +}I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
We need to configure Device Table Entry when we enable/disable SVA.
The DTE should only be changed by bind/unbind of RID/PASID - it has nothing to do with these APIs.
Above functions are called during
enable_feature(SVA)path. Not in PASID bind/unbind path. In PASID bind/unbind path we just update PASID table.
Quoted 6 lines
I know, which is why they can't change anything about the IOMMU HW. Only attach/bind/unbind should change HW state.
I intend to remove these APIs once ARM is fixed, so please don't start to rely on them. The driver is doing something wrong if it is changing HW setups outside of attach/bind/unbind.
If you intend to remove enable_feature(SVA) API completely, then you help us to figure out how to handle below issues?
1) During attach_device() we don't know whether device is really going to use SVA or not. So we want to start with minimal GCR3 table (that can handle just PASID 0) and during enable_feature(SVA) we want to enable GCR3 table that can handle max_pasids supported by a device. That way we can avoid performance penalty if device is not using SVA (extra GCR3 table walk).
Note that to keep things simple current patchset configures max_pasids during boot itself. But we will fix this once this series goes in.
2) If we boot with passthrough mode, then during attach_device() we are not allocating GCR3 table. It will be allocated only when its needed. That way we are not wasting memory.
Do you suggest that the driver setup GCR3 table at the first PASID binding to the device?
-Vasant
Re: [PATCH RESEND 04/10] iommu/amd: Add support to enable/disable SVA feature
Quoted 35 lines · Vasant Hegde
Jason,
Quoted 23 lines · Jason Gunthorpe
Quoted 14 lines · Vasant Hegde
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Review
Quoted 10 lines
(quoted diff) +10
-0+int amd_iommu_sva_disable(struct device *dev) +{ + struct amd_iommu *iommu = get_amd_iommu_from_dev(dev); + struct iommu_dev_data *dev_data = dev_iommu_priv_get(dev); + + if (!iommu || !dev_data) + return -EINVAL; + + return amd_iommu_sva_gcr3_uninit(dev_data); +}I think these features are a mistake, you need to implement them for now but I wouldn't touch the gcr3 table, and disable should be a NOP.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
We need to configure Device Table Entry when we enable/disable SVA.
The DTE should only be changed by bind/unbind of RID/PASID - it has nothing to do with these APIs.
Above functions are called during
enable_feature(SVA)path. Not in PASID bind/unbind path. In PASID bind/unbind path we just update PASID table.Quoted 6 lines
I know, which is why they can't change anything about the IOMMU HW. Only attach/bind/unbind should change HW state.
I intend to remove these APIs once ARM is fixed, so please don't start to rely on them. The driver is doing something wrong if it is changing HW setups outside of attach/bind/unbind.
Quoted 8 lines
If you intend to remove
enable_feature(SVA)API completely, then you help us to figure out how to handle below issues?1) During
attach_device()we don't know whether device is really going to use SVA or not. So we want to start with minimal GCR3 table (that can handle just PASID 0) and duringenable_feature(SVA)we want to enable GCR3 table that can handlemax_pasidssupported by a device. That way we can avoid performance penalty if device is not using SVA (extra GCR3 table walk).
Resize the GCR3 table when the first pasid using attach happens.
We can discuss if/when it should be shrunk back to minimal.
Quoted 6 lines
2) If we boot with passthrough mode, then during
attach_device()we are not allocating GCR3 table. It will be allocated only when its needed. That way we are not wasting memory.Do you suggest that the driver setup GCR3 table at the first PASID binding to the device?
Yes. PASID must work correctly stand alone.
Jason
Vasant Hegde
maintainerPATCH RESEND 5/10 iommu/amd: Move PPR-related functions into ppr.c
In reply to Vasant HegdeFrom: Suravee Suthikulpanit <[email protected]>
In preparation to subsequent PPR-related patches, and also remove static declaration for certain helper functions so that it can be reused in other files.
Also rename below functions:
alloc_ppr_log -> amd_iommu_alloc_ppr_log
iommu_enable_ppr_log -> amd_iommu_enable_ppr_log
free_ppr_log -> amd_iommu_free_ppr_log
iommu_poll_ppr_log -> amd_iommu_poll_ppr_log
--- drivers/iommu/amd/Makefile | 2 +- drivers/iommu/amd/amd_iommu.h | 17 +++++- drivers/iommu/amd/init.c | 65 +++----------------- drivers/iommu/amd/iommu.c | 55 +---------------- drivers/iommu/amd/ppr.c | 112 ++++++++++++++++++++++++++++++++++ 5 files changed, 137 insertions(+), 114 deletions(-) create mode 100644 drivers/iommu/amd/ppr.c
drivers/iommu/amd/Makefile +1 -1
@@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only -obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o sva.o +obj-$(CONFIG_AMD_IOMMU) += iommu.o init.o quirks.o io_pgtable.o io_pgtable_v2.o sva.o ppr.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += debugfs.o
drivers/iommu/amd/amd_iommu.h +14 -3
@@ -17,6 +17,9 @@ irqreturn_t amd_iommu_int_thread_pprlog(int irq, void *data);irqreturn_t amd_iommu_int_thread_galog (int irq,void *data);irqreturn_t amd_iommu_int_handler (int irq,void *data);void amd_iommu_apply_erratum_63 (struct amd_iommu *iommu,u16 devid); +void amd_iommu_restart_log (struct amd_iommu *iommu,const char *evt_type, +u8 cntrl_intr,u8 cntrl_log, +u32 status_run_mask,u32 status_overflow_mask);void amd_iommu_restart_event_logging (struct amd_iommu *iommu);void amd_iommu_restart_ga_log (struct amd_iommu *iommu);void amd_iommu_restart_ppr_log (struct amd_iommu *iommu);
@@ -24,6 +27,9 @@ int amd_iommu_init_devices(void);void amd_iommu_uninit_devices (void );void amd_iommu_init_notifier (void );void amd_iommu_set_rlookup_table (struct amd_iommu *iommu,u16 devid); +void iommu_feature_enable (struct amd_iommu *iommu,u8 bit); +void *__init iommu_alloc_4k_pages (struct amd_iommu *iommu, +gfp_t gfp,size_t size);#ifdef CONFIG_AMD_IOMMU_DEBUGFS void amd_iommu_debugfs_setup (struct amd_iommu *iommu);
@@ -69,6 +75,14 @@ int amd_iommu_set_gcr3(struct iommu_dev_data *dev_data, u32 pasid, unsigned long gcr3);int amd_iommu_clear_gcr3 (struct iommu_dev_data *dev_data,u32 pasid); +/* PPR */ +int __init amd_iommu_alloc_ppr_log (struct amd_iommu *iommu); +void __init amd_iommu_free_ppr_log (struct amd_iommu *iommu); +void amd_iommu_enable_ppr_log (struct amd_iommu *iommu); +void amd_iommu_poll_ppr_log (struct amd_iommu *iommu); +int amd_iommu_complete_ppr (struct pci_dev *pdev,u32 pasid, +int status,int tag); +int amd_iommu_flush_page (struct protection_domain *domain,u32 pasid,u64 address);void amd_iommu_update_and_flush_device_table (struct protection_domain *domain);void amd_iommu_domain_update (struct protection_domain *domain);
@@ -89,9 +103,6 @@ static inline int amd_iommu_create_irq_domain(struct amd_iommu *iommu)#define PPR_INVALID 0x1#define PPR_FAILURE 0xf -int amd_iommu_complete_ppr (struct pci_dev *pdev,u32 pasid, -int status,int tag); -static inline bool is_rd890_iommu (struct pci_dev *pdev) { return (pdev->vendor == PCI_VENDOR_ID_ATI) &&
drivers/iommu/amd/init.c +9 -56
113 lines not on this page · raw message
drivers/iommu/amd/iommu.c +1 -54
67 lines not on this page · raw message
drivers/iommu/amd/ppr.c new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 6/10 iommu/amd: Define per-IOMMU iopf_queue
In reply to Vasant HegdeFrom: Suravee Suthikulpanit <[email protected]>
AMD IOMMU hardware supports PCI Peripheral Paging Request (PPR) using a PPR log, which is a circular buffer containing requests from downstream end-point devices.
There is one PPR log per IOMMU instance. Therefore, allocate an iopf_queue per IOMMU instance during driver initialization, and free the queue during driver deinitialization.
--- drivers/iommu/amd/amd_iommu.h | 4 +++ drivers/iommu/amd/amd_iommu_types.h | 4 +++ drivers/iommu/amd/init.c | 5 ++++ drivers/iommu/amd/ppr.c | 43 +++++++++++++++++++++++++++++ 4 files changed, 56 insertions(+)
drivers/iommu/amd/amd_iommu.h +4 -0
@@ -57,6 +57,10 @@ int amd_iommu_sva_disable(struct device *dev);int amd_iommu_sva_gcr3_init (struct iommu_dev_data *dev_data,int pasids);int amd_iommu_sva_gcr3_uninit (struct iommu_dev_data *dev_data); +/* IOPF */ +int amd_iommu_iopf_init (struct amd_iommu *iommu); +void amd_iommu_iopf_uninit (struct amd_iommu *iommu); +struct amd_iommu *get_amd_iommu (unsigned int idx);u8 amd_iommu_pc_get_max_banks (unsigned int idx);bool amd_iommu_pc_supported (void );
drivers/iommu/amd/amd_iommu_types.h +4 -0
@@ -758,6 +758,10 @@ struct amd_iommu {/* DebugFS Info */ struct dentry *debugfs;#endif + +/* IOPF support */ +struct iopf_queue *iopf_queue; +unsigned char iopfq_name[32 ]; };static inline struct amd_iommu *dev_to_amd_iommu (struct device *dev)
drivers/iommu/amd/init.c +5 -0
@@ -1638,6 +1638,7 @@ static void __init free_iommu_one(struct amd_iommu *iommu)amd_iommu_free_ppr_log (iommu);free_ga_log (iommu);iommu_unmap_mmio_space (iommu); +amd_iommu_iopf_uninit (iommu); }static void __init free_iommu_all (void )
@@ -2791,9 +2792,13 @@ static void enable_iommus_v2(void) {struct amd_iommu *iommu; +if (!amd_iommu_gt_ppr_supported ()) +return ; +for_each_iommu (iommu) {amd_iommu_enable_ppr_log (iommu);iommu_enable_gt (iommu); +amd_iommu_iopf_init (iommu); } }
drivers/iommu/amd/ppr.c +43 -0
52 lines not on this page · raw message
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 7/10 iommu/amd: Add support for page response
In reply to Vasant HegdeFrom: Suravee Suthikulpanit <[email protected]>
This generates AMD IOMMU COMPLETE_PPR_REQUEST for the specified device with the specified PRI Response Code.
--- drivers/iommu/amd/amd_iommu.h | 3 +++ drivers/iommu/amd/iommu.c | 1 + drivers/iommu/amd/ppr.c | 15 +++++++++++++++ 3 files changed, 19 insertions(+)
drivers/iommu/amd/amd_iommu.h +3 -0
@@ -60,6 +60,9 @@ int amd_iommu_sva_gcr3_uninit(struct iommu_dev_data *dev_data);/* IOPF */ int amd_iommu_iopf_init (struct amd_iommu *iommu);void amd_iommu_iopf_uninit (struct amd_iommu *iommu); +int amd_iommu_page_response (struct device *dev, +struct iommu_fault_event *event, +struct iommu_page_response *resp);struct amd_iommu *get_amd_iommu (unsigned int idx);u8 amd_iommu_pc_get_max_banks (unsigned int idx);
drivers/iommu/amd/iommu.c +1 -0
@@ -2622,6 +2622,7 @@ const struct iommu_ops amd_iommu_ops = { .dev_enable_feat = amd_iommu_dev_enable_feature, .dev_disable_feat = amd_iommu_dev_disable_feature, .remove_dev_pasid = amd_iommu_remove_dev_pasid, + .page_response = amd_iommu_page_response, .default_domain_ops = &(const struct iommu_domain_ops ) { .attach_dev = amd_iommu_attach_device, .map_pages = amd_iommu_map_pages,
drivers/iommu/amd/ppr.c +15 -0
@@ -153,3 +153,18 @@ void amd_iommu_iopf_uninit(struct amd_iommu *iommu)raw_spin_unlock_irqrestore (&iommu->lock, flags); } + +int amd_iommu_page_response (struct device *dev, +struct iommu_fault_event *event, +struct iommu_page_response *resp) +{ +struct pci_dev *pdev; + +if (!dev || !dev_is_pci (dev)) +return -ENODEV ; + + pdev =to_pci_dev (dev); + +return amd_iommu_complete_ppr (pdev, resp->pasid, resp->code, + resp->grpid); +}
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 8/10 iommu/amd: Add support for add/remove device for IOPF
In reply to Vasant HegdeFrom: Wei Huang <[email protected]>
Which adds/removes the device to the corresponding per-IOMMU iopf_queue. It also registers device fault handler. These interfaces are called from IOPF feature enable/disable path.
--- drivers/iommu/amd/amd_iommu.h | 2 ++ drivers/iommu/amd/ppr.c | 50 +++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+)
drivers/iommu/amd/amd_iommu.h +2 -0
@@ -60,6 +60,8 @@ int amd_iommu_sva_gcr3_uninit(struct iommu_dev_data *dev_data);/* IOPF */ int amd_iommu_iopf_init (struct amd_iommu *iommu);void amd_iommu_iopf_uninit (struct amd_iommu *iommu); +int amd_iommu_iopf_add_device (struct amd_iommu *iommu,struct device *dev); +int amd_iommu_iopf_remove_device (struct amd_iommu *iommu,struct device *dev);int amd_iommu_page_response (struct device *dev,struct iommu_fault_event *event,struct iommu_page_response *resp);
drivers/iommu/amd/ppr.c +50 -0
53 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH RESEND 08/10] iommu/amd: Add support for add/remove device for IOPF
Quoted 21 lines · Vasant Hegde
drivers/iommu/amd/ppr.c +13
-0@@ -168,3 +168,53 @@ int amd_iommu_page_response(struct device *dev,return amd_iommu_complete_ppr (pdev, resp->pasid, resp->code, resp->grpid); } + +int amd_iommu_iopf_add_device (struct amd_iommu *iommu,struct device *dev) +{ +unsigned long flags; +int ret = -EINVAL ; + +raw_spin_lock_irqsave (&iommu->lock, flags); + +if (!iommu->iopf_queue) + goto out; + + ret =iopf_queue_add_device (iommu->iopf_queue, dev); +if (ret)
drivers/iommu/amd/ppr.c int amd_iommu_page_response(struct device *dev, · line 193
Quoted 10 lines
drivers/iommu/amd/ppr.c +10
-0+ goto out; + + ret =iommu_register_device_fault_handler (dev, iommu_queue_iopf, dev); +if (ret) +iopf_queue_remove_device (iommu->iopf_queue, dev); + +out: +raw_spin_unlock_irqrestore (&iommu->lock, flags); +return ret; +}
Please assume the iopf rework will arrive before your series:
https://lore.kernel.org/linux-iommu/[email protected]/
In the new arrangement the work queue to is used by the SVA code to move the blocking handle_mm_fault out of the global page request processing.
I would expect there to be one (unbound) work_queue for SVA, shared by all SVA domains.
Jason
Vasant Hegde
maintainerRe: [PATCH RESEND 08/10] iommu/amd: Add support for add/remove device for IOPF
Jason,
Quoted 36 lines · Jason Gunthorpe
Quoted 21 lines · Vasant Hegde
drivers/iommu/amd/ppr.c +13
-0@@ -168,3 +168,53 @@ int amd_iommu_page_response(struct device *dev,return amd_iommu_complete_ppr (pdev, resp->pasid, resp->code, resp->grpid); } + +int amd_iommu_iopf_add_device (struct amd_iommu *iommu,struct device *dev) +{ +unsigned long flags; +int ret = -EINVAL ; + +raw_spin_lock_irqsave (&iommu->lock, flags); + +if (!iommu->iopf_queue) + goto out; + + ret =iopf_queue_add_device (iommu->iopf_queue, dev); +if (ret)drivers/iommu/amd/ppr.c int amd_iommu_page_response(struct device *dev, · line 193
Quoted 10 lines
drivers/iommu/amd/ppr.c +10
-0+ goto out; + + ret =iommu_register_device_fault_handler (dev, iommu_queue_iopf, dev); +if (ret) +iopf_queue_remove_device (iommu->iopf_queue, dev); + +out: +raw_spin_unlock_irqrestore (&iommu->lock, flags); +return ret; +}Please assume the iopf rework will arrive before your series:
Sure. I will rebase/rework patches on top of this series.
-Vasant
Vasant Hegde
maintainerPATCH RESEND 9/10 iommu/amd: Add IO page fault notifier handler
In reply to Vasant HegdeFrom: Wei Huang <[email protected]>
Whenever there is a page fault IOMMU logs entry to ppr log and sends interrupt to host. We have to handle the page fault and respond to IOMMU.
Currentlty we have page fault handler support for V2API mode. Let us add support for SVA mode. We will use core IOMMU page fault handler. We just need to validate the request and call core iommu fault handler interface.
Also rename search_dev_data() -> amd_iommu_search_dev_data().
--- drivers/iommu/amd/amd_iommu.h | 2 + drivers/iommu/amd/amd_iommu_types.h | 8 +++ drivers/iommu/amd/iommu.c | 7 +- drivers/iommu/amd/ppr.c | 103 +++++++++++++++++++++++++++- 4 files changed, 116 insertions(+), 4 deletions(-)
drivers/iommu/amd/amd_iommu.h +2 -0
@@ -30,6 +30,8 @@ void amd_iommu_set_rlookup_table(struct amd_iommu *iommu, u16 devid);void iommu_feature_enable (struct amd_iommu *iommu,u8 bit);void *__init iommu_alloc_4k_pages (struct amd_iommu *iommu,gfp_t gfp,size_t size); +struct iommu_dev_data *amd_iommu_search_dev_data (struct amd_iommu *iommu, +u16 devid);#ifdef CONFIG_AMD_IOMMU_DEBUGFS void amd_iommu_debugfs_setup (struct amd_iommu *iommu);
drivers/iommu/amd/amd_iommu_types.h +8 -0
@@ -248,6 +248,14 @@#define PPR_ENTRY_SIZE 16#define PPR_LOG_SIZE (PPR_ENTRY_SIZE * PPR_LOG_ENTRIES) +/* PAGE_SERVICE_REQUEST PPR Log Buffer Entry flags */ +#define PPR_FLAG_EXEC 0x002/* Execute permission requested */ +#define PPR_FLAG_READ 0x004/* Read permission requested */ +#define PPR_FLAG_WRITE 0x020/* Write permission requested */ +#define PPR_FLAG_US 0x040/* 1: User, 0: Supervisor */ +#define PPR_FLAG_RVSD 0x080/* Reserved bit not zero */ +#define PPR_FLAG_GN 0x100/* GVA and PASID is valid */ +#define PPR_REQ_TYPE (x) (((x) >> 60) & 0xfULL)#define PPR_FLAGS (x) (((x) >> 48) & 0xfffULL)#define PPR_DEVID (x) ((x) & 0xffffULL)
drivers/iommu/amd/iommu.c +4 -3
@@ -201,7 +201,8 @@ static struct iommu_dev_data *alloc_dev_data(struct amd_iommu *iommu, u16 devid)return dev_data; } -static struct iommu_dev_data *search_dev_data (struct amd_iommu *iommu,u16 devid) +struct iommu_dev_data *amd_iommu_search_dev_data (struct amd_iommu *iommu, +u16 devid) {struct iommu_dev_data *dev_data;struct llist_node *node;
@@ -285,7 +286,7 @@ static struct iommu_dev_data *find_dev_data(struct amd_iommu *iommu, u16 devid) {struct iommu_dev_data *dev_data; - dev_data =search_dev_data (iommu, devid); + dev_data =amd_iommu_search_dev_data (iommu, devid);if (dev_data ==NULL ) { dev_data =alloc_dev_data (iommu, devid);
@@ -3631,7 +3632,7 @@ static int amd_ir_set_vcpu_affinity(struct irq_data *data, void *vcpu_info)if (ir_data->iommu ==NULL )return -EINVAL ; - dev_data =search_dev_data (ir_data->iommu, irte_info->devid); + dev_data =amd_iommu_search_dev_data (ir_data->iommu, irte_info->devid);/* Note: * This device has never been set up for guest mode.
drivers/iommu/amd/ppr.c +102 -1
115 lines not on this page · raw message
Signature · 1 line
2.31.1
Vasant Hegde
maintainerPATCH RESEND 10/10 iommu/amd: Introduce logic to enable/disable IOPF
In reply to Vasant HegdeFrom: Suravee Suthikulpanit <[email protected]>
Add support to enable/disable IOPF feature. So that PRI capable device can make use of page fault handler in SVA mode.
--- drivers/iommu/amd/amd_iommu.h | 2 ++ drivers/iommu/amd/iommu.c | 6 ++++ drivers/iommu/amd/ppr.c | 56 +++++++++++++++++++++++++++++++++++ 3 files changed, 64 insertions(+)
drivers/iommu/amd/amd_iommu.h +2 -0
@@ -67,6 +67,8 @@ int amd_iommu_iopf_remove_device(struct amd_iommu *iommu, struct device *dev);int amd_iommu_page_response (struct device *dev,struct iommu_fault_event *event,struct iommu_page_response *resp); +int amd_iommu_iopf_enable (struct device *dev); +int amd_iommu_iopf_disable (struct device *dev);struct amd_iommu *get_amd_iommu (unsigned int idx);u8 amd_iommu_pc_get_max_banks (unsigned int idx);
drivers/iommu/amd/iommu.c +6 -0
@@ -2586,6 +2586,9 @@ static int amd_iommu_dev_enable_feature(struct device *dev,case IOMMU_DEV_FEAT_SVA : ret =amd_iommu_sva_enable (dev);break ; +case IOMMU_DEV_FEAT_IOPF : + ret =amd_iommu_iopf_enable (dev); +break ;default : ret = -EINVAL ;break ;
@@ -2602,6 +2605,9 @@ static int amd_iommu_dev_disable_feature(struct device *dev,case IOMMU_DEV_FEAT_SVA : ret =amd_iommu_sva_disable (dev);break ; +case IOMMU_DEV_FEAT_IOPF : + ret =amd_iommu_iopf_disable (dev); +break ;default : ret = -EINVAL ;break ;
drivers/iommu/amd/ppr.c +56 -0
65 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH RESEND 10/10] iommu/amd: Introduce logic to enable/disable IOPF
Quoted 61 lines · Vasant Hegde
drivers/iommu/amd/ppr.c +46
-0@@ -9,6 +9,7 @@#include <linux/amd-iommu.h> #include <linux/delay.h> #include <linux/mmu_notifier.h> +#include <linux/pci-ats.h> #include "amd_iommu.h" #include "amd_iommu_types.h" @@ -319,3 +320,58 @@ int amd_iommu_iopf_remove_device(struct amd_iommu *iommu, struct device *dev)raw_spin_unlock_irqrestore (&iommu->lock, flags);return ret; } + +static int amd_iommu_iopf_update (struct device *dev,bool enable) +{ +unsigned long flags; +int ret; +struct pci_dev *pdev =dev_is_pci (dev) ?to_pci_dev (dev) :NULL ; +struct amd_iommu *iommu =get_amd_iommu_from_dev (dev); +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); +struct protection_domain *pdom =amd_iommu_get_pdomain (dev); + +if (!pdev || !iommu || !dev_data) +return -EINVAL ; + +spin_lock_irqsave (&pdom->lock, flags); + +if (enable) { + ret =amd_iommu_iopf_add_device (iommu, dev); +if (ret) + goto out; + + dev_data->ppr = true; + }else { + ret =amd_iommu_iopf_remove_device (iommu, dev); + dev_data->ppr = false; + } + +amd_iommu_domain_update (pdom); + +out: +spin_unlock_irqrestore (&pdom->lock, flags); +return ret; +} + +int amd_iommu_iopf_enable (struct device *dev) +{ +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!(dev_data->flags &AMD_IOMMU_DEVICE_FLAG_PRI_SUP )) +return -ENODEV ; + +if (!dev_data->ats_enabled || !dev_data->pri_enabled) +return -EINVAL ; + +return amd_iommu_iopf_update (dev, true); +}
drivers/iommu/amd/ppr.c int amd_iommu_iopf_remove_device(struct amd_iommu *iommu, struct device *dev) · line 377
Quoted 10 lines
drivers/iommu/amd/ppr.c +10
-0+ +int amd_iommu_iopf_disable (struct device *dev) +{ +struct iommu_dev_data *dev_data =dev_iommu_priv_get (dev); + +if (!dev_data->pri_enabled) +return -EINVAL ; + +return amd_iommu_iopf_update (dev, false); +}
These features also look like a mistake..
PRI is enabled if the domain has an iopf handler, when that domain is attached to the RID/PASID.
It remains enabled so long as any IOPF enabled domain is present on the device.
So again, these should be checks but otherwise NOPs and any actual working has to be pushed into the domain attach code.
Jason