Intel IOMMU Updates for Linux v5.15
Subsystems (3)
Lu Baolu
maintainerHi Joerg,
The patches queued in this series are for v5.15.
It includes:
- Update the virtual command related registers
- Enable Intel IOMMU scalable mode by default
- Preset A/D bits for user space DMA usage
- Allow devices to have more than 32 outstanding PRs
- Various cleanups
Please pull.
Best regards,
Baolu
Andy Shevchenko (1):
iommu/vt-d: Drop the kernel doc annotation
Liu Yi L (2):
iommu/vt-d: Use pasid_pte_is_present() helper function iommu/vt-d: Add present bit check in pasid entry setup helpers
Lu Baolu (5):
iommu/vt-d: Update the virtual command related registers iommu/vt-d: Refactor Kconfig a bit
iommu/vt-d: Enable Intel IOMMU scalable mode by default iommu/vt-d: Preset A/D bits for user space DMA usage iommu/vt-d: Allow devices to have more than 32 outstanding PRs
Zhen Lei (1):
iommu/vt-d: Remove unnecessary oom message
.../admin-guide/kernel-parameters.txt | 11 +++--- include/linux/intel-iommu.h | 6 +-- include/linux/intel-svm.h | 5 +++ drivers/iommu/intel/pasid.h | 10 ++--- drivers/iommu/intel/dmar.c | 2 - drivers/iommu/intel/iommu.c | 37 ++++++------------- drivers/iommu/intel/pasid.c | 18 ++++++++- drivers/iommu/intel/perf.c | 2 +- drivers/iommu/intel/svm.c | 4 -- drivers/iommu/intel/Kconfig | 19 ++++++---- 10 files changed, 60 insertions(+), 54 deletions(-)
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 1/9 iommu/vt-d: Update the virtual command related registers
The VT-d spec Revision 3.3 updated the virtual command registers, virtual command opcode B register, virtual command response register and virtual command capability register (Section 10.4.43, 10.4.44, 10.4.45, 10.4.46). This updates the virtual command interface implementation in the Intel IOMMU driver accordingly.
--- include/linux/intel-iommu.h | 6 +++--- drivers/iommu/intel/pasid.h | 10 +++++----- 2 files changed, 8 insertions(+), 8 deletions(-)
include/linux/intel-iommu.h +3 -3
@@ -124,9 +124,9 @@#define DMAR_MTRR_PHYSMASK8_REG 0x208#define DMAR_MTRR_PHYSBASE9_REG 0x210#define DMAR_MTRR_PHYSMASK9_REG 0x218 -#define DMAR_VCCAP_REG 0xe00/* Virtual command capability register */ -#define DMAR_VCMD_REG 0xe10/* Virtual command register */ -#define DMAR_VCRSP_REG 0xe20/* Virtual command response register */ +#define DMAR_VCCAP_REG 0xe30/* Virtual command capability register */ +#define DMAR_VCMD_REG 0xe00/* Virtual command register */ +#define DMAR_VCRSP_REG 0xe10/* Virtual command response register */ #define DMAR_IQER_REG_IQEI (reg) FIELD_GET(GENMASK_ULL(3, 0), reg)#define DMAR_IQER_REG_ITESID (reg) FIELD_GET(GENMASK_ULL(47, 32), reg)
drivers/iommu/intel/pasid.h +5 -5
@@ -28,12 +28,12 @@#define VCMD_CMD_ALLOC 0x1#define VCMD_CMD_FREE 0x2#define VCMD_VRSP_IP 0x1 -#define VCMD_VRSP_SC (e) (((e) >> 1) & 0x3) +#define VCMD_VRSP_SC (e) (((e) & 0xff) >> 1)#define VCMD_VRSP_SC_SUCCESS 0 -#define VCMD_VRSP_SC_NO_PASID_AVAIL 2 -#define VCMD_VRSP_SC_INVALID_PASID 2 -#define VCMD_VRSP_RESULT_PASID (e) (((e) >> 8) & 0xfffff) -#define VCMD_CMD_OPERAND (e) ((e) << 8) +#define VCMD_VRSP_SC_NO_PASID_AVAIL 16 +#define VCMD_VRSP_SC_INVALID_PASID 16 +#define VCMD_VRSP_RESULT_PASID (e) (((e) >> 16) & 0xfffff) +#define VCMD_CMD_OPERAND (e) ((e) << 16)/* * Domain ID reserved for pasid entries programmed for first-level * only and pass-through transfer modes.
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 2/9 iommu/vt-d: Remove unnecessary oom message
In reply to Lu BaoluFrom: Zhen Lei <[email protected]>
Fixes scripts/checkpatch.pl warning:
WARNING: Possible unnecessary 'out of memory' message
Remove it can help us save a bit of memory.
--- drivers/iommu/intel/dmar.c | 2 -- drivers/iommu/intel/iommu.c | 6 +----- 2 files changed, 1 insertion(+), 7 deletions(-)
drivers/iommu/intel/dmar.c +0 -2
@@ -149,8 +149,6 @@ dmar_alloc_pci_notify_info(struct pci_dev *dev, unsigned long event) }else { info =kzalloc (size,GFP_KERNEL );if (!info) { -pr_warn ("Out of memory when allocating notify_info " -"for %s.\n" ,pci_name (dev));if (dmar_dev_scope_status ==0 ) dmar_dev_scope_status = -ENOMEM ;return NULL ;
drivers/iommu/intel/iommu.c +1 -5
@@ -1779,11 +1779,8 @@ static int iommu_init_domains(struct intel_iommu *iommu)spin_lock_init (&iommu->lock); iommu->domain_ids =kcalloc (nlongs,sizeof (unsigned long ),GFP_KERNEL ); -if (!iommu->domain_ids) { -pr_err ("%s: Allocating domain id array failed\n" , - iommu->name); +if (!iommu->domain_ids)return -ENOMEM ; - } size = (ALIGN (ndomains,256 ) >>8 ) *sizeof (struct dmar_domain **); iommu->domains =kzalloc (size,GFP_KERNEL );
@@ -3224,7 +3221,6 @@ static int __init init_dmars(void) g_iommus =kcalloc (g_num_of_iommus,sizeof (struct intel_iommu *),GFP_KERNEL );if (!g_iommus) { -pr_err ("Allocating global iommu array failed\n" ); ret = -ENOMEM ; goto error; }
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 3/9 iommu/vt-d: Refactor Kconfig a bit
In reply to Lu BaoluPut all sub-options inside a "if INTEL_IOMMU" so that they don't need to always depend on INTEL_IOMMU. Use IS_ENABLED() instead of #ifdef as well.
--- drivers/iommu/intel/iommu.c | 13 ++----------- drivers/iommu/intel/Kconfig | 18 ++++++++++-------- 2 files changed, 12 insertions(+), 19 deletions(-)
drivers/iommu/intel/iommu.c +2 -11
@@ -327,17 +327,8 @@ static int intel_iommu_attach_device(struct iommu_domain *domain,static phys_addr_t intel_iommu_iova_to_phys (struct iommu_domain *domain,dma_addr_t iova); -#ifdef CONFIG_INTEL_IOMMU_DEFAULT_ON -int dmar_disabled =0 ; -#else -int dmar_disabled =1 ; -#endif /* CONFIG_INTEL_IOMMU_DEFAULT_ON */ - -#ifdef CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON -int intel_iommu_sm =1 ; -#else -int intel_iommu_sm; -#endif /* CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON */ +int dmar_disabled = !IS_ENABLED (CONFIG_INTEL_IOMMU_DEFAULT_ON ); +int intel_iommu_sm =IS_ENABLED (CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON );int intel_iommu_enabled =0 ;EXPORT_SYMBOL_GPL (intel_iommu_enabled);
drivers/iommu/intel/Kconfig +10 -8
@@ -25,9 +25,11 @@ config INTEL_IOMMU and include PCI device scope covered by these DMA remapping devices. +if INTEL_IOMMU + config INTEL_IOMMU_DEBUGFS bool "Export Intel IOMMU internals in Debugfs" - depends on INTEL_IOMMU && IOMMU_DEBUGFS + depends on IOMMU_DEBUGFS select DMAR_PERF help !!!WARNING!!!
@@ -41,7 +43,7 @@ config INTEL_IOMMU_DEBUGFS config INTEL_IOMMU_SVM bool "Support for Shared Virtual Memory with Intel IOMMU" - depends on INTEL_IOMMU && X86_64 + depends on X86_64 select PCI_PASID select PCI_PRI select MMU_NOTIFIER
@@ -53,9 +55,8 @@ config INTEL_IOMMU_SVM means of a Process Address Space ID (PASID). config INTEL_IOMMU_DEFAULT_ON - def_bool y - prompt "Enable Intel DMA Remapping Devices by default" - depends on INTEL_IOMMU + bool "Enable Intel DMA Remapping Devices by default" + default y help Selecting this option will enable a DMAR device at boot time if one is found. If this option is not selected, DMAR support can
@@ -63,7 +64,7 @@ config INTEL_IOMMU_DEFAULT_ON config INTEL_IOMMU_BROKEN_GFX_WA bool "Workaround broken graphics drivers (going away soon)" - depends on INTEL_IOMMU && BROKEN && X86 + depends on BROKEN && X86 help Current Graphics drivers tend to use physical address for DMA and avoid using DMA APIs. Setting this config
@@ -74,7 +75,7 @@ config INTEL_IOMMU_BROKEN_GFX_WA config INTEL_IOMMU_FLOPPY_WA def_bool y - depends on INTEL_IOMMU && X86 + depends on X86 help Floppy disk drivers are known to bypass DMA API calls thereby failing to work when IOMMU is enabled. This
@@ -83,7 +84,6 @@ config INTEL_IOMMU_FLOPPY_WA config INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON bool "Enable Intel IOMMU scalable mode by default" - depends on INTEL_IOMMU help Selecting this option will enable by default the scalable mode if hardware presents the capability. The scalable mode is defined in
@@ -92,3 +92,5 @@ config INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON is not selected, scalable mode support could also be enabled by passing intel_iommu=sm_on to the kernel. If not sure, please use the default value. + +endif # INTEL_IOMMU
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 4/9 iommu/vt-d: Enable Intel IOMMU scalable mode by default
In reply to Lu BaoluThe commit 8950dcd83ae7d ("iommu/vt-d: Leave scalable mode default off") leaves the scalable mode default off and end users could turn it on with "intel_iommu=sm_on". Using the Intel IOMMU scalable mode for kernel DMA, user-level device access and Shared Virtual Address have been enabled. This enables the scalable mode by default if the hardware advertises the support and adds kernel options of "intel_iommu=sm_on/sm_off" for end users to configure it through the kernel parameters.
--- Documentation/admin-guide/kernel-parameters.txt | 11 ++++++----- drivers/iommu/intel/iommu.c | 5 ++++- drivers/iommu/intel/Kconfig | 1 + 3 files changed, 11 insertions(+), 6 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +6 -5
@@ -1946,11 +1946,12 @@ By default, super page will be supported if Intel IOMMU has the capability. With this option, super page will not be supported. - sm_on [Default Off] - By default, scalable mode will be disabled even if the - hardware advertises that it has support for the scalable - mode translation. With this option set, scalable mode - will be used on hardware which claims to support it. + sm_on + Enable the Intel IOMMU scalable mode if the hardware + advertises that it has support for the scalable mode + translation. + sm_off + Disallow use of the Intel IOMMU scalable mode. tboot_noforce [Default Off] Do not force the Intel IOMMU enabled under tboot. By default, tboot will force Intel IOMMU on, which
drivers/iommu/intel/iommu.c +4 -1
@@ -432,8 +432,11 @@ static int __init intel_iommu_setup(char *str)pr_info ("Disable supported super page\n" ); intel_iommu_superpage =0 ; }else if (!strncmp (str,"sm_on" ,5 )) { -pr_info ("Intel-IOMMU: scalable mode supported\n" ); +pr_info ("Enable scalable mode if hardware supports\n" ); intel_iommu_sm =1 ; + }else if (!strncmp (str,"sm_off" ,6 )) { +pr_info ("Scalable mode is disallowed\n" ); + intel_iommu_sm =0 ; }else if (!strncmp (str,"tboot_noforce" ,13 )) {pr_info ("Intel-IOMMU: not forcing on after tboot. This could expose security risk for tboot\n" ); intel_iommu_tboot_noforce =1 ;
drivers/iommu/intel/Kconfig +1 -0
@@ -84,6 +84,7 @@ config INTEL_IOMMU_FLOPPY_WA config INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON bool "Enable Intel IOMMU scalable mode by default" + default y help Selecting this option will enable by default the scalable mode if hardware presents the capability. The scalable mode is defined in
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 5/9 iommu/vt-d: Preset A/D bits for user space DMA usage
In reply to Lu BaoluWe preset the access and dirty bits for IOVA over first level usage only for the kernel DMA (i.e., when domain type is IOMMU_DOMAIN_DMA). We should also preset the FL A/D for user space DMA usage. The idea is that even the user space A/D bit memory write is unnecessary. We should avoid it to minimize the overhead.
--- drivers/iommu/intel/iommu.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
drivers/iommu/intel/iommu.c +3 -7
@@ -2334,13 +2334,9 @@ __domain_mapping(struct dmar_domain *domain, unsigned long iov_pfn, attr = prot & (DMA_PTE_READ |DMA_PTE_WRITE |DMA_PTE_SNP ); attr |=DMA_FL_PTE_PRESENT ;if (domain_use_first_level (domain)) { - attr |=DMA_FL_PTE_XD |DMA_FL_PTE_US ; - -if (iommu_is_dma_domain (&domain->domain)) { - attr |=DMA_FL_PTE_ACCESS ; -if (prot &DMA_PTE_WRITE ) - attr |=DMA_FL_PTE_DIRTY ; - } + attr |=DMA_FL_PTE_XD |DMA_FL_PTE_US |DMA_FL_PTE_ACCESS ; +if (prot &DMA_PTE_WRITE ) + attr |=DMA_FL_PTE_DIRTY ; } pteval = ((phys_addr_t )phys_pfn <<VTD_PAGE_SHIFT ) | attr;
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 6/9 iommu/vt-d: Allow devices to have more than 32 outstanding PRs
In reply to Lu BaoluThe minimum per-IOMMU PRQ queue size is one 4K page, this is more entries than the hardcoded limit of 32 in the current VT-d code. Some devices can support up to 512 outstanding PRQs but underutilized by this limit of 32. Although, 32 gives some rough fairness when multiple devices share the same IOMMU PRQ queue, but far from optimal for customized use case. This extends the per-IOMMU PRQ queue size to four 4K pages and let the devices have as many outstanding page requests as they can.
--- include/linux/intel-svm.h | 5 +++++ drivers/iommu/intel/iommu.c | 3 ++- drivers/iommu/intel/svm.c | 4 ---- 3 files changed, 7 insertions(+), 5 deletions(-)
include/linux/intel-svm.h +5 -0
@@ -14,6 +14,11 @@#define SVM_REQ_EXEC (1<<1)#define SVM_REQ_PRIV (1<<0) +/* Page Request Queue depth */ +#define PRQ_ORDER 2 +#define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) +#define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) +/* * The SVM_FLAG_SUPERVISOR_MODE flag requests a PASID which can be used only * for access to kernel addresses. No IOTLB flushes are automatically done
drivers/iommu/intel/iommu.c +2 -1
@@ -33,6 +33,7 @@#include <linux/iommu.h> #include <linux/dma-iommu.h> #include <linux/intel-iommu.h> +#include <linux/intel-svm.h> #include <linux/syscore_ops.h> #include <linux/tboot.h> #include <linux/dmi.h>
@@ -1541,7 +1542,7 @@ static void iommu_enable_dev_iotlb(struct device_domain_info *info)if (info->pri_supported && (info->pasid_enabled ?pci_prg_resp_pasid_required (pdev) :1 ) && - !pci_reset_pri (pdev) && !pci_enable_pri (pdev,32 )) + !pci_reset_pri (pdev) && !pci_enable_pri (pdev,PRQ_DEPTH )) info->pri_enabled =1 ;#endif if (info->ats_supported && pci_ats_page_aligned(pdev) &&
drivers/iommu/intel/svm.c +0 -4
@@ -31,8 +31,6 @@ static irqreturn_t prq_event_thread(int irq, void *d);static void intel_svm_drain_prq (struct device *dev,u32 pasid);#define to_intel_svm_dev (handle) container_of(handle, struct intel_svm_dev, sva) -#define PRQ_ORDER 0 -static DEFINE_XARRAY_ALLOC (pasid_private_array);static int pasid_private_add (ioasid_t pasid,void *priv) {
@@ -725,8 +723,6 @@ struct page_req_dsc {u64 priv_data[2 ]; }; -#define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) -static bool is_canonical_address (u64 addr) {int shift =64 - (__VIRTUAL_MASK_SHIFT +1 );
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 7/9 iommu/vt-d: Drop the kernel doc annotation
In reply to Lu BaoluFrom: Andy Shevchenko <[email protected]>
Kernel doc validator is unhappy with the following
.../perf.c:16: warning: Function parameter or member 'latency_lock' not described in 'DEFINE_SPINLOCK' .../perf.c:16: warning: expecting prototype for perf.c(). Prototype was for DEFINE_SPINLOCK() instead
Drop kernel doc annotation since the top comment is not in the required format.
--- drivers/iommu/intel/perf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel/perf.c +1 -1
@@ -1,5 +1,5 @@// SPDX-License-Identifier: GPL-2.0 -/** +/* * perf.c - performance monitor * * Copyright (C) 2021 Intel Corporation
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 8/9 iommu/vt-d: Use pasid_pte_is_present() helper function
In reply to Lu BaoluFrom: Liu Yi L <[email protected]>
Use the pasid_pte_is_present() helper for present bit check in the intel_pasid_tear_down_entry().
--- drivers/iommu/intel/pasid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel/pasid.c +1 -1
@@ -517,7 +517,7 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev,if (WARN_ON (!pte))return ; -if (!(pte->val[0 ] &PASID_PTE_PRESENT )) +if (!pasid_pte_is_present (pte))return ; did =pasid_get_domain_id (pte);
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 9/9 iommu/vt-d: Add present bit check in pasid entry setup helpers
In reply to Lu BaoluFrom: Liu Yi L <[email protected]>
The helper functions should not modify the pasid entries which are still in use. Add a check against present bit.
--- drivers/iommu/intel/pasid.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/iommu/intel/pasid.c +16 -0
@@ -540,6 +540,10 @@ void intel_pasid_tear_down_entry(struct intel_iommu *iommu, struct device *dev,devtlb_invalidation_with_pasid (iommu, dev, pasid); } +/* +* This function flushes cache for a newly setup pasid table entry. +* Caller of it should not modify the in-use pasid table entries. +*/ static void pasid_flush_caches (struct intel_iommu *iommu,struct pasid_entry *pte,u32 pasid,u16 did)
@@ -591,6 +595,10 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu,if (WARN_ON (!pte))return -EINVAL ; +/* Caller must ensure PASID entry is not in use. */ +if (pasid_pte_is_present (pte)) +return -EBUSY ; +pasid_clear_entry (pte);/* Setup the first level page table pointer: */
@@ -690,6 +698,10 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu,return -ENODEV ; } +/* Caller must ensure PASID entry is not in use. */ +if (pasid_pte_is_present (pte)) +return -EBUSY ; +pasid_clear_entry (pte);pasid_set_domain_id (pte, did);pasid_set_slptr (pte, pgd_val);
@@ -729,6 +741,10 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,return -ENODEV ; } +/* Caller must ensure PASID entry is not in use. */ +if (pasid_pte_is_present (pte)) +return -EBUSY ; +pasid_clear_entry (pte);pasid_set_domain_id (pte, did);pasid_set_address_width (pte, iommu->agaw);
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Joerg Roedel
maintainerQuoted 14 lines · Lu Baolu
Andy Shevchenko (1):
iommu/vt-d: Drop the kernel doc annotationLiu Yi L (2):
iommu/vt-d: Usepasid_pte_is_present()helper function iommu/vt-d: Add present bit check in pasid entry setup helpersLu Baolu (5):
iommu/vt-d: Update the virtual command related registers iommu/vt-d: RefactorKconfiga bit
iommu/vt-d: Enable Intel IOMMU scalable mode by default iommu/vt-d: Preset A/D bits for user space DMA usage iommu/vt-d: Allow devices to have more than 32 outstanding PRs
Quoted 2 lines
Zhen Lei (1):
iommu/vt-d: Remove unnecessary oom message
Applied, thanks Baolu.
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu