iommu: Enhance IOMMU default DMA mode build options
Subsystems (4)
v15 cover iommu: Enhance IOMMU default DMA mode build options
This is a reboot of Zhen Lei's series from a couple of years ago, which never made it across the line.
I still think that it has some value, so taking up the mantle.
Motivation:
Allow lazy mode be default mode for DMA domains for all ARCHs, and not only those who hardcode it (to be lazy). For ARM64, currently we must use a kernel command line parameter to use lazy mode, which is less than ideal.
I have now included the print for strict/lazy mode, which I originally sent in:
https://lore.kernel.org/linux-iommu/[email protected]/
There was some concern there about drivers and their custom prints conflicting with the print in that patch, but I think that it should be ok.
Based on v5.14-rc1
Differences to v14:
- Add RB tags (Thanks!)
- Rebase
Differences to v13:
- Improve strict mode deprecation messages and cut out some kernel-parameters.txt legacy description
- Add tag in 1/6
- use pr_info_once() for vt-d message about VM and caching
Differences to v12:
- Add Robin's RB tags (thanks!)
- Add a patch to mark x86 strict cmdline params as deprecated
- Improve wording in Kconfig change and tweak iommu_dma_strict declaration
John Garry (3):
iommu: Deprecate Intel and AMD cmdline methods to enable strict mode
iommu: Print strict or lazy mode at init time
iommu: Remove mode argument from iommu_set_dma_strict()
Zhen Lei (3):
iommu: Enhance IOMMU default DMA mode build options iommu/vt-d: Add support for IOMMU default DMA mode build options iommu/amd: Add support for IOMMU default DMA mode build options
.../admin-guide/kernel-parameters.txt | 12 ++---- drivers/iommu/Kconfig | 41 +++++++++++++++++++ drivers/iommu/amd/amd_iommu_types.h | 6 --- drivers/iommu/amd/init.c | 7 ++-- drivers/iommu/amd/iommu.c | 6 --- drivers/iommu/intel/iommu.c | 16 ++++---- drivers/iommu/iommu.c | 12 ++++-- include/linux/iommu.h | 2 +- 8 files changed, 65 insertions(+), 37 deletions(-)
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v15 1/6 iommu: Deprecate Intel and AMD cmdline methods to enable strict mode
Now that the x86 drivers support iommu.strict, deprecate the custom methods.
--- Documentation/admin-guide/kernel-parameters.txt | 9 ++------- drivers/iommu/amd/init.c | 4 +++- drivers/iommu/intel/iommu.c | 1 + 3 files changed, 6 insertions(+), 8 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +2 -7
@@ -290,10 +290,7 @@ amd_iommu= [HW,X86-64] Pass parameters to the AMD IOMMU driver in the system. Possible values are: - fullflush - enable flushing of IO/TLB entries when - they are unmapped. Otherwise they are - flushed before they will be reused, which - is a lot of faster + fullflush - Deprecated, equivalent to iommu.strict=1 off - do not initialize any AMD IOMMU found in the system force_isolation - Force device isolation for all
@@ -1944,9 +1941,7 @@ this case, gfx device will use physical address for DMA. strict [Default Off] - With this option on every unmap_single operation will - result in a hardware IOTLB flush operation as opposed - to batching them for performance. + Deprecated, equivalent to iommu.strict=1. sp_off [Default Off] By default, super page will be supported if Intel IOMMU has the capability. With this option, super page will
drivers/iommu/amd/init.c +3 -1
@@ -3098,8 +3098,10 @@ static int __init parse_amd_iommu_intr(char *str)static int __init parse_amd_iommu_options (char *str) {for (; *str; ++str) { -if (strncmp (str,"fullflush" ,9 ) ==0 ) +if (strncmp (str,"fullflush" ,9 ) ==0 ) { +pr_warn ("amd_iommu=fullflush deprecated; use iommu.strict=1 instead\n" ); amd_iommu_unmap_flush = true; + }if (strncmp (str,"force_enable" ,12 ) ==0 ) amd_iommu_force_enable = true;if (strncmp (str,"off" ,3 ) ==0 )
drivers/iommu/intel/iommu.c +1 -0
@@ -454,6 +454,7 @@ static int __init intel_iommu_setup(char *str)pr_warn ("intel_iommu=forcedac deprecated; use iommu.forcedac instead\n" ); iommu_dma_forcedac = true; }else if (!strncmp (str,"strict" ,6 )) { +pr_warn ("intel_iommu=strict deprecated; use iommu.strict=1 instead\n" );pr_info ("Disable batched IOTLB flush\n" ); intel_iommu_strict =1 ; }else if (!strncmp (str,"sp_off" ,6 )) {
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v15 2/6 iommu: Print strict or lazy mode at init time
In reply to John GarryAs well as the default domain type, it's useful to know whether strict or lazy for DMA domains, so add this info in a separate print.
The (stict/lazy) mode may be also set via iommu.strict earlyparm, but this will be processed prior to iommu_subsys_init(), so that print will be accurate for drivers which don't set the mode via custom means.
For the drivers which set the mode via custom means - AMD and Intel drivers
- they maintain prints to inform a change in policy or that custom cmdline methods to change policy are deprecated.
--- drivers/iommu/iommu.c | 5 +++++ 1 file changed, 5 insertions(+)
drivers/iommu/iommu.c +5 -0
@@ -138,6 +138,11 @@ static int __init iommu_subsys_init(void) (iommu_cmd_line &IOMMU_CMD_LINE_DMA_API ) ? "(set via kernel command line)" : ""); +pr_info ("DMA domain TLB invalidation policy: %s mode %s\n" , + iommu_dma_strict ?"strict" :"lazy" , + (iommu_cmd_line &IOMMU_CMD_LINE_STRICT ) ? +"(set via kernel command line)" :"" ); +return 0 ; }subsys_initcall (iommu_subsys_init);
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v15 3/6 iommu: Enhance IOMMU default DMA mode build options
In reply to John GarryFrom: Zhen Lei <[email protected]>
First, add build options IOMMU_DEFAULT_{LAZY|STRICT}, so that we have the opportunity to set {lazy|strict} mode as default at build time. Then put the two config options in an choice, as they are mutually exclusive.
--- .../admin-guide/kernel-parameters.txt | 3 +- drivers/iommu/Kconfig | 40 +++++++++++++++++++ drivers/iommu/iommu.c | 2 +- 3 files changed, 43 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +2 -1
@@ -2042,9 +2042,10 @@ throughput at the cost of reduced device isolation. Will fall back to strict mode if not supported by the relevant IOMMU driver. - 1 - Strict mode (default). + 1 - Strict mode. DMA unmap operations invalidate IOMMU hardware TLBs synchronously. + unset - Use value of CONFIG_IOMMU_DEFAULT_{LAZY,STRICT}. Note: on x86, the default behaviour depends on the equivalent driver-specific parameters, but a strict mode explicitly specified by either method takes
drivers/iommu/Kconfig +40 -0
@@ -90,6 +90,46 @@ config IOMMU_DEFAULT_PASSTHROUGH If unsure, say N here. +choice + prompt "IOMMU default DMA IOTLB invalidation mode" + depends on IOMMU_DMA + + default IOMMU_DEFAULT_STRICT + help + This option allows an IOMMU DMA IOTLB invalidation mode to be + chosen at build time, to override the default mode of each ARCH, + removing the need to pass in kernel parameters through command line. + It is still possible to provide common boot params to override this + config. + + If unsure, keep the default. + +config IOMMU_DEFAULT_STRICT + bool "strict" + help + For every IOMMU DMA unmap operation, the flush operation of IOTLB and + the free operation of IOVA are guaranteed to be done in the unmap + function. + +config IOMMU_DEFAULT_LAZY + bool "lazy" + help + Support lazy mode, where for every IOMMU DMA unmap operation, the + flush operation of IOTLB and the free operation of IOVA are deferred. + They are only guaranteed to be done before the related IOVA will be + reused. + + The isolation provided in this mode is not as secure as STRICT mode, + such that a vulnerable time window may be created between the DMA + unmap and the mappings cached in the IOMMU IOTLB or device TLB + finally being invalidated, where the device could still access the + memory which has already been unmapped by the device driver. + However this mode may provide better performance in high throughput + scenarios, and is still considerably more secure than passthrough + mode or no IOMMU. + +endchoice + config OF_IOMMU def_bool y depends on OF && IOMMU_API
drivers/iommu/iommu.c +1 -1
@@ -29,7 +29,7 @@ static struct kset *iommu_group_kset;static DEFINE_IDA (iommu_group_ida);static unsigned int iommu_def_domain_type__read_mostly ; -static bool iommu_dma_strict__read_mostly = true; +static bool iommu_dma_strict__read_mostly =IS_ENABLED (CONFIG_IOMMU_DEFAULT_STRICT );static u32 iommu_cmd_line__read_mostly ;struct iommu_group {
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v15 4/6 iommu/vt-d: Add support for IOMMU default DMA mode build options
In reply to John GarryFrom: Zhen Lei <[email protected]>
Make IOMMU_DEFAULT_LAZY default for when INTEL_IOMMU config is set, as is current behaviour.
Also delete global flag intel_iommu_strict:
- In intel_iommu_setup(), call iommu_set_dma_strict(true) directly. Also remove the print, as iommu_subsys_init() prints the mode and we have already marked this param as deprecated.
- For cap_caching_mode() check in intel_iommu_setup(), call iommu_set_dma_strict(true) directly; also reword the accompanying print with a level downgrade and also add the missing '\n'.
- For Ironlake GPU, again call iommu_set_dma_strict(true) directly and keep the accompanying print.
--- drivers/iommu/Kconfig | 1 + drivers/iommu/intel/iommu.c | 15 ++++++--------- 2 files changed, 7 insertions(+), 9 deletions(-)
drivers/iommu/Kconfig +1 -0
@@ -94,6 +94,7 @@ choice prompt "IOMMU default DMA IOTLB invalidation mode" depends on IOMMU_DMA + default IOMMU_DEFAULT_LAZY if INTEL_IOMMU default IOMMU_DEFAULT_STRICT help This option allows an IOMMU DMA IOTLB invalidation mode to be
drivers/iommu/intel/iommu.c +6 -9
@@ -361,7 +361,6 @@ int intel_iommu_enabled = 0;EXPORT_SYMBOL_GPL (intel_iommu_enabled);static int dmar_map_gfx =1 ; -static int intel_iommu_strict;static int intel_iommu_superpage =1 ;static int iommu_identity_mapping;static int iommu_skip_te_disable;
@@ -455,8 +454,7 @@ static int __init intel_iommu_setup(char *str) iommu_dma_forcedac = true; }else if (!strncmp (str,"strict" ,6 )) {pr_warn ("intel_iommu=strict deprecated; use iommu.strict=1 instead\n" ); -pr_info ("Disable batched IOTLB flush\n" ); - intel_iommu_strict =1 ; +iommu_set_dma_strict (true); }else if (!strncmp (str,"sp_off" ,6 )) {pr_info ("Disable supported super page\n" ); intel_iommu_superpage =0 ;
@@ -4382,9 +4380,9 @@ int __init intel_iommu_init(void)* is likely to be much lower than the overhead of synchronizing * the virtual and physical IOMMU page-tables. */ -if (!intel_iommu_strict &&cap_caching_mode (iommu->cap)) { -pr_warn ("IOMMU batching is disabled due to virtualization" ); - intel_iommu_strict =1 ; +if (cap_caching_mode (iommu->cap)) { +pr_info_once ("IOMMU batching disallowed due to virtualization\n" ); +iommu_set_dma_strict (true); } iommu_device_sysfs_add(&iommu->iommu, NULL, intel_iommu_groups,
@@ -4393,7 +4391,6 @@ int __init intel_iommu_init(void) }up_read (&dmar_global_lock); -iommu_set_dma_strict (intel_iommu_strict);bus_set_iommu (&pci_bus_type, &intel_iommu_ops);if (si_domain && !hw_pass_through)register_memory_notifier (&intel_iommu_memory_nb);
@@ -5703,8 +5700,8 @@ static void quirk_calpella_no_shadow_gtt(struct pci_dev *dev) }else if (dmar_map_gfx) {/* we have to ensure the gfx device is idle before we flush */ pci_info (dev,"Disabling batched IOTLB flush on Ironlake\n" ); - intel_iommu_strict =1 ; - } +iommu_set_dma_strict (true); + } }DECLARE_PCI_FIXUP_HEADER (PCI_VENDOR_ID_INTEL ,0x0040 , quirk_calpella_no_shadow_gtt);DECLARE_PCI_FIXUP_HEADER (PCI_VENDOR_ID_INTEL ,0x0044 , quirk_calpella_no_shadow_gtt);
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v15 5/6 iommu/amd: Add support for IOMMU default DMA mode build options
In reply to John GarryFrom: Zhen Lei <[email protected]>
Make IOMMU_DEFAULT_LAZY default for when AMD_IOMMU config is set, which matches current behaviour.
For "fullflush" param, just call iommu_set_dma_strict(true) directly.
Since we get a strict vs lazy mode print already in iommu_subsys_init(), and maintain a deprecation print when "fullflush" param is passed, drop the prints in amd_iommu_init_dma_ops().
Finally drop global flag amd_iommu_unmap_flush, as it has no longer has any purpose.
--- drivers/iommu/Kconfig | 2 +- drivers/iommu/amd/amd_iommu_types.h | 6 ------ drivers/iommu/amd/init.c | 3 +-- drivers/iommu/amd/iommu.c | 6 ------ 4 files changed, 2 insertions(+), 15 deletions(-)
drivers/iommu/Kconfig +1 -1
@@ -94,7 +94,7 @@ choice prompt "IOMMU default DMA IOTLB invalidation mode" depends on IOMMU_DMA - default IOMMU_DEFAULT_LAZY if INTEL_IOMMU + default IOMMU_DEFAULT_LAZY if (AMD_IOMMU || INTEL_IOMMU) default IOMMU_DEFAULT_STRICT help This option allows an IOMMU DMA IOTLB invalidation mode to be
drivers/iommu/amd/amd_iommu_types.h +0 -6
@@ -779,12 +779,6 @@ extern u16 amd_iommu_last_bdf;/* allocation bitmap for domain ids */ extern unsigned long *amd_iommu_pd_alloc_bitmap; -/* -* If true, the addresses will be flushed on unmap time, not when -* they are reused -*/ -extern bool amd_iommu_unmap_flush; -/* Smallest max PASID supported by any IOMMU in the system */ extern u32 amd_iommu_max_pasid;
drivers/iommu/amd/init.c +1 -2
@@ -161,7 +161,6 @@ u16 amd_iommu_last_bdf; /* largest PCI device id we haveto handle */ LIST_HEAD (amd_iommu_unity_map);/* a list of required unity mappings we find in ACPI */ -bool amd_iommu_unmap_flush;/* if true, flush on every unmap */ LIST_HEAD (amd_iommu_list);/* list of all AMD IOMMUs in the system */
@@ -3100,7 +3099,7 @@ static int __init parse_amd_iommu_options(char *str)for (; *str; ++str) {if (strncmp (str,"fullflush" ,9 ) ==0 ) {pr_warn ("amd_iommu=fullflush deprecated; use iommu.strict=1 instead\n" ); - amd_iommu_unmap_flush = true; +iommu_set_dma_strict (true); }if (strncmp (str,"force_enable" ,12 ) ==0 ) amd_iommu_force_enable = true;
drivers/iommu/amd/iommu.c +0 -6
@@ -1775,12 +1775,6 @@ void amd_iommu_domain_update(struct protection_domain *domain)static void __init amd_iommu_init_dma_ops (void ) { swiotlb = (iommu_default_passthrough () || sme_me_mask) ?1 :0 ; - -if (amd_iommu_unmap_flush) -pr_info ("IO/TLB flush on unmap enabled\n" ); -else -pr_info ("Lazy IO/TLB flushing enabled\n" ); -iommu_set_dma_strict (amd_iommu_unmap_flush); }int __init amd_iommu_init_api (void )
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
PATCH v15 6/6 iommu: Remove mode argument from iommu_set_dma_strict()
In reply to John GarryWe only ever now set strict mode enabled in iommu_set_dma_strict(), so just remove the argument.
--- drivers/iommu/amd/init.c | 2 +- drivers/iommu/intel/iommu.c | 6 +++--- drivers/iommu/iommu.c | 5 ++--- include/linux/iommu.h | 2 +- 4 files changed, 7 insertions(+), 8 deletions(-)
drivers/iommu/amd/init.c +1 -1
@@ -3099,7 +3099,7 @@ static int __init parse_amd_iommu_options(char *str)for (; *str; ++str) {if (strncmp (str,"fullflush" ,9 ) ==0 ) {pr_warn ("amd_iommu=fullflush deprecated; use iommu.strict=1 instead\n" ); -iommu_set_dma_strict (true); +iommu_set_dma_strict (); }if (strncmp (str,"force_enable" ,12 ) ==0 ) amd_iommu_force_enable = true;
drivers/iommu/intel/iommu.c +3 -3
@@ -454,7 +454,7 @@ static int __init intel_iommu_setup(char *str) iommu_dma_forcedac = true; }else if (!strncmp (str,"strict" ,6 )) {pr_warn ("intel_iommu=strict deprecated; use iommu.strict=1 instead\n" ); -iommu_set_dma_strict (true); +iommu_set_dma_strict (); }else if (!strncmp (str,"sp_off" ,6 )) {pr_info ("Disable supported super page\n" ); intel_iommu_superpage =0 ;
@@ -4382,7 +4382,7 @@ int __init intel_iommu_init(void)*/ if (cap_caching_mode (iommu->cap)) {pr_info_once ("IOMMU batching disallowed due to virtualization\n" ); -iommu_set_dma_strict (true); +iommu_set_dma_strict (); } iommu_device_sysfs_add(&iommu->iommu, NULL, intel_iommu_groups,
@@ -5700,7 +5700,7 @@ static void quirk_calpella_no_shadow_gtt(struct pci_dev *dev) }else if (dmar_map_gfx) {/* we have to ensure the gfx device is idle before we flush */ pci_info (dev,"Disabling batched IOTLB flush on Ironlake\n" ); -iommu_set_dma_strict (true); +iommu_set_dma_strict (); } }DECLARE_PCI_FIXUP_HEADER (PCI_VENDOR_ID_INTEL ,0x0040 , quirk_calpella_no_shadow_gtt);
drivers/iommu/iommu.c +2 -3
@@ -349,10 +349,9 @@ static int __init iommu_dma_setup(char *str) }early_param ("iommu.strict" , iommu_dma_setup); -void iommu_set_dma_strict (bool strict) +void iommu_set_dma_strict (void ) { -if (strict || !(iommu_cmd_line &IOMMU_CMD_LINE_STRICT )) - iommu_dma_strict = strict; + iommu_dma_strict = true; }bool iommu_get_dma_strict (struct iommu_domain *domain)
include/linux/iommu.h +1 -1
@@ -476,7 +476,7 @@ int iommu_enable_nesting(struct iommu_domain *domain);int iommu_set_pgtable_quirks (struct iommu_domain *domain,unsigned long quirks); -void iommu_set_dma_strict (bool val); +void iommu_set_dma_strict (void );bool iommu_get_dma_strict (struct iommu_domain *domain); extern int report_iommu_fault(struct iommu_domain *domain, struct device *dev,
Signature · 6 lines
2.26.2 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 9 lines · John Garry
John Garry (3):
iommu: Deprecate Intel and AMD cmdline methods to enable strict mode
iommu: Print strict or lazy mode at init time
iommu: Remove mode argument fromiommu_set_dma_strict()Zhen Lei (3):
iommu: Enhance IOMMU default DMA mode build options iommu/vt-d: Add support for IOMMU default DMA mode build options iommu/amd: Add support for IOMMU default DMA mode build options
Applied to iommu/core, thanks.