iommu/vt-d: Convert to use static identity domain
Subsystems (2)
Lu Baolu
maintainerv4 cover iommu/vt-d: Convert to use static identity domain
Intel's IOMMU driver used a special domain called 1:1 mapping domain to support the domain of type IOMMU_DOMAIN_IDENTITY, which enables device drivers to directly utilize physical addresses for DMA access despite the presence of IOMMU units.
The implementation of the 1:1 mapping domain is influenced by hardware differences. While modern Intel VT-d implementations support hardware passthrough translation mode, earlier versions lacked this feature, which requires a more complex implementation approach.
The 1:1 mapping domain for earlier hardware was implemented by associating a DMA domain with an IOVA (IO Virtual Address) equivalent to the physical address. While, for most hardware supporting passthrough mode, simply setting the hardware's passthrough mode is sufficient. These two modes were merged together in si_domain, which is a special DMA domain sharing the domain ops of an ordinary DMA domain.
As the iommu core has evolved, it has introduced global static identity domain with "never fail" attach semantics. This means that the domain is always available and cannot fail to attach. The iommu driver now assigns this domain directly at iommu_ops->identity_domain instead of allocating it through the domain allocation interface.
This converts the Intel IOMMU driver to embrace the global static identity domain. For early legacy hardwares that don't support passthrough translation mode, ask the iommu core to use a DMA type of default domain. For modern hardwares that support passthrough translation mode, implement a static global identity domain.
The whole series is also available at
https://github.com/LuBaolu/intel-iommu/commits/vtd-static-identity-domain-v4
Change log:
v4:
- Add a new patch to remove has_iotlb_device flag as suggested by Jason.
https://lore.kernel.org/linux-iommu/[email protected]/
v3: https://lore.kernel.org/linux-iommu/[email protected]/
- Kevin worried that some graphic devices might still require identity domain. Forcing DMA domain for those drivers might break the existing functionality.
https://lore.kernel.org/linux-iommu/BN9PR11MB52761FF9AB496B422596DDDF8C8AA@BN9PR11MB5276.namprd11.prod.outlook.com/
After confirmed with the graphic community, we decouple "igfx_off" kernel command from graphic identity mapping with the following commits:
ba00196ca41c ("iommu/vt-d: Decouple igfx_off from graphic identity mapping") 4b8d18c0c986 ("iommu/vt-d: Remove INTEL_IOMMU_BROKEN_GFX_WA").
v2: https://lore.kernel.org/linux-iommu/[email protected]/
- Re-orgnize the patches by removing 1:1 mappings before implementing global static domain.
v1: https://lore.kernel.org/linux-iommu/[email protected]/
Lu Baolu (7):
iommu/vt-d: Require DMA domain if hardware not support passthrough iommu/vt-d: Remove identity mappings from si_domain iommu/vt-d: Always reserve a domain ID for identity setup iommu/vt-d: Remove has_iotlb_device flag
iommu/vt-d: Factor out helpers from domain_context_mapping_one() iommu/vt-d: Add support for static identity domain iommu/vt-d: Cleanup si_domain
drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/iommu.c | 468 +++++++++++++++-------------------- drivers/iommu/intel/nested.c | 2 - 3 files changed, 201 insertions(+), 271 deletions(-)
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 1/7 iommu/vt-d: Require DMA domain if hardware not support passthrough
The iommu core defines the def_domain_type callback to query the iommu driver about hardware capability and quirks. The iommu driver should declare IOMMU_DOMAIN_DMA requirement for hardware lacking pass-through capability.
Earlier VT-d hardware implementations did not support pass-through translation mode. The iommu driver relied on a paging domain with all physical system memory addresses identically mapped to the same IOVA to simulate pass-through translation before the def_domain_type was introduced and it has been kept until now. It's time to adjust it now to make the Intel iommu driver follow the def_domain_type semantics.
--- drivers/iommu/intel/iommu.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/iommu/intel/iommu.c +10 -0
@@ -2149,6 +2149,16 @@ static bool device_rmrr_is_relaxable(struct device *dev)static int device_def_domain_type (struct device *dev) { +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; + +/* +* Hardware does not support the passthrough translation mode. +* Always use a dynamaic mapping domain. +*/ +if (!ecap_pass_through (iommu->ecap)) +return IOMMU_DOMAIN_DMA ; +if (dev_is_pci (dev)) {struct pci_dev *pdev =to_pci_dev (dev);
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 2/7 iommu/vt-d: Remove identity mappings from si_domain
In reply to Lu BaoluAs the driver has enforced DMA domains for devices managed by an IOMMU hardware that doesn't support passthrough translation mode, there is no need for static identity mappings in the si_domain. Remove the identity mapping code to avoid dead code.
--- drivers/iommu/intel/iommu.c | 122 ++---------------------------------- 1 file changed, 4 insertions(+), 118 deletions(-)
drivers/iommu/intel/iommu.c +4 -118
@@ -167,14 +167,7 @@ static void device_rbtree_remove(struct device_domain_info *info)spin_unlock_irqrestore (&iommu->device_rbtree_lock, flags); } -/* -* This domain is a statically identity mapping domain. -* 1. This domain creats a static 1:1 mapping to all usable memory. -* 2. It maps to each iommu if successful. -* 3. Each iommu mapps to this domain if successful. -*/ static struct dmar_domain *si_domain; -static int hw_pass_through =1 ;struct dmar_rmrr_unit {struct list_head list;/* list of rmrr units */
@@ -1647,7 +1640,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain,struct context_entry *context;int agaw, ret; -if (hw_pass_through &&domain_type_is_si (domain)) +if (domain_type_is_si (domain)) translation =CONTEXT_TT_PASS_THROUGH ; pr_debug("Set context mapping for %02x:%02x.%d\n",
@@ -1998,29 +1991,10 @@ static bool dev_is_real_dma_subdevice(struct device *dev)pci_real_dma_dev (to_pci_dev (dev)) !=to_pci_dev (dev); } -static int iommu_domain_identity_map (struct dmar_domain *domain, -unsigned long first_vpfn, -unsigned long last_vpfn) -{ -/* -* RMRR range might have overlap with physical memory range, -* clear it first -*/ -dma_pte_clear_range (domain, first_vpfn, last_vpfn); - -return __domain_mapping (domain, first_vpfn, - first_vpfn, last_vpfn - first_vpfn +1 , -DMA_PTE_READ |DMA_PTE_WRITE ,GFP_KERNEL ); -} -static int md_domain_init (struct dmar_domain *domain,int guest_width); -static int __init si_domain_init (int hw) +static int __init si_domain_init (void ) { -struct dmar_rmrr_unit *rmrr; -struct device *dev; -int i, nid, ret; - si_domain =alloc_domain (IOMMU_DOMAIN_IDENTITY );if (!si_domain)return -EFAULT ;
@@ -2031,44 +2005,6 @@ static int __init si_domain_init(int hw)return -EFAULT ; } -if (hw) -return 0 ; - -for_each_online_node (nid) { -unsigned long start_pfn, end_pfn; -int i; - -for_each_mem_pfn_range (i, nid, &start_pfn, &end_pfn,NULL ) { - ret =iommu_domain_identity_map (si_domain, -mm_to_dma_pfn_start (start_pfn), -mm_to_dma_pfn_end (end_pfn-1 )); -if (ret) -return ret; - } - } - -/* -* Identity map the RMRRs so that devices with RMRRs could also use -* the si_domain. -*/ -for_each_rmrr_units (rmrr) { -for_each_active_dev_scope (rmrr->devices, rmrr->devices_cnt, - i, dev) { -unsigned long long start = rmrr->base_address; -unsigned long long end = rmrr->end_address; - -if (WARN_ON (end < start || - end >>agaw_to_width (si_domain->agaw))) -continue ; - - ret =iommu_domain_identity_map (si_domain, -mm_to_dma_pfn_start (start >>PAGE_SHIFT ), -mm_to_dma_pfn_end (end >>PAGE_SHIFT )); -if (ret) -return ret; - } - } -return 0 ; }
@@ -2094,7 +2030,7 @@ static int dmar_domain_attach_device(struct dmar_domain *domain,if (!sm_supported (iommu)) ret =domain_context_mapping (domain, dev); -else if (hw_pass_through &&domain_type_is_si (domain)) +else if (domain_type_is_si (domain)) ret =intel_pasid_setup_pass_through (iommu, dev,IOMMU_NO_PASID );else if (domain->use_first_level) ret =domain_setup_first_level (iommu, domain, dev,IOMMU_NO_PASID );
@@ -2449,8 +2385,6 @@ static int __init init_dmars(void) } } -if (!ecap_pass_through (iommu->ecap)) - hw_pass_through =0 ;intel_svm_check (iommu); }
@@ -2466,7 +2400,7 @@ static int __init init_dmars(void)check_tylersburg_isoch (); - ret =si_domain_init (hw_pass_through); + ret =si_domain_init ();if (ret) goto free_iommu;
@@ -2893,12 +2827,6 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)if (ret) goto out; -if (hw_pass_through && !ecap_pass_through (iommu->ecap)) { -pr_warn ("%s: Doesn't support hardware pass through.\n" , - iommu->name); -return -ENXIO ; - } - sp =domain_update_iommu_superpage (NULL , iommu) -1 ;if (sp >=0 && !(cap_super_page_val (iommu->cap) & (1 << sp))) { pr_warn("%s: Doesn't support large page.\n",
@@ -3149,43 +3077,6 @@ int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)return 0 ; } -static int intel_iommu_memory_notifier (struct notifier_block *nb, -unsigned long val,void *v) -{ -struct memory_notify *mhp = v; -unsigned long start_vpfn =mm_to_dma_pfn_start (mhp->start_pfn); -unsigned long last_vpfn =mm_to_dma_pfn_end (mhp->start_pfn + - mhp->nr_pages -1 ); - -switch (val) { -case MEM_GOING_ONLINE : -if (iommu_domain_identity_map (si_domain, - start_vpfn, last_vpfn)) { -pr_warn ("Failed to build identity map for [%lx-%lx]\n" , - start_vpfn, last_vpfn); -return NOTIFY_BAD ; - } -break ; - -case MEM_OFFLINE : -case MEM_CANCEL_ONLINE : - { -LIST_HEAD (freelist); - -domain_unmap (si_domain, start_vpfn, last_vpfn, &freelist); -iommu_put_pages_list (&freelist); - } -break ; - } - -return NOTIFY_OK ; -} - -static struct notifier_block intel_iommu_memory_nb = { - .notifier_call = intel_iommu_memory_notifier, - .priority =0 -}; -static void intel_disable_iommus (void ) {struct intel_iommu *iommu =NULL ;
@@ -3482,12 +3373,7 @@ int __init intel_iommu_init(void)iommu_pmu_register (iommu); } -up_read (&dmar_global_lock); -if (si_domain && !hw_pass_through) -register_memory_notifier (&intel_iommu_memory_nb); - -down_read (&dmar_global_lock);if (probe_acpi_namespace_devices ())pr_warn ("ACPI name space devices didn't probe correctly\n" );
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 3/7 iommu/vt-d: Always reserve a domain ID for identity setup
In reply to Lu BaoluWe will use a global static identity domain. Reserve a static domain ID for it.
--- drivers/iommu/intel/iommu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/iommu/intel/iommu.c +3 -3
@@ -1440,10 +1440,10 @@ static int iommu_init_domains(struct intel_iommu *iommu)* entry for first-level or pass-through translation modes should * be programmed with a domain id different from those used for * second-level or nested translation. We reserve a domain id for -* this purpose. +* this purpose. This domain id is also used for identity domain +* in legacy mode. */ -if (sm_supported (iommu)) -set_bit (FLPT_DEFAULT_DID , iommu->domain_ids); +set_bit (FLPT_DEFAULT_DID , iommu->domain_ids);return 0 ; }
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 4/7 iommu/vt-d: Remove has_iotlb_device flag
In reply to Lu BaoluThe has_iotlb_device flag was used to indicate if a domain had attached devices with ATS enabled. Domains without this flag didn't require device TLB invalidation during unmap operations, optimizing performance by avoiding unnecessary device iteration.
With the introduction of cache tags, this flag is no longer needed. The code to iterate over attached devices was removed by commit 06792d067989 ("iommu/vt-d: Cleanup use of iommu_flush_iotlb_psi()").
Remove has_iotlb_device to avoid unnecessary code.
--- drivers/iommu/intel/iommu.h | 2 -- drivers/iommu/intel/iommu.c | 34 +--------------------------------- drivers/iommu/intel/nested.c | 2 -- 3 files changed, 1 insertion(+), 37 deletions(-)
drivers/iommu/intel/iommu.h +0 -2
@@ -588,7 +588,6 @@ struct dmar_domain {int nid;/* node id */ struct xarray iommu_array;/* Attached IOMMU array */ - u8 has_iotlb_device: 1; u8 iommu_coherency: 1; /* indicate coherency of iommu access */ u8 force_snooping : 1; /* Create IOPTEs with snoop control */ u8 set_pte_snp:1;
@@ -1104,7 +1103,6 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc,*/ #define QI_OPT_WAIT_DRAIN BIT(0) -void domain_update_iotlb (struct dmar_domain *domain);int domain_attach_iommu (struct dmar_domain *domain,struct intel_iommu *iommu);void domain_detach_iommu (struct dmar_domain *domain,struct intel_iommu *iommu);void device_block_translation (struct device *dev);
drivers/iommu/intel/iommu.c +1 -33
@@ -485,7 +485,6 @@ void domain_update_iommu_cap(struct dmar_domain *domain) domain->domain.geometry.aperture_end =__DOMAIN_MAX_ADDR (domain->gaw); domain->domain.pgsize_bitmap |=domain_super_pgsize_bitmap (domain); -domain_update_iotlb (domain); } struct context_entry *iommu_context_addr(struct intel_iommu *iommu, u8 bus,
@@ -1263,32 +1262,6 @@ domain_lookup_dev_info(struct dmar_domain *domain,return NULL ; } -void domain_update_iotlb (struct dmar_domain *domain) -{ -struct dev_pasid_info *dev_pasid; -struct device_domain_info *info; -bool has_iotlb_device = false; -unsigned long flags; - -spin_lock_irqsave (&domain->lock, flags); -list_for_each_entry (info, &domain->devices, link) { -if (info->ats_enabled) { - has_iotlb_device = true; -break ; - } - } - -list_for_each_entry (dev_pasid, &domain->dev_pasids, link_domain) { - info =dev_iommu_priv_get (dev_pasid->dev); -if (info->ats_enabled) { - has_iotlb_device = true; -break ; - } - } - domain->has_iotlb_device = has_iotlb_device; -spin_unlock_irqrestore (&domain->lock, flags); -} -/* * The extra devTLB flush quirk impacts those QAT devices with PCI device * IDs ranging from 0x4940 to 0x4943. It is exempted from risky_device()
@@ -1325,10 +1298,8 @@ static void iommu_enable_pci_caps(struct device_domain_info *info) info->pasid_enabled =1 ;if (info->ats_supported &&pci_ats_page_aligned (pdev) && - !pci_enable_ats (pdev,VTD_PAGE_SHIFT )) { + !pci_enable_ats (pdev,VTD_PAGE_SHIFT )) info->ats_enabled =1 ; -domain_update_iotlb (info->domain); - } }static void iommu_disable_pci_caps (struct device_domain_info *info)
@@ -1343,7 +1314,6 @@ static void iommu_disable_pci_caps(struct device_domain_info *info)if (info->ats_enabled) {pci_disable_ats (pdev); info->ats_enabled =0 ; -domain_update_iotlb (info->domain); }if (info->pasid_enabled) {
@@ -1517,7 +1487,6 @@ static struct dmar_domain *alloc_domain(unsigned int type) domain->nid =NUMA_NO_NODE ;if (first_level_by_default (type)) domain->use_first_level = true; - domain->has_iotlb_device = false;INIT_LIST_HEAD (&domain->devices);INIT_LIST_HEAD (&domain->dev_pasids);INIT_LIST_HEAD (&domain->cache_tags);
@@ -3518,7 +3487,6 @@ static struct dmar_domain *paging_domain_alloc(struct device *dev, bool first_stxa_init (&domain->iommu_array); domain->nid =dev_to_node (dev); - domain->has_iotlb_device = info->ats_enabled; domain->use_first_level = first_stage;/* calculate the address width */
drivers/iommu/intel/nested.c +0 -2
@@ -66,8 +66,6 @@ static int intel_nested_attach_dev(struct iommu_domain *domain,list_add (&info->link, &dmar_domain->devices);spin_unlock_irqrestore (&dmar_domain->lock, flags); -domain_update_iotlb (dmar_domain); -return 0 ; unassign_tag:cache_tag_unassign_domain (dmar_domain, dev,IOMMU_NO_PASID );
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 5/7 iommu/vt-d: Factor out helpers from domain_context_mapping_one()
In reply to Lu BaoluExtract common code from domain_context_mapping_one() into new helpers, making it reusable by other functions such as the upcoming identity domain implementation. No intentional functional changes.
--- drivers/iommu/intel/iommu.c | 99 ++++++++++++++++++++++--------------- 1 file changed, 58 insertions(+), 41 deletions(-)
drivers/iommu/intel/iommu.c +58 -41
@@ -1597,6 +1597,61 @@ static void domain_exit(struct dmar_domain *domain)kfree (domain); } +/* +* For kdump cases, old valid entries may be cached due to the +* in-flight DMA and copied pgtable, but there is no unmapping +* behaviour for them, thus we need an explicit cache flush for +* the newly-mapped device. For kdump, at this point, the device +* is supposed to finish reset at its driver probe stage, so no +* in-flight DMA will exist, and we don't need to worry anymore +* hereafter. +*/ +static void copied_context_tear_down (struct intel_iommu *iommu, +struct context_entry *context, +u8 bus,u8 devfn) +{ +u16 did_old; + +if (!context_copied (iommu, bus, devfn)) +return ; + +assert_spin_locked (&iommu->lock); + + did_old =context_domain_id (context); +context_clear_entry (context); + +if (did_old <cap_ndoms (iommu->cap)) { + iommu->flush.flush_context (iommu, did_old, + (((u16 )bus) <<8 ) | devfn, +DMA_CCMD_MASK_NOBIT , +DMA_CCMD_DEVICE_INVL ); + iommu->flush.flush_iotlb (iommu, did_old,0 ,0 , +DMA_TLB_DSI_FLUSH ); + } + +clear_context_copied (iommu, bus, devfn); +} + +/* +* It's a non-present to present mapping. If hardware doesn't cache +* non-present entry we only need to flush the write-buffer. If the +* _does_ cache non-present entries, then it does so in the special +* domain #0, which we have to flush: +*/ +static void context_present_cache_flush (struct intel_iommu *iommu,u16 did, +u8 bus,u8 devfn) +{ +if (cap_caching_mode (iommu->cap)) { + iommu->flush.flush_context (iommu,0 , + (((u16 )bus) <<8 ) | devfn, +DMA_CCMD_MASK_NOBIT , +DMA_CCMD_DEVICE_INVL ); + iommu->flush.flush_iotlb (iommu, did,0 ,0 ,DMA_TLB_DSI_FLUSH ); + }else { +iommu_flush_write_buffer (iommu); + } +} +static int domain_context_mapping_one (struct dmar_domain *domain,struct intel_iommu *iommu,u8 bus,u8 devfn)
@@ -1625,31 +1680,9 @@ static int domain_context_mapping_one(struct dmar_domain *domain,if (context_present (context) && !context_copied (iommu, bus, devfn)) goto out_unlock; -/* -* For kdump cases, old valid entries may be cached due to the -* in-flight DMA and copied pgtable, but there is no unmapping -* behaviour for them, thus we need an explicit cache flush for -* the newly-mapped device. For kdump, at this point, the device -* is supposed to finish reset at its driver probe stage, so no -* in-flight DMA will exist, and we don't need to worry anymore -* hereafter. -*/ -if (context_copied (iommu, bus, devfn)) { -u16 did_old =context_domain_id (context); - -if (did_old <cap_ndoms (iommu->cap)) { - iommu->flush.flush_context (iommu, did_old, - (((u16 )bus) <<8 ) | devfn, -DMA_CCMD_MASK_NOBIT , -DMA_CCMD_DEVICE_INVL ); - iommu->flush.flush_iotlb (iommu, did_old,0 ,0 , -DMA_TLB_DSI_FLUSH ); - } - -clear_context_copied (iommu, bus, devfn); - } - +copied_context_tear_down (iommu, context, bus, devfn);context_clear_entry (context); +context_set_domain_id (context, did);if (translation !=CONTEXT_TT_PASS_THROUGH ) {
@@ -1685,23 +1718,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain,context_set_present (context);if (!ecap_coherent (iommu->ecap))clflush_cache_range (context,sizeof (*context)); - -/* -* It's a non-present to present mapping. If hardware doesn't cache -* non-present entry we only need to flush the write-buffer. If the -* _does_ cache non-present entries, then it does so in the special -* domain #0, which we have to flush: -*/ -if (cap_caching_mode (iommu->cap)) { - iommu->flush.flush_context (iommu,0 , - (((u16 )bus) <<8 ) | devfn, -DMA_CCMD_MASK_NOBIT , -DMA_CCMD_DEVICE_INVL ); - iommu->flush.flush_iotlb (iommu, did,0 ,0 ,DMA_TLB_DSI_FLUSH ); - }else { -iommu_flush_write_buffer (iommu); - } - +context_present_cache_flush (iommu, did, bus, devfn); ret =0 ; out_unlock:
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 6/7 iommu/vt-d: Add support for static identity domain
In reply to Lu BaoluSoftware determines VT-d hardware support for passthrough translation by inspecting the capability register. If passthrough translation is not supported, the device is instructed to use DMA domain for its default domain.
Add a global static identity domain with guaranteed attach semantics for IOMMUs that support passthrough translation mode.
The global static identity domain is a dummy domain without corresponding dmar_domain structure. Consequently, the device's info->domain will be NULL with the identity domain is attached. Refactor the code accordingly.
--- drivers/iommu/intel/iommu.c | 114 ++++++++++++++++++++++++++++++++++-- 1 file changed, 110 insertions(+), 4 deletions(-)
drivers/iommu/intel/iommu.c +110 -4
@@ -3691,11 +3691,9 @@ int prepare_domain_attach_device(struct iommu_domain *domain,static int intel_iommu_attach_device (struct iommu_domain *domain,struct device *dev) { -struct device_domain_info *info =dev_iommu_priv_get (dev);int ret; -if (info->domain) -device_block_translation (dev); +device_block_translation (dev); ret =prepare_domain_attach_device (domain, dev);if (ret)
@@ -4301,11 +4299,17 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid, struct iommu_domain *domain) {struct device_domain_info *info =dev_iommu_priv_get (dev); -struct dmar_domain *dmar_domain =to_dmar_domain (domain);struct dev_pasid_info *curr, *dev_pasid =NULL ;struct intel_iommu *iommu = info->iommu; +struct dmar_domain *dmar_domain;unsigned long flags; +if (domain->type ==IOMMU_DOMAIN_IDENTITY ) { +intel_pasid_tear_down_entry (iommu, dev, pasid, false); +return ; + } + + dmar_domain =to_dmar_domain (domain);spin_lock_irqsave (&dmar_domain->lock, flags);list_for_each_entry (curr, &dmar_domain->dev_pasids, link_domain) {if (curr->dev == dev && curr->pasid == pasid) {
@@ -4532,9 +4536,111 @@ static const struct iommu_dirty_ops intel_dirty_ops = { .read_and_clear_dirty = intel_iommu_read_and_clear_dirty, }; +static int context_setup_pass_through (struct device *dev,u8 bus,u8 devfn) +{ +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; +struct context_entry *context; + +spin_lock (&iommu->lock); + context =iommu_context_addr (iommu, bus, devfn,1 ); +if (!context) { +spin_unlock (&iommu->lock); +return -ENOMEM ; + } + +if (context_present (context) && !context_copied (iommu, bus, devfn)) { +spin_unlock (&iommu->lock); +return 0 ; + } + +copied_context_tear_down (iommu, context, bus, devfn); +context_clear_entry (context); +context_set_domain_id (context,FLPT_DEFAULT_DID ); + +/* +* In pass through mode, AW must be programmed to indicate the largest +* AGAW value supported by hardware. And ASR is ignored by hardware. +*/ +context_set_address_width (context, iommu->msagaw); +context_set_translation_type (context,CONTEXT_TT_PASS_THROUGH ); +context_set_fault_enable (context); +context_set_present (context); +if (!ecap_coherent (iommu->ecap)) +clflush_cache_range (context,sizeof (*context)); +context_present_cache_flush (iommu,FLPT_DEFAULT_DID , bus, devfn); +spin_unlock (&iommu->lock); + +return 0 ; +} + +static int context_setup_pass_through_cb (struct pci_dev *pdev,u16 alias,void *data) +{ +struct device *dev = data; + +if (dev != &pdev->dev) +return 0 ; + +return context_setup_pass_through (dev,PCI_BUS_NUM (alias), alias &0xff ); +} + +static int device_setup_pass_through (struct device *dev) +{ +struct device_domain_info *info =dev_iommu_priv_get (dev); + +if (!dev_is_pci (dev)) +return context_setup_pass_through (dev, info->bus, info->devfn); + +return pci_for_each_dma_alias (to_pci_dev (dev), + context_setup_pass_through_cb, dev); +} + +static int identity_domain_attach_dev (struct iommu_domain *domain,struct device *dev) +{ +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; +int ret; + +device_block_translation (dev); + +if (dev_is_real_dma_subdevice (dev)) +return 0 ; + +if (sm_supported (iommu)) { + ret =intel_pasid_setup_pass_through (iommu, dev,IOMMU_NO_PASID ); +if (!ret) +iommu_enable_pci_caps (info); + }else { + ret =device_setup_pass_through (dev); + } + +return ret; +} + +static int identity_domain_set_dev_pasid (struct iommu_domain *domain, +struct device *dev,ioasid_t pasid) +{ +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; + +if (!pasid_supported (iommu) ||dev_is_real_dma_subdevice (dev)) +return -EOPNOTSUPP ; + +return intel_pasid_setup_pass_through (iommu, dev, pasid); +} + +static struct iommu_domain identity_domain = { + .type =IOMMU_DOMAIN_IDENTITY , + .ops = &(const struct iommu_domain_ops ) { + .attach_dev = identity_domain_attach_dev, + .set_dev_pasid = identity_domain_set_dev_pasid, + }, +}; +const struct iommu_ops intel_iommu_ops = { .blocked_domain = &blocking_domain, .release_domain = &blocking_domain, + .identity_domain = &identity_domain, .capable = intel_iommu_capable, .hw_info = intel_iommu_hw_info, .domain_alloc = intel_iommu_domain_alloc,
Signature · 1 line
2.34.1
Lu Baolu
maintainerPATCH v4 7/7 iommu/vt-d: Cleanup si_domain
In reply to Lu BaoluThe static identity domain has been introduced, rendering the si_domain obsolete. Remove si_domain and cleanup the code accordingly.
--- drivers/iommu/intel/iommu.c | 91 ++++++++----------------------------- 1 file changed, 19 insertions(+), 72 deletions(-)
drivers/iommu/intel/iommu.c +19 -72
@@ -167,8 +167,6 @@ static void device_rbtree_remove(struct device_domain_info *info)spin_unlock_irqrestore (&iommu->device_rbtree_lock, flags); } -static struct dmar_domain *si_domain; -struct dmar_rmrr_unit {struct list_head list;/* list of rmrr units */ struct acpi_dmar_header *hdr;/* ACPI header */
@@ -286,11 +284,6 @@ static int __init intel_iommu_setup(char *str) }__setup ("intel_iommu=" , intel_iommu_setup); -static int domain_type_is_si (struct dmar_domain *domain) -{ -return domain->domain.type ==IOMMU_DOMAIN_IDENTITY ; -} -static int domain_pfn_supported (struct dmar_domain *domain,unsigned long pfn) {int addr_width =agaw_to_width (domain->agaw) -VTD_PAGE_SHIFT ;
@@ -1664,9 +1657,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain,struct context_entry *context;int agaw, ret; -if (domain_type_is_si (domain)) - translation =CONTEXT_TT_PASS_THROUGH ; -pr_debug ("Set context mapping for %02x:%02x.%d\n" , bus,PCI_SLOT (devfn),PCI_FUNC (devfn));
@@ -1685,34 +1675,24 @@ static int domain_context_mapping_one(struct dmar_domain *domain,context_set_domain_id (context, did); -if (translation !=CONTEXT_TT_PASS_THROUGH ) { -/* -* Skip top levels of page tables for iommu which has -* less agaw than default. Unnecessary for PT mode. -*/ -for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { - ret = -ENOMEM ; - pgd =phys_to_virt (dma_pte_addr (pgd)); -if (!dma_pte_present (pgd)) - goto out_unlock; - } - -if (info && info->ats_supported) - translation =CONTEXT_TT_DEV_IOTLB ; -else - translation =CONTEXT_TT_MULTI_LEVEL ; - -context_set_address_root (context,virt_to_phys (pgd)); -context_set_address_width (context, agaw); - }else { -/* -* In pass through mode, AW must be programmed to -* indicate the largest AGAW value supported by -* hardware. And ASR is ignored by hardware. -*/ -context_set_address_width (context, iommu->msagaw); +/* +* Skip top levels of page tables for iommu which has +* less agaw than default. Unnecessary for PT mode. +*/ +for (agaw = domain->agaw; agaw > iommu->agaw; agaw--) { + ret = -ENOMEM ; + pgd =phys_to_virt (dma_pte_addr (pgd)); +if (!dma_pte_present (pgd)) + goto out_unlock; } +if (info && info->ats_supported) + translation =CONTEXT_TT_DEV_IOTLB ; +else + translation =CONTEXT_TT_MULTI_LEVEL ; + +context_set_address_root (context,virt_to_phys (pgd)); +context_set_address_width (context, agaw);context_set_translation_type (context, translation);context_set_fault_enable (context);context_set_present (context);
@@ -1977,23 +1957,6 @@ static bool dev_is_real_dma_subdevice(struct device *dev)pci_real_dma_dev (to_pci_dev (dev)) !=to_pci_dev (dev); } -static int md_domain_init (struct dmar_domain *domain,int guest_width); - -static int __init si_domain_init (void ) -{ - si_domain =alloc_domain (IOMMU_DOMAIN_IDENTITY ); -if (!si_domain) -return -EFAULT ; - -if (md_domain_init (si_domain,DEFAULT_DOMAIN_ADDRESS_WIDTH )) { -domain_exit (si_domain); - si_domain =NULL ; -return -EFAULT ; - } - -return 0 ; -} -static int dmar_domain_attach_device (struct dmar_domain *domain,struct device *dev) {
@@ -2016,8 +1979,6 @@ static int dmar_domain_attach_device(struct dmar_domain *domain,if (!sm_supported (iommu)) ret =domain_context_mapping (domain, dev); -else if (domain_type_is_si (domain)) - ret =intel_pasid_setup_pass_through (iommu, dev,IOMMU_NO_PASID );else if (domain->use_first_level) ret =domain_setup_first_level (iommu, domain, dev,IOMMU_NO_PASID );else
@@ -2026,8 +1987,7 @@ static int dmar_domain_attach_device(struct dmar_domain *domain,if (ret) goto out_block_translation; -if (sm_supported (info->iommu) || !domain_type_is_si (info->domain)) -iommu_enable_pci_caps (info); +iommu_enable_pci_caps (info); ret =cache_tag_assign_domain (domain, dev,IOMMU_NO_PASID );if (ret)
@@ -2386,10 +2346,6 @@ static int __init init_dmars(void)check_tylersburg_isoch (); - ret =si_domain_init (); -if (ret) - goto free_iommu; -/* * for each drhd * enable fault log
@@ -2435,10 +2391,6 @@ static int __init init_dmars(void)disable_dmar_iommu (iommu);free_dmar_iommu (iommu); } -if (si_domain) { -domain_exit (si_domain); - si_domain =NULL ; - }return ret; }
@@ -3572,8 +3524,6 @@ static struct iommu_domain *intel_iommu_domain_alloc(unsigned type) domain->geometry.force_aperture = true;return domain; -case IOMMU_DOMAIN_IDENTITY : -return &si_domain->domain;default :return NULL ; }
@@ -3640,8 +3590,7 @@ static void intel_iommu_domain_free(struct iommu_domain *domain)WARN_ON (dmar_domain->nested_parent && !list_empty (&dmar_domain->s1_domains)); -if (domain != &si_domain->domain) -domain_exit (dmar_domain); +domain_exit (dmar_domain); } int prepare_domain_attach_device(struct iommu_domain *domain,
@@ -4364,9 +4313,7 @@ static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,if (ret) goto out_detach_iommu; -if (domain_type_is_si (dmar_domain)) - ret =intel_pasid_setup_pass_through (iommu, dev, pasid); -else if (dmar_domain->use_first_level) +if (dmar_domain->use_first_level) ret =domain_setup_first_level (iommu, dmar_domain, dev, pasid);else
Signature · 1 line
2.34.1
RE: [PATCH v4 1/7] iommu/vt-d: Require DMA domain if hardware not support passthrough
In reply to Lu BaoluQuoted 16 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:54 PMThe iommu core defines the
def_domain_typecallback to query the iommu driver about hardware capability and quirks. The iommu driver should declareIOMMU_DOMAIN_DMArequirement for hardware lacking pass- through
capability.Earlier VT-d hardware implementations did not support pass-through translation mode. The iommu driver relied on a paging domain with all physical system memory addresses identically mapped to the same IOVA to simulate pass-through translation before the
def_domain_typewas introduced and it has been kept until now. It's time to adjust it now to make the Intel iommu driver follow thedef_domain_typesemantics.
Quoted 2 lines
Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Lu Baolu <[email protected]>
RE: [PATCH v4 2/7] iommu/vt-d: Remove identity mappings from si_domain
In reply to Lu BaoluQuoted 10 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:54 PMAs the driver has enforced DMA domains for devices managed by an IOMMU hardware that doesn't support passthrough translation mode, there is no need for static identity mappings in the
si_domain. Remove the identity mapping code to avoid dead code.Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Lu Baolu <[email protected]>
RE: [PATCH v4 3/7] iommu/vt-d: Always reserve a domain ID for identity setup
In reply to Lu BaoluQuoted 8 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:54 PMWe will use a global static identity domain. Reserve a static domain ID for it.
Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Lu Baolu <[email protected]>
RE: [PATCH v4 4/7] iommu/vt-d: Remove has_iotlb_device flag
In reply to Lu BaoluQuoted 8 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:54 PMThe
has_iotlb_deviceflag was used to indicate if a domain had attached devices with ATS enabled. Domains without this flag didn't require device TLB invalidation during unmap operations, optimizing performance by avoiding unnecessary device iteration.
Quoted 8 lines
With the introduction of cache tags, this flag is no longer needed. The code to iterate over attached devices was removed by commit
06792d067989("iommu/vt-d: Cleanup use ofiommu_flush_iotlb_psi()").Remove
has_iotlb_deviceto avoid unnecessary code.Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Lu Baolu <[email protected]>
RE: [PATCH v4 5/7] iommu/vt-d: Factor out helpers from domain_context_mapping_one()
In reply to Lu BaoluQuoted 10 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:54 PMExtract common code from
domain_context_mapping_one()into new helpers,
making it reusable by other functions such as the upcoming identity domain implementation. No intentional functional changes.Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Lu Baolu <[email protected]>
with a nit on naming:
drivers/iommu/intel/iommu.c static void domain_exit(struct dmar_domain *domain) · line 1611
Quoted 3 lines
drivers/iommu/intel/iommu.c +3
-0+static void copied_context_tear_down (struct intel_iommu *iommu, +struct context_entry *context, +u8 bus,u8 devfn)
tear_down_copied_conext
Review
Quoted 3 lines
+static void context_present_cache_flush(struct
intel_iommu*iommu, u16 did,(quoted diff) +1
-0+ u8 bus, u8 devfn)
flush_present_context_cache
RE: [PATCH v4 6/7] iommu/vt-d: Add support for static identity domain
In reply to Lu BaoluQuoted 18 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:55 PMdrivers/iommu/intel/iommu.c +14
-0+static int context_setup_pass_through (struct device *dev,u8 bus,u8 devfn) +{ +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; +struct context_entry *context; + +spin_lock (&iommu->lock); + context =iommu_context_addr (iommu, bus, devfn,1 ); +if (!context) { +spin_unlock (&iommu->lock); +return -ENOMEM ; + } + +if (context_present (context) && !context_copied (iommu, bus, devfn)){
drivers/iommu/intel/iommu.c static const struct iommu_dirty_ops intel_dirty_ops = { · line 4555
Quoted 3 lines
drivers/iommu/intel/iommu.c +3
-0+spin_unlock (&iommu->lock); +return 0 ; + }
Is it a valid case to setup passthrough on a present entry?
RE: [PATCH v4 7/7] iommu/vt-d: Cleanup si_domain
In reply to Lu BaoluQuoted 8 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:55 PMThe static identity domain has been introduced, rendering the
si_domainobsolete. Removesi_domainand cleanup the code accordingly.Reviewed-by Jason Gunthorpe <[email protected]>Signed-off-by Lu Baolu <[email protected]>
Baolu Lu
maintainerRe: [PATCH v4 6/7] iommu/vt-d: Add support for static identity domain
In reply to Tian, KevinQuoted 15 lines · Kevin
Quoted 15 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:55 PMdrivers/iommu/intel/iommu.c +12
-0+static int context_setup_pass_through (struct device *dev,u8 bus,u8 devfn) +{ +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; +struct context_entry *context; + +spin_lock (&iommu->lock); + context =iommu_context_addr (iommu, bus, devfn,1 ); +if (!context) { +spin_unlock (&iommu->lock); +return -ENOMEM ; + }
Quoted 8 lines
drivers/iommu/intel/iommu.c static const struct iommu_dirty_ops intel_dirty_ops = { · line 4555
Quoted 6 lines
(quoted diff) +2
-0+ + if (context_present(context) && !context_copied(iommu, bus, devfn)){
drivers/iommu/intel/iommu.c +3
-0+spin_unlock (&iommu->lock); +return 0 ; + }Is it a valid case to setup passthrough on a present entry?
It's valid but unnecessary.
Since the context is present, it indicates that the configuration has already been setup by a PCI aliased device. The iommu group mandates that all PCI aliased devices must be attached to the same iommu domain. Consequently, there's no need for additional configuration.
While it's feasible to remove this line of code due to the check in the pci_for_each_dma_alias() callback:
static int context_setup_pass_through_cb(struct pci_dev *pdev, u16 alias, void *data)
{
struct device *dev = data;
if (dev != &pdev->dev)
return 0;
return context_setup_pass_through(dev, PCI_BUS_NUM(alias), alias & 0xff);
}
But it's in the original code, I've retained it to prevent any potential regression.
Thanks,
baolu
Tian, Kevin
Reviewed-byRE: [PATCH v4 6/7] iommu/vt-d: Add support for static identity domain
Quoted 48 lines
From: Baolu Lu <[email protected]>
Sent: Saturday, August 10, 2024 4:02 PMQuoted 4 lines · Kevin
Quoted 4 lines
From: Lu Baolu <[email protected]>
Sent: Friday, August 9, 2024 1:55 PM+static int context_setup_pass_through(struct device *dev, u8 bus, u8
devfn)
Quoted 11 lines
Quoted 11 lines
drivers/iommu/intel/iommu.c +11
-0+{ +struct device_domain_info *info =dev_iommu_priv_get (dev); +struct intel_iommu *iommu = info->iommu; +struct context_entry *context; + +spin_lock (&iommu->lock); + context =iommu_context_addr (iommu, bus, devfn,1 ); +if (!context) { +spin_unlock (&iommu->lock); +return -ENOMEM ; + }Quoted 8 lines
drivers/iommu/intel/iommu.c static const struct iommu_dirty_ops intel_dirty_ops = { · line 4555
Quoted 6 lines
(quoted diff) +2
-0+ + if (context_present(context) && !context_copied(iommu, bus, devfn)){
drivers/iommu/intel/iommu.c +3
-0+spin_unlock (&iommu->lock); +return 0 ; + }Is it a valid case to setup passthrough on a present entry?
It's valid but unnecessary.
Since the context is present, it indicates that the configuration has already been setup by a PCI aliased device. The iommu group mandates that all PCI aliased devices must be attached to the same iommu domain. Consequently, there's no need for additional configuration.
While it's feasible to remove this line of code due to the check in the
pci_for_each_dma_alias()callback:static int
context_setup_pass_through_cb(struct pci_dev *pdev, u16 alias, void *data){ struct device *dev = data; if (dev != &pdev->dev) return 0; return context_setup_pass_through(dev, PCI_BUS_NUM(alias),
Quoted 6 lines
alias &
0xff);
}But it's in the original code, I've retained it to prevent any potential regression.
I failed to find the original code which is why I asked above question.
But now I got that you were talking about the check in domain_context_mapping_one() which applied to the si domain before.
So,
Baolu Lu
maintainerQuoted 65 lines · Lu Baolu
Intel's IOMMU driver used a special domain called 1:1 mapping domain to support the domain of type
IOMMU_DOMAIN_IDENTITY, which enables device drivers to directly utilize physical addresses for DMA access despite the presence of IOMMU units.The implementation of the 1:1 mapping domain is influenced by hardware differences. While modern Intel VT-d implementations support hardware passthrough translation mode, earlier versions lacked this feature, which requires a more complex implementation approach.
The 1:1 mapping domain for earlier hardware was implemented by associating a DMA domain with an IOVA (IO Virtual Address) equivalent to the physical address. While, for most hardware supporting passthrough mode, simply setting the hardware's passthrough mode is sufficient. These two modes were merged together in
si_domain, which is a special DMA domain sharing the domain ops of an ordinary DMA domain.As the iommu core has evolved, it has introduced global static identity domain with "never fail" attach semantics. This means that the domain is always available and cannot fail to attach. The iommu driver now assigns this domain directly at
iommu_ops->identity_domaininstead of allocating it through the domain allocation interface.This converts the Intel IOMMU driver to embrace the global static identity domain. For early legacy hardwares that don't support passthrough translation mode, ask the iommu core to use a DMA type of default domain. For modern hardwares that support passthrough translation mode, implement a static global identity domain.
The whole series is also available at
https://github.com/LuBaolu/intel-iommu/commits/vtd-static-identity-domain-v4
Change log:
v4:
- Add a new patch to removehas_iotlb_deviceflag as suggested byJason. https://lore.kernel.org/linux-iommu/[email protected]/v3:https://lore.kernel.org/linux-iommu/[email protected]/
- Kevin worried that some graphic devices might still require identity domain. Forcing DMA domain for those drivers might break the existing functionality.
https://lore.kernel.org/linux-iommu/BN9PR11MB52761FF9AB496B422596DDDF8C8AA@BN9PR11MB5276.namprd11.prod.outlook.com/After confirmed with the graphic community, we decouple "
igfx_off" kernel command from graphic identity mapping with the following commits:
ba00196ca41c("iommu/vt-d: Decoupleigfx_offfrom graphic identity mapping")4b8d18c0c986("iommu/vt-d: RemoveINTEL_IOMMU_BROKEN_GFX_WA").v2:https://lore.kernel.org/linux-iommu/[email protected]/
- Re-orgnize the patches by removing 1:1 mappings before implementing global static domain.v1:https://lore.kernel.org/linux-iommu/[email protected]/
Lu Baolu (7):
iommu/vt-d: Require DMA domain if hardware not support passthrough iommu/vt-d: Remove identity mappings fromsi_domainiommu/vt-d: Always reserve a domain ID for identity setup iommu/vt-d: Removehas_iotlb_deviceflag
iommu/vt-d: Factor out helpers fromdomain_context_mapping_one()iommu/vt-d: Add support for static identity domain iommu/vt-d: Cleanupsi_domain
Quoted 4 lines
drivers/iommu/intel/iommu.h | 2 - drivers/iommu/intel/iommu.c | 468 +++++++++++++++-------------------- drivers/iommu/intel/nested.c | 2 - 3 files changed, 201 insertions(+), 271 deletions(-)
Queued for v6.12-rc1.
Thanks,
baolu