iommu/vt-d: do not use flush-queue when caching-mode is on
Subsystems (3)
PATCH v2 1/1 iommu/vt-d: do not use flush-queue when caching-mode is on
From: Nadav Amit <[email protected]>
When an Intel IOMMU is virtualized, and a physical device is passed-through to the VM, changes of the virtual IOMMU need to be propagated to the physical IOMMU. The hypervisor therefore needs to monitor PTE mappings in the IOMMU page-tables. Intel specifications provide "caching-mode" capability that a virtual IOMMU uses to report that the IOMMU is virtualized and a TLB flush is needed after mapping to allow the hypervisor to propagate virtual IOMMU mappings to the physical IOMMU. To the best of my knowledge no real physical IOMMU reports "caching-mode" as turned on.
Synchronizing the virtual and the physical IOMMU tables is expensive if the hypervisor is unaware which PTEs have changed, as the hypervisor is required to walk all the virtualized tables and look for changes. Consequently, domain flushes are much more expensive than page-specific flushes on virtualized IOMMUs with passthrough devices. The kernel therefore exploited the "caching-mode" indication to avoid domain flushing and use page-specific flushing in virtualized environments. See commit 78d5f0f500e6 ("intel-iommu: Avoid global flushes with caching mode.")
This behavior changed after commit 13cf01744608 ("iommu/vt-d: Make use of iova deferred flushing"). Now, when batched TLB flushing is used (the default), full TLB domain flushes are performed frequently, requiring the hypervisor to perform expensive synchronization between the virtual TLB and the physical one.
Getting batched TLB flushes to use in such circumstances page-specific invalidations again is not easy, since the TLB invalidation scheme assumes that "full" domain TLB flushes are performed for scalability.
Disable batched TLB flushes when caching-mode is on, as the performance benefit from using batched TLB invalidations is likely to be much smaller than the overhead of the virtual-to-physical IOMMU page-tables synchronization.
---v1->v2:
* disable flush queue for all domains if caching-mode is on for any IOMMU (Lu).
--- drivers/iommu/intel/iommu.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-)
drivers/iommu/intel/iommu.c +31 -1
@@ -5373,6 +5373,36 @@ intel_iommu_domain_set_attr(struct iommu_domain *domain,return ret; } +static bool domain_use_flush_queue (void ) +{ +struct dmar_drhd_unit *drhd; +struct intel_iommu *iommu; +bool r = true; + +if (intel_iommu_strict) +return false; + +/* +* The flush queue implementation does not perform page-selective +* invalidations that are required for efficient TLB flushes in virtual +* environments. The benefit of batching is likely to be much lower than +* the overhead of synchronizing the virtual and physical IOMMU +* page-tables. +*/ +rcu_read_lock (); +for_each_active_iommu (iommu, drhd) { +if (!cap_caching_mode (iommu->cap)) +continue ; + +pr_warn_once ("IOMMU batching is disabled due to virtualization" ); + r = false; +break ; + } +rcu_read_unlock (); + +return r; +} +static int intel_iommu_domain_get_attr (struct iommu_domain *domain,enum iommu_attr attr,void *data)
@@ -5383,7 +5413,7 @@ intel_iommu_domain_get_attr(struct iommu_domain *domain,case IOMMU_DOMAIN_DMA :switch (attr) {case DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE : - *(int *)data = !intel_iommu_strict; + *(int *)data =domain_use_flush_queue ();return 0 ;default :return -ENODEV ;
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerQuoted 32 lines · Nadav Amit
From: Nadav Amit <[email protected]>
When an Intel IOMMU is virtualized, and a physical device is passed-through to the VM, changes of the virtual IOMMU need to be propagated to the physical IOMMU. The hypervisor therefore needs to monitor PTE mappings in the IOMMU page-tables. Intel specifications provide "caching-mode" capability that a virtual IOMMU uses to report that the IOMMU is virtualized and a TLB flush is needed after mapping to allow the hypervisor to propagate virtual IOMMU mappings to the physical IOMMU. To the best of my knowledge no real physical IOMMU reports "caching-mode" as turned on.
Synchronizing the virtual and the physical IOMMU tables is expensive if the hypervisor is unaware which PTEs have changed, as the hypervisor is required to walk all the virtualized tables and look for changes. Consequently, domain flushes are much more expensive than page-specific flushes on virtualized IOMMUs with passthrough devices. The kernel therefore exploited the "caching-mode" indication to avoid domain flushing and use page-specific flushing in virtualized environments. See commit
78d5f0f500e6("intel-iommu: Avoid global flushes with caching mode.")This behavior changed after commit
13cf01744608("iommu/vt-d: Make use of iova deferred flushing"). Now, when batched TLB flushing is used (the default), full TLB domain flushes are performed frequently, requiring the hypervisor to perform expensive synchronization between the virtual TLB and the physical one.Getting batched TLB flushes to use in such circumstances page-specific invalidations again is not easy, since the TLB invalidation scheme assumes that "full" domain TLB flushes are performed for scalability.
Quoted 6 lines
Disable batched TLB flushes when caching-mode is on, as the performance benefit from using batched TLB invalidations is likely to be much smaller than the overhead of the virtual-to-physical IOMMU page-tables synchronization.
Fixes 78d5f0f500e6 ("intel-iommu: Avoid global flushes with caching mode.")
Isn't it
?
Best regards,
baolu
Quoted 58 lines
Signed-off-by Nadav Amit <[email protected]>Cc David Woodhouse <[email protected]>Cc Lu Baolu <[email protected]>Cc Joerg Roedel <[email protected]>Cc Will Deacon <[email protected]>Cc [email protected] >---
v1->v2:* disable flush queue for all domains if caching-mode is on for any IOMMU (Lu).
--- drivers/iommu/intel/iommu.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-)drivers/iommu/intel/iommu.c +30
-0@@ -5373,6 +5373,36 @@ intel_iommu_domain_set_attr(struct iommu_domain *domain,return ret; } +static bool domain_use_flush_queue (void ) +{ +struct dmar_drhd_unit *drhd; +struct intel_iommu *iommu; +bool r = true; + +if (intel_iommu_strict) +return false; + +/* +* The flush queue implementation does not perform page-selective +* invalidations that are required for efficient TLB flushes in virtual +* environments. The benefit of batching is likely to be much lower than +* the overhead of synchronizing the virtual and physical IOMMU +* page-tables. +*/ +rcu_read_lock (); +for_each_active_iommu (iommu, drhd) { +if (!cap_caching_mode (iommu->cap)) +continue ; + +pr_warn_once ("IOMMU batching is disabled due to virtualization" ); + r = false; +break ; + } +rcu_read_unlock (); + +return r; +} +static int intel_iommu_domain_get_attr (struct iommu_domain *domain,enum iommu_attr attr,void *data)
drivers/iommu/intel/iommu.c intel_iommu_domain_get_attr(struct iommu_domain *domain, · line 5419
Quoted 8 lines
drivers/iommu/intel/iommu.c +1
-1@@ -5383,7 +5413,7 @@ intel_iommu_domain_get_attr(struct iommu_domain *domain,case IOMMU_DOMAIN_DMA :switch (attr) {case DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE : - *(int *)data = !intel_iommu_strict; + *(int *)data =domain_use_flush_queue ();return 0 ;default :return -ENODEV ;
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Quoted 36 lines · Lu Baolu
Quoted 31 lines · Nadav Amit
From: Nadav Amit <[email protected]>
When an Intel IOMMU is virtualized, and a physical device is passed-through to the VM, changes of the virtual IOMMU need to be propagated to the physical IOMMU. The hypervisor therefore needs to monitor PTE mappings in the IOMMU page-tables. Intel specifications provide "caching-mode" capability that a virtual IOMMU uses to report that the IOMMU is virtualized and a TLB flush is needed after mapping to allow the hypervisor to propagate virtual IOMMU mappings to the physical IOMMU. To the best of my knowledge no real physical IOMMU reports "caching-mode" as turned on.
Synchronizing the virtual and the physical IOMMU tables is expensive if the hypervisor is unaware which PTEs have changed, as the hypervisor is required to walk all the virtualized tables and look for changes. Consequently, domain flushes are much more expensive than page-specific flushes on virtualized IOMMUs with passthrough devices. The kernel therefore exploited the "caching-mode" indication to avoid domain flushing and use page-specific flushing in virtualized environments. See commit78d5f0f500e6("intel-iommu: Avoid global flushes with caching mode.")
This behavior changed after commit13cf01744608("iommu/vt-d: Make use of iova deferred flushing"). Now, when batched TLB flushing is used (the default), full TLB domain flushes are performed frequently, requiring the hypervisor to perform expensive synchronization between the virtual TLB and the physical one.
Getting batched TLB flushes to use in such circumstances page-specific invalidations again is not easy, since the TLB invalidation scheme assumes that "full" domain TLB flushes are performed for scalability. Disable batched TLB flushes when caching-mode is on, as the performance benefit from using batched TLB invalidations is likely to be much smaller than the overhead of the virtual-to-physical IOMMU page-tables synchronization.Quoted 1 line
Fixes 78d5f0f500e6 ("intel-iommu: Avoid global flushes with caching mode.")
Quoted 5 lines
Isn't it
Fixes 13cf01744608 ("iommu/vt-d: Make use of iova deferred flushing")?
Of course it is - bad copy-paste. I will send v3.
Thanks again,
Nadav