iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode
Subsystems (3)
PATCH v2 1/1 iommu/vt-d: Avoid superfluous IOTLB tracking in lazy mode
Intel IOMMU driver implements IOTLB flush queue with domain selective or PASID selective invalidations. In this case there's no need to track IOVA page range and sync IOTLBs, which may cause significant performance hit.
This patch adds a check to avoid IOVA gather page and IOTLB sync for the lazy path.
The performance difference on Sapphire Rapids 100Gb NIC is improved by the following (as measured by iperf send):
w/o this fix~48 Gbits/s. with this fix ~54 Gbits/s
---
v2: use helper function iommu_iotlb_gather_queued() instead of open coding
--- drivers/iommu/intel/iommu.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/iommu/intel/iommu.c +7 -1
@@ -4348,7 +4348,13 @@ static size_t intel_iommu_unmap(struct iommu_domain *domain,if (dmar_domain->max_addr == iova + size) dmar_domain->max_addr = iova; -iommu_iotlb_gather_add_page (domain, gather, iova, size); +/* +* We do not use page-selective IOTLB invalidation in flush queue, +* There is no need to track page and sync iotlb. Domain-selective or +* PASID-selective validation are used in the flush queue. +*/ +if (!iommu_iotlb_gather_queued (gather)) +iommu_iotlb_gather_add_page (domain, gather, iova, size);return size; }
Signature · 1 line
2.25.1
drivers/iommu/intel/iommu.c static size_t intel_iommu_unmap(struct iommu_domain *domain, · line 4353
Quoted 7 lines
From: Jacob Pan <[email protected]>
Sent: Thursday, February 9, 2023 2:19 AMdrivers/iommu/intel/iommu.c +3
-1-iommu_iotlb_gather_add_page (domain, gather, iova, size); +/* +* We do not use page-selective IOTLB invalidation in flush queue, +* There is no need to track page and sync iotlb. Domain-selective or
"in flush queue, so there is ..."
drivers/iommu/intel/iommu.c static size_t intel_iommu_unmap(struct iommu_domain *domain, · line 4354
Quoted 1 line
drivers/iommu/intel/iommu.c +1
-0+* PASID-selective validation are used in the flush queue.
the last sentence can be removed. same meaning as the first one.
drivers/iommu/intel/iommu.c static size_t intel_iommu_unmap(struct iommu_domain *domain, · line 4357
Quoted 4 lines
drivers/iommu/intel/iommu.c +3
-0+*/ +if (!iommu_iotlb_gather_queued (gather)) +iommu_iotlb_gather_add_page (domain, gather, iova, size);
Hi Kevin,
Quoted 12 lines · Kevin
Quoted 9 lines
From: Jacob Pan <[email protected]>
Sent: Thursday, February 9, 2023 2:19 AMdrivers/iommu/intel/iommu.c +2
-1-iommu_iotlb_gather_add_page (domain, gather, iova, size); +/* +* We do not use page-selective IOTLB invalidation in flush queue,
(quoted diff) +1
-0+ * There is no need to track page and sync iotlb.Domain-selective or
"in flush queue, so there is ..."
Quoted 4 lines
drivers/iommu/intel/iommu.c static size_t intel_iommu_unmap(struct iommu_domain *domain, · line 4354
Quoted 1 line
drivers/iommu/intel/iommu.c +1
-0+* PASID-selective validation are used in the flush queue. the last sentence can be removed. same meaning as the first one.
sounds good.
Quoted 7 lines
Quoted 4 lines
drivers/iommu/intel/iommu.c +3
-0+*/ +if (!iommu_iotlb_gather_queued (gather)) + iommu_iotlb_gather_add_page(domain, gather, iova,size);
Reviewed-by Kevin Tian <[email protected]>
Thanks,
Jacob