Skip to content

SMMUv3.2 Range-based TLB Invalidation Support

Eric Auger qemu-arm qemu-devel 5 versions – Applied

Status from this message: Applied to target-arm.next for 5.2, thanks.

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest11 patches 4 repliesApplied
  2. v311 patches 10 repliesSuperseded
  3. v29 patches 7 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v1 posting 2/29 patches17 repliesSuperseded
v1 posting 1/29 patches0 repliesSuperseded

Changes in v4 from the cover letter

- use tg in hash function and directly compare key fields in the equal function
- don't claim 3.2 as we don't have BBML support yet
- fixed a bunch of indent issues

Patches in v4 Applied

#Subject and filesSizeRepliesApplied
1/11hw/arm/smmu-common: Factorize some code in smmu_ptw_64()hw/arm/smmu-common.c+17 -310 replies
2/11hw/arm/smmu-common: Add IOTLB helpershw/arm/smmu-common.c hw/arm/smmuv3.c hw/arm/trace-events +1 more+43 -260 replies
3/11hw/arm/smmu: Introduce smmu_get_iotlb_key()hw/arm/smmu-common.c hw/arm/smmu-internal.h include/hw/arm/smmu-common.h+38 -300 replies
4/11hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB valuehw/arm/smmu-common.c hw/arm/smmuv3.c include/hw/arm/smmu-common.h+31 -230 replies
5/11hw/arm/smmu-common: Manage IOTLB block entrieshw/arm/smmu-common.c hw/arm/smmu-internal.h hw/arm/smmuv3.c +2 more+67 -252 replies
6/11hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helperhw/arm/smmuv3.c hw/arm/trace-events+18 -210 replies
7/11hw/arm/smmuv3: Get prepared for range invalidationhw/arm/smmu-common.c hw/arm/smmuv3-internal.h hw/arm/smmuv3.c +2 more+69 -310 replies
8/11hw/arm/smmuv3: Fix IIDR offsethw/arm/smmuv3-internal.h+1 -10 replies
9/11hw/arm/smmuv3: Let AIDR advertise SMMUv3.0 supporthw/arm/smmuv3-internal.h hw/arm/smmuv3.c include/hw/arm/smmuv3.h+5 -00 replies
10/11hw/arm/smmuv3: Support HAD and advertise SMMUv3.1 supporthw/arm/smmu-common.c hw/arm/smmuv3-internal.h hw/arm/smmuv3.c +2 more+10 -30 replies
11/11hw/arm/smmuv3: Advertise SMMUv3.2 range invalidationhw/arm/smmuv3-internal.h hw/arm/smmuv3.c+2 -00 replies

Reviews trailers per patch

All 11 patches have a Reviewed-by or Acked-by.

PatchPeter Maydell
1/11R-b
2/11R-b
3/11R-b
4/11R-b
5/11R-b
6/11R-b
7/11R-b
8/11R-b
9/11R-b
10/11R-b
11/11R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v4 thread

Cover letter [PATCH for-5.2 v4 00/11] SMMUv3.2 Range-based TLB Invalidation Support

SMMU3.2 brings the support of range-based TLB invalidation and level hint. When this feature is supported, the SMMUv3 driver is allowed to send TLB invalidations for a range of IOVAs instead of using page based invalidation.

Implementing this feature in the virtual SMMUv3 device is mandated for DPDK on guest use case: DPDK uses hugepage buffers and guest sends invalidations for blocks. Without this feature, a guest invalidation of a block of 1GB for instance translates into a storm of page invalidations. Each of them is trapped by the VMM and cascaded downto the physical IOMMU. This completely stalls the execution. This integration issue was initially reported in [1].

Now SMMUv3.2 specifies additional parameters to NH_VA and NH_VAA stage 1 invalidation commands so we can support those extensions.

Supporting block mappings in the IOTLB look sensible in terms of TLB entry consumption. However looking at virtio/vhost device usage, without block mapping and without range invalidation (< 5.7 kernels it may be less performant. However for recent guest kernels supporting range invalidations [2], the performance should be similar.

Best Regards

Eric

This series can be found at:
https://github.com/eauger/qemu.git
branch: v5.1.0-rc1-smmuv3-ril-v4

References:
[1] [RFC v2 4/4] iommu/arm-smmu-v3: add CMD_TLBI_NH_VA_AM command for iova range invalidation
(https://lists.linuxfoundation.org/pipermail/iommu/2017-August/023679.html

[2] 5.7+ kernels featuring
6a481a95d4c1 iommu/arm-smmu-v3: Add SMMUv3.2 range invalidation support

History:

v3 -> v4:
- use tg in hash function and directly compare key fields in the equal function
- don't claim 3.2 as we don't have BBML support yet
- fixed a bunch of indent issues

v2 -> v3:
- restore the Jenkins hash function and keep the key as a struct
- simplify the logic in smmu_hash_remove_by_asid_iova
- added HAD support
- expose AIDR (advertise v3.2 support) and fix IIDR offset

v1 -> v2:
- added "hw/arm/smmu: Introduce smmu_get_iotlb_key()"
- removed "[PATCH 5/9] hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo"
- Collected Peter's R-b
- In this version the key still features TG/LVL.
- More details in individual history logs

Eric Auger (11):
hw/arm/smmu-common: Factorize some code in smmu_ptw_64() hw/arm/smmu-common: Add IOTLB helpers hw/arm/smmu: Introduce smmu_get_iotlb_key() hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value hw/arm/smmu-common: Manage IOTLB block entries hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper hw/arm/smmuv3: Get prepared for range invalidation hw/arm/smmuv3: Fix IIDR offset
hw/arm/smmuv3: Let AIDR advertise SMMUv3.0 support hw/arm/smmuv3: Support HAD and advertise SMMUv3.1 support hw/arm/smmuv3: Advertise SMMUv3.2 range invalidation

 hw/arm/smmu-internal.h       |   8 ++
 hw/arm/smmuv3-internal.h     |  10 +-
 include/hw/arm/smmu-common.h |  19 +++-
 include/hw/arm/smmuv3.h      |   1 +
 hw/arm/smmu-common.c         | 214 ++++++++++++++++++++++++-----------
 hw/arm/smmuv3.c              | 142 +++++++++++------------
 hw/arm/trace-events          |  12 +-
 7 files changed, 258 insertions(+), 148 deletions(-)
Signature · 1 line
2.21.3