Skip to content

hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

Shameer Kolothum qemu-arm qemu-devel 7 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest31 patches 11 repliesIn review
  2. v631 patches 23 repliesSuperseded
  3. v532 patches 51 repliesSuperseded
Earlier versions · 4
VersionPostedPatchesRepliesStatus
v431 patches69 repliesSuperseded
v332 patches59 repliesSuperseded
v224 patches56 repliesSuperseded
v1 RFC16 patches51 repliesSuperseded

Changes in v7 from the cover letter

https://lore.kernel.org/qemu-devel/[email protected]/

- Addressed v6 feedback and picked up R-by/T-by tags. Thanks!
- mmap the host VINTF Page0 at IOMMU_VIOMMU_ALLOC instead of on VINTF enable (patch #14)
- Reset the per-VCMDQ Page0 cache when a queue is freed (patch #19)
- Distributed the reset teardown into the patches that introduce each resource (patches #18, #21)
- Added a VCMDQ readiness helper and renamed the Page0 backing-pointer helper to vintf_lvcmdq_ptr() (patches #18, #19)
- Trace the VCMDQ Page0 backing, hw vs cache (patch #19)
- Renamed patch #24 to reflect the changes accurately
- Updated the design and lifecycle documentation (patch #30)
- Based on top of Nathan's "Resolve AUTO properties" v6 series [0].

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/31backends/iommufd: Update iommufd_backend_get_device_infobackends/iommufd.c hw/arm/smmuv3-accel.c hw/vfio/iommufd.c+10 -30 replies
2/31backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptrbackends/iommufd.c backends/trace-events hw/arm/smmuv3-accel.c +1 more+8 -30 replies
3/31backends/iommufd: Introduce iommufd_backend_alloc_hw_queuebackends/iommufd.c backends/trace-events include/system/iommufd.h+43 -00 replies
4/31backends/iommufd: Introduce iommufd_backend_viommu_mmapbackends/iommufd.c backends/trace-events include/system/iommufd.h+27 -00 replies
5/31system/iommufd: Remove unused viommu pointer from IOMMUFDVeventqhw/arm/smmuv3-accel.c include/system/iommufd.h+0 -20 replies
6/31hw/arm/smmuv3-accel: Introduce CMDQV ops interfacehw/arm/smmuv3-accel.h+34 -00 replies
7/31hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stubhw/arm/Kconfig hw/arm/meson.build hw/arm/tegra241-cmdqv-stubs.c +2 more+94 -00 replies
8/31hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecyclehw/arm/smmuv3-accel.c include/hw/arm/smmuv3.h+89 -70 replies
9/31hw/arm/virt: Use stored SMMUv3 device list for IORT buildhw/arm/virt-acpi-build.c hw/arm/virt.c include/hw/arm/virt.h+35 -390 replies
10/31hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV supporthw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h+34 -20 replies
11/31hw/arm/tegra241-cmdqv: Implement CMDQV inithw/arm/smmuv3-accel.h hw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h+59 -20 replies
12/31hw/arm/virt: Link SMMUv3 CMDQV resources to platform bushw/arm/virt.c+22 -00 replies
13/31hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/freehw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h+48 -10 replies
14/31hw/arm/tegra241-cmdqv: mmap host VINTF Page0 for CMDQVhw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h+18 -11 reply
15/31hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config regionhw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h hw/arm/trace-events+294 -10 replies
16/31hw/arm/tegra241-cmdqv: Emulate VCMDQ register readshw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h hw/arm/trace-events+321 -00 replies
17/31hw/arm/tegra241-cmdqv: Emulate VCMDQ register writeshw/arm/tegra241-cmdqv.c hw/arm/trace-events+159 -20 replies
18/31hw/arm/tegra241-cmdqv: Allocate HW VCMDQs once configuredhw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h+171 -111 reply
19/31hw/arm/tegra241-cmdqv: Route allocated VCMDQ Page0 accesses to the mmap'd host VINTF page0hw/arm/tegra241-cmdqv.c hw/arm/trace-events+91 -22 replies
20/31memory: Allow RAM device regions to skip IOMMU mappinghw/vfio/listener.c hw/vfio/trace-events include/system/memory.h +1 more+38 -00 replies
21/31hw/arm/tegra241-cmdqv: Use mmap'd host VINTF page0 for virtual VINTF page0hw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h+38 -01 reply
22/31hw/arm/smmuv3-accel: Introduce common helper for veventq readhw/arm/smmuv3-accel-stubs.c hw/arm/smmuv3-accel.c hw/arm/smmuv3-accel.h+49 -270 replies
23/31hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errorshw/arm/tegra241-cmdqv.c hw/arm/trace-events+64 -00 replies
24/31hw/arm/tegra241-cmdqv: Initialize register state on resethw/arm/tegra241-cmdqv.c hw/arm/tegra241-cmdqv.h hw/arm/trace-events+42 -00 replies
25/31hw/arm/tegra241-cmdqv: Limit queue size based on backend page sizehw/arm/tegra241-cmdqv.c+20 -00 replies
26/31hw/arm/smmuv3: Add per-device identifier propertyhw/arm/smmuv3.c hw/arm/virt-acpi-build.c hw/arm/virt.c +1 more+19 -10 replies
27/31hw/arm/smmuv3-accel: Introduce helper to query CMDQV typehw/arm/smmuv3-accel-stubs.c hw/arm/smmuv3-accel.c hw/arm/smmuv3-accel.h +1 more+33 -00 replies
28/31hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDThw/arm/trace-events hw/arm/virt-acpi-build.c+53 -00 replies
29/31hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is activehw/arm/smmuv3-accel.c hw/arm/smmuv3-accel.h+19 -00 replies
30/31hw/arm/tegra241-cmdqv: Document the CMDQV design and lifecyclehw/arm/tegra241-cmdqv.c+100 -02 replies
31/31hw/arm/smmuv3: Add cmdqv property for SMMUv3 devicehw/arm/smmuv3.c qemu-options.hx+16 -00 replies

Reviews trailers per patch

All 31 patches have a Reviewed-by or Acked-by.

PatchEric AugerNicolin Chen
SeriesT-b
1/31R-b T-b
2/31R-b T-b
3/31R-b T-b
4/31R-b T-b
5/31R-b R-b T-b
6/31R-b R-b T-b
7/31R-b R-b T-b
8/31R-b R-b T-b
9/31R-b R-b T-b
10/31R-b R-b T-b
11/31R-b T-b
12/31R-b R-b T-b
13/31R-b T-b
14/31R-b T-b
15/31R-b T-b
16/31R-b T-b
17/31R-b T-b
18/31R-b T-b
19/31R-b T-b R-b
20/31R-b T-b T-b
21/31R-b T-b
22/31R-b R-b T-b
23/31R-b R-b T-b
24/31R-b T-b
25/31R-b T-b
26/31R-b R-b T-b
27/31R-b R-b T-b
28/31R-b T-b
29/31R-b R-b T-b
30/31R-b T-b R-b
31/31R-b R-b T-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v7 thread

Cover letter [PATCH v7 00/31] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

Hi,

Changes from v6:
https://lore.kernel.org/qemu-devel/[email protected]/

- Addressed v6 feedback and picked up R-by/T-by tags. Thanks!
- mmap the host VINTF Page0 at IOMMU_VIOMMU_ALLOC instead of on VINTF enable (patch #14)
- Reset the per-VCMDQ Page0 cache when a queue is freed (patch #19)
- Distributed the reset teardown into the patches that introduce each resource (patches #18, #21)
- Added a VCMDQ readiness helper and renamed the Page0 backing-pointer helper to vintf_lvcmdq_ptr() (patches #18, #19)
- Trace the VCMDQ Page0 backing, hw vs cache (patch #19)
- Renamed patch #24 to reflect the changes accurately
- Updated the design and lifecycle documentation (patch #30)
- Based on top of Nathan's "Resolve AUTO properties" v6 series [0].

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-vcmdq-v7-ext

Sanity tested on NVIDIA Grace. Further testing in progress.

Feedback and testing are very welcome.

Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/

---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/[email protected]/

Thanks to Nicolin for the initial patches and testing on which this is based.

Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.

Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)

Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.

VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].
---

Nicolin Chen (15):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free hw/arm/tegra241-cmdqv: mmap host VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate VCMDQ register writes hw/arm/tegra241-cmdqv: Allocate HW VCMDQs once configured hw/arm/tegra241-cmdqv: Use mmap'd host VINTF page0 for virtual VINTF
page0
hw/arm/tegra241-cmdqv: Initialize register state on reset hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Shameer Kolothum (16):
system/iommufd: Remove unused viommu pointer from IOMMUFDVeventq hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle hw/arm/virt: Use stored SMMUv3 device list for IORT build hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus hw/arm/tegra241-cmdqv: Route allocated VCMDQ Page0 accesses to the
mmap'd host VINTF page0
memory: Allow RAM device regions to skip IOMMU mapping hw/arm/smmuv3-accel: Introduce common helper for veventq read hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors hw/arm/smmuv3: Add per-device identifier property hw/arm/smmuv3-accel: Introduce helper to query CMDQV type hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active hw/arm/tegra241-cmdqv: Document the CMDQV design and lifecycle hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

 hw/arm/smmuv3-accel.h         |   48 ++
 hw/arm/tegra241-cmdqv.h       |  384 +++++++++++
 include/hw/arm/smmuv3.h       |    3 +
 include/hw/arm/virt.h         |    1 +
 include/system/iommufd.h      |   17 +-
 include/system/memory.h       |   21 +
 backends/iommufd.c            |   64 ++
 hw/arm/smmuv3-accel-stubs.c   |   12 +
 hw/arm/smmuv3-accel.c         |  194 ++++--
 hw/arm/smmuv3.c               |   10 +
 hw/arm/tegra241-cmdqv-stubs.c |   16 +
 hw/arm/tegra241-cmdqv.c       | 1119 +++++++++++++++++++++++++++++++++
 hw/arm/virt-acpi-build.c      |  127 ++--
 hw/arm/virt.c                 |   37 ++
 hw/vfio/iommufd.c             |    4 +-
 hw/vfio/listener.c            |    6 +
 system/memory.c               |   10 +
 backends/trace-events         |    4 +-
 hw/arm/Kconfig                |    5 +
 hw/arm/meson.build            |    2 +
 hw/arm/trace-events           |   11 +
 hw/vfio/trace-events          |    1 +
 qemu-options.hx               |    8 +
 23 files changed, 2024 insertions(+), 80 deletions(-)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
Signature · 1 line
2.43.0