Skip to contentSkip to conversation

hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

PATCH Superseded Shameer Kolothum 6 participants 81 messages Feb 6 – Feb 16

Subsystems (2)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v2 cover hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

Hi,

Changes from RFCv1:
https://lore.kernel.org/qemu-devel/[email protected]/

- Rebased on top of latest master with SMMUv3 accel support.
- RFC tag dropped as the SMMUv3 accel series is now applied.
- Addressed feedback on RFC and collected tags. Thanks!
- Introduces an ops interface to factor CMDQV specific behavior out of the base implementation(patch #5)
- Split read/write implementation to multiple patches(10-12)
- Only map the VINTF page0 MMIO region to guest(patch #15)
- Add per-device identifier property to SMMUv3 dev so that both Tegra DSDT and SMMUv3 IORT uses the same identifier (patch #22)
- TODO: There was a suggestion to factor out common code for vEVNTQ read side function. I have left it as is for now.

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v4-vcmdq-v2

I have sanity tested this on an NVIDIA GRACE platform and will continue with additional tests.

Feedback and testing are very welcome.

Thanks,
Shameer

---
Background(from RFCv1):

Thanks to Nicolin for the initial patches and testing on which this is based.

Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.

Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)

Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.

VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].

Summary of QEMU changes:

- Integrated into the existing SMMUv3 accel path via a "tegra241-cmdqv" property.
- Support for allocating vIOMMU objects of type IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.
- Mapping and emulation of the CMDQV MMIO register layout.
- VCMDQ/VINTF read/write handling and queue allocation using IOMMUFD APIs.
- Reset and initialisation hooks, including checks for at least one cold-plugged device.
- CMDQV hardware reads guest queue memory using host physical addresses provided through IOMMUFD, which requires that the VCMDQ buffer be physically contiguous not only in guest PA space but also in host PA space. When Tegra241 CMDQV is enabled, QEMU must therefore only expose a CMDQV size that the host can reliably back with contiguous physical memory. Because of this constraint, it is suggested to use huge pages to back the guest RAM.
- ACPI DSDT node generation for CMDQV devices on the virt machine.
---

Nicolin Chen (14):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init and vIOMMU allocation hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Emulate global CMDQV registers hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Shameer Kolothum (10):
hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle system/physmem: Add address_space_is_ram() helper hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev hw/arm/smmuv3: Add per-device identifier poroperty hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

 backends/iommufd.c       |  65 ++++
 backends/trace-events    |   4 +-
 hw/arm/Kconfig           |   5 +
 hw/arm/meson.build       |   3 +-
 hw/arm/smmuv3-accel.c    |  73 +++-
 hw/arm/smmuv3-accel.h    |  24 ++
 hw/arm/smmuv3.c          |  15 +
 hw/arm/tegra241-cmdqv.c  | 778 +++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h  | 340 +++++++++++++++++
 hw/arm/trace-events      |   5 +
 hw/arm/virt-acpi-build.c | 121 ++++--
 hw/arm/virt.c            |   3 +
 hw/vfio/iommufd.c        |   4 +-
 include/hw/arm/smmuv3.h  |   3 +
 include/system/iommufd.h |  17 +
 include/system/memory.h  |  10 +
 system/physmem.c         |  11 +
 17 files changed, 1447 insertions(+), 34 deletions(-)
 create mode 100644 hw/arm/tegra241-cmdqv.c
 create mode 100644 hw/arm/tegra241-cmdqv.h
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/24 backends/iommufd: Update iommufd_backend_get_device_info

From: Nicolin Chen <[email protected]>

The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.

---
 backends/iommufd.c    | 7 +++++++
 hw/arm/smmuv3-accel.c | 2 +-
 hw/vfio/iommufd.c     | 4 ++--
 3 files changed, 10 insertions(+), 3 deletions(-)
backends/iommufd.c +7 -0
index acfab907c0..5daefe505e 100644
@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,
     return true;
 }
 
+/*
+ * @type can carry a desired HW info type defined in the uapi headers. If caller
+ * doesn't have one, indicating it wants the default type, then @type should be
+ * zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT).
+ */
 bool iommufd_backend_get_device_info(IOMMUFDBackend *be, uint32_t devid,
                                      uint32_t *type, void *data, uint32_t len,
                                      uint64_t *caps, uint8_t *max_pasid_log2,
                                      Error **errp)
 {
     struct iommu_hw_info info = {
+        .flags = (*type) ? IOMMU_HW_INFO_FLAG_INPUT_TYPE : 0,
         .size = sizeof(info),
         .dev_id = devid,
         .data_len = len,
         .data_uptr = (uintptr_t)data,
+        .in_data_type = *type,
     };
 
     if (ioctl(be->fd, IOMMU_GET_HW_INFO, &info)) {
hw/arm/smmuv3-accel.c +1 -1
index beffb8aa94..14ea4eac37 100644
@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                            Error **errp)
 {
     struct iommu_hw_info_arm_smmuv3 info;
-    uint32_t data_type;
+    uint32_t data_type = 0;
     uint64_t caps;
 
     if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +2 -2
index 131612eb83..d0f124300d 100644
@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,
     ERRP_GUARD();
     IOMMUFDBackend *iommufd = vbasedev->iommufd;
     VFIOContainer *bcontainer = VFIO_IOMMU(container);
-    uint32_t type, flags = 0;
+    uint32_t type = 0, flags = 0;
     uint64_t hw_caps;
     VendorCaps caps;
     VFIOIOASHwpt *hwpt;
@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,
     HostIOMMUDeviceIOMMUFD *idev;
     HostIOMMUDeviceCaps *caps = &hiod->caps;
     VendorCaps *vendor_caps = &caps->vendor_caps;
-    enum iommu_hw_info_type type;
+    enum iommu_hw_info_type type = 0;
     uint8_t max_pasid_log2;
     uint64_t hw_caps;
 
Signature · 1 line
2.43.0
Reviewed-by on v2 1/24
#2 lore raw

Re: [PATCH v2 01/24] backends/iommufd: Update iommufd_backend_get_device_info

Hi Shameer, Nicolin,

Quoted 50 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.

---
 backends/iommufd.c    | 7 +++++++
 hw/arm/smmuv3-accel.c | 2 +-
 hw/vfio/iommufd.c     | 4 ++--
 3 files changed, 10 insertions(+), 3 deletions(-)
backends/iommufd.c +7 -0
index acfab907c0..5daefe505e 100644
@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,
     return true;
 }
 
+/*
+ * @type can carry a desired HW info type defined in the uapi headers. If caller
+ * doesn't have one, indicating it wants the default type, then @type should be
+ * zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT).
+ */
 bool iommufd_backend_get_device_info(IOMMUFDBackend *be, uint32_t devid,
                                      uint32_t *type, void *data, uint32_t len,
                                      uint64_t *caps, uint8_t *max_pasid_log2,
                                      Error **errp)
 {
     struct iommu_hw_info info = {
+        .flags = (*type) ? IOMMU_HW_INFO_FLAG_INPUT_TYPE : 0,
         .size = sizeof(info),
         .dev_id = devid,
         .data_len = len,
         .data_uptr = (uintptr_t)data,
+        .in_data_type = *type,
     };
 
     if (ioctl(be->fd, IOMMU_GET_HW_INFO, &info)) {
hw/arm/smmuv3-accel.c +0 -0
index beffb8aa94..14ea4eac37 100644

hw/arm/smmuv3-accel.c smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 130

Quoted 5 lines
hw/arm/smmuv3-accel.c +1 -1
@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                            Error **errp)
 {
     struct iommu_hw_info_arm_smmuv3 info;
-    uint32_t data_type;
+    uint32_t data_type = 0;

I would use IOMMU_HW_INFO_TYPE_DEFAULT explicitly.

Quoted 17 lines
hw/arm/smmuv3-accel.c +0 -0
     uint64_t caps;
 
     if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +1 -1
index 131612eb83..d0f124300d 100644
@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,
     ERRP_GUARD();
     IOMMUFDBackend *iommufd = vbasedev->iommufd;
     VFIOContainer *bcontainer = VFIO_IOMMU(container);
-    uint32_t type, flags = 0;
+    uint32_t type = 0, flags = 0;
     uint64_t hw_caps;
     VendorCaps caps;
     VFIOIOASHwpt *hwpt;

hw/vfio/iommufd.c static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, · line 941

Quoted 5 lines
hw/vfio/iommufd.c +1 -1
@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,
     HostIOMMUDeviceIOMMUFD *idev;
     HostIOMMUDeviceCaps *caps = &hiod->caps;
     VendorCaps *vendor_caps = &caps->vendor_caps;
-    enum iommu_hw_info_type type;
+    enum iommu_hw_info_type type = 0;

here also

With that

Thanks

Eric

Quoted 3 lines
hw/vfio/iommufd.c +0 -0
     uint8_t max_pasid_log2;
     uint64_t hw_caps;
 
#3 lore raw

PATCH v2 2/24 backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.

Update the caller accordingly.

---
 backends/iommufd.c       | 4 ++++
 backends/trace-events    | 2 +-
 hw/arm/smmuv3-accel.c    | 4 ++--
 include/system/iommufd.h | 1 +
 4 files changed, 8 insertions(+), 3 deletions(-)
backends/iommufd.c +4 -0
index 5daefe505e..8ab26eb786 100644
@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,
 
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_viommu_id, Error **errp)
 {
     int ret;
@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
         .type = viommu_type,
         .dev_id = dev_id,
         .hwpt_id = hwpt_id,
+        .data_len = data_len,
+        .data_uptr = (uintptr_t)data_ptr,
     };
 
     ret = ioctl(be->fd, IOMMU_VIOMMU_ALLOC, &alloc_viommu);
 
     trace_iommufd_backend_alloc_viommu(be->fd, dev_id, viommu_type, hwpt_id,
+                                       data_len, (uintptr_t)data_ptr,
                                        alloc_viommu.out_viommu_id, ret);
     if (ret) {
         error_setg_errno(errp, errno, "IOMMU_VIOMMU_ALLOC failed");
backends/trace-events +1 -1
index b9365113e7..332888a576 100644
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (%
 iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)"
 iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)"
 iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)"
-iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)"
+iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 
hw/arm/smmuv3-accel.c +2 -2
index 14ea4eac37..9036b14601 100644
@@ -531,8 +531,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     IOMMUFDViommu *viommu;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
-                                      s2_hwpt_id, &viommu_id, errp)) {
+                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
+                                      NULL, 0, &viommu_id, errp)) {
         return false;
     }
 
include/system/iommufd.h +1 -0
index e4ca16da70..5ef23ad9e1 100644
@@ -87,6 +87,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id,
                                 Error **errp);
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_hwpt, Error **errp);
 
 bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
Signature · 1 line
2.43.0
Reviewed-by on v2 2/24
#4 lore raw

Re: [PATCH v2 02/24] backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr

Quoted 51 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.

Update the caller accordingly.

---
 backends/iommufd.c       | 4 ++++
 backends/trace-events    | 2 +-
 hw/arm/smmuv3-accel.c    | 4 ++--
 include/system/iommufd.h | 1 +
 4 files changed, 8 insertions(+), 3 deletions(-)
backends/iommufd.c +4 -0
index 5daefe505e..8ab26eb786 100644
@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,
 
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_viommu_id, Error **errp)
 {
     int ret;
@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
         .type = viommu_type,
         .dev_id = dev_id,
         .hwpt_id = hwpt_id,
+        .data_len = data_len,
+        .data_uptr = (uintptr_t)data_ptr,
     };
 
     ret = ioctl(be->fd, IOMMU_VIOMMU_ALLOC, &alloc_viommu);
 
     trace_iommufd_backend_alloc_viommu(be->fd, dev_id, viommu_type, hwpt_id,
+                                       data_len, (uintptr_t)data_ptr,
                                        alloc_viommu.out_viommu_id, ret);
     if (ret) {
         error_setg_errno(errp, errno, "IOMMU_VIOMMU_ALLOC failed");
backends/trace-events +0 -0
index b9365113e7..332888a576 100644

backends/trace-events iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (% · line 24

Quoted 5 lines
backends/trace-events +1 -1
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (%
 iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)"
 iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)"
 iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)"
-iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)"
+iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)"

nit: I would put the data_ptr first and then the data length to match most prevalent pattern. Also use hexa format for the length?

Besides

Eric

Quoted 30 lines
backends/trace-events +0 -0
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 
hw/arm/smmuv3-accel.c +2 -2
index 14ea4eac37..9036b14601 100644
@@ -531,8 +531,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     IOMMUFDViommu *viommu;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
-                                      s2_hwpt_id, &viommu_id, errp)) {
+                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
+                                      NULL, 0, &viommu_id, errp)) {
         return false;
     }
 
include/system/iommufd.h +1 -0
index e4ca16da70..5ef23ad9e1 100644
@@ -87,6 +87,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id,
                                 Error **errp);
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_hwpt, Error **errp);
 
 bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
#5 lore raw

PATCH v2 3/24 backends/iommufd: Introduce iommufd_backend_alloc_hw_queue

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a helper to allocate an iommufd backed HW queue for a vIOMMU.

While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.

---
 backends/iommufd.c       | 32 ++++++++++++++++++++++++++++++++
 backends/trace-events    |  1 +
 include/system/iommufd.h | 12 ++++++++++++
 3 files changed, 45 insertions(+)
backends/iommufd.c +32 -0
index 8ab26eb786..c6b79da0d4 100644
@@ -546,6 +546,38 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    enum iommu_hw_queue_type queue_type,
+                                    uint32_t index, uint64_t addr,
+                                    uint64_t length, uint32_t *out_hw_queue_id,
+                                    Error **errp)
+{
+    int ret;
+    struct iommu_hw_queue_alloc alloc_hw_queue = {
+        .size = sizeof(alloc_hw_queue),
+        .flags = 0,
+        .viommu_id = viommu_id,
+        .type = queue_type,
+        .index = index,
+        .nesting_parent_iova = addr,
+        .length = length,
+    };
+
+    ret = ioctl(be->fd, IOMMU_HW_QUEUE_ALLOC, &alloc_hw_queue);
+
+    trace_iommufd_backend_alloc_hw_queue(be->fd, viommu_id, queue_type,
+                                         index, addr, length,
+                                         alloc_hw_queue.out_hw_queue_id, ret);
+    if (ret) {
+        error_setg_errno(errp, errno, "IOMMU_HW_QUEUE_ALLOC failed");
+        return false;
+    }
+
+    g_assert(out_hw_queue_id);
+    *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id;
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index 332888a576..3e70338750 100644
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u
 iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
+iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +12 -0
index 5ef23ad9e1..f72ad545f8 100644
@@ -63,6 +63,12 @@ typedef struct IOMMUFDVeventq {
     uint32_t veventq_fd;
 } IOMMUFDVeventq;
 
+/* HW queue object for a vIOMMU-specific HW-accelerated queue */
+typedef struct IOMMUFDHWqueue {
+    IOMMUFDViommu *viommu;
+    uint32_t hw_queue_id;
+} IOMMUFDHWqueue;
+
 bool iommufd_backend_connect(IOMMUFDBackend *be, Error **errp);
 void iommufd_backend_disconnect(IOMMUFDBackend *be);
 
@@ -99,6 +105,12 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
                                    uint32_t *out_veventq_id,
                                    uint32_t *out_veventq_fd, Error **errp);
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    enum iommu_hw_queue_type queue_type,
+                                    uint32_t index, uint64_t addr,
+                                    uint64_t length, uint32_t *out_hw_queue_id,
+                                    Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH v2 03/24] backends/iommufd: Introduce iommufd_backend_alloc_hw_queue

Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a helper to allocate an iommufd backed HW queue for a vIOMMU.

While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.

Quoted 7 lines
---
  backends/iommufd.c       | 32 ++++++++++++++++++++++++++++++++
  backends/trace-events    |  1 +
  include/system/iommufd.h | 12 ++++++++++++

Could you please add to your .gitconfig :

[diff]
orderFile = /path/to/qemu/scripts/git.orderfile

Quoted 75 lines
  3 files changed, 45 insertions(+)
backends/iommufd.c +32 -0
index 8ab26eb786..c6b79da0d4 100644
@@ -546,6 +546,38 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
      return true;
  }
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    enum iommu_hw_queue_type queue_type,
+                                    uint32_t index, uint64_t addr,
+                                    uint64_t length, uint32_t *out_hw_queue_id,
+                                    Error **errp)
+{
+    int ret;
+    struct iommu_hw_queue_alloc alloc_hw_queue = {
+        .size = sizeof(alloc_hw_queue),
+        .flags = 0,
+        .viommu_id = viommu_id,
+        .type = queue_type,
+        .index = index,
+        .nesting_parent_iova = addr,
+        .length = length,
+    };
+
+    ret = ioctl(be->fd, IOMMU_HW_QUEUE_ALLOC, &alloc_hw_queue);
+
+    trace_iommufd_backend_alloc_hw_queue(be->fd, viommu_id, queue_type,
+                                         index, addr, length,
+                                         alloc_hw_queue.out_hw_queue_id, ret);
+    if (ret) {
+        error_setg_errno(errp, errno, "IOMMU_HW_QUEUE_ALLOC failed");
+        return false;
+    }
+
+    g_assert(out_hw_queue_id);
+    *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id;
+    return true;
+}
+
  bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                             uint32_t hwpt_id, Error **errp)
  {
backends/trace-events +1 -0
index 332888a576..3e70338750 100644
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u
  iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)"
  iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
  iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
+iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
 
  # igvm-cfg.c
  igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +6 -0
index 5ef23ad9e1..f72ad545f8 100644
@@ -63,6 +63,12 @@ typedef struct IOMMUFDVeventq {
      uint32_t veventq_fd;
  } IOMMUFDVeventq;
 
+/* HW queue object for a vIOMMU-specific HW-accelerated queue */
+typedef struct IOMMUFDHWqueue {
+    IOMMUFDViommu *viommu;
+    uint32_t hw_queue_id;
+} IOMMUFDHWqueue;
+
  bool iommufd_backend_connect(IOMMUFDBackend *be, Error **errp);
  void iommufd_backend_disconnect(IOMMUFDBackend *be);
 

include/system/iommufd.h bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id, · line 109

Quoted 5 lines
include/system/iommufd.h +2 -0
@@ -99,6 +105,12 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
                                     uint32_t *out_veventq_id,
                                     uint32_t *out_veventq_fd, Error **errp);
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    enum iommu_hw_queue_type queue_type,

Please use a uint32_t for the queue_type parameter. 'enum iommu_hw_queue_type' is a Linux-only type and build will break on other platform.

Thanks,

C.

Quoted 7 lines
include/system/iommufd.h +4 -0
+                                    uint32_t index, uint64_t addr,
+                                    uint64_t length, uint32_t *out_hw_queue_id,
+                                    Error **errp);
+
  bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                          bool start, Error **errp);
  bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
#7 lore raw

PATCH v2 4/24 backends/iommufd: Introduce iommufd_backend_viommu_mmap

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.

The caller is responsible for unmapping the returned region.

---
 backends/iommufd.c       | 22 ++++++++++++++++++++++
 backends/trace-events    |  1 +
 include/system/iommufd.h |  4 ++++
 3 files changed, 27 insertions(+)
backends/iommufd.c +22 -0
index c6b79da0d4..9e30501d38 100644
@@ -578,6 +578,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+/*
+ * Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance.
+ * The caller is responsible for unmapping the mapped region.
+ */
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp)
+{
+    g_assert(viommu_id);
+    g_assert(out_ptr);
+
+    *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, be->fd,
+                   offset);
+    trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size, offset);
+    if (*out_ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno, "IOMMUFD vIOMMU mmap failed");
+        return false;
+    }
+
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index 3e70338750..f824a8d197 100644
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +4 -0
index f72ad545f8..87ddc39041 100644
@@ -111,6 +111,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
                                     uint64_t length, uint32_t *out_hw_queue_id,
                                     Error **errp);
 
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Signature · 1 line
2.43.0
#8 lore raw

Re: [PATCH v2 04/24] backends/iommufd: Introduce iommufd_backend_viommu_mmap

Quoted 22 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.

The caller is responsible for unmapping the returned region.

---
 backends/iommufd.c       | 22 ++++++++++++++++++++++
 backends/trace-events    |  1 +
 include/system/iommufd.h |  4 ++++
 3 files changed, 27 insertions(+)
backends/iommufd.c +0 -0
index c6b79da0d4..9e30501d38 100644

backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, · line 587

Quoted 10 lines
backends/iommufd.c +7 -0
@@ -578,6 +578,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+/*
+ * Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance.
+ * The caller is responsible for unmapping the mapped region.
+ */
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp)

Looking at this again, there is nothing really related to iommufd. This could simply end up as a static helper in tegra241-cmdqv.c

Eric

Quoted 45 lines
backends/iommufd.c +15 -0
+{
+    g_assert(viommu_id);
+    g_assert(out_ptr);
+
+    *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, be->fd,
+                   offset);
+    trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size, offset);
+    if (*out_ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno, "IOMMUFD vIOMMU mmap failed");
+        return false;
+    }
+
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index 3e70338750..f824a8d197 100644
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +4 -0
index f72ad545f8..87ddc39041 100644
@@ -111,6 +111,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
                                     uint64_t length, uint32_t *out_hw_queue_id,
                                     Error **errp);
 
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
#9 lore raw

RE: [PATCH v2 04/24] backends/iommufd: Introduce iommufd_backend_viommu_mmap

Quoted 51 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 13 February 2026 10:19
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 04/24] backends/iommufd: Introduce iommufd_backend_viommu_mmap

External email: Use caution opening links or attachments

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a backend helper to mmap hardware MMIO regions exposed via

iommufd for

backends/iommufd.c bool

Quoted 19 lines

a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.

The caller is responsible for unmapping the returned region.

---
 backends/iommufd.c       | 22 ++++++++++++++++++++++
 backends/trace-events    |  1 +
 include/system/iommufd.h |  4 ++++
 3 files changed, 27 insertions(+)
backends/iommufd.c +0 -0
index c6b79da0d4..9e30501d38 100644
@@ -578,6 +578,28 @@ bool

iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

Quoted 5 lines
backends/iommufd.c +2 -0
     return true;
 }
 
+/*
+ * Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU

instance.

Quoted 8 lines

backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

Quoted 3 lines
backends/iommufd.c +3 -0
+ * The caller is responsible for unmapping the mapped region.
+ */
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t

viommu_id,

backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, · line 587

Quoted 2 lines
backends/iommufd.c +2 -0
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp)

Looking at this again, there is nothing really related to iommufd. This could simply end up as a static helper in tegra241-cmdqv.c

I had that thought as well. However, this operates on the IOMMUFDBackend fd and is not tegra specific. It performs an mmap on the backend fd associated with a vIOMMU instance, which could potentially be reused by others.

Keeping it in backends/iommufd.c also centralizes fd related operations (including tracing) for the IOMMUFD backend.

Thanks,
Shameer

Quoted 64 lines

backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

Quoted 5 lines
backends/iommufd.c +5 -0
+{
+    g_assert(viommu_id);
+    g_assert(out_ptr);
+
+    *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED,

be->fd,

backends/iommufd.c bool · line 594

Quoted 2 lines
backends/iommufd.c +2 -0
+                   offset);
+    trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size,

offset);

backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

Quoted 9 lines
backends/iommufd.c +8 -0
+    if (*out_ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno, "IOMMUFD vIOMMU mmap failed");
+        return false;
+    }
+
+    return true;
+}
+
 bool

host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,

backends/trace-events iommufd_backend_alloc_viommu(int iommufd,

Quoted 7 lines
backends/iommufd.c +0 -0
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +0 -0
index 3e70338750..f824a8d197 100644
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd,

uint32_t dev_id, uint32_t type, uint32

backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 25

Quoted 1 line
backends/trace-events +0 -0
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t

viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"

backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 26

Quoted 1 line
backends/trace-events +0 -0
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id,

uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"

backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 27

Quoted 1 line
backends/trace-events +0 -0
 iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id,

uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"

backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 28

Quoted 1 line
backends/trace-events +1 -0
+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id,

uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64

include/system/iommufd.h bool

Quoted 8 lines
backends/trace-events +0 -0
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +0 -0
index f72ad545f8..87ddc39041 100644
@@ -111,6 +111,10 @@ bool

iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

include/system/iommufd.h bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

Quoted 4 lines
include/system/iommufd.h +1 -0
                                     uint64_t length, uint32_t *out_hw_queue_id,
                                     Error **errp);
 
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t

viommu_id,

include/system/iommufd.h bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,

Quoted 4 lines
include/system/iommufd.h +3 -0
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be,

uint32_t hwpt_id,

include/system/iommufd.h bool · line 120

Quoted 2 lines
include/system/iommufd.h +0 -0
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,

uint32_t hwpt_id,

#10 lore raw

PATCH v2 5/24 hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +19 -0
index c9c10e55c3..ca087240e5 100644
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
Signature · 1 line
2.43.0
#11 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 2 lines · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

Quoted 18 lines

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index c9c10e55c3..ca087240e5 100644

hw/arm/smmuv3-accel.h line 21

Quoted 6 lines
hw/arm/smmuv3-accel.h +3 -0
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.

Ditto

hw/arm/smmuv3-accel.h line 30

Quoted 10 lines
hw/arm/smmuv3-accel.h +9 -0
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
 
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);

I don't see much value of having alloc/free_viommu..

Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.

hw/arm/smmuv3-accel.h line 32

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

alloc_/free_veventq can be moved into alloc/free_viommu functions.

hw/arm/smmuv3-accel.h line 34

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;

Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49

Quoted 9 lines
hw/arm/smmuv3-accel.h +2 -0
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;

Having two pointers for the same extension feels redundant. Maybe merge them with:

typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV struct iommu_viommu_tegra241_cmdqv viommu_data;
} SMMUv3AccelCmdqv;

...

SMMUv3AccelState {
    ...
    SMMUv3AccelCmdqv *cmdqv;
};

?

Nicolin

#12 lore raw

RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 16 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 6 lines
Quoted 1 line · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that

enables

Quoted 1 line

virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.

Quoted 40 lines
Quoted 19 lines

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index c9c10e55c3..ca087240e5 100644

hw/arm/smmuv3-accel.h line 20

Quoted 5 lines
hw/arm/smmuv3-accel.h +2 -0
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3

to

hw/arm/smmuv3-accel.h line 21

Quoted 1 line
hw/arm/smmuv3-accel.h +1 -0
+ * support multiple VCMDQs with virtualization capabilities.

Ditto

hw/arm/smmuv3-accel.h line 30

Quoted 10 lines
hw/arm/smmuv3-accel.h +9 -0
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
 
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
Quoted 4 lines

I don't see much value of having alloc/free_viommu..

Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.

That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.

Quoted 11 lines

hw/arm/smmuv3-accel.h line 32

Quoted 3 lines
hw/arm/smmuv3-accel.h +2 -0
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

alloc_/free_veventq can be moved into alloc/free_viommu functions.

hw/arm/smmuv3-accel.h line 34

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;

Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?

That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.

I'm open to suggestions, and let's wait to hear from others as well.

Quoted 19 lines

hw/arm/smmuv3-accel.h line 37

Quoted 2 lines
hw/arm/smmuv3-accel.h +0 -0
 /*
  * Represents an accelerated SMMU instance backed by an iommufd

vIOMMU object.

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49

Quoted 7 lines
hw/arm/smmuv3-accel.h +2 -0
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;

Having two pointers for the same extension feels redundant. Maybe merge them with:

typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV struct iommu_viommu_tegra241_cmdqv viommu_data;
} SMMUv3AccelCmdqv;

Quoted 8 lines

...

SMMUv3AccelState {
    ...
    SMMUv3AccelCmdqv *cmdqv;
};

?

We could fold the private data and ops into a single struct and store that in SMMUv3AccelState. However, if we want to keep this generic, it may be better to keep them separate.

Thanks,
Shameer

#13 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 11 lines · Shameer Kolothum Thodi
Quoted 6 lines · Shameer Kolothum
Quoted 1 line

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that

enables

Quoted 1 line

virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.

That might be a different naming which we have no idea about yet :-/

Quoted 7 lines
Quoted 6 lines

hw/arm/smmuv3-accel.h line 34

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;

Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?

Quoted 6 lines

That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.

I'm open to suggestions, and let's wait to hear from others as well.

Well, I'll vote for adding that when someday there is second CMDQV. There are only a couple of CMDQ-V functions required to export for SMMUv3 code to call at this moment, which isn't so bad.

Having and naming a structure exclusively for NVIDIA CMDQ-V doesn't make a lot of sense to me. And if we really want an abstraction ops maybe "secondary_cmdq" in the kernel driver could be a good choice?

Nicolin

#14 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 22 lines · Nicolin Chen
Quoted 9 lines · Shameer Kolothum Thodi
Quoted 5 lines · Shameer Kolothum
Quoted 1 line

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that

enables

Quoted 1 line

virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.

That might be a different naming which we have no idea about yet :-/

Quoted 10 lines
Quoted 5 lines

hw/arm/smmuv3-accel.h line 34

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;

Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?

That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.

Quoted 8 lines
Quoted 1 line

I'm open to suggestions, and let's wait to hear from others as well.

Well, I'll vote for adding that when someday there is second CMDQV. There are only a couple of CMDQ-V functions required to export for SMMUv3 code to call at this moment, which isn't so bad.

Having and naming a structure exclusively for NVIDIA CMDQ-V doesn't make a lot of sense to me. And if we really want an abstraction ops maybe "secondary_cmdq" in the kernel driver could be a good choice?

As mentionned earlier mu syggestion was mainly to address specialization with known implementations: accel smmu with vcmdq versus without. And i think we already have some implementation override in that matter.

Eric

Quoted 3 lines

Nicolin

#15 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum Thodi
Quoted 61 lines · Shameer Kolothum Thodi
Quoted 15 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 5 lines
Quoted 1 line · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that

enables

Quoted 1 line

virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.

Quoted 36 lines
Quoted 18 lines

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index c9c10e55c3..ca087240e5 100644

hw/arm/smmuv3-accel.h line 20

Quoted 5 lines
hw/arm/smmuv3-accel.h +2 -0
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3

to

hw/arm/smmuv3-accel.h line 21

Quoted 1 line
hw/arm/smmuv3-accel.h +1 -0
+ * support multiple VCMDQs with virtualization capabilities.

Ditto

Quoted 9 lines
hw/arm/smmuv3-accel.h +9 -0
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
Quoted 7 lines
Quoted 4 lines

I don't see much value of having alloc/free_viommu..

Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.

That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.

My initial thought was callbacks to specialize implementation for accel smmu without vcmdq versus accel smmu with vcmdqs. Effectively we could later imagine to have other implementations with vcmdqs but it is difficult to guess the specialization needs at that stage. I think in the original series you already add the first kind of differentiation.

At least reset and init need to be specialized.

Maybe alloc_viommu does not need to because the differentiation only happens on the type arg. But originally you where also turning off

s->tegra241_cmdqv if IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV failed so I had the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

Quoted 49 lines
Quoted 9 lines

hw/arm/smmuv3-accel.h line 32

Quoted 3 lines
hw/arm/smmuv3-accel.h +2 -0
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

alloc_/free_veventq can be moved into alloc/free_viommu functions.

hw/arm/smmuv3-accel.h line 34

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;

Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?

That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.

I'm open to suggestions, and let's wait to hear from others as well.

Quoted 18 lines

hw/arm/smmuv3-accel.h line 36

Quoted 2 lines
hw/arm/smmuv3-accel.h +0 -0
 /*
  * Represents an accelerated SMMU instance backed by an iommufd

vIOMMU object.

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49

Quoted 7 lines
hw/arm/smmuv3-accel.h +2 -0
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;

Having two pointers for the same extension feels redundant. Maybe merge them with:

typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV struct iommu_viommu_tegra241_cmdqv viommu_data;
} SMMUv3AccelCmdqv;

Quoted 8 lines

...

SMMUv3AccelState {
    ...
    SMMUv3AccelCmdqv *cmdqv;
};

?

We could fold the private data and ops into a single struct and store that in SMMUv3AccelState. However, if we want to keep this generic, it may be better to keep them separate.

Thanks,
Shameer

#16 lore raw

RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Hi Eric,

Quoted 97 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:42
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

External email: Use caution opening links or attachments

Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV

ops

Quoted 33 lines
Quoted 7 lines

interface

Quoted 1 line · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that

enables

Quoted 1 line

virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.

Quoted 21 lines
Quoted 21 lines

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index c9c10e55c3..ca087240e5 100644
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
Quoted 2 lines
Quoted 2 lines

hw/arm/smmuv3-accel.h line 20

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the

SMMUv3

Quoted 13 lines
Quoted 13 lines

to

hw/arm/smmuv3-accel.h line 21

Quoted 1 line
hw/arm/smmuv3-accel.h +1 -0
+ * support multiple VCMDQs with virtualization capabilities.

Ditto

Quoted 9 lines
hw/arm/smmuv3-accel.h +9 -0
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
Quoted 7 lines
Quoted 4 lines

I don't see much value of having alloc/free_viommu..

Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.

That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.

My initial thought was callbacks to specialize implementation for accel smmu without vcmdq versus accel smmu with vcmdqs. Effectively we could later imagine to have other implementations with vcmdqs but it is difficult to guess the specialization needs at that stage. I think in the original series you already add the first kind of differentiation.

At least reset and init need to be specialized.

Quoted 7 lines

Maybe alloc_viommu does not need to because the differentiation only happens on the type arg. But originally you where also turning off

s->tegra241_cmdqv if IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV failed so I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Thanks,
Shameer

#17 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 100 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 98 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:42
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

External email: Use caution opening links or attachments

Quoted 4 lines · Shameer Kolothum Thodi
Quoted 4 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>

Quoted 8 lines
Quoted 8 lines

Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV

ops

Quoted 33 lines
Quoted 7 lines

interface

Quoted 1 line · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that

enables

Quoted 1 line

virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.

Quoted 21 lines
Quoted 21 lines

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index c9c10e55c3..ca087240e5 100644
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
Quoted 2 lines
Quoted 2 lines

hw/arm/smmuv3-accel.h line 20

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the

SMMUv3

Quoted 13 lines
Quoted 13 lines

to

hw/arm/smmuv3-accel.h line 21

Quoted 1 line
hw/arm/smmuv3-accel.h +1 -0
+ * support multiple VCMDQs with virtualization capabilities.

Ditto

Quoted 9 lines
hw/arm/smmuv3-accel.h +9 -0
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
Quoted 7 lines
Quoted 4 lines

I don't see much value of having alloc/free_viommu..

Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.

That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.

My initial thought was callbacks to specialize implementation for accel smmu without vcmdq versus accel smmu with vcmdqs. Effectively we could later imagine to have other implementations with vcmdqs but it is difficult to guess the specialization needs at that stage. I think in the original series you already add the first kind of differentiation.

At least reset and init need to be specialized.

Maybe alloc_viommu does not need to because the differentiation only happens on the type arg. But originally you where also turning off

Quoted 6 lines
Quoted 4 lines

s->tegra241_cmdqv if IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV failed so I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

Thanks

Eric

Quoted 4 lines

Thanks,
Shameer

#18 lore raw

RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 18 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 3 lines
Quoted 3 lines

I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

Quoted 6 lines
Quoted 2 lines

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

It is not an either/or case.

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();

if (accel && cmdqv)

    cmdqv_init();   

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

Thanks,
Shameer

#19 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 30 lines · Shameer Kolothum Thodi
Quoted 14 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

It is not an either/or case.

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();
Quoted 8 lines

if (accel && cmdqv)

    cmdqv_init();   

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

Well I am concerned by the lack of scalability overall. Until now this was more or less generic. Now we may have several implementors coming with their own implementations of even a vendor will come with several
generations: today it is tegra241_cmdq, tomorrow we will have tegraxxx_cmdq, arm_cmdq, ... How to get prepared for this mess? Will we have a property for each one?

Eric

Quoted 3 lines

Thanks,
Shameer

#20 lore raw

RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 61 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 15:29
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

External email: Use caution opening links or attachments

Quoted 8 lines · Shameer Kolothum Thodi
Quoted 8 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan

Chen

Quoted 5 lines
Quoted 5 lines

<[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV

ops

Quoted 17 lines
Quoted 9 lines

interface

Quoted 5 lines
Quoted 3 lines

I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

It is not an either/or case.

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();
Quoted 8 lines

if (accel && cmdqv)

    cmdqv_init();

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

Quoted 6 lines

Well I am concerned by the lack of scalability overall. Until now this was more or less generic. Now we may have several implementors coming with their own implementations of even a vendor will come with several
generations: today it is tegra241_cmdq, tomorrow we will have tegraxxx_cmdq, arm_cmdq, ... How to get prepared for this mess? Will we have a property for each one?

Agree. But at the same time, it is hard to design without knowing the details of other implementations. As a middle path I would propose to keep the ops but just have the additional init/reset callbacks there. This gives an opportunity to extend it in future without over engineering now.

For having separate property for every implementation, that looks like the way it is going to be now. Unless kernel provides us a way to probe the extension type.

Thanks,
Shameer

#21 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum Thodi
Quoted 30 lines · Shameer Kolothum Thodi
Quoted 14 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

It is not an either/or case.

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();
Quoted 8 lines

if (accel && cmdqv)

    cmdqv_init();   

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?

Eric

Quoted 3 lines

Thanks,
Shameer

#22 lore raw

RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 58 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 15:35
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

External email: Use caution opening links or attachments

Quoted 8 lines · Shameer Kolothum Thodi
Quoted 8 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan

Chen

Quoted 5 lines
Quoted 5 lines

<[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV

ops

Quoted 23 lines
Quoted 9 lines

interface

Quoted 5 lines
Quoted 3 lines

I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

It is not an either/or case.

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();

if (accel && cmdqv)

    cmdqv_init();

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

Quoted 6 lines
Quoted 2 lines

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?

Looking at it again, I think we can, by calling IOMMU_GET_HW_INFO with all supported CMDQ types and check which one succeeds.

Thanks,
Shameer

#23 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Hi Shameer,

Quoted 56 lines · Shameer Kolothum Thodi
Quoted 56 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 15:35
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

External email: Use caution opening links or attachments

Quoted 7 lines · Shameer Kolothum Thodi
Quoted 7 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan

Chen

Quoted 5 lines
Quoted 5 lines

<[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV

ops

Quoted 23 lines
Quoted 9 lines

interface

Quoted 5 lines
Quoted 3 lines

I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric

There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?

Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.

It is not an either/or case.

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();

if (accel && cmdqv)

    cmdqv_init();

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?

Looking at it again, I think we can, by calling IOMMU_GET_HW_INFO with all supported CMDQ types and check which one succeeds.

then I think we should endeavour doing that. Regarding the ops, if you feel that reset/init can be managed through a simple add-on code I am fine with it. Maybe Cédric or Peter will have another opinion though ;-)

Eric

Quoted 5 lines

Thanks,
Shameer

#24 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 11 lines · Eric Auger
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.

if (accel)

    accel_init();

if (accel && cmdqv)

    cmdqv_init();

So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.

Yes. And I think it makes more sense by moving cmdqv_init() into accel_init(), completely transparent to the common smmuv3 module maybe?

Quoted 11 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?

Looking at it again, I think we can, by calling IOMMU_GET_HW_INFO with all supported CMDQ types and check which one succeeds.

then I think we should endeavour doing that. Regarding the ops, if you feel that reset/init can be managed through a simple add-on code I am fine with it. Maybe Cédric or Peter will have another opinion though ;-)

I like that on/off/auto depending on the IOMMU_GET_HW_INFO return.

Thanks
Nicolin

#25 lore raw

RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 28 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 12 February 2026 17:23
To: Eric Auger <[email protected]>
Cc: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Quoted 1 line · Eric Auger
Quoted 1 line
Quoted 1 line
Quoted 1 line

The vCMDQ path does not replace the normal accelerated SMMU

init/reset.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Instead, it requires additional init/reset steps on top of the regular accel

flow.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

if (accel)

    accel_init();

if (accel && cmdqv)

    cmdqv_init();
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

So the vCMDQ logic is an extension of the accelerated mode, not a

fallback

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

or alternative implementation.

Yes. And I think it makes more sense by moving cmdqv_init() into accel_init(), completely transparent to the common smmuv3 module maybe?

I have attempted that, IIRC. There is a catch there that the pci iommu_ops needs to be set earlier(before smu_common realize) and we can only call cmdqv_init() after the base SMMUv3 sysbus_init_mmio() and smmu_init_irq().

I need to take another if we can make that work or not.

Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.

do we have any way to introspect from the kernel what cmdq type we

have,

Quoted 1 line
Quoted 1 line
Quoted 1 line

so that we could have a cmdqv option with ON/OFF/AUTO value instead

of

Quoted 2 lines
Quoted 2 lines
Quoted 1 line

duplication cmdq option for each vendor?

Looking at it again, I think we can, by calling IOMMU_GET_HW_INFO with

Quoted 8 lines

all supported

Quoted 5 lines
Quoted 1 line

CMDQ types and check which one succeeds.

then I think we should endeavour doing that. Regarding the ops, if you feel that reset/init can be managed through a simple add-on code I am fine with it. Maybe Cédric or Peter will have another opinion though ;-)

I like that on/off/auto depending on the IOMMU_GET_HW_INFO return.

Yeah. We need to do this.

Thanks,
Shameer

#26 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum
Quoted 2 lines · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).

Let's mention "NVIDIA", noting it's a non-standard extension.

Quoted 18 lines

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index c9c10e55c3..ca087240e5 100644

hw/arm/smmuv3-accel.h line 21

Quoted 6 lines
hw/arm/smmuv3-accel.h +3 -0
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.

Ditto

hw/arm/smmuv3-accel.h line 30

Quoted 9 lines
hw/arm/smmuv3-accel.h +9 -0
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);

I don't see much value of having alloc/free_viommu..

Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.

hw/arm/smmuv3-accel.h line 32

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

alloc_/free_veventq can be moved into alloc/free_viommu functions.

hw/arm/smmuv3-accel.h line 34

Quoted 2 lines
hw/arm/smmuv3-accel.h +2 -0
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;

Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is defined exclusively for Tegra241CMDQV?

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49

Quoted 9 lines
hw/arm/smmuv3-accel.h +2 -0
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;

Having two pointers for the same extension feels redundant. Maybe merge them with:

typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV struct iommu_viommu_tegra241_cmdqv viommu_data;
} SMMUv3AccelCmdqv;

...

SMMUv3AccelState {
    ...
    SMMUv3AccelCmdqv *cmdqv;
};

?

Nicolin

#27 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum
Quoted 45 lines · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +17 -0
index c9c10e55c3..ca087240e5 100644
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49

Quoted 5 lines
hw/arm/smmuv3-accel.h +2 -0
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;

effectively looks duplucate, check !cmdqv_ops ?

Eric

Quoted 3 lines
hw/arm/smmuv3-accel.h +0 -0
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
#28 lore raw

Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum
Quoted 45 lines · Shameer Kolothum

CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).

CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

No functional change

---
 hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +17 -0
index c9c10e55c3..ca087240e5 100644
@@ -16,6 +16,23 @@
 #endif
 #include CONFIG_DEVICES
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 48

Quoted 4 lines
hw/arm/smmuv3-accel.h +1 -0
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;

in my original thought this was not AccelCmdqOps but AccelOps. This should be squashed with the patch that turns existing accel impl into those ops implementation.

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49

Quoted 1 line
hw/arm/smmuv3-accel.h +1 -0
+    void *cmdqv;

Sorry I misread the purpose of that field. This aims at storing cmdqv data. But until this is actually used, I think it is useless to introduce it.

Eric

Quoted 3 lines
hw/arm/smmuv3-accel.h +0 -0
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
#29 lore raw

PATCH v2 6/24 hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

In reply to Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV initialization, vIOMMU/vEVENTQ allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/Kconfig          |  5 ++++
 hw/arm/meson.build      |  1 +
 hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h | 23 ++++++++++++++++
 4 files changed, 89 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.c
 create mode 100644 hw/arm/tegra241-cmdqv.h
hw/arm/Kconfig +5 -0
index c66c452737..3305c6e76e 100644
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +1 -0
index c250487e64..4ec91db50a 100644
@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
+arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
hw/arm/tegra241-cmdqv.c new file +60 -0

60 lines not on this page · raw message

hw/arm/tegra241-cmdqv.h new file +23 -0
new file mode 100644
index 0000000000..81c9deb384
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2025, NVIDIA CORPORATION
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_TEGRA241_CMDQV_H
+#define HW_TEGRA241_CMDQV_H
+
+#include CONFIG_DEVICES
+
+#ifdef CONFIG_TEGRA241_CMDQV
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void);
+#else
+static inline const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void)
+{
+    return NULL;
+}
+#endif
+#endif /* HW_TEGRA241_CMDQV_H */
Signature · 1 line
2.43.0
#30 lore raw

Re: [PATCH v2 06/24] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

Hi Shameer,

Quoted 126 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV initialization, vIOMMU/vEVENTQ allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/Kconfig          |  5 ++++
 hw/arm/meson.build      |  1 +
 hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h | 23 ++++++++++++++++
 4 files changed, 89 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.c
 create mode 100644 hw/arm/tegra241-cmdqv.h
hw/arm/Kconfig +5 -0
index c66c452737..3305c6e76e 100644
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +1 -0
index c250487e64..4ec91db50a 100644
@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
+arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
hw/arm/tegra241-cmdqv.c new file +60 -0
new file mode 100644
index 0000000000..6f30ca035b
@@ -0,0 +1,60 @@
+/*
+ * Copyright (C) 2025, NVIDIA CORPORATION
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
+{
+}
+
+static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
+{
+}
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                            uint32_t *out_viommu_id, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s)
+{
+}
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops_impl = {
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .alloc_veventq = tegra241_cmdqv_alloc_veventq,
+    .free_veventq = tegra241_cmdqv_free_veventq,
+    .reset = tegra241_cmdqv_reset,
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void)
+{
+    return &tegra241_cmdqv_ops_impl;
+}
hw/arm/tegra241-cmdqv.h new file +0 -0
new file mode 100644
index 0000000000..81c9deb384

hw/arm/tegra241-cmdqv.h line 10

Quoted 10 lines
hw/arm/tegra241-cmdqv.h new file +10 -0
@@ -0,0 +1,23 @@
+/*
+ * Copyright (C) 2025, NVIDIA CORPORATION
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_TEGRA241_CMDQV_H

nit: to match existing pattern this should be HW_ARM_TEGRA241_CMDQV_H

Quoted 13 lines
hw/arm/tegra241-cmdqv.h new file +13 -0
+#define HW_TEGRA241_CMDQV_H
+
+#include CONFIG_DEVICES
+
+#ifdef CONFIG_TEGRA241_CMDQV
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void);
+#else
+static inline const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void)
+{
+    return NULL;
+}
+#endif
+#endif /* HW_TEGRA241_CMDQV_H */
#31 lore raw

PATCH v2 7/24 hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

In reply to Shameer Kolothum

Integrate CMDQV ops support into the accelerated SMMUv3 path. When CMDQV is enabled, the backend ops are used to initialize CMDQV state and to allocate the vIOMMU instance. The current implementation connects the Tegra241 CMDQV backend, but does not enable functional support yet.

No functional changes intended.

---
 hw/arm/smmuv3-accel.c   | 65 ++++++++++++++++++++++++++++++++++++-----
 hw/arm/smmuv3-accel.h   |  5 ++++
 hw/arm/smmuv3.c         |  1 +
 include/hw/arm/smmuv3.h |  2 ++
 4 files changed, 66 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +58 -7
index 9036b14601..b1a8ab79b5 100644
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and shared_as_sysmem
@@ -520,6 +521,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -530,12 +532,24 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
-                                      NULL, 0, &viommu_id, errp)) {
+    if (cmdqv_ops && (!cmdqv_ops->alloc_viommu || !cmdqv_ops->alloc_veventq)) {
+        error_setg(errp, "CMDQV vIOMMU allocation not supported");
         return false;
     }
 
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {
+            return false;
+        }
+    }
+
     viommu = g_new0(IOMMUFDViommu, 1);
     viommu->viommu_id = viommu_id;
     viommu->s2_hwpt_id = s2_hwpt_id;
@@ -565,13 +579,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_bypass_hwpt;
     }
 
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+
     /* Attach a HWPT based on SMMUv3 GBPA.ABORT value */
     hwpt_id = smmuv3_accel_gbpa_hwpt(s, accel);
     if (!host_iommu_device_iommufd_attach_hwpt(idev, hwpt_id, errp)) {
-        goto free_veventq;
+        goto free_cmdqv_veventq;
     }
     return true;
 
+free_cmdqv_veventq:
+    if (cmdqv_ops && cmdqv_ops->free_veventq) {
+        cmdqv_ops->free_veventq(s);
+    }
 free_veventq:
     smmuv3_accel_free_veventq(accel);
 free_bypass_hwpt:
@@ -579,7 +601,11 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }
     g_free(viommu);
     accel->viommu = NULL;
     return false;
@@ -865,8 +891,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)
 
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
 static void smmuv3_accel_as_init(SMMUv3State *s)
@@ -886,6 +921,22 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
 }
 
+bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !s->tegra241_cmdqv) {
+        return true;
+    }
+
+    accel->cmdqv_ops = tegra241_cmdqv_ops();
+    if (!accel->cmdqv_ops || !accel->cmdqv_ops->init) {
+        error_setg(errp, "CMDQV support not available");
+        return false;
+    }
+    return accel->cmdqv_ops->init(s, errp);
+}
+
 void smmuv3_accel_init(SMMUv3State *s)
 {
     SMMUState *bs = ARM_SMMU(s);
hw/arm/smmuv3-accel.h +5 -0
index ca087240e5..33da37bfc1 100644
@@ -74,6 +74,7 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp);
 #else
 static inline void smmuv3_accel_init(SMMUv3State *s)
 {
@@ -110,6 +111,10 @@ static inline bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
 static inline void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+static bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    return true;
+}
 #endif
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3.c +1 -0
index 148af80efd..7858bf2c33 100644
@@ -2034,6 +2034,7 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
     smmu_init_irq(s, dev);
     smmuv3_init_id_regs(s);
+    smmuv3_accel_cmdqv_init(s, errp);
 }
 
 static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +2 -0
index 26b2fc42fd..87926f8cb3 100644
@@ -73,6 +73,8 @@ struct SMMUv3State {
     bool ats;
     uint8_t oas;
     uint8_t ssidsize;
+    /* Support for NVIDIA Tegra241 SMMU CMDQV extension */
+    bool tegra241_cmdqv;
 };
 
 typedef enum {
Signature · 1 line
2.43.0
#32 lore raw

Re: [PATCH v2 07/24] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

Hi Shameer,

Quoted 46 lines · Shameer Kolothum

Integrate CMDQV ops support into the accelerated SMMUv3 path. When CMDQV is enabled, the backend ops are used to initialize CMDQV state and to allocate the vIOMMU instance. The current implementation connects the Tegra241 CMDQV backend, but does not enable functional support yet.

No functional changes intended.

---
 hw/arm/smmuv3-accel.c   | 65 ++++++++++++++++++++++++++++++++++++-----
 hw/arm/smmuv3-accel.h   |  5 ++++
 hw/arm/smmuv3.c         |  1 +
 include/hw/arm/smmuv3.h |  2 ++
 4 files changed, 66 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +4 -3
index 9036b14601..b1a8ab79b5 100644
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and shared_as_sysmem
@@ -520,6 +521,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -530,12 +532,24 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
-                                      NULL, 0, &viommu_id, errp)) {
+    if (cmdqv_ops && (!cmdqv_ops->alloc_viommu || !cmdqv_ops->alloc_veventq)) {
+        error_setg(errp, "CMDQV vIOMMU allocation not supported");
         return false;
     }

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 548

Quoted 10 lines
hw/arm/smmuv3-accel.c +9 -0
 
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {

this is not add-on code but rather a switch. Depending on the cmdq type,

sizeof(cmdqv->cmdqv_data) varies. Wondering if ops does not makes sense here either.

Quoted 64 lines
hw/arm/smmuv3-accel.c +29 -4
+            return false;
+        }
+    }
+
     viommu = g_new0(IOMMUFDViommu, 1);
     viommu->viommu_id = viommu_id;
     viommu->s2_hwpt_id = s2_hwpt_id;
@@ -565,13 +579,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_bypass_hwpt;
     }
 
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+
     /* Attach a HWPT based on SMMUv3 GBPA.ABORT value */
     hwpt_id = smmuv3_accel_gbpa_hwpt(s, accel);
     if (!host_iommu_device_iommufd_attach_hwpt(idev, hwpt_id, errp)) {
-        goto free_veventq;
+        goto free_cmdqv_veventq;
     }
     return true;
 
+free_cmdqv_veventq:
+    if (cmdqv_ops && cmdqv_ops->free_veventq) {
+        cmdqv_ops->free_veventq(s);
+    }
 free_veventq:
     smmuv3_accel_free_veventq(accel);
 free_bypass_hwpt:
@@ -579,7 +601,11 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }
     g_free(viommu);
     accel->viommu = NULL;
     return false;
@@ -865,8 +891,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)
 
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
 static void smmuv3_accel_as_init(SMMUv3State *s)

hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State *s) · line 928

Quoted 8 lines
hw/arm/smmuv3-accel.c +5 -0
@@ -886,6 +921,22 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
 }
 
+bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !s->tegra241_cmdqv) {

isn't tegra241_cmdqv also only accelerated. Why do we need to have both checks?

Quoted 54 lines
hw/arm/smmuv3-accel.c +11 -0
+        return true;
+    }
+
+    accel->cmdqv_ops = tegra241_cmdqv_ops();
+    if (!accel->cmdqv_ops || !accel->cmdqv_ops->init) {
+        error_setg(errp, "CMDQV support not available");
+        return false;
+    }
+    return accel->cmdqv_ops->init(s, errp);
+}
+
 void smmuv3_accel_init(SMMUv3State *s)
 {
     SMMUState *bs = ARM_SMMU(s);
hw/arm/smmuv3-accel.h +5 -0
index ca087240e5..33da37bfc1 100644
@@ -74,6 +74,7 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp);
 #else
 static inline void smmuv3_accel_init(SMMUv3State *s)
 {
@@ -110,6 +111,10 @@ static inline bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
 static inline void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+static bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    return true;
+}
 #endif
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3.c +1 -0
index 148af80efd..7858bf2c33 100644
@@ -2034,6 +2034,7 @@ static void smmu_realize(DeviceState *d, Error **errp)
 
     smmu_init_irq(s, dev);
     smmuv3_init_id_regs(s);
+    smmuv3_accel_cmdqv_init(s, errp);
 }
 
 static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +0 -0
index 26b2fc42fd..87926f8cb3 100644

include/hw/arm/smmuv3.h struct SMMUv3State { · line 77

Quoted 5 lines
include/hw/arm/smmuv3.h +2 -0
@@ -73,6 +73,8 @@ struct SMMUv3State {
     bool ats;
     uint8_t oas;
     uint8_t ssidsize;
+    /* Support for NVIDIA Tegra241 SMMU CMDQV extension */
+    bool tegra241_cmdqv;

so I really advocate for auto detection and I think in that context I still think a selection of ops depending on the type of the cmdqv may be more future proof.
Also to me this should be within struct SMMUv3AccelState if cmdqv only works with accel. Or does emulation of cmdqv make sense?

Thanks

Eric

Quoted 3 lines
include/hw/arm/smmuv3.h +0 -0
 };
 
 typedef enum {
#33 lore raw

RE: [PATCH v2 07/24] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

Quoted 84 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 17:38
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 07/24] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 1 line · Shameer Kolothum

Integrate CMDQV ops support into the accelerated SMMUv3 path. When

CMDQV

Quoted 9 lines

is enabled, the backend ops are used to initialize CMDQV state and to allocate the vIOMMU instance. The current implementation connects the Tegra241 CMDQV backend, but does not enable functional support yet.

No functional changes intended.

---
 hw/arm/smmuv3-accel.c   | 65

++++++++++++++++++++++++++++++++++++-----

Quoted 10 lines
 hw/arm/smmuv3-accel.h   |  5 ++++
 hw/arm/smmuv3.c         |  1 +
 include/hw/arm/smmuv3.h |  2 ++
 4 files changed, 66 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +0 -0
index 9036b14601..b1a8ab79b5 100644

hw/arm/smmuv3-accel.c line 24

Quoted 7 lines
hw/arm/smmuv3-accel.c +1 -0
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and

shared_as_sysmem

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,

Quoted 1 line
hw/arm/smmuv3-accel.c +0 -0
@@ -520,6 +521,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,

Quoted 8 lines
hw/arm/smmuv3-accel.c +1 -0
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -530,12 +532,24 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

Quoted 11 lines
hw/arm/smmuv3-accel.c +2 -3
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
-                                      NULL, 0, &viommu_id, errp)) {
+    if (cmdqv_ops && (!cmdqv_ops->alloc_viommu || !cmdqv_ops-
 alloc_veventq)) {
+        error_setg(errp, "CMDQV vIOMMU allocation not supported");
         return false;
     }

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 548

Quoted 10 lines
hw/arm/smmuv3-accel.c +9 -0
 
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {
Quoted 1 line

this is not add-on code but rather a switch. Depending on the cmdq type,

Yeah. This is not add-on but a switch.

Quoted 74 lines

sizeof(cmdqv->cmdqv_data) varies. Wondering if ops does not makes sense here either.

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,

Quoted 8 lines
hw/arm/smmuv3-accel.c +4 -0
+            return false;
+        }
+    }
+
     viommu = g_new0(IOMMUFDViommu, 1);
     viommu->viommu_id = viommu_id;
     viommu->s2_hwpt_id = s2_hwpt_id;
hw/arm/smmuv3-accel.c +0 -0
@@ -565,13 +579,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/smmuv3-accel.c bool

Quoted 36 lines
hw/arm/smmuv3-accel.c +14 -2
         goto free_bypass_hwpt;
     }
 
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+
     /* Attach a HWPT based on SMMUv3 GBPA.ABORT value */
     hwpt_id = smmuv3_accel_gbpa_hwpt(s, accel);
     if (!host_iommu_device_iommufd_attach_hwpt(idev, hwpt_id, errp)) {
-        goto free_veventq;
+        goto free_cmdqv_veventq;
     }
     return true;
 
+free_cmdqv_veventq:
+    if (cmdqv_ops && cmdqv_ops->free_veventq) {
+        cmdqv_ops->free_veventq(s);
+    }
 free_veventq:
     smmuv3_accel_free_veventq(accel);
 free_bypass_hwpt:
@@ -579,7 +601,11 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }
     g_free(viommu);
     accel->viommu = NULL;
     return false;
@@ -865,8 +891,17 @@ bool

smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)

hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State

Quoted 20 lines
hw/arm/smmuv3-accel.c +11 -2
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
 static void smmuv3_accel_as_init(SMMUv3State *s)
@@ -886,6 +921,22 @@ static void smmuv3_accel_as_init(SMMUv3State

*s)

hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State · line 921

Quoted 1 line
hw/arm/smmuv3-accel.c +0 -0
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-

sysmem");

Quoted 2 lines
hw/arm/smmuv3-accel.c +0 -0
 }
 
Quoted 7 lines

hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State *s) · line 928

Quoted 5 lines
hw/arm/smmuv3-accel.c +5 -0
+bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !s->tegra241_cmdqv) {

isn't tegra241_cmdqv also only accelerated. Why do we need to have both checks?

True. accel check is not required.

Quoted 51 lines

hw/arm/smmuv3-accel.h bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s,

Quoted 19 lines
hw/arm/smmuv3-accel.c +11 -0
+        return true;
+    }
+
+    accel->cmdqv_ops = tegra241_cmdqv_ops();
+    if (!accel->cmdqv_ops || !accel->cmdqv_ops->init) {
+        error_setg(errp, "CMDQV support not available");
+        return false;
+    }
+    return accel->cmdqv_ops->init(s, errp);
+}
+
 void smmuv3_accel_init(SMMUv3State *s)
 {
     SMMUState *bs = ARM_SMMU(s);
hw/arm/smmuv3-accel.h +0 -0
index ca087240e5..33da37bfc1 100644
@@ -74,6 +74,7 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s,

void *cmd, SMMUDevice *sdev,

hw/arm/smmuv3-accel.h static inline bool

Quoted 8 lines
hw/arm/smmuv3-accel.h +1 -0
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp);
 #else
 static inline void smmuv3_accel_init(SMMUv3State *s)
 {
@@ -110,6 +111,10 @@ static inline bool

smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)

hw/arm/smmuv3.c static void smmu_realize(DeviceState *d, Error

Quoted 15 lines
hw/arm/smmuv3-accel.h +4 -0
 static inline void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+static bool smmuv3_accel_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    return true;
+}
 #endif
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3.c +0 -0
index 148af80efd..7858bf2c33 100644
@@ -2034,6 +2034,7 @@ static void smmu_realize(DeviceState *d, Error

**errp)

Quoted 6 lines
hw/arm/smmuv3.c +1 -0
     smmu_init_irq(s, dev);
     smmuv3_init_id_regs(s);
+    smmuv3_accel_cmdqv_init(s, errp);
 }
 
Quoted 16 lines

include/hw/arm/smmuv3.h struct SMMUv3State { · line 77

Quoted 11 lines
hw/arm/smmuv3.c +0 -0
 static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +2 -0
index 26b2fc42fd..87926f8cb3 100644
@@ -73,6 +73,8 @@ struct SMMUv3State {
     bool ats;
     uint8_t oas;
     uint8_t ssidsize;
+    /* Support for NVIDIA Tegra241 SMMU CMDQV extension */
+    bool tegra241_cmdqv;

so I really advocate for auto detection and I think in that context I still think a selection of ops depending on the type of the cmdqv may be more future proof.
Also to me this should be within struct SMMUv3AccelState if cmdqv only works with accel. Or does emulation of cmdqv make sense?

Let me see if I can move all cmdqv related stuff to SMMUv3AccelState or not.

Emulation makes no sense as the whole purpose of this extension is to accelerate the performance.

Thanks,
Shameer

#34 lore raw

PATCH v2 8/24 hw/arm/tegra241-cmdqv: Implement CMDQV init and vIOMMU allocation

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add initial Tegra241 CMDQV support by wiring up initialization and IOMMUFD backed vIOMMU allocation. The CMDQV MMIO region and IRQ are registered, while register handling remains a stub and will be completed in subsequent patches.

---
 hw/arm/tegra241-cmdqv.c | 55 ++++++++++++++++++++++++++++++++++++++---
 hw/arm/tegra241-cmdqv.h | 19 ++++++++++++++
 2 files changed, 70 insertions(+), 4 deletions(-)
hw/arm/tegra241-cmdqv.c +51 -4
index 6f30ca035b..596b1c5595 100644
@@ -13,6 +13,16 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+                                 unsigned size)
+{
+}
+
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
 {
 }
@@ -25,24 +35,61 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                             uint32_t *out_viommu_id, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        error_append_hint(errp, "Tegra241 CMDQV support unavailable");
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
 }
 
+static const MemoryRegionOps mmio_cmdqv_ops = {
+    .read = tegra241_cmdqv_read,
+    .write = tegra241_cmdqv_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv;
+
+    if (!accel) {
+        error_setg(errp, "Tegra241 CMDQV requires SMMUv3 acceleration");
+        return false;
+    }
+
+    cmdqv = g_new0(Tegra241CMDQV, 1);
+    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
+                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
+    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
+    sysbus_init_irq(sbd, &cmdqv->irq);
+    cmdqv->s_accel = accel;
+    accel->cmdqv = cmdqv;
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops_impl = {
hw/arm/tegra241-cmdqv.h +19 -0
index 81c9deb384..6ea0087f61 100644
@@ -10,8 +10,27 @@
 #ifndef HW_TEGRA241_CMDQV_H
 #define HW_TEGRA241_CMDQV_H
 
+#include "smmuv3-accel.h"
 #include CONFIG_DEVICES
 
+/*
+ * Tegra241 CMDQV MMIO layout (64KB pages):
+ *
+ * 0x00000: Global CMDQV registers
+ * 0x10000: Global VCMDQ registers, page 0
+ * 0x20000: Global VCMDQ registers, page 1
+ * 0x30000: VINTF0 logical VCMDQ registers, page 0
+ * 0x40000: VINTF0 logical VCMDQ registers, page 1
+ */
+#define TEGRA241_CMDQV_IO_LEN 0x50000
+
+typedef struct Tegra241CMDQV {
+    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
+    SMMUv3AccelState *s_accel;
+    MemoryRegion mmio_cmdqv;
+    qemu_irq irq;
+} Tegra241CMDQV;
+
 #ifdef CONFIG_TEGRA241_CMDQV
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void);
 #else
Signature · 1 line
2.43.0
#35 lore raw

Re: [PATCH v2 08/24] hw/arm/tegra241-cmdqv: Implement CMDQV init and vIOMMU allocation

Hi Shameer,

Quoted 6 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add initial Tegra241 CMDQV support by wiring up initialization and IOMMUFD backed vIOMMU allocation. The CMDQV MMIO region and IRQ are registered, while register handling remains a stub and will be completed in subsequent patches.

Explain what the MMIO region is made of (see below). Also describe what the interrupt is used for. what kind of interrupt is it? Also a work about the state (data)?

Quoted 100 lines
---
 hw/arm/tegra241-cmdqv.c | 55 ++++++++++++++++++++++++++++++++++++++---
 hw/arm/tegra241-cmdqv.h | 19 ++++++++++++++
 2 files changed, 70 insertions(+), 4 deletions(-)
hw/arm/tegra241-cmdqv.c +51 -4
index 6f30ca035b..596b1c5595 100644
@@ -13,6 +13,16 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+                                 unsigned size)
+{
+}
+
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
 {
 }
@@ -25,24 +35,61 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                             uint32_t *out_viommu_id, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        error_append_hint(errp, "Tegra241 CMDQV support unavailable");
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
 }
 
+static const MemoryRegionOps mmio_cmdqv_ops = {
+    .read = tegra241_cmdqv_read,
+    .write = tegra241_cmdqv_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv;
+
+    if (!accel) {
+        error_setg(errp, "Tegra241 CMDQV requires SMMUv3 acceleration");
+        return false;
+    }
+
+    cmdqv = g_new0(Tegra241CMDQV, 1);
+    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
+                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
+    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
+    sysbus_init_irq(sbd, &cmdqv->irq);
+    cmdqv->s_accel = accel;
+    accel->cmdqv = cmdqv;
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops_impl = {
hw/arm/tegra241-cmdqv.h +0 -0
index 81c9deb384..6ea0087f61 100644

hw/arm/tegra241-cmdqv.h line 19

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +5 -0
@@ -10,8 +10,27 @@
 #ifndef HW_TEGRA241_CMDQV_H
 #define HW_TEGRA241_CMDQV_H
 
+#include "smmuv3-accel.h"
 #include CONFIG_DEVICES
 
+/*
+ * Tegra241 CMDQV MMIO layout (64KB pages):
+ *
+ * 0x00000: Global CMDQV registers

I would use the spec terminology, something like

TEGRA241_CMDQV_CFG TEGRA241_CMDQV_VCMDQ0
TEGRA241_CMDQV_VCMDQ1 TEGRA241_CMDQV_VINTF0
TEGRA241_CMDQV_VINTF1 Eric

Quoted 17 lines
hw/arm/tegra241-cmdqv.h +14 -0
+ * 0x10000: Global VCMDQ registers, page 0
+ * 0x20000: Global VCMDQ registers, page 1
+ * 0x30000: VINTF0 logical VCMDQ registers, page 0
+ * 0x40000: VINTF0 logical VCMDQ registers, page 1
+ */
+#define TEGRA241_CMDQV_IO_LEN 0x50000
+
+typedef struct Tegra241CMDQV {
+    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
+    SMMUv3AccelState *s_accel;
+    MemoryRegion mmio_cmdqv;
+    qemu_irq irq;
+} Tegra241CMDQV;
+
 #ifdef CONFIG_TEGRA241_CMDQV
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void);
 #else
#36 lore raw

PATCH v2 9/24 hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages as follows:

0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0 (control/status) 0x20000: Global VCMDQ registers, Page1 (configuration) 0x30000: VINTF0 logical VCMDQ registers, Page0 (control/status) 0x40000: VINTF0 logical VCMDQ registers, Page1 (configuration)

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

Prepare the VINTF0 Page0 mapping by mmapping the kernel provided region when CMDQV is accessed, while keeping all CMDQV MMIO accesses trapped in QEMU. The mapping is not yet exposed to the guest address space. A subsequent patch will install the RAM backed MMIO subregion for VINTF page0 once HW_QUEUE allocation completes.

---
 hw/arm/tegra241-cmdqv.c | 38 ++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h |  3 +++
 2 files changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +38 -0
index 596b1c5595..97c9b9c8dc 100644
@@ -13,14 +13,52 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (!viommu) {
+        return true;
+    }
+
+    g_assert(!cmdqv->vintf_page0);
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_REG_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
+
+    if (!cmdqv->vintf_page0) {
+        if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
+            error_report_err(local_err);
+            local_err = NULL;
+        }
+    }
+
     return 0;
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
+
+    if (!cmdqv->vintf_page0) {
+        if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
+            error_report_err(local_err);
+            local_err = NULL;
+        }
+    }
 }
 
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +3 -0
index 6ea0087f61..94bef8c978 100644
@@ -29,8 +29,11 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 } Tegra241CMDQV;
 
+#define VINTF_REG_PAGE_SIZE 0x10000
+
 #ifdef CONFIG_TEGRA241_CMDQV
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void);
 #else
Signature · 1 line
2.43.0
#37 lore raw

Re: [PATCH v2 09/24] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Hi Shameer,

Quoted 11 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages as follows:

0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0 (control/status) 0x20000: Global VCMDQ registers, Page1 (configuration) 0x30000: VINTF0 logical VCMDQ registers, Page0 (control/status) 0x40000: VINTF0 logical VCMDQ registers, Page1 (configuration)

this shall rather be introduced along with the whole mmio region. In this patch you focus on rather VINTF0 page0

Quoted 5 lines

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

Maybe you shall simply name it VINTF

Quoted 11 lines

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

Prepare the VINTF0 Page0 mapping by mmapping the kernel provided region when CMDQV is accessed, while keeping all CMDQV MMIO accesses trapped in QEMU. The mapping is not yet exposed to the guest address space. A subsequent patch will install the RAM backed MMIO subregion for VINTF page0 once HW_QUEUE allocation completes.

So we are likely to mmap a page whose content is undefined until HW_QUEUE alloc, is that correct? In that case, why don't we do that only after HW_QUEUE alloc? Please can you clarify?

Quoted 19 lines
---
 hw/arm/tegra241-cmdqv.c | 38 ++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h |  3 +++
 2 files changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +3 -0
index 596b1c5595..97c9b9c8dc 100644
@@ -13,14 +13,52 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;

hw/arm/tegra241-cmdqv.c line 28

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+    if (!viommu) {
+        return true;
+    }
+
+    g_assert(!cmdqv->vintf_page0);
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_REG_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {

So I would rather directly implement iommufd_backend_viommu_mmap() here.

hw/arm/tegra241-cmdqv.c line 38

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8 -0
+        return false;
+    }
+
+    return true;
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;

init is not needed since you don't test it.

hw/arm/tegra241-cmdqv.c line 43

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+
+    if (!cmdqv->vintf_page0) {
+        if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
+            error_report_err(local_err);
+            local_err = NULL;

not needed either

hw/arm/tegra241-cmdqv.c line 54

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +5 -0
+        }
+    }
+
     return 0;
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;

same here

Quoted 15 lines
hw/arm/tegra241-cmdqv.c +7 -0
+
+    if (!cmdqv->vintf_page0) {
+        if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
+            error_report_err(local_err);
+            local_err = NULL;
+        }
+    }
 }
 
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +0 -0
index 6ea0087f61..94bef8c978 100644

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +2 -0
@@ -29,8 +29,11 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 } Tegra241CMDQV;
 
+#define VINTF_REG_PAGE_SIZE 0x10000

can't we rename that into VINTF_SIZE. REG does not mean much. Before you talked about Control/Status vs Configuration. Besides, I am not sure having a define brings much in that case.

Thanks

Eric

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +1 -0
+
 #ifdef CONFIG_TEGRA241_CMDQV
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops(void);
 #else
#38 lore raw

RE: [PATCH v2 09/24] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 30 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 13 February 2026 10:39
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 09/24] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV extends SMMUv3 with support for virtual command

queues

Quoted 1 line

(VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is

split

Quoted 2 lines

into 64KB pages as follows:

Quoted 7 lines
Quoted 5 lines

0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0 (control/status) 0x20000: Global VCMDQ registers, Page1 (configuration) 0x30000: VINTF0 logical VCMDQ registers, Page0 (control/status) 0x40000: VINTF0 logical VCMDQ registers, Page1 (configuration)

this shall rather be introduced along with the whole mmio region. In this patch you focus on rather VINTF0 page0

Ok.

Quoted 18 lines
Quoted 5 lines

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

Maybe you shall simply name it VINTF

Quoted 11 lines

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

Prepare the VINTF0 Page0 mapping by mmapping the kernel provided region when CMDQV is accessed, while keeping all CMDQV MMIO accesses trapped in QEMU. The mapping is not yet exposed to the guest address space. A subsequent patch will install the RAM backed MMIO subregion for VINTF page0 once HW_QUEUE allocation completes.

Quoted 3 lines

So we are likely to mmap a page whose content is undefined until HW_QUEUE alloc, is that correct? In that case, why don't we do that only after HW_QUEUE alloc? Please can you clarify?

Correct. Though we mmap it here, as mentioned in commit msg, it is not accessed until we do HW_QUEUE alloc(patch #16). Could delay this as well I suppose.

Quoted 19 lines
Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.c | 38

++++++++++++++++++++++++++++++++++++++

hw/arm/tegra241-cmdqv.c line 16

Quoted 12 lines
 hw/arm/tegra241-cmdqv.h |  3 +++
 2 files changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +1 -0
index 596b1c5595..97c9b9c8dc 100644
@@ -13,14 +13,52 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV

*cmdqv, Error **errp)

Quoted 15 lines

hw/arm/tegra241-cmdqv.c line 28

Quoted 13 lines
hw/arm/tegra241-cmdqv.c +12 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (!viommu) {
+        return true;
+    }
+
+    g_assert(!cmdqv->vintf_page0);
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
 viommu_id,
+                                     VINTF_REG_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {

So I would rather directly implement iommufd_backend_viommu_mmap() here.

I have explained the rationale for making this a common helper in the thread where we introduce it. Please let me know if that reasoning makes sense or not.

Quoted 52 lines

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +6 -0
+        return false;
+    }
+
+    return true;
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,

unsigned size)

hw/arm/tegra241-cmdqv.c line 38

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +2 -0
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;

init is not needed since you don't test it.

hw/arm/tegra241-cmdqv.c line 43

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+
+    if (!cmdqv->vintf_page0) {
+        if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
+            error_report_err(local_err);
+            local_err = NULL;

not needed either

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +3 -0
+        }
+    }
+
     return 0;
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t

value,

hw/arm/tegra241-cmdqv.c line 54

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +2 -0
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;

same here

Quoted 21 lines
hw/arm/tegra241-cmdqv.c +7 -0
+
+    if (!cmdqv->vintf_page0) {
+        if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
+            error_report_err(local_err);
+            local_err = NULL;
+        }
+    }
 }
 

static void tegra241_cmdqv_free_veventq(SMMUv3State *s)

hw/arm/tegra241-cmdqv.h +1 -0
index 6ea0087f61..94bef8c978 100644
@@ -29,8 +29,11 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 } Tegra241CMDQV;
 
Quoted 4 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define VINTF_REG_PAGE_SIZE 0x10000

can't we rename that into VINTF_SIZE. REG does not mean much. Before you talked about Control/Status vs Configuration. Besides, I am not sure having a define brings much in that case.

Ok.

Thanks,
Shameer

#39 lore raw

PATCH v2 10/24 hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

---
 hw/arm/tegra241-cmdqv.c | 102 +++++++++++++++++++++++++++++++++++++++-
 hw/arm/tegra241-cmdqv.h |  86 +++++++++++++++++++++++++++++++++
 2 files changed, 187 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +101 -1
index 97c9b9c8dc..49fca9d536 100644
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
@@ -32,6 +33,25 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
     return true;
 }
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
@@ -44,7 +64,57 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
         }
     }
 
-    return 0;
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
@@ -59,6 +129,36 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
             local_err = NULL;
         }
     }
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
 }
 
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +86 -0

100 lines not on this page · raw message

Signature · 1 line
2.43.0
#40 lore raw

Re: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Quoted 6 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +6 -0
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 51

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;

I wonder if we'd still need to implement VINTF0_SID_MATCH/REPLACE registers as it's a part of HW spec. They don't need to associate with anything but register cache alone to support MMIO.

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    switch (offset) {

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 111

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }

To emulate the HW perfectly, VINTF_EN = 0 might have to disable the VCMDQ feature, i.e. deallocate HW_QUEUEs and munmap page0.

Otherwise, guest could continue using the vcmdq "hardware" after disabling it, which doesn't make sense.

This means that HW_QUEUE allocations would have to be delayed, until VINTF_EN=1 and corresponding VCMDQ_EN=1.

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 136

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +8 -0
@@ -59,6 +129,36 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }

We'd need to do something about this as well. VINTF/VCMDQ should not be functional when CMDQV_EN=0.

Thanks
Nicolin

#41 lore raw

RE: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Quoted 33 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 23:19
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Quoted 1 line · Shameer Kolothum

+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,

hwaddr offset)

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9 -0
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ...

A_VINTF0_LVCMDQ_ERR_MAP_3:

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
Quoted 7 lines

PRIx64 "\n",

Review

Quoted 2 lines
(quoted diff) +2 -0
+                      __func__, offset);
+        return 0;

I wonder if we'd still need to implement VINTF0_SID_MATCH/REPLACE registers as it's a part of HW spec. They don't need to associate with anything but register cache alone to support MMIO.

Ok. But I was thinking Guest should not be allowed to configure it as it belongs to hypervisor, right?

Quoted 20 lines
Quoted 2 lines

+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr

offset,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 111

Quoted 13 lines
hw/arm/tegra241-cmdqv.c +13 -0
+                                       uint64_t value)
+{
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }

To emulate the HW perfectly, VINTF_EN = 0 might have to disable the VCMDQ feature, i.e. deallocate HW_QUEUEs and munmap page0.

Quoted 5 lines

Otherwise, guest could continue using the vcmdq "hardware" after disabling it, which doesn't make sense.

This means that HW_QUEUE allocations would have to be delayed, until VINTF_EN=1 and corresponding VCMDQ_EN=1.

Ok. Will add that.

Quoted 11 lines

Review static void tegra241_cmdqv_write(void *opaque,

Quoted 1 line
(quoted diff) +0 -0
@@ -59,6 +129,36 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 147

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
Quoted 2 lines

We'd need to do something about this as well. VINTF/VCMDQ should not be functional when CMDQV_EN=0.

Ok.

Thanks,
Shameer

#42 lore raw

Re: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Quoted 21 lines · Shameer Kolothum Thodi
Quoted 21 lines · Shameer Kolothum
Quoted 1 line

+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,

hwaddr offset)

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9 -0
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ...

A_VINTF0_LVCMDQ_ERR_MAP_3:

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"

PRIx64 "\n",

Quoted 2 lines
(quoted diff) +2 -0
+                      __func__, offset);
+        return 0;
Quoted 6 lines
Quoted 3 lines

I wonder if we'd still need to implement VINTF0_SID_MATCH/REPLACE registers as it's a part of HW spec. They don't need to associate with anything but register cache alone to support MMIO.

Ok. But I was thinking Guest should not be allowed to configure it as it belongs to hypervisor, right?

That's correct for the physical registers. But I think guest could still MMIO the VM registers? So, what we'd need is just two arrays in the register cache list to support read/write.

Nicolin

#43 lore raw

PATCH v2 11/24 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:

-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.

VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.

---
 hw/arm/tegra241-cmdqv.c |  75 +++++++++++++++
 hw/arm/tegra241-cmdqv.h | 197 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 272 insertions(+)
hw/arm/tegra241-cmdqv.c +75 -0
index 49fca9d536..4d447718d0 100644
@@ -33,6 +33,46 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
     return true;
 }
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses are currently trapped. Use cached registers
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 {
     int i;
@@ -56,6 +96,7 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
     Error *local_err = NULL;
+    int index;
 
     if (!cmdqv->vintf_page0) {
         if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
@@ -88,6 +129,40 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
         return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN:
+        /*
+         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
+         * layout as the global VCMDQ registers, but are based at 0x30000
+         * instead of 0x10000.
+         *
+         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
+         * global VCMDQ offset, then fall through to reuse the common VCMDQ
+         * decoding logic below.
+         */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * VCMDQs are 128 identical instances, each occupying a 0x80-byte window
+         * starting at 0x10000. The MMIO offset is decoded to extract the VCMDQ
+         * index, and the per-instance offset is normalized to a VCMDQ0_*
+         * register (@offset0 = offset - 0x80 * index).
+         *
+         * A single helper then services all VCMDQs, with @index selecting the
+         * instance.
+         */
+        index = (offset - 0x10000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - 0x80 * index, index);
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H:
+        /* Same as A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN case above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H:
+        /* Same as A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN case above */
+        index = (offset - 0x20000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - 0x80 * index, index);
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
                       __func__, offset);
hw/arm/tegra241-cmdqv.h +197 -0

210 lines not on this page · raw message

Signature · 1 line
2.43.0
#44 lore raw

PATCH v2 12/24 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation.

Add write handling for global VCMDQ and VI_VCMDQ register windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.

VI_VCMDQ registers are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.

Writes are backed by cached register state only; no hardware queue mapping is performed yet.

---
 hw/arm/tegra241-cmdqv.c | 84 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 84 insertions(+)
hw/arm/tegra241-cmdqv.c +84 -0
index 4d447718d0..71f9a43bce 100644
@@ -170,6 +170,71 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void
+tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
+                           uint64_t value, unsigned size)
+{
+
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        } else {
+            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_BASE_H:
+        cmdqv->vcmdq_base[index] =
+            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        if (size == 8) {
+            cmdqv->vcmdq_cons_indx_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_cons_indx_base[index] =
+                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        cmdqv->vcmdq_cons_indx_base[index] =
+            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return;
+    }
+}
+
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
                                        uint64_t value)
 {
@@ -197,6 +262,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
     Error *local_err = NULL;
+    int index;
 
     if (!cmdqv->vintf_page0) {
         if (!tegra241_cmdqv_mmap_vintf_page0(cmdqv, &local_err)) {
@@ -230,6 +296,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         tegra241_cmdqv_write_vintf(cmdqv, offset, value);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN:
+        index = (offset - 0x10000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H:
+        index = (offset - 0x20000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
Signature · 1 line
2.43.0
#45 lore raw

PATCH v2 13/24 system/physmem: Add address_space_is_ram() helper

In reply to Shameer Kolothum

Introduce address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.

---
 include/system/memory.h | 10 ++++++++++
 system/physmem.c        | 11 +++++++++++
 2 files changed, 21 insertions(+)
include/system/memory.h +10 -0
index 0562af3136..02b2e83fd7 100644
@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,
  */
 bool address_space_is_io(AddressSpace *as, hwaddr addr);
 
+/**
+ * address_space_is_ram: check whether a guest physical address whithin
+ *                       an address space is RAM.
+ *
+ * @as: #AddressSpace to be accessed
+ * @addr: address within that address space
+ */
+
+bool address_space_is_ram(AddressSpace *as, hwaddr addr);
+
 /* address_space_map: map a physical memory region into a host virtual address
  *
  * May map a subset of the requested range, given by and returned in @plen.
system/physmem.c +11 -0
index 2fb0c25c93..ddcf921311 100644
@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)
     return !(memory_region_is_ram(mr) || memory_region_is_romd(mr));
 }
 
+bool address_space_is_ram(AddressSpace *as, hwaddr addr)
+{
+    MemoryRegion *mr;
+
+    RCU_READ_LOCK_GUARD();
+    mr = address_space_translate(as, addr, &addr, NULL, false,
+                                 MEMTXATTRS_UNSPECIFIED);
+
+    return memory_region_is_ram(mr);
+}
+
 static hwaddr
 flatview_extend_translation(FlatView *fv, hwaddr addr,
                             hwaddr target_len,
Signature · 1 line
2.43.0
Reviewed-by on v2 13/24
#46 lore raw

Re: [PATCH v2 13/24] system/physmem: Add address_space_is_ram() helper

Quoted 5 lines · Shameer Kolothum

Introduce address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.

Eric

Quoted 48 lines
---
 include/system/memory.h | 10 ++++++++++
 system/physmem.c        | 11 +++++++++++
 2 files changed, 21 insertions(+)
include/system/memory.h +10 -0
index 0562af3136..02b2e83fd7 100644
@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,
  */
 bool address_space_is_io(AddressSpace *as, hwaddr addr);
 
+/**
+ * address_space_is_ram: check whether a guest physical address whithin
+ *                       an address space is RAM.
+ *
+ * @as: #AddressSpace to be accessed
+ * @addr: address within that address space
+ */
+
+bool address_space_is_ram(AddressSpace *as, hwaddr addr);
+
 /* address_space_map: map a physical memory region into a host virtual address
  *
  * May map a subset of the requested range, given by and returned in @plen.
system/physmem.c +11 -0
index 2fb0c25c93..ddcf921311 100644
@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)
     return !(memory_region_is_ram(mr) || memory_region_is_romd(mr));
 }
 
+bool address_space_is_ram(AddressSpace *as, hwaddr addr)
+{
+    MemoryRegion *mr;
+
+    RCU_READ_LOCK_GUARD();
+    mr = address_space_translate(as, addr, &addr, NULL, false,
+                                 MEMTXATTRS_UNSPECIFIED);
+
+    return memory_region_is_ram(mr);
+}
+
 static hwaddr
 flatview_extend_translation(FlatView *fv, hwaddr addr,
                             hwaddr target_len,
#47 lore raw

PATCH v2 14/24 hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when VCMDQ base registers are programmed by the guest.

When a VCMDQ BASE register is written with a valid RAM-backed address, allocate a corresponding IOMMUFD hardware queue for the CMDQV device. Any previously allocated queue for the VCMDQ is freed before reallocation. Writes with invalid addresses (e.g. during reset) are ignored.

---
 hw/arm/tegra241-cmdqv.c | 51 ++++++++++++++++++++++++++++++++++++++---
 hw/arm/tegra241-cmdqv.h |  1 +
 2 files changed, 49 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.c +48 -3
index 71f9a43bce..57f47a4997 100644
@@ -170,6 +170,45 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc */
+    if (!address_space_is_ram(&address_space_memory, addr)) {
+        return true;
+    }
+
+    if (vcmdq) {
+        iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+        cmdqv->vcmdq[index] = NULL;
+        g_free(vcmdq);
+    }
+
+    if (!iommufd_backend_alloc_hw_queue(viommu->iommufd, viommu->viommu_id,
+                                        IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV,
+                                        index, addr, size, &hw_queue_id,
+                                        errp)) {
+        return false;
+    }
+    hw_queue = g_new(IOMMUFDHWqueue, 1);
+    hw_queue->hw_queue_id = hw_queue_id;
+    hw_queue->viommu = viommu;
+    cmdqv->vcmdq[index] = hw_queue;
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -178,7 +217,7 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
-                           uint64_t value, unsigned size)
+                           uint64_t value, unsigned size, Error **errp)
 {
 
     switch (offset0) {
@@ -207,11 +246,13 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                 (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
                 (value & 0xffffffffULL);
         }
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_BASE_H:
         cmdqv->vcmdq_base[index] =
             (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
             ((uint64_t)value << 32);
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
         if (size == 8) {
@@ -303,7 +344,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN:
         index = (offset - 0x10000) / 0x80;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
-                                   size);
+                                   size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H:
         /* Same decoding as read() case: See comments above */
@@ -312,12 +353,16 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H:
         index = (offset - 0x20000) / 0x80;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
-                                   size);
+                                   size, &local_err);
         break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +1 -0
index 1bc03c4f97..2f4a8ab35f 100644
@@ -31,6 +31,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     void *vintf_page0;
+    IOMMUFDHWqueue *vcmdq[128];
 
     /* Register Cache */
     uint32_t config;
Signature · 1 line
2.43.0
#48 lore raw

Re: [PATCH v2 14/24] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when VCMDQ base registers are programmed by the guest.

When a VCMDQ BASE register is written with a valid RAM-backed address, allocate a corresponding IOMMUFD hardware queue for the CMDQV device.

Let's mention that this is because QEMU doesn't trap VCMDQ_EN bit, which is a part of VCMDQ_CONFIG register accessed via the mmap'd page0.

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc */
+    if (!address_space_is_ram(&address_space_memory, addr)) {
+        return true;

We should probably check addr and size - 1 + addr?

Nicolin

#49 lore raw

RE: [PATCH v2 14/24] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 49 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 23:46
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 14/24] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when VCMDQ base registers are programmed by the guest.

When a VCMDQ BASE register is written with a valid RAM-backed address, allocate a corresponding IOMMUFD hardware queue for the CMDQV device.

Let's mention that this is because QEMU doesn't trap VCMDQ_EN bit, which is a part of VCMDQ_CONFIG register accessed via the mmap'd page0.

Quoted 1 line

+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int

index,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] &

R_VCMDQ0_BASE_L_LOG2SIZE_MASK;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9 -0
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc */
+    if (!address_space_is_ram(&address_space_memory, addr)) {
+        return true;
Quoted 1 line

We should probably check addr and size - 1 + addr?

Right. Will add.

Thanks,
Shameer

#50 lore raw

PATCH v2 15/24 hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

VINTF page0 is guest accessible and backed by host memory, so it can be mapped directly instead of being trapped and emulated. Map the VINTF0 page0 MMIO region lazily on first vCMDQ setup.

---
 hw/arm/tegra241-cmdqv.c | 21 +++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h |  2 ++
 2 files changed, 23 insertions(+)
hw/arm/tegra241-cmdqv.c +21 -0
index 57f47a4997..153fd70edb 100644
@@ -170,6 +170,26 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static void tegra241_cmdqv_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->vintf_page0_mapped) {
+        return;
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    memory_region_init_ram_device_ptr(&cmdqv->mmio_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_REG_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv, 0x30000,
+                                        &cmdqv->mmio_vintf_page0, 1);
+    g_free(name);
+    cmdqv->vintf_page0_mapped = true;
+}
+
 static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
                                        Error **errp)
 {
@@ -206,6 +226,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_map_vintf_page0(cmdqv);
     return true;
 }
 
hw/arm/tegra241-cmdqv.h +2 -0
index 2f4a8ab35f..b92c34e1d7 100644
@@ -31,6 +31,8 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     void *vintf_page0;
+    MemoryRegion mmio_vintf_page0;
+    bool vintf_page0_mapped;
     IOMMUFDHWqueue *vcmdq[128];
 
     /* Register Cache */
Signature · 1 line
2.43.0
#51 lore raw

Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 34

Quoted 9 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.h +1 -0
index 2f4a8ab35f..b92c34e1d7 100644
@@ -31,6 +31,8 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     void *vintf_page0;
+    MemoryRegion mmio_vintf_page0;

mr_vintf_page0

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+    bool vintf_page0_mapped;

Why do we need another boolean while having vintf_page0 pointer already?

Nicolin

#52 lore raw

RE: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

Quoted 29 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 23:50
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 34

Quoted 9 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.h +1 -0
index 2f4a8ab35f..b92c34e1d7 100644
@@ -31,6 +31,8 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     void *vintf_page0;
+    MemoryRegion mmio_vintf_page0;

mr_vintf_page0

Quoted 4 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+    bool vintf_page0_mapped;

Why do we need another boolean while having vintf_page0 pointer already?

My original idea for this flag was not to track the vintf_page0 mmap itself, but to indicate whether we have allocated a HW queue, since that is when vintf page0 is mapped to the guest.

Looking at this again, the flag may not be necessary. I will double check the flow and remove it.

Shameer
Thanks,
Shameer

#53 lore raw

Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

In reply to Shameer Kolothum
Quoted 6 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +6 -0
+static void tegra241_cmdqv_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->vintf_page0_mapped) {
+        return;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 188

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    memory_region_init_ram_device_ptr(&cmdqv->mmio_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_REG_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv, 0x30000,
+                                        &cmdqv->mmio_vintf_page0, 1);

I wonder if you see this:

[    0.853123] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for vcmdq0
[    0.854739] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for vcmdq1

iommufd_backend_alloc_vdev iommufd=142 dev_id=1 viommu_id=4 virt_id=0x200 vdev_id=8 (0) iommufd_cdev_attach_ioas_hwpt [iommufd=142] Successfully attached device 0009:01:00.0 (143) to id=6 smmuv3_accel_install_ste vSID=0x200 ste type=bypass hwpt_id=0x6
iommufd_viommu_alloc_eventq iommufd=142 viommu_id=4 type=1 veventq_id=9 veventq_fd=153 (0) iommufd_backend_alloc_hw_queue iommufd=142 viommu_id=4 queue_type=1 index=0 addr=0xe2000000 size=0x800000 queue_id=10 (0) vfio_listener_region_add_ram region_add [ram] 0xc0b0000 - 0xc0bffff [0xfffff7f00000] iommufd_backend_map_dma iommufd=142 ioas=2 iova=0xc0b0000 size=0x10000 addr=0xfffff7f00000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?
[Switching to Thread 0xffffdd7aee80 (LWP 52426)]

Thread 38 "qemu-system-aar" hit Breakpoint 2, vfio_container_region_add (bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at ../hw/vfio/listener.c:620
620 error_setg(&err, "vfio_container_dma_map(%p, 0x%"HWADDR_PRIx", "
(gdb) bt

#0  vfio_container_region_add (bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at ../hw/vfio/listener.c:620
#1  0x0000aaaaab8a9290 in vfio_listener_region_add (listener=0xaaaaaf1220d0, section=0xffffdd7ade00) at ../hw/vfio/listener.c:492
#2  0x0000aaaaab3035f8 in address_space_update_topology_pass (as=0xaaaaaefa6de0, old_view=0xffffec5171b0, new_view=0xffffc8001410, adding=true) at ../system/memory.c:1030
#3  0x0000aaaaab303974 in address_space_set_flatview (as=0xaaaaaefa6de0) at ../system/memory.c:1106
#4  0x0000aaaaab303bd4 in memory_region_transaction_commit () at ../system/memory.c:1158
#5  0x0000aaaaab3084e0 in memory_region_update_container_subregions (subregion=0xaaaaaefa7b00) at ../system/memory.c:2653
#6  0x0000aaaaab3085a0 in memory_region_add_subregion_common (mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00) at ../system/memory.c:2668
#7  0x0000aaaaab308644 in memory_region_add_subregion_overlap (mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00, priority=1) at ../system/memory.c:2685
#8  0x0000aaaaab570f68 in tegra241_cmdqv_map_vintf_page0 (cmdqv=0xaaaaaefa79c0) at ../hw/arm/tegra241-cmdqv.c:275
#9  0x0000aaaaab571188 in tegra241_cmdqv_setup_vcmdq (cmdqv=0xaaaaaefa79c0, index=0, errp=0xffffdd7ae0e8) at ../hw/arm/tegra241-cmdqv.c:317

#10 0x0000aaaaab571550 in tegra241_cmdqv_write_vcmdq (cmdqv=0xaaaaaefa79c0, offset0=131072, index=0, value=3791650835, size=4, errp=0xffffdd7ae0e8) at ../hw/arm/tegra241-cmdqv.c:378 #11 0x0000aaaaab571a60 in tegra241_cmdqv_write (opaque=0xaaaaaefa79c0, offset=131072, value=3791650835, size=4) at ../hw/arm/tegra241-cmdqv.c:484

Nicolin

#54 lore raw

RE: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

Quoted 40 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 13 February 2026 23:07
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region

Quoted 5 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +5 -0
+static void tegra241_cmdqv_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->vintf_page0_mapped) {

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        return;
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    memory_region_init_ram_device_ptr(&cmdqv->mmio_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_REG_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,

0x30000,

Quoted 1 line

+ &cmdqv->mmio_vintf_page0, 1);

I wonder if you see this:

[    0.853123] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries
for vcmdq0
[    0.854739] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries
Quoted 16 lines

for vcmdq1
iommufd_backend_alloc_vdev iommufd=142 dev_id=1 viommu_id=4 virt_id=0x200 vdev_id=8 (0)
iommufd_cdev_attach_ioas_hwpt [iommufd=142] Successfully attached device 0009:01:00.0 (143) to id=6
smmuv3_accel_install_ste vSID=0x200 ste type=bypass hwpt_id=0x6 iommufd_viommu_alloc_eventq iommufd=142 viommu_id=4 type=1 veventq_id=9 veventq_fd=153 (0) iommufd_backend_alloc_hw_queue iommufd=142 viommu_id=4 queue_type=1 index=0 addr=0xe2000000 size=0x800000 queue_id=10 (0) vfio_listener_region_add_ram region_add [ram] 0xc0b0000 - 0xc0bffff [0xfffff7f00000]
iommufd_backend_map_dma iommufd=142 ioas=2 iova=0xc0b0000 size=0x10000 addr=0xfffff7f00000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?

Yes. Eric has reported this earlier here:
https://lore.kernel.org/qemu-devel/CH3PR12MB7548771111FF5AC450CB8721AB8EA@CH3PR12MB7548.namprd12.prod.outlook.com/

IIUC, we are using memory_region_init_ram_device_ptr() to avoid trapping accesses to this MMIO space. However, this marks the region as RAM, and when it is added via memory_region_add_subregion_overlap(), the VFIO listener attempts to DMA MAP it. But, the QEMU/kernel does not provide
dma-buf support for this register space.

This region represents VINTF page0 registers and is accessed by the CPU only. My understanding is, there is no expected device DMA or P2P use case targeting it, so an IOMMU_IOAS_MAP for this region should not be required and can be ignored. Please let me know if that is not the case.

Thanks,
Shameer

Quoted 40 lines

[Switching to Thread 0xffffdd7aee80 (LWP 52426)]

Thread 38 "qemu-system-aar" hit Breakpoint 2, vfio_container_region_add (bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at ../hw/vfio/listener.c:620
620 error_setg(&err, "vfio_container_dma_map(%p,
0x%"HWADDR_PRIx", "
(gdb) bt
#0 vfio_container_region_add (bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at ../hw/vfio/listener.c:620 #1 0x0000aaaaab8a9290 in vfio_listener_region_add (listener=0xaaaaaf1220d0, section=0xffffdd7ade00) at ../hw/vfio/listener.c:492 #2 0x0000aaaaab3035f8 in address_space_update_topology_pass (as=0xaaaaaefa6de0, old_view=0xffffec5171b0, new_view=0xffffc8001410, adding=true) at ../system/memory.c:1030
#3 0x0000aaaaab303974 in address_space_set_flatview (as=0xaaaaaefa6de0) at ../system/memory.c:1106
#4 0x0000aaaaab303bd4 in memory_region_transaction_commit () at ../system/memory.c:1158
#5 0x0000aaaaab3084e0 in memory_region_update_container_subregions (subregion=0xaaaaaefa7b00) at ../system/memory.c:2653
#6 0x0000aaaaab3085a0 in memory_region_add_subregion_common (mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00) at ../system/memory.c:2668
#7 0x0000aaaaab308644 in memory_region_add_subregion_overlap (mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00, priority=1) at ../system/memory.c:2685
#8 0x0000aaaaab570f68 in tegra241_cmdqv_map_vintf_page0 (cmdqv=0xaaaaaefa79c0) at ../hw/arm/tegra241-cmdqv.c:275
#9 0x0000aaaaab571188 in tegra241_cmdqv_setup_vcmdq (cmdqv=0xaaaaaefa79c0, index=0, errp=0xffffdd7ae0e8) at ../hw/arm/tegra241-cmdqv.c:317
#10 0x0000aaaaab571550 in tegra241_cmdqv_write_vcmdq (cmdqv=0xaaaaaefa79c0, offset0=131072, index=0, value=3791650835, size=4, errp=0xffffdd7ae0e8) at ../hw/arm/tegra241-cmdqv.c:378 #11 0x0000aaaaab571a60 in tegra241_cmdqv_write (opaque=0xaaaaaefa79c0, offset=131072, value=3791650835, size=4) at ../hw/arm/tegra241-cmdqv.c:484

Nicolin

#55 lore raw

PATCH v2 16/24 hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers

In reply to Shameer Kolothum

When a VCMDQ is mapped and VINTF page0 is available, read and write VCMDQ registers directly via the VINTF page0 backing instead of using cached register values.

---
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 42 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +42 -2
index 153fd70edb..c7e70d8e1d 100644
@@ -38,12 +38,32 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
- *
- * All VCMDQ accesses are currently trapped. Use cached registers
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                           int index)
 {
+
+    /*
+     * If this VCMDQ is mapped and VINTF page0 is available, read directly
+     * from the VINTF page0 backing. Otherwise, fall back to cached state.
+     */
+    if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) {
+        uint64_t off = (index * 0x80) + (offset0 - 0x10000);
+        uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);
+
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -241,6 +261,26 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                            uint64_t value, unsigned size, Error **errp)
 {
 
+    /*
+     * If this VCMDQ is mapped and VINTF page0 is available, write directly
+     * to the VINTF page0 backing. Otherwise, update cached state.
+     */
+    if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) {
+        uint64_t off = (index * 0x80) + (offset0 - 0x10000);
+        uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);
+
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = value;
Signature · 1 line
2.43.0
#56 lore raw

Re: [PATCH v2 16/24] hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers

Hi Shameer,

Quoted 3 lines · Shameer Kolothum

When a VCMDQ is mapped and VINTF page0 is available, read and write VCMDQ registers directly via the VINTF page0 backing instead of using cached register values.

At this stage of the reading I do not really understand the dance between mmap and not mmapped. Please could you elaborate?

Quoted 11 lines
---
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 42 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +0 -0
index 153fd70edb..c7e70d8e1d 100644

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 42

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +0 -2
@@ -38,12 +38,32 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
- *
- * All VCMDQ accesses are currently trapped. Use cached registers
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,

s/offset0/offset?

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 52

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8 -0
                                           int index)
 {
+
+    /*
+     * If this VCMDQ is mapped and VINTF page0 is available, read directly
+     * from the VINTF page0 backing. Otherwise, fall back to cached state.
+     */
+    if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) {
+        uint64_t off = (index * 0x80) + (offset0 - 0x10000);
+        uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);

nit: you could introduce a static helper that return ptr from cmdqv, index, offset0 or returns NULL if the VINTF is not mapped. resuable in write().

Quoted 34 lines
hw/arm/tegra241-cmdqv.c +27 -0
+
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -241,6 +261,26 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                            uint64_t value, unsigned size, Error **errp)
 {
 
+    /*
+     * If this VCMDQ is mapped and VINTF page0 is available, write directly
+     * to the VINTF page0 backing. Otherwise, update cached state.
+     */
+    if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) {
+        uint64_t off = (index * 0x80) + (offset0 - 0x10000);
+        uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);
+
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:

hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, · line 286

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = value;

Thanks

Eric

#57 lore raw

RE: [PATCH v2 16/24] hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers

Quoted 21 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 13 February 2026 10:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 16/24] hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 7 lines
Quoted 3 lines · Shameer Kolothum

When a VCMDQ is mapped and VINTF page0 is available, read and write VCMDQ registers directly via the VINTF page0 backing instead of using cached register values.

At this stage of the reading I do not really understand the dance between mmap and not mmapped. Please could you elaborate?

That’s fair. It is a bit ambiguous, and I will improve it next time.

This patch is basically trying to address what Nicolin explained and suggested here:
https://lore.kernel.org/qemu-devel/aXkTzRM695oTWFaD@Asurada-Nvidia/

In short:
a) if vcmdq is indexed to a mapped global vcmdq, read/write mmio_vintf_page0.
b) if vcmdq is indexed to an unmapped global vcmdq, read/write the value in the register array cached by QEMU.

Quoted 20 lines
Quoted 3 lines
---
 hw/arm/tegra241-cmdqv.c | 44

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.c static bool

Quoted 7 lines
 1 file changed, 42 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +0 -0
index 153fd70edb..c7e70d8e1d 100644
@@ -38,12 +38,32 @@ static bool

tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +0 -2
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
- *
- * All VCMDQ accesses are currently trapped. Use cached registers
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,
Quoted 2 lines

hwaddr offset0,
s/offset0/offset?

Hmm.. I did this because in previous review there was some confusion w.r.t this. There was a suggestion to differentiate this normalized VCMDQ0_* offset from the top level MMIO offset used in tegra241_cmdqv_read()/write().

https://lore.kernel.org/qemu-devel/aXe8QgtdyBmXm5iS@Asurada-Nvidia/

Quoted 10 lines

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 52

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8 -0
                                           int index)
 {
+
+    /*
+     * If this VCMDQ is mapped and VINTF page0 is available, read directly
+     * from the VINTF page0 backing. Otherwise, fall back to cached state.
+     */
+    if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) {
+        uint64_t off = (index * 0x80) + (offset0 - 0x10000);
+        uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);
Quoted 3 lines

nit: you could introduce a static helper that return ptr from cmdqv, index, offset0 or returns NULL if the VINTF is not mapped. resuable in write().

Ok.

Thanks,
Shameer

#58 lore raw

PATCH v2 17/24 hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

In reply to Shameer Kolothum

Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.

---
 hw/arm/tegra241-cmdqv.c | 39 +++++++++++++++++++++++++++++++++++++--
 hw/arm/tegra241-cmdqv.h |  1 +
 2 files changed, 38 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +37 -2
index c7e70d8e1d..9e0e07e85e 100644
@@ -428,12 +428,47 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+
+    if (!veventq) {
+        return;
+    }
+    close(veventq->veventq_fd);
+    iommufd_backend_free_id(accel->viommu->iommufd, veventq->veventq_id);
+    g_free(veventq);
+    cmdqv->veventq = NULL;
 }
 
 static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+    IOMMUFDVeventq *veventq;
+    uint32_t veventq_id;
+    uint32_t veventq_fd;
+
+    if (cmdqv->veventq) {
+        return true;
+    }
+
+    if (!iommufd_backend_alloc_veventq(viommu->iommufd, viommu->viommu_id,
+                                       IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                       1 << 16, &veventq_id, &veventq_fd,
+                                       errp)) {
+        error_append_hint(errp, "Tegra241 CMDQV: failed to alloc veventq");
+        return false;
+    }
+
+    veventq = g_new(IOMMUFDVeventq, 1);
+    veventq->veventq_id = veventq_id;
+    veventq->veventq_fd = veventq_fd;
+    veventq->viommu = viommu;
+    cmdqv->veventq = veventq;
+
+    return true;
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +1 -0
index b92c34e1d7..a3a1621e3a 100644
@@ -34,6 +34,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_vintf_page0;
     bool vintf_page0_mapped;
     IOMMUFDHWqueue *vcmdq[128];
+    IOMMUFDVeventq *veventq;
 
     /* Register Cache */
     uint32_t config;
Signature · 1 line
2.43.0
#60 lore raw

PATCH v2 18/24 hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

In reply to Shameer Kolothum

Install an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().

---
 hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h |  2 +
 hw/arm/trace-events     |  3 ++
 3 files changed, 88 insertions(+)
hw/arm/tegra241-cmdqv.c +83 -0
index 9e0e07e85e..3dc5315677 100644
@@ -8,12 +8,76 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "qemu/log.h"
+#include "trace.h"
 
 #include "hw/arm/smmuv3.h"
+#include "hw/core/irq.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static void tegra241_cmdqv_event_read(void *opaque)
+{
+    Tegra241CMDQV *cmdqv = opaque;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_tegra241_cmdqv vevent;
+    } buf;
+    uint32_t last_seq = cmdqv->last_event_seq;
+    ssize_t bytes;
+
+    bytes = read(cmdqv->veventq->veventq_fd, &buf, sizeof(buf));
+    if (bytes <= 0) {
+        if (errno == EAGAIN || errno == EINTR) {
+            return;
+        }
+        error_report_once("Tegra241 CMDQV: vEVENTQ: read failed (%m)");
+        return;
+    }
+
+    if (bytes == sizeof(buf.hdr) &&
+        (buf.hdr.flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
+        error_report_once("Tegra241 CMDQV: vEVENTQ has lost events");
+        return;
+    }
+
+    if (bytes < sizeof(buf)) {
+        error_report_once("Tegra241 `CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)",
+                          bytes, sizeof(buf));
+        cmdqv->event_start = false;
+        return;
+    }
+
+    /* Check sequence in hdr for lost events if any */
+    if (cmdqv->event_start && (buf.hdr.sequence - last_seq != 1)) {
+        error_report_once("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)",
+                          buf.hdr.sequence - last_seq - 1);
+    }
+
+    if (buf.vevent.lvcmdq_err_map[0] || buf.vevent.lvcmdq_err_map[1]) {
+        cmdqv->vintf_cmdq_err_map[0] =
+            buf.vevent.lvcmdq_err_map[0] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[1] =
+            (buf.vevent.lvcmdq_err_map[0] >> 32) & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[2] =
+            buf.vevent.lvcmdq_err_map[1] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[3] =
+            (buf.vevent.lvcmdq_err_map[1] >> 32) & 0xffffffff;
+        for (int i = 0; i < 4; i++) {
+            cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i];
+        }
+        cmdqv->vi_err_map[0] |= 0x1;
+        qemu_irq_pulse(cmdqv->irq);
+        trace_tegra241_cmdqv_err_map(
+        cmdqv->vintf_cmdq_err_map[3], cmdqv->vintf_cmdq_err_map[2],
+        cmdqv->vintf_cmdq_err_map[1], cmdqv->vintf_cmdq_err_map[0]);
+    }
+
+    cmdqv->last_event_seq = buf.hdr.sequence;
+    cmdqv->event_start = true;
+}
+
 static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
 {
     IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
@@ -435,6 +499,7 @@ static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
     if (!veventq) {
         return;
     }
+    qemu_set_fd_handler(veventq->veventq_fd, NULL, NULL, NULL);
     close(veventq->veventq_fd);
     iommufd_backend_free_id(accel->viommu->iommufd, veventq->veventq_id);
     g_free(veventq);
@@ -449,6 +514,7 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
     IOMMUFDVeventq *veventq;
     uint32_t veventq_id;
     uint32_t veventq_fd;
+    int flags;
 
     if (cmdqv->veventq) {
         return true;
@@ -462,13 +528,30 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
         return false;
     }
 
+    flags = fcntl(veventq_fd, F_GETFL);
+    if (flags < 0) {
+        error_setg(errp, "Failed to get flags for vEVENTQ fd");
+        goto free_veventq;
+    }
+    if (fcntl(veventq_fd, F_SETFL, O_NONBLOCK | flags) < 0) {
+        error_setg(errp, "Failed to set O_NONBLOCK on vEVENTQ fd");
+        goto free_veventq;
+    }
+
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     veventq->viommu = viommu;
     cmdqv->veventq = veventq;
 
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL, s);
     return true;
+
+free_veventq:
+    close(veventq_fd);
+    iommufd_backend_free_id(viommu->iommufd, veventq_id);
+    return false;
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +2 -0
index a3a1621e3a..fddb3ac6e9 100644
@@ -35,6 +35,8 @@ typedef struct Tegra241CMDQV {
     bool vintf_page0_mapped;
     IOMMUFDHWqueue *vcmdq[128];
     IOMMUFDVeventq *veventq;
+    uint32_t last_event_seq;
+    bool event_start;
 
     /* Register Cache */
     uint32_t config;
hw/arm/trace-events +3 -0
index 3457536fb0..76bda0efef 100644
@@ -72,6 +72,9 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev
 smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64
 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x"
 
+# tegra241-cmdqv
+tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
 strongarm_ssp_read_underrun(void) "SSP rx underrun"
Signature · 1 line
2.43.0
#61 lore raw

Re: [PATCH v2 18/24] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

Quoted 2 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +2 -0
+static void tegra241_cmdqv_event_read(void *opaque)
+{

[...]

Quoted 17 lines
hw/arm/tegra241-cmdqv.c +17 -0
+    bytes = read(cmdqv->veventq->veventq_fd, &buf, sizeof(buf));
+    if (bytes <= 0) {
+        if (errno == EAGAIN || errno == EINTR) {
+            return;
+        }
+        error_report_once("Tegra241 CMDQV: vEVENTQ: read failed (%m)");
+        return;
+    }
+
+    if (bytes == sizeof(buf.hdr) &&
+        (buf.hdr.flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
+        error_report_once("Tegra241 CMDQV: vEVENTQ has lost events");
+        return;
+    }
+
+    if (bytes < sizeof(buf)) {
+        error_report_once("Tegra241 `CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)",

hw/arm/tegra241-cmdqv.c line 56

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+                          bytes, sizeof(buf));
+        cmdqv->event_start = false;
+        return;
+    }
+
+    /* Check sequence in hdr for lost events if any */
+    if (cmdqv->event_start && (buf.hdr.sequence - last_seq != 1)) {
+        error_report_once("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)",
+                          buf.hdr.sequence - last_seq - 1);
+    }

It shares the exact same routine with smmuv3_accel_event_read(). Can we put these into a common helper?

I also suggested in the veventq series that event_start/last_seq could be moved into the common structure.

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp) · line 548

Quoted 8 lines
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     veventq->viommu = viommu;
     cmdqv->veventq = veventq;
hw/arm/tegra241-cmdqv.c +2 -0
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL, s);

Should pass in "cmdqv" instead of "s".

Nicolin

#62 lore raw

RE: [PATCH v2 18/24] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

Quoted 46 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 13 February 2026 23:29
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 18/24] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

Quoted 2 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +2 -0
+static void tegra241_cmdqv_event_read(void *opaque)
+{

[...]

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+    bytes = read(cmdqv->veventq->veventq_fd, &buf, sizeof(buf));
+    if (bytes <= 0) {
+        if (errno == EAGAIN || errno == EINTR) {
+            return;
+        }
+        error_report_once("Tegra241 CMDQV: vEVENTQ: read failed (%m)");
+        return;

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    }
+
+    if (bytes == sizeof(buf.hdr) &&
+        (buf.hdr.flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
+        error_report_once("Tegra241 CMDQV: vEVENTQ has lost events");
+        return;
+    }
+
+    if (bytes < sizeof(buf)) {
+        error_report_once("Tegra241 `CMDQV: vEVENTQ: incomplete read

(%zd/%zd bytes)",

hw/arm/tegra241-cmdqv.c

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+                          bytes, sizeof(buf));
+        cmdqv->event_start = false;
+        return;
+    }
+
+    /* Check sequence in hdr for lost events if any */
+    if (cmdqv->event_start && (buf.hdr.sequence - last_seq != 1)) {
+        error_report_once("Tegra241 CMDQV: vEVENTQ: detected lost %u
Quoted 6 lines

event(s)",

Review

Quoted 2 lines
(quoted diff) +2 -0
+                          buf.hdr.sequence - last_seq - 1);
+    }

It shares the exact same routine with smmuv3_accel_event_read(). Can we put these into a common helper?

Yes. I think you had the same comment in RFC as well. I have noted that as a TODO in the cover letter for this. Will do.

Quoted 2 lines

I also suggested in the veventq series that event_start/last_seq could be moved into the common structure.

Yes. I see that.

Quoted 11 lines

Review

Quoted 8 lines
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     veventq->viommu = viommu;
     cmdqv->veventq = veventq;
(quoted diff) +2 -0
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL,

s);

Should pass in "cmdqv" instead of "s".

Ah..Right.

Thanks,
Shameer

#63 lore raw

PATCH v2 19/24 hw/arm/tegra241-cmdqv: Add reset handler

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

CMDQV gets initialized early during guest boot, hence the handler verifies that at least one cold-plugged device is attached to the associated vIOMMU before proceeding. This is required to retrieve host CMDQV info and to validate it against the QEMU implementation support.

---
 hw/arm/smmuv3-accel.c   |   6 ++-
 hw/arm/tegra241-cmdqv.c | 107 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.h |   6 +++
 hw/arm/trace-events     |   1 +
 4 files changed, 119 insertions(+), 1 deletion(-)
hw/arm/smmuv3-accel.c +5 -1
index b1a8ab79b5..96224a7632 100644
@@ -707,7 +707,11 @@ static void smmuv3_accel_unset_iommu_device(PCIBus *bus, void *opaque,
     QLIST_REMOVE(accel_dev, next);
     trace_smmuv3_accel_unset_iommu_device(devfn, idev->devid);
 
-    if (QLIST_EMPTY(&accel->device_list)) {
+   /*
+    * Keep the vIOMMU alive when CMDQV is present, as the vIOMMU to host
+    * SMMUv3 association cannot be changed via device hot-plug.
+    */
+    if (QLIST_EMPTY(&accel->device_list) && !accel->cmdqv) {
         smmuv3_accel_free_viommu(accel);
     }
 }
hw/arm/tegra241-cmdqv.c +107 -0

115 lines not on this page · raw message

hw/arm/tegra241-cmdqv.h +6 -0
index fddb3ac6e9..2aad697170 100644
@@ -25,6 +25,10 @@
  */
 #define TEGRA241_CMDQV_IO_LEN 0x50000
 
+#define TEGRA241_CMDQV_VERSION 0x1
+#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 0x1
+#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 0x4
+
 typedef struct Tegra241CMDQV {
     struct iommu_viommu_tegra241_cmdqv cmdqv_data;
     SMMUv3AccelState *s_accel;
@@ -67,6 +71,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/trace-events +1 -0
index 76bda0efef..ef495c040c 100644
@@ -74,6 +74,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 
 # tegra241-cmdqv
 tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
#64 lore raw

PATCH v2 20/24 hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

CMDQV HW reads guest queue memory in its host physical address setup via IOMUUFD. This requires the guest queue memory isn't only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQV size that the host can safely back with physically contiguous memory. Allowing a CMDQV larger than the host page size could cause the hardware to DMA across page boundaries leading to faults.

Limit IDR1.CMDQS so the guest cannot configure a CMDQV that exceeds the host’s contiguous backing.

---
 hw/arm/tegra241-cmdqv.c | 43 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
hw/arm/tegra241-cmdqv.c +43 -0
index 678e53d23e..19fcc5b68d 100644
@@ -11,10 +11,14 @@
 #include "qemu/error-report.h"
 #include "qemu/log.h"
 #include "trace.h"
+#include <math.h>
 
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 
 static void tegra241_cmdqv_event_read(void *opaque)
@@ -582,6 +586,33 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     return true;
 }
 
+static size_t tegra241_cmdqv_min_ram_pagesize(void)
+{
+    RAMBlock *rb;
+    size_t pg, min_pg = SIZE_MAX;
+
+    RAMBLOCK_FOREACH(rb) {
+        MemoryRegion *mr = rb->mr;
+
+        /* Only consider real RAM regions */
+        if (!mr || !memory_region_is_ram(mr)) {
+            continue;
+        }
+
+        /* Skip RAM regions that are not backed by a memory-backend */
+        if (!object_dynamic_cast(mr->owner, TYPE_MEMORY_BACKEND)) {
+            continue;
+        }
+
+        pg = qemu_ram_pagesize(rb);
+        if (pg && pg < min_pg) {
+            min_pg = pg;
+        }
+    }
+
+    return (min_pg == SIZE_MAX) ? qemu_real_host_page_size() : min_pg;
+}
+
 static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
 {
     SMMUv3AccelState *s_accel = s->s_accel;
@@ -589,7 +620,9 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
     struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
     SMMUv3AccelDevice *accel_dev;
     Error *local_err = NULL;
+    size_t pgsize;
     uint64_t caps;
+    uint32_t val;
     int i;
 
     if (QLIST_EMPTY(&s_accel->device_list)) {
@@ -660,6 +693,16 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
         cmdqv->vcmdq_base[i] = 0;
         cmdqv->vcmdq_cons_indx_base[i] = 0;
     }
+
+   /*
+    * CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the
+    * queue fits entirely within the smallest backend page size, ensuring
+    * the command queue is physically contiguous in host memory.
+    */
+    pgsize = tegra241_cmdqv_min_ram_pagesize();
+    val = FIELD_EX32(s->idr[1], IDR1, CMDQS);
+    s->idr[1] = FIELD_DP32(s->idr[1], IDR1, CMDQS, MIN(log2(pgsize) - 4, val));
+
     return;
 
 out_err:
Signature · 1 line
2.43.0
#65 lore raw

PATCH v2 21/24 virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev

In reply to Shameer Kolothum

Rename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.

No functional changes intended.

---
 hw/arm/virt-acpi-build.c | 44 ++++++++++++++++++++++------------------
 1 file changed, 24 insertions(+), 20 deletions(-)
hw/arm/virt-acpi-build.c +24 -20
index c145678185..ae3b4aac52 100644
@@ -339,24 +339,28 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
     return idmap_a->input_base - idmap_b->input_base;
 }
 
-typedef struct AcpiIortSMMUv3Dev {
+typedef struct AcpiSMMUv3Dev {
     int irq;
     hwaddr base;
+
+    /*
+     * IORT-only fields.
+     * These are used when building IORT SMMUv3 nodes.
+     */
     GArray *rc_smmu_idmaps;
-    /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
-    size_t offset;
+    size_t offset; /* Offset of the SMMUv3 node within the IORT table */
     bool accel;
     bool ats;
-} AcpiIortSMMUv3Dev;
+} AcpiSMMUv3Dev;
 
 /*
- * Populate the struct AcpiIortSMMUv3Dev for the legacy SMMUv3 and
+ * Populate the struct AcpiSMMUv3Dev for the legacy SMMUv3 and
  * return the total number of associated idmaps.
  */
 static int populate_smmuv3_legacy_dev(GArray *sdev_blob)
 {
     VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
+    AcpiSMMUv3Dev sdev = {0};
 
     sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
     object_child_foreach_recursive(object_get_root(), iort_host_bridges,
@@ -376,8 +380,8 @@ static int populate_smmuv3_legacy_dev(GArray *sdev_blob)
 
 static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
 {
-    AcpiIortSMMUv3Dev *sdev_a = (AcpiIortSMMUv3Dev *)a;
-    AcpiIortSMMUv3Dev *sdev_b = (AcpiIortSMMUv3Dev *)b;
+    AcpiSMMUv3Dev *sdev_a = (AcpiSMMUv3Dev *)a;
+    AcpiSMMUv3Dev *sdev_b = (AcpiSMMUv3Dev *)b;
     AcpiIortIdMapping *map_a = &g_array_index(sdev_a->rc_smmu_idmaps,
                                               AcpiIortIdMapping, 0);
     AcpiIortIdMapping *map_b = &g_array_index(sdev_b->rc_smmu_idmaps,
@@ -388,7 +392,7 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
 static int iort_smmuv3_devices(Object *obj, void *opaque)
 {
     VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
+    AcpiSMMUv3Dev sdev = {0};
     GArray *sdev_blob = opaque;
     AcpiIortIdMapping idmap;
     PlatformBusDevice *pbus;
@@ -421,7 +425,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)
 }
 
 /*
- * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and
+ * Populate the struct AcpiSMMUv3Dev for all SMMUv3 devices and
  * return the total number of idmaps.
  */
 static int populate_smmuv3_dev(GArray *sdev_blob)
@@ -442,10 +446,10 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
 {
     AcpiIortIdMapping *idmap;
     AcpiIortIdMapping next_range = {0};
-    AcpiIortSMMUv3Dev *sdev;
+    AcpiSMMUv3Dev *sdev;
 
     for (int i = 0; i < smmuv3_devs->len; i++) {
-        sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
         /*
          * Based on the RID ranges that are directed to the SMMU, determine the
          * bypassed RID ranges, i.e., the ones that are directed to the ITS
@@ -479,7 +483,7 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
 static void
 build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)
 {
-    AcpiIortSMMUv3Dev *sdev;
+    AcpiSMMUv3Dev *sdev;
     AcpiIortIdMapping *idmap;
     int i;
 
@@ -487,7 +491,7 @@ build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)
         uint16_t rmr_len;
         int bdf;
 
-        sdev = &g_array_index(smmuv3_devices, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devices, AcpiSMMUv3Dev, i);
         if (!sdev->accel) {
             continue;
         }
@@ -544,13 +548,13 @@ static void
 build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 {
     int i, nb_nodes, rc_mapping_count;
-    AcpiIortSMMUv3Dev *sdev;
+    AcpiSMMUv3Dev *sdev;
     size_t node_size;
     bool ats_needed = false;
     int num_smmus = 0;
     uint32_t id = 0;
     int rc_smmu_idmaps_len = 0;
-    GArray *smmuv3_devs = g_array_new(false, true, sizeof(AcpiIortSMMUv3Dev));
+    GArray *smmuv3_devs = g_array_new(false, true, sizeof(AcpiSMMUv3Dev));
     GArray *rc_its_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
 
     AcpiTable table = { .sig = "IORT", .rev = 5, .oem_id = vms->oem_id,
@@ -581,7 +585,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         }
         /* Calculate RMR nodes required. One per SMMUv3 with accelerated mode */
         for (i = 0; i < num_smmus; i++) {
-            sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+            sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
             if (sdev->ats) {
                 ats_needed = true;
             }
@@ -620,7 +624,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     }
 
     for (i = 0; i < num_smmus; i++) {
-        sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
         int smmu_mapping_count, offset_to_id_array;
         int irq = sdev->irq;
 
@@ -699,7 +703,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         AcpiIortIdMapping *range;
 
         for (i = 0; i < num_smmus; i++) {
-            sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+            sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
 
             /*
              * Map RIDs (input) from RC to SMMUv3 nodes: RC -> SMMUv3.
@@ -742,7 +746,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_table_end(linker, &table);
     g_array_free(rc_its_idmaps, true);
     for (i = 0; i < num_smmus; i++) {
-        sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
         g_array_free(sdev->rc_smmu_idmaps, true);
     }
     g_array_free(smmuv3_devs, true);
Signature · 1 line
2.43.0
#66 lore raw

Re: [PATCH v2 21/24] virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev

Hi Shameer,

Quoted 25 lines · Shameer Kolothum

Rename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.

No functional changes intended.

---
 hw/arm/virt-acpi-build.c | 44 ++++++++++++++++++++++------------------
 1 file changed, 24 insertions(+), 20 deletions(-)
hw/arm/virt-acpi-build.c +2 -1
index c145678185..ae3b4aac52 100644
@@ -339,24 +339,28 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
     return idmap_a->input_base - idmap_b->input_base;
 }
 
-typedef struct AcpiIortSMMUv3Dev {
+typedef struct AcpiSMMUv3Dev {
     int irq;
     hwaddr base;
+

hw/arm/virt-acpi-build.c static int iort_idmap_compare(gconstpointer a, gconstpointer b) · line 353

Quoted 10 lines
hw/arm/virt-acpi-build.c +5 -2
+    /*
+     * IORT-only fields.
+     * These are used when building IORT SMMUv3 nodes.
+     */
     GArray *rc_smmu_idmaps;
-    /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
-    size_t offset;
+    size_t offset; /* Offset of the SMMUv3 node within the IORT table */
     bool accel;
     bool ats;

accel and ats seem to be only used by IORT too. So it is a bit weird to have a so-called generic datatype which is half iort specific. Why don't use keepAcpiIortSMMUv3Dev and flat encapsulate another object inside with irq and base and id? Eric

Quoted 128 lines
hw/arm/virt-acpi-build.c +17 -17
-} AcpiIortSMMUv3Dev;
+} AcpiSMMUv3Dev;
 
 /*
- * Populate the struct AcpiIortSMMUv3Dev for the legacy SMMUv3 and
+ * Populate the struct AcpiSMMUv3Dev for the legacy SMMUv3 and
  * return the total number of associated idmaps.
  */
 static int populate_smmuv3_legacy_dev(GArray *sdev_blob)
 {
     VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
+    AcpiSMMUv3Dev sdev = {0};
 
     sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
     object_child_foreach_recursive(object_get_root(), iort_host_bridges,
@@ -376,8 +380,8 @@ static int populate_smmuv3_legacy_dev(GArray *sdev_blob)
 
 static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
 {
-    AcpiIortSMMUv3Dev *sdev_a = (AcpiIortSMMUv3Dev *)a;
-    AcpiIortSMMUv3Dev *sdev_b = (AcpiIortSMMUv3Dev *)b;
+    AcpiSMMUv3Dev *sdev_a = (AcpiSMMUv3Dev *)a;
+    AcpiSMMUv3Dev *sdev_b = (AcpiSMMUv3Dev *)b;
     AcpiIortIdMapping *map_a = &g_array_index(sdev_a->rc_smmu_idmaps,
                                               AcpiIortIdMapping, 0);
     AcpiIortIdMapping *map_b = &g_array_index(sdev_b->rc_smmu_idmaps,
@@ -388,7 +392,7 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
 static int iort_smmuv3_devices(Object *obj, void *opaque)
 {
     VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
+    AcpiSMMUv3Dev sdev = {0};
     GArray *sdev_blob = opaque;
     AcpiIortIdMapping idmap;
     PlatformBusDevice *pbus;
@@ -421,7 +425,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)
 }
 
 /*
- * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and
+ * Populate the struct AcpiSMMUv3Dev for all SMMUv3 devices and
  * return the total number of idmaps.
  */
 static int populate_smmuv3_dev(GArray *sdev_blob)
@@ -442,10 +446,10 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
 {
     AcpiIortIdMapping *idmap;
     AcpiIortIdMapping next_range = {0};
-    AcpiIortSMMUv3Dev *sdev;
+    AcpiSMMUv3Dev *sdev;
 
     for (int i = 0; i < smmuv3_devs->len; i++) {
-        sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
         /*
          * Based on the RID ranges that are directed to the SMMU, determine the
          * bypassed RID ranges, i.e., the ones that are directed to the ITS
@@ -479,7 +483,7 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)
 static void
 build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)
 {
-    AcpiIortSMMUv3Dev *sdev;
+    AcpiSMMUv3Dev *sdev;
     AcpiIortIdMapping *idmap;
     int i;
 
@@ -487,7 +491,7 @@ build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)
         uint16_t rmr_len;
         int bdf;
 
-        sdev = &g_array_index(smmuv3_devices, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devices, AcpiSMMUv3Dev, i);
         if (!sdev->accel) {
             continue;
         }
@@ -544,13 +548,13 @@ static void
 build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
 {
     int i, nb_nodes, rc_mapping_count;
-    AcpiIortSMMUv3Dev *sdev;
+    AcpiSMMUv3Dev *sdev;
     size_t node_size;
     bool ats_needed = false;
     int num_smmus = 0;
     uint32_t id = 0;
     int rc_smmu_idmaps_len = 0;
-    GArray *smmuv3_devs = g_array_new(false, true, sizeof(AcpiIortSMMUv3Dev));
+    GArray *smmuv3_devs = g_array_new(false, true, sizeof(AcpiSMMUv3Dev));
     GArray *rc_its_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
 
     AcpiTable table = { .sig = "IORT", .rev = 5, .oem_id = vms->oem_id,
@@ -581,7 +585,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         }
         /* Calculate RMR nodes required. One per SMMUv3 with accelerated mode */
         for (i = 0; i < num_smmus; i++) {
-            sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+            sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
             if (sdev->ats) {
                 ats_needed = true;
             }
@@ -620,7 +624,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     }
 
     for (i = 0; i < num_smmus; i++) {
-        sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
         int smmu_mapping_count, offset_to_id_array;
         int irq = sdev->irq;
 
@@ -699,7 +703,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
         AcpiIortIdMapping *range;
 
         for (i = 0; i < num_smmus; i++) {
-            sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+            sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
 
             /*
              * Map RIDs (input) from RC to SMMUv3 nodes: RC -> SMMUv3.
@@ -742,7 +746,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_table_end(linker, &table);
     g_array_free(rc_its_idmaps, true);
     for (i = 0; i < num_smmus; i++) {
-        sdev = &g_array_index(smmuv3_devs, AcpiIortSMMUv3Dev, i);
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
         g_array_free(sdev->rc_smmu_idmaps, true);
     }
     g_array_free(smmuv3_devs, true);
#67 lore raw

PATCH v2 22/24 hw/arm/smmuv3: Add per-device identifier poroperty

In reply to Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

---
 hw/arm/smmuv3.c          | 1 +
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 include/hw/arm/smmuv3.h  | 1 +
 4 files changed, 8 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -0
index 7858bf2c33..7f84c87a46 100644
@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
index ae3b4aac52..046e930ca5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
 
     /*
      * IORT-only fields.
@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)
     bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
     sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
     sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+    sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
     pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
     sbdev = SYS_BUS_DEVICE(obj);
     sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +3 -0
index 390845c503..22ee5c4a41 100644
@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                        (vms->legacy_smmuv3_present) ?
                        "iommu=smmuv3" : "virtio-iommu");
         } else if (vms->iommu == VIRT_IOMMU_NONE) {
+            static uint8_t id;
+
             /* The new SMMUv3 device is specific to the PCI bus */
             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true, NULL);
             object_property_set_link(OBJECT(dev), "memory",
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", id++, NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
include/hw/arm/smmuv3.h +1 -0
index 87926f8cb3..39cb43506e 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
Signature · 1 line
2.43.0
Reviewed-by on v2 22/24
#68 lore raw

Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

Quoted 10 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

With nits:

hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2117

Quoted 9 lines
hw/arm/smmuv3.c +1 -0
index 7858bf2c33..7f84c87a46 100644
@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),

Perhaps add a line of comments for where we use it.

hw/arm/virt.c static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 3062

Quoted 9 lines
hw/arm/virt.c +1 -0
index 390845c503..22ee5c4a41 100644
@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                        (vms->legacy_smmuv3_present) ?
                        "iommu=smmuv3" : "virtio-iommu");
         } else if (vms->iommu == VIRT_IOMMU_NONE) {
+            static uint8_t id;

Local static feels odd. Perhaps make it global?

#69 lore raw

Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

In reply to Shameer Kolothum

Hi,

Quoted 2 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

typo in the title.

I don't understand what does mandate that change? We already have an identifier set in the IORT table. The rationale behind that patch needs to be clarified

Thanks

Eric

Quoted 88 lines

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

---
 hw/arm/smmuv3.c          | 1 +
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 include/hw/arm/smmuv3.h  | 1 +
 4 files changed, 8 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -0
index 7858bf2c33..7f84c87a46 100644
@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
index ae3b4aac52..046e930ca5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
 
     /*
      * IORT-only fields.
@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)
     bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
     sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
     sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+    sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
     pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
     sbdev = SYS_BUS_DEVICE(obj);
     sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +3 -0
index 390845c503..22ee5c4a41 100644
@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                        (vms->legacy_smmuv3_present) ?
                        "iommu=smmuv3" : "virtio-iommu");
         } else if (vms->iommu == VIRT_IOMMU_NONE) {
+            static uint8_t id;
+
             /* The new SMMUv3 device is specific to the PCI bus */
             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true, NULL);
             object_property_set_link(OBJECT(dev), "memory",
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", id++, NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
include/hw/arm/smmuv3.h +1 -0
index 87926f8cb3..39cb43506e 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
#70 lore raw

Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

In reply to Shameer Kolothum
Quoted 8 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

I think an important info which is missing here is you need the id used in IORT and DSDT to be the same, is that correct

By the way why can't you reuse the QOM id. I think Dan said there is one for all QOM objects.

Thanks

Eric

Quoted 82 lines
---
 hw/arm/smmuv3.c          | 1 +
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 include/hw/arm/smmuv3.h  | 1 +
 4 files changed, 8 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -0
index 7858bf2c33..7f84c87a46 100644
@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
index ae3b4aac52..046e930ca5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
 
     /*
      * IORT-only fields.
@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)
     bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
     sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
     sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+    sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
     pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
     sbdev = SYS_BUS_DEVICE(obj);
     sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +3 -0
index 390845c503..22ee5c4a41 100644
@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                        (vms->legacy_smmuv3_present) ?
                        "iommu=smmuv3" : "virtio-iommu");
         } else if (vms->iommu == VIRT_IOMMU_NONE) {
+            static uint8_t id;
+
             /* The new SMMUv3 device is specific to the PCI bus */
             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true, NULL);
             object_property_set_link(OBJECT(dev), "memory",
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", id++, NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
include/hw/arm/smmuv3.h +1 -0
index 87926f8cb3..39cb43506e 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
#71 lore raw

RE: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

Quoted 23 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 18:30
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

External email: Use caution opening links or attachments

Quoted 3 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

Quoted 7 lines
Quoted 5 lines

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

I think an important info which is missing here is you need the id used in IORT and DSDT to be the same, is that correct

Yes. That is correct. I think I have that explanation in next patch. I will mention it here as well.

Quoted 3 lines

By the way why can't you reuse the QOM id. I think Dan said there is one for all QOM objects.

Yes. QOM device has a global id. But I am not sure we can use that here. For one it is a "char *" and normally user or management layers will use That,
eg:
-device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1-pxb1,

Thanks,
Shameer

#72 lore raw

Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

Quoted 31 lines · Shameer Kolothum Thodi
Quoted 22 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 18:30
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

External email: Use caution opening links or attachments

Quoted 3 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

Quoted 7 lines
Quoted 5 lines

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

I think an important info which is missing here is you need the id used in IORT and DSDT to be the same, is that correct

Yes. That is correct. I think I have that explanation in next patch. I will mention it here as well.

Quoted 7 lines
Quoted 2 lines

By the way why can't you reuse the QOM id. I think Dan said there is one for all QOM objects.

Yes. QOM device has a global id. But I am not sure we can use that here. For one it is a "char *" and normally user or management layers will use That,
eg:
-device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1-pxb1,

oh yes you're correct. Forget about it

Eric

Quoted 4 lines

Thanks,
Shameer

#73 lore raw

Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

In reply to Shameer Kolothum
Quoted 50 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.

No functional change intended.

---
 hw/arm/smmuv3.c          | 1 +
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 include/hw/arm/smmuv3.h  | 1 +
 4 files changed, 8 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -0
index 7858bf2c33..7f84c87a46 100644
@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +2 -0
index ae3b4aac52..046e930ca5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
 
     /*
      * IORT-only fields.
@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)
     bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
     sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
     sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+    sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
     pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
     sbdev = SYS_BUS_DEVICE(obj);
     sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);

hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 647

Quoted 5 lines
hw/arm/virt-acpi-build.c +1 -1
@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */

Would that keep the same ordering for existing qtests?

Eric

Quoted 35 lines
hw/arm/virt-acpi-build.c +0 -0
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +3 -0
index 390845c503..22ee5c4a41 100644
@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                        (vms->legacy_smmuv3_present) ?
                        "iommu=smmuv3" : "virtio-iommu");
         } else if (vms->iommu == VIRT_IOMMU_NONE) {
+            static uint8_t id;
+
             /* The new SMMUv3 device is specific to the PCI bus */
             object_property_set_bool(OBJECT(dev), "smmu_per_bus", true, NULL);
             object_property_set_link(OBJECT(dev), "memory",
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", id++, NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
include/hw/arm/smmuv3.h +1 -0
index 87926f8cb3..39cb43506e 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
#74 lore raw

RE: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

Quoted 16 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 18:49
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty

Quoted 5 lines

Review

Quoted 4 lines
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
(quoted diff) +2 -1
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /*
+ Identifier */

Would that keep the same ordering for existing qtests?

Hmm.. I need to check that.

Thanks,
Shameer

#75 lore raw

PATCH v2 23/24 hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.

The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.

Because virt-acpi-build.c now includes CONFIG_DEVICES via the Tegra241 CMDQV header, the Meson file entry is updated to build it as part of arm_ss instead of arm_common_ss

---
 hw/arm/meson.build       |  2 +-
 hw/arm/trace-events      |  1 +
 hw/arm/virt-acpi-build.c | 73 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+), 1 deletion(-)
hw/arm/meson.build +1 -1
index 4ec91db50a..af0b516df1 100644
@@ -1,7 +1,7 @@
 arm_ss = ss.source_set()
 arm_common_ss = ss.source_set()
 arm_common_ss.add(when: 'CONFIG_ARM_VIRT', if_true: files('virt.c'))
-arm_common_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c'))
+arm_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c'))
 arm_common_ss.add(when: 'CONFIG_DIGIC', if_true: files('digic_boards.c'))
 arm_common_ss.add(when: 'CONFIG_EMCRAFT_SF2', if_true: files('msf2-som.c'))
 arm_common_ss.add(when: 'CONFIG_HIGHBANK', if_true: files('highbank.c'))
hw/arm/trace-events +1 -0
index ef495c040c..e7e3ccfe9f 100644
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s"
 
 # virt-acpi-build.c
 virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out."
+virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d"
 
 # smmu-common.c
 smmu_add_mr(const char *name) "%s"
hw/arm/virt-acpi-build.c +73 -0

91 lines not on this page · raw message

Signature · 1 line
2.43.0
#76 lore raw

Re: [PATCH v2 23/24] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Quoted 84 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.

The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.

Because virt-acpi-build.c now includes CONFIG_DEVICES via the Tegra241 CMDQV header, the Meson file entry is updated to build it as part of arm_ss instead of arm_common_ss

---
 hw/arm/meson.build       |  2 +-
 hw/arm/trace-events      |  1 +
 hw/arm/virt-acpi-build.c | 73 ++++++++++++++++++++++++++++++++++++++++
 3 files changed, 75 insertions(+), 1 deletion(-)
hw/arm/meson.build +1 -1
index 4ec91db50a..af0b516df1 100644
@@ -1,7 +1,7 @@
 arm_ss = ss.source_set()
 arm_common_ss = ss.source_set()
 arm_common_ss.add(when: 'CONFIG_ARM_VIRT', if_true: files('virt.c'))
-arm_common_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c'))
+arm_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c'))
 arm_common_ss.add(when: 'CONFIG_DIGIC', if_true: files('digic_boards.c'))
 arm_common_ss.add(when: 'CONFIG_EMCRAFT_SF2', if_true: files('msf2-som.c'))
 arm_common_ss.add(when: 'CONFIG_HIGHBANK', if_true: files('highbank.c'))
hw/arm/trace-events +1 -0
index ef495c040c..e7e3ccfe9f 100644
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s"
 
 # virt-acpi-build.c
 virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out."
+virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d"
 
 # smmu-common.c
 smmu_add_mr(const char *name) "%s"
hw/arm/virt-acpi-build.c +23 -0
index 046e930ca5..fe2925baaf 100644
@@ -65,6 +65,8 @@
 #include "target/arm/cpu.h"
 #include "target/arm/multiprocessing.h"
 
+#include "tegra241-cmdqv.h"
+
 #define ARM_SPI_BASE 32
 
 #define ACPI_BUILD_TABLE_SIZE             0x20000
@@ -1121,6 +1123,75 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,
     build_fadt(table_data, linker, &fadt, vms->oem_id, vms->oem_table_id);
 }
 
+static int smmuv3_cmdqv_devices(Object *obj, void *opaque)
+{
+    VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
+    GArray *sdev_blob = opaque;
+    PlatformBusDevice *pbus;
+    AcpiSMMUv3Dev sdev;
+    SysBusDevice *sbdev;
+
+    if (!object_dynamic_cast(obj, TYPE_ARM_SMMUV3)) {
+        return 0;
+    }
+
+    if (!object_property_get_bool(obj, "tegra241-cmdqv", NULL)) {
+        return 0;
+    }
+
+    sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
+    pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+    sbdev = SYS_BUS_DEVICE(obj);
+    sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 1);
+    sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;

hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1156

Quoted 10 lines
hw/arm/virt-acpi-build.c +10 -0
+    sdev.irq = platform_bus_get_irqn(pbus, sbdev, NUM_SMMU_IRQS);
+    sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
+    sdev.irq += ARM_SPI_BASE;
+    g_array_append_val(sdev_blob, sdev);
+    return 0;
+}
+
+static void acpi_dsdt_add_tegra241_cmdqv(Aml *scope, VirtMachineState *vms)
+{
+    GArray *smmuv3_devs = g_array_new(false, true, sizeof(AcpiSMMUv3Dev));

wonder if it wouldn't be possible to compute and store the blob in vms once and reuse that for both iort and dsdt?

Eric

Quoted 50 lines
hw/arm/virt-acpi-build.c +40 -0
+    int i;
+
+    if (vms->legacy_smmuv3_present) {
+        return;
+    }
+
+    object_child_foreach_recursive(object_get_root(), smmuv3_cmdqv_devices,
+                                   smmuv3_devs);
+
+    for (i = 0; i < smmuv3_devs->len; i++) {
+        AcpiSMMUv3Dev *sdev;
+        Aml *dev, *crs, *addr;
+
+        sdev = &g_array_index(smmuv3_devs, AcpiSMMUv3Dev, i);
+
+        dev = aml_device("CV%.02u", sdev->id);
+        aml_append(dev, aml_name_decl("_HID", aml_string("NVDA200C")));
+        aml_append(dev, aml_name_decl("_UID", aml_int(sdev->id)));
+        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
+
+        crs = aml_resource_template();
+        addr = aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED, AML_MAX_FIXED,
+                                AML_CACHEABLE, AML_READ_WRITE, 0x0, sdev->base,
+                                sdev->base + TEGRA241_CMDQV_IO_LEN - 0x1, 0x0,
+                                TEGRA241_CMDQV_IO_LEN);
+        aml_append(crs, addr);
+        aml_append(crs, aml_interrupt(AML_CONSUMER, AML_EDGE,
+                                      AML_ACTIVE_HIGH, AML_EXCLUSIVE,
+                                      (uint32_t *)&sdev->irq, 1));
+        aml_append(dev, aml_name_decl("_CRS", crs));
+
+        aml_append(scope, dev);
+
+        trace_virt_acpi_dsdt_tegra241_cmdqv(sdev->id, sdev->base, sdev->irq);
+    }
+    g_array_free(smmuv3_devs, true);
+}
+
 /* DSDT */
 static void
 build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
@@ -1185,6 +1256,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_dsdt_add_tpm(scope, vms);
 #endif
 
+    acpi_dsdt_add_tegra241_cmdqv(scope, vms);
+
     aml_append(dsdt, scope);
 
     pci0_scope = aml_scope("\\_SB.PCI0");
#77 lore raw

PATCH v2 24/24 hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

In reply to Shameer Kolothum

Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

---
 hw/arm/smmuv3.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
hw/arm/smmuv3.c +13 -0
index 7f84c87a46..84dc5351d1 100644
@@ -1956,6 +1956,12 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
     }
 #endif
 
+#ifndef CONFIG_TEGRA241_CMDQV
+    if (s->tegra241_cmdqv) {
+        error_setg(errp, "tegra241_cmdqv=on support not compiled in");
+        return false;
+    }
+#endif
     if (!s->accel) {
         if (!s->ril) {
             error_setg(errp, "ril can only be disabled if accel=on");
@@ -1973,6 +1979,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
             error_setg(errp, "ssidsize can only be set if accel=on");
             return false;
         }
+        if (s->tegra241_cmdqv) {
+            error_setg(errp, "tegra241_cmdqv can only be enabled if accel=on");
+            return false;
+        }
         return true;
     }
 
@@ -2123,6 +2133,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_BOOL("ats", SMMUv3State, ats, false),
     DEFINE_PROP_UINT8("oas", SMMUv3State, oas, 44),
     DEFINE_PROP_UINT8("ssidsize", SMMUv3State, ssidsize, 0),
+    DEFINE_PROP_BOOL("tegra241-cmdqv", SMMUv3State, tegra241_cmdqv, false),
 };
 
 static void smmuv3_instance_init(Object *obj)
@@ -2162,6 +2173,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to 0. A value greater than 0 is required to enable "
         "PASID support.");
+    object_class_property_set_description(klass, "tegra241-cmdqv",
+        "Enable/disable Tegra241 CMDQ-Virtualisation support (for accel=on)");
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
#78 lore raw

Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

Quoted 2 lines · Shameer Kolothum

Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

Hi, do we expect that in the future there will be some other option named {newplatform}-cmdqv?

I'm asking mainly from libvirt PoV where current RFC patch exports this option only as cmdqv.

Pavel

#79 lore raw

RE: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

Quoted 17 lines

-----Original Message-----
From: Pavel Hrdina <[email protected]>
Sent: 13 February 2026 11:55
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

On Fri, Feb 06, 2026 at 02:48:23PM +0000, Shameer Kolothum wrote:

Quoted 8 lines
Quoted 2 lines

Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

Hi, do we expect that in the future there will be some other option named {newplatform}-cmdqv?

I'm asking mainly from libvirt PoV where current RFC patch exports this option only as cmdqv.

Yes, with the current design each implementation would expose its own xxx-cmdqv property.

However, please see the discussion:
https://lore.kernel.org/qemu-devel/CH3PR12MB75484A150D6DCD9D3C3C66F3AB60A@CH3PR12MB7548.namprd12.prod.outlook.com/

I am looking into whether we can instead have a single cmdqv property and probe the kernel to determine the specific implementation type.

Thanks,
Shameer

#80 lore raw

Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

Quoted 26 lines · Shameer Kolothum Thodi
Quoted 17 lines

-----Original Message-----
From: Pavel Hrdina <[email protected]>
Sent: 13 February 2026 11:55
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device

On Fri, Feb 06, 2026 at 02:48:23PM +0000, Shameer Kolothum wrote:

Quoted 8 lines
Quoted 2 lines

Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

Hi, do we expect that in the future there will be some other option named {newplatform}-cmdqv?

I'm asking mainly from libvirt PoV where current RFC patch exports this option only as cmdqv.

Quoted 8 lines

Yes, with the current design each implementation would expose its own xxx-cmdqv property.

However, please see the discussion:
https://lore.kernel.org/qemu-devel/CH3PR12MB75484A150D6DCD9D3C3C66F3AB60A@CH3PR12MB7548.namprd12.prod.outlook.com/

I am looking into whether we can instead have a single cmdqv property and probe the kernel to determine the specific implementation type.

Thanks for the details, yeah that looks better.

I was also thinking about using cmdqv=tegra241 in addition to 'off' and 'auto' if it would be possible for host to support multiple implementations at the same time. Libvirt would probably prefer setting specific implementation instead of using 'auto' to make sure that the VM will not change.

Pavel