Skip to contentSkip to conversation

hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

PATCH Superseded Shameer Kolothum 4 participants 101 messages Apr 15 – May 8

Subsystems (2)

v4 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v4 cover hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

Hi,

Changes from v3:
https://lore.kernel.org/qemu-devel/[email protected]/

- Addressed v3 feedback and picked up Reviewed-by tags.
- Folded veventq alloc/free into alloc_viommu/free_viommu, removing the separate ops callback (patch 13).
- Reworked register and macro names based on feedback.
- Improved documentation around VCMDQ aperture usage, which was a source of confusion in v3. See patches 15, 16, 17, 19 and 20. Patch 20 in particular explains the cached register vs hardware-backed MMIO model for VCMDQ apertures. Hope this is clearer and correct now!.
- Added patch 21 to skip IOMMU mappings for RAM device regions, eliminating spurious "IOMMU_IOAS_MAP failed: Bad address" warnings for the VINTF page0 guest mapping.
- Updated SMMUv3 identifier property to accommodate the ITS node id (patch 27).
- Removed qtest bios-tables blob patches; node id changes are now handled in patch 27.
- Based on top of Nathan's "Resolve AUTO properties" series [0].
- Added patch 30 to enforce viommu association stability when CMDQV is active.

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-vcmdq-v4-ext

Sanity tested on NVIDIA Grace. Further testing in progress.

Feedback and testing are very welcome.

Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]

---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/[email protected]/

Thanks to Nicolin for the initial patches and testing on which this is based.

Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.

Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)

Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.

VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].
---

Nicolin Chen (15):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate VCMDQ register writes hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Shameer Kolothum (16):
system/iommufd: Remove unused viommu pointer from IOMMUFDVeventq hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle hw/arm/virt: Use stored SMMUv3 device list for IORT build hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus system/physmem: Add address_space_is_ram() helper hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing
memory: Allow RAM device regions to skip IOMMU mapping hw/arm/smmuv3-accel: Introduce common helper for veventq read hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors hw/arm/smmuv3: Add per-device identifier property hw/arm/smmuv3-accel: Introduce helper to query CMDQV type hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

 hw/arm/smmuv3-accel.h         |  29 ++
 hw/arm/tegra241-cmdqv.h       | 367 +++++++++++++++
 include/hw/arm/smmuv3.h       |   3 +
 include/hw/arm/virt.h         |   1 +
 include/system/iommufd.h      |  17 +-
 include/system/memory.h       |  12 +
 backends/iommufd.c            |  64 +++
 hw/arm/smmuv3-accel-stubs.c   |  16 +
 hw/arm/smmuv3-accel.c         | 187 ++++++--
 hw/arm/smmuv3.c               |  15 +
 hw/arm/tegra241-cmdqv-stubs.c |  16 +
 hw/arm/tegra241-cmdqv.c       | 817 ++++++++++++++++++++++++++++++++++
 hw/arm/virt-acpi-build.c      | 127 ++++--
 hw/arm/virt.c                 |  37 ++
 hw/vfio/iommufd.c             |   4 +-
 hw/vfio/listener.c            |   5 +
 system/physmem.c              |  11 +
 backends/trace-events         |   4 +-
 hw/arm/Kconfig                |   5 +
 hw/arm/meson.build            |   2 +
 hw/arm/trace-events           |   7 +
 21 files changed, 1666 insertions(+), 80 deletions(-)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
Signature · 1 line
2.43.0
#1 lore raw

PATCH v4 1/31 backends/iommufd: Update iommufd_backend_get_device_info

From: Nicolin Chen <[email protected]>

The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.

---
 backends/iommufd.c    | 7 +++++++
 hw/arm/smmuv3-accel.c | 2 +-
 hw/vfio/iommufd.c     | 4 ++--
 3 files changed, 10 insertions(+), 3 deletions(-)
backends/iommufd.c +7 -0
index 52cb060454..20d4186f29 100644
@@ -397,16 +397,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,
     return true;
 }
 
+/*
+ * @type can carry a desired HW info type defined in the uapi headers. If caller
+ * doesn't have one, indicating it wants the default type, then @type should be
+ * zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT).
+ */
 bool iommufd_backend_get_device_info(IOMMUFDBackend *be, uint32_t devid,
                                      uint32_t *type, void *data, uint32_t len,
                                      uint64_t *caps, uint8_t *max_pasid_log2,
                                      Error **errp)
 {
     struct iommu_hw_info info = {
+        .flags = (*type) ? IOMMU_HW_INFO_FLAG_INPUT_TYPE : 0,
         .size = sizeof(info),
         .dev_id = devid,
         .data_len = len,
         .data_uptr = (uintptr_t)data,
+        .in_data_type = *type,
     };
 
     if (ioctl(be->fd, IOMMU_GET_HW_INFO, &info)) {
hw/arm/smmuv3-accel.c +1 -1
index 48f8017262..d68d4141a0 100644
@@ -172,7 +172,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                            Error **errp)
 {
     struct iommu_hw_info_arm_smmuv3 info;
-    uint32_t data_type;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_DEFAULT;
     uint64_t caps;
 
     if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +2 -2
index 3e33dfbb35..4043111667 100644
@@ -352,7 +352,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,
     ERRP_GUARD();
     IOMMUFDBackend *iommufd = vbasedev->iommufd;
     VFIOContainer *bcontainer = VFIO_IOMMU(container);
-    uint32_t type, flags = 0;
+    uint32_t type = IOMMU_HW_INFO_TYPE_DEFAULT, flags = 0;
     uint64_t hw_caps;
     VendorCaps caps;
     VFIOIOASHwpt *hwpt;
@@ -941,7 +941,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,
     HostIOMMUDeviceIOMMUFD *idev;
     HostIOMMUDeviceCaps *caps = &hiod->caps;
     VendorCaps *vendor_caps = &caps->vendor_caps;
-    enum iommu_hw_info_type type;
+    uint32_t type = IOMMU_HW_INFO_TYPE_DEFAULT;
     uint8_t max_pasid_log2;
     uint64_t hw_caps;
 
Signature · 1 line
2.43.0
#2 lore raw

PATCH v4 2/31 backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.

Update the caller accordingly.

---
 include/system/iommufd.h | 1 +
 backends/iommufd.c       | 4 ++++
 hw/arm/smmuv3-accel.c    | 4 ++--
 backends/trace-events    | 2 +-
 4 files changed, 8 insertions(+), 3 deletions(-)
include/system/iommufd.h +1 -0
index 7062944fe6..e027800c91 100644
@@ -89,6 +89,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id,
                                 Error **errp);
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_hwpt, Error **errp);
 
 bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
backends/iommufd.c +4 -0
index 20d4186f29..9b07ac19c2 100644
@@ -470,6 +470,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,
 
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_viommu_id, Error **errp)
 {
     int ret;
@@ -478,11 +479,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
         .type = viommu_type,
         .dev_id = dev_id,
         .hwpt_id = hwpt_id,
+        .data_len = data_len,
+        .data_uptr = (uintptr_t)data_ptr,
     };
 
     ret = ioctl(be->fd, IOMMU_VIOMMU_ALLOC, &alloc_viommu);
 
     trace_iommufd_backend_alloc_viommu(be->fd, dev_id, viommu_type, hwpt_id,
+                                       (uintptr_t)data_ptr, data_len,
                                        alloc_viommu.out_viommu_id, ret);
     if (ret) {
         error_setg_errno(errp, errno, "IOMMU_VIOMMU_ALLOC failed");
hw/arm/smmuv3-accel.c +2 -2
index d68d4141a0..c356ff9708 100644
@@ -578,8 +578,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     IOMMUFDViommu *viommu;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
-                                      s2_hwpt_id, &viommu_id, errp)) {
+                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
+                                      NULL, 0, &viommu_id, errp)) {
         return false;
     }
 
backends/trace-events +1 -1
index b9365113e7..3ba0c3503c 100644
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (%
 iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)"
 iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)"
 iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)"
-iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)"
+iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 
Signature · 1 line
2.43.0
#3 lore raw

PATCH v4 3/31 backends/iommufd: Introduce iommufd_backend_alloc_hw_queue

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a helper to allocate an iommufd backed HW queue for a vIOMMU.

While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.

---
 include/system/iommufd.h | 11 +++++++++++
 backends/iommufd.c       | 31 +++++++++++++++++++++++++++++++
 backends/trace-events    |  1 +
 3 files changed, 43 insertions(+)
include/system/iommufd.h +11 -0
index e027800c91..8009ce3d31 100644
@@ -65,6 +65,12 @@ typedef struct IOMMUFDVeventq {
     bool event_start; /* True after first valid event; cleared on overflow */
 } IOMMUFDVeventq;
 
+/* HW queue object for a vIOMMU-specific HW-accelerated queue */
+typedef struct IOMMUFDHWqueue {
+    IOMMUFDViommu *viommu;
+    uint32_t hw_queue_id;
+} IOMMUFDHWqueue;
+
 bool iommufd_backend_connect(IOMMUFDBackend *be, Error **errp);
 void iommufd_backend_disconnect(IOMMUFDBackend *be);
 
@@ -101,6 +107,11 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
                                    uint32_t *out_veventq_id,
                                    uint32_t *out_veventq_fd, Error **errp);
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    uint32_t queue_type, uint32_t index,
+                                    uint64_t addr, uint64_t length,
+                                    uint32_t *out_hw_queue_id, Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
backends/iommufd.c +31 -0
index 9b07ac19c2..3be7b07eec 100644
@@ -556,6 +556,37 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    uint32_t queue_type, uint32_t index,
+                                    uint64_t addr, uint64_t length,
+                                    uint32_t *out_hw_queue_id, Error **errp)
+{
+    int ret;
+    struct iommu_hw_queue_alloc alloc_hw_queue = {
+        .size = sizeof(alloc_hw_queue),
+        .flags = 0,
+        .viommu_id = viommu_id,
+        .type = queue_type,
+        .index = index,
+        .nesting_parent_iova = addr,
+        .length = length,
+    };
+
+    ret = ioctl(be->fd, IOMMU_HW_QUEUE_ALLOC, &alloc_hw_queue);
+
+    trace_iommufd_backend_alloc_hw_queue(be->fd, viommu_id, queue_type,
+                                         index, addr, length,
+                                         alloc_hw_queue.out_hw_queue_id, ret);
+    if (ret) {
+        error_setg_errno(errp, errno, "IOMMU_HW_QUEUE_ALLOC failed");
+        return false;
+    }
+
+    g_assert(out_hw_queue_id);
+    *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id;
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index 3ba0c3503c..c5c1d95aad 100644
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u
 iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
+iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
Signature · 1 line
2.43.0
#4 lore raw

PATCH v4 4/31 backends/iommufd: Introduce iommufd_backend_viommu_mmap

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.

The caller is responsible for unmapping the returned region.

---
 include/system/iommufd.h |  4 ++++
 backends/iommufd.c       | 22 ++++++++++++++++++++++
 backends/trace-events    |  1 +
 3 files changed, 27 insertions(+)
include/system/iommufd.h +4 -0
index 8009ce3d31..38cfceca84 100644
@@ -112,6 +112,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
                                     uint64_t addr, uint64_t length,
                                     uint32_t *out_hw_queue_id, Error **errp);
 
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
backends/iommufd.c +22 -0
index 3be7b07eec..e26675990e 100644
@@ -587,6 +587,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+/*
+ * Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance.
+ * The caller is responsible for unmapping the mapped region.
+ */
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp)
+{
+    g_assert(viommu_id);
+    g_assert(out_ptr);
+
+    *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, be->fd,
+                   offset);
+    trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size, offset);
+    if (*out_ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno, "IOMMUFD vIOMMU mmap failed");
+        return false;
+    }
+
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index c5c1d95aad..b63420b73e 100644
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
Signature · 1 line
2.43.0
#5 lore raw

PATCH v4 5/31 system/iommufd: Remove unused viommu pointer from IOMMUFDVeventq

In reply to Shameer Kolothum

The viommu field is assigned but never used. Callers freeing the veventq already have access to the IOMMUFDViommu object through other references, so this field is redundant.

Removing it also simplifies upcoming changes where veventq is allocated based on the viommu id before the IOMMUFDViommu object is created (e.g. vendor CMDQV-based veventq allocation).

No functional change.

---
 include/system/iommufd.h | 1 -
 hw/arm/smmuv3-accel.c    | 1 -
 2 files changed, 2 deletions(-)
include/system/iommufd.h +0 -1
index 38cfceca84..b6599521b8 100644
@@ -58,7 +58,6 @@ typedef struct IOMMUFDVdev {
 
 /* Virtual event queue interface for a vIOMMU */
 typedef struct IOMMUFDVeventq {
-    IOMMUFDViommu *viommu;
     uint32_t veventq_id;
     uint32_t veventq_fd;
     uint32_t last_event_seq; /* Sequence number of last processed event */
hw/arm/smmuv3-accel.c +0 -1
index c356ff9708..f65e654adf 100644
@@ -549,7 +549,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
     veventq = g_new0(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
-    veventq->viommu = accel->viommu;
     accel->veventq = veventq;
 
     /* Set up event handler for veventq fd */
Signature · 1 line
2.43.0
Reviewed-by on v4 5/31
#6 lore raw

Re: [PATCH v4 05/31] system/iommufd: Remove unused viommu pointer from IOMMUFDVeventq

Quoted 11 lines · Shameer Kolothum

The viommu field is assigned but never used. Callers freeing the veventq already have access to the IOMMUFDViommu object through other references, so this field is redundant.

Removing it also simplifies upcoming changes where veventq is allocated based on the viommu id before the IOMMUFDViommu object is created (e.g. vendor CMDQV-based veventq allocation).

No functional change.

Eric

Quoted 29 lines
---
 include/system/iommufd.h | 1 -
 hw/arm/smmuv3-accel.c    | 1 -
 2 files changed, 2 deletions(-)
include/system/iommufd.h +0 -1
index 38cfceca84..b6599521b8 100644
@@ -58,7 +58,6 @@ typedef struct IOMMUFDVdev {
 
 /* Virtual event queue interface for a vIOMMU */
 typedef struct IOMMUFDVeventq {
-    IOMMUFDViommu *viommu;
     uint32_t veventq_id;
     uint32_t veventq_fd;
     uint32_t last_event_seq; /* Sequence number of last processed event */
hw/arm/smmuv3-accel.c +0 -1
index c356ff9708..f65e654adf 100644
@@ -549,7 +549,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
     veventq = g_new0(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
-    veventq->viommu = accel->viommu;
     accel->veventq = veventq;
 
     /* Set up event handler for veventq fd */
Reviewed-by on v4 5/31
#7 lore raw

Re: [PATCH v4 05/31] system/iommufd: Remove unused viommu pointer from IOMMUFDVeventq

In reply to Shameer Kolothum
Quoted 11 lines · Shameer Kolothum

The viommu field is assigned but never used. Callers freeing the veventq already have access to the IOMMUFDViommu object through other references, so this field is redundant.

Removing it also simplifies upcoming changes where veventq is allocated based on the viommu id before the IOMMUFDViommu object is created (e.g. vendor CMDQV-based veventq allocation).

No functional change.

#8 lore raw

PATCH v4 6/31 hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum

Command Queue Virtualization (CMDQV) is a hardware extension available on certain platforms that allows the SMMUv3 command queue to be virtualized and passed through to a VM, improving performance.

For example, NVIDIA Tegra241 implements CMDQV to support virtualization of multiple command queues (VCMDQs).

The term CMDQV is used here generically to refer to any platform that provides hardware support to virtualize the SMMUv3 command queue.

CMDQV support is a specialization of the IOMMUFD-backed accelerated SMMUv3 path. Introduce an ops interface to factor out CMDQV-specific probe, initialization, and vIOMMU allocation logic from the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

This provides an extensible design to support future vendor-specific CMDQV implementations.

No functional change.

---
 hw/arm/smmuv3-accel.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/arm/smmuv3-accel.h +18 -0
index 7b4a0be000..86301afcb4 100644
@@ -10,11 +10,28 @@
 #define HW_ARM_SMMUV3_ACCEL_H
 
 #include "hw/arm/smmu-common.h"
+#include "hw/arm/smmuv3.h"
 #include "system/iommufd.h"
 #ifdef CONFIG_LINUX
 #include <linux/iommufd.h>
 #endif
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*probe)(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp);
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -27,6 +44,7 @@ typedef struct SMMUv3AccelState {
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
     bool auto_mode;
     bool auto_finalised;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
Signature · 1 line
2.43.0
#9 lore raw

Re: [PATCH v4 06/31] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

Hi Shameer,

Quoted 34 lines · Shameer Kolothum

Command Queue Virtualization (CMDQV) is a hardware extension available on certain platforms that allows the SMMUv3 command queue to be virtualized and passed through to a VM, improving performance.

For example, NVIDIA Tegra241 implements CMDQV to support virtualization of multiple command queues (VCMDQs).

The term CMDQV is used here generically to refer to any platform that provides hardware support to virtualize the SMMUv3 command queue.

CMDQV support is a specialization of the IOMMUFD-backed accelerated SMMUv3 path. Introduce an ops interface to factor out CMDQV-specific probe, initialization, and vIOMMU allocation logic from the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

This provides an extensible design to support future vendor-specific CMDQV implementations.

No functional change.

---
 hw/arm/smmuv3-accel.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
hw/arm/smmuv3-accel.h +0 -0
index 7b4a0be000..86301afcb4 100644
@@ -10,11 +10,28 @@
 #define HW_ARM_SMMUV3_ACCEL_H
 
 #include "hw/arm/smmu-common.h"

hw/arm/smmuv3-accel.h line 22

Quoted 10 lines
hw/arm/smmuv3-accel.h +5 -0
+#include "hw/arm/smmuv3.h"
 #include "system/iommufd.h"
 #ifdef CONFIG_LINUX
 #include <linux/iommufd.h>
 #endif
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.

I would doc-comment which of those ops are mandated. For instance it looks alloc_viommu is mandatory if the the whole ops exists. By the way, it looks strange that !cmdqv_ops->free_viommu does not cause any assert in 8/31 because we may not free the viommu_id if it does not exist? Can't we enforce each mandated ops is implemented?

Thanks

Eric

Quoted 23 lines
hw/arm/smmuv3-accel.h +13 -0
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*probe)(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp);
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -27,6 +44,7 @@ typedef struct SMMUv3AccelState {
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
     bool auto_mode;
     bool auto_finalised;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
Reviewed-by on v4 6/31
#10 lore raw

Re: [PATCH v4 06/31] hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum
Quoted 16 lines · Shameer Kolothum

Command Queue Virtualization (CMDQV) is a hardware extension available on certain platforms that allows the SMMUv3 command queue to be virtualized and passed through to a VM, improving performance.

For example, NVIDIA Tegra241 implements CMDQV to support virtualization of multiple command queues (VCMDQs).

The term CMDQV is used here generically to refer to any platform that provides hardware support to virtualize the SMMUv3 command queue.

CMDQV support is a specialization of the IOMMUFD-backed accelerated SMMUv3 path. Introduce an ops interface to factor out CMDQV-specific probe, initialization, and vIOMMU allocation logic from the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

Quoted 6 lines

This provides an extensible design to support future vendor-specific CMDQV implementations.

No functional change.

#11 lore raw

PATCH v4 7/31 hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

In reply to Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 16 ++++++++++
 hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 94 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
hw/arm/tegra241-cmdqv.h new file +15 -0
new file mode 100644
index 0000000000..07e10e86ee
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv-stubs.c new file +16 -0
new file mode 100644
index 0000000000..eabf90daf8
@@ -0,0 +1,16 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return NULL;
+}
hw/arm/tegra241-cmdqv.c new file +56 -0

56 lines not on this page · raw message

hw/arm/Kconfig +5 -0
index 4e50fb1111..073f2ebaaf 100644
@@ -618,6 +618,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -625,6 +629,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +2 -0
index 3be1252c4f..64bcdc5a7c 100644
@@ -87,6 +87,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
Signature · 1 line
2.43.0
Reviewed-by on v4 7/31
#12 lore raw

Re: [PATCH v4 07/31] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

Hi Shameer,

Quoted 27 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 16 ++++++++++
 hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 94 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
hw/arm/tegra241-cmdqv.h new file +0 -0
new file mode 100644
index 0000000000..07e10e86ee

hw/arm/tegra241-cmdqv.h line 3

Quoted 3 lines
hw/arm/tegra241-cmdqv.h new file +3 -0
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3

virtualization

Quoted 18 lines
hw/arm/tegra241-cmdqv.h new file +12 -0
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv-stubs.c new file +0 -0
new file mode 100644
index 0000000000..eabf90daf8

hw/arm/tegra241-cmdqv-stubs.c line 4

Quoted 4 lines
hw/arm/tegra241-cmdqv-stubs.c new file +4 -0
@@ -0,0 +1,16 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3

same

Quoted 102 lines
hw/arm/tegra241-cmdqv-stubs.c new file +12 -0
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return NULL;
+}
hw/arm/tegra241-cmdqv.c new file +56 -0
new file mode 100644
index 0000000000..ad5a0d4611
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
+{
+}
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                            uint32_t *out_viommu_id, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s)
+{
+}
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                                 Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
+    .probe = tegra241_cmdqv_probe,
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .reset = tegra241_cmdqv_reset,
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return &tegra241_cmdqv_ops;
+}
hw/arm/Kconfig +5 -0
index 4e50fb1111..073f2ebaaf 100644
@@ -618,6 +618,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -625,6 +629,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +0 -0
index 3be1252c4f..64bcdc5a7c 100644

hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) · line 94

Quoted 8 lines
hw/arm/meson.build +2 -0
@@ -87,6 +87,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(

Besides

Eric

Reviewed-by on v4 7/31
#13 lore raw

Re: [PATCH v4 07/31] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

In reply to Shameer Kolothum
Quoted 10 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

#14 lore raw

PATCH v4 8/31 hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

In reply to Shameer Kolothum

Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.

If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.

The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.

No functional change.

---
 include/hw/arm/smmuv3.h |  2 +
 hw/arm/smmuv3-accel.c   | 93 +++++++++++++++++++++++++++++++++++++----
 2 files changed, 88 insertions(+), 7 deletions(-)
include/hw/arm/smmuv3.h +2 -0
index fe0493c1aa..aa6a79237a 100644
@@ -74,6 +74,8 @@ struct SMMUv3State {
     OnOffAuto ats;
     OasMode oas;
     SsidSizeMode ssidsize;
+    /* SMMU CMDQV extension */
+    OnOffAuto cmdqv;
 
     Notifier machine_done;
 };
hw/arm/smmuv3-accel.c +86 -7

134 lines not on this page · raw message

Signature · 1 line
2.43.0
Reviewed-by on v4 8/31
#15 lore raw

Re: [PATCH v4 08/31] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

Quoted 86 lines · Shameer Kolothum

Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.

If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.

The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.

No functional change.

---
 include/hw/arm/smmuv3.h |  2 +
 hw/arm/smmuv3-accel.c   | 93 +++++++++++++++++++++++++++++++++++++----
 2 files changed, 88 insertions(+), 7 deletions(-)
include/hw/arm/smmuv3.h +2 -0
index fe0493c1aa..aa6a79237a 100644
@@ -74,6 +74,8 @@ struct SMMUv3State {
     OnOffAuto ats;
     OasMode oas;
     SsidSizeMode ssidsize;
+    /* SMMU CMDQV extension */
+    OnOffAuto cmdqv;
 
     Notifier machine_done;
 };
hw/arm/smmuv3-accel.c +13 -4
index f65e654adf..9068e65e2b 100644
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and shared_as_sysmem
@@ -566,6 +567,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -576,10 +578,17 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
-                                      NULL, 0, &viommu_id, errp)) {
-        return false;
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {
+            return false;
+        }
     }
 
     viommu = g_new0(IOMMUFDViommu, 1);

hw/arm/smmuv3-accel.c free_bypass_hwpt: · line 638

Quoted 6 lines
hw/arm/smmuv3-accel.c +2 -1
@@ -625,12 +634,69 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);

hum actually we do free the iommu id below in case the free_viommu is not implemented. So forget my previously comment. Besides this means free_viommu shall be documented as not mandatory.

Quoted 39 lines
hw/arm/smmuv3-accel.c +34 -0
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }
     g_free(viommu);
     accel->viommu = NULL;
     return false;
 }
 
+static const SMMUv3AccelCmdqvOps *
+smmuv3_accel_probe_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = tegra241_cmdqv_get_ops();
+
+    if (!ops || !ops->probe) {
+        error_setg(errp, "No CMDQV ops found");
+        return NULL;
+    }
+
+    if (!ops->probe(s, idev, errp)) {
+        return NULL;
+    }
+    return ops;
+}
+
+static bool
+smmuv3_accel_select_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = NULL;
+
+    if (s->s_accel->cmdqv_ops) {
+        return true;
+    }
+
+    switch (s->cmdqv) {
+    case ON_OFF_AUTO_OFF:
+        s->s_accel->cmdqv_ops = NULL;
+        return true;

hw/arm/smmuv3-accel.c free_bypass_hwpt: · line 687

Quoted 10 lines
hw/arm/smmuv3-accel.c +10 -0
+    case ON_OFF_AUTO_AUTO:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, NULL);
+        break;
+    case ON_OFF_AUTO_ON:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, errp);
+        if (!ops) {
+            error_append_hint(errp, "CMDQV requested but not supported");
+            return false;
+        }
+        s->s_accel->cmdqv_ops = ops;

Shouldn't you remove the above setting as you do it again below in case init succeeds?

Quoted 36 lines
hw/arm/smmuv3-accel.c +20 -2
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    if (ops && ops->init && !ops->init(s, errp)) {
+        return false;
+    }
+    s->s_accel->cmdqv_ops = ops;
+    return true;
+}
+
 static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
                                           HostIOMMUDevice *hiod, Error **errp)
 {
@@ -665,6 +731,10 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
         goto done;
     }
 
+    if (!smmuv3_accel_select_cmdqv(s, idev, errp)) {
+        return false;
+    }
+
     if (!smmuv3_accel_alloc_viommu(s, idev, errp)) {
         error_append_hint(errp, "Unable to alloc vIOMMU: idev devid 0x%x: ",
                           idev->devid);
@@ -936,8 +1006,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)
 
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;

hw/arm/smmuv3-accel.c bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp) · line 1022

Quoted 10 lines
hw/arm/smmuv3-accel.c +7 -0
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
 static void smmuv3_accel_as_init(SMMUv3State *s)

Besides

Eric

#16 lore raw

RE: [PATCH v4 08/31] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

Quoted 112 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 04 May 2026 16:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 08/31] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

External email: Use caution opening links or attachments

Quoted 4 lines · Shameer Kolothum

Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed

allocation

Quoted 3 lines

path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during device

setup.

Quoted 4 lines

If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.

Quoted 8 lines

If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.

The current implementation wires up the Tegra241 CMDQV backend

through the

Quoted 9 lines

generic ops interface. Functional CMDQV behaviour is added in subsequent patches.

No functional change.

---
 include/hw/arm/smmuv3.h |  2 +
 hw/arm/smmuv3-accel.c   | 93

+++++++++++++++++++++++++++++++++++++----

Quoted 20 lines
 2 files changed, 88 insertions(+), 7 deletions(-)
include/hw/arm/smmuv3.h +2 -0
index fe0493c1aa..aa6a79237a 100644
@@ -74,6 +74,8 @@ struct SMMUv3State {
     OnOffAuto ats;
     OasMode oas;
     SsidSizeMode ssidsize;
+    /* SMMU CMDQV extension */
+    OnOffAuto cmdqv;
 
     Notifier machine_done;
 };
hw/arm/smmuv3-accel.c +0 -0
index f65e654adf..9068e65e2b 100644

hw/arm/smmuv3-accel.c line 24

Quoted 7 lines
hw/arm/smmuv3-accel.c +1 -0
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and

shared_as_sysmem

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,

Quoted 1 line
hw/arm/smmuv3-accel.c +0 -0
@@ -566,6 +567,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,

Quoted 8 lines
hw/arm/smmuv3-accel.c +1 -0
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -576,10 +578,17 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,

Quoted 5 lines
hw/arm/smmuv3-accel.c +0 -2
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3,

s2_hwpt_id,

Quoted 15 lines
hw/arm/smmuv3-accel.c +11 -2
-                                      NULL, 0, &viommu_id, errp)) {
-        return false;
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {
+            return false;
+        }
     }
 
Quoted 11 lines

hw/arm/smmuv3-accel.c free_bypass_hwpt: · line 638

Quoted 8 lines
hw/arm/smmuv3-accel.c +2 -1
     viommu = g_new0(IOMMUFDViommu, 1);
@@ -625,12 +634,69 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);

hum actually we do free the iommu id below in case the free_viommu is not implemented. So forget my previously comment. Besides this means free_viommu shall be documented as not mandatory.

Sure. Will add documentation to make the mandatory/not mandatory clear.

Quoted 45 lines

hw/arm/smmuv3-accel.c free_bypass_hwpt:

Quoted 10 lines
hw/arm/smmuv3-accel.c +5 -0
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }
     g_free(viommu);
     accel->viommu = NULL;
     return false;
 }
 
+static const SMMUv3AccelCmdqvOps *
+smmuv3_accel_probe_cmdqv(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

Quoted 7 lines
hw/arm/smmuv3-accel.c +7 -0
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = tegra241_cmdqv_get_ops();
+
+    if (!ops || !ops->probe) {
+        error_setg(errp, "No CMDQV ops found");
+        return NULL;

hw/arm/smmuv3-accel.c free_bypass_hwpt:

Quoted 10 lines
hw/arm/smmuv3-accel.c +10 -0
+    }
+
+    if (!ops->probe(s, idev, errp)) {
+        return NULL;
+    }
+    return ops;
+}
+
+static bool
+smmuv3_accel_select_cmdqv(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

Quoted 16 lines
hw/arm/smmuv3-accel.c +16 -0
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = NULL;
+
+    if (s->s_accel->cmdqv_ops) {
+        return true;
+    }
+
+    switch (s->cmdqv) {
+    case ON_OFF_AUTO_OFF:
+        s->s_accel->cmdqv_ops = NULL;
+        return true;
+    case ON_OFF_AUTO_AUTO:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, NULL);
+        break;
+    case ON_OFF_AUTO_ON:
Quoted 8 lines

hw/arm/smmuv3-accel.c free_bypass_hwpt: · line 687

Quoted 6 lines
hw/arm/smmuv3-accel.c +6 -0
+        ops = smmuv3_accel_probe_cmdqv(s, idev, errp);
+        if (!ops) {
+            error_append_hint(errp, "CMDQV requested but not supported");
+            return false;
+        }
+        s->s_accel->cmdqv_ops = ops;

Shouldn't you remove the above setting as you do it again below in case init succeeds?

Correct. We don’t need to assign it here.

Quoted 48 lines

hw/arm/smmuv3-accel.c free_bypass_hwpt:

Quoted 13 lines
hw/arm/smmuv3-accel.c +12 -0
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    if (ops && ops->init && !ops->init(s, errp)) {
+        return false;
+    }
+    s->s_accel->cmdqv_ops = ops;
+    return true;
+}
+
 static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque,

int devfn,

hw/arm/smmuv3-accel.c static bool

Quoted 3 lines
hw/arm/smmuv3-accel.c +0 -0
                                           HostIOMMUDevice *hiod, Error **errp)
 {
@@ -665,6 +731,10 @@ static bool

smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,

hw/arm/smmuv3-accel.c bool

Quoted 11 lines
hw/arm/smmuv3-accel.c +4 -0
         goto done;
     }
 
+    if (!smmuv3_accel_select_cmdqv(s, idev, errp)) {
+        return false;
+    }
+
     if (!smmuv3_accel_alloc_viommu(s, idev, errp)) {
         error_append_hint(errp, "Unable to alloc vIOMMU: idev devid 0x%x: ",
                           idev->devid);
@@ -936,8 +1006,17 @@ bool

smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)

Quoted 18 lines
hw/arm/smmuv3-accel.c +11 -2
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
Quoted 3 lines

hw/arm/smmuv3-accel.c bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp) · line 1022

Quoted 1 line
hw/arm/smmuv3-accel.c +0 -0
 static void smmuv3_accel_as_init(SMMUv3State *s)

Besides

Thanks,
Shameer

Reviewed-by on v4 8/31
#17 lore raw

Re: [PATCH v4 08/31] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

In reply to Shameer Kolothum
Quoted 18 lines · Shameer Kolothum

Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.

If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.

Quoted 7 lines

The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.

No functional change.

With Eric's comments addressed,

#18 lore raw

PATCH v4 9/31 hw/arm/virt: Use stored SMMUv3 device list for IORT build

In reply to Shameer Kolothum

Introduce a GPtrArray in VirtMachineState to track all SMMUv3 devices created on the virt machine, and use it when building the IORT table instead of relying on object_child_foreach_recursive() walks of the object tree.

This avoids recursive object traversal and provides a foundation for subsequent patches that need direct access to SMMUv3 instances for CMDQV-related handling.

No functional change. No bios-tables qtest failures observed.

---
 include/hw/arm/virt.h    |  1 +
 hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++----------------------
 hw/arm/virt.c            |  3 ++
 3 files changed, 35 insertions(+), 39 deletions(-)
include/hw/arm/virt.h +1 -0
index 5fcbd1c76f..a840a97de8 100644
@@ -187,6 +187,7 @@ struct VirtMachineState {
     MemoryRegion *sysmem;
     MemoryRegion *secure_sysmem;
     bool pci_preserve_config;
+    GPtrArray *smmuv3_devices;
 };
 
 #define VIRT_ECAM_ID(high) (high ? VIRT_HIGH_PCIE_ECAM : VIRT_PCIE_ECAM)
hw/arm/virt-acpi-build.c +31 -39

87 lines not on this page · raw message

hw/arm/virt.c +3 -0
index ec0d8475ca..68464ceb14 100644
@@ -3261,6 +3261,7 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev,
             }
 
             create_smmuv3_dev_dtb(vms, dev, bus, errp);
+            g_ptr_array_add(vms->smmuv3_devices, dev);
         }
     }
 
@@ -3697,6 +3698,8 @@ static void virt_instance_init(Object *obj)
     vms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6);
     vms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
     cxl_machine_init(obj, &vms->cxl_devices_state);
+
+    vms->smmuv3_devices = g_ptr_array_new_with_free_func(NULL);
 }
 
 static const TypeInfo virt_machine_info = {
Signature · 1 line
2.43.0
Reviewed-by on v4 9/31
#19 lore raw

Re: [PATCH v4 09/31] hw/arm/virt: Use stored SMMUv3 device list for IORT build

Quoted 24 lines · Shameer Kolothum
hw/arm/virt-acpi-build.c +0 -24
-static int iort_smmuv3_devices(Object *obj, void *opaque)
-{
-    VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
-    GArray *sdev_blob = opaque;
-    AcpiIortIdMapping idmap;
-    PlatformBusDevice *pbus;
-    int min_bus, max_bus;
-    SysBusDevice *sbdev;
-    PCIBus *bus;
-
-    if (!object_dynamic_cast(obj, TYPE_ARM_SMMUV3)) {
-        return 0;
-    }
-
-    bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
-    sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
-    sdev.ats = smmuv3_ats_enabled(ARM_SMMUV3(obj));
-    pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
-    sbdev = SYS_BUS_DEVICE(obj);
-    sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
-    sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
-    sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
-    sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];

hw/arm/virt-acpi-build.c static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b) · line 421

Quoted 10 lines
hw/arm/virt-acpi-build.c +0 -10
-    sdev.irq += ARM_SPI_BASE;
-
-    pci_bus_range(bus, &min_bus, &max_bus);
-    sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
-    idmap.input_base = min_bus << 8,
-    idmap.id_count = (max_bus - min_bus + 1) << 8,
-    g_array_append_val(sdev.rc_smmu_idmaps, idmap);
-    g_array_append_val(sdev_blob, sdev);
-    return 0;
-}

This could just stay as:

static int __populate_smmuv3_dev(VirtMachineState *vms, GArray *sdev_blob)

?

Either way,

#20 lore raw

PATCH v4 10/31 hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

In reply to Shameer Kolothum

Use IOMMU_GET_HW_INFO to query host support for Tegra241 CMDQV.

Validate the returned data type, version, and minimum number of vCMDQs and SIDs per Tegra241 CMDQ Virtual Interface(VI). Fail the probe if the host does not meet these requirements.

The QEMU model supports one Virtual Interface(VI) per VM with 2 vCMDQs and 16 SIDs per VI, so the probe ensures the host implementation is compatible with these limits.

---
 hw/arm/tegra241-cmdqv.h |  4 ++++
 hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
 2 files changed, 34 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +4 -0
index 07e10e86ee..c1866084f8 100644
@@ -10,6 +10,10 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#define CMDQV_VER                 1
+#define CMDQV_NUM_CMDQ_LOG2       1
+#define CMDQV_NUM_SID_PER_VI_LOG2 4
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +30 -2
index ad5a0d4611..3a19a1af56 100644
@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
+    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
+    uint64_t caps;
+
+    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
+                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
+                                         NULL, errp)) {
+        return false;
+    }
+    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
+        error_setg(errp, "Host CMDQV: unexpected data type %u (expected %u)",
+                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
+        return false;
+    }
+    if (cmdqv_info.version != CMDQV_VER) {
+        error_setg(errp, "Host CMDQV: unsupported version %u (expected %u)",
+                   cmdqv_info.version, CMDQV_VER);
+        return false;
+    }
+    if (cmdqv_info.log2vcmdqs < CMDQV_NUM_CMDQ_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)",
+                   cmdqv_info.log2vcmdqs, CMDQV_NUM_CMDQ_LOG2);
+        return false;
+    }
+    if (cmdqv_info.log2vsids < CMDQV_NUM_SID_PER_VI_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >= %u)",
+                   cmdqv_info.log2vsids, CMDQV_NUM_SID_PER_VI_LOG2);
+        return false;
+    }
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
Signature · 1 line
2.43.0
Reviewed-by on v4 10/31
#21 lore raw

Re: [PATCH v4 10/31] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

Quoted 6 lines · Shameer Kolothum

Use IOMMU_GET_HW_INFO to query host support for Tegra241 CMDQV.

Validate the returned data type, version, and minimum number of vCMDQs and SIDs per Tegra241 CMDQ Virtual Interface(VI). Fail the probe if the host does not meet these requirements.

Quoted 6 lines

The QEMU model supports one Virtual Interface(VI) per VM with 2 vCMDQs and 16 SIDs per VI, so the probe ensures the host implementation is compatible with these limits.

#22 lore raw

PATCH v4 11/31 hw/arm/tegra241-cmdqv: Implement CMDQV init

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages:

0x00000  (CMDQ-V Config page)
0x10000  (CMDQ-V CMDQ Page0)
0x20000  (CMDQ-V CMDQ Page1)
0x30000  (Virtual Interface Page0)
0x40000  (Virtual Interface Page1)

This patch wires up the Tegra241 CMDQV init callback and allocates vendor-specific CMDQV state. The state pointer is stored in SMMUv3AccelState for use by subsequent CMDQV operations.

The CMDQV MMIO region and a dedicated IRQ line are registered with the SMMUv3 device. The MMIO read/write handlers are currently stubs and will be implemented in later patches.

The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF error conditions. This patch only registers the IRQ line. Interrupt generation and propagation to the guest will be added in a subsequent patch.

---
 hw/arm/smmuv3-accel.h   |  1 +
 hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++--
 3 files changed, 47 insertions(+), 2 deletions(-)
hw/arm/smmuv3-accel.h +1 -0
index 86301afcb4..28bceca061 100644
@@ -45,6 +45,7 @@ typedef struct SMMUv3AccelState {
     bool auto_mode;
     bool auto_finalised;
     const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;  /* vendor specific CMDQV state */
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
hw/arm/tegra241-cmdqv.h +18 -0
index c1866084f8..2a34a4b6b4 100644
@@ -14,6 +14,24 @@
 #define CMDQV_NUM_CMDQ_LOG2       1
 #define CMDQV_NUM_SID_PER_VI_LOG2 4
 
+/*
+ * Tegra241 CMDQV MMIO layout (64KB pages)
+ *
+ * 0x00000  (CMDQ-V Config page)
+ * 0x10000  (CMDQ-V CMDQ Page0)
+ * 0x20000  (CMDQ-V CMDQ Page1)
+ * 0x30000  (Virtual Interface Page0)
+ * 0x40000  (Virtual Interface Page1)
+ */
+#define TEGRA241_CMDQV_IO_LEN 0x50000
+
+typedef struct Tegra241CMDQV {
+    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
+    SMMUv3AccelState *s_accel;
+    MemoryRegion mmio_cmdqv;
+    qemu_irq irq;
+} Tegra241CMDQV;
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +28 -2
index 3a19a1af56..ccd3c6d275 100644
@@ -13,6 +13,16 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+                                 unsigned size)
+{
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
 }
@@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
 }
 
+static const MemoryRegionOps mmio_cmdqv_ops = {
+    .read = tegra241_cmdqv_read,
+    .write = tegra241_cmdqv_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv;
+
+    cmdqv = g_new0(Tegra241CMDQV, 1);
+    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
+                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
+    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
+    sysbus_init_irq(sbd, &cmdqv->irq);
+    cmdqv->s_accel = accel;
+    accel->cmdqv = cmdqv;
+    return true;
 }
 
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Signature · 1 line
2.43.0
#23 lore raw

PATCH v4 12/31 hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus

In reply to Shameer Kolothum

SMMUv3 devices with acceleration may enable CMDQV extensions after device realize. In that case, additional MMIO regions and IRQ lines may be registered but not yet mapped to the platform bus.

Ensure SMMUv3 device resources are linked to the platform bus during machine_done().

This is safe to do unconditionally since the platform bus helpers skip resources that are already mapped.

---
 hw/arm/virt.c | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
hw/arm/virt.c +22 -0
index 68464ceb14..6c5e51af37 100644
@@ -1832,6 +1832,25 @@ static void virt_build_smbios(VirtMachineState *vms)
     }
 }
 
+/*
+ * SMMUv3 devices with acceleration may enable CMDQV extensions
+ * after device realize. In that case, additional MMIO regions and
+ * IRQ lines may be registered but not yet mapped to the platform bus.
+ *
+ * Ensure all resources are linked to the platform bus before final
+ * machine setup.
+ */
+
+static void virt_smmuv3_dev_link_cmdqv(VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        DeviceState *dev = g_ptr_array_index(vms->smmuv3_devices, i);
+
+        platform_bus_link_device(PLATFORM_BUS_DEVICE(vms->platform_bus_dev),
+                                 SYS_BUS_DEVICE(dev));
+    }
+}
+
 static
 void virt_machine_done(Notifier *notifier, void *data)
 {
@@ -1848,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)
     if (vms->cxl_devices_state.is_enabled) {
         cxl_fmws_link_targets(&error_fatal);
     }
+
+    virt_smmuv3_dev_link_cmdqv(vms);
+
     /*
      * If the user provided a dtb, we assume the dynamic sysbus nodes
      * already are integrated there. This corresponds to a use case where
Signature · 1 line
2.43.0
Reviewed-by on v4 12/31
#24 lore raw

Re: [PATCH v4 12/31] hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus

Quoted 4 lines · Shameer Kolothum

SMMUv3 devices with acceleration may enable CMDQV extensions after device realize. In that case, additional MMIO regions and IRQ lines may be registered but not yet mapped to the platform bus.

Quoted 8 lines

Ensure SMMUv3 device resources are linked to the platform bus during machine_done().

This is safe to do unconditionally since the platform bus helpers skip resources that are already mapped.

#25 lore raw

PATCH v4 13/31 hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.

Allocate a matching vEVENTQ together with the vIOMMU, since it is specific to the Tegra241 CMDQV vIOMMU and used to receive CMDQV events.

Free both objects on teardown.

---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 46 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 46 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +1 -0
index 2a34a4b6b4..fa0aa3ab04 100644
@@ -30,6 +30,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    IOMMUFDVeventq *veventq;
 } Tegra241CMDQV;
 
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
hw/arm/tegra241-cmdqv.c +45 -1

58 lines not on this page · raw message

Signature · 1 line
2.43.0
Reviewed-by on v4 13/31
#26 lore raw

Re: [PATCH v4 13/31] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 9 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.

Allocate a matching vEVENTQ together with the vIOMMU, since it is specific to the Tegra241 CMDQV vIOMMU and used to receive CMDQV events.

Quoted 5 lines

Free both objects on teardown.

Eric

Quoted 80 lines
---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 46 ++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 46 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +1 -0
index 2a34a4b6b4..fa0aa3ab04 100644
@@ -30,6 +30,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    IOMMUFDVeventq *veventq;
 } Tegra241CMDQV;
 
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
hw/arm/tegra241-cmdqv.c +45 -1
index ccd3c6d275..2f1084b55f 100644
@@ -25,13 +25,57 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+
+    if (!viommu) {
+        return;
+    }
+    if (veventq) {
+        close(veventq->veventq_fd);
+        iommufd_backend_free_id(viommu->iommufd, veventq->veventq_id);
+        g_free(veventq);
+        cmdqv->veventq = NULL;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                             uint32_t *out_viommu_id, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+    uint32_t viommu_id, veventq_id, veventq_fd;
+    IOMMUFDVeventq *veventq;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), &viommu_id,
+                                      errp)) {
+        return false;
+    }
+
+    if (!iommufd_backend_alloc_veventq(idev->iommufd, viommu_id,
+                                       IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                       1 << 16, &veventq_id, &veventq_fd,
+                                       errp)) {
+        error_append_hint(errp, "Tegra241 CMDQV: failed to alloc veventq");
+        goto free_viommu;
+    }
+
+    veventq = g_new(IOMMUFDVeventq, 1);
+    veventq->veventq_id = veventq_id;
+    veventq->veventq_fd = veventq_fd;
+    cmdqv->veventq = veventq;
+
+    *out_viommu_id = viommu_id;
+    return true;
+
+free_viommu:
+    iommufd_backend_free_id(idev->iommufd, viommu_id);
     return false;
 }
 
#27 lore raw

Re: [PATCH v4 13/31] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

In reply to Shameer Kolothum

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 63

Quoted 3 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +3 -0
+    if (!iommufd_backend_alloc_veventq(idev->iommufd, viommu_id,
+                                       IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                       1 << 16, &veventq_id, &veventq_fd,

I forgot why we use 16 here. But perhaps we could do SMMU_EVENTQS?

Nicolin

#28 lore raw

PATCH v4 14/31 hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes control and status registers in the CMDQ-V Config page (offset [0x0, 0x10000)) used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the CMDQ-V Config region ([CMDQV_BASE, CMDQV_CMDQ_BASE]), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the CMDQ-V Config space. Only VINTF0 is supported; VINTF1-63 are not.

---
 hw/arm/tegra241-cmdqv.h | 127 +++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 163 ++++++++++++++++++++++++++++++++++++++--
 hw/arm/trace-events     |   4 +
 3 files changed, 288 insertions(+), 6 deletions(-)
hw/arm/tegra241-cmdqv.h +127 -0
index fa0aa3ab04..965670066d 100644
@@ -10,10 +10,14 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+
 #define CMDQV_VER                 1
 #define CMDQV_NUM_CMDQ_LOG2       1
 #define CMDQV_NUM_SID_PER_VI_LOG2 4
 
+#define TEGRA241_CMDQV_MAX_CMDQ   (1U << CMDQV_NUM_CMDQ_LOG2)
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -31,8 +35,131 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* CMDQ-V Config page registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VI_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VI_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+/* SMMU_CMDQV_VI_ERR_MAP_0/1 definitions */
+#define A_VI_ERR_MAP_0 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+/* SMMU_CMDQV_VI_INT_MASK_0/1 definitions */
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+/* SMMU_CMDQV_CMDQ_ERR_MAP_0-3 definitions */
+#define A_CMDQ_ERR_MAP_0 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/*
+ * CMDQ_ALLOC_MAP: one entry per physical VCMDQ. Hardware supports up to 128
+ * entries (CMDQV_NUM_CMDQ_LOG2=7), but QEMU only exposes
+ * TEGRA241_CMDQV_MAX_CMDQ (=2) VCMDQs per VM so only entries 0 and 1 are
+ * defined here.
+ */
+/* 2 identical register entries */
+#define SMMU_CMDQV_CMDQ_ALLOC_MAP_(i)        \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(0)
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(1)
+
+
+/* Only VINTF0 is exposed to the guest; vintf = 0 */
+#define SMMU_CMDQV_VINTFi_CONFIG_(vi)                 \
+    REG32(VINTF##vi##_CONFIG, 0x1000 + vi * 0x100) \
+    FIELD(VINTF##vi##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##vi##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##vi##_CONFIG, HYP_OWN, 17, 1)
+
+SMMU_CMDQV_VINTFi_CONFIG_(0)
+
+#define SMMU_CMDQV_VINTFi_STATUS_(vi)                 \
+    REG32(VINTF##vi##_STATUS, 0x1004 + vi * 0x100) \
+    FIELD(VINTF##vi##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##vi##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##vi##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+SMMU_CMDQV_VINTFi_STATUS_(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_ERROR (1 << 1)
+
+/*
+ * SID_MATCH/SID_REPLACE: 16 entries per VINTF (CMDQV_NUM_SID_PER_VI_LOG2=4).
+ * vintf = 0, 16 identical register entries
+ */
+#define SMMU_CMDQV_VINTFi_SID_MATCH_(vi, j)                          \
+    REG32(VINTF##vi##_SID_MATCH_##j, 0x1040 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##vi##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+SMMU_CMDQV_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+SMMU_CMDQV_VINTFi_SID_MATCH_(0, 15)
+
+/* vintf = 0, 16 identical register entries */
+#define SMMU_CMDQV_VINTFi_SID_REPLACE_(vi, j)                          \
+    REG32(VINTF##vi##_SID_REPLACE_##j, 0x1080 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_SID_REPLACE_##j, PHYS_SID, 0, 20)
+
+SMMU_CMDQV_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+SMMU_CMDQV_VINTFi_SID_REPLACE_(0, 15)
+
+/*
+ * LVCMDQ_ERR_MAP: hardware defines 4 registers per VINTF (offset
+ * 0x10c0..0x10cc), each covering 32 logical VCMDQs. All 4 are accessible
+ * by the guest. With TEGRA241_CMDQV_MAX_CMDQ=2 only MAP_0 bits [1:0]
+ * carry meaningful error state; MAP_1..MAP_3 always read as 0.
+ * vintf = 0, 4 identical register entries
+ */
+#define SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(vi, j)                          \
+    REG32(VINTF##vi##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* MAP_1 and MAP_2 omitted; not referenced directly */
+SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +157 -6

184 lines not on this page · raw message

hw/arm/trace-events +4 -0
index 3457536fb0..8c61d66a26 100644
@@ -72,6 +72,10 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev
 smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64
 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x"
 
+# tegra241-cmdqv
+tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
+tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
+
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
 strongarm_ssp_read_underrun(void) "SSP rx underrun"
Signature · 1 line
2.43.0
#29 lore raw

Re: [PATCH v4 14/31] hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region

On Wed, Apr 15, 2026 at 11:55:35AM +0100, Shameer Kolothum wrote:

LGTM. Some nits:

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +0 -0
index fa0aa3ab04..965670066d 100644

hw/arm/tegra241-cmdqv.h line 19

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +3 -0
@@ -10,10 +10,14 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+
 #define CMDQV_VER                 1
 #define CMDQV_NUM_CMDQ_LOG2       1
 #define CMDQV_NUM_SID_PER_VI_LOG2 4
 
+#define TEGRA241_CMDQV_MAX_CMDQ   (1U << CMDQV_NUM_CMDQ_LOG2)

Maybe add:
#define TEGRA241_CMDQV_MAX_NUM_SID (1U << CMDQV_NUM_SID_PER_VI_LOG2)

Quoted 18 lines
hw/arm/tegra241-cmdqv.h +11 -0
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -31,8 +35,131 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;

[...]

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 50

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];

Then s/16/TEGRA241_CMDQV_MAX_NUM_SID

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 105

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+#define SMMU_CMDQV_CMDQ_ALLOC_MAP_(i)        \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(0)
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(1)
+
+

Can drop the extra line.

hw/arm/tegra241-cmdqv.h static void tegra241_cmdqv_reset(SMMUv3State *s) · line 240

Quoted 10 lines

static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

hw/arm/tegra241-cmdqv.h +2 -2
@@ -84,8 +235,8 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
 }
 
 static const MemoryRegionOps mmio_cmdqv_ops = {
-    .read = tegra241_cmdqv_read,
-    .write = tegra241_cmdqv_write,
+    .read = tegra241_cmdqv_read_mmio,
+    .write = tegra241_cmdqv_write_mmio,
     .endianness = DEVICE_LITTLE_ENDIAN,

This could squash to PATCH-11.

Nicolin

#30 lore raw

Re: [PATCH v4 14/31] hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region

In reply to Shameer Kolothum

Hi Shameer,

Quoted 74 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes control and status registers in the CMDQ-V Config page (offset [0x0, 0x10000)) used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the CMDQ-V Config region ([CMDQV_BASE, CMDQV_CMDQ_BASE]), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the CMDQ-V Config space. Only VINTF0 is supported; VINTF1-63 are not.

---
 hw/arm/tegra241-cmdqv.h | 127 +++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 163 ++++++++++++++++++++++++++++++++++++++--
 hw/arm/trace-events     |   4 +
 3 files changed, 288 insertions(+), 6 deletions(-)
hw/arm/tegra241-cmdqv.h +33 -0
index fa0aa3ab04..965670066d 100644
@@ -10,10 +10,14 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+
 #define CMDQV_VER                 1
 #define CMDQV_NUM_CMDQ_LOG2       1
 #define CMDQV_NUM_SID_PER_VI_LOG2 4
 
+#define TEGRA241_CMDQV_MAX_CMDQ   (1U << CMDQV_NUM_CMDQ_LOG2)
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -31,8 +35,131 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* CMDQ-V Config page registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VI_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VI_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 78

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+/* SMMU_CMDQV_VI_ERR_MAP_0/1 definitions */
+#define A_VI_ERR_MAP_0 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+/* SMMU_CMDQV_VI_INT_MASK_0/1 definitions */
+#define A_VI_INT_MASK 0x1c

pls use spec terminology: _0

Quoted 18 lines
hw/arm/tegra241-cmdqv.h +18 -0
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+/* SMMU_CMDQV_CMDQ_ERR_MAP_0-3 definitions */
+#define A_CMDQ_ERR_MAP_0 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/*
+ * CMDQ_ALLOC_MAP: one entry per physical VCMDQ. Hardware supports up to 128
+ * entries (CMDQV_NUM_CMDQ_LOG2=7), but QEMU only exposes
+ * TEGRA241_CMDQV_MAX_CMDQ (=2) VCMDQs per VM so only entries 0 and 1 are
+ * defined here.
+ */
+/* 2 identical register entries */
+#define SMMU_CMDQV_CMDQ_ALLOC_MAP_(i)        \

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 106

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(0)
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(1)
+
+
+/* Only VINTF0 is exposed to the guest; vintf = 0 */

is it supposed to evolve? Otherwise we can keep things simple and just define a single

SMMU_CMDQV_VINTF0_CONFIG_0 and same for the rest.

Quoted 14 lines
hw/arm/tegra241-cmdqv.h +14 -0
+#define SMMU_CMDQV_VINTFi_CONFIG_(vi)                 \
+    REG32(VINTF##vi##_CONFIG, 0x1000 + vi * 0x100) \
+    FIELD(VINTF##vi##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##vi##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##vi##_CONFIG, HYP_OWN, 17, 1)
+
+SMMU_CMDQV_VINTFi_CONFIG_(0)
+
+#define SMMU_CMDQV_VINTFi_STATUS_(vi)                 \
+    REG32(VINTF##vi##_STATUS, 0x1004 + vi * 0x100) \
+    FIELD(VINTF##vi##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##vi##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##vi##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+SMMU_CMDQV_VINTFi_STATUS_(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_ERROR (1 << 1)
+
+/*
+ * SID_MATCH/SID_REPLACE: 16 entries per VINTF (CMDQV_NUM_SID_PER_VI_LOG2=4).
+ * vintf = 0, 16 identical register entries
+ */
+#define SMMU_CMDQV_VINTFi_SID_MATCH_(vi, j)  

this could also be simplified

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5 -0
                        \
+    REG32(VINTF##vi##_SID_MATCH_##j, 0x1040 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##vi##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+SMMU_CMDQV_VINTFi_SID_MATCH_(0, 0)

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 145

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+/* Omitting [0][1~14] as not being directly called */
+SMMU_CMDQV_VINTFi_SID_MATCH_(0, 15)
+
+/* vintf = 0, 16 identical register entries */
+#define SMMU_CMDQV_VINTFi_SID_REPLACE_(vi, j)                          \
+    REG32(VINTF##vi##_SID_REPLACE_##j, 0x1080 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_SID_REPLACE_##j, PHYS_SID, 0, 20)
+
+SMMU_CMDQV_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */

why?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+SMMU_CMDQV_VINTFi_SID_REPLACE_(0, 15)
+
+/*
+ * LVCMDQ_ERR_MAP: hardware defines 4 registers per VINTF (offset
+ * 0x10c0..0x10cc), each covering 32 logical VCMDQs. All 4 are accessible
+ * by the guest. With TEGRA241_CMDQV_MAX_CMDQ=2 only MAP_0 bits [1:0]
+ * carry meaningful error state; MAP_1..MAP_3 always read as 0.
+ * vintf = 0, 4 identical register entries
+ */
+#define SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(vi, j)  

Again I won't add this extra complexity, get rid of vi everywhere for now.

Quoted 206 lines
hw/arm/tegra241-cmdqv.h +7 -0
                        \
+    REG32(VINTF##vi##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* MAP_1 and MAP_2 omitted; not referenced directly */
+SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);

#endif /* HW_ARM_TEGRA241_CMDQV_H */

hw/arm/tegra241-cmdqv.c +157 -6
index 2f1084b55f..3b08ed0ff3 100644
@@ -8,19 +8,170 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
+#include "trace.h"
 
-static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+static uint64_t tegra241_cmdqv_config_vintf_read(Tegra241CMDQV *cmdqv,
+                                                 hwaddr offset)
 {
-    return 0;
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        /*
+         * GLB_FILT_CFG_0 (offset 0xC) and GLB_FILT_DATA_0 (offset 0x10) are
+         * filter config and filter data registers. They are not required for
+         * normal VINTF operation and are not emulated.
+         */
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,
+                                              hwaddr offset, uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /*
+         * Mask out HYP_OWN on guest writes. This bit selects Hypervisor (1) vs
+         * Guest (0) ownership of the CMDQ. Force it to 0 so the VINTF always
+         * remains guest-owned.
+         */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        /*
+         * GLB_FILT_CFG_0 (offset 0xC) and GLB_FILT_DATA_0 (offset 0x10) are
+         * filter config and filter data registers. They are not required for
+         * normal VINTF operation and are not emulated.
+         */
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
+}
+
+static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,
+                                         unsigned size)
+{
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    uint64_t val = 0;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x%x)\n", __func__,
+                      offset, TEGRA241_CMDQV_IO_LEN);
+        goto out;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        val = cmdqv->config;
+        break;
+    case A_PARAM:
+        val = cmdqv->param;
+        break;
+    case A_STATUS:
+        val = cmdqv->status;
+        break;
+    case A_VI_ERR_MAP_0 ... A_VI_ERR_MAP_1:
+        val = cmdqv->vi_err_map[(offset - A_VI_ERR_MAP_0) / 4];
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        val = cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+        break;
+    case A_CMDQ_ERR_MAP_0 ... A_CMDQ_ERR_MAP_3:
+        val = cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP_0) / 4];
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        val = cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        val = tegra241_cmdqv_config_vintf_read(cmdqv, offset);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
+
+out:
+    trace_tegra241_cmdqv_read_mmio(offset, val, size);
+    return val;
 }
 
-static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
-                                 unsigned size)
+static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
+                                      uint64_t value, unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x%x)\n", __func__,
+                      offset, TEGRA241_CMDQV_IO_LEN);
+        goto out;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_config_vintf_write(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
+
+out:
+    trace_tegra241_cmdqv_write_mmio(offset, value, size);
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
@@ -84,8 +235,8 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
 }
 
 static const MemoryRegionOps mmio_cmdqv_ops = {
-    .read = tegra241_cmdqv_read,
-    .write = tegra241_cmdqv_write,
+    .read = tegra241_cmdqv_read_mmio,
+    .write = tegra241_cmdqv_write_mmio,
     .endianness = DEVICE_LITTLE_ENDIAN,
 };
 
hw/arm/trace-events +0 -0
index 3457536fb0..8c61d66a26 100644

hw/arm/trace-events smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev · line 81

Quoted 10 lines
hw/arm/trace-events +4 -0
@@ -72,6 +72,10 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev
 smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64
 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x"
 
+# tegra241-cmdqv
+tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
+tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
+
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
 strongarm_ssp_read_underrun(void) "SSP rx underrun"

Thanks

Eric

#31 lore raw

RE: [PATCH v4 14/31] hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region

Quoted 101 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 05 May 2026 08:27
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 14/31] hw/arm/tegra241-cmdqv: Emulate CMDQ-V Config region

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes control and status registers in the CMDQ-V Config page (offset [0x0, 0x10000)) used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the CMDQ-V Config region ([CMDQV_BASE, CMDQV_CMDQ_BASE]), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the CMDQ-V Config space. Only VINTF0 is supported; VINTF1-63 are not.

Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.h | 127 +++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 163

++++++++++++++++++++++++++++++++++++++--

Quoted 8 lines
 hw/arm/trace-events     |   4 +
 3 files changed, 288 insertions(+), 6 deletions(-)
hw/arm/tegra241-cmdqv.h +0 -0
index fa0aa3ab04..965670066d 100644

hw/arm/tegra241-cmdqv.h line 19

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +3 -0
@@ -10,10 +10,14 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+
 #define CMDQV_VER                 1
 #define CMDQV_NUM_CMDQ_LOG2       1
 #define CMDQV_NUM_SID_PER_VI_LOG2 4
 
+#define TEGRA241_CMDQV_MAX_CMDQ   (1U <<

CMDQV_NUM_CMDQ_LOG2)

Quoted 42 lines
hw/arm/tegra241-cmdqv.h +33 -0
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -31,8 +35,131 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* CMDQ-V Config page registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VI_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VI_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+/* SMMU_CMDQV_VI_ERR_MAP_0/1 definitions */
Quoted 8 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 78

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+#define A_VI_ERR_MAP_0 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+/* SMMU_CMDQV_VI_INT_MASK_0/1 definitions */
+#define A_VI_INT_MASK 0x1c

pls use spec terminology: _0

Sure.

Quoted 18 lines
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +2 -0
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+#define V_VI_INT_MASK_MASKED (1)
+
+/* SMMU_CMDQV_CMDQ_ERR_MAP_0-3 definitions */
+#define A_CMDQ_ERR_MAP_0 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/*
+ * CMDQ_ALLOC_MAP: one entry per physical VCMDQ. Hardware supports

up to 128

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 92

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+ * entries (CMDQV_NUM_CMDQ_LOG2=7), but QEMU only exposes
+ * TEGRA241_CMDQV_MAX_CMDQ (=2) VCMDQs per VM so only entries 0

and 1 are

Quoted 16 lines
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+ * defined here.
+ */
+/* 2 identical register entries */
+#define SMMU_CMDQV_CMDQ_ALLOC_MAP_(i)        \

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 106

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(0)
+SMMU_CMDQV_CMDQ_ALLOC_MAP_(1)
+
+
+/* Only VINTF0 is exposed to the guest; vintf = 0 */

is it supposed to evolve? Otherwise we can keep things simple and just define a single

I don’t think it will in near future. Will change to a single VINTF.

Quoted 38 lines

SMMU_CMDQV_VINTF0_CONFIG_0 and same for the rest.

Quoted 11 lines
hw/arm/tegra241-cmdqv.h +11 -0
+#define SMMU_CMDQV_VINTFi_CONFIG_(vi)                 \
+    REG32(VINTF##vi##_CONFIG, 0x1000 + vi * 0x100) \
+    FIELD(VINTF##vi##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##vi##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##vi##_CONFIG, HYP_OWN, 17, 1)
+
+SMMU_CMDQV_VINTFi_CONFIG_(0)
+
+#define SMMU_CMDQV_VINTFi_STATUS_(vi)                 \
+    REG32(VINTF##vi##_STATUS, 0x1004 + vi * 0x100) \
+    FIELD(VINTF##vi##_STATUS, ENABLE_OK, 0, 1)    \

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+    FIELD(VINTF##vi##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##vi##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+SMMU_CMDQV_VINTFi_STATUS_(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_ERROR (1 << 1)
+
+/*
+ * SID_MATCH/SID_REPLACE: 16 entries per VINTF

(CMDQV_NUM_SID_PER_VI_LOG2=4).

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3 -0
+ * vintf = 0, 16 identical register entries
+ */
+#define SMMU_CMDQV_VINTFi_SID_MATCH_(vi, j)

this could also be simplified

Quoted 9 lines
hw/arm/tegra241-cmdqv.h +8 -0
                        \
+    REG32(VINTF##vi##_SID_MATCH_##j, 0x1040 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##vi##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+SMMU_CMDQV_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+SMMU_CMDQV_VINTFi_SID_MATCH_(0, 15)
+
Quoted 8 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 145

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+/* vintf = 0, 16 identical register entries */
+#define SMMU_CMDQV_VINTFi_SID_REPLACE_(vi, j)                          \
+    REG32(VINTF##vi##_SID_REPLACE_##j, 0x1080 + j * 4 + vi * 0x100) \
+    FIELD(VINTF##vi##_SID_REPLACE_##j, PHYS_SID, 0, 20)
+
+SMMU_CMDQV_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */

why?

Because the only use in switch uses range bounds:
A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15

I will make it clear.

Quoted 7 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+SMMU_CMDQV_VINTFi_SID_REPLACE_(0, 15)
+
+/*
+ * LVCMDQ_ERR_MAP: hardware defines 4 registers per VINTF (offset
+ * 0x10c0..0x10cc), each covering 32 logical VCMDQs. All 4 are accessible
+ * by the guest. With TEGRA241_CMDQV_MAX_CMDQ=2 only MAP_0 bits

[1:0]

Quoted 5 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+ * carry meaningful error state; MAP_1..MAP_3 always read as 0.
+ * vintf = 0, 4 identical register entries
+ */
+#define SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(vi, j)

Again I won't add this extra complexity, get rid of vi everywhere for now.

Ok.

Thanks,
Shameer

#32 lore raw

PATCH v4 15/31 hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO apertures:

CMDQV_CMDQ_BASE (0x10000/0x20000): VCMDQ Page0/Page1 CMDQV_VI_CMDQ_BASE (0x30000/0x40000): VINTF VCMDQ Page0/Page1

VINTF Page0 (0x30000) and VCMDQ Page0 (0x10000) are hardware aliases addressing the same underlying registers. Add read emulation for both apertures, backed by a register cache. VINTF Page0 reads are translated to their VCMDQ Page0 equivalent and served from the same cached state.

Once IOMMU_HW_QUEUE_ALLOC and viommu_mmap are wired up in a subsequent patch, Page0 register reads will be served directly from the hardware backed mmap'd page instead of the cache. Page1 registers are always served from cache.

---
 hw/arm/tegra241-cmdqv.h | 185 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c |  73 ++++++++++++++++
 2 files changed, 258 insertions(+)
hw/arm/tegra241-cmdqv.h +185 -0
index 965670066d..b8bd8cd8ff 100644
@@ -29,6 +29,13 @@
  */
 #define TEGRA241_CMDQV_IO_LEN 0x50000
 
+/* CMDQV MMIO aperture bases and VCMDQ stride */
+#define CMDQV_VCMDQ_PAGE0_BASE  0x10000  /* CMDQV_CMDQ_BASE */
+#define CMDQV_VCMDQ_PAGE1_BASE  0x20000
+#define CMDQV_VINTF_PAGE0_BASE  0x30000  /* CMDQV_VI_CMDQ_BASE */
+#define CMDQV_VINTF_PAGE1_BASE  0x40000
+#define CMDQV_VCMDQ_STRIDE      0x80
+
 typedef struct Tegra241CMDQV {
     struct iommu_viommu_tegra241_cmdqv cmdqv_data;
     SMMUv3AccelState *s_accel;
@@ -49,6 +56,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* CMDQ-V Config page registers (offset 0x00000) */
@@ -160,6 +175,176 @@ SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* MAP_1 and MAP_2 omitted; not referenced directly */
 SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
+ */
+#define A_VCMDQi_CONS_INDX(i)                       \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)                             \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)                             \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)
+
+#define A_VCMDQi_BASE_L(i)                       \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
+ */
+#define A_VI_VCMDQi_CONS_INDX(i)                       \
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +73 -0

91 lines not on this page · raw message

Signature · 1 line
2.43.0
#33 lore raw

Re: [PATCH v4 15/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads

Hi,

Quoted 6 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO apertures:

CMDQV_CMDQ_BASE (0x10000/0x20000): VCMDQ Page0/Page1

(global)

Quoted 1 line

CMDQV_VI_CMDQ_BASE (0x30000/0x40000): VINTF VCMDQ Page0/Page1

(logical)

Quoted 2 lines

VINTF Page0 (0x30000) and VCMDQ Page0 (0x10000) are hardware aliases

what about page1?

Quoted 8 lines

addressing the same underlying registers. Add read emulation for both apertures, backed by a register cache. VINTF Page0 reads are translated to their VCMDQ Page0 equivalent and served from the same cached state.

Once IOMMU_HW_QUEUE_ALLOC and viommu_mmap are wired up in a subsequent patch, Page0 register reads will be served directly from the hardware backed mmap'd page instead of the cache. Page1 registers are always served from cache.

I would add add that
Page 0 contains VCMDQ control and status registers while Page 1 contains VCMDQ base and DRAM address registers

I would add Nicolin's explanation
"

The global page 0 programmable at any time so long as CMDQV_EN is enabled.

The logical (VINTF) page 0 are programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
"

Quoted 26 lines
---
 hw/arm/tegra241-cmdqv.h | 185 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c |  73 ++++++++++++++++
 2 files changed, 258 insertions(+)
hw/arm/tegra241-cmdqv.h +7 -0
index 965670066d..b8bd8cd8ff 100644
@@ -29,6 +29,13 @@
  */
 #define TEGRA241_CMDQV_IO_LEN 0x50000
 
+/* CMDQV MMIO aperture bases and VCMDQ stride */
+#define CMDQV_VCMDQ_PAGE0_BASE  0x10000  /* CMDQV_CMDQ_BASE */
+#define CMDQV_VCMDQ_PAGE1_BASE  0x20000
+#define CMDQV_VINTF_PAGE0_BASE  0x30000  /* CMDQV_VI_CMDQ_BASE */
+#define CMDQV_VINTF_PAGE1_BASE  0x40000
+#define CMDQV_VCMDQ_STRIDE      0x80
+
 typedef struct Tegra241CMDQV {
     struct iommu_viommu_tegra241_cmdqv cmdqv_data;
     SMMUv3AccelState *s_accel;

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 58

Quoted 3 lines
hw/arm/tegra241-cmdqv.h +0 -0
@@ -49,6 +56,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];

I would add a comment explaining that those cached registers store the consistent/idenftical values for both the global and logical regs (they are same)

Quoted 12 lines
hw/arm/tegra241-cmdqv.h +8 -0
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* CMDQ-V Config page registers (offset 0x00000) */

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 182

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +5 -0
@@ -160,6 +175,176 @@ SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* MAP_1 and MAP_2 omitted; not referenced directly */
 SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers

Can you clearly separate regs that belong to page 0 from regs that belong to page 1

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 184

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+ */
+#define A_VCMDQi_CONS_INDX(i)        

same remark as for config registers. Please use the spec terminology

SMMU_CMDQV_VCMDQi_CONS_INDX_0

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +4 -0
               \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)      

SMMU_CMDQV_VCMDQi_PROD_INDX_0

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
                       \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)           

SMMU_CMDQV_VCMDQi_CONFIG_0(i)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 209

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +1 -0
                  \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \

0x80, isn't it the stride you defined earlier, ie

CMDQV_VCMDQ_STRIDE ?

Quoted 19 lines
hw/arm/tegra241-cmdqv.h +19 -0
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 235

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)

/* Page 1 */

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 240

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+
+#define A_VCMDQi_BASE_L(i)         

ditto remove the A_ prefix and use spec terminology

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 246

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9 -0
              \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \

may instead of using 0x20000 define macros for the base address of each region

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +8 -0
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 264

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)

I would group all regs definition par page. The layout would look clearer to me.

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 270

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers

same here, pls separate page 0 and page 1 definitions

Quoted 2 lines

+ */
+#define A_VI_VCMDQi_CONS_INDX(i)

A_

Quoted 164 lines
                 \
hw/arm/tegra241-cmdqv.h +75 -0
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +63 -0
index 3b08ed0ff3..35e6f0bbd6 100644
@@ -15,6 +15,46 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_config_vintf_read(Tegra241CMDQV *cmdqv,
                                                  hwaddr offset)
 {
@@ -92,6 +132,7 @@ static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
     uint64_t val = 0;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -125,6 +166,38 @@ static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         val = tegra241_cmdqv_config_vintf_read(cmdqv, offset);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VINTF Page0 registers have the same per-VCMDQ layout as the
+         * VCMDQ Page0 registers. Translate the VINTF aperture offset to the
+         * equivalent VCMDQ aperture offset, then fall through to reuse the
+         * common VCMDQ decoding logic below.
+         */
+        offset -= CMDQV_VINTF_PAGE0_BASE - CMDQV_VCMDQ_PAGE0_BASE;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
+         * occupies a CMDQV_VCMDQ_STRIDE-byte window within the page.
+         *
+         * Extract the VCMDQ index and normalize to the VCMDQ0_* register
+         * offset. A single helper services all instances via @index.
+         */
+        index = (offset - CMDQV_VCMDQ_PAGE0_BASE) / CMDQV_VCMDQ_STRIDE;
+        return tegra241_cmdqv_read_vcmdq(cmdqv,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset, · line 200

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+                offset - index * CMDQV_VCMDQ_STRIDE, index);
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
+        offset -= CMDQV_VINTF_PAGE1_BASE - CMDQV_VCMDQ_PAGE1_BASE;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as VCMDQ Page0 case above */
+        index = (offset - CMDQV_VCMDQ_PAGE1_BASE) / CMDQV_VCMDQ_STRIDE;
+        return tegra241_cmdqv_read_vcmdq(cmdqv,
+                offset - index * CMDQV_VCMDQ_STRIDE, index);

Please add trace points for read & write accesses

Thanks

Eric

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
                       __func__, offset);
#34 lore raw

RE: [PATCH v4 15/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads

Quoted 28 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 05 May 2026 11:13
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 15/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads

External email: Use caution opening links or attachments

Hi,

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two

MMIO

Quoted 2 lines

apertures:

Quoted 8 lines
Quoted 1 line

CMDQV_CMDQ_BASE (0x10000/0x20000): VCMDQ Page0/Page1

(global)

Quoted 1 line

CMDQV_VI_CMDQ_BASE (0x30000/0x40000): VINTF VCMDQ

Page0/Page1
(logical)

Quoted 2 lines

VINTF Page0 (0x30000) and VCMDQ Page0 (0x10000) are hardware aliases

what about page1?

Right. Page1 is also an alias, but kernel only provides mmap support for Page0. Hence Page1 is always served from register cache. I will update.

Quoted 18 lines
Quoted 5 lines

addressing the same underlying registers. Add read emulation for both apertures, backed by a register cache. VINTF Page0 reads are translated to their VCMDQ Page0 equivalent and served from the same cached state.

Once IOMMU_HW_QUEUE_ALLOC and viommu_mmap are wired up in a

subsequent

Quoted 3 lines

patch, Page0 register reads will be served directly from the hardware backed mmap'd page instead of the cache. Page1 registers are always served from cache.

I would add add that
Page 0 contains VCMDQ control and status registers while Page 1 contains VCMDQ base and DRAM address registers

I would add Nicolin's explanation
"

Quoted 8 lines

The global page 0 programmable at any time so long as CMDQV_EN is enabled.

The logical (VINTF) page 0 are programmable only when SW allocates and maps
global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
"

Ok. Will add.

Quoted 353 lines
Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.h | 185

++++++++++++++++++++++++++++++++++++++++

hw/arm/tegra241-cmdqv.h line 33

Quoted 13 lines
 hw/arm/tegra241-cmdqv.c |  73 ++++++++++++++++
 2 files changed, 258 insertions(+)
hw/arm/tegra241-cmdqv.h +2 -0
index 965670066d..b8bd8cd8ff 100644
@@ -29,6 +29,13 @@
  */
 #define TEGRA241_CMDQV_IO_LEN 0x50000
 
+/* CMDQV MMIO aperture bases and VCMDQ stride */
+#define CMDQV_VCMDQ_PAGE0_BASE  0x10000  /* CMDQV_CMDQ_BASE

*/

hw/arm/tegra241-cmdqv.h line 35

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+#define CMDQV_VCMDQ_PAGE1_BASE  0x20000
+#define CMDQV_VINTF_PAGE0_BASE  0x30000  /*

CMDQV_VI_CMDQ_BASE */

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 58

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +3 -0
+#define CMDQV_VINTF_PAGE1_BASE  0x40000
+#define CMDQV_VCMDQ_STRIDE      0x80
+
 typedef struct Tegra241CMDQV {
     struct iommu_viommu_tegra241_cmdqv cmdqv_data;
     SMMUv3AccelState *s_accel;
@@ -49,6 +56,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];

I would add a comment explaining that those cached registers store the consistent/idenftical values for both the global and logical regs (they are same)

hw/arm/tegra241-cmdqv.h

Quoted 12 lines
hw/arm/tegra241-cmdqv.h +8 -0
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* CMDQ-V Config page registers (offset 0x00000) */
@@ -160,6 +175,176 @@

SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 182

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +5 -0
 /* MAP_1 and MAP_2 omitted; not referenced directly */
 SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers

Can you clearly separate regs that belong to page 0 from regs that belong to page 1

hw/arm/tegra241-cmdqv.h line 184

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+ */
+#define A_VCMDQi_CONS_INDX(i)

same remark as for config registers. Please use the spec terminology

SMMU_CMDQV_VCMDQi_CONS_INDX_0

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +4 -0
               \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)

SMMU_CMDQV_VCMDQi_PROD_INDX_0

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
                       \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)

SMMU_CMDQV_VCMDQi_CONFIG_0(i)

hw/arm/tegra241-cmdqv.h line 209

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +1 -0
                  \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \

0x80, isn't it the stride you defined earlier, ie

CMDQV_VCMDQ_STRIDE ?

Quoted 19 lines
hw/arm/tegra241-cmdqv.h +19 -0
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 235

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)

/* Page 1 */

hw/arm/tegra241-cmdqv.h line 240

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+
+#define A_VCMDQi_BASE_L(i)

ditto remove the A_ prefix and use spec terminology

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 246

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9 -0
              \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \

may instead of using 0x20000 define macros for the base address of each region

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i *

0x80) \

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i *

0x80) \

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 264

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)

I would group all regs definition par page. The layout would look clearer to me.

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 270

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers

same here, pls separate page 0 and page 1 definitions

Quoted 2 lines

+ */
+#define A_VI_VCMDQi_CONS_INDX(i)

A_

Quoted 54 lines
                 \
hw/arm/tegra241-cmdqv.h +53 -0
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i

* 0x80) \

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i

* 0x80) \

Quoted 15 lines
hw/arm/tegra241-cmdqv.h +5 -0
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index 3b08ed0ff3..35e6f0bbd6 100644
@@ -15,6 +15,46 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"

hw/arm/tegra241-cmdqv.c line 26

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,

hwaddr offset0,

Quoted 22 lines
hw/arm/tegra241-cmdqv.c +22 -0
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_config_vintf_read(Tegra241CMDQV

*cmdqv,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
                                                  hwaddr offset)
 {
@@ -92,6 +132,7 @@ static uint64_t tegra241_cmdqv_read_mmio(void

*opaque, hwaddr offset,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
     uint64_t val = 0;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -125,6 +166,38 @@ static uint64_t tegra241_cmdqv_read_mmio(void

*opaque, hwaddr offset,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +8 -0
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         val = tegra241_cmdqv_config_vintf_read(cmdqv, offset);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VINTF Page0 registers have the same per-VCMDQ layout as the
+         * VCMDQ Page0 registers. Translate the VINTF aperture offset to the
+         * equivalent VCMDQ aperture offset, then fall through to reuse the
+         * common VCMDQ decoding logic below.
+         */
+        offset -= CMDQV_VINTF_PAGE0_BASE -

CMDQV_VCMDQ_PAGE0_BASE;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each

VCMDQ

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+         * occupies a CMDQV_VCMDQ_STRIDE-byte window within the page.
+         *
+         * Extract the VCMDQ index and normalize to the VCMDQ0_* register
+         * offset. A single helper services all instances via @index.
+         */
+        index = (offset - CMDQV_VCMDQ_PAGE0_BASE) /

CMDQV_VCMDQ_STRIDE;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+        return tegra241_cmdqv_read_vcmdq(cmdqv,
+                offset - index * CMDQV_VCMDQ_STRIDE, index);
+    case A_VI_VCMDQ0_BASE_L ...

A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void · line 194

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+        /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
+        offset -= CMDQV_VINTF_PAGE1_BASE -

CMDQV_VCMDQ_PAGE1_BASE;

Quoted 8 lines

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as VCMDQ Page0 case above */
+        index = (offset - CMDQV_VCMDQ_PAGE1_BASE) /

CMDQV_VCMDQ_STRIDE;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void · line 200

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+        return tegra241_cmdqv_read_vcmdq(cmdqv,
+                offset - index * CMDQV_VCMDQ_STRIDE, index);

Please add trace points for read & write accesses

Just to clarify, we already have trace_tegra241_cmdqv_read_mmio and trace_tegra241_cmdqv_write_mmio, so this suggestion is for to have another pair for vcmdq read/write, right?

(Agree with all other suggestions above)

Thanks,
Shameer

#35 lore raw

Re: [PATCH v4 15/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads

Quoted 419 lines · Shameer Kolothum Thodi
Quoted 27 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 05 May 2026 11:13
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 15/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register reads

External email: Use caution opening links or attachments

Hi,

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two

MMIO

Quoted 2 lines

apertures:

Quoted 7 lines
Quoted 1 line

CMDQV_CMDQ_BASE (0x10000/0x20000): VCMDQ Page0/Page1

(global)

Quoted 1 line

CMDQV_VI_CMDQ_BASE (0x30000/0x40000): VINTF VCMDQ

Page0/Page1
(logical)

Quoted 1 line

VINTF Page0 (0x30000) and VCMDQ Page0 (0x10000) are hardware aliases

what about page1?

Right. Page1 is also an alias, but kernel only provides mmap support for Page0. Hence Page1 is always served from register cache. I will update.

Quoted 17 lines
Quoted 5 lines

addressing the same underlying registers. Add read emulation for both apertures, backed by a register cache. VINTF Page0 reads are translated to their VCMDQ Page0 equivalent and served from the same cached state.

Once IOMMU_HW_QUEUE_ALLOC and viommu_mmap are wired up in a

subsequent

Quoted 3 lines

patch, Page0 register reads will be served directly from the hardware backed mmap'd page instead of the cache. Page1 registers are always served from cache.

I would add add that
Page 0 contains VCMDQ control and status registers while Page 1 contains VCMDQ base and DRAM address registers

I would add Nicolin's explanation
"

Quoted 8 lines

The global page 0 programmable at any time so long as CMDQV_EN is enabled.

The logical (VINTF) page 0 are programmable only when SW allocates and maps
global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
"

Ok. Will add.

Quoted 355 lines
Quoted 4 lines
---
 hw/arm/tegra241-cmdqv.h | 185

++++++++++++++++++++++++++++++++++++++++

hw/arm/tegra241-cmdqv.h line 33

Quoted 13 lines
 hw/arm/tegra241-cmdqv.c |  73 ++++++++++++++++
 2 files changed, 258 insertions(+)
hw/arm/tegra241-cmdqv.h +2 -0
index 965670066d..b8bd8cd8ff 100644
@@ -29,6 +29,13 @@
  */
 #define TEGRA241_CMDQV_IO_LEN 0x50000
 
+/* CMDQV MMIO aperture bases and VCMDQ stride */
+#define CMDQV_VCMDQ_PAGE0_BASE  0x10000  /* CMDQV_CMDQ_BASE

*/

hw/arm/tegra241-cmdqv.h line 35

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+#define CMDQV_VCMDQ_PAGE1_BASE  0x20000
+#define CMDQV_VINTF_PAGE0_BASE  0x30000  /*

CMDQV_VI_CMDQ_BASE */

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 58

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +3 -0
+#define CMDQV_VINTF_PAGE1_BASE  0x40000
+#define CMDQV_VCMDQ_STRIDE      0x80
+
 typedef struct Tegra241CMDQV {
     struct iommu_viommu_tegra241_cmdqv cmdqv_data;
     SMMUv3AccelState *s_accel;
@@ -49,6 +56,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];

I would add a comment explaining that those cached registers store the consistent/idenftical values for both the global and logical regs (they are same)

hw/arm/tegra241-cmdqv.h

Quoted 12 lines
hw/arm/tegra241-cmdqv.h +8 -0
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* CMDQ-V Config page registers (offset 0x00000) */
@@ -160,6 +175,176 @@

SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 182

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +5 -0
 /* MAP_1 and MAP_2 omitted; not referenced directly */
 SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers

Can you clearly separate regs that belong to page 0 from regs that belong to page 1

hw/arm/tegra241-cmdqv.h line 184

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+ */
+#define A_VCMDQi_CONS_INDX(i)

same remark as for config registers. Please use the spec terminology

SMMU_CMDQV_VCMDQi_CONS_INDX_0

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +4 -0
               \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)

SMMU_CMDQV_VCMDQi_PROD_INDX_0

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
                       \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)

SMMU_CMDQV_VCMDQi_CONFIG_0(i)

hw/arm/tegra241-cmdqv.h line 209

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +1 -0
                  \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \

0x80, isn't it the stride you defined earlier, ie

CMDQV_VCMDQ_STRIDE ?

Quoted 19 lines
hw/arm/tegra241-cmdqv.h +19 -0
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 235

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)

/* Page 1 */

hw/arm/tegra241-cmdqv.h line 240

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+
+#define A_VCMDQi_BASE_L(i)

ditto remove the A_ prefix and use spec terminology

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 246

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9 -0
              \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \

may instead of using 0x20000 define macros for the base address of each region

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i *

0x80) \

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i *

0x80) \

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 264

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)

I would group all regs definition par page. The layout would look clearer to me.

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 270

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers

same here, pls separate page 0 and page 1 definitions

Quoted 2 lines

+ */
+#define A_VI_VCMDQi_CONS_INDX(i)

A_

Quoted 54 lines
                 \
hw/arm/tegra241-cmdqv.h +53 -0
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i

* 0x80) \

hw/arm/tegra241-cmdqv.h SMMU_CMDQV_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i

* 0x80) \

Quoted 15 lines
hw/arm/tegra241-cmdqv.h +5 -0
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index 3b08ed0ff3..35e6f0bbd6 100644
@@ -15,6 +15,46 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"

hw/arm/tegra241-cmdqv.c line 26

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,

hwaddr offset0,

Quoted 22 lines
hw/arm/tegra241-cmdqv.c +22 -0
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_config_vintf_read(Tegra241CMDQV

*cmdqv,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
                                                  hwaddr offset)
 {
@@ -92,6 +132,7 @@ static uint64_t tegra241_cmdqv_read_mmio(void

*opaque, hwaddr offset,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
     uint64_t val = 0;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -125,6 +166,38 @@ static uint64_t tegra241_cmdqv_read_mmio(void

*opaque, hwaddr offset,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +8 -0
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         val = tegra241_cmdqv_config_vintf_read(cmdqv, offset);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VINTF Page0 registers have the same per-VCMDQ layout as the
+         * VCMDQ Page0 registers. Translate the VINTF aperture offset to the
+         * equivalent VCMDQ aperture offset, then fall through to reuse the
+         * common VCMDQ decoding logic below.
+         */
+        offset -= CMDQV_VINTF_PAGE0_BASE -

CMDQV_VCMDQ_PAGE0_BASE;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each

VCMDQ

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+         * occupies a CMDQV_VCMDQ_STRIDE-byte window within the page.
+         *
+         * Extract the VCMDQ index and normalize to the VCMDQ0_* register
+         * offset. A single helper services all instances via @index.
+         */
+        index = (offset - CMDQV_VCMDQ_PAGE0_BASE) /

CMDQV_VCMDQ_STRIDE;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void *opaque, hwaddr offset,

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+        return tegra241_cmdqv_read_vcmdq(cmdqv,
+                offset - index * CMDQV_VCMDQ_STRIDE, index);
+    case A_VI_VCMDQ0_BASE_L ...

A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void · line 194

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+        /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
+        offset -= CMDQV_VINTF_PAGE1_BASE -

CMDQV_VCMDQ_PAGE1_BASE;

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as VCMDQ Page0 case above */
+        index = (offset - CMDQV_VCMDQ_PAGE1_BASE) /
Quoted 7 lines
Quoted 4 lines

CMDQV_VCMDQ_STRIDE;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_mmio(void · line 200

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+        return tegra241_cmdqv_read_vcmdq(cmdqv,
+                offset - index * CMDQV_VCMDQ_STRIDE, index);

Please add trace points for read & write accesses

Just to clarify, we already have trace_tegra241_cmdqv_read_mmio and trace_tegra241_cmdqv_write_mmio, so this suggestion is for to have another pair for vcmdq read/write, right?

nope I did not see that

trace_tegra241_cmdqv_write_mmio also applied to those accesses.
To be honest I would have prefered to have different traces points for different types of MMIO (CMDQV Config, VCMDQ P0 (when not mmaped), VCMDQ P1) to easily differentiate them but if you fell that's too heavy that's enough.

Eric

Quoted 5 lines

(Agree with all other suggestions above)

Thanks,
Shameer

#36 lore raw

PATCH v4 16/31 hw/arm/tegra241-cmdqv: Emulate VCMDQ register writes

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation. Add write handling for both CMDQV_CMDQ_BASE and CMDQV_VI_CMDQ_BASE apertures using the same index decoding and VINTF-to-VCMDQ translation logic as the read path.

VINTF aperture writes are translated to their CMDQV_CMDQ_BASE equivalent and update the same cached state. Page1 registers (BASE, CONS_INDX_BASE) always update the cache. Once IOMMU_HW_QUEUE_ALLOC and viommu_mmap are wired up in a subsequent patch, Page0 register writes will be forwarded to the hardware-backed mmap'd page.

---
 hw/arm/tegra241-cmdqv.c | 99 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
hw/arm/tegra241-cmdqv.c +99 -0
index 35e6f0bbd6..d4ba2ada92 100644
@@ -55,6 +55,70 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                       int index, uint64_t value,
+                                       unsigned size)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        } else {
+            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_BASE_H:
+        cmdqv->vcmdq_base[index] =
+            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        if (size == 8) {
+            cmdqv->vcmdq_cons_indx_base[index] = value;
+        } else {
+            cmdqv->vcmdq_cons_indx_base[index] =
+                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        cmdqv->vcmdq_cons_indx_base[index] =
+            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return;
+    }
+}
+
 static uint64_t tegra241_cmdqv_config_vintf_read(Tegra241CMDQV *cmdqv,
                                                  hwaddr offset)
 {
@@ -212,6 +276,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
                                       uint64_t value, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -238,6 +303,40 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         tegra241_cmdqv_config_vintf_write(cmdqv, offset, value);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VINTF Page0 registers have the same per-VCMDQ layout as the
+         * VCMDQ Page0 registers. Translate the VINTF aperture offset to the
+         * equivalent VCMDQ aperture offset, then fall through to reuse the
+         * common VCMDQ decoding logic below.
+         */
+        offset -= CMDQV_VINTF_PAGE0_BASE - CMDQV_VCMDQ_PAGE0_BASE;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
+         * occupies a CMDQV_VCMDQ_STRIDE-byte window within the page.
+         *
+         * Extract the VCMDQ index and normalize to the VCMDQ0_* register
+         * offset. A single helper services all instances via @index.
+         */
+        index = (offset - CMDQV_VCMDQ_PAGE0_BASE) / CMDQV_VCMDQ_STRIDE;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
+                                   index, value, size);
+        break;
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
+        offset -= CMDQV_VINTF_PAGE1_BASE - CMDQV_VCMDQ_PAGE1_BASE;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as VCMDQ Page0 case above */
+        index = (offset - CMDQV_VCMDQ_PAGE1_BASE) / CMDQV_VCMDQ_STRIDE;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
+                                   index, value, size);
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
Signature · 1 line
2.43.0
#37 lore raw

Re: [PATCH v4 16/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register writes

Hi,

Quoted 52 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation. Add write handling for both CMDQV_CMDQ_BASE and CMDQV_VI_CMDQ_BASE apertures using the same index decoding and VINTF-to-VCMDQ translation logic as the read path.

VINTF aperture writes are translated to their CMDQV_CMDQ_BASE equivalent and update the same cached state. Page1 registers (BASE, CONS_INDX_BASE) always update the cache. Once IOMMU_HW_QUEUE_ALLOC and viommu_mmap are wired up in a subsequent patch, Page0 register writes will be forwarded to the hardware-backed mmap'd page.

---
 hw/arm/tegra241-cmdqv.c | 99 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
hw/arm/tegra241-cmdqv.c +25 -0
index 35e6f0bbd6..d4ba2ada92 100644
@@ -55,6 +55,70 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                       int index, uint64_t value,
+                                       unsigned size)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        } else {
+            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, · line 92

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    case A_VCMDQ0_GERRORN:
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);

can you explain the access pattern?
this does not look like A_STRTAB_BASE smmuv3 writes/read

Quoted 71 lines
hw/arm/tegra241-cmdqv.c +57 -0
+        }
+        return;
+    case A_VCMDQ0_BASE_H:
+        cmdqv->vcmdq_base[index] =
+            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        if (size == 8) {
+            cmdqv->vcmdq_cons_indx_base[index] = value;
+        } else {
+            cmdqv->vcmdq_cons_indx_base[index] =
+                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        cmdqv->vcmdq_cons_indx_base[index] =
+            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return;
+    }
+}
+
 static uint64_t tegra241_cmdqv_config_vintf_read(Tegra241CMDQV *cmdqv,
                                                  hwaddr offset)
 {
@@ -212,6 +276,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
                                       uint64_t value, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -238,6 +303,40 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         tegra241_cmdqv_config_vintf_write(cmdqv, offset, value);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VINTF Page0 registers have the same per-VCMDQ layout as the
+         * VCMDQ Page0 registers. Translate the VINTF aperture offset to the
+         * equivalent VCMDQ aperture offset, then fall through to reuse the
+         * common VCMDQ decoding logic below.
+         */
+        offset -= CMDQV_VINTF_PAGE0_BASE - CMDQV_VCMDQ_PAGE0_BASE;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
+         * occupies a CMDQV_VCMDQ_STRIDE-byte window within the page.
+         *
+         * Extract the VCMDQ index and normalize to the VCMDQ0_* register
+         * offset. A single helper services all instances via @index.
+         */
+        index = (offset - CMDQV_VCMDQ_PAGE0_BASE) / CMDQV_VCMDQ_STRIDE;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
+                                   index, value, size);
+        break;
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
+        offset -= CMDQV_VINTF_PAGE1_BASE - CMDQV_VCMDQ_PAGE1_BASE;

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset, · line 342

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as VCMDQ Page0 case above */
+        index = (offset - CMDQV_VCMDQ_PAGE1_BASE) / CMDQV_VCMDQ_STRIDE;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
+                                   index, value, size);
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);

Thanks

Eric

#38 lore raw

RE: [PATCH v4 16/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register writes

Quoted 89 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 05 May 2026 11:43
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 16/31] hw/arm/tegra241-cmdqv: Emulate VCMDQ register writes

External email: Use caution opening links or attachments

Hi,

Quoted 10 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation. Add write handling for both CMDQV_CMDQ_BASE and CMDQV_VI_CMDQ_BASE apertures using the same index decoding and VINTF-to-VCMDQ translation logic as the read path.

VINTF aperture writes are translated to their CMDQV_CMDQ_BASE equivalent and update the same cached state. Page1 registers (BASE, CONS_INDX_BASE) always update the cache. Once

IOMMU_HW_QUEUE_ALLOC and

hw/arm/tegra241-cmdqv.c static uint64_t

Quoted 15 lines

viommu_mmap are wired up in a subsequent patch, Page0 register writes will be forwarded to the hardware-backed mmap'd page.

---
 hw/arm/tegra241-cmdqv.c | 99
+++++++++++++++++++++++++++++++++++++++++
 1 file changed, 99 insertions(+)
hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0 -0

35e6f0bbd6..d4ba2ada92 100644

hw/arm/tegra241-cmdqv.c +0 -0
@@ -55,6 +55,70 @@ static uint64_t

tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +8 -0
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always
+refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv,

hwaddr offset0,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,

Quoted 12 lines
hw/arm/tegra241-cmdqv.c +12 -0
+                                       int index, uint64_t value,
+                                       unsigned size) {
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |=

R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;

hw/arm/tegra241-cmdqv.c static uint64_t · line 79

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+        } else {
+            cmdqv->vcmdq_status[index] &=

~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, · line 92

Quoted 13 lines
hw/arm/tegra241-cmdqv.c +13 -0
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
Quoted 2 lines

can you explain the access pattern?
this does not look like A_STRTAB_BASE smmuv3 writes/read

The size == 8 branch was added to handle a potential single 64-bit write to BASE, but you're right this is inconsistent with the SMMUv3 pattern where size dispatch happens before the per register switch, not inside it.

Also, mmio_cmdqv_ops is missing .valid/.impl size constraints. I will take a look on next respin and update.

Thanks,
Shameer

#39 lore raw

PATCH v4 17/31 hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

The CMDQ-V CMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and set ENABLE_OK only if the mmap succeeds. Unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++---
 2 files changed, 47 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
index b8bd8cd8ff..88572ad939 100644
@@ -18,6 +18,8 @@
 
 #define TEGRA241_CMDQV_MAX_CMDQ   (1U << CMDQV_NUM_CMDQ_LOG2)
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -42,6 +44,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +44 -3

84 lines not on this page · raw message

Signature · 1 line
2.43.0
#40 lore raw

PATCH v4 18/31 system/physmem: Add address_space_is_ram() helper

In reply to Shameer Kolothum

Introduce address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.

---
 include/system/memory.h | 10 ++++++++++
 system/physmem.c        | 11 +++++++++++
 2 files changed, 21 insertions(+)
include/system/memory.h +10 -0
index d7b18b632d..7aed255e81 100644
@@ -2841,6 +2841,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,
  */
 bool address_space_is_io(AddressSpace *as, hwaddr addr);
 
+/**
+ * address_space_is_ram: check whether a guest physical address whithin
+ *                       an address space is RAM.
+ *
+ * @as: #AddressSpace to be accessed
+ * @addr: address within that address space
+ */
+
+bool address_space_is_ram(AddressSpace *as, hwaddr addr);
+
 /* address_space_map: map a physical memory region into a host virtual address
  *
  * May map a subset of the requested range, given by and returned in @plen.
system/physmem.c +11 -0
index 4e26f1a1d4..b67dde80fb 100644
@@ -3674,6 +3674,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)
     return !(memory_region_is_ram(mr) || memory_region_is_romd(mr));
 }
 
+bool address_space_is_ram(AddressSpace *as, hwaddr addr)
+{
+    MemoryRegion *mr;
+
+    RCU_READ_LOCK_GUARD();
+    mr = address_space_translate(as, addr, &addr, NULL, false,
+                                 MEMTXATTRS_UNSPECIFIED);
+
+    return memory_region_is_ram(mr);
+}
+
 static hwaddr
 flatview_extend_translation(FlatView *fv, hwaddr addr,
                             hwaddr target_len,
Signature · 1 line
2.43.0
#42 lore raw

PATCH v4 19/31 hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. A subsequent patch maps this region directly into the guest address space, so QEMU does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.

If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 70 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 78 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 88572ad939..039d86374f 100644
@@ -44,6 +44,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -348,6 +349,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +67 -3

117 lines not on this page · raw message

Signature · 1 line
2.43.0
#43 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 6 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +6 -0
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;

hw/arm/tegra241-cmdqv.c line 54

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {

Overflow prevention:
size - 1 + addr

hw/arm/tegra241-cmdqv.c line 60

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+    if (!tegra241_cmdq_enabled(cmdqv) || !tegra241_vintf_enabled(cmdqv)) {
+        return true;
+    }

It's good to have these two checks. But if vcmdq setup is skipped for vintf=disabled, we need to call this setup() again upon vintf gets enabled?

Also, do we fence against unassigned vcmdq? Corner case is that a guest might write base address registers via direct (global) MMIO space.

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +1 -1
@@ -363,7 +427,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
          */
         index = (offset - CMDQV_VCMDQ_PAGE0_BASE) / CMDQV_VCMDQ_STRIDE;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
-                                   index, value, size);
+                                   index, value, size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset, · line 440

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +1 -1
@@ -373,7 +437,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         /* Same decode logic as VCMDQ Page0 case above */
         index = (offset - CMDQV_VCMDQ_PAGE1_BASE) / CMDQV_VCMDQ_STRIDE;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
-                                   index, value, size);
+                                   index, value, size, &local_err);

Should these two be squashed into an earlier patch?

Nicolin

#44 lore raw

RE: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 36 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 05 May 2026 01:41
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 1 line · Shameer Kolothum

+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int

index,

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] &

R_VCMDQ0_BASE_L_LOG2SIZE_MASK;

hw/arm/tegra241-cmdqv.c line 54

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
Quoted 2 lines

Overflow prevention:
size - 1 + addr

Ok.

Quoted 9 lines
Quoted 2 lines

+ if (!tegra241_cmdq_enabled(cmdqv) ||

!tegra241_vintf_enabled(cmdqv)) {

Review

Quoted 2 lines
(quoted diff) +2 -0
+        return true;
+    }

It's good to have these two checks. But if vcmdq setup is skipped for vintf=disabled, we need to call this setup() again upon vintf gets enabled?

Yeah. It looks like more restrictive than required as per spec. I will address this.

Quoted 5 lines

Also, do we fence against unassigned vcmdq? Corner case is that a guest might write base address registers via direct (global) MMIO space.

Not sure I get that completely.

Spec(p. 176) has:

"While the software can program the Virtual CMDQ(s) directly using the direct VCMDQ aperture (and not through the Virtual Interface), it is required that the VCMDQ be allocated to a Virtual Interface before it is used to send commands to the SMMU."

The spec only restricts sending commands before allocation, not programming BASE. In our model, the BASE write itself triggers alloc_hw_queue so there's nothing to fence there. For other's (Page 0: CONS_INDX, PROD_INDX etc.), the vintf_ptr() check already drops them silently if vcmdq[index] is not yet allocated, consistent with spec p.172:

"If no Virtual CMDQ is mapped to the Guest, or if the logical CMDQ index in the Virtual Interface being accessed by the software does not map to any Virtual CMDQ, the access is dropped with no Fault/Interrupt".

Please let me know if the above interpretation is correct or not.

Quoted 17 lines

Review static void tegra241_cmdqv_write_mmio(void

Quoted 1 line
(quoted diff) +0 -0
@@ -363,7 +427,7 @@ static void tegra241_cmdqv_write_mmio(void

*opaque, hwaddr offset,

Review static void tegra241_cmdqv_write_mmio(void · line 428

Quoted 2 lines
(quoted diff) +0 -0
          */
         index = (offset - CMDQV_VCMDQ_PAGE0_BASE) /

CMDQV_VCMDQ_STRIDE;

Review static void tegra241_cmdqv_write_mmio(void · line 429

Quoted 1 line
(quoted diff) +0 -0
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index *

CMDQV_VCMDQ_STRIDE,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -1
-                                   index, value, size);
+                                   index, value, size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ...

A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +0 -0
         /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
@@ -373,7 +437,7 @@ static void tegra241_cmdqv_write_mmio(void

*opaque, hwaddr offset,

Review static void tegra241_cmdqv_write_mmio(void · line 438

Quoted 2 lines
(quoted diff) +0 -0
         /* Same decode logic as VCMDQ Page0 case above */
         index = (offset - CMDQV_VCMDQ_PAGE1_BASE) /
Quoted 7 lines

CMDQV_VCMDQ_STRIDE;

Review static void tegra241_cmdqv_write_mmio(void · line 439

Quoted 1 line
(quoted diff) +0 -0
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index *

CMDQV_VCMDQ_STRIDE,

Review static void tegra241_cmdqv_write_mmio(void · line 440

Quoted 2 lines
(quoted diff) +1 -1
-                                   index, value, size);
+                                   index, value, size, &local_err);

Should these two be squashed into an earlier patch?

Yes.

Thanks,
Shameer

#45 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 9 lines · Shameer Kolothum Thodi
Quoted 4 lines

Also, do we fence against unassigned vcmdq? Corner case is that a guest might write base address registers via direct (global) MMIO space.

Not sure I get that completely.

Spec(p. 176) has:

Quoted 8 lines

"While the software can program the Virtual CMDQ(s) directly using the direct VCMDQ aperture (and not through the Virtual Interface), it is required that the VCMDQ be allocated to a Virtual Interface before it is used to send commands to the SMMU."

The spec only restricts sending commands before allocation, not programming BASE. In our model, the BASE write itself triggers alloc_hw_queue so there's nothing to fence there.

Our model has an assumption that guest would map a VCMDQ to the VINTF0 first before doing any meaningful programming.

What if a guest programs BASE before it maps a VCMDQ to VINTF0? This can end up with allocating hw_queue that will map the VCMDQ to physical VINTFx, while the guest hasn't map it yet.

Quoted 8 lines

For other's
(Page 0: CONS_INDX, PROD_INDX etc.), the vintf_ptr() check already drops them silently if vcmdq[index] is not yet allocated, consistent with spec p.172:

"If no Virtual CMDQ is mapped to the Guest, or if the logical CMDQ index in the Virtual Interface being accessed by the software does not map to any Virtual CMDQ, the access is dropped with no Fault/Interrupt".

My reading: "the access" means accessing logical VCMDQ via VTINF's MMIO space (0x30000 and 0x40000). Yes, silently dropping it is what we want.

My concern here is about the access to the direct/global VCMDQ MMIO space (0x10000 and 0x20000).

So, if this concern is a real case, should we:
1. If the guest maps the VCMDQ to VINTF0 before it writes BASE, call setup() to allocate hw_queue as our model expects.
2. If the guest writes BASE before it maps VCMDQ to VINTF0, we need to fence against it.
?

One more thing to check for case (2): after the guest programmed BASE and then maps it to VINTF0, and it never writes BASE again, should we call setup() at map?

Thanks
Nicolin

#46 lore raw

RE: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 32 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 05 May 2026 20:38
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 14 lines · Shameer Kolothum Thodi
Quoted 4 lines

Also, do we fence against unassigned vcmdq? Corner case is that a guest might write base address registers via direct (global) MMIO space.

Not sure I get that completely.

Spec(p. 176) has:

"While the software can program the Virtual CMDQ(s) directly using the direct VCMDQ aperture (and not through the Virtual Interface), it is required that the VCMDQ be allocated to a Virtual Interface before it is used to send commands to the SMMU."

Quoted 6 lines
Quoted 3 lines

The spec only restricts sending commands before allocation, not programming BASE. In our model, the BASE write itself triggers alloc_hw_queue so there's nothing to fence there.

Our model has an assumption that guest would map a VCMDQ to the VINTF0 first before doing any meaningful programming.

Yes. And I think by "map" here, you mean the writing to the _ALLOC_MAP register. If so, yes, we don't check that.

Quoted 4 lines

What if a guest programs BASE before it maps a VCMDQ to VINTF0? This can end up with allocating hw_queue that will map the VCMDQ to physical VINTFx, while the guest hasn't map it yet.

Yes. We probably need to gate setup_vcmdq() with:

if (!(cmdqv->cmdq_alloc_map[index] & R_CMDQ_ALLOC_MAP_0_ALLOC_MASK)) {
      return true;

}

Quoted 5 lines
Quoted 5 lines

For other's
(Page 0: CONS_INDX, PROD_INDX etc.), the vintf_ptr() check already drops them silently if vcmdq[index] is not yet allocated, consistent with spec p.172:

Quoted 7 lines
Quoted 3 lines

"If no Virtual CMDQ is mapped to the Guest, or if the logical CMDQ index in the Virtual Interface being accessed by the software does not map to any Virtual CMDQ, the access is dropped with no Fault/Interrupt".

My reading: "the access" means accessing logical VCMDQ via VTINF's MMIO space (0x30000 and 0x40000). Yes, silently dropping it is what we want.

Ok.

Quoted 10 lines

My concern here is about the access to the direct/global VCMDQ MMIO space (0x10000 and 0x20000).

So, if this concern is a real case, should we:
1. If the guest maps the VCMDQ to VINTF0 before it writes BASE, call setup() to allocate hw_queue as our model expects.
2. If the guest writes BASE before it maps VCMDQ to VINTF0, we need to fence against it.
?

Quoted 3 lines

One more thing to check for case (2): after the guest programmed BASE and then maps it to VINTF0, and it never writes BASE again, should we call setup() at map?

That is a non-spec compliant behaviour I guess:.

In p.176: "Allocate a VCMDQ to the Virtual Interface", this is the order:

-Program the CMDQV_CMDQ_ALLOC_MAP_X to map CMDQ X to the
Logical CMDQ (L) on Virtual Interface (V). Logical CMDQ allocated in the Guest must be in order starting from 0.
-CMDQ is allocated to be used by the software. Guest can now program
the VCMDQ to use it as described in the "Enabling the Virtual CMDQ"
section.

"Enabling the Virtual CMDQ" (p.175) is where BASE is programmed. So the spec explicitly places ALLOC_MAP before BASE.

Should we support a non-spec compliant Guest is another question. IMHO, supporting non-compliant behaviour adds complexity with no real benefit.

Thanks,
Shameer

#47 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines
Quoted 12 lines

Not sure I get that completely.

Spec(p. 176) has:

"While the software can program the Virtual CMDQ(s) directly using the direct VCMDQ aperture (and not through the Virtual Interface), it is required that the VCMDQ be allocated to a Virtual Interface before it is used to send commands to the SMMU."

The spec only restricts sending commands before allocation, not programming BASE. In our model, the BASE write itself triggers alloc_hw_queue so there's nothing to fence there.

Quoted 5 lines
Quoted 2 lines

Our model has an assumption that guest would map a VCMDQ to the VINTF0 first before doing any meaningful programming.

Yes. And I think by "map" here, you mean the writing to the _ALLOC_MAP register. If so, yes, we don't check that.

Yea, map == ALLOC_MAP.

Quoted 26 lines
Quoted 10 lines

My concern here is about the access to the direct/global VCMDQ MMIO space (0x10000 and 0x20000).

So, if this concern is a real case, should we:
1. If the guest maps the VCMDQ to VINTF0 before it writes BASE, call setup() to allocate hw_queue as our model expects.
2. If the guest writes BASE before it maps VCMDQ to VINTF0, we need to fence against it.
?

Quoted 3 lines

One more thing to check for case (2): after the guest programmed BASE and then maps it to VINTF0, and it never writes BASE again, should we call setup() at map?

That is a non-spec compliant behaviour I guess:.

In p.176: "Allocate a VCMDQ to the Virtual Interface", this is the order:

(quoted diff) +0 -2
-Program the CMDQV_CMDQ_ALLOC_MAP_X to map CMDQ X to the
 Logical CMDQ (L) on Virtual Interface (V). Logical CMDQ allocated in the
 Guest must be in order starting from 0.
-CMDQ is allocated to be used by the software. Guest can now program
  the VCMDQ to use it   as described in the "Enabling the Virtual CMDQ"
  section.
Quoted 6 lines

"Enabling the Virtual CMDQ" (p.175) is where BASE is programmed. So the spec explicitly places ALLOC_MAP before BASE.

Should we support a non-spec compliant Guest is another question. IMHO, supporting non-compliant behaviour adds complexity with no real benefit.

I read that as a spec recommendation (IOW, a SW guidance) not an ordering enforcement (IOW, not a HW contract).

The real restriction for BASE registers is, per spec:

    Should be programmed before CMDQEN == 1
and HW behaviour for breaking this restriction is:
    Ignore Writes to the register when the CMDQEN == 1

My reading is that, so long as SW follows this minimal rule, it can do whatever it wants.

(Btw, seems we haven't applies this restriction into our code?)

The point is about how we are going to simulate the HW. If real HW allows this (maybe worth hacking the driver to confirm this), there seems to be no reason for us not to support that?

Nicolin

#48 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

In reply to Shameer Kolothum

Hi,

Quoted 10 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. A subsequent patch maps this region directly into the guest address space, so QEMU does not trap writes to VCMDQ_CONFIG.

Quoted 6 lines

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled. If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

the asymetric alloc/free sounds unusual. Are there other alternatives?

Quoted 55 lines

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 70 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 78 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 88572ad939..039d86374f 100644
@@ -44,6 +44,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -348,6 +349,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +6 -0
index cdd941cec9..b5f2f74cf2 100644
@@ -15,6 +15,66 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {

hw/arm/tegra241-cmdqv.c line 33

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
+    cmdqv->vcmdq[index] = NULL;
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in reverse order to avoid "resource busy" error */

can you provide additional details about the above problematic. Is it documented in the spec?

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +11 -0
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;

hw/arm/tegra241-cmdqv.c line 54

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {

this check looks a little bit risky, no? Don't we have a better way to test the address has been properly set?

hw/arm/tegra241-cmdqv.c line 62

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+        return true;
+    }
+
+    if (!tegra241_cmdq_enabled(cmdqv) || !tegra241_vintf_enabled(cmdqv)) {
+        return true;
+    }
+
+    tegra241_cmdqv_free_vcmdq(cmdqv, index);

would deserve a comment also here.

Quoted 67 lines
hw/arm/tegra241-cmdqv.c +21 -2
+
+    if (!iommufd_backend_alloc_hw_queue(viommu->iommufd, viommu->viommu_id,
+                                        IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV,
+                                        index, addr, size, &hw_queue_id,
+                                        errp)) {
+        return false;
+    }
+    hw_queue = g_new(IOMMUFDHWqueue, 1);
+    hw_queue->hw_queue_id = hw_queue_id;
+    hw_queue->viommu = viommu;
+    cmdqv->vcmdq[index] = hw_queue;
+
+    return true;
+}
+
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -63,7 +123,7 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
  */
 static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                        int index, uint64_t value,
-                                       unsigned size)
+                                       unsigned size, Error **errp)
 {
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
@@ -91,11 +151,13 @@ static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                 (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
                 (value & 0xffffffffULL);
         }
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_BASE_H:
         cmdqv->vcmdq_base[index] =
             (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
             ((uint64_t)value << 32);
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
         if (size == 8) {
@@ -204,6 +266,7 @@ static void tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,
                 cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
             }
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
@@ -329,6 +392,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
         break;
@@ -363,7 +427,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
          */
         index = (offset - CMDQV_VCMDQ_PAGE0_BASE) / CMDQV_VCMDQ_STRIDE;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
-                                   index, value, size);
+                                   index, value, size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset, · line 443

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -1
@@ -373,7 +437,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         /* Same decode logic as VCMDQ Page0 case above */
         index = (offset - CMDQV_VCMDQ_PAGE1_BASE) / CMDQV_VCMDQ_STRIDE;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
-                                   index, value, size);
+                                   index, value, size, &local_err);
         break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",

Eric

#49 lore raw

RE: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 32 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 05 May 2026 14:26
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

External email: Use caution opening links or attachments

Hi,

Quoted 10 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. A subsequent patch maps this region directly into the guest address space, so QEMU does not trap writes to VCMDQ_CONFIG.

Quoted 7 lines
Quoted 6 lines

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled. If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

the asymetric alloc/free sounds unusual. Are there other alternatives?

Nothing that comes to my mind now. There is no "update hw_queue address" hence we have to free and allocate. May be we can add a small optimisation and skip free + alloc if the vcmdq_base[index] hasn't changed from previous alloc.

Quoted 61 lines
Quoted 10 lines

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 70

+++++++++++++++++++++++++++++++++++++++--

Quoted 36 lines
 2 files changed, 78 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 88572ad939..039d86374f 100644
@@ -44,6 +44,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -348,6 +349,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index cdd941cec9..b5f2f74cf2 100644

hw/arm/tegra241-cmdqv.c line 18

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -15,6 +15,66 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int

index)

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
Quoted 8 lines

hw/arm/tegra241-cmdqv.c line 33

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    cmdqv->vcmdq[index] = NULL;
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in reverse order to avoid "resource busy" error */

can you provide additional details about the above problematic. Is it documented in the spec?

See p.176:

Deallocate a VCMDQ from a Virtual Interface
Logical CMDQ being deallocated for a Guest must be in decreasing order starting from the highest numbered LVCMDQ.

Quoted 23 lines

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int

index,

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] &

R_VCMDQ0_BASE_L_LOG2SIZE_MASK;

hw/arm/tegra241-cmdqv.c line 54

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
Quoted 2 lines

this check looks a little bit risky, no? Don't we have a better way to test the address has been properly set?

I think eventually kernel will handle any attempt to use an invalid address through IOMMU_HW_QUEUE_ALLOC IOCTL:
iommufd_hw_queue_alloc_phys()/iommufd_access_pin_pages() etc.

Any attempt to pass an invalid address will return error, I think.

@Nicolin, is that a safe assumption to make?

Quoted 10 lines

hw/arm/tegra241-cmdqv.c

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        return true;
+    }
+
+    if (!tegra241_cmdq_enabled(cmdqv) ||

!tegra241_vintf_enabled(cmdqv)) {

hw/arm/tegra241-cmdqv.c line 62

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        return true;
+    }
+
+    tegra241_cmdqv_free_vcmdq(cmdqv, index);

would deserve a comment also here.

Sure.

Thanks,
Shameer

#50 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 11 lines · Shameer Kolothum Thodi
Quoted 5 lines

hw/arm/tegra241-cmdqv.c line 33

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in reverse order to avoid "resource busy" error */

can you provide additional details about the above problematic. Is it documented in the spec?

See p.176:

Deallocate a VCMDQ from a Virtual Interface
Logical CMDQ being deallocated for a Guest must be in decreasing order starting from the highest numbered LVCMDQ.

Maybe add "Spec demands" to things like this.

Quoted 26 lines
Quoted 23 lines

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int

index,

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] &

R_VCMDQ0_BASE_L_LOG2SIZE_MASK;

hw/arm/tegra241-cmdqv.c line 54

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
Quoted 2 lines

this check looks a little bit risky, no? Don't we have a better way to test the address has been properly set?

Quoted 7 lines

I think eventually kernel will handle any attempt to use an invalid address through IOMMU_HW_QUEUE_ALLOC IOCTL:
iommufd_hw_queue_alloc_phys()/iommufd_access_pin_pages() etc.

Any attempt to pass an invalid address will return error, I think.

@Nicolin, is that a safe assumption to make?

Kernel validates if the gPA range is in the stage-2 page table and physically contiguous. It might be nicer if QEMU has a way to make sure that is the range is in RAM.

Nicolin

#51 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

In reply to Shameer Kolothum Thodi
Quoted 37 lines · Shameer Kolothum Thodi
Quoted 37 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 05 May 2026 14:26
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

External email: Use caution opening links or attachments

Hi,

Quoted 16 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. A subsequent patch maps this region directly into the guest address space, so QEMU does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled. If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

Quoted 5 lines
Quoted 1 line

the asymetric alloc/free sounds unusual. Are there other alternatives?

Nothing that comes to my mind now. There is no "update hw_queue address" hence we have to free and allocate. May be we can add a small optimisation and skip free + alloc if the vcmdq_base[index] hasn't changed from previous alloc.

OK

Thanks

Eric

Quoted 122 lines
Quoted 61 lines
Quoted 10 lines

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 70

+++++++++++++++++++++++++++++++++++++++--

Quoted 36 lines
 2 files changed, 78 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 88572ad939..039d86374f 100644
@@ -44,6 +44,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -348,6 +349,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index cdd941cec9..b5f2f74cf2 100644

hw/arm/tegra241-cmdqv.c line 18

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -15,6 +15,66 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int

index)

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
Quoted 8 lines

hw/arm/tegra241-cmdqv.c line 33

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    cmdqv->vcmdq[index] = NULL;
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in reverse order to avoid "resource busy" error */

can you provide additional details about the above problematic. Is it documented in the spec?

See p.176:

Deallocate a VCMDQ from a Virtual Interface
Logical CMDQ being deallocated for a Guest must be in decreasing order starting from the highest numbered LVCMDQ.

Quoted 23 lines

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int

index,

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] &

R_VCMDQ0_BASE_L_LOG2SIZE_MASK;

hw/arm/tegra241-cmdqv.c line 54

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
Quoted 2 lines

this check looks a little bit risky, no? Don't we have a better way to test the address has been properly set?

I think eventually kernel will handle any attempt to use an invalid address through IOMMU_HW_QUEUE_ALLOC IOCTL:
iommufd_hw_queue_alloc_phys()/iommufd_access_pin_pages() etc.

Any attempt to pass an invalid address will return error, I think.

@Nicolin, is that a safe assumption to make?

Quoted 10 lines

hw/arm/tegra241-cmdqv.c

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        return true;
+    }
+
+    if (!tegra241_cmdq_enabled(cmdqv) ||

!tegra241_vintf_enabled(cmdqv)) {

hw/arm/tegra241-cmdqv.c line 62

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        return true;
+    }
+
+    tegra241_cmdqv_free_vcmdq(cmdqv, index);

would deserve a comment also here.

Sure.

Thanks,
Shameer

#52 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

In reply to Shameer Kolothum
Quoted 74 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. A subsequent patch maps this region directly into the guest address space, so QEMU does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.

If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 70 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 78 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 88572ad939..039d86374f 100644
@@ -44,6 +44,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDVeventq *veventq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -348,6 +349,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdqv)
+{
+    return cmdqv->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +8 -0
index cdd941cec9..b5f2f74cf2 100644
@@ -15,6 +15,66 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {
+        return;
+    }

hw/arm/tegra241-cmdqv.c line 35

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
+    cmdqv->vcmdq[index] = NULL;
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in reverse order to avoid "resource busy" error */
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);

uapi/linux/iommufd.h says:
* - alloc starts from the lowest @index=0 in ascending order
* - destroy starts from the last allocated @index in descending order

so this seems a requirement of the uapi if I understand correctly

Eric

Quoted 102 lines
hw/arm/tegra241-cmdqv.c +49 -3
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc. */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
+        return true;
+    }
+
+    if (!tegra241_cmdq_enabled(cmdqv) || !tegra241_vintf_enabled(cmdqv)) {
+        return true;
+    }
+
+    tegra241_cmdqv_free_vcmdq(cmdqv, index);
+
+    if (!iommufd_backend_alloc_hw_queue(viommu->iommufd, viommu->viommu_id,
+                                        IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV,
+                                        index, addr, size, &hw_queue_id,
+                                        errp)) {
+        return false;
+    }
+    hw_queue = g_new(IOMMUFDHWqueue, 1);
+    hw_queue->hw_queue_id = hw_queue_id;
+    hw_queue->viommu = viommu;
+    cmdqv->vcmdq[index] = hw_queue;
+
+    return true;
+}
+
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -63,7 +123,7 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
  */
 static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                        int index, uint64_t value,
-                                       unsigned size)
+                                       unsigned size, Error **errp)
 {
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
@@ -91,11 +151,13 @@ static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                 (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
                 (value & 0xffffffffULL);
         }
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_BASE_H:
         cmdqv->vcmdq_base[index] =
             (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
             ((uint64_t)value << 32);
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
         if (size == 8) {
@@ -204,6 +266,7 @@ static void tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,
                 cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
             }
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
@@ -329,6 +392,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
         break;
@@ -363,7 +427,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
          */
         index = (offset - CMDQV_VCMDQ_PAGE0_BASE) / CMDQV_VCMDQ_STRIDE;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
-                                   index, value, size);
+                                   index, value, size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         /* Same VINTF-to-VCMDQ translation as VINTF Page0 case above */
@@ -373,7 +437,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         /* Same decode logic as VCMDQ Page0 case above */
         index = (offset - CMDQV_VCMDQ_PAGE1_BASE) / CMDQV_VCMDQ_STRIDE;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - index * CMDQV_VCMDQ_STRIDE,
-                                   index, value, size);
+                                   index, value, size, &local_err);
         break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
#53 lore raw

Re: [PATCH v4 19/31] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 11 lines · Eric Auger

hw/arm/tegra241-cmdqv.c line 35

Quoted 5 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +5 -0
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in reverse order to avoid "resource busy" error */
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);

uapi/linux/iommufd.h says:
* - alloc starts from the lowest @index=0 in ascending order
* - destroy starts from the last allocated @index in descending order

so this seems a requirement of the uapi if I understand correctly

Oh yea. I forgot I put there too. Fundamentally, it's demanded by the HW contract. But I think you are right, for QEMU, saying that it follows the uAPI fits better.

Nicolin

#54 lore raw

PATCH v4 20/31 hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

In reply to Shameer Kolothum

Introduce tegra241_cmdqv_vintf_ptr() to route VCMDQ register accesses through the mmap'd VINTF page0 backing once a hardware queue has been allocated.

There are two QEMU trapped MMIO apertures for VCMDQ registers:

- Direct VCMDQ aperture (offset 0x10000)
- VINTF Page0 (offset 0x30000)

These are hardware aliases: they address the same underlying registers. A subsequent patch maps the VINTF aperture as a guest-direct RAM region; in this patch both remain QEMU-trapped.

VCMDQ register accesses operate in one of two mutually exclusive modes, depending on whether a hardware queue (IOMMU_HW_QUEUE_ALLOC) has been allocated for the VCMDQ:

Pre-alloc: vintf_ptr is NULL. Both apertures use QEMU's register cache. Hardware is not yet engaged;

Post-alloc: vintf_ptr is valid. Both QEMU trapped apertures access registers directly via the mmap'd vintf_page0 pointer, bypassing the cache. Hardware is the single source of truth.

The pre-to-post-alloc transition is triggered by the BASE register write that initiates IOMMU_HW_QUEUE_ALLOC. No cache-to-hardware synchronisation is needed at transition time. The hardware mandated init sequence requires BASE to be written first; PROD_INDX, CONS_INDX and CONFIG.CMDQ_EN are programmed only after BASE and are therefore always post-alloc.

Any pre-alloc writes to those registers update only the register cache, which is discarded at the transition.

CMDQV acceleration only becomes active once the guest enables VINTF and programs the VCMDQ BASE register. Until then, all VCMDQ accesses are served from the emulated register cache with no real hardware command processing. This matches the CMDQV hardware specification: if the logical CMDQ index does not map to any allocated Virtual CMDQ, "the access is dropped with no Fault/Interrupt".

---
 hw/arm/tegra241-cmdqv.c | 48 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 47 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +47 -1
index b5f2f74cf2..eb619e1134 100644
@@ -75,17 +75,45 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     return true;
 }
 
+static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
+                                                 int index, hwaddr offset0)
+{
+    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
+        return NULL;
+    }
+    return (uint32_t *)(cmdqv->vintf_page0 +
+                        (index * CMDQV_VCMDQ_STRIDE) +
+                        (offset0 - CMDQV_VCMDQ_PAGE0_BASE));
+}
+
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
  *
- * All VCMDQ accesses return cached registers.
+ * If the VCMDQ is allocated and VINTF page0 is mmap'd, read directly
+ * from the VINTF page0 backing. Otherwise, fall back to cached state.
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                           int index)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -120,11 +148,29 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * If the VCMDQ is allocated and VINTF page0 is mmap'd, write directly
+ * to the VINTF page0 backing. Otherwise, update cached state.
  */
 static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                        int index, uint64_t value,
                                        unsigned size, Error **errp)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = (uint32_t)value;
Signature · 1 line
2.43.0
#55 lore raw

Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 8 lines · Shameer Kolothum

The pre-to-post-alloc transition is triggered by the BASE register write that initiates IOMMU_HW_QUEUE_ALLOC. No cache-to-hardware synchronisation is needed at transition time. The hardware mandated init sequence requires BASE to be written first; PROD_INDX, CONS_INDX and CONFIG.CMDQ_EN are programmed only after BASE and are therefore always post-alloc.

Any pre-alloc writes to those registers update only the register cache, which is discarded at the transition.

Is "discard" the correct action?

Guest OS might expect HW (VM) to retain what it writes to those page0 registers?

Nicolin

#56 lore raw

RE: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 24 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 05 May 2026 01:50
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 5 lines · Shameer Kolothum

The pre-to-post-alloc transition is triggered by the BASE register write that initiates IOMMU_HW_QUEUE_ALLOC. No cache-to-hardware synchronisation is needed at transition time. The hardware mandated init sequence requires BASE to be written first; PROD_INDX, CONS_INDX and CONFIG.CMDQ_EN are programmed only after BASE and are therefore

always post-alloc.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

Any pre-alloc writes to those registers update only the register cache, which is discarded at the transition.

Is "discard" the correct action?

Guest OS might expect HW (VM) to retain what it writes to those page0 registers?

As explained above, I reached that conclusion based on spec p.174/175:

Under "Enabling the Virtual CMDQ" it specifies the init order as below:

- Program the VIRT_CMDQ_BASE
- Init the PROD_INDX/ CONS_INDX to 0 or a consistent value
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Since we set up the HW QUEUE on BASE write, a spec compliant guest is expected to program the Page0 registers as above after that, and we pass those writes directly to the VINTF Page0. There are no GERRORN writes before HW queue setup either.

Please let me know if there is still a case to sync the registers.

Thanks,
Shameer

#57 lore raw

Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 21 lines · Shameer Kolothum Thodi
Quoted 9 lines

From: Nicolin Chen <[email protected]>

Quoted 5 lines · Shameer Kolothum

The pre-to-post-alloc transition is triggered by the BASE register write that initiates IOMMU_HW_QUEUE_ALLOC. No cache-to-hardware synchronisation is needed at transition time. The hardware mandated init sequence requires BASE to be written first; PROD_INDX, CONS_INDX and CONFIG.CMDQ_EN are programmed only after BASE and are therefore

always post-alloc.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

Any pre-alloc writes to those registers update only the register cache, which is discarded at the transition.

Is "discard" the correct action?

Guest OS might expect HW (VM) to retain what it writes to those page0 registers?

As explained above, I reached that conclusion based on spec p.174/175:

Under "Enabling the Virtual CMDQ" it specifies the init order as below:

Quoted 8 lines

- Program the VIRT_CMDQ_BASE
- Init the PROD_INDX/ CONS_INDX to 0 or a consistent value
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Since we set up the HW QUEUE on BASE write, a spec compliant guest is expected to program the Page0 registers as above after that, and we pass those writes directly to the VINTF Page0. There are no GERRORN writes before HW queue setup either.

I don't think we can make an assumption that guest would follow what spec suggests.

Could this be a case:
- Write the PROD_INDX/ CONS_INDX with a consistent value (0xf)
- Program the VIRT_CMDQ_BASE
- Read the PROD_INDX/CONS_INDX, expecting 0xf
- Write the PROD_INDX/ CONS_INDX with 0x0
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ
?

Nicolin

#58 lore raw

RE: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 52 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 05 May 2026 20:52
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 7 lines · Shameer Kolothum Thodi
Quoted 7 lines

From: Nicolin Chen <[email protected]>

Quoted 5 lines · Shameer Kolothum

The pre-to-post-alloc transition is triggered by the BASE register write that initiates IOMMU_HW_QUEUE_ALLOC. No cache-to-hardware synchronisation is needed at transition time. The hardware mandated init sequence requires BASE to be written first; PROD_INDX, CONS_INDX and CONFIG.CMDQ_EN are programmed only after BASE and are

therefore

Quoted 14 lines
Quoted 9 lines

always post-alloc.

Quoted 3 lines

Any pre-alloc writes to those registers update only the register cache, which is discarded at the transition.

Is "discard" the correct action?

Guest OS might expect HW (VM) to retain what it writes to those page0 registers?

As explained above, I reached that conclusion based on spec p.174/175:

Under "Enabling the Virtual CMDQ" it specifies the init order as below:

Quoted 8 lines

- Program the VIRT_CMDQ_BASE
- Init the PROD_INDX/ CONS_INDX to 0 or a consistent value
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Since we set up the HW QUEUE on BASE write, a spec compliant guest is expected to program the Page0 registers as above after that, and we pass those writes directly to the VINTF Page0. There are no GERRORN writes before HW queue setup either.

I don't think we can make an assumption that guest would follow what spec suggests.

Quoted 6 lines

Could this be a case:
- Write the PROD_INDX/ CONS_INDX with a consistent value (0xf)
- Program the VIRT_CMDQ_BASE
- Read the PROD_INDX/CONS_INDX, expecting 0xf
- Write the PROD_INDX/ CONS_INDX with 0x0
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Yes, if Guest is not spec compliant then this can be any order.

However,

Quoted 1 line

- Program the VIRT_CMDQ_BASE -- suppose we call setup_vcmdq()

then kernel will reset PROD/CONS, right?
https://lore.kernel.org/all/[email protected]/

Again, the question is whether we should handle all these non-compliant cases to prevent the guest from shooting itself in the foot, or simply propagate any hardware error events due to the non-compliant behaviour.

On the other side, if we discard the register values, the read-what-you-write contract is broken.

I am not sure what is QEMU guideline w.r.t a non-spec compliant Guest behaviour.

@Eric, any thoughts?

We already have a spec-compliant behaviour assumption for accel vEVENTQ support, as discussed here:

https://lore.kernel.org/qemu-devel/CH3PR12MB75485AFF456B4133B9FB3770ABA0A@CH3PR12MB7548.namprd12.prod.outlook.com/

Thanks,
Shameer

#59 lore raw

Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 8 lines · Shameer Kolothum Thodi
Quoted 7 lines

From: Nicolin Chen <[email protected]>
Could this be a case:
- Write the PROD_INDX/ CONS_INDX with a consistent value (0xf)
- Program the VIRT_CMDQ_BASE
- Read the PROD_INDX/CONS_INDX, expecting 0xf
- Write the PROD_INDX/ CONS_INDX with 0x0
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Quoted 7 lines

Yes, if Guest is not spec compliant then this can be any order.

However,

Quoted 1 line

- Program the VIRT_CMDQ_BASE -- suppose we call setup_vcmdq()

then kernel will reset PROD/CONS, right?

But I don't have an impression that QEMU only supports Guest to run Linux...

Quoted 4 lines

Again, the question is whether we should handle all these non-compliant cases to prevent the guest from shooting itself in the foot, or simply propagate any hardware error events due to the non-compliant behaviour.

I don't really feel it convincing to call it non-compliant. HW does not disallow that behaviour. Also, the flow in my example does not shoot itself in the foot: it basically follows the spec recommended flow by adding a pre-write before programming BASE. The remaining steps are still exactly follow the spec recommendation. Yes, it is odd to do so, but it is not insane yet. There might be some similar corner case to exist for certain reasons that we can't imagine yet.

And adding these to making the QEMU model closer to a real HW isn't seemingly very convoluted? I'd feed the spec to an AI and run some analysis to see what else we need to cover here.

Nicolin

#60 lore raw

RE: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 26 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 06 May 2026 19:35
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 10 lines · Shameer Kolothum Thodi
Quoted 7 lines

From: Nicolin Chen <[email protected]>
Could this be a case:
- Write the PROD_INDX/ CONS_INDX with a consistent value (0xf)
- Program the VIRT_CMDQ_BASE
- Read the PROD_INDX/CONS_INDX, expecting 0xf
- Write the PROD_INDX/ CONS_INDX with 0x0
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Yes, if Guest is not spec compliant then this can be any order.

Quoted 8 lines
Quoted 5 lines

However,

Quoted 1 line

- Program the VIRT_CMDQ_BASE -- suppose we call setup_vcmdq()

then kernel will reset PROD/CONS, right?

But I don't have an impression that QEMU only supports Guest to run Linux...

Oh no, I was referring to host kernel behaviour on IOMMU_HW_QUEUE_ALLOC IOCTL. It seems to reset PROD/CONS to zero in that path and I thought it was following spec in that case.

Quoted 14 lines
Quoted 5 lines

Again, the question is whether we should handle all these non-compliant cases to prevent the guest from shooting itself in the foot, or simply propagate any hardware error events due to the non-compliant behaviour.

I don't really feel it convincing to call it non-compliant. HW does not disallow that behaviour. Also, the flow in my example does not shoot itself in the foot: it basically follows the spec recommended flow by adding a pre-write before programming BASE. The remaining steps are still exactly follow the spec recommendation. Yes, it is odd to do so, but it is not insane yet. There might be some similar corner case to exist for certain reasons that we can't imagine yet.

Quoted 3 lines

And adding these to making the QEMU model closer to a real HW isn't seemingly very convoluted? I'd feed the spec to an AI and run some analysis to see what else we need to cover here.

I did ask Claude on the need to sync vs discard(feeding spec and all this discussion) and had some fun😊

1)First, it said:

This scenario writes PROD_INDX before BASE, which is explicitly non-spec-compliant — TRM p.175 "Enabling the Virtual CMDQ" mandates BASE as the first step, with PROD_INDX/CONS_INDX initialization following. For a spec-compliant guest, PROD_INDX is only written after BASE, at which point vintf_ptr() is valid and writes go directly to hardware with no discard involved.

Supporting this non-compliant ordering would require replaying cached values after alloc, adding complexity for behaviour the spec does not permit.

2) But then after some prompt it says:

Looking at SMMUv3's register write handlers, QEMU doesn't enforce
any programming order there either — registers are written to cache
unconditionally. CMDQV should follow the same convention. Will fix by replaying cached PROD_INDX/CONS_INDX values to hardware after alloc_hw_queue succeeds, making the pre-to-post-alloc transition transparent to the guest.

3) But it changes position again if you ask in a different way:

SMMUv3 has no equivalent transition point where hardware takes over. In CMDQV, the BASE write is a fundamental transition — before it, there is no hardware queue; after it, hardware owns the registers. The BASE write is a fundamental transition point — before it, no hardware queue exists; after it, hardware owns the Page0 registers. Pre-alloc writes to PROD_INDX/CONS_INDX are undefined behavior by
spec, and "discard" at transition is the correct consequence.
There's nothing to "preserve across" in the SMMUv3 case.

So it goes on...

I think the spec is vague and doesn’t demand a strict order. So may be better to sync the registers after the alloc.

Thanks,
Shameer

#61 lore raw

Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

Quoted 16 lines · Shameer Kolothum Thodi
Quoted 16 lines

-----Original Message-----

Quoted 13 lines · Shameer Kolothum Thodi
Quoted 7 lines

From: Nicolin Chen <[email protected]>
Could this be a case:
- Write the PROD_INDX/ CONS_INDX with a consistent value (0xf)
- Program the VIRT_CMDQ_BASE
- Read the PROD_INDX/CONS_INDX, expecting 0xf
- Write the PROD_INDX/ CONS_INDX with 0x0
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

Yes, if Guest is not spec compliant then this can be any order.

However,

Quoted 1 line

- Program the VIRT_CMDQ_BASE -- suppose we call setup_vcmdq()

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

then kernel will reset PROD/CONS, right?

But I don't have an impression that QEMU only supports Guest to run Linux...

Oh no, I was referring to host kernel behaviour on IOMMU_HW_QUEUE_ALLOC IOCTL. It seems to reset PROD/CONS to zero in that path and I thought it was following spec in that case.

Ah, it was fixing a kernel bug, where the VCMDQ registers never got cleared even after a host reset (IIRC). And the solution is to clear them before handing over to the guest.

Once QEMU allocates the hw_queue that ALLOC_MAPs it to the VINTF, it should probably sync what the register caches were programmed by the guest.

Quoted 7 lines

This scenario writes PROD_INDX before BASE, which is explicitly non-spec-compliant — TRM p.175 "Enabling the Virtual CMDQ" mandates BASE as the first step, with PROD_INDX/CONS_INDX initialization following. For a spec-compliant guest, PROD_INDX is only written after BASE, at which point vintf_ptr() is valid and writes go directly to hardware with no discard involved.

I think the goal of VMM is that a guest VM cannot distinguish between real HW and emulated HW. If we just follow the minimal "spec compliant" flow, it fails to do so. So, "spec compliant" itself doesn't make a lot of sense to me.

With that being said, if QEMU is okay with the minimal support for "spec compliant" flow only, I certainly won't be against submitting it (for the initial version maybe?).

Quoted 8 lines

2) But then after some prompt it says:

Looking at SMMUv3's register write handlers, QEMU doesn't enforce
any programming order there either — registers are written to cache
unconditionally. CMDQV should follow the same convention. Will fix by replaying cached PROD_INDX/CONS_INDX values to hardware after alloc_hw_queue succeeds, making the pre-to-post-alloc transition transparent to the guest.

Yes. We are indeed missing a few details in the implementation.

Quoted 10 lines

3) But it changes position again if you ask in a different way:

SMMUv3 has no equivalent transition point where hardware takes over. In CMDQV, the BASE write is a fundamental transition — before it, there is no hardware queue; after it, hardware owns the registers. The BASE write is a fundamental transition point — before it, no hardware queue exists; after it, hardware owns the Page0 registers. Pre-alloc writes to PROD_INDX/CONS_INDX are undefined behavior by
spec, and "discard" at transition is the correct consequence.
There's nothing to "preserve across" in the SMMUv3 case.

Ask it to do a deeper research before answer any question, or to pinpoint where spec declares that "undefined behaviour" for example.

Quoted 2 lines

I think the spec is vague and doesn’t demand a strict order. So may be better to sync the registers after the alloc.

Agreed.

Nicolin

#62 lore raw

Re: [PATCH v4 20/31] hw/arm/tegra241-cmdqv: Use mmap'd VINTF page0 as VCMDQ backing

In reply to Shameer Kolothum
Quoted 5 lines · Shameer Kolothum

Introduce tegra241_cmdqv_vintf_ptr() to route VCMDQ register accesses through the mmap'd VINTF page0 backing once a hardware queue has been allocated.

There are two QEMU trapped MMIO apertures for VCMDQ registers:

s/VCMDQ registers/ VCMDQ Page0 registers

Quoted 2 lines

- Direct VCMDQ aperture (offset 0x10000)

emphasize this is page 0

Quoted 72 lines

- VINTF Page0 (offset 0x30000)

These are hardware aliases: they address the same underlying registers. A subsequent patch maps the VINTF aperture as a guest-direct RAM region; in this patch both remain QEMU-trapped.

VCMDQ register accesses operate in one of two mutually exclusive modes, depending on whether a hardware queue (IOMMU_HW_QUEUE_ALLOC) has been allocated for the VCMDQ:

Pre-alloc: vintf_ptr is NULL. Both apertures use QEMU's register cache. Hardware is not yet engaged;

Post-alloc: vintf_ptr is valid. Both QEMU trapped apertures access registers directly via the mmap'd vintf_page0 pointer, bypassing the cache. Hardware is the single source of truth.

The pre-to-post-alloc transition is triggered by the BASE register write that initiates IOMMU_HW_QUEUE_ALLOC. No cache-to-hardware synchronisation is needed at transition time. The hardware mandated init sequence requires BASE to be written first; PROD_INDX, CONS_INDX and CONFIG.CMDQ_EN are programmed only after BASE and are therefore always post-alloc.

Any pre-alloc writes to those registers update only the register cache, which is discarded at the transition.

CMDQV acceleration only becomes active once the guest enables VINTF and programs the VCMDQ BASE register. Until then, all VCMDQ accesses are served from the emulated register cache with no real hardware command processing. This matches the CMDQV hardware specification: if the logical CMDQ index does not map to any allocated Virtual CMDQ, "the access is dropped with no Fault/Interrupt".

---
 hw/arm/tegra241-cmdqv.c | 48 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 47 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +15 -1
index b5f2f74cf2..eb619e1134 100644
@@ -75,17 +75,45 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     return true;
 }
 
+static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
+                                                 int index, hwaddr offset0)
+{
+    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
+        return NULL;
+    }
+    return (uint32_t *)(cmdqv->vintf_page0 +
+                        (index * CMDQV_VCMDQ_STRIDE) +
+                        (offset0 - CMDQV_VCMDQ_PAGE0_BASE));
+}
+
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
  *
- * All VCMDQ accesses return cached registers.
+ * If the VCMDQ is allocated and VINTF page0 is mmap'd, read directly
+ * from the VINTF page0 backing. Otherwise, fall back to cached state.
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                           int index)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 112

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:

are there other page0 regs which are not modeled for which this implement would return 0?

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 115

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+            break;
+        }
+    }

I would prefer we have a clear separation between page0 accesses and page1 accesses (meaning 2 different helpers).

This would help understand that page1 accesses are always trapped whereas page0 access can be either cached or mmapped.

Quoted 24 lines
hw/arm/tegra241-cmdqv.c +12 -0
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -120,11 +148,29 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * If the VCMDQ is allocated and VINTF page0 is mmap'd, write directly
+ * to the VINTF page0 backing. Otherwise, update cached state.
  */
 static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                        int index, uint64_t value,
                                        unsigned size, Error **errp)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, · line 176

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = (uint32_t)value;

Thanks

Eric

#63 lore raw

PATCH v4 21/31 memory: Allow RAM device regions to skip IOMMU mapping

In reply to Shameer Kolothum

Some RAM device regions created with memory_region_init_ram_device_ptr() are not intended to be P2P DMA targets.

The VFIO listener currently treats all RAM device regions as DMA capable and attempts to map them into the IOMMU. For regions without dma-buf backing this fails and prints warnings such as:

IOMMU_IOAS_MAP failed: Bad address, PCI BAR?

Introduce a MemoryRegion flag (ram_device_skip_iommu_map) to mark RAM device regions that should not be IOMMU mapped. When set, the VFIO listener skips DMA mapping for that region.

---
 include/system/memory.h | 2 ++
 hw/vfio/listener.c      | 5 +++++
 2 files changed, 7 insertions(+)
include/system/memory.h +2 -0
index 7aed255e81..9df15e833a 100644
@@ -864,6 +864,8 @@ struct MemoryRegion {
 
     /* For devices designed to perform re-entrant IO into their own IO MRs */
     bool disable_reentrancy_guard;
+    /* RAM device region that does not require IOMMU mapping for P2P */
+    bool ram_device_skip_iommu_map;
 };
 
 struct IOMMUMemoryRegion {
hw/vfio/listener.c +5 -0
index 960da9e0a9..32d33a740a 100644
@@ -614,6 +614,11 @@ void vfio_container_region_add(VFIOContainer *bcontainer,
         }
     }
 
+    if (memory_region_is_ram_device(section->mr) &&
+        section->mr->ram_device_skip_iommu_map) {
+        return;
+    }
+
     ret = vfio_container_dma_map(bcontainer, iova, int128_get64(llsize),
                                  vaddr, section->readonly, section->mr);
     if (ret) {
Signature · 1 line
2.43.0
#64 lore raw

Re: [PATCH v4 21/31] memory: Allow RAM device regions to skip IOMMU mapping

Quoted 37 lines · Shameer Kolothum

Some RAM device regions created with memory_region_init_ram_device_ptr() are not intended to be P2P DMA targets.

The VFIO listener currently treats all RAM device regions as DMA capable and attempts to map them into the IOMMU. For regions without dma-buf backing this fails and prints warnings such as:

IOMMU_IOAS_MAP failed: Bad address, PCI BAR?

Introduce a MemoryRegion flag (ram_device_skip_iommu_map) to mark RAM device regions that should not be IOMMU mapped. When set, the VFIO listener skips DMA mapping for that region.

---
 include/system/memory.h | 2 ++
 hw/vfio/listener.c      | 5 +++++
 2 files changed, 7 insertions(+)
include/system/memory.h +2 -0
index 7aed255e81..9df15e833a 100644
@@ -864,6 +864,8 @@ struct MemoryRegion {
 
     /* For devices designed to perform re-entrant IO into their own IO MRs */
     bool disable_reentrancy_guard;
+    /* RAM device region that does not require IOMMU mapping for P2P */
+    bool ram_device_skip_iommu_map;
 };
 
 struct IOMMUMemoryRegion {
hw/vfio/listener.c +0 -0
index 960da9e0a9..32d33a740a 100644

hw/vfio/listener.c void vfio_container_region_add(VFIOContainer *bcontainer, · line 618

Quoted 5 lines
hw/vfio/listener.c +2 -0
@@ -614,6 +614,11 @@ void vfio_container_region_add(VFIOContainer *bcontainer,
         }
     }
 
+    if (memory_region_is_ram_device(section->mr) &&
+        section->mr->ram_device_skip_iommu_map) {

nit might be better the create an accessor including both checks like memory_region_is_protected

Thanks

Eric

Quoted 6 lines
hw/vfio/listener.c +3 -0
+        return;
+    }
+
     ret = vfio_container_dma_map(bcontainer, iova, int128_get64(llsize),
                                  vaddr, section->readonly, section->mr);
     if (ret) {
#65 lore raw

PATCH v4 22/31 hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region directly into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path CONS/PROD index updates.

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

The mapping is installed lazily on first successful VCMDQ hardware queue allocation and removed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)
hw/arm/tegra241-cmdqv.h +1 -0
index 039d86374f..2befa6205e 100644
@@ -46,6 +46,7 @@ typedef struct Tegra241CMDQV {
     IOMMUFDVeventq *veventq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
+    MemoryRegion *mr_vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +37 -0
index eb619e1134..bf989dd51f 100644
@@ -15,6 +15,40 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_guest_unmap_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    if (!cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    memory_region_del_subregion(&cmdqv->mmio_cmdqv, cmdqv->mr_vintf_page0);
+    object_unparent(OBJECT(cmdqv->mr_vintf_page0));
+    g_free(cmdqv->mr_vintf_page0);
+    cmdqv->mr_vintf_page0 = NULL;
+}
+
+static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    cmdqv->mr_vintf_page0 = g_malloc0(sizeof(*cmdqv->mr_vintf_page0));
+    memory_region_init_ram_device_ptr(cmdqv->mr_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,
+                                        CMDQV_VINTF_PAGE0_BASE,
+                                        cmdqv->mr_vintf_page0, 1);
+    g_free(name);
+}
+
 static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
 {
     IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
@@ -72,6 +106,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_guest_map_vintf_page0(cmdqv);
     return true;
 }
 
@@ -312,6 +347,7 @@ static void tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,
                 cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
             }
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
@@ -438,6 +474,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
Signature · 1 line
2.43.0
#66 lore raw

Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

Hi Shameer,

Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region directly into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path CONS/PROD index updates.

Quoted 5 lines

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

I fail to understand when the previous trapped path using ptr in tegra241_cmdqv_read/write_vcmdq gets used versus that path. Is it eventually used?

Quoted 50 lines

The mapping is installed lazily on first successful VCMDQ hardware queue allocation and removed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 37 +++++++++++++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)
hw/arm/tegra241-cmdqv.h +1 -0
index 039d86374f..2befa6205e 100644
@@ -46,6 +46,7 @@ typedef struct Tegra241CMDQV {
     IOMMUFDVeventq *veventq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
+    MemoryRegion *mr_vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +18 -0
index eb619e1134..bf989dd51f 100644
@@ -15,6 +15,40 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_guest_unmap_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    if (!cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    memory_region_del_subregion(&cmdqv->mmio_cmdqv, cmdqv->mr_vintf_page0);
+    object_unparent(OBJECT(cmdqv->mr_vintf_page0));
+    g_free(cmdqv->mr_vintf_page0);
+    cmdqv->mr_vintf_page0 = NULL;
+}
+
+static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->mr_vintf_page0) {
+        return;

hw/arm/tegra241-cmdqv.c line 45

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    cmdqv->mr_vintf_page0 = g_malloc0(sizeof(*cmdqv->mr_vintf_page0));
+    memory_region_init_ram_device_ptr(cmdqv->mr_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;

I guess you need a setter, here to.

Quoted 26 lines
hw/arm/tegra241-cmdqv.c +8 -0
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,
+                                        CMDQV_VINTF_PAGE0_BASE,
+                                        cmdqv->mr_vintf_page0, 1);
+    g_free(name);
+}
+
 static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
 {
     IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
@@ -72,6 +106,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_guest_map_vintf_page0(cmdqv);
     return true;
 }
 
@@ -312,6 +347,7 @@ static void tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,
                 cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
             }
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset, · line 480

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -438,6 +474,7 @@ static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }

Thanks

Eric

#67 lore raw

RE: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

Hi Eric,

Quoted 33 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 06 May 2026 13:45
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region directly into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path CONS/PROD index updates.

Quoted 5 lines

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

Quoted 3 lines

I fail to understand when the previous trapped path using ptr in tegra241_cmdqv_read/write_vcmdq gets used versus that path. Is it eventually used?

The spec does not prevent a guest from using the 0x10000 path for allocated VCMDQs, so the trap path remains valid and QEMU forwards those accesses to the mmap'd vintf_page0 via vintf_ptr.

We cannot map 0x10000 directly to the guest as RAM because the kernel mmap only backs VCMDQs actually allocated via HW_QUEUE ioctl. If the guest allocates only 1 of 2 VCMDQs, exposing the full direct aperture page0 as RAM would give the guest access to unallocated VCMDQ slots. Hence, it remains trapped and QEMU only fwds to vintf page0 for allocated queues.

Thanks,
Shameer

Quoted 105 lines
Quoted 10 lines

The mapping is installed lazily on first successful VCMDQ hardware queue allocation and removed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 37

+++++++++++++++++++++++++++++++++++++

Quoted 19 lines
 2 files changed, 38 insertions(+)
hw/arm/tegra241-cmdqv.h +1 -0
index 039d86374f..2befa6205e 100644
@@ -46,6 +46,7 @@ typedef struct Tegra241CMDQV {
     IOMMUFDVeventq *veventq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
+    MemoryRegion *mr_vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index eb619e1134..bf989dd51f 100644

hw/arm/tegra241-cmdqv.c line 18

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -15,6 +15,40 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_guest_unmap_vintf_page0(Tegra241CMDQV

*cmdqv)

hw/arm/tegra241-cmdqv.c

Quoted 13 lines
hw/arm/tegra241-cmdqv.c +12 -0
+{
+    if (!cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    memory_region_del_subregion(&cmdqv->mmio_cmdqv, cmdqv-
 mr_vintf_page0);
+    object_unparent(OBJECT(cmdqv->mr_vintf_page0));
+    g_free(cmdqv->mr_vintf_page0);
+    cmdqv->mr_vintf_page0 = NULL;
+}
+
+static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV

*cmdqv)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+{
+    char *name;
+
+    if (cmdqv->mr_vintf_page0) {
+        return;

hw/arm/tegra241-cmdqv.c line 45

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    cmdqv->mr_vintf_page0 = g_malloc0(sizeof(*cmdqv->mr_vintf_page0));
+    memory_region_init_ram_device_ptr(cmdqv->mr_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;

I guess you need a setter, here to.

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,
+                                        CMDQV_VINTF_PAGE0_BASE,
+                                        cmdqv->mr_vintf_page0, 1);
+    g_free(name);
+}
+
 static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int

index)

hw/arm/tegra241-cmdqv.c static bool

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 {
     IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
hw/arm/tegra241-cmdqv.c +0 -0
@@ -72,6 +106,7 @@ static bool

tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,

hw/arm/tegra241-cmdqv.c static void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_guest_map_vintf_page0(cmdqv);
     return true;
 }
 
@@ -312,6 +347,7 @@ static void

tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
                 cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
             }
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
@@ -438,6 +474,7 @@ static void tegra241_cmdqv_write_mmio(void

*opaque, hwaddr offset,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset, · line 480

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +1 -0
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }

Thanks

Eric

#68 lore raw

Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

Quoted 41 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 27 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 06 May 2026 13:45
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region directly into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path CONS/PROD index updates.

Quoted 8 lines
Quoted 5 lines

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

I fail to understand when the previous trapped path using ptr in tegra241_cmdqv_read/write_vcmdq gets used versus that path. Is it eventually used?

The spec does not prevent a guest from using the 0x10000 path for allocated VCMDQs, so the trap path remains valid and QEMU forwards those accesses to the mmap'd vintf_page0 via vintf_ptr.

Quoted 5 lines

We cannot map 0x10000 directly to the guest as RAM because the kernel mmap only backs VCMDQs actually allocated via HW_QUEUE ioctl. If the guest allocates only 1 of 2 VCMDQs, exposing the full direct aperture page0 as RAM would give the guest access to unallocated VCMDQ slots. Hence, it remains trapped and QEMU only fwds to vintf page0 for allocated queues.

Actually my question rather was why don't we use a subregion_overlap for global vcmdq page0, mapping to vintf_page0 once cmdqv->vintf_page0 is set, just as we do for VINTF0 page0? I understand tegra241_cmdqv_read/write_vcmdq would return the same content anyway.

Besides, I would recommend that all over the code, page0 and page1 are clearly differentiated in terms of accessors and comments and we use the same terminology through the code for global vcmdq and vintf vcmdqs.

Thanks

Eric

Quoted 108 lines

Thanks,
Shameer

Quoted 104 lines
Quoted 9 lines

The mapping is installed lazily on first successful VCMDQ hardware queue allocation and removed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 37

+++++++++++++++++++++++++++++++++++++

Quoted 19 lines
 2 files changed, 38 insertions(+)
hw/arm/tegra241-cmdqv.h +1 -0
index 039d86374f..2befa6205e 100644
@@ -46,6 +46,7 @@ typedef struct Tegra241CMDQV {
     IOMMUFDVeventq *veventq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
+    MemoryRegion *mr_vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index eb619e1134..bf989dd51f 100644

hw/arm/tegra241-cmdqv.c line 18

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -15,6 +15,40 @@
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
+static void tegra241_cmdqv_guest_unmap_vintf_page0(Tegra241CMDQV

*cmdqv)

hw/arm/tegra241-cmdqv.c

Quoted 13 lines
hw/arm/tegra241-cmdqv.c +12 -0
+{
+    if (!cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    memory_region_del_subregion(&cmdqv->mmio_cmdqv, cmdqv-
 mr_vintf_page0);
+    object_unparent(OBJECT(cmdqv->mr_vintf_page0));
+    g_free(cmdqv->mr_vintf_page0);
+    cmdqv->mr_vintf_page0 = NULL;
+}
+
+static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV

*cmdqv)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+{
+    char *name;
+
+    if (cmdqv->mr_vintf_page0) {
+        return;

hw/arm/tegra241-cmdqv.c line 45

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    cmdqv->mr_vintf_page0 = g_malloc0(sizeof(*cmdqv->mr_vintf_page0));
+    memory_region_init_ram_device_ptr(cmdqv->mr_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;

I guess you need a setter, here to.

hw/arm/tegra241-cmdqv.c

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,
+                                        CMDQV_VINTF_PAGE0_BASE,
+                                        cmdqv->mr_vintf_page0, 1);
+    g_free(name);
+}
+
 static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int

index)

hw/arm/tegra241-cmdqv.c static bool

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 {
     IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
hw/arm/tegra241-cmdqv.c +0 -0
@@ -72,6 +106,7 @@ static bool

tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,

hw/arm/tegra241-cmdqv.c static void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_guest_map_vintf_page0(cmdqv);
     return true;
 }
 
@@ -312,6 +347,7 @@ static void

tegra241_cmdqv_config_vintf_write(Tegra241CMDQV *cmdqv,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
                 cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
             }
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
@@ -438,6 +474,7 @@ static void tegra241_cmdqv_write_mmio(void

*opaque, hwaddr offset,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write_mmio(void *opaque, hwaddr offset, · line 480

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +1 -0
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }

Thanks

Eric

#69 lore raw

RE: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

Hi Eric,

Quoted 67 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Quoted 10 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 8 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 06 May 2026 13:45
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason

Gunthorpe

Quoted 24 lines
Quoted 24 lines

<[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region directly into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path CONS/PROD index updates.

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

Quoted 4 lines
Quoted 3 lines

I fail to understand when the previous trapped path using ptr in tegra241_cmdqv_read/write_vcmdq gets used versus that path. Is it eventually used?

The spec does not prevent a guest from using the 0x10000 path for

allocated

Quoted 1 line

VCMDQs, so the trap path remains valid and QEMU forwards those accesses

to

Quoted 7 lines

the mmap'd vintf_page0 via vintf_ptr.

We cannot map 0x10000 directly to the guest as RAM because the kernel mmap only backs VCMDQs actually allocated via HW_QUEUE ioctl. If the guest allocates only 1 of 2 VCMDQs, exposing the full direct aperture page0 as RAM would give the guest access to unallocated VCMDQ slots. Hence, it remains trapped and QEMU only fwds to vintf page0 for allocated queues.

Quoted 4 lines

Actually my question rather was why don't we use a subregion_overlap for global vcmdq page0, mapping to vintf_page0 once cmdqv->vintf_page0 is set, just as we do for VINTF0 page0? I understand tegra241_cmdqv_read/write_vcmdq would return the same content anyway.

This is my understanding why we cannot map direct VCMDQ aperture(0x10000) to vintf_page0:

- Unallocated VCMDQs accessed via vintf_page0 get "access dropped with no Fault/Interrupt" behaviour per spec((p.172)

- Mapping the direct VCMDQ aperture (0x10000) to vintf_page0 would replicate this behaviour for unallocated VCMDQ accesses via the direct aperture too.

- However, per spec p.176, software can program VCMDQs via the direct VCMDQ aperture without going through the VINTF. Mapping vintf_page0 at 0x10000 violates this.

- Partial allocation case (VCMDQ0 allocated, VCMDQ1 not): the trap correctly serves VCMDQ1 accesses from QEMU cache while forwarding VCMDQ0 accesses to vintf_page0. A direct RAM mapping cannot make this distinction.

Quoted 4 lines

Besides, I would recommend that all over the code, page0 and page1 are clearly differentiated in terms of accessors and comments and we use the same terminology through the code for global vcmdq and vintf vcmdqs.

Sure. I will address this.

Thanks,
Shameer

#70 lore raw

Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

Quoted 82 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 67 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Quoted 10 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 8 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 06 May 2026 13:45
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason

Gunthorpe

Quoted 24 lines
Quoted 24 lines

<[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region directly into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path CONS/PROD index updates.

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

Quoted 4 lines
Quoted 3 lines

I fail to understand when the previous trapped path using ptr in tegra241_cmdqv_read/write_vcmdq gets used versus that path. Is it eventually used?

The spec does not prevent a guest from using the 0x10000 path for

allocated

Quoted 1 line

VCMDQs, so the trap path remains valid and QEMU forwards those accesses

to

Quoted 7 lines

the mmap'd vintf_page0 via vintf_ptr.

We cannot map 0x10000 directly to the guest as RAM because the kernel mmap only backs VCMDQs actually allocated via HW_QUEUE ioctl. If the guest allocates only 1 of 2 VCMDQs, exposing the full direct aperture page0 as RAM would give the guest access to unallocated VCMDQ slots. Hence, it remains trapped and QEMU only fwds to vintf page0 for allocated queues.

Quoted 4 lines

Actually my question rather was why don't we use a subregion_overlap for global vcmdq page0, mapping to vintf_page0 once cmdqv->vintf_page0 is set, just as we do for VINTF0 page0? I understand tegra241_cmdqv_read/write_vcmdq would return the same content anyway.

This is my understanding why we cannot map direct VCMDQ aperture(0x10000) to vintf_page0:

- Unallocated VCMDQs accessed via vintf_page0 get "access dropped with no Fault/Interrupt" behaviour per spec((p.172)

- Mapping the direct VCMDQ aperture (0x10000) to vintf_page0 would replicate this behaviour for unallocated VCMDQ accesses via the direct aperture too.

Quoted 7 lines

- However, per spec p.176, software can program VCMDQs via the direct VCMDQ aperture without going through the VINTF. Mapping vintf_page0 at 0x10000 violates this.

- Partial allocation case (VCMDQ0 allocated, VCMDQ1 not): the trap correctly serves VCMDQ1 accesses from QEMU cache while forwarding VCMDQ0 accesses to vintf_page0. A direct RAM mapping cannot make this distinction.

I agree that normally we shouldn't use VINTF view for the global view.

When you say "the trap correctly serves VCMDQ1 accesses from QEMU cache while forwarding VCMDQ0 accesses to vintf_page0", the filtering is done by tegra241_cmdqv_vintf_ptr. Whatever the queue being enabled, cmdqv->vintf_page0 is allocated. So only the test on cmdqv->vcmdq[index] does the filtering in that case, correct?

If so I now share your pov

Eric

Quoted 9 lines
Quoted 4 lines

Besides, I would recommend that all over the code, page0 and page1 are clearly differentiated in terms of accessors and comments and we use the same terminology through the code for global vcmdq and vintf vcmdqs.

Sure. I will address this.

Thanks,
Shameer

#71 lore raw

RE: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

Quoted 124 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 08 May 2026 15:35
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

External email: Use caution opening links or attachments

Quoted 14 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 12 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF

page0

Quoted 6 lines
Quoted 6 lines

into guest MMIO space

External email: Use caution opening links or attachments

On 5/6/26 4:24 PM, Shameer Kolothum Thodi wrote:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hi Eric,

Quoted 6 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 06 May 2026 13:45
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];

Nicolin

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Chen <[email protected]>; Nathan Chen <[email protected]>;

Matt

Quoted 6 lines
Quoted 6 lines
Quoted 1 line
Quoted 1 line

Ochs <[email protected]>; Jiandi An <[email protected]>; Jason

Gunthorpe

Quoted 4 lines
Quoted 4 lines

<[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 22/31] hw/arm/tegra241-cmdqv: Map VINTF

page0

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

into guest MMIO space

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Once a VCMDQ is allocated, map the mmap'd vintf_page0 region

directly

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

into the guest-visible MMIO space at offset 0x30000 as a RAM-backed MemoryRegion. This eliminates QEMU trapping for hot-path

CONS/PROD

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

index updates.

After this patch, the two VCMDQ apertures use different access paths:
the direct aperture (0x10000) remains QEMU-trapped and writes via vintf_ptr, while the VI aperture (0x30000) is a direct guest RAM mapping. Both paths write to the same underlying vintf_page0 memory, so no synchronisation between the apertures is needed.

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 3 lines

I fail to understand when the previous trapped path using ptr in tegra241_cmdqv_read/write_vcmdq gets used versus that path. Is it eventually used?

The spec does not prevent a guest from using the 0x10000 path for

allocated

Quoted 1 line

VCMDQs, so the trap path remains valid and QEMU forwards those

accesses

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines

the mmap'd vintf_page0 via vintf_ptr.

We cannot map 0x10000 directly to the guest as RAM because the kernel mmap only backs VCMDQs actually allocated via HW_QUEUE ioctl. If the guest allocates only 1 of 2 VCMDQs, exposing the full direct aperture

page0

Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

as RAM would give the guest access to unallocated VCMDQ slots. Hence, it remains trapped and QEMU only fwds to vintf page0 for allocated queues.

Actually my question rather was why don't we use a subregion_overlap for global vcmdq page0, mapping to vintf_page0 once cmdqv->vintf_page0 is set, just as we do for VINTF0 page0? I understand tegra241_cmdqv_read/write_vcmdq would return the same content

anyway.

Quoted 1 line

This is my understanding why we cannot map direct VCMDQ

aperture(0x10000)

Quoted 3 lines

to vintf_page0:

- Unallocated VCMDQs accessed via vintf_page0 get "access dropped with

no

Quoted 3 lines

Fault/Interrupt" behaviour per spec((p.172)

- Mapping the direct VCMDQ aperture (0x10000) to vintf_page0 would

replicate

Quoted 7 lines

this behaviour for unallocated VCMDQ accesses via the direct aperture too.

- However, per spec p.176, software can program VCMDQs via the direct VCMDQ aperture without going through the VINTF. Mapping vintf_page0 at 0x10000 violates this.

- Partial allocation case (VCMDQ0 allocated, VCMDQ1 not): the trap

correctly

Quoted 1 line

serves VCMDQ1 accesses from QEMU cache while forwarding VCMDQ0

accesses

Quoted 1 line

to vintf_page0. A direct RAM mapping cannot make this distinction.

Quoted 7 lines

I agree that normally we shouldn't use VINTF view for the global view.

When you say "the trap correctly serves VCMDQ1 accesses from QEMU cache while forwarding VCMDQ0 accesses to vintf_page0", the filtering is done by tegra241_cmdqv_vintf_ptr. Whatever the queue being enabled, cmdqv->vintf_page0 is allocated. So only the test on cmdqv->vcmdq[index] does the filtering in that case, correct?

Yes. That is correct.

Thanks,
Shameer

#72 lore raw

PATCH v4 23/31 hw/arm/smmuv3-accel: Introduce common helper for veventq read

In reply to Shameer Kolothum

Move the vEVENTQ read and validation logic into a common helper smmuv3_accel_event_read_validate(). The helper performs the read(), checks for overflow and short reads, validates the sequence number, and updates the sequence state.

This helper can be reused for Tegra241 CMDQV vEVENTQ support in a subsequent patch.

Error handling is slightly adjusted: instead of reporting errors directly in the read handler, the helper now returns errors via Error **. Sequence gaps are reported as warnings.

---
 hw/arm/smmuv3-accel.h       |  2 ++
 hw/arm/smmuv3-accel-stubs.c | 11 ++++++
 hw/arm/smmuv3-accel.c       | 67 ++++++++++++++++++++++---------------
 3 files changed, 53 insertions(+), 27 deletions(-)
hw/arm/smmuv3-accel.h +2 -0
index 28bceca061..448f47c0ca 100644
@@ -71,6 +71,8 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,
                                 Error **errp);
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +11 -0
index c08caa6fa4..e8f08dc833 100644
@@ -41,6 +41,17 @@ void smmuv3_accel_idr_override(SMMUv3State *s)
 {
 }
 
+bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
+{
+    return true;
+}
+
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp)
+{
+    return true;
+}
+
 void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
hw/arm/smmuv3-accel.c +40 -27

87 lines not on this page · raw message

Signature · 1 line
2.43.0
Reviewed-by on v4 23/31
#73 lore raw

Re: [PATCH v4 23/31] hw/arm/smmuv3-accel: Introduce common helper for veventq read

Quoted 5 lines · Shameer Kolothum

Move the vEVENTQ read and validation logic into a common helper smmuv3_accel_event_read_validate(). The helper performs the read(), checks for overflow and short reads, validates the sequence number, and updates the sequence state.

Quoted 8 lines

This helper can be reused for Tegra241 CMDQV vEVENTQ support in a subsequent patch.

Error handling is slightly adjusted: instead of reporting errors directly in the read handler, the helper now returns errors via Error **. Sequence gaps are reported as warnings.

Reviewed-by on v4 23/31
#74 lore raw

Re: [PATCH v4 23/31] hw/arm/smmuv3-accel: Introduce common helper for veventq read

In reply to Shameer Kolothum
Quoted 5 lines · Shameer Kolothum

Move the vEVENTQ read and validation logic into a common helper smmuv3_accel_event_read_validate(). The helper performs the read(), checks for overflow and short reads, validates the sequence number, and updates the sequence state.

Quoted 8 lines

This helper can be reused for Tegra241 CMDQV vEVENTQ support in a subsequent patch.

Error handling is slightly adjusted: instead of reporting errors directly in the read handler, the helper now returns errors via Error **. Sequence gaps are reported as warnings.

Eric

Quoted 133 lines
---
 hw/arm/smmuv3-accel.h       |  2 ++
 hw/arm/smmuv3-accel-stubs.c | 11 ++++++
 hw/arm/smmuv3-accel.c       | 67 ++++++++++++++++++++++---------------
 3 files changed, 53 insertions(+), 27 deletions(-)
hw/arm/smmuv3-accel.h +2 -0
index 28bceca061..448f47c0ca 100644
@@ -71,6 +71,8 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,
                                 Error **errp);
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +11 -0
index c08caa6fa4..e8f08dc833 100644
@@ -41,6 +41,17 @@ void smmuv3_accel_idr_override(SMMUv3State *s)
 {
 }
 
+bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
+{
+    return true;
+}
+
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp)
+{
+    return true;
+}
+
 void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
hw/arm/smmuv3-accel.c +40 -27
index 9068e65e2b..230f608f03 100644
@@ -436,47 +436,60 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *bs, void *cmd, SMMUDevice *sdev,
                    sizeof(Cmd), &entry_num, cmd, errp);
 }
 
-static void smmuv3_accel_event_read(void *opaque)
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp)
 {
-    SMMUv3State *s = opaque;
-    IOMMUFDVeventq *veventq = s->s_accel->veventq;
-    struct {
-        struct iommufd_vevent_header hdr;
-        struct iommu_vevent_arm_smmuv3 vevent;
-    } buf;
-    enum iommu_veventq_type type = IOMMU_VEVENTQ_TYPE_ARM_SMMUV3;
-    uint32_t id = veventq->veventq_id;
     uint32_t last_seq = veventq->last_event_seq;
+    uint32_t id = veventq->veventq_id;
+    struct iommufd_vevent_header *hdr;
     ssize_t bytes;
 
-    bytes = read(veventq->veventq_fd, &buf, sizeof(buf));
+    bytes = read(veventq->veventq_fd, buf, size);
     if (bytes <= 0) {
         if (errno == EAGAIN || errno == EINTR) {
-            return;
+            return true;
         }
-        error_report_once("vEVENTQ(type %u id %u): read failed (%m)", type, id);
-        return;
+        error_setg(errp, "vEVENTQ(type %u id %u): read failed (%m)", type, id);
+        return false;
     }
-
-    if (bytes == sizeof(buf.hdr) &&
-        (buf.hdr.flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
-        error_report_once("vEVENTQ(type %u id %u): overflowed", type, id);
+    hdr = (struct iommufd_vevent_header *)buf;
+    if (bytes == sizeof(*hdr) &&
+        (hdr->flags & IOMMU_VEVENTQ_FLAG_LOST_EVENTS)) {
+        error_setg(errp, "vEVENTQ(type %u id %u): overflowed", type, id);
         veventq->event_start = false;
-        return;
+        return false;
     }
-    if (bytes < sizeof(buf)) {
-        error_report_once("vEVENTQ(type %u id %u): short read(%zd/%zd bytes)",
-                          type, id, bytes, sizeof(buf));
-        return;
+    if (bytes < size) {
+        error_setg(errp, "vEVENTQ(type %u id %u): short read(%zd/%zd bytes)",
+                          type, id, bytes, size);
+        return false;
     }
-
     /* Check sequence in hdr for lost events if any */
-    if (veventq->event_start && (buf.hdr.sequence - last_seq != 1)) {
-        error_report_once("vEVENTQ(type %u id %u): lost %u event(s)",
-                          type, id, buf.hdr.sequence - last_seq - 1);
+    if (veventq->event_start && (hdr->sequence - last_seq != 1)) {
+        warn_report("vEVENTQ(type %u id %u): lost %u event(s)",
+                    type, id, hdr->sequence - last_seq - 1);
     }
-    veventq->last_event_seq = buf.hdr.sequence;
+    veventq->last_event_seq = hdr->sequence;
     veventq->event_start = true;
+    return true;
+}
+
+static void smmuv3_accel_event_read(void *opaque)
+{
+    SMMUv3State *s = opaque;
+    IOMMUFDVeventq *veventq = s->s_accel->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_arm_smmuv3 vevent;
+    } buf;
+    Error *local_err = NULL;
+
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_ARM_SMMUV3, &buf,
+                                          sizeof(buf), &local_err)) {
+        warn_report_err_once(local_err);
+        return;
+    }
     smmuv3_propagate_event(s, (Evt *)&buf.vevent);
 }
 
#75 lore raw

PATCH v4 24/31 hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

In reply to Shameer Kolothum

Install an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().

---
 hw/arm/tegra241-cmdqv.c | 55 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 2 files changed, 56 insertions(+)
hw/arm/tegra241-cmdqv.c +55 -0
index bf989dd51f..9c2fc02b92 100644
@@ -11,6 +11,7 @@
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
+#include "hw/core/irq.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
@@ -534,6 +535,43 @@ out:
     trace_tegra241_cmdqv_write_mmio(offset, value, size);
 }
 
+static void tegra241_cmdqv_event_read(void *opaque)
+{
+    Tegra241CMDQV *cmdqv = opaque;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_tegra241_cmdqv vevent;
+    } buf;
+    Error *local_err = NULL;
+
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                          &buf, sizeof(buf), &local_err)) {
+        warn_report_err_once(local_err);
+        return;
+    }
+
+    if (buf.vevent.lvcmdq_err_map[0] || buf.vevent.lvcmdq_err_map[1]) {
+        cmdqv->vintf_cmdq_err_map[0] =
+            buf.vevent.lvcmdq_err_map[0] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[1] =
+            (buf.vevent.lvcmdq_err_map[0] >> 32) & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[2] =
+            buf.vevent.lvcmdq_err_map[1] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[3] =
+            (buf.vevent.lvcmdq_err_map[1] >> 32) & 0xffffffff;
+        for (int i = 0; i < 4; i++) {
+            cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i];
+        }
+        cmdqv->vi_err_map[0] |= 0x1;
+        qemu_irq_pulse(cmdqv->irq);
+        trace_tegra241_cmdqv_err_map(
+            cmdqv->vintf_cmdq_err_map[3], cmdqv->vintf_cmdq_err_map[2],
+            cmdqv->vintf_cmdq_err_map[1], cmdqv->vintf_cmdq_err_map[0]);
+    }
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
     SMMUv3AccelState *accel = s->s_accel;
@@ -545,6 +583,7 @@ static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
         return;
     }
     if (veventq) {
+        qemu_set_fd_handler(veventq->veventq_fd, NULL, NULL, NULL);
         close(veventq->veventq_fd);
         iommufd_backend_free_id(viommu->iommufd, veventq->veventq_id);
         g_free(veventq);
@@ -560,6 +599,7 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
     uint32_t viommu_id, veventq_id, veventq_fd;
     IOMMUFDVeventq *veventq;
+    int flags;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
                                       IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
@@ -577,14 +617,29 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_viommu;
     }
 
+    flags = fcntl(veventq_fd, F_GETFL);
+    if (flags < 0) {
+        error_setg(errp, "Failed to get flags for vEVENTQ fd");
+        goto free_veventq;
+    }
+    if (fcntl(veventq_fd, F_SETFL, O_NONBLOCK | flags) < 0) {
+        error_setg(errp, "Failed to set O_NONBLOCK on vEVENTQ fd");
+        goto free_veventq;
+    }
+
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     cmdqv->veventq = veventq;
 
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL, cmdqv);
     *out_viommu_id = viommu_id;
     return true;
 
+free_veventq:
+    close(veventq_fd);
+    iommufd_backend_free_id(idev->iommufd, veventq_id);
 free_viommu:
     iommufd_backend_free_id(idev->iommufd, viommu_id);
     return false;
hw/arm/trace-events +1 -0
index 8c61d66a26..fd6441bfa7 100644
@@ -75,6 +75,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 # tegra241-cmdqv
 tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
+tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
Reviewed-by on v4 24/31
#76 lore raw

Re: [PATCH v4 24/31] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

Quoted 7 lines · Shameer Kolothum

Install an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().

#77 lore raw

Re: [PATCH v4 24/31] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

In reply to Shameer Kolothum

Hi Shameer,

Quoted 38 lines · Shameer Kolothum

Install an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().

---
 hw/arm/tegra241-cmdqv.c | 55 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 2 files changed, 56 insertions(+)
hw/arm/tegra241-cmdqv.c +11 -0
index bf989dd51f..9c2fc02b92 100644
@@ -11,6 +11,7 @@
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
+#include "hw/core/irq.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
@@ -534,6 +535,43 @@ out:
     trace_tegra241_cmdqv_write_mmio(offset, value, size);
 }
 
+static void tegra241_cmdqv_event_read(void *opaque)
+{
+    Tegra241CMDQV *cmdqv = opaque;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_tegra241_cmdqv vevent;
+    } buf;
+    Error *local_err = NULL;
+

hw/arm/tegra241-cmdqv.c out: · line 557

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                          &buf, sizeof(buf), &local_err)) {
+        warn_report_err_once(local_err);
+        return;
+    }
+
+    if (buf.vevent.lvcmdq_err_map[0] || buf.vevent.lvcmdq_err_map[1]) {
+        cmdqv->vintf_cmdq_err_map[0] =
+            buf.vevent.lvcmdq_err_map[0] & 0xffffffff;

can't you use extract64() here and below?

hw/arm/tegra241-cmdqv.c out: · line 565

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+        cmdqv->vintf_cmdq_err_map[1] =
+            (buf.vevent.lvcmdq_err_map[0] >> 32) & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[2] =
+            buf.vevent.lvcmdq_err_map[1] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[3] =
+            (buf.vevent.lvcmdq_err_map[1] >> 32) & 0xffffffff;
+        for (int i = 0; i < 4; i++) {
+            cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i];

if the cached regs are same to we need to have both?

Quoted 63 lines
hw/arm/tegra241-cmdqv.c +26 -0
+        }
+        cmdqv->vi_err_map[0] |= 0x1;
+        qemu_irq_pulse(cmdqv->irq);
+        trace_tegra241_cmdqv_err_map(
+            cmdqv->vintf_cmdq_err_map[3], cmdqv->vintf_cmdq_err_map[2],
+            cmdqv->vintf_cmdq_err_map[1], cmdqv->vintf_cmdq_err_map[0]);
+    }
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
     SMMUv3AccelState *accel = s->s_accel;
@@ -545,6 +583,7 @@ static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
         return;
     }
     if (veventq) {
+        qemu_set_fd_handler(veventq->veventq_fd, NULL, NULL, NULL);
         close(veventq->veventq_fd);
         iommufd_backend_free_id(viommu->iommufd, veventq->veventq_id);
         g_free(veventq);
@@ -560,6 +599,7 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
     uint32_t viommu_id, veventq_id, veventq_fd;
     IOMMUFDVeventq *veventq;
+    int flags;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
                                       IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
@@ -577,14 +617,29 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_viommu;
     }
 
+    flags = fcntl(veventq_fd, F_GETFL);
+    if (flags < 0) {
+        error_setg(errp, "Failed to get flags for vEVENTQ fd");
+        goto free_veventq;
+    }
+    if (fcntl(veventq_fd, F_SETFL, O_NONBLOCK | flags) < 0) {
+        error_setg(errp, "Failed to set O_NONBLOCK on vEVENTQ fd");
+        goto free_veventq;
+    }
+
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     cmdqv->veventq = veventq;
 
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL, cmdqv);
     *out_viommu_id = viommu_id;
     return true;
 
+free_veventq:
+    close(veventq_fd);
+    iommufd_backend_free_id(idev->iommufd, veventq_id);
 free_viommu:
     iommufd_backend_free_id(idev->iommufd, viommu_id);
     return false;
hw/arm/trace-events +0 -0
index 8c61d66a26..fd6441bfa7 100644

hw/arm/trace-events smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS · line 81

Quoted 7 lines
hw/arm/trace-events +1 -0
@@ -75,6 +75,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 # tegra241-cmdqv
 tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
+tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"

Eric

#78 lore raw

RE: [PATCH v4 24/31] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

Quoted 62 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 17:41
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 24/31] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 9 lines · Shameer Kolothum

Install an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().

---
 hw/arm/tegra241-cmdqv.c | 55

+++++++++++++++++++++++++++++++++++++++++

Quoted 30 lines
 hw/arm/trace-events     |  1 +
 2 files changed, 56 insertions(+)
hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0 -0

bf989dd51f..9c2fc02b92 100644

hw/arm/tegra241-cmdqv.c +13 -0
@@ -11,6 +11,7 @@
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
+#include "hw/core/irq.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
@@ -534,6 +535,43 @@ out:
     trace_tegra241_cmdqv_write_mmio(offset, value, size);  }
 
+static void tegra241_cmdqv_event_read(void *opaque) {
+    Tegra241CMDQV *cmdqv = opaque;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_tegra241_cmdqv vevent;
+    } buf;
+    Error *local_err = NULL;
+
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                          &buf, sizeof(buf), &local_err)) {
Quoted 8 lines

hw/arm/tegra241-cmdqv.c out: · line 557

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+        warn_report_err_once(local_err);
+        return;
+    }
+
+    if (buf.vevent.lvcmdq_err_map[0] || buf.vevent.lvcmdq_err_map[1]) {
+        cmdqv->vintf_cmdq_err_map[0] =
+            buf.vevent.lvcmdq_err_map[0] & 0xffffffff;

can't you use extract64() here and below?

Yes. that’s better.

Quoted 9 lines

hw/arm/tegra241-cmdqv.c out: · line 565

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+        cmdqv->vintf_cmdq_err_map[1] =
+            (buf.vevent.lvcmdq_err_map[0] >> 32) & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[2] =
+            buf.vevent.lvcmdq_err_map[1] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[3] =
+            (buf.vevent.lvcmdq_err_map[1] >> 32) & 0xffffffff;
+        for (int i = 0; i < 4; i++) {
+            cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i];

if the cached regs are same to we need to have both?

They are same for now because we only support one VINTF0. I think better to keep them separate as per spec. I will add a comment.

Thanks,
Shameer

#79 lore raw

PATCH v4 25/31 hw/arm/tegra241-cmdqv: Add reset handler

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/tegra241-cmdqv.c | 50 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 3 files changed, 53 insertions(+)
hw/arm/tegra241-cmdqv.h +2 -0
index 2befa6205e..b2a444daef 100644
@@ -79,6 +79,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/tegra241-cmdqv.c +50 -0

64 lines not on this page · raw message

hw/arm/trace-events +1 -0
index fd6441bfa7..6f602b9eda 100644
@@ -76,6 +76,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
#80 lore raw

Re: [PATCH v4 25/31] hw/arm/tegra241-cmdqv: Add reset handler

Quoted 39 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/tegra241-cmdqv.c | 50 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 3 files changed, 53 insertions(+)
hw/arm/tegra241-cmdqv.h +2 -0
index 2befa6205e..b2a444daef 100644
@@ -79,6 +79,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/tegra241-cmdqv.c +1 -0
index 9c2fc02b92..af68add2f0 100644
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"

hw/arm/tegra241-cmdqv.c free_viommu: · line 654

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6 -0
@@ -645,8 +646,57 @@ free_viommu:
     return false;
 }
 
+static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
+{
+    int i;
+
+    cmdqv->config = V_CONFIG_RESET;
+    cmdqv->param = FIELD_DP32(0, PARAM, CMDQV_VER, CMDQV_VER);

hw/core/registerfields.h:#define FIELD_DP32(storage, reg, field, val) what does this 0?

hw/arm/tegra241-cmdqv.c free_viommu: · line 656

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM, CMDQV_NUM_CMDQ_LOG2,
+                              CMDQV_NUM_CMDQ_LOG2);

I was a bit puzzled by the FIELD name being the value name but well.

hw/arm/tegra241-cmdqv.c free_viommu: · line 667

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +11 -0
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM, CMDQV_NUM_SID_PER_VI_LOG2,
+                              CMDQV_NUM_SID_PER_VI_LOG2);
+    trace_tegra241_cmdqv_init_regs(cmdqv->param);
+    cmdqv->status = R_STATUS_CMDQV_ENABLED_MASK;
+    for (i = 0; i < 2; i++) {
+        cmdqv->vi_err_map[i] = 0;
+        cmdqv->vi_int_mask[i] = 0;
+        cmdqv->cmdq_err_map[i] = 0;
+    }
+    cmdqv->cmdq_err_map[2] = 0;
+    cmdqv->cmdq_err_map[3] = 0;

the split looks pretty strange, ie. loop of 2 + continue with individual setting. why don't you do the init of

cmdqv->cmdq_err_map in the x4 loop below

Quoted 40 lines
hw/arm/tegra241-cmdqv.c +30 -0
+    cmdqv->vintf_config = 0;
+    cmdqv->vintf_status = 0;
+    for (i = 0; i < 4; i++) {
+        cmdqv->vintf_cmdq_err_map[i] = 0;
+    }
+    for (i = 0; i < TEGRA241_CMDQV_MAX_CMDQ; i++) {
+        cmdqv->cmdq_alloc_map[i] = 0;
+        cmdqv->vcmdq_cons_indx[i] = 0;
+        cmdqv->vcmdq_prod_indx[i] = 0;
+        cmdqv->vcmdq_config[i] = 0;
+        cmdqv->vcmdq_status[i] = 0;
+        cmdqv->vcmdq_gerror[i] = 0;
+        cmdqv->vcmdq_gerrorn[i] = 0;
+        cmdqv->vcmdq_base[i] = 0;
+        cmdqv->vcmdq_cons_indx_base[i] = 0;
+    }
+}
+
 static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+
+    if (!cmdqv) {
+        return;
+    }
+
+    tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
+    tegra241_cmdqv_munmap_vintf_page0(cmdqv, NULL);
+    tegra241_cmdqv_free_all_vcmdq(cmdqv);
+
+    tegra241_cmdqv_init_regs(s, cmdqv);
 }
 
 static const MemoryRegionOps mmio_cmdqv_ops = {
hw/arm/trace-events +0 -0
index fd6441bfa7..6f602b9eda 100644

hw/arm/trace-events smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS · line 82

Quoted 7 lines
hw/arm/trace-events +1 -0
@@ -76,6 +76,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"

Thanks

Eric

#81 lore raw

RE: [PATCH v4 25/31] hw/arm/tegra241-cmdqv: Add reset handler

Quoted 64 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 17:52
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 25/31] hw/arm/tegra241-cmdqv: Add reset handler

External email: Use caution opening links or attachments

Quoted 10 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/tegra241-cmdqv.c | 50

+++++++++++++++++++++++++++++++++++++++++

Quoted 29 lines
 hw/arm/trace-events     |  1 +
 3 files changed, 53 insertions(+)
hw/arm/tegra241-cmdqv.h +2 -0
index 2befa6205e..b2a444daef 100644
@@ -79,6 +79,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/tegra241-cmdqv.c +1 -0
index 9c2fc02b92..af68add2f0 100644
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"

hw/arm/tegra241-cmdqv.c free_viommu: · line 649

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -645,8 +646,57 @@ free_viommu:
     return false;
 }
 
+static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV

*cmdqv)

Quoted 7 lines

hw/arm/tegra241-cmdqv.c free_viommu: · line 654

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+{
+    int i;
+
+    cmdqv->config = V_CONFIG_RESET;
+    cmdqv->param = FIELD_DP32(0, PARAM, CMDQV_VER, CMDQV_VER);

hw/core/registerfields.h:#define FIELD_DP32(storage, reg, field, val) what does this 0?

I think it is to make sure we have no garbage set for non-field bits. Starting from 0 guarantees non-field bits are clean.

Quoted 16 lines

hw/arm/tegra241-cmdqv.c free_viommu: · line 655

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM,

CMDQV_NUM_CMDQ_LOG2,

hw/arm/tegra241-cmdqv.c free_viommu: · line 656

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+                              CMDQV_NUM_CMDQ_LOG2);

I was a bit puzzled by the FIELD name being the value name but well.

hw/arm/tegra241-cmdqv.c free_viommu: · line 657

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM,

CMDQV_NUM_SID_PER_VI_LOG2,

hw/arm/tegra241-cmdqv.c free_viommu: · line 667

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+                              CMDQV_NUM_SID_PER_VI_LOG2);
+    trace_tegra241_cmdqv_init_regs(cmdqv->param);
+    cmdqv->status = R_STATUS_CMDQV_ENABLED_MASK;
+    for (i = 0; i < 2; i++) {
+        cmdqv->vi_err_map[i] = 0;
+        cmdqv->vi_int_mask[i] = 0;
+        cmdqv->cmdq_err_map[i] = 0;
+    }
+    cmdqv->cmdq_err_map[2] = 0;
+    cmdqv->cmdq_err_map[3] = 0;
Quoted 4 lines

the split looks pretty strange, ie. loop of 2 + continue with individual setting. why don't you do the init of

cmdqv->cmdq_err_map in the x4 loop below

😊. True, that does look strange. I will change.

Thanks,
Shameer

#82 lore raw

Re: [PATCH v4 25/31] hw/arm/tegra241-cmdqv: Add reset handler

Quoted 70 lines · Shameer Kolothum Thodi
Quoted 63 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 17:52
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 25/31] hw/arm/tegra241-cmdqv: Add reset handler

External email: Use caution opening links or attachments

Quoted 10 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/tegra241-cmdqv.c | 50

+++++++++++++++++++++++++++++++++++++++++

Quoted 29 lines
 hw/arm/trace-events     |  1 +
 3 files changed, 53 insertions(+)
hw/arm/tegra241-cmdqv.h +2 -0
index 2befa6205e..b2a444daef 100644
@@ -79,6 +79,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/tegra241-cmdqv.c +1 -0
index 9c2fc02b92..af68add2f0 100644
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"

hw/arm/tegra241-cmdqv.c free_viommu: · line 649

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -645,8 +646,57 @@ free_viommu:
     return false;
 }
 
+static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV

*cmdqv)

Quoted 7 lines

hw/arm/tegra241-cmdqv.c free_viommu: · line 654

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+{
+    int i;
+
+    cmdqv->config = V_CONFIG_RESET;
+    cmdqv->param = FIELD_DP32(0, PARAM, CMDQV_VER, CMDQV_VER);

hw/core/registerfields.h:#define FIELD_DP32(storage, reg, field, val) what does this 0?

Quoted 2 lines

I think it is to make sure we have no garbage set for non-field bits. Starting from 0 guarantees non-field bits are clean.

OK thanks ;-)

Eric

Quoted 25 lines
Quoted 17 lines

hw/arm/tegra241-cmdqv.c free_viommu: · line 655

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +1 -0
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM,

CMDQV_NUM_CMDQ_LOG2,

hw/arm/tegra241-cmdqv.c free_viommu: · line 656

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+                              CMDQV_NUM_CMDQ_LOG2);

I was a bit puzzled by the FIELD name being the value name but well.

hw/arm/tegra241-cmdqv.c free_viommu: · line 657

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM,

CMDQV_NUM_SID_PER_VI_LOG2,

hw/arm/tegra241-cmdqv.c free_viommu: · line 667

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+                              CMDQV_NUM_SID_PER_VI_LOG2);
+    trace_tegra241_cmdqv_init_regs(cmdqv->param);
+    cmdqv->status = R_STATUS_CMDQV_ENABLED_MASK;
+    for (i = 0; i < 2; i++) {
+        cmdqv->vi_err_map[i] = 0;
+        cmdqv->vi_int_mask[i] = 0;
+        cmdqv->cmdq_err_map[i] = 0;
+    }
+    cmdqv->cmdq_err_map[2] = 0;
+    cmdqv->cmdq_err_map[3] = 0;
Quoted 4 lines

the split looks pretty strange, ie. loop of 2 + continue with individual setting. why don't you do the init of

cmdqv->cmdq_err_map in the x4 loop below

😊. True, that does look strange. I will change.

Thanks,
Shameer

#83 lore raw

Re: [PATCH v4 25/31] hw/arm/tegra241-cmdqv: Add reset handler

In reply to Shameer Kolothum
Quoted 18 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/tegra241-cmdqv.c | 50 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 3 files changed, 53 insertions(+)
hw/arm/tegra241-cmdqv.h +0 -0
index 2befa6205e..b2a444daef 100644

hw/arm/tegra241-cmdqv.h FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) · line 82

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +1 -0
@@ -79,6 +79,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403

please add a comment to explain what

0x00020403 value means

Eric

Quoted 86 lines
hw/arm/tegra241-cmdqv.h +1 -0
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/tegra241-cmdqv.c +50 -0
index 9c2fc02b92..af68add2f0 100644
@@ -8,6 +8,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
 #include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
@@ -645,8 +646,57 @@ free_viommu:
     return false;
 }
 
+static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
+{
+    int i;
+
+    cmdqv->config = V_CONFIG_RESET;
+    cmdqv->param = FIELD_DP32(0, PARAM, CMDQV_VER, CMDQV_VER);
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM, CMDQV_NUM_CMDQ_LOG2,
+                              CMDQV_NUM_CMDQ_LOG2);
+    cmdqv->param = FIELD_DP32(cmdqv->param, PARAM, CMDQV_NUM_SID_PER_VI_LOG2,
+                              CMDQV_NUM_SID_PER_VI_LOG2);
+    trace_tegra241_cmdqv_init_regs(cmdqv->param);
+    cmdqv->status = R_STATUS_CMDQV_ENABLED_MASK;
+    for (i = 0; i < 2; i++) {
+        cmdqv->vi_err_map[i] = 0;
+        cmdqv->vi_int_mask[i] = 0;
+        cmdqv->cmdq_err_map[i] = 0;
+    }
+    cmdqv->cmdq_err_map[2] = 0;
+    cmdqv->cmdq_err_map[3] = 0;
+    cmdqv->vintf_config = 0;
+    cmdqv->vintf_status = 0;
+    for (i = 0; i < 4; i++) {
+        cmdqv->vintf_cmdq_err_map[i] = 0;
+    }
+    for (i = 0; i < TEGRA241_CMDQV_MAX_CMDQ; i++) {
+        cmdqv->cmdq_alloc_map[i] = 0;
+        cmdqv->vcmdq_cons_indx[i] = 0;
+        cmdqv->vcmdq_prod_indx[i] = 0;
+        cmdqv->vcmdq_config[i] = 0;
+        cmdqv->vcmdq_status[i] = 0;
+        cmdqv->vcmdq_gerror[i] = 0;
+        cmdqv->vcmdq_gerrorn[i] = 0;
+        cmdqv->vcmdq_base[i] = 0;
+        cmdqv->vcmdq_cons_indx_base[i] = 0;
+    }
+}
+
 static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv = accel->cmdqv;
+
+    if (!cmdqv) {
+        return;
+    }
+
+    tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
+    tegra241_cmdqv_munmap_vintf_page0(cmdqv, NULL);
+    tegra241_cmdqv_free_all_vcmdq(cmdqv);
+
+    tegra241_cmdqv_init_regs(s, cmdqv);
 }
 
 static const MemoryRegionOps mmio_cmdqv_ops = {
hw/arm/trace-events +1 -0
index fd6441bfa7..6f602b9eda 100644
@@ -76,6 +76,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 tegra241_cmdqv_read_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_write_mmio(uint64_t offset, uint64_t val, unsigned size) "offset: 0x%"PRIx64" val: 0x%"PRIx64" size: 0x%x"
 tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
#84 lore raw

PATCH v4 26/31 hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

CMDQV HW reads guest queue memory in its host physical address setup via IOMMUFD. This requires the guest queue memory is not only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQS that the host can safely back with physically contiguous memory. Allowing a queue larger than the host page size could cause the hardware to DMA across page boundaries, leading to faults.

Walk the RAMBlock list to find the smallest memory-backend page size, then limit IDR1.CMDQS so the guest cannot configure a command queue that exceeds that contiguous backing. Fall back to the real host page size if no memory-backend RAM blocks are found.

---
 hw/arm/tegra241-cmdqv.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +41 -0
index af68add2f0..2870886783 100644
@@ -14,6 +14,9 @@
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
@@ -646,9 +649,38 @@ free_viommu:
     return false;
 }
 
+static size_t tegra241_cmdqv_min_ram_pagesize(void)
+{
+    RAMBlock *rb;
+    size_t pg, min_pg = SIZE_MAX;
+
+    RAMBLOCK_FOREACH(rb) {
+        MemoryRegion *mr = rb->mr;
+
+        /* Only consider real RAM regions */
+        if (!mr || !memory_region_is_ram(mr)) {
+            continue;
+        }
+
+        /* Skip RAM regions that are not backed by a memory-backend */
+        if (!object_dynamic_cast(mr->owner, TYPE_MEMORY_BACKEND)) {
+            continue;
+        }
+
+        pg = qemu_ram_pagesize(rb);
+        if (pg && pg < min_pg) {
+            min_pg = pg;
+        }
+    }
+
+    return (min_pg == SIZE_MAX) ? qemu_real_host_page_size() : min_pg;
+}
+
 static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
 {
     int i;
+    size_t pgsize;
+    uint32_t val;
 
     cmdqv->config = V_CONFIG_RESET;
     cmdqv->param = FIELD_DP32(0, PARAM, CMDQV_VER, CMDQV_VER);
@@ -681,6 +713,15 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
         cmdqv->vcmdq_base[i] = 0;
         cmdqv->vcmdq_cons_indx_base[i] = 0;
     }
+
+    /*
+     * CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the
+     * queue fits entirely within the smallest backend page size, ensuring
+     * the command queue is physically contiguous in host memory.
+     */
+    pgsize = tegra241_cmdqv_min_ram_pagesize();
+    val = FIELD_EX32(s->idr[1], IDR1, CMDQS);
+    s->idr[1] = FIELD_DP32(s->idr[1], IDR1, CMDQS, MIN(ctz64(pgsize) - 4, val));
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
Signature · 1 line
2.43.0
#85 lore raw

Re: [PATCH v4 26/31] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv) · line 724

Quoted 8 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +8 -0
+    /*
+     * CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the
+     * queue fits entirely within the smallest backend page size, ensuring
+     * the command queue is physically contiguous in host memory.
+     */
+    pgsize = tegra241_cmdqv_min_ram_pagesize();
+    val = FIELD_EX32(s->idr[1], IDR1, CMDQS);
+    s->idr[1] = FIELD_DP32(s->idr[1], IDR1, CMDQS, MIN(ctz64(pgsize) - 4, val));

Might be helpful to add in the note:
idr1.cmdqs = log2(max_qsz) - entry shift

Otherwise, lgtm.

Nicolin

#86 lore raw

Re: [PATCH v4 26/31] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

In reply to Shameer Kolothum

Hi Shameer,

Quoted 6 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

CMDQV HW reads guest queue memory in its host physical address setup via IOMMUFD. This requires the guest queue memory is not only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQS that the host can safely back with physically

s/a CMDQS/ a command queue size (CMDQS)

Quoted 1 line

contiguous memory. Allowing a queue larger than the host page size could

a queue size

Quoted 4 lines

cause the hardware to DMA across page boundaries, leading to faults.

Walk the RAMBlock list to find the smallest memory-backend page size, then limit IDR1.CMDQS so the guest cannot configure a command queue that exceeds

what is the minimal CMDQS required?

Quoted 24 lines

that contiguous backing. Fall back to the real host page size if no memory-backend RAM blocks are found.

---
 hw/arm/tegra241-cmdqv.c | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +3 -0
index af68add2f0..2870886783 100644
@@ -14,6 +14,9 @@
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 

hw/arm/tegra241-cmdqv.c free_viommu: · line 652

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -646,9 +649,38 @@ free_viommu:
     return false;
 }
 
+static size_t tegra241_cmdqv_min_ram_pagesize(void)

shouldn't we put that rather in system/physmem.c there we also have qemu_ram_pagesize_largest() for instance

Quoted 40 lines
hw/arm/tegra241-cmdqv.c +30 -0
+{
+    RAMBlock *rb;
+    size_t pg, min_pg = SIZE_MAX;
+
+    RAMBLOCK_FOREACH(rb) {
+        MemoryRegion *mr = rb->mr;
+
+        /* Only consider real RAM regions */
+        if (!mr || !memory_region_is_ram(mr)) {
+            continue;
+        }
+
+        /* Skip RAM regions that are not backed by a memory-backend */
+        if (!object_dynamic_cast(mr->owner, TYPE_MEMORY_BACKEND)) {
+            continue;
+        }
+
+        pg = qemu_ram_pagesize(rb);
+        if (pg && pg < min_pg) {
+            min_pg = pg;
+        }
+    }
+
+    return (min_pg == SIZE_MAX) ? qemu_real_host_page_size() : min_pg;
+}
+
 static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
 {
     int i;
+    size_t pgsize;
+    uint32_t val;
 
     cmdqv->config = V_CONFIG_RESET;
     cmdqv->param = FIELD_DP32(0, PARAM, CMDQV_VER, CMDQV_VER);
@@ -681,6 +713,15 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
         cmdqv->vcmdq_base[i] = 0;
         cmdqv->vcmdq_cons_indx_base[i] = 0;
     }
+
+    /*

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv) · line 727

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+     * CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the
+     * queue fits entirely within the smallest backend page size, ensuring
+     * the command queue is physically contiguous in host memory.
+     */
+    pgsize = tegra241_cmdqv_min_ram_pagesize();
+    val = FIELD_EX32(s->idr[1], IDR1, CMDQS);
+    s->idr[1] = FIELD_DP32(s->idr[1], IDR1, CMDQS, MIN(ctz64(pgsize) - 4, val));
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)

Eric

#87 lore raw

RE: [PATCH v4 26/31] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

Quoted 30 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 18:24
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 26/31] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 5 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

CMDQV HW reads guest queue memory in its host physical address setup via IOMMUFD. This requires the guest queue memory is not only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled,

we

Quoted 1 line

must only advertise a CMDQS that the host can safely back with physically

s/a CMDQS/ a command queue size (CMDQS)

Quoted 8 lines
Quoted 1 line

contiguous memory. Allowing a queue larger than the host page size could

a queue size

Quoted 4 lines

cause the hardware to DMA across page boundaries, leading to faults.

Walk the RAMBlock list to find the smallest memory-backend page size, then limit IDR1.CMDQS so the guest cannot configure a command queue that

exceeds
what is the minimal CMDQS required?

AFAICS, spec doesn't specify a minimum. But in practical terms I think, it will be 4K page size(CMDQS=8).

Quoted 24 lines
Quoted 7 lines

that contiguous backing. Fall back to the real host page size if no memory-backend RAM blocks are found.

---
 hw/arm/tegra241-cmdqv.c | 41

+++++++++++++++++++++++++++++++++++++++++

Quoted 16 lines
 1 file changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +3 -0
index af68add2f0..2870886783 100644
@@ -14,6 +14,9 @@
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
Quoted 7 lines

hw/arm/tegra241-cmdqv.c free_viommu: · line 652

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +1 -0
@@ -646,9 +649,38 @@ free_viommu:
     return false;
 }
 
+static size_t tegra241_cmdqv_min_ram_pagesize(void)

shouldn't we put that rather in system/physmem.c there we also have qemu_ram_pagesize_largest() for instance

Could do. But we may have to rename it differently compared to qemu_ram_pagesize_largest() as we are checking the size among memory-backend RAM only.

May be qemu_ram_backend_pagesize_min() ?

Thanks,
Shameer

#88 lore raw

Re: [PATCH v4 26/31] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

Quoted 31 lines · Shameer Kolothum Thodi
Quoted 31 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 18:24
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 26/31] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 5 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

CMDQV HW reads guest queue memory in its host physical address setup via IOMMUFD. This requires the guest queue memory is not only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled,

we

Quoted 1 line

must only advertise a CMDQS that the host can safely back with physically

s/a CMDQS/ a command queue size (CMDQS)

Quoted 1 line

contiguous memory. Allowing a queue larger than the host page size could

a queue size

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

cause the hardware to DMA across page boundaries, leading to faults.

Walk the RAMBlock list to find the smallest memory-backend page size, then limit IDR1.CMDQS so the guest cannot configure a command queue that

exceeds
what is the minimal CMDQS required?

AFAICS, spec doesn't specify a minimum. But in practical terms I think, it will be 4K page size(CMDQS=8).

OK

Quoted 29 lines
Quoted 29 lines
Quoted 8 lines

that contiguous backing. Fall back to the real host page size if no memory-backend RAM blocks are found.

---
 hw/arm/tegra241-cmdqv.c | 41

+++++++++++++++++++++++++++++++++++++++++

Quoted 20 lines
 1 file changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +3 -0
index af68add2f0..2870886783 100644
@@ -14,6 +14,9 @@
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 #include "trace.h"
 
@@ -646,9 +649,38 @@ free_viommu:
     return false;
 }
 
Quoted 8 lines
Quoted 3 lines

hw/arm/tegra241-cmdqv.c free_viommu: · line 652

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+static size_t tegra241_cmdqv_min_ram_pagesize(void)

shouldn't we put that rather in system/physmem.c there we also have qemu_ram_pagesize_largest() for instance

Could do. But we may have to rename it differently compared to qemu_ram_pagesize_largest() as we are checking the size among memory-backend RAM only.

May be qemu_ram_backend_pagesize_min() ?

Yes sounds reasonable to me. Let's add the physmem.c maintainers in cc.

Eric

Quoted 3 lines

Thanks,
Shameer

#89 lore raw

PATCH v4 27/31 hw/arm/smmuv3: Add per-device identifier property

In reply to Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.

The identifier is assigned at pre-plug time, accounting for the ITS Group node that build_iort() places before SMMUv3 nodes in the IORT table, so that identifiers are globally unique across all IORT nodes.

No functional change: IORT blob content for bios-tables qtest is identical to before.

---
 include/hw/arm/smmuv3.h  |  1 +
 hw/arm/smmuv3.c          |  2 ++
 hw/arm/virt-acpi-build.c |  5 ++++-
 hw/arm/virt.c            | 12 ++++++++++++
 4 files changed, 19 insertions(+), 1 deletion(-)
include/hw/arm/smmuv3.h +1 -0
index aa6a79237a..0fce564619 100644
@@ -64,6 +64,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
hw/arm/smmuv3.c +2 -0
index 682d89c3ea..1d6fdd776c 100644
@@ -2144,6 +2144,8 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +4 -1
index 521443de87..65ccc96349 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiIortSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
     GArray *rc_smmu_idmaps;
     /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
     size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(VirtMachineState *vms, GArray *sdev_blob)
                                                &error_abort));
         sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
         sdev.ats = smmuv3_ats_enabled(ARM_SMMUV3(obj));
+        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
         pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
         sbdev = SYS_BUS_DEVICE(obj);
         sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -630,7 +632,8 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
+        id++;  /* advance shared counter for RC/RMR node uniqueness */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +12 -0
index 6c5e51af37..22d6b9eec9 100644
@@ -240,6 +240,9 @@ static MemMapEntry extended_memmap[] = {
     /* Any CXL Fixed memory windows come here */
 };
 
+/* Counts SMMUv3 devices plugged; used to assign stable IORT identifiers */
+static uint8_t smmuv3_dev_id;
+
 static const int a15irqmap[] = {
     [VIRT_UART0] = 1,
     [VIRT_RTC] = 2,
@@ -3226,6 +3229,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            /*
+             * In build_iort(), the ITS node(id=0) precedes SMMUv3 nodes
+             * when present. Account for it so this SMMUv3's identifier
+             * is globally unique across all IORT nodes.
+             */
+            uint8_t its_offset = (vms->msi_controller == VIRT_MSI_CTRL_ITS)
+                                  ? 1 : 0;
+            object_property_set_uint(OBJECT(dev), "identifier",
+                                     its_offset + smmuv3_dev_id++, NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
Signature · 1 line
2.43.0
Reviewed-by on v4 27/31
#90 lore raw

Re: [PATCH v4 27/31] hw/arm/smmuv3: Add per-device identifier property

Quoted 12 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.

The identifier is assigned at pre-plug time, accounting for the ITS Group node that build_iort() places before SMMUv3 nodes in the IORT table, so that identifiers are globally unique across all IORT nodes.

Quoted 5 lines

No functional change: IORT blob content for bios-tables qtest is identical to before.

#91 lore raw

PATCH v4 28/31 hw/arm/smmuv3-accel: Introduce helper to query CMDQV type

In reply to Shameer Kolothum

Introduce a SMMUv3AccelCmdqvType enum and a helper to query the CMDQV implementation type associated with an accelerated SMMUv3 instance.

A subsequent patch will use this helper when generating the Tegra241 CMDQV DSDT.

---
 hw/arm/smmuv3-accel.h       |  7 +++++++
 hw/arm/smmuv3-accel-stubs.c |  5 +++++
 hw/arm/smmuv3-accel.c       | 12 ++++++++++++
 hw/arm/tegra241-cmdqv.c     |  6 ++++++
 4 files changed, 30 insertions(+)
hw/arm/smmuv3-accel.h +7 -0
index 448f47c0ca..3ed94ed05c 100644
@@ -16,6 +16,11 @@
 #include <linux/iommufd.h>
 #endif
 
+typedef enum SMMUv3AccelCmdqvType {
+    SMMUV3_CMDQV_NONE = 0,
+    SMMUV3_CMDQV_TEGRA241,
+} SMMUv3AccelCmdqvType;
+
 /*
  * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
  * support multiple VCMDQs with virtualization capabilities.
@@ -29,6 +34,7 @@ typedef struct SMMUv3AccelCmdqvOps {
                          uint32_t *out_viommu_id,
                          Error **errp);
     void (*free_viommu)(SMMUv3State *s);
+    SMMUv3AccelCmdqvType (*get_type)(void);
     void (*reset)(SMMUv3State *s);
 } SMMUv3AccelCmdqvOps;
 
@@ -74,5 +80,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
                                       void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +5 -0
index e8f08dc833..08de01d909 100644
@@ -55,3 +55,8 @@ bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
 void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    return SMMUV3_CMDQV_NONE;
+}
hw/arm/smmuv3-accel.c +12 -0
index 230f608f03..a58815ded2 100644
@@ -1049,6 +1049,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
 }
 
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    SMMUv3State *s = ARM_SMMUV3(obj);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) {
+        return SMMUV3_CMDQV_NONE;
+    }
+
+    return accel->cmdqv_ops->get_type();
+}
+
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp)
 {
     SMMUState *bs = ARM_SMMU(s);
hw/arm/tegra241-cmdqv.c +6 -0
index 2870886783..71f89abcb4 100644
@@ -762,6 +762,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type(void)
+{
+    return SMMUV3_CMDQV_TEGRA241;
+}
+
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
@@ -802,6 +807,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
     .init = tegra241_cmdqv_init,
     .alloc_viommu = tegra241_cmdqv_alloc_viommu,
     .free_viommu = tegra241_cmdqv_free_viommu,
+    .get_type = tegra241_cmdqv_get_type,
     .reset = tegra241_cmdqv_reset,
 };
 
Signature · 1 line
2.43.0
Reviewed-by on v4 28/31
#92 lore raw

Re: [PATCH v4 28/31] hw/arm/smmuv3-accel: Introduce helper to query CMDQV type

Quoted 9 lines · Shameer Kolothum

Introduce a SMMUv3AccelCmdqvType enum and a helper to query the CMDQV implementation type associated with an accelerated SMMUv3 instance.

A subsequent patch will use this helper when generating the Tegra241 CMDQV DSDT.

#93 lore raw

PATCH v4 29/31 hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.

The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.

---
 hw/arm/virt-acpi-build.c | 52 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events      |  1 +
 2 files changed, 53 insertions(+)
hw/arm/virt-acpi-build.c +52 -0
index 65ccc96349..fbc793d06e 100644
@@ -65,6 +65,9 @@
 #include "target/arm/cpu.h"
 #include "target/arm/multiprocessing.h"
 
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
 #define ARM_SPI_BASE 32
 
 #define ACPI_BUILD_TABLE_SIZE             0x20000
@@ -1114,6 +1117,51 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,
     build_fadt(table_data, linker, &fadt, vms->oem_id, vms->oem_table_id);
 }
 
+static void acpi_dsdt_add_tegra241_cmdqv(Aml *scope, VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        PlatformBusDevice *pbus;
+        Aml *dev, *crs, *addr;
+        SysBusDevice *sbdev;
+        hwaddr base;
+        uint32_t id;
+        int irq;
+
+        if (smmuv3_accel_cmdqv_type(obj) != SMMUV3_CMDQV_TEGRA241) {
+            continue;
+        }
+        id = object_property_get_uint(obj, "identifier", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        base = platform_bus_get_mmio_addr(pbus, sbdev, 1);
+        base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        irq = platform_bus_get_irqn(pbus, sbdev, NUM_SMMU_IRQS);
+        irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        irq += ARM_SPI_BASE;
+
+        dev = aml_device("CV%.02u", id);
+        aml_append(dev, aml_name_decl("_HID", aml_string("NVDA200C")));
+        aml_append(dev, aml_name_decl("_UID", aml_int(id)));
+        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
+
+        crs = aml_resource_template();
+        addr = aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED, AML_MAX_FIXED,
+                                AML_CACHEABLE, AML_READ_WRITE, 0x0, base,
+                                base + TEGRA241_CMDQV_IO_LEN - 0x1, 0x0,
+                                TEGRA241_CMDQV_IO_LEN);
+        aml_append(crs, addr);
+        aml_append(crs, aml_interrupt(AML_CONSUMER, AML_EDGE,
+                                      AML_ACTIVE_HIGH, AML_EXCLUSIVE,
+                                      (uint32_t *)&irq, 1));
+        aml_append(dev, aml_name_decl("_CRS", crs));
+
+        aml_append(scope, dev);
+
+        trace_virt_acpi_dsdt_tegra241_cmdqv(id, base, irq);
+    }
+}
+
 /* DSDT */
 static void
 build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
@@ -1178,6 +1226,10 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_dsdt_add_tpm(scope, vms);
 #endif
 
+    if (!vms->legacy_smmuv3_present) {
+        acpi_dsdt_add_tegra241_cmdqv(scope, vms);
+    }
+
     aml_append(dsdt, scope);
 
     pci0_scope = aml_scope("\\_SB.PCI0");
hw/arm/trace-events +1 -0
index 6f602b9eda..e5e4e93324 100644
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s"
 
 # virt-acpi-build.c
 virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out."
+virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d"
 
 # smmu-common.c
 smmu_add_mr(const char *name) "%s"
Signature · 1 line
2.43.0
#94 lore raw

Re: [PATCH v4 29/31] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Quoted 8 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.

The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.

Could you add a link to the ACPI spec to check the content of what is generated.

maybe you can also describe the aml in the coommit msg

Thanks

Eric

Quoted 98 lines
---
 hw/arm/virt-acpi-build.c | 52 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events      |  1 +
 2 files changed, 53 insertions(+)
hw/arm/virt-acpi-build.c +52 -0
index 65ccc96349..fbc793d06e 100644
@@ -65,6 +65,9 @@
 #include "target/arm/cpu.h"
 #include "target/arm/multiprocessing.h"
 
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
 #define ARM_SPI_BASE 32
 
 #define ACPI_BUILD_TABLE_SIZE             0x20000
@@ -1114,6 +1117,51 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,
     build_fadt(table_data, linker, &fadt, vms->oem_id, vms->oem_table_id);
 }
 
+static void acpi_dsdt_add_tegra241_cmdqv(Aml *scope, VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        PlatformBusDevice *pbus;
+        Aml *dev, *crs, *addr;
+        SysBusDevice *sbdev;
+        hwaddr base;
+        uint32_t id;
+        int irq;
+
+        if (smmuv3_accel_cmdqv_type(obj) != SMMUV3_CMDQV_TEGRA241) {
+            continue;
+        }
+        id = object_property_get_uint(obj, "identifier", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        base = platform_bus_get_mmio_addr(pbus, sbdev, 1);
+        base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        irq = platform_bus_get_irqn(pbus, sbdev, NUM_SMMU_IRQS);
+        irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        irq += ARM_SPI_BASE;
+
+        dev = aml_device("CV%.02u", id);
+        aml_append(dev, aml_name_decl("_HID", aml_string("NVDA200C")));
+        aml_append(dev, aml_name_decl("_UID", aml_int(id)));
+        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
+
+        crs = aml_resource_template();
+        addr = aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED, AML_MAX_FIXED,
+                                AML_CACHEABLE, AML_READ_WRITE, 0x0, base,
+                                base + TEGRA241_CMDQV_IO_LEN - 0x1, 0x0,
+                                TEGRA241_CMDQV_IO_LEN);
+        aml_append(crs, addr);
+        aml_append(crs, aml_interrupt(AML_CONSUMER, AML_EDGE,
+                                      AML_ACTIVE_HIGH, AML_EXCLUSIVE,
+                                      (uint32_t *)&irq, 1));
+        aml_append(dev, aml_name_decl("_CRS", crs));
+
+        aml_append(scope, dev);
+
+        trace_virt_acpi_dsdt_tegra241_cmdqv(id, base, irq);
+    }
+}
+
 /* DSDT */
 static void
 build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
@@ -1178,6 +1226,10 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_dsdt_add_tpm(scope, vms);
 #endif
 
+    if (!vms->legacy_smmuv3_present) {
+        acpi_dsdt_add_tegra241_cmdqv(scope, vms);
+    }
+
     aml_append(dsdt, scope);
 
     pci0_scope = aml_scope("\\_SB.PCI0");
hw/arm/trace-events +1 -0
index 6f602b9eda..e5e4e93324 100644
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s"
 
 # virt-acpi-build.c
 virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out."
+virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d"
 
 # smmu-common.c
 smmu_add_mr(const char *name) "%s"
#95 lore raw

PATCH v4 30/31 hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active

In reply to Shameer Kolothum

When CMDQV is active, the first cold-plugged VFIO device establishes the viommu to host SMMUv3 association. Block its hot-unplug to preserve this association and the guest's boot time CMDQV configuration.

Also abort at machine_done if cmdqv=on is requested but no cold-plugged VFIO device was present to initialize it.

---
 hw/arm/smmuv3-accel.h |  1 +
 hw/arm/smmuv3-accel.c | 12 ++++++++++++
 hw/arm/smmuv3.c       |  6 ++++++
 3 files changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +1 -0
index 3ed94ed05c..c4441d5b3f 100644
@@ -65,6 +65,7 @@ typedef struct SMMUv3AccelDevice {
     IOMMUFDVdev *vdev;
     QLIST_ENTRY(SMMUv3AccelDevice) next;
     SMMUv3AccelState *s_accel;
+    Error *unplug_blocker; /* set when CMDQV is active to block hot-unplug */
 } SMMUv3AccelDevice;
 
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp);
hw/arm/smmuv3-accel.c +12 -0
index a58815ded2..f381702a08 100644
@@ -754,6 +754,18 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
         return false;
     }
 
+    /*
+     * CMDQV is active: block hot-unplug of the device that established the
+     * viommu association. Removing it would cause the vIOMMU to host SMMUv3
+     * association be changed via device hot-plug.
+     */
+    if (s->s_accel->cmdqv_ops) {
+        PCIDevice *pdev = pci_find_device(bus, pci_bus_num(bus), devfn);
+        error_setg(&accel_dev->unplug_blocker,
+                   "CMDQV is active: removing the device that established the "
+                   "viommu association would break the guest CMDQV");
+        qdev_add_unplug_blocker(DEVICE(pdev), accel_dev->unplug_blocker);
+    }
 done:
     accel_dev->idev = idev;
     accel_dev->s_accel = s->s_accel;
hw/arm/smmuv3.c +6 -0
index 1d6fdd776c..c9ff6298f5 100644
@@ -2020,6 +2020,12 @@ static void smmuv3_machine_done(Notifier *notifier, void *data)
                      "at least one cold-plugged VFIO device");
         exit(1);
     }
+
+    if (s->cmdqv == ON_OFF_AUTO_ON && !accel->cmdqv) {
+        error_report("arm-smmuv3 cmdqv=on requires at least one cold-plugged "
+                     "VFIO device");
+        exit(1);
+    }
 }
 
 static void smmu_realize(DeviceState *d, Error **errp)
Signature · 1 line
2.43.0
Reviewed-by on v4 30/31
#96 lore raw

Re: [PATCH v4 30/31] hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active

Quoted 8 lines · Shameer Kolothum

When CMDQV is active, the first cold-plugged VFIO device establishes the viommu to host SMMUv3 association. Block its hot-unplug to preserve this association and the guest's boot time CMDQV configuration.

Also abort at machine_done if cmdqv=on is requested but no cold-plugged VFIO device was present to initialize it.

#97 lore raw

Re: [PATCH v4 30/31] hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active

In reply to Shameer Kolothum
Quoted 34 lines · Shameer Kolothum

When CMDQV is active, the first cold-plugged VFIO device establishes the viommu to host SMMUv3 association. Block its hot-unplug to preserve this association and the guest's boot time CMDQV configuration.

Also abort at machine_done if cmdqv=on is requested but no cold-plugged VFIO device was present to initialize it.

---
 hw/arm/smmuv3-accel.h |  1 +
 hw/arm/smmuv3-accel.c | 12 ++++++++++++
 hw/arm/smmuv3.c       |  6 ++++++
 3 files changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +1 -0
index 3ed94ed05c..c4441d5b3f 100644
@@ -65,6 +65,7 @@ typedef struct SMMUv3AccelDevice {
     IOMMUFDVdev *vdev;
     QLIST_ENTRY(SMMUv3AccelDevice) next;
     SMMUv3AccelState *s_accel;
+    Error *unplug_blocker; /* set when CMDQV is active to block hot-unplug */
 } SMMUv3AccelDevice;
 
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp);
hw/arm/smmuv3-accel.c +0 -0
index a58815ded2..f381702a08 100644
@@ -754,6 +754,18 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
         return false;
     }
 

hw/arm/smmuv3-accel.c static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn, · line 766

Quoted 10 lines
hw/arm/smmuv3-accel.c +10 -0
+    /*
+     * CMDQV is active: block hot-unplug of the device that established the
+     * viommu association. Removing it would cause the vIOMMU to host SMMUv3
+     * association be changed via device hot-plug.
+     */
+    if (s->s_accel->cmdqv_ops) {
+        PCIDevice *pdev = pci_find_device(bus, pci_bus_num(bus), devfn);
+        error_setg(&accel_dev->unplug_blocker,
+                   "CMDQV is active: removing the device that established the "
+                   "viommu association would break the guest CMDQV");

It is something that is planned to be fixed later on?

can you provide more details about the nature of the breakage

Thanks

Eric

Quoted 22 lines
hw/arm/smmuv3-accel.c +2 -0
+        qdev_add_unplug_blocker(DEVICE(pdev), accel_dev->unplug_blocker);
+    }
 done:
     accel_dev->idev = idev;
     accel_dev->s_accel = s->s_accel;
hw/arm/smmuv3.c +6 -0
index 1d6fdd776c..c9ff6298f5 100644
@@ -2020,6 +2020,12 @@ static void smmuv3_machine_done(Notifier *notifier, void *data)
                      "at least one cold-plugged VFIO device");
         exit(1);
     }
+
+    if (s->cmdqv == ON_OFF_AUTO_ON && !accel->cmdqv) {
+        error_report("arm-smmuv3 cmdqv=on requires at least one cold-plugged "
+                     "VFIO device");
+        exit(1);
+    }
 }
 
 static void smmu_realize(DeviceState *d, Error **errp)
#98 lore raw

RE: [PATCH v4 30/31] hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active

Quoted 60 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 07 May 2026 18:36
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v4 30/31] hw/arm/smmuv3-accel: Enforce viommu association when CMDQV is active

External email: Use caution opening links or attachments

hw/arm/smmuv3-accel.c static bool

Quoted 32 lines · Shameer Kolothum

When CMDQV is active, the first cold-plugged VFIO device establishes the viommu to host SMMUv3 association. Block its hot-unplug to preserve this association and the guest's boot time CMDQV configuration.

Also abort at machine_done if cmdqv=on is requested but no cold-plugged VFIO device was present to initialize it.

---
 hw/arm/smmuv3-accel.h |  1 +
 hw/arm/smmuv3-accel.c | 12 ++++++++++++
 hw/arm/smmuv3.c       |  6 ++++++
 3 files changed, 19 insertions(+)
hw/arm/smmuv3-accel.h index renamed from hw/arm/smmuv3-accel.h +0 -0

3ed94ed05c..c4441d5b3f 100644

hw/arm/smmuv3-accel.h +2 -0
@@ -65,6 +65,7 @@ typedef struct SMMUv3AccelDevice {
     IOMMUFDVdev *vdev;
     QLIST_ENTRY(SMMUv3AccelDevice) next;
     SMMUv3AccelState *s_accel;
+    Error *unplug_blocker; /* set when CMDQV is active to block
+ hot-unplug */
 } SMMUv3AccelDevice;
 
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp); diff --git

a/hw/arm/smmuv3-accel.c b/hw/arm/smmuv3-accel.c index a58815ded2..f381702a08 100644

hw/arm/smmuv3-accel.c +0 -0
@@ -754,6 +754,18 @@ static bool

smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,

hw/arm/smmuv3-accel.c static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,

Quoted 6 lines
hw/arm/smmuv3-accel.c +3 -0
         return false;
     }
 
+    /*
+     * CMDQV is active: block hot-unplug of the device that established the
+     * viommu association. Removing it would cause the vIOMMU to host

SMMUv3

Quoted 8 lines

hw/arm/smmuv3-accel.c static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn, · line 766

Quoted 7 lines
hw/arm/smmuv3-accel.c +7 -0
+     * association be changed via device hot-plug.
+     */
+    if (s->s_accel->cmdqv_ops) {
+        PCIDevice *pdev = pci_find_device(bus, pci_bus_num(bus), devfn);
+        error_setg(&accel_dev->unplug_blocker,
+                   "CMDQV is active: removing the device that established the "
+                   "viommu association would break the guest CMDQV");

It is something that is planned to be fixed later on?

Not really. It is not straightforward to support hot unplug of the device that enabled the VCMDQ on the associated SMMUv3.

Quoted 1 line

can you provide more details about the nature of the breakage

One scenario could be:
- Consider two SMMUv3+CMDQV instances, each with a passthrough device:
- dev_A → SMMUv3_A + CMDQV_A
- dev_B → SMMUv3_B + CMDQV_B
- Guest boots with dev_A cold-plugged; CMDQV_A is probed, viommu allocated against SMMUv3_A/CMDQV_A, and during guest boot time CMDQV configuration is established
- If dev_A is hot-unplugged, QEMU would have to release the viommu and tears down all associated CMDQV state (VINTFs, VCMDQs)
- If dev_B is subsequently hot-plugged, QEMU allocates a new viommu against SMMUv3_B/CMDQV_B.
- The guest, however, still holds the boot time SMMUv3_A/CMDQV_A configuration (VINTFs, VCMDQs) and will continue issuing commands against it.

Blocking hot-unplug of the establishing device prevents this viommu/CMDQV teardown from happening while the guest is actively using it.

Supporting hot-unplug may be possible if we retain the SMMUv3-to-viommu association even after the device is unplugged. However, that would require different lifetime management in the SMMUv3 accel and VFIO code paths, which would be non-trivial.

There is no strong user requirement for supporting hot unplug of the device like in the above scenario on these platforms either.

Thanks,
Shameer

#99 lore raw

PATCH v4 31/31 hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

In reply to Shameer Kolothum

Introduce a "cmdqv" property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

---
 hw/arm/smmuv3.c | 7 +++++++
 1 file changed, 7 insertions(+)
hw/arm/smmuv3.c +7 -0
index c9ff6298f5..51b7d01da5 100644
@@ -1993,6 +1993,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
             error_setg(errp, "ssidsize can only be set if accel=on");
             return false;
         }
+        if (s->cmdqv == ON_OFF_AUTO_ON) {
+            error_setg(errp, "cmdqv can only be enabled if accel=on");
+            return false;
+        }
         return true;
     }
 
@@ -2161,6 +2165,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_AUTO),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
                               SSID_SIZE_MODE_AUTO),
+    DEFINE_PROP_ON_OFF_AUTO("cmdqv", SMMUv3State, cmdqv, ON_OFF_AUTO_AUTO),
 };
 
 static void smmuv3_instance_init(Object *obj)
@@ -2200,6 +2205,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to auto. A value greater than 0 is required to enable "
         "PASID support.");
+    object_class_property_set_description(klass, "cmdqv",
+        "Enable/disable CMDQ-Virtualisation support (for accel=on)");
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
#100 lore raw

Re: [PATCH v4 31/31] hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

Quoted 33 lines · Shameer Kolothum

Introduce a "cmdqv" property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

---
 hw/arm/smmuv3.c | 7 +++++++
 1 file changed, 7 insertions(+)
hw/arm/smmuv3.c +5 -0
index c9ff6298f5..51b7d01da5 100644
@@ -1993,6 +1993,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
             error_setg(errp, "ssidsize can only be set if accel=on");
             return false;
         }
+        if (s->cmdqv == ON_OFF_AUTO_ON) {
+            error_setg(errp, "cmdqv can only be enabled if accel=on");
+            return false;
+        }
         return true;
     }
 
@@ -2161,6 +2165,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_OAS_MODE("oas", SMMUv3State, oas, OAS_MODE_AUTO),
     DEFINE_PROP_SSIDSIZE_MODE("ssidsize", SMMUv3State, ssidsize,
                               SSID_SIZE_MODE_AUTO),
+    DEFINE_PROP_ON_OFF_AUTO("cmdqv", SMMUv3State, cmdqv, ON_OFF_AUTO_AUTO),
 };
 
 static void smmuv3_instance_init(Object *obj)

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2209

Quoted 5 lines
hw/arm/smmuv3.c +2 -0
@@ -2200,6 +2205,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to auto. A value greater than 0 is required to enable "
         "PASID support.");
+    object_class_property_set_description(klass, "cmdqv",
+        "Enable/disable CMDQ-Virtualisation support (for accel=on)");

s/CMDQ-Virtualisation/CMDQ-V

hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2212

Quoted 3 lines
hw/arm/smmuv3.c +0 -0
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,

qemu-options.hx also needs to be updated

Eric