Skip to contentSkip to conversation

hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

PATCH Superseded Shameer Kolothum 5 participants 92 messages Feb 26 – Apr 15 1 message not in the archive

Subsystems (2)

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

Hi,

Changes from v2:
https://lore.kernel.org/qemu-devel/[email protected]/

- Addressed feedback on v2 and picked up R-by tags. Thanks!
- Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto

    "cmdqv".
  - Introduced a probe() callback to detect and initialize CMDQV
    support during device attach (patch #7).
  - Since CMDQV initialization is now deferred to device attach time,
    added a helper to link any unmapped MMIO/IRQ resources (patch #12).
  - Moved VINTF enable handling to the CONFIG ENABLE bit write path
    instead of any read/write access path (patch #17). Hopefully this
    won't break any corner cases.
  - Added checks for VINTF and CMDQV ENABLE bits before allocating
    vCMDQ HW queues, and free any allocated resources if disabled
    (patch #19).
  - Introduced a common helper for vEVENTQ read and validation
    (patch #23).
  - Updated bios-tables test IORT blobs for SMMUv3 device identifier
    changes (patch #29).
  - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
    using tegra241-cmdqv-stubs.c (patch #6).

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcmdq-v3

I have sanity tested this on an NVIDIA GRACE platform and will continue with additional tests.

Feedback and testing are very welcome.

Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/

---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/[email protected]/

Thanks to Nicolin for the initial patches and testing on which this is based.

Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.

Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)

Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.

VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].

---

Nicolin Chen (15):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free hw/arm/tegra241-cmdqv: Emulate global CMDQV registers hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Shameer Kolothum (17):
hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle hw/arm/virt: Store SMMUv3 device objects in VirtMachineState. hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus system/physmem: Add address_space_is_ram() helper hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free hw/arm/smmuv3-accel: Introduce common helper for veventq read hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier
change
hw/arm/smmuv3: Add per-device identifier property tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier
change
hw/arm/smmuv3-accel: Introduce helper to query CMDQV type hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

 hw/arm/smmuv3-accel.h                         |  29 +
 hw/arm/tegra241-cmdqv.h                       | 342 ++++++++
 include/hw/arm/smmuv3.h                       |   3 +
 include/hw/arm/virt.h                         |   1 +
 include/system/iommufd.h                      |  16 +
 include/system/memory.h                       |  10 +
 backends/iommufd.c                            |  64 ++
 hw/arm/smmuv3-accel-stubs.c                   |  11 +
 hw/arm/smmuv3-accel.c                         | 196 ++++-
 hw/arm/smmuv3.c                               |   9 +
 hw/arm/tegra241-cmdqv-stubs.c                 |  18 +
 hw/arm/tegra241-cmdqv.c                       | 793 ++++++++++++++++++
 hw/arm/virt-acpi-build.c                      | 126 ++-
 hw/arm/virt.c                                 |  27 +
 hw/vfio/iommufd.c                             |   4 +-
 system/physmem.c                              |  11 +
 backends/trace-events                         |   4 +-
 hw/arm/Kconfig                                |   5 +
 hw/arm/meson.build                            |   2 +
 hw/arm/trace-events                           |   5 +
 tests/data/acpi/aarch64/virt/IORT.its_off     | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m  | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-dev  | Bin 260 -> 260 bytes
 .../data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes
 24 files changed, 1596 insertions(+), 80 deletions(-)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
Signature · 1 line
2.43.0
#1 lore raw

PATCH v3 1/32 backends/iommufd: Update iommufd_backend_get_device_info

From: Nicolin Chen <[email protected]>

The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.

---
 backends/iommufd.c    | 7 +++++++
 hw/arm/smmuv3-accel.c | 2 +-
 hw/vfio/iommufd.c     | 4 ++--
 3 files changed, 10 insertions(+), 3 deletions(-)
backends/iommufd.c +7 -0
index acfab907c0..5daefe505e 100644
@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,
     return true;
 }
 
+/*
+ * @type can carry a desired HW info type defined in the uapi headers. If caller
+ * doesn't have one, indicating it wants the default type, then @type should be
+ * zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT).
+ */
 bool iommufd_backend_get_device_info(IOMMUFDBackend *be, uint32_t devid,
                                      uint32_t *type, void *data, uint32_t len,
                                      uint64_t *caps, uint8_t *max_pasid_log2,
                                      Error **errp)
 {
     struct iommu_hw_info info = {
+        .flags = (*type) ? IOMMU_HW_INFO_FLAG_INPUT_TYPE : 0,
         .size = sizeof(info),
         .dev_id = devid,
         .data_len = len,
         .data_uptr = (uintptr_t)data,
+        .in_data_type = *type,
     };
 
     if (ioctl(be->fd, IOMMU_GET_HW_INFO, &info)) {
hw/arm/smmuv3-accel.c +1 -1
index f2fd926160..2c4228b7bd 100644
@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                            Error **errp)
 {
     struct iommu_hw_info_arm_smmuv3 info;
-    uint32_t data_type;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_DEFAULT;
     uint64_t caps;
 
     if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +2 -2
index 131612eb83..3854f8b9fd 100644
@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,
     ERRP_GUARD();
     IOMMUFDBackend *iommufd = vbasedev->iommufd;
     VFIOContainer *bcontainer = VFIO_IOMMU(container);
-    uint32_t type, flags = 0;
+    uint32_t type = IOMMU_HW_INFO_TYPE_DEFAULT, flags = 0;
     uint64_t hw_caps;
     VendorCaps caps;
     VFIOIOASHwpt *hwpt;
@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,
     HostIOMMUDeviceIOMMUFD *idev;
     HostIOMMUDeviceCaps *caps = &hiod->caps;
     VendorCaps *vendor_caps = &caps->vendor_caps;
-    enum iommu_hw_info_type type;
+    uint32_t type = IOMMU_HW_INFO_TYPE_DEFAULT;
     uint8_t max_pasid_log2;
     uint64_t hw_caps;
 
Signature · 1 line
2.43.0
#2 lore raw

PATCH v3 2/32 backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.

Update the caller accordingly.

---
 include/system/iommufd.h | 1 +
 backends/iommufd.c       | 4 ++++
 hw/arm/smmuv3-accel.c    | 4 ++--
 backends/trace-events    | 2 +-
 4 files changed, 8 insertions(+), 3 deletions(-)
include/system/iommufd.h +1 -0
index 7062944fe6..e027800c91 100644
@@ -89,6 +89,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id,
                                 Error **errp);
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_hwpt, Error **errp);
 
 bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
backends/iommufd.c +4 -0
index 5daefe505e..400946810d 100644
@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,
 
 bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
                                   uint32_t viommu_type, uint32_t hwpt_id,
+                                  void *data_ptr, uint32_t data_len,
                                   uint32_t *out_viommu_id, Error **errp)
 {
     int ret;
@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id,
         .type = viommu_type,
         .dev_id = dev_id,
         .hwpt_id = hwpt_id,
+        .data_len = data_len,
+        .data_uptr = (uintptr_t)data_ptr,
     };
 
     ret = ioctl(be->fd, IOMMU_VIOMMU_ALLOC, &alloc_viommu);
 
     trace_iommufd_backend_alloc_viommu(be->fd, dev_id, viommu_type, hwpt_id,
+                                       (uintptr_t)data_ptr, data_len,
                                        alloc_viommu.out_viommu_id, ret);
     if (ret) {
         error_setg_errno(errp, errno, "IOMMU_VIOMMU_ALLOC failed");
hw/arm/smmuv3-accel.c +2 -2
index 2c4228b7bd..ab1b0a3669 100644
@@ -533,8 +533,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     IOMMUFDViommu *viommu;
 
     if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
-                                      s2_hwpt_id, &viommu_id, errp)) {
+                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
+                                      NULL, 0, &viommu_id, errp)) {
         return false;
     }
 
backends/trace-events +1 -1
index b9365113e7..3ba0c3503c 100644
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (%
 iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)"
 iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)"
 iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)"
-iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)"
+iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 
Signature · 1 line
2.43.0
#3 lore raw

PATCH v3 3/32 backends/iommufd: Introduce iommufd_backend_alloc_hw_queue

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a helper to allocate an iommufd backed HW queue for a vIOMMU.

While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.

---
 include/system/iommufd.h | 11 +++++++++++
 backends/iommufd.c       | 31 +++++++++++++++++++++++++++++++
 backends/trace-events    |  1 +
 3 files changed, 43 insertions(+)
include/system/iommufd.h +11 -0
index e027800c91..8009ce3d31 100644
@@ -65,6 +65,12 @@ typedef struct IOMMUFDVeventq {
     bool event_start; /* True after first valid event; cleared on overflow */
 } IOMMUFDVeventq;
 
+/* HW queue object for a vIOMMU-specific HW-accelerated queue */
+typedef struct IOMMUFDHWqueue {
+    IOMMUFDViommu *viommu;
+    uint32_t hw_queue_id;
+} IOMMUFDHWqueue;
+
 bool iommufd_backend_connect(IOMMUFDBackend *be, Error **errp);
 void iommufd_backend_disconnect(IOMMUFDBackend *be);
 
@@ -101,6 +107,11 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
                                    uint32_t *out_veventq_id,
                                    uint32_t *out_veventq_fd, Error **errp);
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    uint32_t queue_type, uint32_t index,
+                                    uint64_t addr, uint64_t length,
+                                    uint32_t *out_hw_queue_id, Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
backends/iommufd.c +31 -0
index 400946810d..86f9c9f4cc 100644
@@ -546,6 +546,37 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
+                                    uint32_t queue_type, uint32_t index,
+                                    uint64_t addr, uint64_t length,
+                                    uint32_t *out_hw_queue_id, Error **errp)
+{
+    int ret;
+    struct iommu_hw_queue_alloc alloc_hw_queue = {
+        .size = sizeof(alloc_hw_queue),
+        .flags = 0,
+        .viommu_id = viommu_id,
+        .type = queue_type,
+        .index = index,
+        .nesting_parent_iova = addr,
+        .length = length,
+    };
+
+    ret = ioctl(be->fd, IOMMU_HW_QUEUE_ALLOC, &alloc_hw_queue);
+
+    trace_iommufd_backend_alloc_hw_queue(be->fd, viommu_id, queue_type,
+                                         index, addr, length,
+                                         alloc_hw_queue.out_hw_queue_id, ret);
+    if (ret) {
+        error_setg_errno(errp, errno, "IOMMU_HW_QUEUE_ALLOC failed");
+        return false;
+    }
+
+    g_assert(out_hw_queue_id);
+    *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id;
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index 3ba0c3503c..c5c1d95aad 100644
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u
 iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)"
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
+iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
Signature · 1 line
2.43.0
#4 lore raw

PATCH v3 4/32 backends/iommufd: Introduce iommufd_backend_viommu_mmap

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.

The caller is responsible for unmapping the returned region.

---
 include/system/iommufd.h |  4 ++++
 backends/iommufd.c       | 22 ++++++++++++++++++++++
 backends/trace-events    |  1 +
 3 files changed, 27 insertions(+)
include/system/iommufd.h +4 -0
index 8009ce3d31..38cfceca84 100644
@@ -112,6 +112,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
                                     uint64_t addr, uint64_t length,
                                     uint32_t *out_hw_queue_id, Error **errp);
 
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp);
+
 bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be, uint32_t hwpt_id,
                                         bool start, Error **errp);
 bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
backends/iommufd.c +22 -0
index 86f9c9f4cc..6defdf6a52 100644
@@ -577,6 +577,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
     return true;
 }
 
+/*
+ * Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance.
+ * The caller is responsible for unmapping the mapped region.
+ */
+bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t viommu_id,
+                                 uint64_t size, off_t offset, void **out_ptr,
+                                 Error **errp)
+{
+    g_assert(viommu_id);
+    g_assert(out_ptr);
+
+    *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, be->fd,
+                   offset);
+    trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size, offset);
+    if (*out_ptr == MAP_FAILED) {
+        error_setg_errno(errp, errno, "IOMMUFD vIOMMU mmap failed");
+        return false;
+    }
+
+    return true;
+}
+
 bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
                                            uint32_t hwpt_id, Error **errp)
 {
backends/trace-events +1 -0
index c5c1d95aad..b63420b73e 100644
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32
 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"
 iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
 iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"
+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64
 
 # igvm-cfg.c
 igvm_reset_enter(int type) "type=%u"
Signature · 1 line
2.43.0
#5 lore raw

PATCH v3 5/32 hw/arm/smmuv3-accel: Introduce CMDQV ops interface

In reply to Shameer Kolothum

Command Queue Virtualization (CMDQV) is a hardware extension available on certain platforms that allows the SMMUv3 command queue to be virtualized and passed through to a VM, improving performance.

For example, NVIDIA Tegra241 implements CMDQV to support virtualization of multiple command queues (VCMDQs).

The term CMDQV is used here generically to refer to any platform that provides hardware support to virtualize the SMMUv3 command queue.

CMDQV support is a specialization of the IOMMUFD-backed accelerated SMMUv3 path. Introduce an ops interface to factor out CMDQV-specific probe, initialization, and vIOMMU allocation logic from the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.

This provides an extensible design to support future vendor-specific CMDQV implementations.

No functional change.

---
 hw/arm/smmuv3-accel.h | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
hw/arm/smmuv3-accel.h +17 -0
index 85669d0e00..5bdd01afb5 100644
@@ -15,6 +15,22 @@
 #include <linux/iommufd.h>
 #endif
 
+/*
+ * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
+ * support multiple VCMDQs with virtualization capabilities.
+ * CMDQV specific behavior is factored behind this ops interface.
+ */
+typedef struct SMMUv3AccelCmdqvOps {
+    bool (*probe)(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp);
+    bool (*init)(SMMUv3State *s, Error **errp);
+    bool (*alloc_viommu)(SMMUv3State *s,
+                         HostIOMMUDeviceIOMMUFD *idev,
+                         uint32_t *out_viommu_id,
+                         Error **errp);
+    void (*free_viommu)(SMMUv3State *s);
+    void (*reset)(SMMUv3State *s);
+} SMMUv3AccelCmdqvOps;
+
 /*
  * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
  * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -25,6 +41,7 @@ typedef struct SMMUv3AccelState {
     uint32_t bypass_hwpt_id;
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops;
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
Signature · 1 line
2.43.0
#6 lore raw

PATCH v3 6/32 hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

In reply to Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
 hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 96 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
hw/arm/tegra241-cmdqv.h new file +15 -0
new file mode 100644
index 0000000000..07e10e86ee
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv-stubs.c new file +18 -0
new file mode 100644
index 0000000000..eedc7bfdcd
@@ -0,0 +1,18 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return NULL;
+}
+
hw/arm/tegra241-cmdqv.c new file +56 -0

56 lines not on this page · raw message

hw/arm/Kconfig +5 -0
index c66c452737..3305c6e76e 100644
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +2 -0
index 8f834c32b1..fc83b635e7 100644
@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
Signature · 1 line
2.43.0
#7 lore raw

Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

Hi Shameer,

Quoted 48 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
 hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 96 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
hw/arm/tegra241-cmdqv.h new file +15 -0
new file mode 100644
index 0000000000..07e10e86ee
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv-stubs.c new file +0 -0
new file mode 100644
index 0000000000..eedc7bfdcd

hw/arm/tegra241-cmdqv-stubs.c line 4

Quoted 4 lines
hw/arm/tegra241-cmdqv-stubs.c new file +4 -0
@@ -0,0 +1,18 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3

^typo

Quoted 60 lines
hw/arm/tegra241-cmdqv-stubs.c new file +14 -0
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return NULL;
+}
+
hw/arm/tegra241-cmdqv.c new file +40 -0
new file mode 100644
index 0000000000..ad5a0d4611
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
+{
+}
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                            uint32_t *out_viommu_id, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s)
+{
+}
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                                 Error **errp)
+{

hw/arm/tegra241-cmdqv.c line 50

Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10 -0
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
+    .probe = tegra241_cmdqv_probe,
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .reset = tegra241_cmdqv_reset,

I think I would create a no_cmdqv_ops to convert the existing code without any cmdqv support to the same api. This would avoid the repeating:

if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.

Also this looks a bit strange to have different signatures for the functions used with both vcmdq on and off

Eric

Quoted 42 lines
hw/arm/tegra241-cmdqv.c new file +6 -0
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return &tegra241_cmdqv_ops;
+}
hw/arm/Kconfig +5 -0
index c66c452737..3305c6e76e 100644
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +2 -0
index 8f834c32b1..fc83b635e7 100644
@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
#8 lore raw

RE: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

Hi Eric,

Quoted 140 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 09:18
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 7 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Quoted 7 lines

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
 hw/arm/tegra241-cmdqv.c       | 56

+++++++++++++++++++++++++++++++++++

Quoted 33 lines
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 96 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h  create mode 100644

hw/arm/tegra241-cmdqv-stubs.c create mode 100644
hw/arm/tegra241-cmdqv.c

hw/arm/tegra241-cmdqv.h new renamed from hw/arm/tegra241-cmdqv.h +0 -0

file mode 100644 index 0000000000..07e10e86ee

hw/arm/tegra241-cmdqv.h new file +15 -0
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
+rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later  */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
(quoted diff) +0 -0

b/hw/arm/tegra241-cmdqv-stubs.c new file mode 100644 index 0000000000..eedc7bfdcd

hw/arm/tegra241-cmdqv-stubs.c line 5

Quoted 5 lines
hw/arm/tegra241-cmdqv-stubs.c new file +5 -0
@@ -0,0 +1,18 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
+rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3

^typo

Quoted 26 lines
hw/arm/tegra241-cmdqv-stubs.c new file +12 -0
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
+    return NULL;
+}
+
hw/arm/tegra241-cmdqv.c new renamed from hw/arm/tegra241-cmdqv.c +0 -0

file mode 100644 index 0000000000..ad5a0d4611

hw/arm/tegra241-cmdqv.c new file +9 -0
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
+rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later  */
+

hw/arm/tegra241-cmdqv.c line 19

Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10 -0
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s) { }
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c

Quoted 13 lines
hw/arm/tegra241-cmdqv.c new file +13 -0
+                            uint32_t *out_viommu_id, Error **errp) {
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s) { }
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) {
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static bool tegra241_cmdqv_probe(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c line 50

Quoted 11 lines
hw/arm/tegra241-cmdqv.c new file +11 -0
+                                 Error **errp) {
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
+    .probe = tegra241_cmdqv_probe,
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .reset = tegra241_cmdqv_reset,

I think I would create a no_cmdqv_ops to convert the existing code without any cmdqv support to the same api. This would avoid the repeating:

Quoted 1 line

if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.

I guess you are suggesting having something like,

static const SMMUv3AccelCmdqvOps no_cmdqv_ops = {
.probe = no_cmdqv_probe,
.alloc_viommu = no_cmdqv_alloc_viommu,
...
};

static bool no_cmdqv_probe(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev, Error **errp)

{
    return true;
}

static bool no_cmdqv_alloc_viommu(SMMUv3State *s,

                                  HostIOMMUDeviceIOMMUFD *idev,
                                  uint32_t *id,
                                  Error **errp)

{
return false; /* We have to call the default vIOMMU alloc*/
}

And then call them directly. I will take a look at that approach.

Quoted 3 lines

Also this looks a bit strange to have different signatures for the functions used with both vcmdq on and off

Is the concern here is mainly around alloc_viommu( that is the one which require fallback to default vIOMMU alloc)?

static bool
smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Error **errp)

and

bool (*alloc_viommu)(SMMUv3State *s,

                         HostIOMMUDeviceIOMMUFD *idev,
                         uint32_t *out_viommu_id,
                         Error **errp);

Making those consistent would likely require some refactoring of the existing path. I will have a play with that and see.

Thanks,
Shameer

Quoted 50 lines

Eric

hw/arm/Kconfig config ARM_SMMUV3_ACCEL config

Quoted 21 lines
hw/arm/tegra241-cmdqv.c new file +5 -0
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
+    return &tegra241_cmdqv_ops;
+}
hw/arm/Kconfig index renamed from hw/arm/Kconfig +0 -0

c66c452737..3305c6e76e 100644

hw/arm/Kconfig +4 -0
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL  config

ARM_SMMUV3

hw/arm/meson.build arm_common_ss.add(when:

Quoted 11 lines
hw/arm/Kconfig +1 -0
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +0 -0
index 8f834c32b1..fc83b635e7 100644
@@ -88,6 +88,8 @@ arm_common_ss.add(when:

'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 88

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true:

files('smmuv3.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 89

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:

files('smmuv3-accel.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 91

Quoted 2 lines
hw/arm/meson.build +1 -0
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true:

files('tegra241-cmdqv.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 93

Quoted 2 lines
hw/arm/meson.build +1 -0
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-

imx6ul.c', 'mcimx6ul-evk.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 94

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:

files('nrf51_soc.c'))

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
#9 lore raw

Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

Hi Shameer,

Quoted 158 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 139 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 09:18
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 7 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Quoted 7 lines

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
 hw/arm/tegra241-cmdqv.c       | 56

+++++++++++++++++++++++++++++++++++

Quoted 33 lines
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 96 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h  create mode 100644

hw/arm/tegra241-cmdqv-stubs.c create mode 100644
hw/arm/tegra241-cmdqv.c

hw/arm/tegra241-cmdqv.h new renamed from hw/arm/tegra241-cmdqv.h +0 -0

file mode 100644 index 0000000000..07e10e86ee

hw/arm/tegra241-cmdqv.h new file +15 -0
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
+rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later  */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
(quoted diff) +0 -0

b/hw/arm/tegra241-cmdqv-stubs.c new file mode 100644 index 0000000000..eedc7bfdcd

hw/arm/tegra241-cmdqv-stubs.c line 5

Quoted 5 lines
hw/arm/tegra241-cmdqv-stubs.c new file +5 -0
@@ -0,0 +1,18 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
+rights reserved
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3

^typo

Quoted 26 lines
hw/arm/tegra241-cmdqv-stubs.c new file +12 -0
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
+    return NULL;
+}
+
hw/arm/tegra241-cmdqv.c new renamed from hw/arm/tegra241-cmdqv.c +0 -0

file mode 100644 index 0000000000..ad5a0d4611

hw/arm/tegra241-cmdqv.c new file +9 -0
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All
+rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later  */
+

hw/arm/tegra241-cmdqv.c line 19

Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10 -0
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s) { }
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c

Quoted 13 lines
hw/arm/tegra241-cmdqv.c new file +13 -0
+                            uint32_t *out_viommu_id, Error **errp) {
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s) { }
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) {
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static bool tegra241_cmdqv_probe(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c line 50

Quoted 11 lines
hw/arm/tegra241-cmdqv.c new file +11 -0
+                                 Error **errp) {
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
+    .probe = tegra241_cmdqv_probe,
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .reset = tegra241_cmdqv_reset,

I think I would create a no_cmdqv_ops to convert the existing code without any cmdqv support to the same api. This would avoid the repeating:

Quoted 1 line

if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.

I guess you are suggesting having something like,

static const SMMUv3AccelCmdqvOps no_cmdqv_ops = {
.probe = no_cmdqv_probe,
.alloc_viommu = no_cmdqv_alloc_viommu,
...
};

static bool no_cmdqv_probe(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev, Error **errp)

{
    return true;
}

static bool no_cmdqv_alloc_viommu(SMMUv3State *s,

Quoted 8 lines
                                  HostIOMMUDeviceIOMMUFD *idev,
                                  uint32_t *id,
                                  Error **errp)

{
return false; /* We have to call the default vIOMMU alloc*/
}

And then call them directly. I will take a look at that approach.

Well forget it, I don't think it is worth the pain.

Quoted 12 lines
Quoted 3 lines

Also this looks a bit strange to have different signatures for the functions used with both vcmdq on and off

Is the concern here is mainly around alloc_viommu( that is the one which require fallback to default vIOMMU alloc)?

static bool
smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Error **errp)

and

Quoted 7 lines

bool (*alloc_viommu)(SMMUv3State *s,

                         HostIOMMUDeviceIOMMUFD *idev,
                         uint32_t *out_viommu_id,
                         Error **errp);

Making those consistent would likely require some refactoring of the existing path. I will have a play with that and see.

Same here. That's OK for now.

Thanks

Eric

Quoted 54 lines

Thanks,
Shameer

Quoted 50 lines

Eric

hw/arm/Kconfig config ARM_SMMUV3_ACCEL config

Quoted 21 lines
hw/arm/tegra241-cmdqv.c new file +5 -0
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void) {
+    return &tegra241_cmdqv_ops;
+}
hw/arm/Kconfig index renamed from hw/arm/Kconfig +0 -0

c66c452737..3305c6e76e 100644

hw/arm/Kconfig +4 -0
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL  config

ARM_SMMUV3

hw/arm/meson.build arm_common_ss.add(when:

Quoted 11 lines
hw/arm/Kconfig +1 -0
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +0 -0
index 8f834c32b1..fc83b635e7 100644
@@ -88,6 +88,8 @@ arm_common_ss.add(when:

'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 88

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true:

files('smmuv3.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 89

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:

files('smmuv3-accel.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 91

Quoted 2 lines
hw/arm/meson.build +1 -0
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true:

files('tegra241-cmdqv.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 93

Quoted 2 lines
hw/arm/meson.build +1 -0
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-

imx6ul.c', 'mcimx6ul-evk.c'))

hw/arm/meson.build arm_common_ss.add(when: · line 94

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:

files('nrf51_soc.c'))

Quoted 1 line
hw/arm/meson.build +0 -0
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
#10 lore raw

Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub

In reply to Shameer Kolothum

Hi Shameer,

Quoted 50 lines · Shameer Kolothum

Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.

This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.

Functional CMDQV support is added in follow-up patches.

---
 hw/arm/tegra241-cmdqv.h       | 15 ++++++++++
 hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++
 hw/arm/tegra241-cmdqv.c       | 56 +++++++++++++++++++++++++++++++++++
 hw/arm/Kconfig                |  5 ++++
 hw/arm/meson.build            |  2 ++
 5 files changed, 96 insertions(+)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
hw/arm/tegra241-cmdqv.h new file +15 -0
new file mode 100644
index 0000000000..07e10e86ee
@@ -0,0 +1,15 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_ARM_TEGRA241_CMDQV_H
+#define HW_ARM_TEGRA241_CMDQV_H
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
+
+#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv-stubs.c new file +2 -0
new file mode 100644
index 0000000000..eedc7bfdcd
@@ -0,0 +1,18 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved

hw/arm/tegra241-cmdqv-stubs.c line 12

Quoted 10 lines
hw/arm/tegra241-cmdqv-stubs.c new file +10 -0
+ *
+ * Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "hw/arm/tegra241-cmdqv.h"

not sure you need all those headers for the stub

Eric

Quoted 104 lines
hw/arm/tegra241-cmdqv-stubs.c new file +6 -0
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return NULL;
+}
+
hw/arm/tegra241-cmdqv.c new file +56 -0
new file mode 100644
index 0000000000..ad5a0d4611
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
+ * NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3
+ *
+ * Written by Nicolin Chen, Shameer Kolothum
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+
+#include "hw/arm/smmuv3.h"
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
+static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
+{
+}
+
+static bool
+tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                            uint32_t *out_viommu_id, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static void tegra241_cmdqv_reset(SMMUv3State *s)
+{
+}
+
+static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                                 Error **errp)
+{
+    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
+    return false;
+}
+
+static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
+    .probe = tegra241_cmdqv_probe,
+    .init = tegra241_cmdqv_init,
+    .alloc_viommu = tegra241_cmdqv_alloc_viommu,
+    .free_viommu = tegra241_cmdqv_free_viommu,
+    .reset = tegra241_cmdqv_reset,
+};
+
+const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void)
+{
+    return &tegra241_cmdqv_ops;
+}
hw/arm/Kconfig +5 -0
index c66c452737..3305c6e76e 100644
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK
     depends on TCG
     select FSL_IMX8MP
 
+config TEGRA241_CMDQV
+    bool
+    depends on ARM_SMMUV3_ACCEL
+
 config ARM_SMMUV3_ACCEL
     bool
     depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL
 config ARM_SMMUV3
     bool
     select ARM_SMMUV3_ACCEL if IOMMUFD
+    imply TEGRA241_CMDQV
 
 config FSL_IMX6UL
     bool
hw/arm/meson.build +2 -0
index 8f834c32b1..fc83b635e7 100644
@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c'))
 arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c'))
 stub_ss.add(files('smmuv3-accel-stubs.c'))
+arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c'))
+stub_ss.add(files('tegra241-cmdqv-stubs.c'))
 arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c'))
 arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c'))
 arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
#11 lore raw

PATCH v3 7/32 hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

In reply to Shameer Kolothum

Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.

If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.

The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.

No functional change.

---
 include/hw/arm/smmuv3.h |  2 +
 hw/arm/smmuv3-accel.c   | 93 +++++++++++++++++++++++++++++++++++++----
 2 files changed, 88 insertions(+), 7 deletions(-)
include/hw/arm/smmuv3.h +2 -0
index 26b2fc42fd..648412cafc 100644
@@ -73,6 +73,8 @@ struct SMMUv3State {
     bool ats;
     uint8_t oas;
     uint8_t ssidsize;
+    /* SMMU CMDQV extension */
+    OnOffAuto cmdqv;
 };
 
 typedef enum {
hw/arm/smmuv3-accel.c +86 -7

134 lines not on this page · raw message

Signature · 1 line
2.43.0
#12 lore raw

Re: [PATCH v3 07/32] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle

Quoted 86 lines · Shameer Kolothum

Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.

If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.

If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.

If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.

When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.

The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.

No functional change.

---
 include/hw/arm/smmuv3.h |  2 +
 hw/arm/smmuv3-accel.c   | 93 +++++++++++++++++++++++++++++++++++++----
 2 files changed, 88 insertions(+), 7 deletions(-)
include/hw/arm/smmuv3.h +2 -0
index 26b2fc42fd..648412cafc 100644
@@ -73,6 +73,8 @@ struct SMMUv3State {
     bool ats;
     uint8_t oas;
     uint8_t ssidsize;
+    /* SMMU CMDQV extension */
+    OnOffAuto cmdqv;
 };
 
 typedef enum {
hw/arm/smmuv3-accel.c +13 -4
index ab1b0a3669..4373bbd97b 100644
@@ -18,6 +18,7 @@
 
 #include "smmuv3-internal.h"
 #include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
 
 /*
  * The root region aliases the global system memory, and shared_as_sysmem
@@ -522,6 +523,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                           Error **errp)
 {
     SMMUv3AccelState *accel = s->s_accel;
+    const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;
     struct iommu_hwpt_arm_smmuv3 bypass_data = {
         .ste = { SMMU_STE_CFG_BYPASS | SMMU_STE_VALID, 0x0ULL },
     };
@@ -532,10 +534,17 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     uint32_t viommu_id, hwpt_id;
     IOMMUFDViommu *viommu;
 
-    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
-                                      IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id,
-                                      NULL, 0, &viommu_id, errp)) {
-        return false;
+    if (cmdqv_ops) {
+        if (!cmdqv_ops->alloc_viommu(s, idev, &viommu_id, errp)) {
+            return false;
+        }
+    } else {
+        if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                          IOMMU_VIOMMU_TYPE_ARM_SMMUV3,
+                                          s2_hwpt_id, NULL, 0, &viommu_id,
+                                          errp)) {
+            return false;
+        }
     }
 
     viommu = g_new0(IOMMUFDViommu, 1);

hw/arm/smmuv3-accel.c free_bypass_hwpt: · line 597

Quoted 9 lines
hw/arm/smmuv3-accel.c +5 -1
@@ -581,12 +590,69 @@ free_bypass_hwpt:
 free_abort_hwpt:
     iommufd_backend_free_id(idev->iommufd, accel->abort_hwpt_id);
 free_viommu:
-    iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    if (cmdqv_ops && cmdqv_ops->free_viommu) {
+        cmdqv_ops->free_viommu(s);
+    } else {
+        iommufd_backend_free_id(idev->iommufd, viommu->viommu_id);
+    }

reading [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

tegra241_cmdqv_free_viommu() looks identifical in both implementations

So maybe this is not needed

Eric

Quoted 92 lines
hw/arm/smmuv3-accel.c +68 -2
     g_free(viommu);
     accel->viommu = NULL;
     return false;
 }
 
+static const SMMUv3AccelCmdqvOps *
+smmuv3_accel_probe_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = tegra241_cmdqv_get_ops();
+
+    if (!ops || !ops->probe) {
+        error_setg(errp, "No CMDQV ops found");
+        return NULL;
+    }
+
+    if (!ops->probe(s, idev, errp)) {
+        return NULL;
+    }
+    return ops;
+}
+
+static bool
+smmuv3_accel_select_cmdqv(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
+                          Error **errp)
+{
+    const SMMUv3AccelCmdqvOps *ops = NULL;
+
+    if (s->s_accel->cmdqv_ops) {
+        return true;
+    }
+
+    switch (s->cmdqv) {
+    case ON_OFF_AUTO_OFF:
+        s->s_accel->cmdqv_ops = NULL;
+        return true;
+    case ON_OFF_AUTO_AUTO:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, NULL);
+        break;
+    case ON_OFF_AUTO_ON:
+        ops = smmuv3_accel_probe_cmdqv(s, idev, errp);
+        if (!ops) {
+            error_append_hint(errp, "CMDQV requested but not supported");
+            return false;
+        }
+        s->s_accel->cmdqv_ops = ops;
+        break;
+    default:
+        g_assert_not_reached();
+    }
+
+    if (ops && ops->init && !ops->init(s, errp)) {
+        return false;
+    }
+    s->s_accel->cmdqv_ops = ops;
+    return true;
+}
+
 static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
                                           HostIOMMUDevice *hiod, Error **errp)
 {
@@ -621,6 +687,10 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn,
         goto done;
     }
 
+    if (!smmuv3_accel_select_cmdqv(s, idev, errp)) {
+        return false;
+    }
+
     if (!smmuv3_accel_alloc_viommu(s, idev, errp)) {
         error_append_hint(errp, "Unable to alloc vIOMMU: idev devid 0x%x: ",
                           idev->devid);
@@ -867,8 +937,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)
 
 void smmuv3_accel_reset(SMMUv3State *s)
 {
-     /* Attach a HWPT based on GBPA reset value */
-     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel) {
+        return;
+    }
+    /* Attach a HWPT based on GBPA reset value */
+    smmuv3_accel_attach_gbpa_hwpt(s, NULL);
+
+    if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
+        accel->cmdqv_ops->reset(s);
+    }
 }
 
 static void smmuv3_accel_as_init(SMMUv3State *s)
#13 lore raw

PATCH v3 8/32 hw/arm/virt: Store SMMUv3 device objects in VirtMachineState.

In reply to Shameer Kolothum

Introduce a GPtrArray in VirtMachineState to keep track of all SMMUv3 devices created on the virt machine.

This will avoid relying on object_child_foreach_recursive() walks of the object tree when accessing SMMUv3 instances. Subsequent patches will use this list during ACPI IORT table generation and for CMDQV-related handling.

No functional change.

---
 include/hw/arm/virt.h | 1 +
 hw/arm/virt.c         | 3 +++
 2 files changed, 4 insertions(+)
include/hw/arm/virt.h +1 -0
index 8069422769..f9437e6410 100644
@@ -186,6 +186,7 @@ struct VirtMachineState {
     MemoryRegion *sysmem;
     MemoryRegion *secure_sysmem;
     bool pci_preserve_config;
+    GPtrArray *smmuv3_devices;
 };
 
 #define VIRT_ECAM_ID(high) (high ? VIRT_HIGH_PCIE_ECAM : VIRT_PCIE_ECAM)
hw/arm/virt.c +3 -0
index 50865e8115..292e523664 100644
@@ -3232,6 +3232,7 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev,
             }
 
             create_smmuv3_dev_dtb(vms, dev, bus, errp);
+            g_ptr_array_add(vms->smmuv3_devices, dev);
         }
     }
 
@@ -3659,6 +3660,8 @@ static void virt_instance_init(Object *obj)
     vms->oem_id = g_strndup(ACPI_BUILD_APPNAME6, 6);
     vms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
     cxl_machine_init(obj, &vms->cxl_devices_state);
+
+    vms->smmuv3_devices = g_ptr_array_new_with_free_func(NULL);
 }
 
 static const TypeInfo virt_machine_info = {
Signature · 1 line
2.43.0
#14 lore raw

PATCH v3 9/32 hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build

In reply to Shameer Kolothum

Stop walking the object tree to discover SMMUv3 devices when building the IORT table.

Use the SMMUv3 device array maintained by the virt machine instead, avoiding recursive object traversal.

No functional change intended.

---
 hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++----------------------
 1 file changed, 31 insertions(+), 39 deletions(-)
hw/arm/virt-acpi-build.c +31 -39
index 544004615d..ae78e9b9e0 100644
@@ -385,49 +385,41 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
     return map_a->input_base - map_b->input_base;
 }
 
-static int iort_smmuv3_devices(Object *obj, void *opaque)
-{
-    VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
-    GArray *sdev_blob = opaque;
-    AcpiIortIdMapping idmap;
-    PlatformBusDevice *pbus;
-    int min_bus, max_bus;
-    SysBusDevice *sbdev;
-    PCIBus *bus;
-
-    if (!object_dynamic_cast(obj, TYPE_ARM_SMMUV3)) {
-        return 0;
-    }
-
-    bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
-    sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
-    sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
-    pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
-    sbdev = SYS_BUS_DEVICE(obj);
-    sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
-    sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
-    sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
-    sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
-    sdev.irq += ARM_SPI_BASE;
-
-    pci_bus_range(bus, &min_bus, &max_bus);
-    sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
-    idmap.input_base = min_bus << 8,
-    idmap.id_count = (max_bus - min_bus + 1) << 8,
-    g_array_append_val(sdev.rc_smmu_idmaps, idmap);
-    g_array_append_val(sdev_blob, sdev);
-    return 0;
-}
-
 /*
  * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and
  * return the total number of idmaps.
  */
-static int populate_smmuv3_dev(GArray *sdev_blob)
+static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
 {
-    object_child_foreach_recursive(object_get_root(),
-                                   iort_smmuv3_devices, sdev_blob);
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        AcpiIortSMMUv3Dev sdev = {0};
+        AcpiIortIdMapping idmap;
+        PlatformBusDevice *pbus;
+        int min_bus, max_bus;
+        SysBusDevice *sbdev;
+        PCIBus *bus;
+
+        bus = PCI_BUS(object_property_get_link(obj, "primary-bus",
+                                               &error_abort));
+        sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
+        sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
+        sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
+        sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        sdev.irq += ARM_SPI_BASE;
+
+        pci_bus_range(bus, &min_bus, &max_bus);
+        sdev.rc_smmu_idmaps = g_array_new(false, true,
+                                          sizeof(AcpiIortIdMapping));
+        idmap.input_base = min_bus << 8;
+        idmap.id_count = (max_bus - min_bus + 1) << 8;
+        g_array_append_val(sdev.rc_smmu_idmaps, idmap);
+        g_array_append_val(sdev_blob, sdev);
+    }
     /* Sort the smmuv3 devices(if any) by smmu idmap input_base */
     g_array_sort(sdev_blob, smmuv3_dev_idmap_compare);
     /*
@@ -561,7 +553,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     if (vms->legacy_smmuv3_present) {
         rc_smmu_idmaps_len = populate_smmuv3_legacy_dev(smmuv3_devs);
     } else {
-        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs);
+        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs, vms);
     }
 
     num_smmus = smmuv3_devs->len;
Signature · 1 line
2.43.0
Reviewed-by on v3 9/32
#15 lore raw

Re: [PATCH v3 09/32] hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build

Quoted 52 lines · Shameer Kolothum

Stop walking the object tree to discover SMMUv3 devices when building the IORT table.

Use the SMMUv3 device array maintained by the virt machine instead, avoiding recursive object traversal.

No functional change intended.

---
 hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++----------------------
 1 file changed, 31 insertions(+), 39 deletions(-)
hw/arm/virt-acpi-build.c +0 -31
index 544004615d..ae78e9b9e0 100644
@@ -385,49 +385,41 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)
     return map_a->input_base - map_b->input_base;
 }
 
-static int iort_smmuv3_devices(Object *obj, void *opaque)
-{
-    VirtMachineState *vms = VIRT_MACHINE(qdev_get_machine());
-    AcpiIortSMMUv3Dev sdev = {0};
-    GArray *sdev_blob = opaque;
-    AcpiIortIdMapping idmap;
-    PlatformBusDevice *pbus;
-    int min_bus, max_bus;
-    SysBusDevice *sbdev;
-    PCIBus *bus;
-
-    if (!object_dynamic_cast(obj, TYPE_ARM_SMMUV3)) {
-        return 0;
-    }
-
-    bus = PCI_BUS(object_property_get_link(obj, "primary-bus", &error_abort));
-    sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
-    sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
-    pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
-    sbdev = SYS_BUS_DEVICE(obj);
-    sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
-    sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
-    sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
-    sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
-    sdev.irq += ARM_SPI_BASE;
-
-    pci_bus_range(bus, &min_bus, &max_bus);
-    sdev.rc_smmu_idmaps = g_array_new(false, true, sizeof(AcpiIortIdMapping));
-    idmap.input_base = min_bus << 8,
-    idmap.id_count = (max_bus - min_bus + 1) << 8,
-    g_array_append_val(sdev.rc_smmu_idmaps, idmap);

hw/arm/virt-acpi-build.c static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b) · line 392

Quoted 10 lines
hw/arm/virt-acpi-build.c +1 -5
-    g_array_append_val(sdev_blob, sdev);
-    return 0;
-}
-
 /*
  * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and
  * return the total number of idmaps.
  */
-static int populate_smmuv3_dev(GArray *sdev_blob)
+static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)

nit: I would prefer vms to be the 1st arg

Quoted 36 lines
hw/arm/virt-acpi-build.c +29 -2
 {
-    object_child_foreach_recursive(object_get_root(),
-                                   iort_smmuv3_devices, sdev_blob);
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        AcpiIortSMMUv3Dev sdev = {0};
+        AcpiIortIdMapping idmap;
+        PlatformBusDevice *pbus;
+        int min_bus, max_bus;
+        SysBusDevice *sbdev;
+        PCIBus *bus;
+
+        bus = PCI_BUS(object_property_get_link(obj, "primary-bus",
+                                               &error_abort));
+        sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
+        sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
+        sdev.base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        sdev.irq = platform_bus_get_irqn(pbus, sbdev, 0);
+        sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        sdev.irq += ARM_SPI_BASE;
+
+        pci_bus_range(bus, &min_bus, &max_bus);
+        sdev.rc_smmu_idmaps = g_array_new(false, true,
+                                          sizeof(AcpiIortIdMapping));
+        idmap.input_base = min_bus << 8;
+        idmap.id_count = (max_bus - min_bus + 1) << 8;
+        g_array_append_val(sdev.rc_smmu_idmaps, idmap);
+        g_array_append_val(sdev_blob, sdev);
+    }
     /* Sort the smmuv3 devices(if any) by smmu idmap input_base */
     g_array_sort(sdev_blob, smmuv3_dev_idmap_compare);
     /*

hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 556

Quoted 5 lines
hw/arm/virt-acpi-build.c +1 -1
@@ -561,7 +553,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     if (vms->legacy_smmuv3_present) {
         rc_smmu_idmaps_len = populate_smmuv3_legacy_dev(smmuv3_devs);
     } else {
-        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs);
+        rc_smmu_idmaps_len = populate_smmuv3_dev(smmuv3_devs, vms);

So did you check this does not break existing bios qtests (as we change the way we iterate on the devices)? If so maybe add a comment in commit description.

hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 559

Quoted 3 lines
hw/arm/virt-acpi-build.c +0 -0
     }
 
     num_smmus = smmuv3_devs->len;

I would squash previous patch in that one.

Eric

#16 lore raw

PATCH v3 10/32 hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

In reply to Shameer Kolothum

Use IOMMU_GET_HW_INFO to check whether the host supports Tegra241 CMDQV.

Validate the returned data type, version, number of vCMDQs and SIDs per VM. Fail the probe if the host does not meet the expected requirements.

---
 hw/arm/tegra241-cmdqv.h |  5 +++++
 hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
 2 files changed, 35 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +5 -0
index 07e10e86ee..312064a081 100644
@@ -10,6 +10,11 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#define TEGRA241_CMDQV_VERSION             1
+#define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
+#define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
+#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +30 -2
index ad5a0d4611..a270fa7ce4 100644
@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
+    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
+    uint64_t caps;
+
+    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
+                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
+                                         NULL, errp)) {
+        return false;
+    }
+    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
+        error_setg(errp, "Host CMDQV: unexpected data type %u (expected %u)",
+                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
+        return false;
+    }
+    if (cmdqv_info.version != TEGRA241_CMDQV_VERSION) {
+        error_setg(errp, "Host CMDQV: unsupported version %u (expected %u)",
+                   cmdqv_info.version, TEGRA241_CMDQV_VERSION);
+        return false;
+    }
+    if (cmdqv_info.log2vcmdqs < TEGRA241_CMDQV_NUM_CMDQ_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)",
+                   cmdqv_info.log2vcmdqs, TEGRA241_CMDQV_NUM_CMDQ_LOG2);
+        return false;
+    }
+    if (cmdqv_info.log2vsids < TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >= %u)",
+                   cmdqv_info.log2vsids, TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);
+        return false;
+    }
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
Signature · 1 line
2.43.0
Reviewed-by on v3 10/32
#17 lore raw

Re: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

Quoted 4 lines · Shameer Kolothum

Use IOMMU_GET_HW_INFO to check whether the host supports Tegra241 CMDQV.

Validate the returned data type, version, number of vCMDQs and SIDs per VM. Fail the probe if the host does not meet the expected requirements.

Can you explain why it is important to check

number of vCMDQs and SIDs per VM?

Quoted 56 lines
---
 hw/arm/tegra241-cmdqv.h |  5 +++++
 hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
 2 files changed, 35 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +5 -0
index 07e10e86ee..312064a081 100644
@@ -10,6 +10,11 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#define TEGRA241_CMDQV_VERSION             1
+#define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
+#define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
+#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +23 -2
index ad5a0d4611..a270fa7ce4 100644
@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
+    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
+    uint64_t caps;
+
+    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
+                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
+                                         NULL, errp)) {
+        return false;
+    }
+    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
+        error_setg(errp, "Host CMDQV: unexpected data type %u (expected %u)",
+                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
+        return false;
+    }
+    if (cmdqv_info.version != TEGRA241_CMDQV_VERSION) {
+        error_setg(errp, "Host CMDQV: unsupported version %u (expected %u)",
+                   cmdqv_info.version, TEGRA241_CMDQV_VERSION);
+        return false;
+    }
+    if (cmdqv_info.log2vcmdqs < TEGRA241_CMDQV_NUM_CMDQ_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)",
+                   cmdqv_info.log2vcmdqs, TEGRA241_CMDQV_NUM_CMDQ_LOG2);
+        return false;

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) · line 73

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+    }
+    if (cmdqv_info.log2vsids < TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >= %u)",
+                   cmdqv_info.log2vsids, TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);
+        return false;
+    }
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {

Besides

Eric

#18 lore raw

RE: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

Quoted 21 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 10:31
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support

External email: Use caution opening links or attachments

Quoted 1 line · Shameer Kolothum

Use IOMMU_GET_HW_INFO to check whether the host supports Tegra241

Quoted 8 lines

CMDQV.

Quoted 3 lines

Validate the returned data type, version, number of vCMDQs and SIDs per VM. Fail the probe if the host does not meet the expected requirements.

Can you explain why it is important to check

number of vCMDQs and SIDs per VM?

Right. I will add that.

The QEMU Tegra241 CMDQV implementation supports only two VCMDQs and 16 SIDs per VM. The check ensures the host implementation is compatible with that.

Thanks,
Shameer

Quoted 78 lines
Quoted 13 lines
---
 hw/arm/tegra241-cmdqv.h |  5 +++++
 hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++--
 2 files changed, 35 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h index renamed from hw/arm/tegra241-cmdqv.h +0 -0

07e10e86ee..312064a081 100644

hw/arm/tegra241-cmdqv.h line 15

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +3 -0
@@ -10,6 +10,11 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#define TEGRA241_CMDQV_VERSION             1
+#define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
+#define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 38

Quoted 11 lines
hw/arm/tegra241-cmdqv.h +2 -0
+#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0 -0

ad5a0d4611..a270fa7ce4 100644

hw/arm/tegra241-cmdqv.c +0 -0
@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s,
 Error **errp)  static bool tegra241_cmdqv_probe(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +5 -2
                                  Error **errp)  {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV;
+    struct iommu_hw_info_tegra241_cmdqv cmdqv_info;
+    uint64_t caps;
+
+    if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid,

&data_type,

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+                                         &cmdqv_info, sizeof(cmdqv_info), &caps,
+                                         NULL, errp)) {
+        return false;
+    }
+    if (data_type != IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV) {
+        error_setg(errp, "Host CMDQV: unexpected data type %u (expected

%u)",

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+                   data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV);
+        return false;
+    }
+    if (cmdqv_info.version != TEGRA241_CMDQV_VERSION) {
+        error_setg(errp, "Host CMDQV: unsupported version %u (expected

%u)",

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+                   cmdqv_info.version, TEGRA241_CMDQV_VERSION);
+        return false;
+    }
+    if (cmdqv_info.log2vcmdqs < TEGRA241_CMDQV_NUM_CMDQ_LOG2) {
+        error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >=

%u)",

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 61

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+                   cmdqv_info.log2vcmdqs,

TEGRA241_CMDQV_NUM_CMDQ_LOG2);

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+        return false;
+    }
+    if (cmdqv_info.log2vsids <

TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 65

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+        error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >=

%u)",

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 66

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+                   cmdqv_info.log2vsids,

TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) · line 73

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +3 -0
+        return false;
+    }
+    return true;
 }
 
 static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {

Besides

#19 lore raw

PATCH v3 11/32 hw/arm/tegra241-cmdqv: Implement CMDQV init

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages:

0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0
0x20000: Global VCMDQ registers, Page1
0x30000: VINTF0 logical VCMDQ registers, Page0
0x40000: VINTF0 logical VCMDQ registers, Page1

This patch wires up the Tegra241 CMDQV init callback and allocates vendor-specific CMDQV state. The state pointer is stored in SMMUv3AccelState for use by subsequent CMDQV operations.

The CMDQV MMIO region and a dedicated IRQ line are registered with the SMMUv3 device. The MMIO read/write handlers are currently stubs and will be implemented in later patches.

The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF error conditions. This patch only registers the IRQ line. Interrupt generation and propagation to the guest will be added in a subsequent patch.

---
 hw/arm/smmuv3-accel.h   |  1 +
 hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++--
 3 files changed, 47 insertions(+), 2 deletions(-)
hw/arm/smmuv3-accel.h +1 -0
index 5bdd01afb5..7d6e4c6b76 100644
@@ -42,6 +42,7 @@ typedef struct SMMUv3AccelState {
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
     const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;  /* vendor specific CMDQV state */
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
hw/arm/tegra241-cmdqv.h +18 -0
index 312064a081..46aa9e8a9f 100644
@@ -15,6 +15,24 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+/*
+ * Tegra241 CMDQV MMIO layout (64KB pages)
+ *
+ * 0x00000  TEGRA241_CMDQV_CFG    (Global CMDQV configuration)
+ * 0x10000  TEGRA241_VCMDQ_PAGE0  (Virtual CMDQ page 0)
+ * 0x20000  TEGRA241_VCMDQ_PAGE1  (Virtual CMDQ page 1)
+ * 0x30000  TEGRA241_VINTF0_PAGE0 (Virtual interface 0, page 0)
+ * 0x40000  TEGRA241_VINTF0_PAGE1 (Virtual interface 0, page 1)
+ */
+#define TEGRA241_CMDQV_IO_LEN 0x50000
+
+typedef struct Tegra241CMDQV {
+    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
+    SMMUv3AccelState *s_accel;
+    MemoryRegion mmio_cmdqv;
+    qemu_irq irq;
+} Tegra241CMDQV;
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +28 -2
index a270fa7ce4..6959766129 100644
@@ -13,6 +13,16 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+                                 unsigned size)
+{
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
 }
@@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
 }
 
+static const MemoryRegionOps mmio_cmdqv_ops = {
+    .read = tegra241_cmdqv_read,
+    .write = tegra241_cmdqv_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv;
+
+    cmdqv = g_new0(Tegra241CMDQV, 1);
+    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
+                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
+    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
+    sysbus_init_irq(sbd, &cmdqv->irq);
+    cmdqv->s_accel = accel;
+    accel->cmdqv = cmdqv;
+    return true;
 }
 
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Signature · 1 line
2.43.0
Reviewed-by on v3 11/32
#20 lore raw

Re: [PATCH v3 11/32] hw/arm/tegra241-cmdqv: Implement CMDQV init

Quoted 114 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages:

0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0
0x20000: Global VCMDQ registers, Page1
0x30000: VINTF0 logical VCMDQ registers, Page0
0x40000: VINTF0 logical VCMDQ registers, Page1

This patch wires up the Tegra241 CMDQV init callback and allocates vendor-specific CMDQV state. The state pointer is stored in SMMUv3AccelState for use by subsequent CMDQV operations.

The CMDQV MMIO region and a dedicated IRQ line are registered with the SMMUv3 device. The MMIO read/write handlers are currently stubs and will be implemented in later patches.

The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF error conditions. This patch only registers the IRQ line. Interrupt generation and propagation to the guest will be added in a subsequent patch.

---
 hw/arm/smmuv3-accel.h   |  1 +
 hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++--
 3 files changed, 47 insertions(+), 2 deletions(-)
hw/arm/smmuv3-accel.h +1 -0
index 5bdd01afb5..7d6e4c6b76 100644
@@ -42,6 +42,7 @@ typedef struct SMMUv3AccelState {
     uint32_t abort_hwpt_id;
     QLIST_HEAD(, SMMUv3AccelDevice) device_list;
     const SMMUv3AccelCmdqvOps *cmdqv_ops;
+    void *cmdqv;  /* vendor specific CMDQV state */
 } SMMUv3AccelState;
 
 typedef struct SMMUS1Hwpt {
hw/arm/tegra241-cmdqv.h +18 -0
index 312064a081..46aa9e8a9f 100644
@@ -15,6 +15,24 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+/*
+ * Tegra241 CMDQV MMIO layout (64KB pages)
+ *
+ * 0x00000  TEGRA241_CMDQV_CFG    (Global CMDQV configuration)
+ * 0x10000  TEGRA241_VCMDQ_PAGE0  (Virtual CMDQ page 0)
+ * 0x20000  TEGRA241_VCMDQ_PAGE1  (Virtual CMDQ page 1)
+ * 0x30000  TEGRA241_VINTF0_PAGE0 (Virtual interface 0, page 0)
+ * 0x40000  TEGRA241_VINTF0_PAGE1 (Virtual interface 0, page 1)
+ */
+#define TEGRA241_CMDQV_IO_LEN 0x50000
+
+typedef struct Tegra241CMDQV {
+    struct iommu_viommu_tegra241_cmdqv cmdqv_data;
+    SMMUv3AccelState *s_accel;
+    MemoryRegion mmio_cmdqv;
+    qemu_irq irq;
+} Tegra241CMDQV;
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +21 -2
index a270fa7ce4..6959766129 100644
@@ -13,6 +13,16 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
+{
+    return 0;
+}
+
+static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
+                                 unsigned size)
+{
+}
+
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
 }
@@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s)
 {
 }
 
+static const MemoryRegionOps mmio_cmdqv_ops = {
+    .read = tegra241_cmdqv_read,
+    .write = tegra241_cmdqv_write,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
 static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    SysBusDevice *sbd = SYS_BUS_DEVICE(OBJECT(s));
+    SMMUv3AccelState *accel = s->s_accel;
+    Tegra241CMDQV *cmdqv;
+
+    cmdqv = g_new0(Tegra241CMDQV, 1);

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_reset(SMMUv3State *s) · line 64

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+    memory_region_init_io(&cmdqv->mmio_cmdqv, OBJECT(s), &mmio_cmdqv_ops, cmdqv,
+                          "tegra241-cmdqv", TEGRA241_CMDQV_IO_LEN);
+    sysbus_init_mmio(sbd, &cmdqv->mmio_cmdqv);
+    sysbus_init_irq(sbd, &cmdqv->irq);
+    cmdqv->s_accel = accel;
+    accel->cmdqv = cmdqv;
+    return true;
 }
 
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,

Eric

#21 lore raw

PATCH v3 12/32 hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus

In reply to Shameer Kolothum

SMMUv3 devices with acceleration may enable CMDQV extensions after device realize. In that case, additional MMIO regions and IRQ lines may be registered but not yet mapped to the platform bus.

Ensure SMMUv3 device resources are linked to the platform bus during machine_done().

This is safe to do unconditionally since the platform bus helpers skip resources that are already mapped.

---
 hw/arm/virt.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
hw/arm/virt.c +21 -0
index 292e523664..c75a8d6e9e 100644
@@ -1833,6 +1833,24 @@ static void virt_build_smbios(VirtMachineState *vms)
     }
 }
 
+/*
+ * SMMUv3 devices with acceleration may enable CMDQV extensions
+ * after device realize. In that case, additional MMIO regions and
+ * IRQ lines may be registered but not yet mapped to the platform bus.
+ *
+ * Ensure all resources are linked to the platform bus before final
+ * machine setup.
+ */
+
+static void virt_smmuv3_dev_link_cmdqv(VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        DeviceState *dev = g_ptr_array_index(vms->smmuv3_devices, i);
+        platform_bus_link_device(PLATFORM_BUS_DEVICE(vms->platform_bus_dev),
+                                 SYS_BUS_DEVICE(dev));
+    }
+}
+
 static
 void virt_machine_done(Notifier *notifier, void *data)
 {
@@ -1849,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)
     if (vms->cxl_devices_state.is_enabled) {
         cxl_fmws_link_targets(&error_fatal);
     }
+
+    virt_smmuv3_dev_link_cmdqv(vms);
+
     /*
      * If the user provided a dtb, we assume the dynamic sysbus nodes
      * already are integrated there. This corresponds to a use case where
Signature · 1 line
2.43.0
Reviewed-by on v3 12/32
#22 lore raw

Re: [PATCH v3 12/32] hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus

Quoted 26 lines · Shameer Kolothum

SMMUv3 devices with acceleration may enable CMDQV extensions after device realize. In that case, additional MMIO regions and IRQ lines may be registered but not yet mapped to the platform bus.

Ensure SMMUv3 device resources are linked to the platform bus during machine_done().

This is safe to do unconditionally since the platform bus helpers skip resources that are already mapped.

---
 hw/arm/virt.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
hw/arm/virt.c +3 -0
index 292e523664..c75a8d6e9e 100644
@@ -1833,6 +1833,24 @@ static void virt_build_smbios(VirtMachineState *vms)
     }
 }
 
+/*
+ * SMMUv3 devices with acceleration may enable CMDQV extensions
+ * after device realize. In that case, additional MMIO regions and

hw/arm/virt.c static void virt_build_smbios(VirtMachineState *vms) · line 1848

Quoted 10 lines
hw/arm/virt.c +10 -0
+ * IRQ lines may be registered but not yet mapped to the platform bus.
+ *
+ * Ensure all resources are linked to the platform bus before final
+ * machine setup.
+ */
+
+static void virt_smmuv3_dev_link_cmdqv(VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        DeviceState *dev = g_ptr_array_index(vms->smmuv3_devices, i);

need extra line

Quoted 9 lines
hw/arm/virt.c +5 -0
+        platform_bus_link_device(PLATFORM_BUS_DEVICE(vms->platform_bus_dev),
+                                 SYS_BUS_DEVICE(dev));
+    }
+}
+
 static
 void virt_machine_done(Notifier *notifier, void *data)
 {

hw/arm/virt.c void virt_machine_done(Notifier *notifier, void *data) · line 1875

Quoted 9 lines
hw/arm/virt.c +3 -0
@@ -1849,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)
     if (vms->cxl_devices_state.is_enabled) {
         cxl_fmws_link_targets(&error_fatal);
     }
+
+    virt_smmuv3_dev_link_cmdqv(vms);
+
     /*
      * If the user provided a dtb, we assume the dynamic sysbus nodes
      * already are integrated there. This corresponds to a use case where

Eric

#23 lore raw

PATCH v3 13/32 hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.

Free the vIOMMU ID on teardown.

---
 hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +17 -2
index 6959766129..d487612ba2 100644
@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                             uint32_t *out_viommu_id, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
Signature · 1 line
2.43.0
#24 lore raw

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 30 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.

Free the vIOMMU ID on teardown.

---
 hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +7 -0
index 6959766129..d487612ba2 100644
@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
 {
+    SMMUv3AccelState *accel = s->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +4 -2
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                             uint32_t *out_viommu_id, Error **errp)
 {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+                                      IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,

if the only think that differs compared to no cmdq is the type, ie.

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv that is being used in the smmu_accel device. Then you can get rid of alloc and free

Eric

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6 -0
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
#25 lore raw

RE: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 63 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 11:09
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

External email: Use caution opening links or attachments

Quoted 22 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.

Free the vIOMMU ID on teardown.

---
 hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0 -0

6959766129..d487612ba2 100644

hw/arm/tegra241-cmdqv.c +1 -0
@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque,
 hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)  {
+    SMMUv3AccelState *accel = s->s_accel;

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 38

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +5 -2
                             uint32_t *out_viommu_id, Error **errp)  {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+
+ IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,

if the only think that differs compared to no cmdq is the type, ie.

Quoted 6 lines

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in the smmu_accel device. Then you can get rid of
+ alloc and free

It is not just the type. Based on the type we also need to pass,

* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *

And the above is implementation specific.

If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.

Thanks,
Shameer

Quoted 13 lines

Eric

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6 -0
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
#26 lore raw

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 75 lines · Shameer Kolothum Thodi
Quoted 62 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 11:09
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

External email: Use caution opening links or attachments

Quoted 22 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.

Free the vIOMMU ID on teardown.

---
 hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0 -0

6959766129..d487612ba2 100644

hw/arm/tegra241-cmdqv.c +1 -0
@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque,
 hwaddr offset, uint64_t value,
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)  {
+    SMMUv3AccelState *accel = s->s_accel;

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 38

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +6 -0
+    IOMMUFDViommu *viommu = accel->viommu;
+
+    if (!viommu) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, viommu->viommu_id);
 }
 
 static bool
 tegra241_cmdqv_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +5 -2
                             uint32_t *out_viommu_id, Error **errp)  {
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+
+ IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,

if the only think that differs compared to no cmdq is the type, ie.

Quoted 6 lines

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in the smmu_accel device. Then you can get rid of
+ alloc and free

It is not just the type. Based on the type we also need to pass,

* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *

Quoted 4 lines

And the above is implementation specific.

If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.

OK I missed the fact data_len and IOMMU data needed specialization.

Thanks

Eric

Quoted 16 lines

Thanks,
Shameer

Quoted 12 lines

Eric

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6 -0
+                                      idev->hwpt_id, &cmdqv->cmdqv_data,
+                                      sizeof(cmdqv->cmdqv_data), out_viommu_id,
+                                      errp)) {
+        return false;
+    }
+    return true;
 }
 
 static void tegra241_cmdqv_reset(SMMUv3State *s)
#27 lore raw

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

In reply to Shameer Kolothum Thodi
Quoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +5 -2
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+
+ IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,

if the only think that differs compared to no cmdq is the type, ie.

Quoted 6 lines

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in the smmu_accel device. Then you can get rid of
+ alloc and free

It is not just the type. Based on the type we also need to pass,

* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *

Quoted 4 lines

And the above is implementation specific.

If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.

Any future "cmdqv" will likely have a different viommu type. So, essentially this is just a viommu type override, as Eric pointed out above.

What we actually need to expose per viommu is type/data_len. The data_uptr can be a union viommu_data in the accel structure?

Nicolin

#28 lore raw

RE: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 46 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:10
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +5 -2
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+
+ IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,

if the only think that differs compared to no cmdq is the type, ie.

Quoted 6 lines

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in the smmu_accel device. Then you can get rid of
+ alloc and free

It is not just the type. Based on the type we also need to pass,

* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *

Quoted 4 lines

And the above is implementation specific.

If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.

Quoted 6 lines

Any future "cmdqv" will likely have a different viommu type. So, essentially this is just a viommu type override, as Eric pointed out above.

What we actually need to expose per viommu is type/data_len. The data_uptr can be a union viommu_data in the accel structure?

So the suggestion is to get rid of alloc_viommu() and retrieve type/len during probe() and store the data_uptr in struct accel . Doable, I guess.

But then we need to do something similar for veventq as well, right?

Isn't that actually bloating struct accel then?
IMHO, the ops way looks better.

Thanks,
Shameer

#29 lore raw

Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 53 lines · Shameer Kolothum Thodi
Quoted 46 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:10
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free

Quoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +5 -2
-    error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported");
-    return false;
+    Tegra241CMDQV *cmdqv = s->s_accel->cmdqv;
+
+    if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid,
+
+ IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,

if the only think that differs compared to no cmdq is the type, ie.

Quoted 6 lines

IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV

+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in the smmu_accel device. Then you can get rid of
+ alloc and free

It is not just the type. Based on the type we also need to pass,

* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *

Quoted 4 lines

And the above is implementation specific.

If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.

Quoted 6 lines

Any future "cmdqv" will likely have a different viommu type. So, essentially this is just a viommu type override, as Eric pointed out above.

What we actually need to expose per viommu is type/data_len. The data_uptr can be a union viommu_data in the accel structure?

Quoted 7 lines

So the suggestion is to get rid of alloc_viommu() and retrieve type/len during probe() and store the data_uptr in struct accel . Doable, I guess.

But then we need to do something similar for veventq as well, right?

Isn't that actually bloating struct accel then?
IMHO, the ops way looks better.

The data_uptr could be dynamically allocated corresponding to the data_len, to avoid bloating.

But you are right we do need a place to alloc/free a 2nd veventq.

Maybe we can have a pair of alloc/free_viommu for now.

Thanks
Nicolin

#30 lore raw

PATCH v3 14/32 hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

---
 hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +108 -0
index 46aa9e8a9f..50bcecee9d 100644
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+#define A_VI_ERR_MAP 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
+#define A_CMDQ_ALLOC_MAP_(i)                 \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)                       \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_MATCH_(i, j)                               \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +120 -1

139 lines not on this page · raw message

Signature · 1 line
2.43.0
#31 lore raw

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Hi Shameer,

Quoted 55 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

---
 hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +19 -0
index 46aa9e8a9f..50bcecee9d 100644
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 64

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)

this is called CMDQV_NUM_VI_LOG2 in the spec I have access to VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naming

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 65

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)

same here s/VM/VI

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 69

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+

I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 70

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_VI_ERR_MAP 0x14

_0?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 74

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+

same SMMU_CMDQV_VI_ING_MASK0/1

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 79

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +5 -0
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+

SMMU_CMDQV_CMDQ_ERR_MAP0-3

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
 
+#define A_CMDQ_ALLOC_MAP_(i)   

why A_? Since you rely on the REG macros, A_ should prefix the address offset
Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in the spec?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 91

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5 -0
              \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+

Please explain why we only expose 2 of those regs among 128?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)     

again why don't we use the spec terminology
SMMU_CMDQV_VINTF0_CONFIG_0

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
                  \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 114

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)

Nit: the spec also contains the typo but I would rather fix it in the code

Explicitly state we only expose 1 VINTF

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 119

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+
+/* i = [0, 0], j = [0, 15] */

does that mean that we can have a max 16 SIDs per VM?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 120

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_VINTFi_SID_MATCH_(i, j) 

if matched vi, I would prefer vi

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 126

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9 -0
                              \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */

vint = 0, 16 identical register entries

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 129

Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3 -0
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)

s/19/20

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 135

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */

vi = 0, 4 identical regs

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 141

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */

I don't get this comment

Quoted 43 lines
hw/arm/tegra241-cmdqv.h +2 -0
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +25 -0
index d487612ba2..a3830a02d6 100644
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}

hw/arm/tegra241-cmdqv.c line 49

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -1
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);

If all those regs belong to the Global CMDQV registers page this shall be within the first 64KB

Quoted 26 lines
hw/arm/tegra241-cmdqv.c +26 -0
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}

hw/arm/tegra241-cmdqv.c line 85

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;

Can you explain why this needed?

Quoted 12 lines
hw/arm/tegra241-cmdqv.c +12 -0
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;

hw/arm/tegra241-cmdqv.c line 107

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
 }

nit: if you can put write_vintf just after its read fellow, I think it would ease the comparison/review

Quoted 28 lines
hw/arm/tegra241-cmdqv.c +24 -0
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;

hw/arm/tegra241-cmdqv.c line 145

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

At this stage of the reading it is unclear to me why we need to expose all those regs to the guest. I would assume that a subset of them are used by the hyp and not supposed to be accessed by the guest. For instance the SID match/replace, ...

Please could you clairfy what is absolutely needed to expose to the guest?

Thanks

Eric

#32 lore raw

RE: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Hi Eric,

Quoted 119 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 16:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

Quoted 4 lines
---
 hw/arm/tegra241-cmdqv.h | 108

+++++++++++++++++++++++++++++++++++

Quoted 1 line
 hw/arm/tegra241-cmdqv.c | 121

+++++++++++++++++++++++++++++++++++++++-

Quoted 7 lines
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +0 -0
index 46aa9e8a9f..50bcecee9d 100644

hw/arm/tegra241-cmdqv.h line 18

Quoted 9 lines
hw/arm/tegra241-cmdqv.h +3 -0
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

Quoted 22 lines
hw/arm/tegra241-cmdqv.h +16 -0
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 64

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)

this is called CMDQV_NUM_VI_LOG2 in the spec I have access to VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naming

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 65

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)

same here s/VM/VI

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 69

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+

I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 70

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_VI_ERR_MAP 0x14

_0?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 74

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+

same SMMU_CMDQV_VI_ING_MASK0/1

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 79

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +5 -0
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+

SMMU_CMDQV_CMDQ_ERR_MAP0-3

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
Quoted 5 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 86

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_CMDQ_ALLOC_MAP_(i)

why A_? Since you rely on the REG macros, A_ should prefix the address offset
Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in the spec?

I will incorporate all suggestions related to naming etc and will make it same as used in the spec.

Quoted 7 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 91

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5 -0
              \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+

Please explain why we only expose 2 of those regs among 128?

QEMU currently models only two VCMDQs as we expose that by setting TEGRA241_CMDQV_NUM_CMDQ_LOG2 = 1 via SMMU_CMDQV_PARAM_0. Therefore, only the corresponding CMDQ allocation map entries are implemented. I will add a comment to make it explicit here.

Quoted 56 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)

again why don't we use the spec terminology
SMMU_CMDQV_VINTF0_CONFIG_0

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
                  \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 114

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)

Nit: the spec also contains the typo but I would rather fix it in the code

Explicitly state we only expose 1 VINTF

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 118

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+
+/* i = [0, 0], j = [0, 15] */

does that mean that we can have a max 16 SIDs per VM?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 119

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_VINTFi_SID_MATCH_(i, j)

if matched vi, I would prefer vi

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 126

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9 -0
                              \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */

vint = 0, 16 identical register entries

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 129

Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3 -0
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)

s/19/20

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 135

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */

vi = 0, 4 identical regs

Quoted 7 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 141

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */

I don't get this comment

the hardware defines four registers ([0..3]). They are handled using the range based switch case,

case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:

And the index to arary is calculated using:
i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4

I will reword the comment something like:

Only the first and last offsets are declared explicitly since the intermediate registers are handled via the range based switch case. The register index is derived as:
(offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4

Quoted 57 lines
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +2 -0
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+

const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);

#endif /* HW_ARM_TEGRA241_CMDQV_H */

hw/arm/tegra241-cmdqv.c +0 -0
index d487612ba2..a3830a02d6 100644

hw/arm/tegra241-cmdqv.c line 17

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -0
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,

hwaddr offset)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ...

A_VINTF0_LVCMDQ_ERR_MAP_3:

hw/arm/tegra241-cmdqv.c

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"

PRIx64 "\n",

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +5 -0
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,

unsigned size)

hw/arm/tegra241-cmdqv.c line 49

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +6 -1
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
Quoted 2 lines

If all those regs belong to the Global CMDQV registers page this shall be within the first 64KB

Yes, in this patch that is the case. However, subsequent patches in the series will add support for the other page windows as well. Hence the full MMIO range check here.

Quoted 31 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /

4];

hw/arm/tegra241-cmdqv.c

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"

PRIx64 "\n",

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr

offset,

Quoted 9 lines

hw/arm/tegra241-cmdqv.c line 85

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;

Can you explain why this needed?

The HYP_OWN bit is not guest controllable. It is owned by the host kernel/hypervisor and is wired to zero when running in guest kernel.

Please see tegra241_vintf_hw_init() in kernel,
drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c

The guest cannot enable this bit and the effective value seen by the guest remains zero.

Quoted 65 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

hw/arm/tegra241-cmdqv.c line 107

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +3 -0
+                      __func__, offset);
+        return;
+    }
 }

nit: if you can put write_vintf just after its read fellow, I think it would ease the comparison/review

hw/arm/tegra241-cmdqv.c line 108

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +0 -0
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t

value,

Quoted 15 lines
hw/arm/tegra241-cmdqv.c +13 -0
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =

value;

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +2 -0
+                      __func__, offset);
+    }
 }
 
Quoted 8 lines

hw/arm/tegra241-cmdqv.c line 145

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

At this stage of the reading it is unclear to me why we need to expose all those regs to the guest. I would assume that a subset of them are used by the hyp and not supposed to be accessed by the guest. For instance the SID match/replace, ...

Please could you clairfy what is absolutely needed to expose to the guest?

The intention here is to model the architectural register interface defined by the CMDQV specification so the guest visible MMIO space matches the hardware.

And some of the registers are backed by a simple register cache(like the SID match/replace). Access to the actual hardware state remains controlled by QEMU, so guest writes do not necessarily translate to direct hardware programming.

Thanks,
Shameer

#33 lore raw

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

Quoted 321 lines · Shameer Kolothum Thodi

Hi Eric,

Quoted 112 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 16:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

Quoted 4 lines
---
 hw/arm/tegra241-cmdqv.h | 108

+++++++++++++++++++++++++++++++++++

Quoted 1 line
 hw/arm/tegra241-cmdqv.c | 121

+++++++++++++++++++++++++++++++++++++++-

Quoted 7 lines
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +0 -0
index 46aa9e8a9f..50bcecee9d 100644

hw/arm/tegra241-cmdqv.h line 18

Quoted 9 lines
hw/arm/tegra241-cmdqv.h +3 -0
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

Quoted 22 lines
hw/arm/tegra241-cmdqv.h +16 -0
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 64

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)

this is called CMDQV_NUM_VI_LOG2 in the spec I have access to VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naming

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 65

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)

same here s/VM/VI

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 69

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+

I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 70

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_VI_ERR_MAP 0x14

_0?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 74

Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4 -0
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+

same SMMU_CMDQV_VI_ING_MASK0/1

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 79

Quoted 5 lines
hw/arm/tegra241-cmdqv.h +5 -0
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+

SMMU_CMDQV_CMDQ_ERR_MAP0-3

Quoted 11 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
+#define A_CMDQ_ALLOC_MAP_(i)

why A_? Since you rely on the REG macros, A_ should prefix the address offset
Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in the spec?

I will incorporate all suggestions related to naming etc and will make it same as used in the spec.

Quoted 7 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 91

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5 -0
              \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+

Please explain why we only expose 2 of those regs among 128?

QEMU currently models only two VCMDQs as we expose that by setting TEGRA241_CMDQV_NUM_CMDQ_LOG2 = 1 via SMMU_CMDQV_PARAM_0. Therefore, only the corresponding CMDQ allocation map entries are implemented. I will add a comment to make it explicit here.

Quoted 56 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)

again why don't we use the spec terminology
SMMU_CMDQV_VINTF0_CONFIG_0

Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7 -0
                  \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 114

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)

Nit: the spec also contains the typo but I would rather fix it in the code

Explicitly state we only expose 1 VINTF

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 118

Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2 -0
+
+/* i = [0, 0], j = [0, 15] */

does that mean that we can have a max 16 SIDs per VM?

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 119

Quoted 1 line
hw/arm/tegra241-cmdqv.h +1 -0
+#define A_VINTFi_SID_MATCH_(i, j)

if matched vi, I would prefer vi

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 126

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9 -0
                              \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */

vint = 0, 16 identical register entries

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 129

Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3 -0
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)

s/19/20

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 135

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */

vi = 0, 4 identical regs

Quoted 7 lines

hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 141

Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6 -0
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */

I don't get this comment

the hardware defines four registers ([0..3]). They are handled using the range based switch case,

case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:

And the index to arary is calculated using:
i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4

I will reword the comment something like:

Only the first and last offsets are declared explicitly since the intermediate registers are handled via the range based switch case. The register index is derived as:

   (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4
Quoted 57 lines
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +2 -0
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+

const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);

#endif /* HW_ARM_TEGRA241_CMDQV_H */

hw/arm/tegra241-cmdqv.c +0 -0
index d487612ba2..a3830a02d6 100644

hw/arm/tegra241-cmdqv.c line 17

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -0
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,

hwaddr offset)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5 -0
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ...

A_VINTF0_LVCMDQ_ERR_MAP_3:

hw/arm/tegra241-cmdqv.c

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"

PRIx64 "\n",

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +5 -0
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,

unsigned size)

hw/arm/tegra241-cmdqv.c line 49

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +6 -1
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
Quoted 2 lines

If all those regs belong to the Global CMDQV registers page this shall be within the first 64KB

Yes, in this patch that is the case. However, subsequent patches in the series will add support for the other page windows as well. Hence the full MMIO range check here.

Quoted 31 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /

4];

hw/arm/tegra241-cmdqv.c

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"

PRIx64 "\n",

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr

offset,

Quoted 9 lines

hw/arm/tegra241-cmdqv.c line 85

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8 -0
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;

Can you explain why this needed?

Quoted 8 lines

The HYP_OWN bit is not guest controllable. It is owned by the host kernel/hypervisor and is wired to zero when running in guest kernel.

Please see tegra241_vintf_hw_init() in kernel,
drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c

The guest cannot enable this bit and the effective value seen by the guest remains zero.

Well nothing in the spec tells you you cannot set this bit from a guest.

When you allocate a vcmdq for passthrough to a VM you need to set that bit to 0 for additional checks to be done. When host is asked to prepare the vcmdq for assignment, on tegra241_cmdqv_init_vintf_user(), hyp_owned is set to false.

However in usual tegra241_cmdqv_hw_reset() call, hyp_own is set to true. So I expect the guest to set the bit.

What do I miss?

Eric

Quoted 83 lines
Quoted 65 lines
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +8 -0
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

hw/arm/tegra241-cmdqv.c line 107

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +3 -0
+                      __func__, offset);
+        return;
+    }
 }

nit: if you can put write_vintf just after its read fellow, I think it would ease the comparison/review

hw/arm/tegra241-cmdqv.c line 108

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t

value,

Quoted 15 lines
hw/arm/tegra241-cmdqv.c +13 -0
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =

value;

hw/arm/tegra241-cmdqv.c

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +2 -0
+                      __func__, offset);
+    }
 }
 
Quoted 7 lines

hw/arm/tegra241-cmdqv.c line 145

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

At this stage of the reading it is unclear to me why we need to expose all those regs to the guest. I would assume that a subset of them are used by the hyp and not supposed to be accessed by the guest. For instance the SID match/replace, ...

Please could you clairfy what is absolutely needed to expose to the guest?

The intention here is to model the architectural register interface defined by the CMDQV specification so the guest visible MMIO space matches the hardware.

And some of the registers are backed by a simple register cache(like the SID match/replace). Access to the actual hardware state remains controlled by QEMU, so guest writes do not necessarily translate to direct hardware programming.

Thanks,
Shameer

#34 lore raw

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

In reply to Shameer Kolothum
Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page

In the spec this is named CMDQ-V Config. This 64kB page contains CMDQ-V Config + VI-n Configs

Eric

Quoted 284 lines

(offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

---
 hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +108 -0
index 46aa9e8a9f..50bcecee9d 100644
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+#define A_VI_ERR_MAP 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
+#define A_CMDQ_ALLOC_MAP_(i)                 \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)                       \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_MATCH_(i, j)                               \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +120 -1
index d487612ba2..a3830a02d6 100644
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#35 lore raw

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

In reply to Shameer Kolothum
Quoted 281 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

---
 hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +108 -0
index 46aa9e8a9f..50bcecee9d 100644
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+#define A_VI_ERR_MAP 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
+#define A_CMDQ_ALLOC_MAP_(i)                 \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)                       \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_MATCH_(i, j)                               \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +113 -1
index d487612ba2..a3830a02d6 100644
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;

hw/arm/tegra241-cmdqv.c line 145

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Please could you add trace points for mmio access as this is generally done for SMMU access? See trace_smmuv3_read/write_mmio()

Eric

#36 lore raw

Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers

In reply to Shameer Kolothum

Hi Shameer,

Quoted 11 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.

Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.

This is called CMDQ-V Config within [CMDQV_BASE, CMDQV-CMDQ_BASE] in the spec.

Precise we only support a single VINTF0 and VINTF1-63 are not supported.

Quoted 141 lines
---
 hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++-
 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +108 -0
index 46aa9e8a9f..50bcecee9d 100644
@@ -10,6 +10,9 @@
 #ifndef HW_ARM_TEGRA241_CMDQV_H
 #define HW_ARM_TEGRA241_CMDQV_H
 
+#include "hw/core/registerfields.h"
+#include "smmuv3-accel.h"
+
 #define TEGRA241_CMDQV_VERSION             1
 #define TEGRA241_CMDQV_NUM_CMDQ_LOG2       1
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+
+    /* Register Cache */
+    uint32_t config;
+    uint32_t param;
+    uint32_t status;
+    uint32_t vi_err_map[2];
+    uint32_t vi_int_mask[2];
+    uint32_t cmdq_err_map[4];
+    uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vintf_config;
+    uint32_t vintf_status;
+    uint32_t vintf_sid_match[16];
+    uint32_t vintf_sid_replace[16];
+    uint32_t vintf_cmdq_err_map[4];
 } Tegra241CMDQV;
 
+/* Global CMDQV MMIO registers (offset 0x00000) */
+REG32(CONFIG, 0x0)
+FIELD(CONFIG, CMDQV_EN, 0, 1)
+FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3)
+FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
+FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
+FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
+
+REG32(PARAM, 0x4)
+FIELD(PARAM, CMDQV_VER, 0, 4)
+FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
+FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4)
+FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4)
+
+REG32(STATUS, 0x8)
+FIELD(STATUS, CMDQV_ENABLED, 0, 1)
+
+#define A_VI_ERR_MAP 0x14
+#define A_VI_ERR_MAP_1 0x18
+#define V_VI_ERR_MAP_NO_ERROR (0)
+#define V_VI_ERR_MAP_ERROR (1)
+
+#define A_VI_INT_MASK 0x1c
+#define A_VI_INT_MASK_1 0x20
+#define V_VI_INT_MASK_NOT_MASKED (0)
+#define V_VI_INT_MASK_MASKED (1)
+
+#define A_CMDQ_ERR_MAP 0x24
+#define A_CMDQ_ERR_MAP_1 0x28
+#define A_CMDQ_ERR_MAP_2 0x2c
+#define A_CMDQ_ERR_MAP_3 0x30
+
+/* i = [0, 1] */
+#define A_CMDQ_ALLOC_MAP_(i)                 \
+    REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \
+    FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1)   \
+    FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7)  \
+    FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6)
+
+A_CMDQ_ALLOC_MAP_(0)
+A_CMDQ_ALLOC_MAP_(1)
+
+
+/* i = [0, 0] */
+#define A_VINTFi_CONFIG(i)                       \
+    REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \
+    FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1)       \
+    FIELD(VINTF##i##_CONFIG, VMID, 1, 16)        \
+    FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1)
+
+A_VINTFi_CONFIG(0)
+
+#define A_VINTFi_STATUS(i)                       \
+    REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \
+    FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1)    \
+    FIELD(VINTF##i##_STATUS, STATUS, 1, 3)       \
+    FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8)
+
+A_VINTFi_STATUS(0)
+
+#define V_VINTF_STATUS_NO_ERROR (0 << 1)
+#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_MATCH_(i, j)                               \
+    REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1)               \
+    FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20)
+
+A_VINTFi_SID_MATCH_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_MATCH_(0, 15)
+
+/* i = [0, 0], j = [0, 15] */
+#define A_VINTFi_SID_REPLACE_(i, j)                               \
+    REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
+
+A_VINTFi_SID_REPLACE_(0, 0)
+/* Omitting [0][1~14] as not being directly called */
+A_VINTFi_SID_REPLACE_(0, 15)
+
+/* i = [0, 0], j = [0, 3] */
+#define A_VINTFi_LVCMDQ_ERR_MAP_(i, j)                               \
+    REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \
+    FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32)
+
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
+/* Omitting [0][1~2] as not being directly called */
+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index d487612ba2..a3830a02d6 100644

hw/arm/tegra241-cmdqv.c line 17

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -0
@@ -8,19 +8,138 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/log.h"
 
 #include "hw/arm/smmuv3.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)

since it belong to the Config region, can we normalize the names

tegra241_cmdqv_config_vintf_read()

Quoted 15 lines
hw/arm/tegra241-cmdqv.c +15 -0
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:
+        return cmdqv->vintf_config;
+    case A_VINTF0_STATUS:
+        return cmdqv->vintf_status;
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        return cmdqv->vintf_sid_match[i];
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        return cmdqv->vintf_sid_replace[i];
+    case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:

hw/arm/tegra241-cmdqv.c line 42

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
+        i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4;
+        return cmdqv->vintf_cmdq_err_map[i];
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

tegra241_cmdqv_config_read
and same for the rest

Quoted 42 lines
hw/arm/tegra241-cmdqv.c +40 -1
 {
-    return 0;
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return 0;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        return cmdqv->config;
+    case A_PARAM:
+        return cmdqv->param;
+    case A_STATUS:
+        return cmdqv->status;
+    case A_VI_ERR_MAP ... A_VI_ERR_MAP_1:
+        return cmdqv->vi_err_map[(offset - A_VI_ERR_MAP) / 4];
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        return cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4];
+    case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3:
+        return cmdqv->cmdq_err_map[(offset - A_CMDQ_ERR_MAP) / 4];
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return 0;
+    }
+}
+
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
+                                       uint64_t value)
+{
+    int i;
+
+    switch (offset) {
+    case A_VINTF0_CONFIG:

hw/arm/tegra241-cmdqv.c line 93

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        /* Strip off HYP_OWN setting from guest kernel */
+        value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;
+
+        cmdqv->vintf_config = value;
+        if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
+        } else {
+            cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
+        }
+        break;

what about the GLB_FILT_CFG_0 and FILT_DATA-0, at least document why you don't support it neither in R nor W

Quoted 42 lines
hw/arm/tegra241-cmdqv.c +37 -0
+    case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15:
+        i = (offset - A_VINTF0_SID_MATCH_0) / 4;
+        cmdqv->vintf_sid_match[i] = value;
+        break;
+    case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15:
+        i = (offset - A_VINTF0_SID_REPLACE_0) / 4;
+        cmdqv->vintf_sid_replace[i] = value;
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+        return;
+    }
 }
 
 static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
+    Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+
+    if (offset >= TEGRA241_CMDQV_IO_LEN) {
+        qemu_log_mask(LOG_UNIMP,
+                      "%s offset 0x%" PRIx64 " off limit (0x50000)\n", __func__,
+                      offset);
+        return;
+    }
+
+    switch (offset) {
+    case A_CONFIG:
+        cmdqv->config = value;
+        if (value & R_CONFIG_CMDQV_EN_MASK) {
+            cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
+        } else {
+            cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
+        }
+        break;
+    case A_VI_INT_MASK ... A_VI_INT_MASK_1:
+        cmdqv->vi_int_mask[(offset - A_VI_INT_MASK) / 4] = value;
+        break;
+    case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1:
+        cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
+        break;

hw/arm/tegra241-cmdqv.c line 145

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+    case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        break;
+    default:
+        qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)

Otherwise I reviewed the macros and they look good to me

Eric

#37 lore raw

PATCH v3 15/32 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:

-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.

VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.

---
 hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
 2 files changed, 255 insertions(+)
hw/arm/tegra241-cmdqv.h +178 -0
index 50bcecee9d..d379b8860c 100644
@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* Global CMDQV MMIO registers (offset 0x00000) */
@@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* Omitting [0][1~2] as not being directly called */
 A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
+ */
+#define A_VCMDQi_CONS_INDX(i)                       \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)                             \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)                             \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)
+
+#define A_VCMDQi_BASE_L(i)                       \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
+ */
+#define A_VI_VCMDQi_CONS_INDX(i)                       \
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +77 -0

95 lines not on this page · raw message

Signature · 1 line
2.43.0
#38 lore raw

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Quoted 16 lines · On Thu, 26 Feb 2026 10:50:39 +0000 Shameer Kolothum <[email protected]>…

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:

-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.

Quoted 6 lines

VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.

Hi Shameer,

As noted below there are a lot of repeats of 0x80 and the register window offsets in here. Maybe some defines would make things clearer?

Quoted 16 lines

static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)

 {
     int i;
hw/arm/tegra241-cmdqv.c +2 -0
@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
         return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 121

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        /*
+         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
+         * layout as the global VCMDQ registers, but are based at 0x30000
+         * instead of 0x10000.
+         *
+         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
+         * global VCMDQ offset, then fall through to reuse the common VCMDQ
+         * decoding logic below.
+         */
+        offset -= 0x20000;

There are a lot of repeated numeric values of offsets and sizes in here. I'm a bit in two minds about whether they are clearer as numbers or you should add a few more defines.

Jonathan

#39 lore raw

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

In reply to Shameer Kolothum

Shameer,

Quoted 5 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:

-Global VCMDQ registers at 0x10000/0x20000

Spec mentions CMDQV_CMDQ_BASE and CMDV_VI_CMDQ_BASE offset. You could refer to that here and in the code (See fig 3.1).
Also there are size macros that could be used too.

By the way why do you name them "Global". Is that a spec terminology?

What I am really missing overall is the guest pov. I think I understand the VMM is supposed to create some VIs, assign some VCMDQs to those VI (that the guest will be able to have direct mapping to), define which VMID, SIDs are going to use this VI/VCMDQS + conversion from vSID to pSID. But then? what does the guest sees and what does he needs to program.

Quoted 5 lines

-VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if he shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use VI0?

Thanks

Eric

Quoted 315 lines

Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.

VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.

---
 hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
 2 files changed, 255 insertions(+)
hw/arm/tegra241-cmdqv.h +178 -0
index 50bcecee9d..d379b8860c 100644
@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* Global CMDQV MMIO registers (offset 0x00000) */
@@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* Omitting [0][1~2] as not being directly called */
 A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
+ */
+#define A_VCMDQi_CONS_INDX(i)                       \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)                             \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)                             \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)
+
+#define A_VCMDQi_BASE_L(i)                       \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
+ */
+#define A_VI_VCMDQi_CONS_INDX(i)                       \
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +77 -0
index a3830a02d6..d2e6938e44 100644
@@ -14,6 +14,46 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 {
     int i;
@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
         return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
+         * layout as the global VCMDQ registers, but are based at 0x30000
+         * instead of 0x10000.
+         *
+         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
+         * global VCMDQ offset, then fall through to reuse the common VCMDQ
+         * decoding logic below.
+         */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
+         * occupies a 0x80-byte window starting at 0x10000.
+         *
+         * The MMIO offset is decoded to extract the VCMDQ index and normalized
+         * to the corresponding VCMDQ0_* register by subtracting index * 0x80.
+         *
+         * A single helper then services all VCMDQs, with @index selecting the
+         * instance.
+         */
+        index = (offset - 0x10000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
+        index = (offset - 0x20000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
                       __func__, offset);
#40 lore raw

RE: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Hi Eric,

Quoted 43 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 17:44
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

External email: Use caution opening links or attachments

Shameer,

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two

MMIO views:

Quoted 2 lines

-Global VCMDQ registers at 0x10000/0x20000

Spec mentions CMDQV_CMDQ_BASE and CMDV_VI_CMDQ_BASE offset. You could
refer to that here and in the code (See fig 3.1). Also there are size macros that could be used too.

By the way why do you name them "Global". Is that a spec terminology?

What I am really missing overall is the guest pov. I think I understand the VMM is supposed to create some VIs, assign some VCMDQs to those VI (that the guest will be able to have direct mapping to), define which VMID, SIDs are going to use this VI/VCMDQS + conversion from vSID to pSID. But then? what does the guest sees and what does he needs to program.

Quoted 2 lines

-VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

Quoted 8 lines
Quoted 3 lines

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if he
shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use VI0?

The CMDQ registers can be accessed via both the so called "Global VCMDQ registers Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.

The difference being(from Nicolin's explanation here[0]):

"
The global page0 is programmable at any time so long as CMDQV_EN is enabled.

The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".

IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".

The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.

@Nicolin, please correct me if that assumption is not right.

Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/aXkTzRM695oTWFaD@Asurada-Nvidia/

Quoted 336 lines

Thanks

Eric

Quoted 6 lines

Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.

Quoted 8 lines

VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.

---
 hw/arm/tegra241-cmdqv.h | 178

++++++++++++++++++++++++++++++++++++++++

Quoted 92 lines
 hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
 2 files changed, 255 insertions(+)
hw/arm/tegra241-cmdqv.h +74 -0
index 50bcecee9d..d379b8860c 100644
@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* Global CMDQV MMIO registers (offset 0x00000) */
@@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* Omitting [0][1~2] as not being directly called */
 A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
+ */
+#define A_VCMDQi_CONS_INDX(i)                       \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)                             \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)                             \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)
+
+#define A_VCMDQi_BASE_L(i)                       \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)

hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 227

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i *

0x80) \

hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i *

0x80) \

Quoted 65 lines
hw/arm/tegra241-cmdqv.h +65 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
+ */
+#define A_VI_VCMDQi_CONS_INDX(i)                       \
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)

hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i

* 0x80) \

hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)

Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7 -0
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i

* 0x80) \

Quoted 15 lines
hw/arm/tegra241-cmdqv.h +5 -0
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index a3830a02d6..d2e6938e44 100644
@@ -14,6 +14,46 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"

hw/arm/tegra241-cmdqv.c line 25

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,

hwaddr offset0,

Quoted 22 lines
hw/arm/tegra241-cmdqv.c +22 -0
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,

hwaddr offset)

hw/arm/tegra241-cmdqv.c static uint64_t

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 {
     int i;
@@ -42,6 +82,7 @@ static uint64_t

tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)

hw/arm/tegra241-cmdqv.c static uint64_t · line 82

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,

unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +1 -0
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 108

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /

4];

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8 -0
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
+         * layout as the global VCMDQ registers, but are based at 0x30000
+         * instead of 0x10000.
+         *
+         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
+         * global VCMDQ offset, then fall through to reuse the common VCMDQ

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+         * decoding logic below.
+         */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each

VCMDQ

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3 -0
+         * occupies a 0x80-byte window starting at 0x10000.
+         *
+         * The MMIO offset is decoded to extract the VCMDQ index and

normalized

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 132

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+         * to the corresponding VCMDQ0_* register by subtracting index *

0x80.

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+         *
+         * A single helper then services all VCMDQs, with @index selecting the
+         * instance.
+         */
+        index = (offset - 0x10000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80,

index);

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 139

Quoted 1 line
hw/arm/tegra241-cmdqv.c +1 -0
+    case A_VI_VCMDQ0_BASE_L ...

A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7 -0
+        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
+        index = (offset - 0x20000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80,

index);

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 148

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +0 -0
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"

PRIx64 "\n",

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
                       __func__, offset);
#41 lore raw

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Quoted 19 lines · Shameer Kolothum Thodi
Quoted 9 lines
Quoted 3 lines

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if he
shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use VI0?

The CMDQ registers can be accessed via both the so called "Global VCMDQ registers Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.

The difference being(from Nicolin's explanation here[0]):

"
The global page0 is programmable at any time so long as CMDQV_EN is enabled.

Quoted 7 lines

The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".

IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".

Yes.

Quoted 3 lines

The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.

Yes.

The point is that we can't reject guest's access to the global MMIO space. So, QEMU has to support both. But guest OS is allowed to do a partial map/alloc, so global MMIO page isn't identical to the VINTF logical page in this case.

In an extreme corner case, guest OS might try to enable a VCMDQ via the global MMIO space, though the vcmdq isn't allocated/mapped to a VINTF. QEMU should still allow these MMIOs but does not essentially make the vcmdq functional, since this would be "unpredictable" in a physical HW situation.

Maybe we should elaborate these in the commit message.

Nicolin

Quoted 1 line

@Nicolin, please correct me if that assumption is not right.

#42 lore raw

RE: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Quoted 54 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:41
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Quoted 6 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 3 lines

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the SMMU_CMDQ_PARAM_0) how does he

know if

Quoted 3 lines
Quoted 3 lines

he
shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use

VI0?

Quoted 2 lines

The CMDQ registers can be accessed via both the so called "Global VCMDQ

registers

Quoted 8 lines

Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.

The difference being(from Nicolin's explanation here[0]):

"
The global page0 is programmable at any time so long as CMDQV_EN is enabled.

Quoted 7 lines

The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".

IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".

Yes.

Quoted 3 lines

The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.

Quoted 6 lines

Yes.

The point is that we can't reject guest's access to the global MMIO space. So, QEMU has to support both. But guest OS is allowed to do a partial map/alloc, so global MMIO page isn't identical to the VINTF logical page in this case.

Are there any benefits for a guest implementation making use of the "Global VCMDQ registers Page 0"?

If not, the VINTF-based access path seems preferable from a hardware isolation point of view. In that case, would it make sense to restrict usage to the VINTF logical page and recommend that guests use that interface?

Thanks,
Shameer

#43 lore raw

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Quoted 61 lines · Shameer Kolothum Thodi
Quoted 54 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:41
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

Quoted 6 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 3 lines

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the SMMU_CMDQ_PARAM_0) how does he

know if

Quoted 3 lines
Quoted 3 lines

he
shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use

VI0?

Quoted 2 lines

The CMDQ registers can be accessed via both the so called "Global VCMDQ

registers

Quoted 8 lines

Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.

The difference being(from Nicolin's explanation here[0]):

"
The global page0 is programmable at any time so long as CMDQV_EN is enabled.

Quoted 7 lines

The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".

IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".

Yes.

Quoted 3 lines

The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.

Quoted 6 lines

Yes.

The point is that we can't reject guest's access to the global MMIO space. So, QEMU has to support both. But guest OS is allowed to do a partial map/alloc, so global MMIO page isn't identical to the VINTF logical page in this case.

Quoted 5 lines

Are there any benefits for a guest implementation making use of the "Global VCMDQ registers Page 0"?

If not, the VINTF-based access path seems preferable from a hardware isolation point of view.

Yea. That's why Linux was implemented in that way.

Quoted 3 lines

In that case, would it make sense to restrict
usage to the VINTF logical page and recommend that guests use that interface?

I don't think QEMU can recommend that... Linux kernel made a good example but it doesn't make a lot of sense to recommend or force all guest SWs to follow.

From hardware emulation perspective, QEMU should just re-act like a real HW. It's okay to make it unfunctional v.s. unpredictable. But it shouldn't reject any MMIO access, since the HW wouldn't.

Nicolin

#44 lore raw

Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads

In reply to Shameer Kolothum
Quoted 8 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:

-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000

The VI_VCMDQ register ranges are an alias of the global VCMDQ registers

I would recommend to use the same terminology anywhere, either VINTF or VI and stick to it.

Why do we need emulation for VI_VCMDQ. I thought this was mmapped?

Explicitly state we only support 2 VCMDQs

Quoted 125 lines

and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.

Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.

VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.

---
 hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/tegra241-cmdqv.c |  77 +++++++++++++++++
 2 files changed, 255 insertions(+)
hw/arm/tegra241-cmdqv.h +91 -0
index 50bcecee9d..d379b8860c 100644
@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {
     uint32_t vintf_sid_match[16];
     uint32_t vintf_sid_replace[16];
     uint32_t vintf_cmdq_err_map[4];
+    uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ];
+    uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ];
+    uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ];
 } Tegra241CMDQV;
 
 /* Global CMDQV MMIO registers (offset 0x00000) */
@@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
 /* Omitting [0][1~2] as not being directly called */
 A_VINTFi_LVCMDQ_ERR_MAP_(0, 3)
 
+/*
+ * VCMDQ register windows.
+ *
+ * Page 0 @ 0x10000: VCMDQ control and status registers
+ * Page 1 @ 0x20000: VCMDQ base and DRAM address registers
+ */
+#define A_VCMDQi_CONS_INDX(i)                       \
+    REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VCMDQi_CONS_INDX(0)
+A_VCMDQi_CONS_INDX(1)
+
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3
+#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4
+
+#define A_VCMDQi_PROD_INDX(i)                             \
+    REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VCMDQi_PROD_INDX(0)
+A_VCMDQi_PROD_INDX(1)
+
+#define A_VCMDQi_CONFIG(i)                             \
+    REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VCMDQi_CONFIG(0)
+A_VCMDQi_CONFIG(1)
+
+#define A_VCMDQi_STATUS(i)                             \
+    REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \
+    FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VCMDQi_STATUS(0)
+A_VCMDQi_STATUS(1)
+
+#define A_VCMDQi_GERROR(i)                               \
+    REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERROR(0)
+A_VCMDQi_GERROR(1)
+
+#define A_VCMDQi_GERRORN(i)                               \
+    REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80)  \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VCMDQi_GERRORN(0)
+A_VCMDQi_GERRORN(1)
+
+#define A_VCMDQi_BASE_L(i)                       \
+    REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VCMDQi_BASE_L(0)
+A_VCMDQi_BASE_L(1)
+
+#define A_VCMDQi_BASE_H(i)                             \
+    REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \
+    FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VCMDQi_BASE_H(0)
+A_VCMDQi_BASE_H(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \

hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 244

Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10 -0
+    FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
+/*
+ * VI_VCMDQ register windows (VCMDQs mapped via VINTF).
+ *
+ * Page 0 @ 0x30000: VI_VCMDQ control and status registers
+ * Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers

I don't get why it is needed.

Quoted 172 lines
hw/arm/tegra241-cmdqv.h +77 -0
+ */
+#define A_VI_VCMDQi_CONS_INDX(i)                       \
+    REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20)          \
+    FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7)
+
+A_VI_VCMDQi_CONS_INDX(0)
+A_VI_VCMDQi_CONS_INDX(1)
+
+#define A_VI_VCMDQi_PROD_INDX(i)                             \
+    REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20)
+
+A_VI_VCMDQi_PROD_INDX(0)
+A_VI_VCMDQi_PROD_INDX(1)
+
+#define A_VI_VCMDQi_CONFIG(i)                             \
+    REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1)
+
+A_VI_VCMDQi_CONFIG(0)
+A_VI_VCMDQi_CONFIG(1)
+
+#define A_VI_VCMDQi_STATUS(i)                             \
+    REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1)
+
+A_VI_VCMDQi_STATUS(0)
+A_VI_VCMDQi_STATUS(1)
+
+#define A_VI_VCMDQi_GERROR(i)                               \
+    REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERROR(0)
+A_VI_VCMDQi_GERROR(1)
+
+#define A_VI_VCMDQi_GERRORN(i)                               \
+    REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80)  \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1)             \
+    FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \
+    FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1)
+
+A_VI_VCMDQi_GERRORN(0)
+A_VI_VCMDQi_GERRORN(1)
+
+#define A_VI_VCMDQi_BASE_L(i)                       \
+    REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5)     \
+    FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27)
+
+A_VI_VCMDQi_BASE_L(0)
+A_VI_VCMDQi_BASE_L(1)
+
+#define A_VI_VCMDQi_BASE_H(i)                             \
+    REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_BASE_H(0)
+A_VI_VCMDQi_BASE_H(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
+
+#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(i)                             \
+    REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \
+    FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16)
+
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
+A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +70 -0
index a3830a02d6..d2e6938e44 100644
@@ -14,6 +14,46 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+/*
+ * Read a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * All VCMDQ accesses return cached registers.
+ */
+static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
+                                          int index)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        return cmdqv->vcmdq_cons_indx[index];
+    case A_VCMDQ0_PROD_INDX:
+        return cmdqv->vcmdq_prod_indx[index];
+    case A_VCMDQ0_CONFIG:
+        return cmdqv->vcmdq_config[index];
+    case A_VCMDQ0_STATUS:
+        return cmdqv->vcmdq_status[index];
+    case A_VCMDQ0_GERROR:
+        return cmdqv->vcmdq_gerror[index];
+    case A_VCMDQ0_GERRORN:
+        return cmdqv->vcmdq_gerrorn[index];
+    case A_VCMDQ0_BASE_L:
+        return cmdqv->vcmdq_base[index];
+    case A_VCMDQ0_BASE_H:
+        return cmdqv->vcmdq_base[index] >> 32;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        return cmdqv->vcmdq_cons_indx_base[index];
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        return cmdqv->vcmdq_cons_indx_base[index] >> 32;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled read access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return 0;
+    }
+}
+
 static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 {
     int i;
@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)
 static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
         return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4];
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         return tegra241_cmdqv_read_vintf(cmdqv, offset);
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /*
+         * VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ
+         * layout as the global VCMDQ registers, but are based at 0x30000
+         * instead of 0x10000.
+         *
+         * Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent
+         * global VCMDQ offset, then fall through to reuse the common VCMDQ
+         * decoding logic below.
+         */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        /*
+         * Decode a per-VCMDQ register access.
+         *
+         * The hardware supports up to 128 identical VCMDQ instances; we
+         * currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
+         * occupies a 0x80-byte window starting at 0x10000.
+         *
+         * The MMIO offset is decoded to extract the VCMDQ index and normalized
+         * to the corresponding VCMDQ0_* register by subtracting index * 0x80.
+         *
+         * A single helper then services all VCMDQs, with @index selecting the
+         * instance.
+         */
+        index = (offset - 0x10000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 149

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7 -0
+        /* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decode logic as A_VCMDQx_CONS_INDX case above */
+        index = (offset - 0x20000) / 0x80;
+        return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80, index);
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%" PRIx64 "\n",
                       __func__, offset);

Thanks

Eric

#45 lore raw

PATCH v3 16/32 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation.

Add write handling for global VCMDQ and VI_VCMDQ register windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.

VI_VCMDQ registers are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.

Writes are backed by cached register state only; no hardware queue mapping is performed yet.

---
 hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
hw/arm/tegra241-cmdqv.c +83 -0
index d2e6938e44..e1f1562c44 100644
@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void
+tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
+                           uint64_t value, unsigned size)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        } else {
+            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_BASE_H:
+        cmdqv->vcmdq_base[index] =
+            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        if (size == 8) {
+            cmdqv->vcmdq_cons_indx_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_cons_indx_base[index] =
+                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        cmdqv->vcmdq_cons_indx_base[index] =
+            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return;
+    }
+}
+
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
                                        uint64_t value)
 {
@@ -187,6 +251,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -213,6 +278,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         tegra241_cmdqv_write_vintf(cmdqv, offset, value);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        index = (offset - 0x10000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        index = (offset - 0x20000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
Signature · 1 line
2.43.0
#46 lore raw

Re: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

Quoted 54 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation.

Add write handling for global VCMDQ and VI_VCMDQ register windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.

VI_VCMDQ registers are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.

Writes are backed by cached register state only; no hardware queue mapping is performed yet.

---
 hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
hw/arm/tegra241-cmdqv.c +26 -0
index d2e6938e44..e1f1562c44 100644
@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void
+tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
+                           uint64_t value, unsigned size)
+{
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |= R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        } else {
+            cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }

If we compare to smmuv3 std command queue API, we miss some checks. See smmu_writel()
        if (!smmu_cmdq_base_writable(s, reg_sec_sid)) {
for CONS
        if (!smmu_cmdq_disabled_stable(s, reg_sec_sid)) {

In the spec I have access to I failed to find any details about equivalent restrictions

Can you check if those write accesses are always allowed?

Thanks

Eric

Quoted 64 lines
hw/arm/tegra241-cmdqv.c +47 -0
+        return;
+    case A_VCMDQ0_BASE_H:
+        cmdqv->vcmdq_base[index] =
+            (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
+        if (size == 8) {
+            cmdqv->vcmdq_cons_indx_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_cons_indx_base[index] =
+                (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }
+        return;
+    case A_VCMDQ0_CONS_INDX_BASE_DRAM_H:
+        cmdqv->vcmdq_cons_indx_base[index] =
+            (cmdqv->vcmdq_cons_indx_base[index] & 0xffffffffULL) |
+            ((uint64_t)value << 32);
+        return;
+    default:
+        qemu_log_mask(LOG_UNIMP,
+                      "%s unhandled write access at 0x%" PRIx64 "\n",
+                      __func__, offset0);
+        return;
+    }
+}
+
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
                                        uint64_t value)
 {
@@ -187,6 +251,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
         qemu_log_mask(LOG_UNIMP,
@@ -213,6 +278,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
         tegra241_cmdqv_write_vintf(cmdqv, offset, value);
         break;
+    case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
+        index = (offset - 0x10000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
+    case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        /* Same decoding as read() case: See comments above */
+        offset -= 0x20000;
+        QEMU_FALLTHROUGH;
+    case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
+        index = (offset - 0x20000) / 0x80;
+        tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
+                                   size);
+        break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
#47 lore raw

RE: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

Hi Eric,

Quoted 93 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:32
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes

External email: Use caution opening links or attachments

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 26 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

This is the write side counterpart of the VCMDQ read emulation.

Add write handling for global VCMDQ and VI_VCMDQ register windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.

VI_VCMDQ registers are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.

Writes are backed by cached register state only; no hardware queue mapping is performed yet.

---
 hw/arm/tegra241-cmdqv.c | 83
+++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)
hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0 -0

d2e6938e44..e1f1562c44 100644

hw/arm/tegra241-cmdqv.c +0 -0
@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 12 lines
hw/arm/tegra241-cmdqv.c +9 -0
     }
 }
 
+/*
+ * Write a VCMDQ register using VCMDQ0_* offsets.
+ *
+ * The caller normalizes the MMIO offset such that @offset0 always
+refers
+ * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ */
+static void
+tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,

int index,

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +11 -0
+                           uint64_t value, unsigned size) {
+    switch (offset0) {
+    case A_VCMDQ0_CONS_INDX:
+        cmdqv->vcmdq_cons_indx[index] = value;
+        return;
+    case A_VCMDQ0_PROD_INDX:
+        cmdqv->vcmdq_prod_indx[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_CONFIG:
+        if (value & R_VCMDQ0_CONFIG_CMDQ_EN_MASK) {
+            cmdqv->vcmdq_status[index] |=

R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 175

Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2 -0
+        } else {
+            cmdqv->vcmdq_status[index] &=

~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+        }
+        cmdqv->vcmdq_config[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_GERRORN:

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+        cmdqv->vcmdq_gerrorn[index] = (uint32_t)value;
+        return;
+    case A_VCMDQ0_BASE_L:
+        if (size == 8) {
+            cmdqv->vcmdq_base[index] = value;
+        } else if (size == 4) {
+            cmdqv->vcmdq_base[index] =
+                (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
+                (value & 0xffffffffULL);
+        }

If we compare to smmuv3 std command queue API, we miss some checks. See
smmu_writel()
if (!smmu_cmdq_base_writable(s, reg_sec_sid)) { for CONS if (!smmu_cmdq_disabled_stable(s, reg_sec_sid)) {

Quoted 4 lines

In the spec I have access to I failed to find any details about equivalent restrictions

Can you check if those write accesses are always allowed?

I couldn't find any explicit restriction in the spec either. The spec details the "Enabling the Virtual CMDQ" sequence as:

- Program the VIRT_CMDQ_BASE register
- Initialize CMDQ_ registers
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ

From this, looks like, the expected behaviour is that BASE is programmed only when the VCMDQ is in a disabled state.

We can add a defensive check by reading VCMDQ_CONFIG directly from vintf_ptr before allowing a BASE write. If CMDQ_EN is set, we reject the write. This will ensure Guest has disabled the VCMDQ before reprogramming BASE register.

For v4(that I just send out), this part is unchanged. If required, we can add the above check in the next respin.

Thanks,
Shameer

#48 lore raw

PATCH v3 17/32 hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +42 -2

82 lines not on this page · raw message

Signature · 1 line
2.43.0
#49 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 4 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real

Again I fail to understand the guest POV. If the goal is to let the guest have a direct access to some vcmdqs why do we need to emulate the CMDQ regs?

Eric

Quoted 132 lines

hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +42 -2
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
 
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#50 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

In reply to Shameer Kolothum
Quoted 7 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

Quoted 8 lines

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How do we transfer the state of the cmdq from one to the other?

Thanks

Eric

Quoted 121 lines
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +42 -2
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
 
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#51 lore raw

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 32 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

Quoted 6 lines
Quoted 3 lines

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How do we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.

So when the mapping between a global VCMDQ and a logical VCMDQ is created through the HW_QUEUE ioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.

I will double check this.

Thanks,
Shameer

Quoted 138 lines

Thanks

Eric

Quoted 6 lines
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.h line 18

Quoted 8 lines
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +0 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3 -0
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2 -0
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error

**errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV

*cmdqv, Error **errp)

hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV

Quoted 22 lines
hw/arm/tegra241-cmdqv.c +17 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
 viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.

*

hw/arm/tegra241-cmdqv.c +0 -0
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV

*cmdqv, hwaddr offset0, int index,

hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV · line 251

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr

offset,

hw/arm/tegra241-cmdqv.c static void

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1 -1
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void

tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +2 -0
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =

value;

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -1
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 336

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4 -0
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#52 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 34 lines · Shameer Kolothum Thodi
Quoted 34 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 15 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

Quoted 7 lines
Quoted 3 lines

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How do we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.

Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of smmuv3_cmdq_consume()), no?
I don't see it done at the moment.

By the way, do we want support of VCMDQs in fully emulated mode. I guess it would be sensible?

Thanks

Eric

Quoted 146 lines

So when the mapping between a global VCMDQ and a logical VCMDQ is created through the HW_QUEUE ioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.

I will double check this.

Thanks,
Shameer

Quoted 136 lines

Thanks

Eric

Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.h line 18

Quoted 8 lines
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +0 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3 -0
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2 -0
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error

**errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV

*cmdqv, Error **errp)

hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV

Quoted 22 lines
hw/arm/tegra241-cmdqv.c +17 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
 viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.

*

hw/arm/tegra241-cmdqv.c +0 -0
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV

*cmdqv, hwaddr offset0, int index,

hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV · line 251

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr

offset,

hw/arm/tegra241-cmdqv.c static void

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1 -1
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void

tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -0
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =

value;

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -1
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 336

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4 -0
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#53 lore raw

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 64 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF

Page0

Quoted 18 lines
Quoted 18 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How

do

Quoted 5 lines
Quoted 1 line

we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.

Quoted 7 lines

Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of smmuv3_cmdq_consume()), no?
I don't see it done at the moment.

By the way, do we want support of VCMDQs in fully emulated mode. I guess it would be sensible?

Yes, command handling is not implemented now.

Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then forward the supported invalidation commands back to the host again.

The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.

Thanks,
Shameer

Quoted 153 lines

Thanks

Eric

Quoted 49 lines

So when the mapping between a global VCMDQ and a logical VCMDQ is created through the HW_QUEUE ioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.

I will double check this.

Thanks,
Shameer

Quoted 39 lines

Thanks

Eric

Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.h line 18

Quoted 8 lines
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +0 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3 -0
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

Quoted 5 lines
Quoted 5 lines

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2 -0
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv,

Error

Quoted 30 lines
Quoted 30 lines

**errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV

*cmdqv, Error **errp)

Quoted 14 lines
hw/arm/tegra241-cmdqv.c +13 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
 viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
Quoted 8 lines
Quoted 8 lines

hw/arm/tegra241-cmdqv.c

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +4 -0
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.

*

hw/arm/tegra241-cmdqv.c +0 -0
@@ -216,7 +249,7 @@

tegra241_cmdqv_write_vcmdq(Tegra241CMDQV

Quoted 4 lines
Quoted 4 lines

*cmdqv, hwaddr offset0, int index,

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv,

hwaddr

Quoted 30 lines
Quoted 30 lines

offset,

hw/arm/tegra241-cmdqv.c static void

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1 -1
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void

tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -0
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =

value;

Quoted 8 lines
Quoted 8 lines

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -1
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void

*opaque,

Quoted 12 lines
Quoted 12 lines

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void · line 336

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4 -0
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#54 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 70 lines · Shameer Kolothum Thodi
Quoted 61 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 4 lines · Shameer Kolothum Thodi
Quoted 4 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu-

Quoted 8 lines
Quoted 8 lines

[email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF

Page0

Quoted 18 lines
Quoted 18 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 12 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How

do

Quoted 5 lines
Quoted 1 line

we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.

Quoted 7 lines

Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of smmuv3_cmdq_consume()), no?
I don't see it done at the moment.

By the way, do we want support of VCMDQs in fully emulated mode. I guess it would be sensible?

Yes, command handling is not implemented now.

Quoted 7 lines

Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then forward the supported invalidation commands back to the host again.

The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.

OK but really the whole user model is really confusing. One starts using a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are not. Also when the guest is using the global vcmdq, it can access the registers but they actually do not trigger any commands. This really needs to be clarified.

Then I understand we want to focus on accel mode but above points need to be clatified. I just wanted to mention that vcmdq should logically also work in fully emulated mode (and you seemed to implement a mix actually when you have emulated access to vi vcmdq) as it replaces the SMMU standard cmdq, if I understand correctly

Eric

Quoted 156 lines

Thanks,
Shameer

Quoted 152 lines

Thanks

Eric

Quoted 48 lines

So when the mapping between a global VCMDQ and a logical VCMDQ is created through the HW_QUEUE ioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.

I will double check this.

Thanks,
Shameer

Quoted 39 lines

Thanks

Eric

Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.h line 18

Quoted 8 lines
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +0 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3 -0
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

Quoted 5 lines
Quoted 5 lines

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2 -0
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv,

Error

Quoted 30 lines
Quoted 30 lines

**errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV

*cmdqv, Error **errp)

Quoted 14 lines
hw/arm/tegra241-cmdqv.c +13 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
 viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
Quoted 8 lines
Quoted 8 lines

hw/arm/tegra241-cmdqv.c

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +4 -0
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.

*

hw/arm/tegra241-cmdqv.c +0 -0
@@ -216,7 +249,7 @@

tegra241_cmdqv_write_vcmdq(Tegra241CMDQV

Quoted 4 lines
Quoted 4 lines

*cmdqv, hwaddr offset0, int index,

Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0 -0
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv,

hwaddr

Quoted 30 lines
Quoted 30 lines

offset,

hw/arm/tegra241-cmdqv.c static void

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1 -1
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void

tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -0
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -0
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =

value;

Quoted 8 lines
Quoted 8 lines

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void

Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1 -1
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void

*opaque,

Quoted 12 lines
Quoted 12 lines

hwaddr offset, uint64_t value,

hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void · line 336

Quoted 1 line
hw/arm/tegra241-cmdqv.c +0 -0
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"

PRIx64 "\n",

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4 -0
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#55 lore raw

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 107 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:19
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF

Page0

Quoted 5 lines
Quoted 5 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 7 lines
Quoted 7 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 6 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];

Nicolin

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Chen <[email protected]>; Nathan Chen <[email protected]>;

Matt

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Ochs <[email protected]>; Jiandi An <[email protected]>; Jason

Gunthorpe

Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines

<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF

Page0

Quoted 8 lines
Quoted 8 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 2 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association

between

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

VCMDQs and a VINTF is only established after HW_QUEUE allocation.

Prior

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page.

How

Quoted 6 lines
Quoted 6 lines

do

Quoted 5 lines
Quoted 1 line

we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.

Quoted 6 lines
Quoted 6 lines

Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of smmuv3_cmdq_consume()), no?
I don't see it done at the moment.

By the way, do we want support of VCMDQs in fully emulated mode. I

guess

Quoted 5 lines
Quoted 1 line

it would be sensible?

Yes, command handling is not implemented now.

Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then

forward

Quoted 5 lines

the supported invalidation commands back to the host again.

The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.

Quoted 6 lines

OK but really the whole user model is really confusing. One starts using a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are not. Also when the guest is using the global vcmdq, it can access the registers but they actually do not trigger any commands. This really needs to be clarified.

Yes, this could use better documentation. I can clarify the model in the commit message and code comments.

One option we discussed earlier to avoid the confusion was to restrict usage to the "VINTF logical VCMDQ" interface and return an error if the guest accesses the "Global VCMDQ registers".

However, since this is still MMIO space, the thinking was that it may be better to allow the access even if it is effectively non-functional, rather than making the behaviour unpredictable.

See the discussion:
https://lore.kernel.org/qemu-devel/aa8cJszkOKGLUYdD@Asurada-Nvidia/

Quoted 5 lines

Then I understand we want to focus on accel mode but above points need to be clatified. I just wanted to mention that vcmdq should logically also work in fully emulated mode (and you seemed to implement a mix actually when you have emulated access to vi vcmdq) as it replaces the SMMU standard cmdq, if I understand correctly

IIUC, in the current Linux implementation VCMDQ does not replace the standard SMMU CMDQ from the guest perspective. This is also why we do not expose the HYP_OWN bit to the guest.

When HYP_OWN is not set, guest still uses the normal SMMU CMDQ. CMDQ is only used as a fast-path for a small set of invalidation commands that can be sent directly to the host hardware.

See:

static bool tegra241_guest_vcmdq_supports_cmd(struct arm_smmu_cmdq_ent *ent)

{
        switch (ent->opcode) {
        case CMDQ_OP_TLBI_NH_ASID:
        case CMDQ_OP_TLBI_NH_VA:
        case CMDQ_OP_ATC_INV:
                return true;
        default:
                return false;
        }
}

So VCMDQ is not a replacement for the SMMU CMDQ in the guest. It is only used to accelerate these invalidation commands.

Thanks,
Shameer

A message that is not in the archive (sent off-list or not yet received): [email protected]

#57 lore raw

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 191 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 March 2026 15:11
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:19
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF

Page0

Quoted 5 lines
Quoted 5 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 7 lines
Quoted 7 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 6 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];

Nicolin

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Chen <[email protected]>; Nathan Chen <[email protected]>;

Matt

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Ochs <[email protected]>; Jiandi An <[email protected]>; Jason

Gunthorpe

Quoted 17 lines
Quoted 17 lines
Quoted 4 lines
Quoted 4 lines

<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF

Page0

Quoted 12 lines
Quoted 12 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 6 lines · Shameer Kolothum Thodi
Quoted 6 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];

Quoted 7 lines
Quoted 7 lines

Nicolin

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Chen <[email protected]>; Nathan Chen <[email protected]>;

Matt

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Ochs <[email protected]>; Jiandi An <[email protected]>; Jason

Gunthorpe

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant

Jaju

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

<[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap

VINTF

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

Page0

Quoted 9 lines
Quoted 9 lines

for CMDQV

External email: Use caution opening links or attachments

Quoted 3 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while

the

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes

a

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region

during

Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

vIOMMU allocation. However, the logical-to-physical association

between

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

VCMDQs and a VINTF is only established after HW_QUEUE allocation.

Prior

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation

support.

Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page.

How

Quoted 7 lines
Quoted 7 lines

do

Quoted 5 lines
Quoted 1 line

we transfer the state of the cmdq from one to the other?

Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.

Also assuming the global VCMDQs are accessible and thus used, I guess

we

Quoted 49 lines
Quoted 17 lines
Quoted 5 lines
Quoted 5 lines

shall properly handle the actual commands (equivalent of smmuv3_cmdq_consume()), no?
I don't see it done at the moment.

By the way, do we want support of VCMDQs in fully emulated mode. I

guess

Quoted 5 lines
Quoted 1 line

it would be sensible?

Yes, command handling is not implemented now.

Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then

forward

Quoted 5 lines

the supported invalidation commands back to the host again.

The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.

Quoted 6 lines

OK but really the whole user model is really confusing. One starts using a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are not. Also when the guest is using the global vcmdq, it can access the registers but they actually do not trigger any commands. This really needs to be clarified.

Yes, this could use better documentation. I can clarify the model in the commit message and code comments.

One option we discussed earlier to avoid the confusion was to restrict usage to the "VINTF logical VCMDQ" interface and return an error if the guest accesses the "Global VCMDQ registers".

However, since this is still MMIO space, the thinking was that it may be better to allow the access even if it is effectively non-functional, rather than making the behaviour unpredictable.

See the discussion:
https://lore.kernel.org/qemu-devel/aa8cJszkOKGLUYdD@Asurada-Nvidia/

Quoted 5 lines

Then I understand we want to focus on accel mode but above points need to be clatified. I just wanted to mention that vcmdq should logically also work in fully emulated mode (and you seemed to implement a mix actually when you have emulated access to vi vcmdq) as it replaces the SMMU standard cmdq, if I understand correctly

IIUC, in the current Linux implementation VCMDQ does not replace the standard SMMU CMDQ from the guest perspective. This is also why we do not expose the HYP_OWN bit to the guest.
When HYP_OWN is not set, guest still uses the normal SMMU CMDQ. CMDQ is only used as a fast-path for a small set of invalidation commands that can be sent directly to the host hardware.

Quoted 3 lines

See:

static bool tegra241_guest_vcmdq_supports_cmd(struct

arm_smmu_cmdq_ent *ent)

Quoted 11 lines
{
        switch (ent->opcode) {
        case CMDQ_OP_TLBI_NH_ASID:
        case CMDQ_OP_TLBI_NH_VA:
        case CMDQ_OP_ATC_INV:
                return true;
        default:
                return false;
        }
}
Quoted 2 lines

So VCMDQ is not a replacement for the SMMU CMDQ in the guest. It is only used to accelerate these invalidation commands.

I understand. But guest works in HYP_OWN mode as it sets the HYP_OWN bit. So resetting it looks like a hack to me, not documented in any way in the spec. Besides, as tegra241_guest_vcmdq_supports_cmd() forces to send on the cmdqv cmdq only legal cmds I don't see any problem leaving the bit to 1. What is important is that the host does the check (and the host sets HYP_OWN to 0 for that cmdq). I did the trial to comment

Quoted 3 lines
        //value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;

and it seems to work properly

Ah.. that’s interesting. .I was under the impression that we are toggling the HYP_OWN so that guest does sets the tegra241_guest_vcmdq_supports_cmd() correctly and delegates the supported cmds through that queue.

See,

    tegra241_cmdqv_init_structures()
        tegra241_vintf_alloc_lvcmdq()
          tegra241_vcmdq_alloc_smmu_cmdq()
          {
               .... 
                if (!vcmdq->vintf->hyp_own)
                      cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;

However, it looks like the above happens before tegra241_vintf_hw_init(), which sets HYP_OWN. That may explain why toggling the bit here does not make any difference.

I need to double check why we did the toggling in the QEMU then.

@Nicolin, do you remember?

Thanks,
Shameer

#58 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Quoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines

I understand. But guest works in HYP_OWN mode as it sets the HYP_OWN bit. So resetting it looks like a hack to me, not documented in any way in the spec. Besides, as tegra241_guest_vcmdq_supports_cmd() forces to send on the cmdqv cmdq only legal cmds I don't see any problem leaving the bit to 1. What is important is that the host does the check (and the host sets HYP_OWN to 0 for that cmdq). I did the trial to comment

        //value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;

and it seems to work properly

Ah.. that’s interesting. .I was under the impression that we are toggling the HYP_OWN so that guest does sets the tegra241_guest_vcmdq_supports_cmd() correctly and delegates the supported cmds through that queue.

See,

    tegra241_cmdqv_init_structures()
        tegra241_vintf_alloc_lvcmdq()
          tegra241_vcmdq_alloc_smmu_cmdq()
          {
               .... 
                if (!vcmdq->vintf->hyp_own)
                      cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;
Quoted 3 lines

However, it looks like the above happens before tegra241_vintf_hw_init(), which sets HYP_OWN. That may explain why toggling the bit here does not make any difference.

Well, looks like we exposed a kernel bug...

It seems that the sequence of setting hyp_own and supports_cmd got inverted during a rework when I got the driver upstream.

It turned out that both host and guest kernels sets supports_cmd to tegra241_guest_vcmdq_supports_cmd().. IOW, the host is sending non- invalidation commands to the SMMU CMDQ rather than the HYP_OWNed VCMDQ, although this is completely harmless and probably does not affect host performance at all since non-invalidation commands are not that frequently issued.

This explains why Eric's experiment didn't fail. I'll submit a fix.

With that being said, in QEMU we do need to wire HYP_OWN bit to 0. This is not decided by the spec, but by the host SW implementation:

550 /*

 551          * Note that HYP_OWN bit is wired to zero when running in guest kernel,
 552          * whether enabling it here or not, as !HYP_OWN cmdq HWs only support a
 553          * restricted set of supported commands.
 554          */
 555         regval = FIELD_PREP(VINTF_HYP_OWN, hyp_own) | 
 556                  FIELD_PREP(VINTF_VMID, vintf->vsmmu.vmid);
 557         writel(regval, REG_VINTF(vintf, CONFIG));
 558
 559         ret = vintf_write_config(vintf, regval | VINTF_EN);
 560         if (ret)
 561                 return ret;
 562         /*
 563          * As being mentioned above, HYP_OWN bit is wired to zero for a guest
 564          * kernel, so read it back from HW to ensure that reflects in hyp_own
 565          */

566 vintf->hyp_own = !!(VINTF_HYP_OWN & readl(REG_VINTF(vintf, CONFIG)));

This should have been mentioned in the uAPI header. I'll submit a patch to add this line near IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.

One thing we need to get our head around is that a guest hardware is different than the host hardware:
- Host HW is backed by a VINTF (HYP_OWN=1)
- Guest HW is backed by a VINTF (HYP_OWN=0)
This is a fixed configuration, decided by the hardware and the spec also.

Thus, the guest version of the VINTF must align with its underlying hardware, so QEMU cannot allow setting HYP_OWN=1.

Otherwise, guest OS would have no idea this is a guest version of hardware and would issue unsupported commands that will fail.

Nicolin

#59 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

In reply to Shameer Kolothum
Quoted 11 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

So what does happen if the guest attempts to access the VI CMDQ before its mmapping.

At the moment MMIO accesses are valid as VI_VCMDQ0 MMIOs are handled in the same MMIO region as the Config page.
Is it the expected behavior? I guess you should signal an error and maaybe signal an interrupt if it is not masked for this VI?

Spec says:"While the SW can program the virtual CMDQ(s) directly using the direct VCMDQ aperture (and not though the Virtual Interface), it is required hat the VCMDQ be allocated to a VI before it is used to send commands to the SMMU

Note the terminology used here: direct VCMDQ aparture (not Global)

Thanks

Eric

Quoted 125 lines

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +42 -2
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
 
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#60 lore raw

Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

In reply to Shameer Kolothum
Quoted 68 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +16 -0
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 179

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,

If I am not wrong the mmap_offset is only available after iommufd_backend_alloc_hw_queue(). So I would suggest to first introduce the patch that does the iommufd_backend_alloc_hw_queue (ie. 19/32) and then this one

Eric

Quoted 58 lines
hw/arm/tegra241-cmdqv.c +16 -2
+                                     &cmdqv->vintf_page0, errp)) {
+        return false;
+    }
+
+    return true;
+}
+
 /*
  * Write a VCMDQ register using VCMDQ0_* offsets.
  *
@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
 }
 
 static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
-                                       uint64_t value)
+                                       uint64_t value, Error **errp)
 {
     int i;
 
@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
 
         cmdqv->vintf_config = value;
         if (value & R_VINTF0_CONFIG_ENABLE_MASK) {
+            tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
         break;
@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
                                  unsigned size)
 {
     Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
+    Error *local_err = NULL;
     int index;
 
     if (offset >= TEGRA241_CMDQV_IO_LEN) {
@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] = value;
         break;
     case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:
-        tegra241_cmdqv_write_vintf(cmdqv, offset, value);
+        tegra241_cmdqv_write_vintf(cmdqv, offset, value, &local_err);
         break;
     case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
         /* Same decoding as read() case: See comments above */
@@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
                       __func__, offset);
     }
+
+    if (local_err) {
+        error_report_err(local_err);
+    }
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
#61 lore raw

RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

Hi Eric,

Quoted 91 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:59
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV

External email: Use caution opening links or attachments

Quoted 16 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.

The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.

When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.

Quoted 5 lines
---
 hw/arm/tegra241-cmdqv.h |  3 +++
 hw/arm/tegra241-cmdqv.c | 44

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.h line 18

Quoted 8 lines
 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +0 -0
index d379b8860c..3ce9f539ae 100644
@@ -18,6 +18,8 @@
 #define TEGRA241_CMDQV_MAX_CMDQ            (1U <<

TEGRA241_CMDQV_NUM_CMDQ_LOG2)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3 -0
 #define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4
 
+#define VINTF_PAGE_SIZE 0x10000
+
 /*
  * Tegra241 CMDQV MMIO layout (64KB pages)
  *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    void *vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +0 -0
index e1f1562c44..a3767a85a3 100644
@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2 -0
     }
 }
 
+static bool
+tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error

**errp)

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4 -0
+{
+    if (!cmdqv->vintf_page0) {
+        return true;
+    }

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+
+    if (munmap(cmdqv->vintf_page0, VINTF_PAGE_SIZE) < 0) {
+        error_setg_errno(errp, errno, "Failed to unmap VINTF page0");
+        return false;
+    }
+    cmdqv->vintf_page0 = NULL;
+    return true;
+}
+
+static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv,

Error **errp)

Quoted 15 lines

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 179

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +10 -0
+{
+    IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
+
+    if (cmdqv->vintf_page0) {
+        return true;
+    }
+
+    if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu-
 viommu_id,
+                                     VINTF_PAGE_SIZE,
+                                     cmdqv->cmdqv_data.out_vintf_mmap_offset,

If I am not wrong the mmap_offset is only available after iommufd_backend_alloc_hw_queue(). So I would suggest to first introduce the patch that does the iommufd_backend_alloc_hw_queue (ie. 19/32) and then this one

The out_vintf_mmap_offset is actually populated during IOMMU_VIOMMU_ALLOC (via iommufd_backend_alloc_viommu()), not IOMMU_HW_QUEUE_ALLOC. It is stored in cmdqv->cmdqv_data.out_vintf_mmap_offset at viommu allocation time and is therefore already available. iommufd_backend_alloc_hw_queue() only returns the hw_queue_id,
not the mmap offset.

Thanks,
Shameer

#62 lore raw

PATCH v3 18/32 system/physmem: Add address_space_is_ram() helper

In reply to Shameer Kolothum

Introduce address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.

---
 include/system/memory.h | 10 ++++++++++
 system/physmem.c        | 11 +++++++++++
 2 files changed, 21 insertions(+)
include/system/memory.h +10 -0
index 0562af3136..02b2e83fd7 100644
@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,
  */
 bool address_space_is_io(AddressSpace *as, hwaddr addr);
 
+/**
+ * address_space_is_ram: check whether a guest physical address whithin
+ *                       an address space is RAM.
+ *
+ * @as: #AddressSpace to be accessed
+ * @addr: address within that address space
+ */
+
+bool address_space_is_ram(AddressSpace *as, hwaddr addr);
+
 /* address_space_map: map a physical memory region into a host virtual address
  *
  * May map a subset of the requested range, given by and returned in @plen.
system/physmem.c +11 -0
index 2fb0c25c93..ddcf921311 100644
@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)
     return !(memory_region_is_ram(mr) || memory_region_is_romd(mr));
 }
 
+bool address_space_is_ram(AddressSpace *as, hwaddr addr)
+{
+    MemoryRegion *mr;
+
+    RCU_READ_LOCK_GUARD();
+    mr = address_space_translate(as, addr, &addr, NULL, false,
+                                 MEMTXATTRS_UNSPECIFIED);
+
+    return memory_region_is_ram(mr);
+}
+
 static hwaddr
 flatview_extend_translation(FlatView *fv, hwaddr addr,
                             hwaddr target_len,
Signature · 1 line
2.43.0
#63 lore raw

PATCH v3 19/32 hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. This region is mapped directly into the guest address space (introduced in a subsequent patch), so QEMU does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.

If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 71 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 79 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 3ce9f539ae..139e14b61b 100644
@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +68 -3

118 lines not on this page · raw message

Signature · 1 line
2.43.0
#64 lore raw

Re: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 74 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. This region is mapped directly into the guest address space (introduced in a subsequent patch), so QEMU does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.

If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

Writes with invalid addresses are ignored.

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 71 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 79 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 3ce9f539ae..139e14b61b 100644
@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +8 -0
index a3767a85a3..002dde50fc 100644
@@ -151,6 +151,67 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {
+        return;

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 171

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10 -0
+    }
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
+    cmdqv->vcmdq[index] = NULL;
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in the reverse order to avoid "resource busy" error */
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {

Is it future proof to restrict the number of queues to 2? Shouldn't it be configurable?

Eric

Quoted 103 lines
hw/arm/tegra241-cmdqv.c +50 -3
+        tegra241_cmdqv_free_vcmdq(cmdqv, i);
+    }
+}
+
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
+                                       Error **errp)
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    uint64_t base_mask = (uint64_t)R_VCMDQ0_BASE_L_ADDR_MASK |
+                         (uint64_t)R_VCMDQ0_BASE_H_ADDR_MASK << 32;
+    uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
+    uint64_t log2 = cmdqv->vcmdq_base[index] & R_VCMDQ0_BASE_L_LOG2SIZE_MASK;
+    uint64_t size = 1ULL << (log2 + 4);
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *hw_queue;
+    uint32_t hw_queue_id;
+
+    /* Ignore any invalid address. This may come as part of reset etc */
+    if (!address_space_is_ram(&address_space_memory, addr) ||
+        !address_space_is_ram(&address_space_memory, addr + size - 1)) {
+        return true;
+    }
+
+    if (!tegra241_cmdq_enabled(cmdqv) || !tegra241_vintf_enabled(cmdqv)) {
+        return true;
+    }
+
+    tegra241_cmdqv_free_vcmdq(cmdqv, index);
+
+    if (!iommufd_backend_alloc_hw_queue(viommu->iommufd, viommu->viommu_id,
+                                        IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV,
+                                        index, addr, size, &hw_queue_id,
+                                        errp)) {
+        return false;
+    }
+    hw_queue = g_new(IOMMUFDHWqueue, 1);
+    hw_queue->hw_queue_id = hw_queue_id;
+    hw_queue->viommu = viommu;
+    cmdqv->vcmdq[index] = hw_queue;
+
+    return true;
+}
+
 static bool
 tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
 {
@@ -192,7 +253,7 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
-                           uint64_t value, unsigned size)
+                           uint64_t value, unsigned size, Error **errp)
 {
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
@@ -220,11 +281,13 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                 (cmdqv->vcmdq_base[index] & 0xffffffff00000000ULL) |
                 (value & 0xffffffffULL);
         }
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_BASE_H:
         cmdqv->vcmdq_base[index] =
             (cmdqv->vcmdq_base[index] & 0xffffffffULL) |
             ((uint64_t)value << 32);
+        tegra241_cmdqv_setup_vcmdq(cmdqv, index, errp);
         return;
     case A_VCMDQ0_CONS_INDX_BASE_DRAM_L:
         if (size == 8) {
@@ -263,6 +326,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
             tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
         }
@@ -302,6 +366,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
         break;
@@ -321,7 +386,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN:
         index = (offset - 0x10000) / 0x80;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
-                                   size);
+                                   size, &local_err);
         break;
     case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         /* Same decoding as read() case: See comments above */
@@ -330,7 +395,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H:
         index = (offset - 0x20000) / 0x80;
         tegra241_cmdqv_write_vcmdq(cmdqv, offset - 0x80 * index, index, value,
-                                   size);
+                                   size, &local_err);
         break;
     default:
         qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
#65 lore raw

RE: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

Quoted 101 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 14:34
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming

External email: Use caution opening links or attachments

Quoted 20 lines · Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.

VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. This region is mapped directly into the guest address space (introduced in a subsequent patch), so QEMU does not trap writes to VCMDQ_CONFIG.

Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.

If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.

Writes with invalid addresses are ignored.

Quoted 7 lines

All allocated VCMDQs are freed when CMDQV or VINTF is disabled.

---
 hw/arm/tegra241-cmdqv.h | 11 +++++++
 hw/arm/tegra241-cmdqv.c | 71

+++++++++++++++++++++++++++++++++++++++--

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void

Quoted 36 lines
 2 files changed, 79 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
index 3ce9f539ae..139e14b61b 100644
@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {
     SMMUv3AccelState *s_accel;
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
+    IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
 
     /* Register Cache */
@@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0)
 A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1)
 
+static inline bool tegra241_cmdq_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->status & R_STATUS_CMDQV_ENABLED_MASK;
+}
+
+static inline bool tegra241_vintf_enabled(Tegra241CMDQV *cmdq)
+{
+    return cmdq->vintf_status & R_VINTF0_STATUS_ENABLE_OK_MASK;
+}
+
 const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops(void);
 
 #endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +0 -0
index a3767a85a3..002dde50fc 100644
@@ -151,6 +151,67 @@ static uint64_t tegra241_cmdqv_read(void

*opaque, hwaddr offset, unsigned size)

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 154

Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1 -0
     }
 }
 
+static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int

index)

Quoted 11 lines
hw/arm/tegra241-cmdqv.c +11 -0
+{
+    SMMUv3AccelState *accel = cmdqv->s_accel;
+    IOMMUFDViommu *viommu = accel->viommu;
+    IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index];
+
+    if (!vcmdq) {
+        return;
+    }
+    iommufd_backend_free_id(viommu->iommufd, vcmdq->hw_queue_id);
+    g_free(vcmdq);
+    cmdqv->vcmdq[index] = NULL;
Quoted 8 lines

hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 171

Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6 -0
+}
+
+static void tegra241_cmdqv_free_all_vcmdq(Tegra241CMDQV *cmdqv)
+{
+    /* Free in the reverse order to avoid "resource busy" error */
+    for (int i = (TEGRA241_CMDQV_MAX_CMDQ - 1); i >= 0; i--) {

Is it future proof to restrict the number of queues to 2? Shouldn't it be configurable?

On current platforms the host reports log2vcmdqs = 1 (2 VCMDQs). I think, for the initial implementation we can keep the number of queues to 2. We can make it configurable if the requirement comes up in the future.

Thanks,
Shameer

#66 lore raw

PATCH v3 20/32 hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

In reply to Shameer Kolothum

When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the kernel, access the VCMDQ registers directly through the VINTF page0 backing instead of using QEMU's cached register state.

VINTF page0 provides the backing memory region for VCMDQ registers once a hardware queue is created. In that case, reads and writes should reflect the live backing state.

If a VCMDQ is not allocated, or if VINTF page0 is not available, continue to use the cached register values maintained by QEMU.

---
 hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +46 -1
index 002dde50fc..17b9552906 100644
@@ -14,17 +14,44 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
+                                                 int index, hwaddr offset0)
+{
+    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
+        return NULL;
+    }
+
+    return (uint32_t *)(cmdqv->vintf_page0 + (index * 0x80) +
+                        (offset0 - 0x10000));
+}
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
  *
- * All VCMDQ accesses return cached registers.
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly
+ * from the VINTF page0 backing. Otherwise, fall back to cached state.
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                           int index)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -250,11 +277,29 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly
+ * to the VINTF page0 backing. Otherwise, update cached state.
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                            uint64_t value, unsigned size, Error **errp)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = value;
Signature · 1 line
2.43.0
#67 lore raw

Re: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

Quoted 10 lines · Shameer Kolothum

When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the kernel, access the VCMDQ registers directly through the VINTF page0 backing instead of using QEMU's cached register state.

VINTF page0 provides the backing memory region for VCMDQ registers once a hardware queue is created. In that case, reads and writes should reflect the live backing state.

If a VCMDQ is not allocated, or if VINTF page0 is not available, continue to use the cached register values maintained by QEMU.

But if both coexist shouldn't you also sync emulated/mmaped VINTF CMDQV. Somehow this means that you have 3 states of the same CMDQV: the global/direct one, the emulated VI CMDQV and the mmapped VI CMDQV (host backed).
Is this understanding correct?

Thanks

Eric

Quoted 86 lines
---
 hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +46 -1
index 002dde50fc..17b9552906 100644
@@ -14,17 +14,44 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
+                                                 int index, hwaddr offset0)
+{
+    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
+        return NULL;
+    }
+
+    return (uint32_t *)(cmdqv->vintf_page0 + (index * 0x80) +
+                        (offset0 - 0x10000));
+}
 /*
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
  *
- * All VCMDQ accesses return cached registers.
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly
+ * from the VINTF page0 backing. Otherwise, fall back to cached state.
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
                                           int index)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -250,11 +277,29 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly
+ * to the VINTF page0 backing. Otherwise, update cached state.
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index,
                            uint64_t value, unsigned size, Error **errp)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = value;
#68 lore raw

RE: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

Quoted 28 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 14:53
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing

External email: Use caution opening links or attachments

Quoted 8 lines · Shameer Kolothum

When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the kernel, access the VCMDQ registers directly through the VINTF page0 backing instead of using QEMU's cached register state.

VINTF page0 provides the backing memory region for VCMDQ registers once a hardware queue is created. In that case, reads and writes should reflect the live backing state.

Quoted 8 lines
Quoted 2 lines

If a VCMDQ is not allocated, or if VINTF page0 is not available, continue to use the cached register values maintained by QEMU.

But if both coexist shouldn't you also sync emulated/mmaped VINTF CMDQV. Somehow this means that you have 3 states of the same CMDQV: the global/direct one, the emulated VI CMDQV and the mmapped VI CMDQV (host
backed).
Is this understanding correct?

Yes. However, in the current implementation the VCMDQ is functionally valid only if QEMU successfully sets up the VINTF page0 mmap and creates a backing VCMDQ through the HW_QUEUE ioctl.

All other cases it just uses the cached registers for MMIO read/write.

Maybe we should expose such failures to the guest, by injecting an error or by setting STATUS/CONFIG to indicate that the VCMDQ is not functional.

Regarding keeping the Global VCMDQ and the VI CMDQ in sync always, this would only be possible if we do not map (tegra241_cmdqv_guest_map_vintf_page0) the VINTF Page0 into guest memory. Once it is mapped, accesses to the VI CMDQ (control/status registers) no longer trap to QEMU.

However, not mapping it is not advisable as it would hurt performance, since every CONS/PROD index update would trap to QEMU.

Thanks,
Shameer

Quoted 95 lines

Thanks

Eric

Quoted 4 lines
---
 hw/arm/tegra241-cmdqv.c | 47

++++++++++++++++++++++++++++++++++++++++-

hw/arm/tegra241-cmdqv.c line 17

Quoted 11 lines
 1 file changed, 46 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +1 -0
index 002dde50fc..17b9552906 100644
@@ -14,17 +14,44 @@
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
+static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV

*cmdqv,

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9 -0
+                                                 int index, hwaddr offset0)
+{
+    if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) {
+        return NULL;
+    }
+
+    return (uint32_t *)(cmdqv->vintf_page0 + (index * 0x80) +
+                        (offset0 - 0x10000));
+}
 /*

hw/arm/tegra241-cmdqv.c

Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2 -1
  * Read a VCMDQ register using VCMDQ0_* offsets.
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
  *
- * All VCMDQ accesses return cached registers.
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly
+ * from the VINTF page0 backing. Otherwise, fall back to cached state.
  */
 static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,

hwaddr offset0,

hw/arm/tegra241-cmdqv.c static bool

Quoted 22 lines
hw/arm/tegra241-cmdqv.c +16 -0
                                           int index)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_STATUS:
+        case A_VCMDQ0_GERROR:
+        case A_VCMDQ0_GERRORN:
+            return *ptr;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         return cmdqv->vcmdq_cons_indx[index];
@@ -250,11 +277,29 @@ static bool

tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)

Quoted 9 lines
hw/arm/tegra241-cmdqv.c +3 -0
  *
  * The caller normalizes the MMIO offset such that @offset0 always refers
  * to a VCMDQ0_* register, while @index selects the VCMDQ instance.
+ *
+ * If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly
+ * to the VINTF page0 backing. Otherwise, update cached state.
  */
 static void
 tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,

int index,

Quoted 20 lines
hw/arm/tegra241-cmdqv.c +15 -0
                            uint64_t value, unsigned size, Error **errp)
 {
+    uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0);
+
+    if (ptr) {
+        switch (offset0) {
+        case A_VCMDQ0_CONS_INDX:
+        case A_VCMDQ0_PROD_INDX:
+        case A_VCMDQ0_CONFIG:
+        case A_VCMDQ0_GERRORN:
+            *ptr = (uint32_t)value;
+            return;
+        default:
+            break;
+        }
+    }
+
     switch (offset0) {
     case A_VCMDQ0_CONS_INDX:
         cmdqv->vcmdq_cons_indx[index] = value;
#69 lore raw

PATCH v3 21/32 hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

VINTF page0 is backed by host memory returned by the kernel via mmap. Instead of trapping and emulating accesses to this region, map it directly into the guest-visible MMIO space.

The VINTF page0 MMIO region is created lazily when the first VCMDQ hardware queue is allocated. When CMDQV or VINTF is disabled, the region is removed.

---
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/tegra241-cmdqv.c | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
hw/arm/tegra241-cmdqv.h +1 -0
index 139e14b61b..914977c2ef 100644
@@ -38,6 +38,7 @@ typedef struct Tegra241CMDQV {
     qemu_irq irq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
     void *vintf_page0;
+    MemoryRegion *mr_vintf_page0;
 
     /* Register Cache */
     uint32_t config;
hw/arm/tegra241-cmdqv.c +35 -0
index 17b9552906..ce144add54 100644
@@ -178,6 +178,38 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
     }
 }
 
+static void tegra241_cmdqv_guest_unmap_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    if (!cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    memory_region_del_subregion(&cmdqv->mmio_cmdqv, cmdqv->mr_vintf_page0);
+    object_unparent(OBJECT(cmdqv->mr_vintf_page0));
+    g_free(cmdqv->mr_vintf_page0);
+    cmdqv->mr_vintf_page0 = NULL;
+}
+
+static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)
+{
+    char *name;
+
+    if (cmdqv->mr_vintf_page0) {
+        return;
+    }
+
+    name = g_strdup_printf("%s vintf-page0",
+                           memory_region_name(&cmdqv->mmio_cmdqv));
+    cmdqv->mr_vintf_page0 = g_malloc0(sizeof(*cmdqv->mr_vintf_page0));
+    memory_region_init_ram_device_ptr(cmdqv->mr_vintf_page0,
+                                      memory_region_owner(&cmdqv->mmio_cmdqv),
+                                      name, VINTF_PAGE_SIZE,
+                                      cmdqv->vintf_page0);
+    memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv, 0x30000,
+                                        cmdqv->mr_vintf_page0, 1);
+    g_free(name);
+}
+
 static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, int index)
 {
     SMMUv3AccelState *accel = cmdqv->s_accel;
@@ -236,6 +268,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
     hw_queue->viommu = viommu;
     cmdqv->vcmdq[index] = hw_queue;
 
+    tegra241_cmdqv_guest_map_vintf_page0(cmdqv);
     return true;
 }
 
@@ -371,6 +404,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
             tegra241_cmdqv_mmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status |= R_VINTF0_STATUS_ENABLE_OK_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             tegra241_cmdqv_munmap_vintf_page0(cmdqv, errp);
             cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK;
@@ -411,6 +445,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
         if (value & R_CONFIG_CMDQV_EN_MASK) {
             cmdqv->status |= R_STATUS_CMDQV_ENABLED_MASK;
         } else {
+            tegra241_cmdqv_guest_unmap_vintf_page0(cmdqv);
             tegra241_cmdqv_free_all_vcmdq(cmdqv);
             cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK;
         }
Signature · 1 line
2.43.0
#70 lore raw

PATCH v3 22/32 hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

In reply to Shameer Kolothum

Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.

---
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/smmuv3-accel.c   | 10 ++++++++-
 hw/arm/tegra241-cmdqv.c | 47 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 59 insertions(+), 1 deletion(-)
hw/arm/smmuv3-accel.h +2 -0
index 7d6e4c6b76..4bff90e2c1 100644
@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
                          uint32_t *out_viommu_id,
                          Error **errp);
     void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);
     void (*reset)(SMMUv3State *s);
 } SMMUv3AccelCmdqvOps;
 
hw/arm/tegra241-cmdqv.h +1 -0
index 914977c2ef..01d446474a 100644
@@ -37,6 +37,7 @@ typedef struct Tegra241CMDQV {
     MemoryRegion mmio_cmdqv;
     qemu_irq irq;
     IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ];
+    IOMMUFDVeventq *veventq;
     void *vintf_page0;
     MemoryRegion *mr_vintf_page0;
 
hw/arm/smmuv3-accel.c +9 -1
index 4373bbd97b..f6602f51aa 100644
@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_bypass_hwpt;
     }
 
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+
     /* Attach a HWPT based on SMMUv3 GBPA.ABORT value */
     hwpt_id = smmuv3_accel_gbpa_hwpt(s, accel);
     if (!host_iommu_device_iommufd_attach_hwpt(idev, hwpt_id, errp)) {
-        goto free_veventq;
+        goto free_cmdqv_veventq;
     }
     return true;
 
+free_cmdqv_veventq:
+    if (cmdqv_ops && cmdqv_ops->free_veventq) {
+        cmdqv_ops->free_veventq(s);
+    }
 free_veventq:
     smmuv3_accel_free_veventq(accel);
 free_bypass_hwpt:
hw/arm/tegra241-cmdqv.c +47 -0

59 lines not on this page · raw message

Signature · 1 line
2.43.0
#71 lore raw

Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

Quoted 16 lines · Shameer Kolothum

Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.

---
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/smmuv3-accel.c   | 10 ++++++++-
 hw/arm/tegra241-cmdqv.c | 47 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 59 insertions(+), 1 deletion(-)
hw/arm/smmuv3-accel.h +0 -0
index 7d6e4c6b76..4bff90e2c1 100644

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelCmdqvOps { · line 32

Quoted 5 lines
hw/arm/smmuv3-accel.h +2 -0
@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
                          uint32_t *out_viommu_id,
                          Error **errp);
     void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

As I replied in v2, this should really depend on viommu and should be simply added to tegra241_cmdqv_alloc/free_viommu().

The alloc_viommu is an override of the standard viommu type, so it might be useful to have an op.

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 582

Quoted 12 lines
hw/arm/smmuv3-accel.c +4 -0
index 4373bbd97b..f6602f51aa 100644
@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
         goto free_bypass_hwpt;
     }
 
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+

But, this veventq op is completely in parallel to the standard one. There is no need for the smmuv3-accel to initiate the allocation?

Thanks
Nicolin

#72 lore raw

RE: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

Quoted 29 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

Quoted 9 lines · Shameer Kolothum

Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.

---
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/smmuv3-accel.c   | 10 ++++++++-
 hw/arm/tegra241-cmdqv.c | 47

+++++++++++++++++++++++++++++++++++++++++

Quoted 2 lines
 4 files changed, 59 insertions(+), 1 deletion(-)
Quoted 13 lines

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelCmdqvOps { · line 32

Quoted 10 lines
hw/arm/smmuv3-accel.h +2 -0
index 7d6e4c6b76..4bff90e2c1 100644
@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
                          uint32_t *out_viommu_id,
                          Error **errp);
     void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

As I replied in v2, this should really depend on viommu and should be simply added to tegra241_cmdqv_alloc/free_viommu().

Yes, I remember that. That was actually my initial plan, which is why this was not part of the original callbacks in the ops struct.

However, I then realised that we are storing the viommu pointer in veventq,

veventq->viommu = accel->viommu;

And that struct viommu is only allocated after alloc_viommu() returns.

I didn't find an easy way to resolve that. Need to take another look.

Quoted 13 lines

The alloc_viommu is an override of the standard viommu type, so it might be useful to have an op.

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,

Quoted 5 lines
hw/arm/smmuv3-accel.c +0 -0
index 4373bbd97b..f6602f51aa 100644
@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,

HostIOMMUDeviceIOMMUFD *idev,

Quoted 3 lines
hw/arm/smmuv3-accel.c +0 -0
         goto free_bypass_hwpt;
     }
 
Quoted 7 lines

hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 582

Quoted 4 lines
hw/arm/smmuv3-accel.c +4 -0
+    if (cmdqv_ops && !cmdqv_ops->alloc_veventq(s, errp)) {
+        goto free_veventq;
+    }
+

But, this veventq op is completely in parallel to the standard one. There is no need for the smmuv3-accel to initiate the allocation?

If we can resolve the above dependency, then yes it can be tucked inside .

Thanks,
Shameer

#73 lore raw

Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

Quoted 46 lines · Shameer Kolothum Thodi
Quoted 29 lines

-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free

Quoted 9 lines · Shameer Kolothum

Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.

---
 hw/arm/smmuv3-accel.h   |  2 ++
 hw/arm/tegra241-cmdqv.h |  1 +
 hw/arm/smmuv3-accel.c   | 10 ++++++++-
 hw/arm/tegra241-cmdqv.c | 47

+++++++++++++++++++++++++++++++++++++++++

Quoted 2 lines
 4 files changed, 59 insertions(+), 1 deletion(-)
Quoted 13 lines

hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelCmdqvOps { · line 32

Quoted 10 lines
hw/arm/smmuv3-accel.h +2 -0
index 7d6e4c6b76..4bff90e2c1 100644
@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {
                          uint32_t *out_viommu_id,
                          Error **errp);
     void (*free_viommu)(SMMUv3State *s);
+    bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
+    void (*free_veventq)(SMMUv3State *s);

As I replied in v2, this should really depend on viommu and should be simply added to tegra241_cmdqv_alloc/free_viommu().

Yes, I remember that. That was actually my initial plan, which is why this was not part of the original callbacks in the ops struct.

Quoted 8 lines

However, I then realised that we are storing the viommu pointer in veventq,

veventq->viommu = accel->viommu;

And that struct viommu is only allocated after alloc_viommu() returns.

I didn't find an easy way to resolve that. Need to take another look.

Looks like we only needed iommufd pointer v.s. accel->viommu? This should allow us to merge alloc_veventq into alloc_viommu.

Alternatively, as we discussed on the other side, we could drop the alloc/free_viommu. Then, have a pair of viommu_init/destroy to have the veventq part (and any viommu specific routine in the future).

I feel okay with either approach. So, pick one that works the best.

Nicolin

#74 lore raw

PATCH v3 23/32 hw/arm/smmuv3-accel: Introduce common helper for veventq read

In reply to Shameer Kolothum

Move the vEVENTQ read and validation logic into a common helper.

The helper performs the read(), checks for overflow and short reads, validates the sequence number, and updates the sequence state.

This helper can be reused in the subsequent patch for Tegra241 CMDQV vEVENTQ support.

Error handling is slightly adjusted: instead of reporting errors directly in the read handler, the helper now returns errors via Error **. Sequence gaps are reported as warnings.

---
 hw/arm/smmuv3-accel.h       |  2 ++
 hw/arm/smmuv3-accel-stubs.c |  6 ++++
 hw/arm/smmuv3-accel.c       | 67 ++++++++++++++++++++++---------------
 3 files changed, 48 insertions(+), 27 deletions(-)
hw/arm/smmuv3-accel.h +2 -0
index 4bff90e2c1..c349981e79 100644
@@ -70,6 +70,8 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,
                                 Error **errp);
 void smmuv3_accel_idr_override(SMMUv3State *s);
 bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +6 -0
index 870fc2a71c..1d5d3bb10c 100644
@@ -42,6 +42,12 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)
     return true;
 }
 
+bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
+                                      void *buf, size_t size, Error **errp)
+{
+    return true;
+}
+
 void smmuv3_accel_idr_override(SMMUv3State *s)
 {
 }
hw/arm/smmuv3-accel.c +40 -27

87 lines not on this page · raw message

Signature · 1 line
2.43.0
#75 lore raw

PATCH v3 24/32 hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors

In reply to Shameer Kolothum

Install an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.

The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().

---
 hw/arm/tegra241-cmdqv.c | 58 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  3 +++
 2 files changed, 61 insertions(+)
hw/arm/tegra241-cmdqv.c +58 -0
index 8cde459b4f..99b85e698f 100644
@@ -9,8 +9,10 @@
 
 #include "qemu/osdep.h"
 #include "qemu/log.h"
+#include "trace.h"
 
 #include "hw/arm/smmuv3.h"
+#include "hw/core/irq.h"
 #include "smmuv3-accel.h"
 #include "tegra241-cmdqv.h"
 
@@ -487,6 +489,43 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
     }
 }
 
+static void tegra241_cmdqv_event_read(void *opaque)
+{
+    Tegra241CMDQV *cmdqv = opaque;
+    IOMMUFDVeventq *veventq = cmdqv->veventq;
+    struct {
+        struct iommufd_vevent_header hdr;
+        struct iommu_vevent_tegra241_cmdqv vevent;
+    } buf;
+    Error *local_err;
+
+    if (!smmuv3_accel_event_read_validate(veventq,
+                                          IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV,
+                                          &buf, sizeof(buf), &local_err)) {
+        warn_report_err_once(local_err);
+        return;
+    }
+
+    if (buf.vevent.lvcmdq_err_map[0] || buf.vevent.lvcmdq_err_map[1]) {
+        cmdqv->vintf_cmdq_err_map[0] =
+            buf.vevent.lvcmdq_err_map[0] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[1] =
+            (buf.vevent.lvcmdq_err_map[0] >> 32) & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[2] =
+            buf.vevent.lvcmdq_err_map[1] & 0xffffffff;
+        cmdqv->vintf_cmdq_err_map[3] =
+            (buf.vevent.lvcmdq_err_map[1] >> 32) & 0xffffffff;
+        for (int i = 0; i < 4; i++) {
+            cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i];
+        }
+        cmdqv->vi_err_map[0] |= 0x1;
+        qemu_irq_pulse(cmdqv->irq);
+        trace_tegra241_cmdqv_err_map(
+        cmdqv->vintf_cmdq_err_map[3], cmdqv->vintf_cmdq_err_map[2],
+        cmdqv->vintf_cmdq_err_map[1], cmdqv->vintf_cmdq_err_map[0]);
+    }
+}
+
 static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
 {
     SMMUv3AccelState *accel = s->s_accel;
@@ -496,6 +535,7 @@ static void tegra241_cmdqv_free_veventq(SMMUv3State *s)
     if (!veventq) {
         return;
     }
+    qemu_set_fd_handler(veventq->veventq_fd, NULL, NULL, NULL);
     close(veventq->veventq_fd);
     iommufd_backend_free_id(veventq->viommu->iommufd, veventq->veventq_id);
     g_free(veventq);
@@ -510,6 +550,7 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
     IOMMUFDVeventq *veventq;
     uint32_t veventq_id;
     uint32_t veventq_fd;
+    int flags;
 
     if (cmdqv->veventq) {
         return true;
@@ -523,13 +564,30 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)
         return false;
     }
 
+    flags = fcntl(veventq_fd, F_GETFL);
+    if (flags < 0) {
+        error_setg(errp, "Failed to get flags for vEVENTQ fd");
+        goto free_veventq;
+    }
+    if (fcntl(veventq_fd, F_SETFL, O_NONBLOCK | flags) < 0) {
+        error_setg(errp, "Failed to set O_NONBLOCK on vEVENTQ fd");
+        goto free_veventq;
+    }
+
     veventq = g_new(IOMMUFDVeventq, 1);
     veventq->veventq_id = veventq_id;
     veventq->veventq_fd = veventq_fd;
     veventq->viommu = accel->viommu;
     cmdqv->veventq = veventq;
 
+    /* Set up event handler for veventq fd */
+    qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL, cmdqv);
     return true;
+
+free_veventq:
+    close(veventq_fd);
+    iommufd_backend_free_id(viommu->iommufd, veventq_id);
+    return false;
 }
 
 static void tegra241_cmdqv_free_viommu(SMMUv3State *s)
hw/arm/trace-events +3 -0
index 3457536fb0..76bda0efef 100644
@@ -72,6 +72,9 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev
 smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64
 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x"
 
+# tegra241-cmdqv
+tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
 strongarm_ssp_read_underrun(void) "SSP rx underrun"
Signature · 1 line
2.43.0
#76 lore raw

PATCH v3 25/32 hw/arm/tegra241-cmdqv: Add reset handler

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.

CMDQV gets initialized early during guest boot, hence the handler verifies that at least one cold-plugged device is attached to the associated vIOMMU before proceeding. This is required to retrieve host CMDQV info and to validate it against the QEMU implementation support.

---
 hw/arm/tegra241-cmdqv.h |  2 ++
 hw/arm/smmuv3-accel.c   | 12 +++++++++-
 hw/arm/tegra241-cmdqv.c | 50 +++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events     |  1 +
 4 files changed, 64 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +2 -0
index 01d446474a..96a737eb4e 100644
@@ -72,6 +72,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8)
 FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8)
 FIELD(CONFIG, CONS_DRAM_EN, 20, 1)
 
+#define V_CONFIG_RESET 0x00020403
+
 REG32(PARAM, 0x4)
 FIELD(PARAM, CMDQV_VER, 0, 4)
 FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4)
hw/arm/smmuv3-accel.c +11 -1
index 5f296ea763..9a570b8af9 100644
@@ -774,7 +774,11 @@ static void smmuv3_accel_unset_iommu_device(PCIBus *bus, void *opaque,
     QLIST_REMOVE(accel_dev, next);
     trace_smmuv3_accel_unset_iommu_device(devfn, idev->devid);
 
-    if (QLIST_EMPTY(&accel->device_list)) {
+   /*
+    * Keep the vIOMMU alive when CMDQV is present, as the vIOMMU to host
+    * SMMUv3 association cannot be changed via device hot-plug.
+    */
+    if (QLIST_EMPTY(&accel->device_list) && !accel->cmdqv) {
         smmuv3_accel_free_viommu(accel);
     }
 }
@@ -966,6 +970,12 @@ void smmuv3_accel_reset(SMMUv3State *s)
     /* Attach a HWPT based on GBPA reset value */
     smmuv3_accel_attach_gbpa_hwpt(s, NULL);
 
+    if (s->cmdqv == ON_OFF_AUTO_ON && QLIST_EMPTY(&accel->device_list)) {
+        error_report("cmdqv=on: requires at least one cold-plugged "
+                     "vfio-pci device");
+        exit(1);
+    }
+
     if (accel->cmdqv_ops && accel->cmdqv_ops->reset) {
         accel->cmdqv_ops->reset(s);
     }
hw/arm/tegra241-cmdqv.c +50 -0

64 lines not on this page · raw message

hw/arm/trace-events +1 -0
index 76bda0efef..ef495c040c 100644
@@ -74,6 +74,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS
 
 # tegra241-cmdqv
 tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X"
+tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X"
 
 # strongarm.c
 strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
#77 lore raw

PATCH v3 26/32 hw/arm/tegra241-cmdqv: Limit queue size based on backend page size

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

CMDQV HW reads guest queue memory in its host physical address setup via IOMUUFD. This requires the guest queue memory isn't only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQV size that the host can safely back with physically contiguous memory. Allowing a CMDQV larger than the host page size could cause the hardware to DMA across page boundaries leading to faults.

Limit IDR1.CMDQS so the guest cannot configure a CMDQV that exceeds the host’s contiguous backing.

---
 hw/arm/tegra241-cmdqv.c | 43 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)
hw/arm/tegra241-cmdqv.c +43 -0
index 5afdc5c8a4..a379341c0a 100644
@@ -11,10 +11,14 @@
 #include "qemu/error-report.h"
 #include "qemu/log.h"
 #include "trace.h"
+#include <math.h>
 
 #include "hw/arm/smmuv3.h"
 #include "hw/core/irq.h"
 #include "smmuv3-accel.h"
+#include "smmuv3-internal.h"
+#include "system/ramblock.h"
+#include "system/ramlist.h"
 #include "tegra241-cmdqv.h"
 
 static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
@@ -618,9 +622,38 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
     return true;
 }
 
+static size_t tegra241_cmdqv_min_ram_pagesize(void)
+{
+    RAMBlock *rb;
+    size_t pg, min_pg = SIZE_MAX;
+
+    RAMBLOCK_FOREACH(rb) {
+        MemoryRegion *mr = rb->mr;
+
+        /* Only consider real RAM regions */
+        if (!mr || !memory_region_is_ram(mr)) {
+            continue;
+        }
+
+        /* Skip RAM regions that are not backed by a memory-backend */
+        if (!object_dynamic_cast(mr->owner, TYPE_MEMORY_BACKEND)) {
+            continue;
+        }
+
+        pg = qemu_ram_pagesize(rb);
+        if (pg && pg < min_pg) {
+            min_pg = pg;
+        }
+    }
+
+    return (min_pg == SIZE_MAX) ? qemu_real_host_page_size() : min_pg;
+}
+
 static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
 {
     int i;
+    size_t pgsize;
+    uint32_t val;
 
     cmdqv->config = V_CONFIG_RESET;
     cmdqv->param =
@@ -652,6 +685,16 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)
         cmdqv->vcmdq_base[i] = 0;
         cmdqv->vcmdq_cons_indx_base[i] = 0;
     }
+
+   /*
+    * CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the
+    * queue fits entirely within the smallest backend page size, ensuring
+    * the command queue is physically contiguous in host memory.
+    */
+    pgsize = tegra241_cmdqv_min_ram_pagesize();
+    val = FIELD_EX32(s->idr[1], IDR1, CMDQS);
+    s->idr[1] = FIELD_DP32(s->idr[1], IDR1, CMDQS, MIN(log2(pgsize) - 4, val));
+
     return;
 }
 
Signature · 1 line
2.43.0
#78 lore raw

PATCH v3 27/32 tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier change

In reply to Shameer Kolothum

The IORT SMMUv3 node Identifier field will be derived from a new per-device "identifier" property instead of relying on enumeration order.

Add the affected IORT blobs to allowed-diff list for bios-table tests.

---
 tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++
 1 file changed, 4 insertions(+)
tests/qtest/bios-tables-test-allowed-diff.h +4 -0
index dfb8523c8b..df5fe417c0 100644
@@ -1 +1,5 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/aarch64/virt/IORT.its_off",
+"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m",
+"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy",
+"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
Signature · 1 line
2.43.0
Reviewed-by on v3 27/32
#79 lore raw

Re: [PATCH v3 27/32] tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier change

Quoted 7 lines · Shameer Kolothum

The IORT SMMUv3 node Identifier field will be derived from a new per-device "identifier" property instead of relying on enumeration order.

Add the affected IORT blobs to allowed-diff list for bios-table tests.

Eric

Quoted 14 lines
---
 tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++
 1 file changed, 4 insertions(+)
tests/qtest/bios-tables-test-allowed-diff.h +4 -0
index dfb8523c8b..df5fe417c0 100644
@@ -1 +1,5 @@
 /* List of comma-separated changed AML files to ignore */
+"tests/data/acpi/aarch64/virt/IORT.its_off",
+"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m",
+"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy",
+"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
#80 lore raw

PATCH v3 28/32 hw/arm/smmuv3: Add per-device identifier property

In reply to Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.

No functional change intended.

---
 include/hw/arm/smmuv3.h  | 1 +
 hw/arm/smmuv3.c          | 2 ++
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 4 files changed, 9 insertions(+), 1 deletion(-)
include/hw/arm/smmuv3.h +1 -0
index 648412cafc..73b8f39aaa 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
hw/arm/smmuv3.c +2 -0
index 468135bb24..c1f84bedd4 100644
@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
index ae78e9b9e0..20605185c5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiIortSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
     GArray *rc_smmu_idmaps;
     /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
     size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
                                                &error_abort));
         sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
         sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
         pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
         sbdev = SYS_BUS_DEVICE(obj);
         sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +3 -0
index c75a8d6e9e..44c6b99c96 100644
@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
     }
 }
 
+static uint8_t smmuv3_dev_id;
 static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                             DeviceState *dev, Error **errp)
 {
@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", smmuv3_dev_id++,
+                                     NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
Signature · 1 line
2.43.0
Reviewed-by on v3 28/32
#81 lore raw

Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

Shameer,

Quoted 9 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.

No functional change intended.

besides changes the IORT table

Quoted 69 lines
---
 include/hw/arm/smmuv3.h  | 1 +
 hw/arm/smmuv3.c          | 2 ++
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 4 files changed, 9 insertions(+), 1 deletion(-)
include/hw/arm/smmuv3.h +1 -0
index 648412cafc..73b8f39aaa 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
hw/arm/smmuv3.c +2 -0
index 468135bb24..c1f84bedd4 100644
@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
index ae78e9b9e0..20605185c5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiIortSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
     GArray *rc_smmu_idmaps;
     /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
     size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
                                                &error_abort));
         sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
         sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
         pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
         sbdev = SYS_BUS_DEVICE(obj);
         sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +0 -0
index c75a8d6e9e..44c6b99c96 100644

hw/arm/virt.c static void virt_memory_plug(HotplugHandler *hotplug_dev, · line 3141

Quoted 4 lines
hw/arm/virt.c +1 -0
@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
     }
 }
 
+static uint8_t smmuv3_dev_id;

nit: I would rather put that at the top of the file

hw/arm/virt.c static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 3204

Quoted 12 lines
hw/arm/virt.c +2 -0
 static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                             DeviceState *dev, Error **errp)
 {
@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", smmuv3_dev_id++,
+                                     NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;

Eric

#82 lore raw

Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

In reply to Shameer Kolothum
Quoted 65 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.

No functional change intended.

---
 include/hw/arm/smmuv3.h  | 1 +
 hw/arm/smmuv3.c          | 2 ++
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 4 files changed, 9 insertions(+), 1 deletion(-)
include/hw/arm/smmuv3.h +1 -0
index 648412cafc..73b8f39aaa 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
hw/arm/smmuv3.c +2 -0
index 468135bb24..c1f84bedd4 100644
@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +2 -0
index ae78e9b9e0..20605185c5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)
 typedef struct AcpiIortSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
     GArray *rc_smmu_idmaps;
     /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
     size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms)
                                                &error_abort));
         sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
         sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
         pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
         sbdev = SYS_BUS_DEVICE(obj);
         sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);

hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 635

Quoted 5 lines
hw/arm/virt-acpi-build.c +1 -1
@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */

Hum actually I see that previously the id scope was shared by all nodes, including ITS, SMMU, RC.
So now can have its and smmu node having both id=0. And this is why you have a change in the IORT table for its=off test.
Is it legal?

IORT spec says:

Unique identifier for this node that can be used to locate it in the parent table. This identifier enables other ACPI tables and DSDT objects to locate this node. This field serves in the same capacity as the _UID object associated with ACPI device objects. Note IMPLEMENTATION NOTE: In the simplest scheme, the Identifier might be set to the node’s index in the array of IORT nodes in the parent IORT table. Other schemes are also possible and permitted.

So I don't think this is valid

Eric

Quoted 24 lines
hw/arm/virt-acpi-build.c +0 -0
         /* Number of ID mappings */
         build_append_int_noprefix(table_data, smmu_mapping_count, 4);
         /* Reference to ID Array */
hw/arm/virt.c +3 -0
index c75a8d6e9e..44c6b99c96 100644
@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev,
     }
 }
 
+static uint8_t smmuv3_dev_id;
 static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                             DeviceState *dev, Error **errp)
 {
@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev,
                                      OBJECT(vms->sysmem), NULL);
             object_property_set_link(OBJECT(dev), "secure-memory",
                                      OBJECT(vms->secure_sysmem), NULL);
+            object_property_set_uint(OBJECT(dev), "identifier", smmuv3_dev_id++,
+                                     NULL);
         }
         if (object_property_get_bool(OBJECT(dev), "accel", &error_abort)) {
             hwaddr db_start = 0;
#83 lore raw

RE: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

Quoted 94 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 18:23
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property

External email: Use caution opening links or attachments

Quoted 36 lines · Shameer Kolothum

Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.

This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.

No functional change intended.

---
 include/hw/arm/smmuv3.h  | 1 +
 hw/arm/smmuv3.c          | 2 ++
 hw/arm/virt-acpi-build.c | 4 +++-
 hw/arm/virt.c            | 3 +++
 4 files changed, 9 insertions(+), 1 deletion(-)
include/hw/arm/smmuv3.h +1 -0
index 648412cafc..73b8f39aaa 100644
@@ -63,6 +63,7 @@ struct SMMUv3State {
     qemu_irq     irq[4];
     QemuMutex mutex;
     char *stage;
+    uint8_t identifier;
 
     /* SMMU has HW accelerator support for nested S1 + s2 */
     bool accel;
hw/arm/smmuv3.c +0 -0
index 468135bb24..c1f84bedd4 100644

hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2112

Quoted 4 lines
hw/arm/smmuv3.c +1 -0
@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {
      * Defaults to stage 1
      */
     DEFINE_PROP_STRING("stage", SMMUv3State, stage),
+    /* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV)

generation */

hw/arm/virt-acpi-build.c static int iort_idmap_compare(gconstpointer a,

Quoted 9 lines
hw/arm/smmuv3.c +1 -0
+    DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
     DEFINE_PROP_BOOL("accel", SMMUv3State, accel, false),
     /* GPA of MSI doorbell, for SMMUv3 accel use. */
     DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +0 -0
index ae78e9b9e0..20605185c5 100644
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a,

gconstpointer b)

hw/arm/virt-acpi-build.c static int populate_smmuv3_dev(GArray

Quoted 8 lines
hw/arm/virt-acpi-build.c +1 -0
 typedef struct AcpiIortSMMUv3Dev {
     int irq;
     hwaddr base;
+    uint8_t id;
     GArray *rc_smmu_idmaps;
     /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
     size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray

*sdev_blob, VirtMachineState *vms)

hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker,

Quoted 8 lines
hw/arm/virt-acpi-build.c +1 -0
                                                &error_abort));
         sdev.accel = object_property_get_bool(obj, "accel", &error_abort);
         sdev.ats = object_property_get_bool(obj, "ats", &error_abort);
+        sdev.id = object_property_get_uint(obj, "identifier", &error_abort);
         pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
         sbdev = SYS_BUS_DEVICE(obj);
         sdev.base = platform_bus_get_mmio_addr(pbus, sbdev, 0);
@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker,

VirtMachineState *vms)

hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 635

Quoted 5 lines
hw/arm/virt-acpi-build.c +1 -1
                      (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);
         build_append_int_noprefix(table_data, node_size, 2); /* Length */
         build_append_int_noprefix(table_data, 4, 1); /* Revision */
-        build_append_int_noprefix(table_data, id++, 4); /* Identifier */
+        build_append_int_noprefix(table_data, sdev->id, 4); /* Identifier */
Quoted 4 lines

Hum actually I see that previously the id scope was shared by all nodes, including ITS, SMMU, RC.
So now can have its and smmu node having both id=0. And this is why you have a change in the IORT table for its=off test.

Hmm... I had that doubt as well. But I read it as a unique identifier for each type of IORT node.

Quoted 12 lines

Is it legal?

IORT spec says:

Unique identifier for this node that can be used to locate it in the parent table. This identifier enables other ACPI tables and DSDT objects to locate this node. This field serves in the same capacity as the _UID object associated with ACPI device objects. Note IMPLEMENTATION NOTE: In the simplest scheme, the Identifier might be set to the node’s index in the array of IORT nodes in the parent IORT table. Other schemes are also possible and permitted.

Quoted 1 line

So I don't think this is valid

I think we need another way to sort out the identifier then.

Thanks,
Shameer

#84 lore raw

PATCH v3 29/32 tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier change

In reply to Shameer Kolothum

Update the reference IORT blobs after identifier change for SMMUv3 nodes. This affects the aarch64 'virt' IORT tests.

IORT diff is the identifier change in SMMUv3 nodes:

/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20230628 (64-bit version)
* Copyright (c) 2000 - 2023 Intel Corporation
*
- * Disassembly of tests/data/acpi/aarch64/virt/IORT.its_off, Wed Feb 25 16:36:44 2026
+ * Disassembly of /tmp/aml-KVO1K3, Wed Feb 25 16:36:44 2026
*
* ACPI Data Table [IORT]
*
* Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue (in hex) */

[000h 0000 004h] Signature : "IORT" [IO Remapping Table]
[004h 0004 004h] Table Length : 000000AC

 [008h 0008 001h]                    Revision : 05
-[009h 0009 001h]                    Checksum : 95
+[009h 0009 001h]                    Checksum : 96

[00Ah 0010 006h] Oem ID : "BOCHS "
[010h 0016 008h] Oem Table ID : "BXPC "
[018h 0024 004h] Oem Revision : 00000001
[01Ch 0028 004h] Asl Compiler ID : "BXPC"
[020h 0032 004h] Asl Compiler Revision : 00000001

[024h 0036 004h] Node Count : 00000002
[028h 0040 004h] Node Offset : 00000030
[02Ch 0044 004h] Reserved : 00000000
...

[074h 0116 001h] Type : 02
[075h 0117 002h] Length : 0038
[077h 0119 001h] Revision : 03
-[078h 0120 004h] Identifier : 00000001
+[078h 0120 004h] Identifier : 00000000
[07Ch 0124 004h] Mapping Count : 00000001
[080h 0128 004h] Mapping Offset : 00000024

[084h 0132 008h] Memory Properties : [IORT Memory Access Properties]
[084h 0132 004h] Cache Coherency : 00000001
[088h 0136 001h] Hints (decoded below) : 00
Transient : 0
Write Allocate : 0
Read Allocate : 0
Override : 0
[089h 0137 002h] Reserved : 0000
[08Bh 0139 001h] Memory Flags (decoded below) : 03
Coherency : 1
Device Attribute : 1
[08Ch 0140 004h] ATS Attribute : 00000000
[090h 0144 004h] PCI Segment Number : 00000000
[094h 0148 001h] Memory Size Limit : 40
[095h 0149 002h] PASID Capabilities : 0000
[097h 0151 001h] Reserved : 00

[098h 0152 004h] Input base : 00000000
[09Ch 0156 004h] ID Count : 000000FF
[0wA0h 0160 004h] Output Base : 00000000
[0A4h 0164 004h] Output Reference : 00000030
[0A8h 0168 004h] Flags (decoded below) : 00000000
Single Mapping : 0

...

---
 tests/qtest/bios-tables-test-allowed-diff.h     |   4 ----
 tests/data/acpi/aarch64/virt/IORT.its_off       | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m    | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-dev    | Bin 260 -> 260 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes
 5 files changed, 4 deletions(-)
tests/qtest/bios-tables-test-allowed-diff.h +0 -4
index df5fe417c0..dfb8523c8b 100644
@@ -1,5 +1 @@
 /* List of comma-separated changed AML files to ignore */
-"tests/data/acpi/aarch64/virt/IORT.its_off",
-"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m",
-"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy",
-"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
tests/data/acpi/aarch64/virt/IORT.its_off binary +0 -0
index 0cf52b52f671637bf4dbc9e0fc80c3c73d0b01d3..295dc381063fa4c415a933c5c690ca0fc8ffd6dc 100644
GIT binary patch
tests/data/acpi/aarch64/virt/IORT.msi_gicv2m binary +0 -0
index 0cf52b52f671637bf4dbc9e0fc80c3c73d0b01d3..295dc381063fa4c415a933c5c690ca0fc8ffd6dc 100644
GIT binary patch
tests/data/acpi/aarch64/virt/IORT.smmuv3-dev binary +0 -0
index 60cfed1361976ef26b280c11ba2e233f1cfd9383..12f3e0f01864acf24fc86d4cde2de561aa92f878 100644
GIT binary patch
tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy binary +0 -0
index e2de4049f044bdfe5b3e656b94282ddf17832d9f..f940b696af2abcbb3b46fcc865acde7456aa3389 100644
GIT binary patch
Signature · 1 line
2.43.0
#85 lore raw

PATCH v3 30/32 hw/arm/smmuv3-accel: Introduce helper to query CMDQV type

In reply to Shameer Kolothum

Introduce a SMMUv3AccelCmdqvType enum and a helper to query the CMDQV implementation type associated with an accelerated SMMUv3 instance.

A subsequent patch will use this helper when generating the Tegra241 CMDQV DSDT.

---
 hw/arm/smmuv3-accel.h       |  7 +++++++
 hw/arm/smmuv3-accel-stubs.c |  5 +++++
 hw/arm/smmuv3-accel.c       | 12 ++++++++++++
 hw/arm/tegra241-cmdqv.c     |  6 ++++++
 4 files changed, 30 insertions(+)
hw/arm/smmuv3-accel.h +7 -0
index c349981e79..6d21788006 100644
@@ -15,6 +15,11 @@
 #include <linux/iommufd.h>
 #endif
 
+typedef enum SMMUv3AccelCmdqvType {
+    SMMUV3_CMDQV_NONE = 0,
+    SMMUV3_CMDQV_TEGRA241,
+} SMMUv3AccelCmdqvType;
+
 /*
  * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
  * support multiple VCMDQs with virtualization capabilities.
@@ -30,6 +35,7 @@ typedef struct SMMUv3AccelCmdqvOps {
     void (*free_viommu)(SMMUv3State *s);
     bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
     void (*free_veventq)(SMMUv3State *s);
+    SMMUv3AccelCmdqvType (*get_type)(void);
     void (*reset)(SMMUv3State *s);
 } SMMUv3AccelCmdqvOps;
 
@@ -73,5 +79,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
                                       void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +5 -0
index 1d5d3bb10c..5ca94d605f 100644
@@ -55,3 +55,8 @@ void smmuv3_accel_idr_override(SMMUv3State *s)
 void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    return SMMUV3_CMDQV_NONE;
+}
hw/arm/smmuv3-accel.c +12 -0
index 9a570b8af9..585b460943 100644
@@ -998,6 +998,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
 }
 
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    SMMUv3State *s = ARM_SMMUV3(obj);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) {
+        return SMMUV3_CMDQV_NONE;
+    }
+
+    return accel->cmdqv_ops->get_type();
+}
+
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp)
 {
     SMMUState *bs = ARM_SMMU(s);
hw/arm/tegra241-cmdqv.c +6 -0
index a379341c0a..42d7dbfde7 100644
@@ -736,6 +736,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type(void)
+{
+    return SMMUV3_CMDQV_TEGRA241;
+};
+
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
@@ -778,6 +783,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
     .free_viommu = tegra241_cmdqv_free_viommu,
     .alloc_veventq = tegra241_cmdqv_alloc_veventq,
     .free_veventq = tegra241_cmdqv_free_veventq,
+    .get_type = tegra241_cmdqv_get_type,
     .reset = tegra241_cmdqv_reset,
 };
 
Signature · 1 line
2.43.0
Reviewed-by on v3 30/32
#86 lore raw

Re: [PATCH v3 30/32] hw/arm/smmuv3-accel: Introduce helper to query CMDQV type

Quoted 8 lines · Shameer Kolothum

Introduce a SMMUv3AccelCmdqvType enum and a helper to query the CMDQV implementation type associated with an accelerated SMMUv3 instance.

A subsequent patch will use this helper when generating the Tegra241 CMDQV DSDT.

Eric

Quoted 98 lines
---
 hw/arm/smmuv3-accel.h       |  7 +++++++
 hw/arm/smmuv3-accel-stubs.c |  5 +++++
 hw/arm/smmuv3-accel.c       | 12 ++++++++++++
 hw/arm/tegra241-cmdqv.c     |  6 ++++++
 4 files changed, 30 insertions(+)
hw/arm/smmuv3-accel.h +7 -0
index c349981e79..6d21788006 100644
@@ -15,6 +15,11 @@
 #include <linux/iommufd.h>
 #endif
 
+typedef enum SMMUv3AccelCmdqvType {
+    SMMUV3_CMDQV_NONE = 0,
+    SMMUV3_CMDQV_TEGRA241,
+} SMMUv3AccelCmdqvType;
+
 /*
  * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
  * support multiple VCMDQs with virtualization capabilities.
@@ -30,6 +35,7 @@ typedef struct SMMUv3AccelCmdqvOps {
     void (*free_viommu)(SMMUv3State *s);
     bool (*alloc_veventq)(SMMUv3State *s,  Error **errp);
     void (*free_veventq)(SMMUv3State *s);
+    SMMUv3AccelCmdqvType (*get_type)(void);
     void (*reset)(SMMUv3State *s);
 } SMMUv3AccelCmdqvOps;
 
@@ -73,5 +79,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);
 bool smmuv3_accel_event_read_validate(IOMMUFDVeventq *veventq, uint32_t type,
                                       void *buf, size_t size, Error **errp);
 void smmuv3_accel_reset(SMMUv3State *s);
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj);
 
 #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +5 -0
index 1d5d3bb10c..5ca94d605f 100644
@@ -55,3 +55,8 @@ void smmuv3_accel_idr_override(SMMUv3State *s)
 void smmuv3_accel_reset(SMMUv3State *s)
 {
 }
+
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    return SMMUV3_CMDQV_NONE;
+}
hw/arm/smmuv3-accel.c +12 -0
index 9a570b8af9..585b460943 100644
@@ -998,6 +998,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)
     address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
 }
 
+SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type(Object *obj)
+{
+    SMMUv3State *s = ARM_SMMUV3(obj);
+    SMMUv3AccelState *accel = s->s_accel;
+
+    if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) {
+        return SMMUV3_CMDQV_NONE;
+    }
+
+    return accel->cmdqv_ops->get_type();
+}
+
 bool smmuv3_accel_init(SMMUv3State *s, Error **errp)
 {
     SMMUState *bs = ARM_SMMU(s);
hw/arm/tegra241-cmdqv.c +6 -0
index a379341c0a..42d7dbfde7 100644
@@ -736,6 +736,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
     return true;
 }
 
+static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type(void)
+{
+    return SMMUV3_CMDQV_TEGRA241;
+};
+
 static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
                                  Error **errp)
 {
@@ -778,6 +783,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
     .free_viommu = tegra241_cmdqv_free_viommu,
     .alloc_veventq = tegra241_cmdqv_alloc_veventq,
     .free_veventq = tegra241_cmdqv_free_veventq,
+    .get_type = tegra241_cmdqv_get_type,
     .reset = tegra241_cmdqv_reset,
 };
 
#87 lore raw

PATCH v3 31/32 hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

In reply to Shameer Kolothum

From: Nicolin Chen <[email protected]>

Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.

The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.

Because virt-acpi-build.c now includes CONFIG_DEVICES via the Tegra241 CMDQV header, the Meson file entry is updated to build it as part of arm_ss instead of arm_common_ss

---
 hw/arm/virt-acpi-build.c | 52 ++++++++++++++++++++++++++++++++++++++++
 hw/arm/trace-events      |  1 +
 2 files changed, 53 insertions(+)
hw/arm/virt-acpi-build.c +52 -0
index 20605185c5..462faa6a2b 100644
@@ -65,6 +65,9 @@
 #include "target/arm/cpu.h"
 #include "target/arm/multiprocessing.h"
 
+#include "smmuv3-accel.h"
+#include "tegra241-cmdqv.h"
+
 #define ARM_SPI_BASE 32
 
 #define ACPI_BUILD_TABLE_SIZE             0x20000
@@ -1113,6 +1116,51 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,
     build_fadt(table_data, linker, &fadt, vms->oem_id, vms->oem_table_id);
 }
 
+static void acpi_dsdt_add_tegra241_cmdqv(Aml *scope, VirtMachineState *vms)
+{
+    for (int i = 0; i < vms->smmuv3_devices->len; i++) {
+        Object *obj = OBJECT(g_ptr_array_index(vms->smmuv3_devices, i));
+        PlatformBusDevice *pbus;
+        Aml *dev, *crs, *addr;
+        SysBusDevice *sbdev;
+        hwaddr base;
+        uint32_t id;
+        int irq;
+
+        if (smmuv3_accel_cmdqv_type(obj) != SMMUV3_CMDQV_TEGRA241) {
+            continue;
+        }
+        id = object_property_get_uint(obj, "identifier", &error_abort);
+        pbus = PLATFORM_BUS_DEVICE(vms->platform_bus_dev);
+        sbdev = SYS_BUS_DEVICE(obj);
+        base = platform_bus_get_mmio_addr(pbus, sbdev, 1);
+        base += vms->memmap[VIRT_PLATFORM_BUS].base;
+        irq = platform_bus_get_irqn(pbus, sbdev, NUM_SMMU_IRQS);
+        irq += vms->irqmap[VIRT_PLATFORM_BUS];
+        irq += ARM_SPI_BASE;
+
+        dev = aml_device("CV%.02u", id);
+        aml_append(dev, aml_name_decl("_HID", aml_string("NVDA200C")));
+        aml_append(dev, aml_name_decl("_UID", aml_int(id)));
+        aml_append(dev, aml_name_decl("_CCA", aml_int(1)));
+
+        crs = aml_resource_template();
+        addr = aml_qword_memory(AML_POS_DECODE, AML_MIN_FIXED, AML_MAX_FIXED,
+                                AML_CACHEABLE, AML_READ_WRITE, 0x0, base,
+                                base + TEGRA241_CMDQV_IO_LEN - 0x1, 0x0,
+                                TEGRA241_CMDQV_IO_LEN);
+        aml_append(crs, addr);
+        aml_append(crs, aml_interrupt(AML_CONSUMER, AML_EDGE,
+                                      AML_ACTIVE_HIGH, AML_EXCLUSIVE,
+                                      (uint32_t *)&irq, 1));
+        aml_append(dev, aml_name_decl("_CRS", crs));
+
+        aml_append(scope, dev);
+
+        trace_virt_acpi_dsdt_tegra241_cmdqv(id, base, irq);
+    }
+}
+
 /* DSDT */
 static void
 build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
@@ -1177,6 +1225,10 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
     acpi_dsdt_add_tpm(scope, vms);
 #endif
 
+    if (!vms->legacy_smmuv3_present) {
+        acpi_dsdt_add_tegra241_cmdqv(scope, vms);
+    }
+
     aml_append(dsdt, scope);
 
     pci0_scope = aml_scope("\\_SB.PCI0");
hw/arm/trace-events +1 -0
index ef495c040c..e7e3ccfe9f 100644
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s"
 
 # virt-acpi-build.c
 virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out."
+virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d"
 
 # smmu-common.c
 smmu_add_mr(const char *name) "%s"
Signature · 1 line
2.43.0
#88 lore raw

PATCH v3 32/32 hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

In reply to Shameer Kolothum

Introduce a “cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.

---
 hw/arm/smmuv3.c | 7 +++++++
 1 file changed, 7 insertions(+)
hw/arm/smmuv3.c +7 -0
index c1f84bedd4..70135af015 100644
@@ -1967,6 +1967,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)
             error_setg(errp, "ssidsize can only be set if accel=on");
             return false;
         }
+        if (s->cmdqv == ON_OFF_AUTO_ON) {
+            error_setg(errp, "cmdqv can only be enabled if accel=on");
+            return false;
+        }
         return true;
     }
 
@@ -2119,6 +2123,7 @@ static const Property smmuv3_properties[] = {
     DEFINE_PROP_BOOL("ats", SMMUv3State, ats, false),
     DEFINE_PROP_UINT8("oas", SMMUv3State, oas, 44),
     DEFINE_PROP_UINT8("ssidsize", SMMUv3State, ssidsize, 0),
+    DEFINE_PROP_ON_OFF_AUTO("cmdqv", SMMUv3State, cmdqv, ON_OFF_AUTO_AUTO),
 };
 
 static void smmuv3_instance_init(Object *obj)
@@ -2158,6 +2163,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)
         "Valid range is 0-20, where 0 disables SubstreamID support. "
         "Defaults to 0. A value greater than 0 is required to enable "
         "PASID support.");
+    object_class_property_set_description(klass, "cmdqv",
+        "Enable/disable CMDQ-Virtualisation support (for accel=on)");
 }
 
 static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
#89 lore raw
In reply to Shameer Kolothum

Hi Shameer,

Quoted 25 lines · Shameer Kolothum

Hi,

Changes from v2:
https://lore.kernel.org/qemu-devel/[email protected]/

- Addressed feedback on v2 and picked up R-by tags. Thanks!
- Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto

    "cmdqv".
  - Introduced a probe() callback to detect and initialize CMDQV
    support during device attach (patch #7).
  - Since CMDQV initialization is now deferred to device attach time,
    added a helper to link any unmapped MMIO/IRQ resources (patch #12).
  - Moved VINTF enable handling to the CONFIG ENABLE bit write path
    instead of any read/write access path (patch #17). Hopefully this
    won't break any corner cases.
  - Added checks for VINTF and CMDQV ENABLE bits before allocating
    vCMDQ HW queues, and free any allocated resources if disabled
    (patch #19).
  - Introduced a common helper for vEVENTQ read and validation
    (patch #23).
  - Updated bios-tables test IORT blobs for SMMUv3 device identifier
    changes (patch #29).
  - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
    using tegra241-cmdqv-stubs.c (patch #6).
Quoted 2 lines

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcmdq-v3

with this version I still get those traces:

iommufd_backend_map_dma  iommufd=28 ioas=2 iova=0xc0b0000 size=0x10000 addr=0xffff0bd30000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?
qemu-system-aarch64: warning: vfio_container_dma_map(0xaaaea11e2870, 0xc0b0000, 0x10000, 0xffff0bd30000) = -14 (Bad address)

Eric

Quoted 107 lines

I have sanity tested this on an NVIDIA GRACE platform and will continue with additional tests.

Feedback and testing are very welcome.

Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/

---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/[email protected]/

Thanks to Nicolin for the initial patches and testing on which this is based.

Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.

Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)

Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.

VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].

---

Nicolin Chen (15):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free hw/arm/tegra241-cmdqv: Emulate global CMDQV registers hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT

Shameer Kolothum (17):
hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle hw/arm/virt: Store SMMUv3 device objects in VirtMachineState. hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus system/physmem: Add address_space_is_ram() helper hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free hw/arm/smmuv3-accel: Introduce common helper for veventq read hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier
change
hw/arm/smmuv3: Add per-device identifier property tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier
change
hw/arm/smmuv3-accel: Introduce helper to query CMDQV type hw/arm/smmuv3: Add cmdqv property for SMMUv3 device

 hw/arm/smmuv3-accel.h                         |  29 +
 hw/arm/tegra241-cmdqv.h                       | 342 ++++++++
 include/hw/arm/smmuv3.h                       |   3 +
 include/hw/arm/virt.h                         |   1 +
 include/system/iommufd.h                      |  16 +
 include/system/memory.h                       |  10 +
 backends/iommufd.c                            |  64 ++
 hw/arm/smmuv3-accel-stubs.c                   |  11 +
 hw/arm/smmuv3-accel.c                         | 196 ++++-
 hw/arm/smmuv3.c                               |   9 +
 hw/arm/tegra241-cmdqv-stubs.c                 |  18 +
 hw/arm/tegra241-cmdqv.c                       | 793 ++++++++++++++++++
 hw/arm/virt-acpi-build.c                      | 126 ++-
 hw/arm/virt.c                                 |  27 +
 hw/vfio/iommufd.c                             |   4 +-
 system/physmem.c                              |  11 +
 backends/trace-events                         |   4 +-
 hw/arm/Kconfig                                |   5 +
 hw/arm/meson.build                            |   2 +
 hw/arm/trace-events                           |   5 +
 tests/data/acpi/aarch64/virt/IORT.its_off     | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.msi_gicv2m  | Bin 172 -> 172 bytes
 tests/data/acpi/aarch64/virt/IORT.smmuv3-dev  | Bin 260 -> 260 bytes
 .../data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes
 24 files changed, 1596 insertions(+), 80 deletions(-)
 create mode 100644 hw/arm/tegra241-cmdqv.h
 create mode 100644 hw/arm/tegra241-cmdqv-stubs.c
 create mode 100644 hw/arm/tegra241-cmdqv.c
#90 lore raw
Quoted 57 lines

-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 18:25
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

External email: Use caution opening links or attachments

Hi Shameer,

Quoted 5 lines · Shameer Kolothum

Hi,

Changes from v2:

https://lore.kernel.org/qemu-devel/20260206144823.80655-1-

skolothumtho

Quoted 22 lines

@nvidia.com/

- Addressed feedback on v2 and picked up R-by tags. Thanks!
- Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto

    "cmdqv".
  - Introduced a probe() callback to detect and initialize CMDQV
    support during device attach (patch #7).
  - Since CMDQV initialization is now deferred to device attach time,
    added a helper to link any unmapped MMIO/IRQ resources (patch #12).
  - Moved VINTF enable handling to the CONFIG ENABLE bit write path
    instead of any read/write access path (patch #17). Hopefully this
    won't break any corner cases.
  - Added checks for VINTF and CMDQV ENABLE bits before allocating
    vCMDQ HW queues, and free any allocated resources if disabled
    (patch #19).
  - Introduced a common helper for vEVENTQ read and validation
    (patch #23).
  - Updated bios-tables test IORT blobs for SMMUv3 device identifier
    changes (patch #29).
  - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
    using tegra241-cmdqv-stubs.c (patch #6).
Quoted 2 lines

Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-

vcm

Quoted 1 line

dq-v3

with this version I still get those traces:

Quoted 7 lines

iommufd_backend_map_dma iommufd=28 ioas=2 iova=0xc0b0000 size=0x10000
addr=0xffff0bd30000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?
qemu-system-aarch64: warning: vfio_container_dma_map(0xaaaea11e2870, 0xc0b0000, 0x10000, 0xffff0bd30000) = -14 (Bad address)

Yes. Please see my reply to Nicolin on v2:
https://lore.kernel.org/qemu-devel/CH3PR12MB7548C2DCA4D321D7FDE23754AB6CA@CH3PR12MB7548.namprd12.prod.outlook.com/

I do not have a way to suppress that warning currently. It should be safe to ignore, similar to the behaviour before the dma-buf support series for vfio-pci PCI BAR mappings.

Thanks,
Shameer

#91 lore raw
Quoted 3 lines · Shameer Kolothum Thodi

I do not have a way to suppress that warning currently. It should be safe to ignore, similar to the behaviour before the dma-buf support series for vfio-pci PCI BAR mappings.

qemu should have the concept of memory regions that are not available for DMA peer to peer operation - ie are not mapped to the iommu.

Now that the kernel strictly enforces this it is essential qemu itself keep track if it thinks p2p should work or not.

Jason

#92 lore raw
Quoted 21 lines

-----Original Message-----
From: Jason Gunthorpe <[email protected]>
Sent: 11 March 2026 20:00
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3

Quoted 3 lines · Shameer Kolothum Thodi

I do not have a way to suppress that warning currently. It should be safe to ignore, similar to the behaviour before the dma-buf support series for vfio-pci PCI BAR mappings.

Quoted 5 lines

qemu should have the concept of memory regions that are not available for DMA peer to peer operation - ie are not mapped to the iommu.

Now that the kernel strictly enforces this it is essential qemu itself keep track if it thinks p2p should work or not.

We could introduce something like the below to address this. I will include it in the next respin.

Thanks,
Shameer

---
 include/system/memory.h | 2 ++
 hw/arm/tegra241-cmdqv.c | 1 +
 hw/vfio/listener.c      | 5 +++++
 3 files changed, 8 insertions(+)
include/system/memory.h +2 -0
index 02b2e83fd7..fb37616503 100644
@@ -874,6 +874,8 @@ struct MemoryRegion {
 
     /* For devices designed to perform re-entrant IO into their own IO MRs */
     bool disable_reentrancy_guard;
+    /* RAM device region that does not require IOMMU mapping for P2P */
+    bool ram_device_skip_iommu_map;
 };
 
 struct IOMMUMemoryRegion {
hw/arm/tegra241-cmdqv.c +1 -0
index 42d7dbfde7..4647b4db54 100644
@@ -212,6 +212,7 @@ static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)
                                       memory_region_owner(&cmdqv->mmio_cmdqv),
                                       name, VINTF_PAGE_SIZE,
                                       cmdqv->vintf_page0);
+    cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;
     memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv, 0x30000,
                                         cmdqv->mr_vintf_page0, 1);
     g_free(name);
hw/vfio/listener.c +5 -0
index 1087fdc142..d47439befb 100644
@@ -614,6 +614,11 @@ void vfio_container_region_add(VFIOContainer *bcontainer,
         }
     }
 
+    if (memory_region_is_ram_device(section->mr) &&
+        section->mr->ram_device_skip_iommu_map) {
+        return;
+    }
+
     ret = vfio_container_dma_map(bcontainer, iova, int128_get64(llsize),
                                  vaddr, section->readonly, section->mr);
     if (ret) {

--
2.43.0