hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3
Subsystems (2)
v3 cover hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3
Hi,
Changes from v2:
https://lore.kernel.org/qemu-devel/[email protected]/
- Addressed feedback on v2 and picked up R-by tags. Thanks!
- Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto
"cmdqv".
- Introduced a probe() callback to detect and initialize CMDQV
support during device attach (patch #7).
- Since CMDQV initialization is now deferred to device attach time,
added a helper to link any unmapped MMIO/IRQ resources (patch #12).
- Moved VINTF enable handling to the CONFIG ENABLE bit write path
instead of any read/write access path (patch #17). Hopefully this
won't break any corner cases.
- Added checks for VINTF and CMDQV ENABLE bits before allocating
vCMDQ HW queues, and free any allocated resources if disabled
(patch #19).
- Introduced a common helper for vEVENTQ read and validation
(patch #23).
- Updated bios-tables test IORT blobs for SMMUv3 device identifier
changes (patch #29).
- Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0],
using tegra241-cmdqv-stubs.c (patch #6).Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcmdq-v3
I have sanity tested this on an NVIDIA GRACE platform and will continue with additional tests.
Feedback and testing are very welcome.
Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/
---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/[email protected]/
Thanks to Nicolin for the initial patches and testing on which this is based.
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)
Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].
---
Nicolin Chen (15):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free hw/arm/tegra241-cmdqv: Emulate global CMDQV registers hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Shameer Kolothum (17):
hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle hw/arm/virt: Store SMMUv3 device objects in VirtMachineState. hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus system/physmem: Add address_space_is_ram() helper hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free hw/arm/smmuv3-accel: Introduce common helper for veventq read hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier
change
hw/arm/smmuv3: Add per-device identifier property tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier
change
hw/arm/smmuv3-accel: Introduce helper to query CMDQV type hw/arm/smmuv3: Add cmdqv property for SMMUv3 device
hw/arm/smmuv3-accel.h | 29 + hw/arm/tegra241-cmdqv.h | 342 ++++++++ include/hw/arm/smmuv3.h | 3 + include/hw/arm/virt.h | 1 + include/system/iommufd.h | 16 + include/system/memory.h | 10 + backends/iommufd.c | 64 ++ hw/arm/smmuv3-accel-stubs.c | 11 + hw/arm/smmuv3-accel.c | 196 ++++- hw/arm/smmuv3.c | 9 + hw/arm/tegra241-cmdqv-stubs.c | 18 + hw/arm/tegra241-cmdqv.c | 793 ++++++++++++++++++ hw/arm/virt-acpi-build.c | 126 ++- hw/arm/virt.c | 27 + hw/vfio/iommufd.c | 4 +- system/physmem.c | 11 + backends/trace-events | 4 +- hw/arm/Kconfig | 5 + hw/arm/meson.build | 2 + hw/arm/trace-events | 5 + tests/data/acpi/aarch64/virt/IORT.its_off | Bin 172 -> 172 bytes tests/data/acpi/aarch64/virt/IORT.msi_gicv2m | Bin 172 -> 172 bytes tests/data/acpi/aarch64/virt/IORT.smmuv3-dev | Bin 260 -> 260 bytes .../data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes 24 files changed, 1596 insertions(+), 80 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644 hw/arm/tegra241-cmdqv-stubs.c create mode 100644 hw/arm/tegra241-cmdqv.c
Signature · 1 line
2.43.0
PATCH v3 1/32 backends/iommufd: Update iommufd_backend_get_device_info
From: Nicolin Chen <[email protected]>
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.
--- backends/iommufd.c | 7 +++++++ hw/arm/smmuv3-accel.c | 2 +- hw/vfio/iommufd.c | 4 ++-- 3 files changed, 10 insertions(+), 3 deletions(-)
backends/iommufd.c +7 -0
@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,return true; } +/* +* @type can carry a desired HW info type defined in the uapi headers. If caller +* doesn't have one, indicating it wants the default type, then @type should be +* zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT). +*/ bool iommufd_backend_get_device_info (IOMMUFDBackend *be,uint32_t devid,uint32_t *type,void *data,uint32_t len,uint64_t *caps,uint8_t *max_pasid_log2,Error **errp) {struct iommu_hw_info info = { + .flags = (*type) ?IOMMU_HW_INFO_FLAG_INPUT_TYPE :0 , .size =sizeof (info), .dev_id = devid, .data_len = len, .data_uptr = (uintptr_t )data, + .in_data_type = *type, };if (ioctl (be->fd,IOMMU_GET_HW_INFO , &info)) {
hw/arm/smmuv3-accel.c +1 -1
@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) { struct iommu_hw_info_arm_smmuv3 info; - uint32_t data_type; + uint32_t data_type = IOMMU_HW_INFO_TYPE_DEFAULT; uint64_t caps; if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +2 -2
@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,ERRP_GUARD ();IOMMUFDBackend *iommufd = vbasedev->iommufd;VFIOContainer *bcontainer =VFIO_IOMMU (container); -uint32_t type, flags =0 ; +uint32_t type =IOMMU_HW_INFO_TYPE_DEFAULT , flags =0 ;uint64_t hw_caps;VendorCaps caps;VFIOIOASHwpt *hwpt;
@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,HostIOMMUDeviceIOMMUFD *idev;HostIOMMUDeviceCaps *caps = &hiod->caps;VendorCaps *vendor_caps = &caps->vendor_caps; -enum iommu_hw_info_type type; +uint32_t type =IOMMU_HW_INFO_TYPE_DEFAULT ;uint8_t max_pasid_log2;uint64_t hw_caps;
Signature · 1 line
2.43.0
PATCH v3 2/32 backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.
Update the caller accordingly.
--- include/system/iommufd.h | 1 + backends/iommufd.c | 4 ++++ hw/arm/smmuv3-accel.c | 4 ++-- backends/trace-events | 2 +- 4 files changed, 8 insertions(+), 3 deletions(-)
include/system/iommufd.h +1 -0
@@ -89,6 +89,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id, Error **errp);bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t data_len,uint32_t *out_hwpt,Error **errp); bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
backends/iommufd.c +4 -0
@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t data_len,uint32_t *out_viommu_id,Error **errp) {int ret;
@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, .type = viommu_type, .dev_id = dev_id, .hwpt_id = hwpt_id, + .data_len = data_len, + .data_uptr = (uintptr_t)data_ptr, }; ret =ioctl (be->fd,IOMMU_VIOMMU_ALLOC , &alloc_viommu);trace_iommufd_backend_alloc_viommu (be->fd, dev_id, viommu_type, hwpt_id, + (uintptr_t )data_ptr, data_len, alloc_viommu.out_viommu_id, ret);if (ret) {error_setg_errno (errp, errno,"IOMMU_VIOMMU_ALLOC failed" );
hw/arm/smmuv3-accel.c +2 -2
@@ -533,8 +533,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,IOMMUFDViommu *viommu;if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , - s2_hwpt_id, &viommu_id, errp)) { +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, +NULL ,0 , &viommu_id, errp)) {return false; }
backends/trace-events +1 -1
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (% iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)" iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)" iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)" -iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)" +iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
Signature · 1 line
2.43.0
PATCH v3 3/32 backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add a helper to allocate an iommufd backed HW queue for a vIOMMU.
While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.
--- include/system/iommufd.h | 11 +++++++++++ backends/iommufd.c | 31 +++++++++++++++++++++++++++++++ backends/trace-events | 1 + 3 files changed, 43 insertions(+)
include/system/iommufd.h +11 -0
@@ -65,6 +65,12 @@ typedef struct IOMMUFDVeventq {bool event_start;/* True after first valid event; cleared on overflow */ } IOMMUFDVeventq; +/* HW queue object for a vIOMMU-specific HW-accelerated queue */ +typedef struct IOMMUFDHWqueue { +IOMMUFDViommu *viommu; +uint32_t hw_queue_id; +}IOMMUFDHWqueue ; +bool iommufd_backend_connect (IOMMUFDBackend *be,Error **errp);void iommufd_backend_disconnect (IOMMUFDBackend *be);
@@ -101,6 +107,11 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,uint32_t *out_veventq_id, uint32_t *out_veventq_fd, Error **errp); +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +uint32_t queue_type,uint32_t index, +uint64_t addr,uint64_t length, +uint32_t *out_hw_queue_id,Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
backends/iommufd.c +31 -0
@@ -546,6 +546,37 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +uint32_t queue_type,uint32_t index, +uint64_t addr,uint64_t length, +uint32_t *out_hw_queue_id,Error **errp) +{ +int ret; +struct iommu_hw_queue_alloc alloc_hw_queue = { + .size =sizeof (alloc_hw_queue), + .flags =0 , + .viommu_id = viommu_id, + .type = queue_type, + .index = index, + .nesting_parent_iova = addr, + .length = length, + }; + + ret =ioctl (be->fd,IOMMU_HW_QUEUE_ALLOC , &alloc_hw_queue); + +trace_iommufd_backend_alloc_hw_queue (be->fd, viommu_id, queue_type, + index, addr, length, + alloc_hw_queue.out_hw_queue_id, ret); +if (ret) { +error_setg_errno (errp, errno,"IOMMU_HW_QUEUE_ALLOC failed" ); +return false; + } + +g_assert (out_hw_queue_id); + *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id; +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {
backends/trace-events +1 -0
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint64_t data_ptr, uint32_t data_len, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_ptr=0x%"PRIx64" data_len=0x%x viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" +iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)" # igvm-cfg.c igvm_reset_enter(int type) "type=%u"
Signature · 1 line
2.43.0
PATCH v3 4/32 backends/iommufd: Introduce iommufd_backend_viommu_mmap
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.
The caller is responsible for unmapping the returned region.
--- include/system/iommufd.h | 4 ++++ backends/iommufd.c | 22 ++++++++++++++++++++++ backends/trace-events | 1 + 3 files changed, 27 insertions(+)
include/system/iommufd.h +4 -0
@@ -112,6 +112,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, uint64_t addr, uint64_t length, uint32_t *out_hw_queue_id, Error **errp); +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
backends/iommufd.c +22 -0
@@ -577,6 +577,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +/* +* Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance. +* The caller is responsible for unmapping the mapped region. +*/ +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp) +{ +g_assert (viommu_id); +g_assert (out_ptr); + + *out_ptr =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , be->fd, + offset); +trace_iommufd_backend_viommu_mmap (be->fd, viommu_id, size, offset); +if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"IOMMUFD vIOMMU mmap failed" ); +return false; + } + +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {
backends/trace-events +1 -0
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)" +iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64 # igvm-cfg.c igvm_reset_enter(int type) "type=%u"
Signature · 1 line
2.43.0
PATCH v3 5/32 hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer KolothumCommand Queue Virtualization (CMDQV) is a hardware extension available on certain platforms that allows the SMMUv3 command queue to be virtualized and passed through to a VM, improving performance.
For example, NVIDIA Tegra241 implements CMDQV to support virtualization of multiple command queues (VCMDQs).
The term CMDQV is used here generically to refer to any platform that provides hardware support to virtualize the SMMUv3 command queue.
CMDQV support is a specialization of the IOMMUFD-backed accelerated SMMUv3 path. Introduce an ops interface to factor out CMDQV-specific probe, initialization, and vIOMMU allocation logic from the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
This provides an extensible design to support future vendor-specific CMDQV implementations.
No functional change.
--- hw/arm/smmuv3-accel.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
hw/arm/smmuv3-accel.h +17 -0
@@ -15,6 +15,22 @@#include <linux/iommufd.h> #endif +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to +* support multiple VCMDQs with virtualization capabilities. +* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { +bool (*probe)(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,Error **errp); +bool (*init)(SMMUv3State *s,Error **errp); +bool (*alloc_viommu)(SMMUv3State *s, +HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id, +Error **errp); +void (*free_viommu)(SMMUv3State *s); +void (*reset)(SMMUv3State *s); +}SMMUv3AccelCmdqvOps ; +/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object. * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -25,6 +41,7 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; } SMMUv3AccelState; typedef struct SMMUS1Hwpt {
Signature · 1 line
2.43.0
PATCH v3 6/32 hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
In reply to Shameer KolothumIntroduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.
Functional CMDQV support is added in follow-up patches.
--- hw/arm/tegra241-cmdqv.h | 15 ++++++++++ hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++ hw/arm/tegra241-cmdqv.c | 56 +++++++++++++++++++++++++++++++++++ hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 2 ++ 5 files changed, 96 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644 hw/arm/tegra241-cmdqv-stubs.c create mode 100644 hw/arm/tegra241-cmdqv.c
hw/arm/tegra241-cmdqv.h new file +15 -0
@@ -0,0 +1,15 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_ARM_TEGRA241_CMDQV_H +#define HW_ARM_TEGRA241_CMDQV_H + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ); + +#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv-stubs.c new file +18 -0
@@ -0,0 +1,18 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* +* Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "hw/arm/tegra241-cmdqv.h" + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) +{ +return NULL ; +} +
hw/arm/tegra241-cmdqv.c new file +56 -0
56 lines not on this page · raw message
hw/arm/Kconfig +5 -0
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config ARM_SMMUV3 bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL bool
hw/arm/meson.build +2 -0
@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) stub_ss.add(files('smmuv3-accel-stubs.c')) +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) +stub_ss.add(files('tegra241-cmdqv-stubs.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
Signature · 1 line
2.43.0
Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
Hi Shameer,
Quoted 48 lines · Shameer Kolothum
Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.
Functional CMDQV support is added in follow-up patches.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 15 ++++++++++ hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++ hw/arm/tegra241-cmdqv.c | 56 +++++++++++++++++++++++++++++++++++ hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 2 ++ 5 files changed, 96 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644 hw/arm/tegra241-cmdqv-stubs.c create mode 100644 hw/arm/tegra241-cmdqv.chw/arm/tegra241-cmdqv.h new file +15
-0@@ -0,0 +1,15 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_ARM_TEGRA241_CMDQV_H +#define HW_ARM_TEGRA241_CMDQV_H + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ); + +#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv-stubs.c new file +0
-0
hw/arm/tegra241-cmdqv-stubs.c line 4
Quoted 4 lines
hw/arm/tegra241-cmdqv-stubs.c new file +4
-0@@ -0,0 +1,18 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* +* Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3
^typo
Quoted 60 lines
hw/arm/tegra241-cmdqv-stubs.c new file +14
-0+* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "hw/arm/tegra241-cmdqv.h" + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) +{ +return NULL ; +} +hw/arm/tegra241-cmdqv.c new file +40
-0@@ -0,0 +1,56 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) +{ +} + +static bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static void tegra241_cmdqv_reset (SMMUv3State *s) +{ +} + +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static bool tegra241_cmdqv_probe (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +Error **errp) +{
hw/arm/tegra241-cmdqv.c line 50
Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10
-0+error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = { + .probe = tegra241_cmdqv_probe, + .init = tegra241_cmdqv_init, + .alloc_viommu = tegra241_cmdqv_alloc_viommu, + .free_viommu = tegra241_cmdqv_free_viommu, + .reset = tegra241_cmdqv_reset,
I think I would create a no_cmdqv_ops to convert the existing code without any cmdqv support to the same api. This would avoid the repeating:
if (cmdqv_ops && cmdqv_ops-><func>) check in subsequent patches.
Also this looks a bit strange to have different signatures for the functions used with both vcmdq on and off
Eric
Quoted 42 lines
hw/arm/tegra241-cmdqv.c new file +6
-0+}; + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) +{ +return &tegra241_cmdqv_ops; +}hw/arm/Kconfig +5
-0@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config ARM_SMMUV3 bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL boolhw/arm/meson.build +2
-0@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) stub_ss.add(files('smmuv3-accel-stubs.c')) +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) +stub_ss.add(files('tegra241-cmdqv-stubs.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
RE: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
Hi Eric,
Quoted 140 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 09:18
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stubExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 7 lines · Shameer Kolothum
Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.
Quoted 7 lines
Functional CMDQV support is added in follow-up patches.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 15 ++++++++++ hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++ hw/arm/tegra241-cmdqv.c | 56+++++++++++++++++++++++++++++++++++
Quoted 33 lines
hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 2 ++ 5 files changed, 96 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644
hw/arm/tegra241-cmdqv-stubs.ccreate mode 100644hw/arm/tegra241-cmdqv.chw/arm/tegra241-cmdqv.h new renamed from hw/arm/tegra241-cmdqv.h +0
-0file mode 100644 index 0000000000..07e10e86ee
hw/arm/tegra241-cmdqv.h new file +15
-0@@ -0,0 +1,15 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All +rights reserved +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef HW_ARM_TEGRA241_CMDQV_H +#define HW_ARM_TEGRA241_CMDQV_H + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ); + +#endif /* HW_ARM_TEGRA241_CMDQV_H */ (quoted diff) +0
-0
b/hw/arm/tegra241-cmdqv-stubs.cnew file mode 100644 index 0000000000..eedc7bfdcdhw/arm/tegra241-cmdqv-stubs.c line 5
Quoted 5 lines
hw/arm/tegra241-cmdqv-stubs.c new file +5
-0@@ -0,0 +1,18 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All +rights reserved +* +* Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3 ^typo
Quoted 26 lines
hw/arm/tegra241-cmdqv-stubs.c new file +12
-0+* +* SPDX-License-Identifier: GPL-2.0-or-later */ + +#include "qemu/osdep.h" +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "hw/arm/tegra241-cmdqv.h" + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) { +return NULL ; +} +hw/arm/tegra241-cmdqv.c new renamed from hw/arm/tegra241-cmdqv.c +0
-0file mode 100644 index 0000000000..ad5a0d4611
hw/arm/tegra241-cmdqv.c new file +9
-0@@ -0,0 +1,56 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All +rights reserved +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later */ +hw/arm/tegra241-cmdqv.c line 19
Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10
-0+#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { } + +static bool +tegra241_cmdqv_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c
Quoted 13 lines
hw/arm/tegra241-cmdqv.c new file +13
-0+ uint32_t *out_viommu_id, Error **errp) { + error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported"); +return false; +} + +static void tegra241_cmdqv_reset (SMMUv3State *s) { } + +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) { +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static bool tegra241_cmdqv_probe(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c line 50
Quoted 11 lines
hw/arm/tegra241-cmdqv.c new file +11
-0+ Error **errp) { +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = { + .probe = tegra241_cmdqv_probe, + .init = tegra241_cmdqv_init, + .alloc_viommu = tegra241_cmdqv_alloc_viommu, + .free_viommu = tegra241_cmdqv_free_viommu, + .reset = tegra241_cmdqv_reset,I think I would create a
no_cmdqv_opsto convert the existing code without any cmdqv support to the same api. This would avoid the repeating:
Quoted 1 line
if (
cmdqv_ops&& cmdqv_ops-><func>) check in subsequent patches.
I guess you are suggesting having something like,
static const SMMUv3AccelCmdqvOps no_cmdqv_ops = {
.probe = no_cmdqv_probe,
.alloc_viommu = no_cmdqv_alloc_viommu,
...
};
static bool no_cmdqv_probe(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev, Error **errp)
{
return true;
}static bool no_cmdqv_alloc_viommu(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev,
uint32_t *id,
Error **errp){
return false; /* We have to call the default vIOMMU alloc*/
}
And then call them directly. I will take a look at that approach.
Quoted 3 lines
Also this looks a bit strange to have different signatures for the functions used with both vcmdq on and off
Is the concern here is mainly around alloc_viommu( that is the one which require fallback to default vIOMMU alloc)?
static bool
smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Error **errp)
and
bool (*alloc_viommu)(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev,
uint32_t *out_viommu_id,
Error **errp);Making those consistent would likely require some refactoring of the existing path. I will have a play with that and see.
Thanks,
Shameer
Quoted 50 lines
Eric
hw/arm/Kconfig config ARM_SMMUV3_ACCEL config
Quoted 21 lines
hw/arm/tegra241-cmdqv.c new file +5
-0+}; + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) { +return &tegra241_cmdqv_ops; +}hw/arm/Kconfig index renamed from hw/arm/Kconfig +0
-0
c66c452737..3305c6e76e100644hw/arm/Kconfig +4
-0@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config
ARM_SMMUV3hw/arm/meson.build arm_common_ss.add(when:
Quoted 11 lines
hw/arm/Kconfig +1
-0bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL boolhw/arm/meson.build +0
-0@@ -88,6 +88,8 @@ arm_common_ss.add(when:'
CONFIG_FSL_IMX8MP_EVK',if_true: files('imx8mp-evk.c'))hw/arm/meson.build arm_common_ss.add(when: · line 88
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true:files('smmuv3.c'))
hw/arm/meson.build arm_common_ss.add(when: · line 89
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:files('smmuv3-accel.c'))
hw/arm/meson.build arm_common_ss.add(when: · line 91
Quoted 2 lines
hw/arm/meson.build +1
-0stub_ss.add(files('smmuv3-accel-stubs.c')) +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true:files('tegra241-cmdqv.c'))
hw/arm/meson.build arm_common_ss.add(when: · line 93
Quoted 2 lines
hw/arm/meson.build +1
-0+stub_ss.add(files('tegra241-cmdqv-stubs.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-
imx6ul.c', 'mcimx6ul-evk.c'))hw/arm/meson.build arm_common_ss.add(when: · line 94
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:
files('nrf51_soc.c'))Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
Hi Shameer,
Quoted 158 lines · Shameer Kolothum Thodi
Hi Eric,
Quoted 139 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 09:18
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stubExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 7 lines · Shameer Kolothum
Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.
Quoted 7 lines
Functional CMDQV support is added in follow-up patches.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 15 ++++++++++ hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++ hw/arm/tegra241-cmdqv.c | 56+++++++++++++++++++++++++++++++++++
Quoted 33 lines
hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 2 ++ 5 files changed, 96 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644
hw/arm/tegra241-cmdqv-stubs.ccreate mode 100644hw/arm/tegra241-cmdqv.chw/arm/tegra241-cmdqv.h new renamed from hw/arm/tegra241-cmdqv.h +0
-0file mode 100644 index 0000000000..07e10e86ee
hw/arm/tegra241-cmdqv.h new file +15
-0@@ -0,0 +1,15 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All +rights reserved +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef HW_ARM_TEGRA241_CMDQV_H +#define HW_ARM_TEGRA241_CMDQV_H + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ); + +#endif /* HW_ARM_TEGRA241_CMDQV_H */ (quoted diff) +0
-0
b/hw/arm/tegra241-cmdqv-stubs.cnew file mode 100644 index 0000000000..eedc7bfdcdhw/arm/tegra241-cmdqv-stubs.c line 5
Quoted 5 lines
hw/arm/tegra241-cmdqv-stubs.c new file +5
-0@@ -0,0 +1,18 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All +rights reserved +* +* Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3 ^typo
Quoted 26 lines
hw/arm/tegra241-cmdqv-stubs.c new file +12
-0+* +* SPDX-License-Identifier: GPL-2.0-or-later */ + +#include "qemu/osdep.h" +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "hw/arm/tegra241-cmdqv.h" + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) { +return NULL ; +} +hw/arm/tegra241-cmdqv.c new renamed from hw/arm/tegra241-cmdqv.c +0
-0file mode 100644 index 0000000000..ad5a0d4611
hw/arm/tegra241-cmdqv.c new file +9
-0@@ -0,0 +1,56 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All +rights reserved +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later */ +hw/arm/tegra241-cmdqv.c line 19
Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10
-0+#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { } + +static bool +tegra241_cmdqv_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c
Quoted 13 lines
hw/arm/tegra241-cmdqv.c new file +13
-0+ uint32_t *out_viommu_id, Error **errp) { + error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported"); +return false; +} + +static void tegra241_cmdqv_reset (SMMUv3State *s) { } + +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) { +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static bool tegra241_cmdqv_probe(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c line 50
Quoted 11 lines
hw/arm/tegra241-cmdqv.c new file +11
-0+ Error **errp) { +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = { + .probe = tegra241_cmdqv_probe, + .init = tegra241_cmdqv_init, + .alloc_viommu = tegra241_cmdqv_alloc_viommu, + .free_viommu = tegra241_cmdqv_free_viommu, + .reset = tegra241_cmdqv_reset,I think I would create a
no_cmdqv_opsto convert the existing code without any cmdqv support to the same api. This would avoid the repeating:Quoted 1 line
if (
cmdqv_ops&& cmdqv_ops-><func>) check in subsequent patches.I guess you are suggesting having something like,
static const SMMUv3AccelCmdqvOps
no_cmdqv_ops= {
.probe =no_cmdqv_probe,
.alloc_viommu =no_cmdqv_alloc_viommu,
...
};static bool no_cmdqv_probe(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev, Error **errp){ return true; }static bool no_cmdqv_alloc_viommu(SMMUv3State *s,
Quoted 8 lines
HostIOMMUDeviceIOMMUFD *idev, uint32_t *id, Error **errp){
return false; /* We have to call the default vIOMMU alloc*/
}And then call them directly. I will take a look at that approach.
Well forget it, I don't think it is worth the pain.
Quoted 12 lines
Quoted 3 lines
Also this looks a bit strange to have different signatures for the functions used with both vcmdq on and off
Is the concern here is mainly around
alloc_viommu( that is the one which require fallback to default vIOMMU alloc)?static bool
smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Error **errp)and
Quoted 7 lines
bool (*alloc_viommu)(SMMUv3State *s,
HostIOMMUDeviceIOMMUFD *idev, uint32_t *out_viommu_id, Error **errp);Making those consistent would likely require some refactoring of the existing path. I will have a play with that and see.
Same here. That's OK for now.
Thanks
Eric
Quoted 54 lines
Thanks,
ShameerQuoted 50 lines
Eric
hw/arm/Kconfig config ARM_SMMUV3_ACCEL config
Quoted 21 lines
hw/arm/tegra241-cmdqv.c new file +5
-0+}; + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) { +return &tegra241_cmdqv_ops; +}hw/arm/Kconfig index renamed from hw/arm/Kconfig +0
-0
c66c452737..3305c6e76e100644hw/arm/Kconfig +4
-0@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config
ARM_SMMUV3hw/arm/meson.build arm_common_ss.add(when:
Quoted 11 lines
hw/arm/Kconfig +1
-0bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL boolhw/arm/meson.build +0
-0@@ -88,6 +88,8 @@ arm_common_ss.add(when:'
CONFIG_FSL_IMX8MP_EVK',if_true: files('imx8mp-evk.c'))hw/arm/meson.build arm_common_ss.add(when: · line 88
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true:files('smmuv3.c'))
hw/arm/meson.build arm_common_ss.add(when: · line 89
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:files('smmuv3-accel.c'))
hw/arm/meson.build arm_common_ss.add(when: · line 91
Quoted 2 lines
hw/arm/meson.build +1
-0stub_ss.add(files('smmuv3-accel-stubs.c')) +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true:files('tegra241-cmdqv.c'))
hw/arm/meson.build arm_common_ss.add(when: · line 93
Quoted 2 lines
hw/arm/meson.build +1
-0+stub_ss.add(files('tegra241-cmdqv-stubs.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-
imx6ul.c', 'mcimx6ul-evk.c'))hw/arm/meson.build arm_common_ss.add(when: · line 94
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:
files('nrf51_soc.c'))Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
Re: [PATCH v3 06/32] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
In reply to Shameer KolothumHi Shameer,
Quoted 50 lines · Shameer Kolothum
Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV probe, initialization, vIOMMU allocation and reset handling.
Functional CMDQV support is added in follow-up patches.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 15 ++++++++++ hw/arm/tegra241-cmdqv-stubs.c | 18 +++++++++++ hw/arm/tegra241-cmdqv.c | 56 +++++++++++++++++++++++++++++++++++ hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 2 ++ 5 files changed, 96 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644 hw/arm/tegra241-cmdqv-stubs.c create mode 100644 hw/arm/tegra241-cmdqv.chw/arm/tegra241-cmdqv.h new file +15
-0@@ -0,0 +1,15 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_ARM_TEGRA241_CMDQV_H +#define HW_ARM_TEGRA241_CMDQV_H + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ); + +#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv-stubs.c new file +2
-0@@ -0,0 +1,18 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved
hw/arm/tegra241-cmdqv-stubs.c line 12
Quoted 10 lines
hw/arm/tegra241-cmdqv-stubs.c new file +10
-0+* +* Stubs for Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "hw/arm/tegra241-cmdqv.h"
not sure you need all those headers for the stub
Eric
Quoted 104 lines
hw/arm/tegra241-cmdqv-stubs.c new file +6
-0+ +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) +{ +return NULL ; +} +hw/arm/tegra241-cmdqv.c new file +56
-0@@ -0,0 +1,56 @@ +/* +* Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) +{ +} + +static bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static void tegra241_cmdqv_reset (SMMUv3State *s) +{ +} + +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static bool tegra241_cmdqv_probe (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = { + .probe = tegra241_cmdqv_probe, + .init = tegra241_cmdqv_init, + .alloc_viommu = tegra241_cmdqv_alloc_viommu, + .free_viommu = tegra241_cmdqv_free_viommu, + .reset = tegra241_cmdqv_reset, +}; + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void ) +{ +return &tegra241_cmdqv_ops; +}hw/arm/Kconfig +5
-0@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config ARM_SMMUV3 bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL boolhw/arm/meson.build +2
-0@@ -88,6 +88,8 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_common_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) stub_ss.add(files('smmuv3-accel-stubs.c')) +arm_common_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) +stub_ss.add(files('tegra241-cmdqv-stubs.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
PATCH v3 7/32 hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
In reply to Shameer KolothumAdd support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.
If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.
If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.
If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.
When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.
The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.
No functional change.
--- include/hw/arm/smmuv3.h | 2 + hw/arm/smmuv3-accel.c | 93 +++++++++++++++++++++++++++++++++++++---- 2 files changed, 88 insertions(+), 7 deletions(-)
include/hw/arm/smmuv3.h +2 -0
@@ -73,6 +73,8 @@ struct SMMUv3State {bool ats;uint8_t oas;uint8_t ssidsize; +/* SMMU CMDQV extension */ +OnOffAuto cmdqv; }; typedef enum {
hw/arm/smmuv3-accel.c +86 -7
134 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 07/32] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
Quoted 86 lines · Shameer Kolothum
Add support for selecting and initializing a CMDQV backend based on the cmdqv OnOffAuto property.
If set to OFF, CMDQV is not used and the default IOMMUFD-backed allocation path is taken.
If set to AUTO, QEMU attempts to probe a CMDQV backend during device setup. If probing succeeds, the selected ops are stored in the accelerated SMMUv3 state and used. If probing fails, QEMU silently falls back to the default path.
If set to ON, QEMU requires CMDQV support. Probing is performed during setup and failure results in an error.
When a CMDQV backend is active, its callbacks are used for vIOMMU allocation, free, and reset handling. Otherwise, the base implementation is used.
The current implementation wires up the Tegra241 CMDQV backend through the generic ops interface. Functional CMDQV behaviour is added in subsequent patches.
No functional change.
Signed-off-by Shameer Kolothum <[email protected]>--- include/hw/arm/smmuv3.h | 2 + hw/arm/smmuv3-accel.c | 93 +++++++++++++++++++++++++++++++++++++---- 2 files changed, 88 insertions(+), 7 deletions(-)include/hw/arm/smmuv3.h +2
-0@@ -73,6 +73,8 @@ struct SMMUv3State {bool ats;uint8_t oas;uint8_t ssidsize; +/* SMMU CMDQV extension */ +OnOffAuto cmdqv; }; typedef enum {hw/arm/smmuv3-accel.c +13
-4@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem @@ -522,6 +523,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) {SMMUv3AccelState *accel = s->s_accel; +const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;struct iommu_hwpt_arm_smmuv3 bypass_data = { .ste = {SMMU_STE_CFG_BYPASS |SMMU_STE_VALID ,0x0ULL }, };@@ -532,10 +534,17 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,uint32_t viommu_id, hwpt_id;IOMMUFDViommu *viommu; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, -NULL ,0 , &viommu_id, errp)) { -return false; +if (cmdqv_ops) { +if (!cmdqv_ops->alloc_viommu (s, idev, &viommu_id, errp)) { +return false; + } + }else { +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , + s2_hwpt_id,NULL ,0 , &viommu_id, + errp)) { +return false; + } } viommu =g_new0 (IOMMUFDViommu,1 );
hw/arm/smmuv3-accel.c free_bypass_hwpt: · line 597
Quoted 9 lines
hw/arm/smmuv3-accel.c +5
-1@@ -581,12 +590,69 @@ free_bypass_hwpt: free_abort_hwpt:iommufd_backend_free_id (idev->iommufd, accel->abort_hwpt_id); free_viommu: -iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); +if (cmdqv_ops && cmdqv_ops->free_viommu) { + cmdqv_ops->free_viommu (s); + }else { +iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); + }
reading [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
tegra241_cmdqv_free_viommu() looks identifical in both implementations
So maybe this is not needed
Eric
Quoted 92 lines
hw/arm/smmuv3-accel.c +68
-2g_free (viommu); accel->viommu =NULL ;return false; } +static const SMMUv3AccelCmdqvOps * +smmuv3_accel_probe_cmdqv (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +Error **errp) +{ +const SMMUv3AccelCmdqvOps *ops =tegra241_cmdqv_get_ops (); + +if (!ops || !ops->probe) { +error_setg (errp,"No CMDQV ops found" ); +return NULL ; + } + +if (!ops->probe (s, idev, errp)) { +return NULL ; + } +return ops; +} + +static bool +smmuv3_accel_select_cmdqv (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +Error **errp) +{ +const SMMUv3AccelCmdqvOps *ops =NULL ; + +if (s->s_accel->cmdqv_ops) { +return true; + } + +switch (s->cmdqv) { +case ON_OFF_AUTO_OFF : + s->s_accel->cmdqv_ops =NULL ; +return true; +case ON_OFF_AUTO_AUTO : + ops =smmuv3_accel_probe_cmdqv (s, idev,NULL ); +break ; +case ON_OFF_AUTO_ON : + ops =smmuv3_accel_probe_cmdqv (s, idev, errp); +if (!ops) { +error_append_hint (errp,"CMDQV requested but not supported" ); +return false; + } + s->s_accel->cmdqv_ops = ops; +break ; +default : +g_assert_not_reached (); + } + +if (ops && ops->init && !ops->init (s, errp)) { +return false; + } + s->s_accel->cmdqv_ops = ops; +return true; +} +static bool smmuv3_accel_set_iommu_device (PCIBus *bus,void *opaque,int devfn,HostIOMMUDevice *hiod,Error **errp) {@@ -621,6 +687,10 @@ static bool smmuv3_accel_set_iommu_device(PCIBus *bus, void *opaque, int devfn, goto done; } +if (!smmuv3_accel_select_cmdqv (s, idev, errp)) { +return false; + } +if (!smmuv3_accel_alloc_viommu (s, idev, errp)) {error_append_hint (errp,"Unable to alloc vIOMMU: idev devid 0x%x: " , idev->devid);@@ -867,8 +937,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)void smmuv3_accel_reset (SMMUv3State *s) { -/* Attach a HWPT based on GBPA reset value */ -smmuv3_accel_attach_gbpa_hwpt (s,NULL ); +SMMUv3AccelState *accel = s->s_accel; + +if (!accel) { +return ; + } +/* Attach a HWPT based on GBPA reset value */ +smmuv3_accel_attach_gbpa_hwpt (s,NULL ); + +if (accel->cmdqv_ops && accel->cmdqv_ops->reset) { + accel->cmdqv_ops->reset (s); + } }static void smmuv3_accel_as_init (SMMUv3State *s)
PATCH v3 8/32 hw/arm/virt: Store SMMUv3 device objects in VirtMachineState.
In reply to Shameer KolothumIntroduce a GPtrArray in VirtMachineState to keep track of all SMMUv3 devices created on the virt machine.
This will avoid relying on object_child_foreach_recursive() walks of the object tree when accessing SMMUv3 instances. Subsequent patches will use this list during ACPI IORT table generation and for CMDQV-related handling.
No functional change.
--- include/hw/arm/virt.h | 1 + hw/arm/virt.c | 3 +++ 2 files changed, 4 insertions(+)
include/hw/arm/virt.h +1 -0
@@ -186,6 +186,7 @@ struct VirtMachineState {MemoryRegion *sysmem;MemoryRegion *secure_sysmem;bool pci_preserve_config; +GPtrArray *smmuv3_devices; };#define VIRT_ECAM_ID (high) (high ? VIRT_HIGH_PCIE_ECAM : VIRT_PCIE_ECAM)
hw/arm/virt.c +3 -0
@@ -3232,6 +3232,7 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev, }create_smmuv3_dev_dtb (vms, dev, bus, errp); +g_ptr_array_add (vms->smmuv3_devices, dev); } }
@@ -3659,6 +3660,8 @@ static void virt_instance_init(Object *obj) vms->oem_id =g_strndup (ACPI_BUILD_APPNAME6 ,6 ); vms->oem_table_id =g_strndup (ACPI_BUILD_APPNAME8 ,8 );cxl_machine_init (obj, &vms->cxl_devices_state); + + vms->smmuv3_devices =g_ptr_array_new_with_free_func (NULL ); }static const TypeInfo virt_machine_info = {
Signature · 1 line
2.43.0
PATCH v3 9/32 hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build
In reply to Shameer KolothumStop walking the object tree to discover SMMUv3 devices when building the IORT table.
Use the SMMUv3 device array maintained by the virt machine instead, avoiding recursive object traversal.
No functional change intended.
--- hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 39 deletions(-)
hw/arm/virt-acpi-build.c +31 -39
@@ -385,49 +385,41 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)return map_a->input_base - map_b->input_base; } -static int iort_smmuv3_devices (Object *obj,void *opaque) -{ -VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev = {0 }; -GArray *sdev_blob = opaque; -AcpiIortIdMapping idmap; -PlatformBusDevice *pbus; -int min_bus, max_bus; -SysBusDevice *sbdev; -PCIBus *bus; - -if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { -return 0 ; - } - - bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , &error_abort)); - sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); - sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); - pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); - sbdev =SYS_BUS_DEVICE (obj); - sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 ); - sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; - sdev.irq =platform_bus_get_irqn (pbus, sbdev,0 ); - sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; - sdev.irq +=ARM_SPI_BASE ; - -pci_bus_range (bus, &min_bus, &max_bus); - sdev.rc_smmu_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping)); - idmap.input_base = min_bus <<8 , - idmap.id_count = (max_bus - min_bus +1 ) <<8 , -g_array_append_val (sdev.rc_smmu_idmaps, idmap); -g_array_append_val (sdev_blob, sdev); -return 0 ; -} -/* * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and * return the total number of idmaps. */ -static int populate_smmuv3_dev (GArray *sdev_blob) +static int populate_smmuv3_dev (GArray *sdev_blob,VirtMachineState *vms) { -object_child_foreach_recursive (object_get_root (), - iort_smmuv3_devices, sdev_blob); +for (int i =0 ; i < vms->smmuv3_devices->len; i++) { +Object *obj =OBJECT (g_ptr_array_index (vms->smmuv3_devices, i)); +AcpiIortSMMUv3Dev sdev = {0 }; +AcpiIortIdMapping idmap; +PlatformBusDevice *pbus; +int min_bus, max_bus; +SysBusDevice *sbdev; +PCIBus *bus; + + bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , + &error_abort)); + sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); + sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; + sdev.irq =platform_bus_get_irqn (pbus, sbdev,0 ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; + +pci_bus_range (bus, &min_bus, &max_bus); + sdev.rc_smmu_idmaps =g_array_new (false, true, +sizeof (AcpiIortIdMapping)); + idmap.input_base = min_bus <<8 ; + idmap.id_count = (max_bus - min_bus +1 ) <<8 ; +g_array_append_val (sdev.rc_smmu_idmaps, idmap); +g_array_append_val (sdev_blob, sdev); + }/* Sort the smmuv3 devices(if any) by smmu idmap input_base */ g_array_sort (sdev_blob, smmuv3_dev_idmap_compare);/*
@@ -561,7 +553,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)if (vms->legacy_smmuv3_present) { rc_smmu_idmaps_len =populate_smmuv3_legacy_dev (smmuv3_devs); }else { - rc_smmu_idmaps_len =populate_smmuv3_dev (smmuv3_devs); + rc_smmu_idmaps_len =populate_smmuv3_dev (smmuv3_devs, vms); } num_smmus = smmuv3_devs->len;
Signature · 1 line
2.43.0
Re: [PATCH v3 09/32] hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build
Quoted 52 lines · Shameer Kolothum
Stop walking the object tree to discover SMMUv3 devices when building the IORT table.
Use the SMMUv3 device array maintained by the virt machine instead, avoiding recursive object traversal.
No functional change intended.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/virt-acpi-build.c | 70 ++++++++++++++++++---------------------- 1 file changed, 31 insertions(+), 39 deletions(-)hw/arm/virt-acpi-build.c +0
-31@@ -385,49 +385,41 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)return map_a->input_base - map_b->input_base; } -static int iort_smmuv3_devices (Object *obj,void *opaque) -{ -VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev = {0 }; -GArray *sdev_blob = opaque; -AcpiIortIdMapping idmap; -PlatformBusDevice *pbus; -int min_bus, max_bus; -SysBusDevice *sbdev; -PCIBus *bus; - -if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { -return 0 ; - } - - bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , &error_abort)); - sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); - sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); - pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); - sbdev =SYS_BUS_DEVICE (obj); - sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 ); - sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; - sdev.irq =platform_bus_get_irqn (pbus, sbdev,0 ); - sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; - sdev.irq +=ARM_SPI_BASE ; - -pci_bus_range (bus, &min_bus, &max_bus); - sdev.rc_smmu_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping)); - idmap.input_base = min_bus <<8 , - idmap.id_count = (max_bus - min_bus +1 ) <<8 , -g_array_append_val (sdev.rc_smmu_idmaps, idmap);
hw/arm/virt-acpi-build.c static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b) · line 392
Quoted 10 lines
hw/arm/virt-acpi-build.c +1
-5-g_array_append_val (sdev_blob, sdev); -return 0 ; -} -/* * Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and * return the total number of idmaps. */ -static int populate_smmuv3_dev (GArray *sdev_blob) +static int populate_smmuv3_dev (GArray *sdev_blob,VirtMachineState *vms)
nit: I would prefer vms to be the 1st arg
Quoted 36 lines
hw/arm/virt-acpi-build.c +29
-2{ -object_child_foreach_recursive (object_get_root (), - iort_smmuv3_devices, sdev_blob); +for (int i =0 ; i < vms->smmuv3_devices->len; i++) { +Object *obj =OBJECT (g_ptr_array_index (vms->smmuv3_devices, i)); +AcpiIortSMMUv3Dev sdev = {0 }; +AcpiIortIdMapping idmap; +PlatformBusDevice *pbus; +int min_bus, max_bus; +SysBusDevice *sbdev; +PCIBus *bus; + + bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , + &error_abort)); + sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); + sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; + sdev.irq =platform_bus_get_irqn (pbus, sbdev,0 ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; + +pci_bus_range (bus, &min_bus, &max_bus); + sdev.rc_smmu_idmaps =g_array_new (false, true, +sizeof (AcpiIortIdMapping)); + idmap.input_base = min_bus <<8 ; + idmap.id_count = (max_bus - min_bus +1 ) <<8 ; +g_array_append_val (sdev.rc_smmu_idmaps, idmap); +g_array_append_val (sdev_blob, sdev); + }/* Sort the smmuv3 devices(if any) by smmu idmap input_base */ g_array_sort (sdev_blob, smmuv3_dev_idmap_compare);/*
hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 556
Quoted 5 lines
hw/arm/virt-acpi-build.c +1
-1@@ -561,7 +553,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)if (vms->legacy_smmuv3_present) { rc_smmu_idmaps_len =populate_smmuv3_legacy_dev (smmuv3_devs); }else { - rc_smmu_idmaps_len =populate_smmuv3_dev (smmuv3_devs); + rc_smmu_idmaps_len =populate_smmuv3_dev (smmuv3_devs, vms);
So did you check this does not break existing bios qtests (as we change the way we iterate on the devices)? If so maybe add a comment in commit description.
hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 559
Quoted 3 lines
hw/arm/virt-acpi-build.c +0
-0} num_smmus = smmuv3_devs->len;
I would squash previous patch in that one.
Eric
PATCH v3 10/32 hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support
In reply to Shameer KolothumUse IOMMU_GET_HW_INFO to check whether the host supports Tegra241 CMDQV.
Validate the returned data type, version, number of vCMDQs and SIDs per VM. Fail the probe if the host does not meet the expected requirements.
--- hw/arm/tegra241-cmdqv.h | 5 +++++ hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 35 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +5 -0
@@ -10,6 +10,11 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#define TEGRA241_CMDQV_VERSION 1 +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1 +#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2) +#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +30 -2
@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)static bool tegra241_cmdqv_probe (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +uint32_t data_type =IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ; +struct iommu_hw_info_tegra241_cmdqv cmdqv_info; +uint64_t caps; + +if (!iommufd_backend_get_device_info (idev->iommufd, idev->devid, &data_type, + &cmdqv_info,sizeof (cmdqv_info), &caps, +NULL , errp)) { +return false; + } +if (data_type !=IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ) { +error_setg (errp,"Host CMDQV: unexpected data type %u (expected %u)" , + data_type,IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ); +return false; + } +if (cmdqv_info.version !=TEGRA241_CMDQV_VERSION ) { +error_setg (errp,"Host CMDQV: unsupported version %u (expected %u)" , + cmdqv_info.version,TEGRA241_CMDQV_VERSION ); +return false; + } +if (cmdqv_info.log2vcmdqs <TEGRA241_CMDQV_NUM_CMDQ_LOG2 ) { +error_setg (errp,"Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)" , + cmdqv_info.log2vcmdqs,TEGRA241_CMDQV_NUM_CMDQ_LOG2 ); +return false; + } +if (cmdqv_info.log2vsids <TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 ) { +error_setg (errp,"Host CMDQV: insufficient SIDs log2=%u (need >= %u)" , + cmdqv_info.log2vsids,TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 ); +return false; + } +return true; }static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
Signature · 1 line
2.43.0
Re: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support
Quoted 4 lines · Shameer Kolothum
Use
IOMMU_GET_HW_INFOto check whether the host supports Tegra241 CMDQV.Validate the returned data type, version, number of vCMDQs and SIDs per VM. Fail the probe if the host does not meet the expected requirements.
Can you explain why it is important to check
number of vCMDQs and SIDs per VM?
Quoted 56 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 5 +++++ hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 35 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +5
-0@@ -10,6 +10,11 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#define TEGRA241_CMDQV_VERSION 1 +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1 +#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2) +#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +23
-2@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)static bool tegra241_cmdqv_probe (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +uint32_t data_type =IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ; +struct iommu_hw_info_tegra241_cmdqv cmdqv_info; +uint64_t caps; + +if (!iommufd_backend_get_device_info (idev->iommufd, idev->devid, &data_type, + &cmdqv_info,sizeof (cmdqv_info), &caps, +NULL , errp)) { +return false; + } +if (data_type !=IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ) { +error_setg (errp,"Host CMDQV: unexpected data type %u (expected %u)" , + data_type,IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ); +return false; + } +if (cmdqv_info.version !=TEGRA241_CMDQV_VERSION ) { +error_setg (errp,"Host CMDQV: unsupported version %u (expected %u)" , + cmdqv_info.version,TEGRA241_CMDQV_VERSION ); +return false; + } +if (cmdqv_info.log2vcmdqs <TEGRA241_CMDQV_NUM_CMDQ_LOG2 ) { +error_setg (errp,"Host CMDQV: insufficient vCMDQs log2=%u (need >= %u)" , + cmdqv_info.log2vcmdqs,TEGRA241_CMDQV_NUM_CMDQ_LOG2 ); +return false;
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) · line 73
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+ } +if (cmdqv_info.log2vsids <TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 ) { +error_setg (errp,"Host CMDQV: insufficient SIDs log2=%u (need >= %u)" , + cmdqv_info.log2vsids,TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 ); +return false; + } +return true; }static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {
Besides
Eric
RE: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support
Quoted 21 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 10:31
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 10/32] hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV supportExternal email: Use caution opening links or attachments
Quoted 1 line · Shameer Kolothum
Use
IOMMU_GET_HW_INFOto check whether the host supports Tegra241
Quoted 8 lines
CMDQV.
Quoted 3 lines
Validate the returned data type, version, number of vCMDQs and SIDs per VM. Fail the probe if the host does not meet the expected requirements.
Can you explain why it is important to check
number of vCMDQs and SIDs per VM?
Right. I will add that.
The QEMU Tegra241 CMDQV implementation supports only two VCMDQs and 16 SIDs per VM. The check ensures the host implementation is compatible with that.
Thanks,
Shameer
Quoted 78 lines
Quoted 13 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 5 +++++ hw/arm/tegra241-cmdqv.c | 32 ++++++++++++++++++++++++++++++-- 2 files changed, 35 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h index renamed from hw/arm/tegra241-cmdqv.h +0
-0
07e10e86ee..312064a081100644hw/arm/tegra241-cmdqv.h line 15
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +3
-0@@ -10,6 +10,11 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#define TEGRA241_CMDQV_VERSION 1 +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1 +#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 38
Quoted 11 lines
hw/arm/tegra241-cmdqv.h +2
-0+#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0
-0
ad5a0d4611..a270fa7ce4100644hw/arm/tegra241-cmdqv.c +0
-0@@ -38,8 +38,36 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) static bool tegra241_cmdqv_probe(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +5
-2Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; + uint32_t data_type = IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV; + struct iommu_hw_info_tegra241_cmdqv cmdqv_info; + uint64_t caps; + + if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid,
&data_type,hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+ &cmdqv_info, sizeof(cmdqv_info), &caps, + NULL, errp)) { +return false; + } +if (data_type !=IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV ) { + error_setg(errp, "Host CMDQV: unexpected data type %u (expected%u)",
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5
-0+ data_type, IOMMU_HW_INFO_TYPE_TEGRA241_CMDQV); +return false; + } +if (cmdqv_info.version !=TEGRA241_CMDQV_VERSION ) { + error_setg(errp, "Host CMDQV: unsupported version %u (expected%u)",
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5
-0+ cmdqv_info.version, TEGRA241_CMDQV_VERSION); +return false; + } +if (cmdqv_info.log2vcmdqs <TEGRA241_CMDQV_NUM_CMDQ_LOG2 ) { + error_setg(errp, "Host CMDQV: insufficient vCMDQs log2=%u (need >=%u)",
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 61
Quoted 1 line
hw/arm/tegra241-cmdqv.c +1
-0+ cmdqv_info.log2vcmdqs,
TEGRA241_CMDQV_NUM_CMDQ_LOG2);hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3
-0+return false; + } + if (cmdqv_info.log2vsids <
TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2) {hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 65
Quoted 1 line
hw/arm/tegra241-cmdqv.c +1
-0+ error_setg(errp, "Host CMDQV: insufficient SIDs log2=%u (need >=%u)",
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, · line 66
Quoted 1 line
hw/arm/tegra241-cmdqv.c +1
-0+ cmdqv_info.log2vsids,
TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2);hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp) · line 73
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +3
-0+return false; + } +return true; }static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = {Besides
Reviewed-by Eric Auger <[email protected]> Eric
PATCH v3 11/32 hw/arm/tegra241-cmdqv: Implement CMDQV init
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages:
0x00000: Global CMDQV registers0x10000: Global VCMDQ registers, Page00x20000: Global VCMDQ registers, Page10x30000: VINTF0 logical VCMDQ registers, Page00x40000: VINTF0 logical VCMDQ registers, Page1
This patch wires up the Tegra241 CMDQV init callback and allocates vendor-specific CMDQV state. The state pointer is stored in SMMUv3AccelState for use by subsequent CMDQV operations.
The CMDQV MMIO region and a dedicated IRQ line are registered with the SMMUv3 device. The MMIO read/write handlers are currently stubs and will be implemented in later patches.
The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF error conditions. This patch only registers the IRQ line. Interrupt generation and propagation to the guest will be added in a subsequent patch.
--- hw/arm/smmuv3-accel.h | 1 + hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++-- 3 files changed, 47 insertions(+), 2 deletions(-)
hw/arm/smmuv3-accel.h +1 -0
@@ -42,6 +42,7 @@ typedef struct SMMUv3AccelState {uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list;const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;/* vendor specific CMDQV state */ } SMMUv3AccelState; typedef struct SMMUS1Hwpt {
hw/arm/tegra241-cmdqv.h +18 -0
@@ -15,6 +15,24 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +/* +* Tegra241 CMDQV MMIO layout (64KB pages) +* +* 0x00000 TEGRA241_CMDQV_CFG (Global CMDQV configuration) +* 0x10000 TEGRA241_VCMDQ_PAGE0 (Virtual CMDQ page 0) +* 0x20000 TEGRA241_VCMDQ_PAGE1 (Virtual CMDQ page 1) +* 0x30000 TEGRA241_VINTF0_PAGE0 (Virtual interface 0, page 0) +* 0x40000 TEGRA241_VINTF0_PAGE1 (Virtual interface 0, page 1) +*/ +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3AccelState *s_accel; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +28 -2
@@ -13,6 +13,16 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { }
@@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s) { } +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +SMMUv3AccelState *accel = s->s_accel; +Tegra241CMDQV *cmdqv; + + cmdqv =g_new0 (Tegra241CMDQV,1 ); +memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->s_accel = accel; + accel->cmdqv = cmdqv; +return true; } static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Signature · 1 line
2.43.0
Re: [PATCH v3 11/32] hw/arm/tegra241-cmdqv: Implement CMDQV init
Quoted 114 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages:
0x00000: Global CMDQV registers0x10000: Global VCMDQ registers, Page00x20000: Global VCMDQ registers, Page10x30000: VINTF0 logical VCMDQ registers, Page00x40000: VINTF0 logical VCMDQ registers, Page1This patch wires up the Tegra241 CMDQV init callback and allocates vendor-specific CMDQV state. The state pointer is stored in SMMUv3AccelState for use by subsequent CMDQV operations.
The CMDQV MMIO region and a dedicated IRQ line are registered with the SMMUv3 device. The MMIO read/write handlers are currently stubs and will be implemented in later patches.
The CMDQV interrupt is edge-triggered and indicates VCMDQ or VINTF error conditions. This patch only registers the IRQ line. Interrupt generation and propagation to the guest will be added in a subsequent patch.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 1 + hw/arm/tegra241-cmdqv.h | 18 ++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 30 ++++++++++++++++++++++++++++-- 3 files changed, 47 insertions(+), 2 deletions(-)hw/arm/smmuv3-accel.h +1
-0@@ -42,6 +42,7 @@ typedef struct SMMUv3AccelState {uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list;const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;/* vendor specific CMDQV state */ } SMMUv3AccelState; typedef struct SMMUS1Hwpt {hw/arm/tegra241-cmdqv.h +18
-0@@ -15,6 +15,24 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +/* +* Tegra241 CMDQV MMIO layout (64KB pages) +* +* 0x00000 TEGRA241_CMDQV_CFG (Global CMDQV configuration) +* 0x10000 TEGRA241_VCMDQ_PAGE0 (Virtual CMDQ page 0) +* 0x20000 TEGRA241_VCMDQ_PAGE1 (Virtual CMDQ page 1) +* 0x30000 TEGRA241_VINTF0_PAGE0 (Virtual interface 0, page 0) +* 0x40000 TEGRA241_VINTF0_PAGE1 (Virtual interface 0, page 1) +*/ +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3AccelState *s_accel; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +21
-2@@ -13,6 +13,16 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { }@@ -29,10 +39,26 @@ static void tegra241_cmdqv_reset(SMMUv3State *s) { } +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +SMMUv3AccelState *accel = s->s_accel; +Tegra241CMDQV *cmdqv; + + cmdqv =g_new0 (Tegra241CMDQV,1 );
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_reset(SMMUv3State *s) · line 64
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->s_accel = accel; + accel->cmdqv = cmdqv; +return true; } static bool tegra241_cmdqv_probe(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Eric
PATCH v3 12/32 hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus
In reply to Shameer KolothumSMMUv3 devices with acceleration may enable CMDQV extensions after device realize. In that case, additional MMIO regions and IRQ lines may be registered but not yet mapped to the platform bus.
Ensure SMMUv3 device resources are linked to the platform bus during machine_done().
This is safe to do unconditionally since the platform bus helpers skip resources that are already mapped.
--- hw/arm/virt.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
hw/arm/virt.c +21 -0
@@ -1833,6 +1833,24 @@ static void virt_build_smbios(VirtMachineState *vms) } } +/* +* SMMUv3 devices with acceleration may enable CMDQV extensions +* after device realize. In that case, additional MMIO regions and +* IRQ lines may be registered but not yet mapped to the platform bus. +* +* Ensure all resources are linked to the platform bus before final +* machine setup. +*/ + +static void virt_smmuv3_dev_link_cmdqv (VirtMachineState *vms) +{ +for (int i =0 ; i < vms->smmuv3_devices->len; i++) { +DeviceState *dev =g_ptr_array_index (vms->smmuv3_devices, i); +platform_bus_link_device (PLATFORM_BUS_DEVICE (vms->platform_bus_dev), +SYS_BUS_DEVICE (dev)); + } +} +static void virt_machine_done (Notifier *notifier,void *data) {
@@ -1849,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)if (vms->cxl_devices_state.is_enabled) {cxl_fmws_link_targets (&error_fatal); } + +virt_smmuv3_dev_link_cmdqv (vms); +/* * If the user provided a dtb, we assume the dynamic sysbus nodes * already are integrated there. This corresponds to a use case where
Signature · 1 line
2.43.0
Re: [PATCH v3 12/32] hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus
Quoted 26 lines · Shameer Kolothum
SMMUv3 devices with acceleration may enable CMDQV extensions after device realize. In that case, additional MMIO regions and IRQ lines may be registered but not yet mapped to the platform bus.
Ensure SMMUv3 device resources are linked to the platform bus during
machine_done().This is safe to do unconditionally since the platform bus helpers skip resources that are already mapped.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/virt.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)hw/arm/virt.c +3
-0@@ -1833,6 +1833,24 @@ static void virt_build_smbios(VirtMachineState *vms) } } +/* +* SMMUv3 devices with acceleration may enable CMDQV extensions +* after device realize. In that case, additional MMIO regions and
hw/arm/virt.c static void virt_build_smbios(VirtMachineState *vms) · line 1848
Quoted 10 lines
hw/arm/virt.c +10
-0+* IRQ lines may be registered but not yet mapped to the platform bus. +* +* Ensure all resources are linked to the platform bus before final +* machine setup. +*/ + +static void virt_smmuv3_dev_link_cmdqv (VirtMachineState *vms) +{ +for (int i =0 ; i < vms->smmuv3_devices->len; i++) { +DeviceState *dev =g_ptr_array_index (vms->smmuv3_devices, i);
need extra line
Quoted 9 lines
hw/arm/virt.c +5
-0+platform_bus_link_device (PLATFORM_BUS_DEVICE (vms->platform_bus_dev), +SYS_BUS_DEVICE (dev)); + } +} +static void virt_machine_done (Notifier *notifier,void *data) {
hw/arm/virt.c void virt_machine_done(Notifier *notifier, void *data) · line 1875
Quoted 9 lines
hw/arm/virt.c +3
-0@@ -1849,6 +1867,9 @@ void virt_machine_done(Notifier *notifier, void *data)if (vms->cxl_devices_state.is_enabled) {cxl_fmws_link_targets (&error_fatal); } + +virt_smmuv3_dev_link_cmdqv (vms); +/* * If the user provided a dtb, we assume the dynamic sysbus nodes * already are integrated there. This corresponds to a use case where
Eric
PATCH v3 13/32 hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.
Free the vIOMMU ID on teardown.
--- hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +17 -2
@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { +SMMUv3AccelState *accel = s->s_accel; +IOMMUFDViommu *viommu = accel->viommu; + +if (!viommu) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, viommu->viommu_id); }static bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV , + idev->hwpt_id, &cmdqv->cmdqv_data, +sizeof (cmdqv->cmdqv_data), out_viommu_id, + errp)) { +return false; + } +return true; }static void tegra241_cmdqv_reset (SMMUv3State *s)
Signature · 1 line
2.43.0
Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
Quoted 30 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.
Free the vIOMMU ID on teardown.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.c +7
-0@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { +SMMUv3AccelState *accel = s->s_accel; +IOMMUFDViommu *viommu = accel->viommu; + +if (!viommu) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, viommu->viommu_id); }
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +4
-2static bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, uint32_t *out_viommu_id, Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; + Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + + if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,
if the only think that differs compared to no cmdq is the type, ie.
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV vs IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV
+ passing true args, maybe you can just record the type of the cmdqv that is being used in the smmu_accel device. Then you can get rid of alloc and free
Eric
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6
-0+ idev->hwpt_id, &cmdqv->cmdqv_data, + sizeof(cmdqv->cmdqv_data), out_viommu_id, + errp)) { +return false; + } +return true; }static void tegra241_cmdqv_reset (SMMUv3State *s)
RE: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
Quoted 63 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 11:09
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/freeExternal email: Use caution opening links or attachments
Quoted 22 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.
Free the vIOMMU ID on teardown.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0
-06959766129..d487612ba2 100644
hw/arm/tegra241-cmdqv.c +1
-0@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, static void tegra241_cmdqv_free_viommu(SMMUv3State *s) { + SMMUv3AccelState *accel = s->s_accel;hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 38
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +6
-0+IOMMUFDViommu *viommu = accel->viommu; + +if (!viommu) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, viommu->viommu_id); } static bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +5
-2uint32_t *out_viommu_id, Error **errp) { - error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported"); -return false; + Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + + if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,if the only think that differs compared to no cmdq is the type, ie.
Quoted 6 lines
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQVvsIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in thesmmu_acceldevice. Then you can get rid of
+ alloc and free
It is not just the type. Based on the type we also need to pass,
* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *
And the above is implementation specific.
If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.
Thanks,
Shameer
Quoted 13 lines
Eric
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6
-0+ idev->hwpt_id, &cmdqv->cmdqv_data, + sizeof(cmdqv->cmdqv_data), out_viommu_id, + errp)) { +return false; + } +return true; }static void tegra241_cmdqv_reset (SMMUv3State *s)
Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
Quoted 75 lines · Shameer Kolothum Thodi
Quoted 62 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 11:09
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/freeExternal email: Use caution opening links or attachments
Quoted 22 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Replace the stub implementation with real vIOMMU allocation for Tegra241 CMDQV.
Free the vIOMMU ID on teardown.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0
-06959766129..d487612ba2 100644
hw/arm/tegra241-cmdqv.c +1
-0@@ -25,14 +25,29 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, static void tegra241_cmdqv_free_viommu(SMMUv3State *s) { + SMMUv3AccelState *accel = s->s_accel;hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 38
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +6
-0+IOMMUFDViommu *viommu = accel->viommu; + +if (!viommu) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, viommu->viommu_id); } static bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +5
-2uint32_t *out_viommu_id, Error **errp) { - error_setg(errp, "NVIDIA Tegra241 CMDQV is unsupported"); -return false; + Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + + if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,if the only think that differs compared to no cmdq is the type, ie.
Quoted 6 lines
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQVvsIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in thesmmu_acceldevice. Then you can get rid of
+ alloc and freeIt is not just the type. Based on the type we also need to pass,
* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *
Quoted 4 lines
And the above is implementation specific.
If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.
OK I missed the fact data_len and IOMMU data needed specialization.
Thanks
Eric
Quoted 16 lines
Thanks,
ShameerQuoted 12 lines
Eric
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +6
-0+ idev->hwpt_id, &cmdqv->cmdqv_data, + sizeof(cmdqv->cmdqv_data), out_viommu_id, + errp)) { +return false; + } +return true; }static void tegra241_cmdqv_reset (SMMUv3State *s)
Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
In reply to Shameer Kolothum ThodiQuoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +5
-2-error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + + if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,if the only think that differs compared to no cmdq is the type, ie.
Quoted 6 lines
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQVvsIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in thesmmu_acceldevice. Then you can get rid of
+ alloc and freeIt is not just the type. Based on the type we also need to pass,
* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *
Quoted 4 lines
And the above is implementation specific.
If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.
Any future "cmdqv" will likely have a different viommu type. So, essentially this is just a viommu type override, as Eric pointed out above.
What we actually need to expose per viommu is type/data_len. The data_uptr can be a union viommu_data in the accel structure?
Nicolin
RE: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
Quoted 46 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:10
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/freeQuoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +5
-2-error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + + if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,if the only think that differs compared to no cmdq is the type, ie.
Quoted 6 lines
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQVvsIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in thesmmu_acceldevice. Then you can get rid of
+ alloc and freeIt is not just the type. Based on the type we also need to pass,
* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *Quoted 4 lines
And the above is implementation specific.
If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.
Quoted 6 lines
Any future "cmdqv" will likely have a different viommu type. So, essentially this is just a viommu type override, as Eric pointed out above.
What we actually need to expose per viommu is type/data_len. The
data_uptrcan be a unionviommu_datain the accel structure?
So the suggestion is to get rid of alloc_viommu() and retrieve type/len during probe() and store the data_uptr in struct accel . Doable, I guess.
But then we need to do something similar for veventq as well, right?
Isn't that actually bloating struct accel then?
IMHO, the ops way looks better.
Thanks,
Shameer
Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free
Quoted 53 lines · Shameer Kolothum Thodi
Quoted 46 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:10
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 13/32] hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/freeQuoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 44
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +5
-2-error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + + if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + + IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV,if the only think that differs compared to no cmdq is the type, ie.
Quoted 6 lines
IOMMU_VIOMMU_TYPE_TEGRA241_CMDQVvsIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV+ passing true args, maybe you can just record the type of the cmdqv
+ that is being used in thesmmu_acceldevice. Then you can get rid of
+ alloc and freeIt is not just the type. Based on the type we also need to pass,
* @data_len: Length of the type specific data
* @__reserved: Must be 0
* @data_uptr: User pointer to a driver-specific virtual IOMMU data *Quoted 4 lines
And the above is implementation specific.
If our idea of the "ops" is to allow easier support for different implementations in future, it probably makes sense to keep this.
Quoted 6 lines
Any future "cmdqv" will likely have a different viommu type. So, essentially this is just a viommu type override, as Eric pointed out above.
What we actually need to expose per viommu is type/data_len. The
data_uptrcan be a unionviommu_datain the accel structure?
Quoted 7 lines
So the suggestion is to get rid of
alloc_viommu()and retrieve type/len duringprobe()and store thedata_uptrin struct accel . Doable, I guess.But then we need to do something similar for veventq as well, right?
Isn't that actually bloating struct accel then?
IMHO, the ops way looks better.
The data_uptr could be dynamically allocated corresponding to the data_len, to avoid bloating.
But you are right we do need a place to alloc/free a 2nd veventq.
Maybe we can have a pair of alloc/free_viommu for now.
Thanks
Nicolin
PATCH v3 14/32 hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.
--- hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 228 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +108 -0
@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)
@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32(CONFIG, 0x0) +FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4) +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4) + +REG32(STATUS, 0x8) +FIELD(STATUS, CMDQV_ENABLED, 0, 1) + +#define A_VI_ERR_MAP 0x14 +#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) + +#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) + +#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */ +#define A_CMDQ_ALLOC_MAP_ (i) \ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) + +A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i) \ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG(0) + +#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_MATCH_ (i, j) \ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19) + +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */ +#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */ +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +120 -1
139 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
Hi Shameer,
Quoted 55 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset
0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 228 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.h +19
-0@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32 (CONFIG ,0x0 )
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 64
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD (PARAM ,CMDQV_NUM_VM_LOG2 ,8 ,4 )
this is called CMDQV_NUM_VI_LOG2 in the spec I have access to VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naming
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 65
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+FIELD (PARAM ,CMDQV_NUM_SID_PER_VM_LOG2 ,12 ,4 )
same here s/VM/VI
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 69
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4
-0+ +REG32(STATUS, 0x8) +FIELD (STATUS ,CMDQV_ENABLED ,0 ,1 ) +
I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 70
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_VI_ERR_MAP 0x14
_0?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 74
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4
-0+#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) +
same SMMU_CMDQV_VI_ING_MASK0/1
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 79
Quoted 5 lines
hw/arm/tegra241-cmdqv.h +5
-0+#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) +
SMMU_CMDQV_CMDQ_ERR_MAP0-3
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {
Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7
-0+#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */ +#define A_CMDQ_ALLOC_MAP_ (i)
why A_? Since you rely on the REG macros, A_ should prefix the address offset
Can't we call that SMMU_CMDQV_CMDQ_ALLOC_MAP_() as it is refered to in the spec?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 91
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5
-0\ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) +
Please explain why we only expose 2 of those regs among 128?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i)
again why don't we use the spec terminologySMMU_CMDQV_VINTF0_CONFIG_0
Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7
-0\ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG (0 ) +
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 114
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)
Nit: the spec also contains the typo but I would rather fix it in the code
Explicitly state we only expose 1 VINTF
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 119
Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2
-0+ +/* i = [0, 0], j = [0, 15] */
does that mean that we can have a max 16 SIDs per VM?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 120
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_VINTFi_SID_MATCH_ (i, j)
if matched vi, I would prefer vi
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 126
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9
-0\ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_(0, 0) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */
vint = 0, 16 identical register entries
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 129
Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3
-0+#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)
s/19/20
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 135
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+ +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */
vi = 0, 4 identical regs
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 141
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */
I don't get this comment
Quoted 43 lines
hw/arm/tegra241-cmdqv.h +2
-0+A_VINTFi_LVCMDQ_ERR_MAP_(0, 3) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +25
-0@@ -8,19 +8,138 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; +return cmdqv->vintf_sid_match[i]; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; +return cmdqv->vintf_sid_replace[i]; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +}
hw/arm/tegra241-cmdqv.c line 49
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-1+static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { -return 0 ; +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset);
If all those regs belong to the Global CMDQV registers page this shall be within the first 64KB
Quoted 26 lines
hw/arm/tegra241-cmdqv.c +26
-0+return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +}
hw/arm/tegra241-cmdqv.c line 85
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ;
Can you explain why this needed?
Quoted 12 lines
hw/arm/tegra241-cmdqv.c +12
-0+ + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; + cmdqv->vintf_sid_match[i] = value; +break ;
hw/arm/tegra241-cmdqv.c line 107
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9
-0+ case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; + cmdqv->vintf_sid_replace[i] = value; +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } }
nit: if you can put write_vintf just after its read fellow, I think it would ease the comparison/review
Quoted 28 lines
hw/arm/tegra241-cmdqv.c +24
-0static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ;
hw/arm/tegra241-cmdqv.c line 145
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+ case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
At this stage of the reading it is unclear to me why we need to expose all those regs to the guest. I would assume that a subset of them are used by the hyp and not supposed to be accessed by the guest. For instance the SID match/replace, ...
Please could you clairfy what is absolutely needed to expose to the guest?
Thanks
Eric
RE: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
Hi Eric,
Quoted 119 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 16:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registersExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 12 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset
0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.Quoted 4 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 108+++++++++++++++++++++++++++++++++++
Quoted 1 line
hw/arm/tegra241-cmdqv.c | 121+++++++++++++++++++++++++++++++++++++++-
Quoted 7 lines
2 files changed, 228 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.h +0
-0hw/arm/tegra241-cmdqv.h line 18
Quoted 9 lines
hw/arm/tegra241-cmdqv.h +3
-0@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)Quoted 22 lines
hw/arm/tegra241-cmdqv.h +16
-0@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32 (CONFIG ,0x0 )hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 64
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD (PARAM ,CMDQV_NUM_VM_LOG2 ,8 ,4 )this is called
CMDQV_NUM_VI_LOG2in the spec I have access to VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naminghw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 65
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+FIELD (PARAM ,CMDQV_NUM_SID_PER_VM_LOG2 ,12 ,4 )same here s/VM/VI
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 69
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4
-0+ +REG32(STATUS, 0x8) +FIELD (STATUS ,CMDQV_ENABLED ,0 ,1 ) +I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 70
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_VI_ERR_MAP 0x14_0?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 74
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4
-0+#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) +same SMMU_CMDQV_VI_ING_MASK0/1
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 79
Quoted 5 lines
hw/arm/tegra241-cmdqv.h +5
-0+#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) +SMMU_CMDQV_CMDQ_ERR_MAP0-3
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */
Quoted 5 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 86
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_CMDQ_ALLOC_MAP_ (i)why A_? Since you rely on the REG macros, A_ should prefix the address offset
Can't we call thatSMMU_CMDQV_CMDQ_ALLOC_MAP_()as it is refered to in the spec?
I will incorporate all suggestions related to naming etc and will make it same as used in the spec.
Quoted 7 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 91
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5
-0\ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) +Please explain why we only expose 2 of those regs among 128?
QEMU currently models only two VCMDQs as we expose that by setting TEGRA241_CMDQV_NUM_CMDQ_LOG2 = 1 via SMMU_CMDQV_PARAM_0. Therefore, only the corresponding CMDQ allocation map entries are implemented. I will add a comment to make it explicit here.
Quoted 56 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i)again why don't we use the spec terminology
SMMU_CMDQV_VINTF0_CONFIG_0Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7
-0\ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG (0 ) +hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 114
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)Nit: the spec also contains the typo but I would rather fix it in the code
Explicitly state we only expose 1 VINTF
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 118
Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2
-0+ +/* i = [0, 0], j = [0, 15] */ does that mean that we can have a max 16 SIDs per VM?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 119
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_VINTFi_SID_MATCH_ (i, j)if matched vi, I would prefer vi
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 126
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9
-0\ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_(0, 0) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */ vint = 0, 16 identical register entries
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 129
Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3
-0+#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)s/19/20
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 135
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+ +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */ vi = 0, 4 identical regs
Quoted 7 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 141
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */ I don't get this comment
the hardware defines four registers ([0..3]). They are handled using the range based switch case,
case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
And the index to arary is calculated using:
i = (offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4
I will reword the comment something like:
Only the first and last offsets are declared explicitly since the intermediate registers are handled via the range based switch case. The register index is derived as:
(offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4
Quoted 57 lines
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +2
-0+A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) +const SMMUv3AccelCmdqvOps
*tegra241_cmdqv_get_ops(void);#endif /*
HW_ARM_TEGRA241_CMDQV_H*/hw/arm/tegra241-cmdqv.c +0
-0hw/arm/tegra241-cmdqv.c line 17
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-0@@ -8,19 +8,138 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,hwaddr offset)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5
-0+{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG :hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; +return cmdqv->vintf_sid_match[i]; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; +return cmdqv->vintf_sid_replace[i]; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ...
A_VINTF0_LVCMDQ_ERR_MAP_3:hw/arm/tegra241-cmdqv.c
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"PRIx64 "\n",
hw/arm/tegra241-cmdqv.c
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +5
-0+ __func__, offset); +return 0 ; + } +} + static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,unsigned size)
hw/arm/tegra241-cmdqv.c line 49
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +6
-1{ -return 0 ; +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset);
Quoted 2 lines
If all those regs belong to the Global CMDQV registers page this shall be within the first 64KB
Yes, in this patch that is the case. However, subsequent patches in the series will add support for the other page windows as well. Hence the full MMIO range check here.
Quoted 31 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param;hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /4];
hw/arm/tegra241-cmdqv.c
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"PRIx64 "\n",
hw/arm/tegra241-cmdqv.c
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+ __func__, offset); +return 0 ; + } +} + +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddroffset,
Quoted 9 lines
hw/arm/tegra241-cmdqv.c line 85
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+ uint64_t value) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ;Can you explain why this needed?
The HYP_OWN bit is not guest controllable. It is owned by the host kernel/hypervisor and is wired to zero when running in guest kernel.
Please see tegra241_vintf_hw_init() in kernel,drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c
The guest cannot enable this bit and the effective value seen by the guest remains zero.
Quoted 65 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+ + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ;hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; + cmdqv->vintf_sid_match[i] = value; +break ; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; + cmdqv->vintf_sid_replace[i] = value; +break ; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
hw/arm/tegra241-cmdqv.c line 107
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +3
-0+ __func__, offset); +return ; + } }nit: if you can put
write_vintfjust after its read fellow, I think it would ease the comparison/reviewhw/arm/tegra241-cmdqv.c line 108
Quoted 2 lines
hw/arm/tegra241-cmdqv.c +0
-0static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_tvalue,
Quoted 15 lines
hw/arm/tegra241-cmdqv.c +13
-0unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) {hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =value;
hw/arm/tegra241-cmdqv.c
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+break ; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +2
-0+ __func__, offset); + } }
Quoted 8 lines
hw/arm/tegra241-cmdqv.c line 145
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0static void tegra241_cmdqv_free_viommu (SMMUv3State *s)At this stage of the reading it is unclear to me why we need to expose all those regs to the guest. I would assume that a subset of them are used by the hyp and not supposed to be accessed by the guest. For instance the SID match/replace, ...
Please could you clairfy what is absolutely needed to expose to the guest?
The intention here is to model the architectural register interface defined by the CMDQV specification so the guest visible MMIO space matches the hardware.
And some of the registers are backed by a simple register cache(like the SID match/replace). Access to the actual hardware state remains controlled by QEMU, so guest writes do not necessarily translate to direct hardware programming.
Thanks,
Shameer
Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
Quoted 321 lines · Shameer Kolothum Thodi
Hi Eric,
Quoted 112 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 16:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registersExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 12 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset
0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.Quoted 4 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 108+++++++++++++++++++++++++++++++++++
Quoted 1 line
hw/arm/tegra241-cmdqv.c | 121+++++++++++++++++++++++++++++++++++++++-
Quoted 7 lines
2 files changed, 228 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.h +0
-0hw/arm/tegra241-cmdqv.h line 18
Quoted 9 lines
hw/arm/tegra241-cmdqv.h +3
-0@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)Quoted 22 lines
hw/arm/tegra241-cmdqv.h +16
-0@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32 (CONFIG ,0x0 )hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 64
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD (PARAM ,CMDQV_NUM_VM_LOG2 ,8 ,4 )this is called
CMDQV_NUM_VI_LOG2in the spec I have access to VI = virtual interface
I guess there is 1-1 mapping vetween VI and VM but I would keep spec naminghw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 65
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+FIELD (PARAM ,CMDQV_NUM_SID_PER_VM_LOG2 ,12 ,4 )same here s/VM/VI
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 69
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4
-0+ +REG32(STATUS, 0x8) +FIELD (STATUS ,CMDQV_ENABLED ,0 ,1 ) +I would add "SMMU_CMDQV_VI_ERR_MAP_0/1 definitions"
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 70
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_VI_ERR_MAP 0x14_0?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 74
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +4
-0+#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) +same SMMU_CMDQV_VI_ING_MASK0/1
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 79
Quoted 5 lines
hw/arm/tegra241-cmdqv.h +5
-0+#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) +SMMU_CMDQV_CMDQ_ERR_MAP0-3
Quoted 11 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {
Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7
-0+#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */ +#define A_CMDQ_ALLOC_MAP_ (i)why A_? Since you rely on the REG macros, A_ should prefix the address offset
Can't we call thatSMMU_CMDQV_CMDQ_ALLOC_MAP_()as it is refered to in the spec?I will incorporate all suggestions related to naming etc and will make it same as used in the spec.
Quoted 7 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 91
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +5
-0\ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) +Please explain why we only expose 2 of those regs among 128?
QEMU currently models only two VCMDQs as we expose that by setting
TEGRA241_CMDQV_NUM_CMDQ_LOG2= 1 viaSMMU_CMDQV_PARAM_0. Therefore, only the corresponding CMDQ allocation map entries are implemented. I will add a comment to make it explicit here.Quoted 56 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV {
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i)again why don't we use the spec terminology
SMMU_CMDQV_VINTF0_CONFIG_0Quoted 8 lines
hw/arm/tegra241-cmdqv.h +7
-0\ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG (0 ) +hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 114
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1)Nit: the spec also contains the typo but I would rather fix it in the code
Explicitly state we only expose 1 VINTF
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 118
Quoted 2 lines
hw/arm/tegra241-cmdqv.h +2
-0+ +/* i = [0, 0], j = [0, 15] */ does that mean that we can have a max 16 SIDs per VM?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 119
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define A_VINTFi_SID_MATCH_ (i, j)if matched vi, I would prefer vi
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 126
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +9
-0\ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_(0, 0) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */ vint = 0, 16 identical register entries
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 129
Quoted 3 lines
hw/arm/tegra241-cmdqv.h +3
-0+#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19)s/19/20
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 135
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+ +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */ vi = 0, 4 identical regs
Quoted 7 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 141
Quoted 6 lines
hw/arm/tegra241-cmdqv.h +6
-0+#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */ I don't get this comment
the hardware defines four registers ([0..3]). They are handled using the range based switch case,
case
A_VINTF0_LVCMDQ_ERR_MAP_0...A_VINTF0_LVCMDQ_ERR_MAP_3:And the index to arary is calculated using:
i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0) / 4I will reword the comment something like:
Only the first and last offsets are declared explicitly since the intermediate registers are handled via the range based switch case. The register index is derived as:
(offset - A_VINTF0_LVCMDQ_ERR_MAP_0) / 4Quoted 57 lines
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +2
-0+A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) +const SMMUv3AccelCmdqvOps
*tegra241_cmdqv_get_ops(void);#endif /*
HW_ARM_TEGRA241_CMDQV_H*/hw/arm/tegra241-cmdqv.c +0
-0hw/arm/tegra241-cmdqv.c line 17
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-0@@ -8,19 +8,138 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,hwaddr offset)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5
-0+{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG :hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; +return cmdqv->vintf_sid_match[i]; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; +return cmdqv->vintf_sid_replace[i]; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ...
A_VINTF0_LVCMDQ_ERR_MAP_3:hw/arm/tegra241-cmdqv.c
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"PRIx64 "\n",
hw/arm/tegra241-cmdqv.c
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +5
-0+ __func__, offset); +return 0 ; + } +} + static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,unsigned size)
hw/arm/tegra241-cmdqv.c line 49
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +6
-1{ -return 0 ; +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset);Quoted 2 lines
If all those regs belong to the Global CMDQV registers page this shall be within the first 64KB
Yes, in this patch that is the case. However, subsequent patches in the series will add support for the other page windows as well. Hence the full MMIO range check here.
Quoted 31 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param;hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /4];
hw/arm/tegra241-cmdqv.c
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"PRIx64 "\n",
hw/arm/tegra241-cmdqv.c
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+ __func__, offset); +return 0 ; + } +} + +static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddroffset,
Quoted 9 lines
hw/arm/tegra241-cmdqv.c line 85
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+ uint64_t value) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ;Can you explain why this needed?
Quoted 8 lines
The
HYP_OWNbit is not guest controllable. It is owned by the host kernel/hypervisor and is wired to zero when running in guest kernel.Please see
tegra241_vintf_hw_init()in kernel,drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.cThe guest cannot enable this bit and the effective value seen by the guest remains zero.
Well nothing in the spec tells you you cannot set this bit from a guest.
When you allocate a vcmdq for passthrough to a VM you need to set that bit to 0 for additional checks to be done. When host is asked to prepare the vcmdq for assignment, on tegra241_cmdqv_init_vintf_user(), hyp_owned is set to false.
However in usual tegra241_cmdqv_hw_reset() call, hyp_own is set to true. So I expect the guest to set the bit.
What do I miss?
Eric
Quoted 83 lines
Quoted 65 lines
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +8
-0+ + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ;hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; + cmdqv->vintf_sid_match[i] = value; +break ; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; + cmdqv->vintf_sid_replace[i] = value; +break ; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
hw/arm/tegra241-cmdqv.c line 107
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +3
-0+ __func__, offset); +return ; + } }nit: if you can put
write_vintfjust after its read fellow, I think it would ease the comparison/reviewhw/arm/tegra241-cmdqv.c line 108
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_tvalue,
Quoted 15 lines
hw/arm/tegra241-cmdqv.c +13
-0unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) {hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =value;
hw/arm/tegra241-cmdqv.c
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+break ; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +2
-0+ __func__, offset); + } }Quoted 7 lines
hw/arm/tegra241-cmdqv.c line 145
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0static void tegra241_cmdqv_free_viommu (SMMUv3State *s)At this stage of the reading it is unclear to me why we need to expose all those regs to the guest. I would assume that a subset of them are used by the hyp and not supposed to be accessed by the guest. For instance the SID match/replace, ...
Please could you clairfy what is absolutely needed to expose to the guest?
The intention here is to model the architectural register interface defined by the CMDQV specification so the guest visible MMIO space matches the hardware.
And some of the registers are backed by a simple register cache(like the SID match/replace). Access to the actual hardware state remains controlled by QEMU, so guest writes do not necessarily translate to direct hardware programming.
Thanks,
Shameer
Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
In reply to Shameer KolothumQuoted 7 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page
In the spec this is named CMDQ-V Config. This 64kB page contains CMDQ-V Config + VI-n Configs
Eric
Quoted 284 lines
(offset
0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 228 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.h +108
-0@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32(CONFIG, 0x0) +FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4) +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4) + +REG32(STATUS, 0x8) +FIELD(STATUS, CMDQV_ENABLED, 0, 1) + +#define A_VI_ERR_MAP 0x14 +#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) + +#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) + +#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */ +#define A_CMDQ_ALLOC_MAP_ (i) \ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) + +A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i) \ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG(0) + +#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_MATCH_ (i, j) \ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19) + +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */ +#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */ +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +120
-1@@ -8,19 +8,138 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; +return cmdqv->vintf_sid_match[i]; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; +return cmdqv->vintf_sid_replace[i]; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { -return 0 ; +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} + +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; + cmdqv->vintf_sid_match[i] = value; +break ; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; + cmdqv->vintf_sid_replace[i] = value; +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
In reply to Shameer KolothumQuoted 281 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset
0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 228 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.h +108
-0@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32(CONFIG, 0x0) +FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4) +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4) + +REG32(STATUS, 0x8) +FIELD(STATUS, CMDQV_ENABLED, 0, 1) + +#define A_VI_ERR_MAP 0x14 +#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) + +#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) + +#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */ +#define A_CMDQ_ALLOC_MAP_ (i) \ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) + +A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i) \ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG(0) + +#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_MATCH_ (i, j) \ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19) + +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */ +#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */ +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +113
-1@@ -8,19 +8,138 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; +return cmdqv->vintf_sid_match[i]; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; +return cmdqv->vintf_sid_replace[i]; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { -return 0 ; +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} + +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; + cmdqv->vintf_sid_match[i] = value; +break ; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; + cmdqv->vintf_sid_replace[i] = value; +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ;
hw/arm/tegra241-cmdqv.c line 145
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+ case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Please could you add trace points for mmio access as this is generally done for SMMU access? See trace_smmuv3_read/write_mmio()
Eric
Re: [PATCH v3 14/32] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
In reply to Shameer KolothumHi Shameer,
Quoted 11 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset
0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.
This is called CMDQ-V Config within [CMDQV_BASE, CMDQV-CMDQ_BASE] in the spec.
Precise we only support a single VINTF0 and VINTF1-63 are not supported.
Quoted 141 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 108 +++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 121 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 228 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.h +108
-0@@ -10,6 +10,9 @@#ifndef HW_ARM_TEGRA241_CMDQV_H #define HW_ARM_TEGRA241_CMDQV_H +#include "hw/core/registerfields.h" +#include "smmuv3-accel.h" +#define TEGRA241_CMDQV_VERSION 1#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 1#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)@@ -31,8 +34,113 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_sid_match[16 ]; +uint32_t vintf_sid_replace[16 ]; +uint32_t vintf_cmdq_err_map[4 ]; } Tegra241CMDQV; +/* Global CMDQV MMIO registers (offset 0x00000) */ +REG32(CONFIG, 0x0) +FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4) +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4) + +REG32(STATUS, 0x8) +FIELD(STATUS, CMDQV_ENABLED, 0, 1) + +#define A_VI_ERR_MAP 0x14 +#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) + +#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) + +#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 1] */ +#define A_CMDQ_ALLOC_MAP_ (i) \ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) + +A_CMDQ_ALLOC_MAP_(0) +A_CMDQ_ALLOC_MAP_ (1 ) + + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i) \ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG(0) + +#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + +A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_MATCH_ (i, j) \ + REG32(VINTF##i##_SID_MATCH_##j, 0x1040 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_MATCH_##j, ENABLE, 0, 1) \ + FIELD(VINTF##i##_SID_MATCH_##j, VIRT_SID, 1, 20) + +A_VINTFi_SID_MATCH_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_MATCH_ (0 ,15 ) + +/* i = [0, 0], j = [0, 15] */ +#define A_VINTFi_SID_REPLACE_ (i, j) \ + REG32(VINTF##i##_SID_REPLACE_##j, 0x1080 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_SID_REPLACE_##j, PHYS_SID, 0, 19) + +A_VINTFi_SID_REPLACE_ (0 ,0 ) +/* Omitting [0][1~14] as not being directly called */ +A_VINTFi_SID_REPLACE_ (0 ,15 ) + +/* i = [0, 0], j = [0, 3] */ +#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) +/* Omitting [0][1~2] as not being directly called */ +A_VINTFi_LVCMDQ_ERR_MAP_(0, 3) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +0
-0
hw/arm/tegra241-cmdqv.c line 17
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-0@@ -8,19 +8,138 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset)
since it belong to the Config region, can we normalize the names
tegra241_cmdqv_config_vintf_read()
Quoted 15 lines
hw/arm/tegra241-cmdqv.c +15
-0+{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; +return cmdqv->vintf_sid_match[i]; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; +return cmdqv->vintf_sid_replace[i]; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3:
hw/arm/tegra241-cmdqv.c line 42
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9
-0+ i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size)
tegra241_cmdqv_config_read
and same for the rest
Quoted 42 lines
hw/arm/tegra241-cmdqv.c +40
-1{ -return 0 ; +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} + +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG :
hw/arm/tegra241-cmdqv.c line 93
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ;
what about the GLB_FILT_CFG_0 and FILT_DATA-0, at least document why you don't support it neither in R nor W
Quoted 42 lines
hw/arm/tegra241-cmdqv.c +37
-0+ case A_VINTF0_SID_MATCH_0 ... A_VINTF0_SID_MATCH_15: + i = (offset -A_VINTF0_SID_MATCH_0 ) /4 ; + cmdqv->vintf_sid_match[i] = value; +break ; + case A_VINTF0_SID_REPLACE_0 ... A_VINTF0_SID_REPLACE_15: + i = (offset -A_VINTF0_SID_REPLACE_0 ) /4 ; + cmdqv->vintf_sid_replace[i] = value; +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_1: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ;
hw/arm/tegra241-cmdqv.c line 145
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+ case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Otherwise I reviewed the macros and they look good to me
Eric
PATCH v3 15/32 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000
The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.
Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.
VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.
--- hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 77 +++++++++++++++++ 2 files changed, 255 insertions(+)
hw/arm/tegra241-cmdqv.h +178 -0
@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {uint32_t vintf_sid_match[16 ];uint32_t vintf_sid_replace[16 ];uint32_t vintf_cmdq_err_map[4 ]; +uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ ]; } Tegra241CMDQV;/* Global CMDQV MMIO registers (offset 0x00000) */
@@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)/* Omitting [0][1~2] as not being directly called */ A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) +/* +* VCMDQ register windows. +* +* Page 0 @ 0x10000: VCMDQ control and status registers +* Page 1 @ 0x20000: VCMDQ base and DRAM address registers + */ +#define A_VCMDQi_CONS_INDX (i) \ + REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VCMDQi_CONS_INDX(0) +A_VCMDQi_CONS_INDX(1) + +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4 + +#define A_VCMDQi_PROD_INDX (i) \ + REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VCMDQi_PROD_INDX(0) +A_VCMDQi_PROD_INDX(1) + +#define A_VCMDQi_CONFIG (i) \ + REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VCMDQi_CONFIG(0) +A_VCMDQi_CONFIG(1) + +#define A_VCMDQi_STATUS (i) \ + REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VCMDQi_STATUS(0) +A_VCMDQi_STATUS(1) + +#define A_VCMDQi_GERROR (i) \ + REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERROR(0) +A_VCMDQi_GERROR(1) + +#define A_VCMDQi_GERRORN (i) \ + REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERRORN(0) +A_VCMDQi_GERRORN(1) + +#define A_VCMDQi_BASE_L (i) \ + REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VCMDQi_BASE_L(0) +A_VCMDQi_BASE_L(1) + +#define A_VCMDQi_BASE_H (i) \ + REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VCMDQi_BASE_H(0) +A_VCMDQi_BASE_H(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_H (1 ) + +/* +* VI_VCMDQ register windows (VCMDQs mapped via VINTF). +* +* Page 0 @ 0x30000: VI_VCMDQ control and status registers +* Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers + */ +#define A_VI_VCMDQi_CONS_INDX (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VI_VCMDQi_CONS_INDX(0) +A_VI_VCMDQi_CONS_INDX(1) + +#define A_VI_VCMDQi_PROD_INDX (i) \ + REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VI_VCMDQi_PROD_INDX(0) +A_VI_VCMDQi_PROD_INDX(1) + +#define A_VI_VCMDQi_CONFIG (i) \ + REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VI_VCMDQi_CONFIG(0) +A_VI_VCMDQi_CONFIG(1) + +#define A_VI_VCMDQi_STATUS (i) \ + REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VI_VCMDQi_STATUS(0) +A_VI_VCMDQi_STATUS(1) + +#define A_VI_VCMDQi_GERROR (i) \ + REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERROR(0) +A_VI_VCMDQi_GERROR(1) + +#define A_VI_VCMDQi_GERRORN (i) \ + REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERRORN(0) +A_VI_VCMDQi_GERRORN(1) + +#define A_VI_VCMDQi_BASE_L (i) \ + REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VI_VCMDQi_BASE_L(0) +A_VI_VCMDQi_BASE_L(1) + +#define A_VI_VCMDQi_BASE_H (i) \ + REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VI_VCMDQi_BASE_H(0) +A_VI_VCMDQi_BASE_H(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +77 -0
95 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
Quoted 16 lines · On Thu, 26 Feb 2026 10:50:39 +0000 Shameer Kolothum <[email protected]>…
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (
VI_VCMDQ) registers at 0x30000/0x40000The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctlIOMMU_HW_QUEUE_ALLOC.Add read side emulation for both global VCMDQ and
VI_VCMDQregister ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.
Quoted 6 lines
VI_VCMDQaccesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>
Hi Shameer,
As noted below there are a lot of repeats of 0x80 and the register window offsets in here. Maybe some defines would make things clearer?
Quoted 16 lines
static
uint64_ttegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset){ int i;hw/arm/tegra241-cmdqv.c +2
-0@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) { qemu_log_mask(LOG_UNIMP,@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 121
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+/* +* VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ +* layout as the global VCMDQ registers, but are based at 0x30000 +* instead of 0x10000. +* +* Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent +* global VCMDQ offset, then fall through to reuse the common VCMDQ +* decoding logic below. +*/ + offset -=0x20000 ;
There are a lot of repeated numeric values of offsets and sizes in here. I'm a bit in two minds about whether they are clearer as numbers or you should add a few more defines.
Jonathan
Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
In reply to Shameer KolothumShameer,
Quoted 5 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
-Global VCMDQ registers at 0x10000/0x20000
Spec mentions CMDQV_CMDQ_BASE and CMDV_VI_CMDQ_BASE offset. You could refer to that here and in the code (See fig 3.1).
Also there are size macros that could be used too.
By the way why do you name them "Global". Is that a spec terminology?
What I am really missing overall is the guest pov. I think I understand the VMM is supposed to create some VIs, assign some VCMDQs to those VI (that the guest will be able to have direct mapping to), define which VMID, SIDs are going to use this VI/VCMDQS + conversion from vSID to pSID. But then? what does the guest sees and what does he needs to program.
Quoted 5 lines
-VINTF VCMDQ (
VI_VCMDQ) registers at 0x30000/0x40000The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctlIOMMU_HW_QUEUE_ALLOC.
If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the SMMU_CMDQ_PARAM_0) how does he know if he shall access your so-called "Global VCMD regs" at 0x10000 or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use VI0?
Thanks
Eric
Quoted 315 lines
Add read side emulation for both global VCMDQ and
VI_VCMDQregister ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.
VI_VCMDQaccesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 77 +++++++++++++++++ 2 files changed, 255 insertions(+)hw/arm/tegra241-cmdqv.h +178
-0@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {uint32_t vintf_sid_match[16 ];uint32_t vintf_sid_replace[16 ];uint32_t vintf_cmdq_err_map[4 ]; +uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ ]; } Tegra241CMDQV;/* Global CMDQV MMIO registers (offset 0x00000) */ @@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)/* Omitting [0][1~2] as not being directly called */ A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) +/* +* VCMDQ register windows. +* +* Page 0 @ 0x10000: VCMDQ control and status registers +* Page 1 @ 0x20000: VCMDQ base and DRAM address registers + */ +#define A_VCMDQi_CONS_INDX (i) \ + REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VCMDQi_CONS_INDX(0) +A_VCMDQi_CONS_INDX(1) + +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4 + +#define A_VCMDQi_PROD_INDX (i) \ + REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VCMDQi_PROD_INDX(0) +A_VCMDQi_PROD_INDX(1) + +#define A_VCMDQi_CONFIG (i) \ + REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VCMDQi_CONFIG(0) +A_VCMDQi_CONFIG(1) + +#define A_VCMDQi_STATUS (i) \ + REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VCMDQi_STATUS(0) +A_VCMDQi_STATUS(1) + +#define A_VCMDQi_GERROR (i) \ + REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERROR(0) +A_VCMDQi_GERROR(1) + +#define A_VCMDQi_GERRORN (i) \ + REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERRORN(0) +A_VCMDQi_GERRORN(1) + +#define A_VCMDQi_BASE_L (i) \ + REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VCMDQi_BASE_L(0) +A_VCMDQi_BASE_L(1) + +#define A_VCMDQi_BASE_H (i) \ + REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VCMDQi_BASE_H(0) +A_VCMDQi_BASE_H(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_H (1 ) + +/* +* VI_VCMDQ register windows (VCMDQs mapped via VINTF). +* +* Page 0 @ 0x30000: VI_VCMDQ control and status registers +* Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers + */ +#define A_VI_VCMDQi_CONS_INDX (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VI_VCMDQi_CONS_INDX(0) +A_VI_VCMDQi_CONS_INDX(1) + +#define A_VI_VCMDQi_PROD_INDX (i) \ + REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VI_VCMDQi_PROD_INDX(0) +A_VI_VCMDQi_PROD_INDX(1) + +#define A_VI_VCMDQi_CONFIG (i) \ + REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VI_VCMDQi_CONFIG(0) +A_VI_VCMDQi_CONFIG(1) + +#define A_VI_VCMDQi_STATUS (i) \ + REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VI_VCMDQi_STATUS(0) +A_VI_VCMDQi_STATUS(1) + +#define A_VI_VCMDQi_GERROR (i) \ + REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERROR(0) +A_VI_VCMDQi_GERROR(1) + +#define A_VI_VCMDQi_GERRORN (i) \ + REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERRORN(0) +A_VI_VCMDQi_GERRORN(1) + +#define A_VI_VCMDQi_BASE_L (i) \ + REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VI_VCMDQi_BASE_L(0) +A_VI_VCMDQi_BASE_L(1) + +#define A_VI_VCMDQi_BASE_H (i) \ + REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VI_VCMDQi_BASE_H(0) +A_VI_VCMDQi_BASE_H(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +77
-0@@ -14,6 +14,46 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +/* +* Read a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* All VCMDQ accesses return cached registers. +*/ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0, +int index) +{ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +return cmdqv->vcmdq_cons_indx[index]; +case A_VCMDQ0_PROD_INDX : +return cmdqv->vcmdq_prod_indx[index]; +case A_VCMDQ0_CONFIG : +return cmdqv->vcmdq_config[index]; +case A_VCMDQ0_STATUS : +return cmdqv->vcmdq_status[index]; +case A_VCMDQ0_GERROR : +return cmdqv->vcmdq_gerror[index]; +case A_VCMDQ0_GERRORN : +return cmdqv->vcmdq_gerrorn[index]; +case A_VCMDQ0_BASE_L : +return cmdqv->vcmdq_base[index]; +case A_VCMDQ0_BASE_H : +return cmdqv->vcmdq_base[index] >>32 ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +return cmdqv->vcmdq_cons_indx_base[index]; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : +return cmdqv->vcmdq_cons_indx_base[index] >>32 ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset0); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) {int i;@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) { qemu_log_mask(LOG_UNIMP,@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN: +/* +* VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ +* layout as the global VCMDQ registers, but are based at 0x30000 +* instead of 0x10000. +* +* Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent +* global VCMDQ offset, then fall through to reuse the common VCMDQ +* decoding logic below. +*/ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN: +/* +* Decode a per-VCMDQ register access. +* +* The hardware supports up to 128 identical VCMDQ instances; we +* currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ +* occupies a 0x80-byte window starting at 0x10000. +* +* The MMIO offset is decoded to extract the VCMDQ index and normalized +* to the corresponding VCMDQ0_* register by subtracting index * 0x80. +* +* A single helper then services all VCMDQs, with @index selecting the +* instance. +*/ + index = (offset -0x10000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset - index *0x80 , index); + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H: +/* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H: +/* Same decode logic as A_VCMDQx_CONS_INDX case above */ + index = (offset -0x20000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset - index *0x80 , index);default :qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , __func__, offset);
RE: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
Hi Eric,
Quoted 43 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 17:44
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register readsExternal email: Use caution opening links or attachments
Shameer,
Quoted 3 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV exposes per-VCMDQ register windows through two
MMIO views:
Quoted 2 lines
-Global VCMDQ registers at 0x10000/0x20000
Spec mentions
CMDQV_CMDQ_BASEandCMDV_VI_CMDQ_BASEoffset. You could
refer to that here and in the code (See fig 3.1). Also there are size macros that could be used too.By the way why do you name them "Global". Is that a spec terminology?
What I am really missing overall is the guest pov. I think I understand the VMM is supposed to create some VIs, assign some VCMDQs to those VI (that the guest will be able to have direct mapping to), define which VMID, SIDs are going to use this VI/VCMDQS + conversion from vSID to pSID. But then? what does the guest sees and what does he needs to program.
Quoted 2 lines
-VINTF VCMDQ (
VI_VCMDQ) registers at 0x30000/0x40000
Quoted 8 lines
Quoted 3 lines
The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctlIOMMU_HW_QUEUE_ALLOC.If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the
SMMU_CMDQ_PARAM_0) how does he know if he
shall access your so-called "Global VCMD regs" at0x10000or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use VI0?
The CMDQ registers can be accessed via both the so called "Global VCMDQ registers Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
The difference being(from Nicolin's explanation here[0]):
"
The global page0 is programmable at any time so long as CMDQV_EN is enabled.
The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".
IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".
The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.
@Nicolin, please correct me if that assumption is not right.
Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/aXkTzRM695oTWFaD@Asurada-Nvidia/
Quoted 336 lines
Thanks
Eric
Quoted 6 lines
Add read side emulation for both global VCMDQ and
VI_VCMDQregister ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.Quoted 8 lines
VI_VCMDQaccesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 178++++++++++++++++++++++++++++++++++++++++
Quoted 92 lines
hw/arm/tegra241-cmdqv.c | 77 +++++++++++++++++ 2 files changed, 255 insertions(+)hw/arm/tegra241-cmdqv.h +74
-0@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {uint32_t vintf_sid_match[16 ];uint32_t vintf_sid_replace[16 ];uint32_t vintf_cmdq_err_map[4 ]; +uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ ]; } Tegra241CMDQV;/* Global CMDQV MMIO registers (offset 0x00000) */ @@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)/* Omitting [0][1~2] as not being directly called */ A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) +/* +* VCMDQ register windows. +* +* Page 0 @ 0x10000: VCMDQ control and status registers +* Page 1 @ 0x20000: VCMDQ base and DRAM address registers + */ +#define A_VCMDQi_CONS_INDX (i) \ + REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VCMDQi_CONS_INDX(0) +A_VCMDQi_CONS_INDX(1) + +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4 + +#define A_VCMDQi_PROD_INDX (i) \ + REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VCMDQi_PROD_INDX(0) +A_VCMDQi_PROD_INDX(1) + +#define A_VCMDQi_CONFIG (i) \ + REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VCMDQi_CONFIG(0) +A_VCMDQi_CONFIG(1) + +#define A_VCMDQi_STATUS (i) \ + REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VCMDQi_STATUS(0) +A_VCMDQi_STATUS(1) + +#define A_VCMDQi_GERROR (i) \ + REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERROR(0) +A_VCMDQi_GERROR(1) + +#define A_VCMDQi_GERRORN (i) \ + REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERRORN(0) +A_VCMDQi_GERRORN(1) + +#define A_VCMDQi_BASE_L (i) \ + REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VCMDQi_BASE_L(0) +A_VCMDQi_BASE_L (1 )hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 227
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+ +#define A_VCMDQi_BASE_H (i) \ + REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VCMDQi_BASE_H(0) +A_VCMDQi_BASE_H(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i *
0x80) \hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7
-0+ FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i *
0x80) \Quoted 65 lines
hw/arm/tegra241-cmdqv.h +65
-0+ FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_H (1 ) + +/* +* VI_VCMDQ register windows (VCMDQs mapped via VINTF). +* +* Page 0 @ 0x30000: VI_VCMDQ control and status registers +* Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers + */ +#define A_VI_VCMDQi_CONS_INDX (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VI_VCMDQi_CONS_INDX(0) +A_VI_VCMDQi_CONS_INDX(1) + +#define A_VI_VCMDQi_PROD_INDX (i) \ + REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VI_VCMDQi_PROD_INDX(0) +A_VI_VCMDQi_PROD_INDX(1) + +#define A_VI_VCMDQi_CONFIG (i) \ + REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VI_VCMDQi_CONFIG(0) +A_VI_VCMDQi_CONFIG(1) + +#define A_VI_VCMDQi_STATUS (i) \ + REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VI_VCMDQi_STATUS(0) +A_VI_VCMDQi_STATUS(1) + +#define A_VI_VCMDQi_GERROR (i) \ + REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERROR(0) +A_VI_VCMDQi_GERROR(1) + +#define A_VI_VCMDQi_GERRORN (i) \ + REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERRORN(0) +A_VI_VCMDQi_GERRORN(1) + +#define A_VI_VCMDQi_BASE_L (i) \ + REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VI_VCMDQi_BASE_L(0) +A_VI_VCMDQi_BASE_L (1 )hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+ +#define A_VI_VCMDQi_BASE_H (i) \ + REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VI_VCMDQi_BASE_H(0) +A_VI_VCMDQi_BASE_H(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i*
0x80) \hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)
Quoted 7 lines
hw/arm/tegra241-cmdqv.h +7
-0+ FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i*
0x80) \Quoted 15 lines
hw/arm/tegra241-cmdqv.h +5
-0+ FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +0
-0@@ -14,6 +14,46 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" hw/arm/tegra241-cmdqv.c line 25
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9
-0+/* +* Read a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* All VCMDQ accesses return cached registers. +*/ +static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,hwaddr offset0,
Quoted 22 lines
hw/arm/tegra241-cmdqv.c +22
-0+ int index) +{ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +return cmdqv->vcmdq_cons_indx[index]; +case A_VCMDQ0_PROD_INDX : +return cmdqv->vcmdq_prod_indx[index]; +case A_VCMDQ0_CONFIG : +return cmdqv->vcmdq_config[index]; +case A_VCMDQ0_STATUS : +return cmdqv->vcmdq_status[index]; +case A_VCMDQ0_GERROR : +return cmdqv->vcmdq_gerror[index]; +case A_VCMDQ0_GERRORN : +return cmdqv->vcmdq_gerrorn[index]; +case A_VCMDQ0_BASE_L : +return cmdqv->vcmdq_base[index]; +case A_VCMDQ0_BASE_H : +return cmdqv->vcmdq_base[index] >>32 ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +return cmdqv->vcmdq_cons_indx_base[index]; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H :hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9
-0+return cmdqv->vcmdq_cons_indx_base[index] >>32 ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset0); +return 0 ; + } +} + static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,hwaddr offset)
hw/arm/tegra241-cmdqv.c static uint64_t
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0{int i;@@ -42,6 +82,7 @@ static uint64_t
tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)hw/arm/tegra241-cmdqv.c static uint64_t · line 82
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +1
-0{Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) { qemu_log_mask(LOG_UNIMP,@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 108
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0return cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) /4];
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8
-0case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN: +/* +* VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ +* layout as the global VCMDQ registers, but are based at 0x30000 +* instead of 0x10000. +* +* Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent +* global VCMDQ offset, then fall through to reuse the common VCMDQ hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+* decoding logic below. +*/ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN: +/* +* Decode a per-VCMDQ register access. +* +* The hardware supports up to 128 identical VCMDQ instances; we +* currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3
-0+* occupies a 0x80-byte window starting at 0x10000. +* +* The MMIO offset is decoded to extract the VCMDQ index and normalized
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 132
Quoted 1 line
hw/arm/tegra241-cmdqv.c +1
-0+* to the corresponding VCMDQ0_* register by subtracting index *
0x80.hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+* +* A single helper then services all VCMDQs, with @index selecting the +* instance. +*/ + index = (offset -0x10000 ) /0x80 ; + return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80,index);
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 139
Quoted 1 line
hw/arm/tegra241-cmdqv.c +1
-0+ case A_VI_VCMDQ0_BASE_L ...
A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7
-0+/* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H: +/* Same decode logic as A_VCMDQx_CONS_INDX case above */ + index = (offset -0x20000 ) /0x80 ; + return tegra241_cmdqv_read_vcmdq(cmdqv, offset - index * 0x80,index);
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 148
Quoted 2 lines
hw/arm/tegra241-cmdqv.c +0
-0default : qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"PRIx64 "\n",
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0__func__, offset);
Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
Quoted 19 lines · Shameer Kolothum Thodi
Quoted 9 lines
Quoted 3 lines
The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctlIOMMU_HW_QUEUE_ALLOC.If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the
SMMU_CMDQ_PARAM_0) how does he know if he
shall access your so-called "Global VCMD regs" at0x10000or VI0 regs (0x30000-40000). How does the guest know that he is supposed to use VI0?The CMDQ registers can be accessed via both the so called "Global VCMDQ registers Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
The difference
being(from Nicolin's explanation here[0]):"
The global page0 is programmable at any time so long asCMDQV_ENis enabled.
Quoted 7 lines
The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".
Yes.
Quoted 3 lines
The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.
Yes.
The point is that we can't reject guest's access to the global MMIO space. So, QEMU has to support both. But guest OS is allowed to do a partial map/alloc, so global MMIO page isn't identical to the VINTF logical page in this case.
In an extreme corner case, guest OS might try to enable a VCMDQ via the global MMIO space, though the vcmdq isn't allocated/mapped to a VINTF. QEMU should still allow these MMIOs but does not essentially make the vcmdq functional, since this would be "unpredictable" in a physical HW situation.
Maybe we should elaborate these in the commit message.
Nicolin
Quoted 1 line
@Nicolin, please correct me if that assumption is not right.
RE: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
Quoted 54 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:41
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register readsQuoted 6 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 3 lines
The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctlIOMMU_HW_QUEUE_ALLOC.If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the
SMMU_CMDQ_PARAM_0) how does heknow if
Quoted 3 lines
Quoted 3 lines
he
shall access your so-called "Global VCMD regs" at0x10000or VI0 regs (0x30000-40000). How does the guest know that he is supposed to useVI0?
Quoted 2 lines
The CMDQ registers can be accessed via both the so called "Global VCMDQ
registers
Quoted 8 lines
Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
The difference
being(from Nicolin's explanation here[0]):"
The global page0 is programmable at any time so long asCMDQV_ENis enabled.Quoted 7 lines
The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".
Yes.
Quoted 3 lines
The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.
Quoted 6 lines
Yes.
The point is that we can't reject guest's access to the global MMIO space. So, QEMU has to support both. But guest OS is allowed to do a partial map/alloc, so global MMIO page isn't identical to the VINTF logical page in this case.
Are there any benefits for a guest implementation making use of the "Global VCMDQ registers Page 0"?
If not, the VINTF-based access path seems preferable from a hardware isolation point of view. In that case, would it make sense to restrict usage to the VINTF logical page and recommend that guests use that interface?
Thanks,
Shameer
Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
Quoted 61 lines · Shameer Kolothum Thodi
Quoted 54 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 18:41
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register readsQuoted 6 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 3 lines
The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctlIOMMU_HW_QUEUE_ALLOC.If the guest detects he can use a single VMCDQs (I guess he can introspect this by reading the
SMMU_CMDQ_PARAM_0) how does heknow if
Quoted 3 lines
Quoted 3 lines
he
shall access your so-called "Global VCMD regs" at0x10000or VI0 regs (0x30000-40000). How does the guest know that he is supposed to useVI0?
Quoted 2 lines
The CMDQ registers can be accessed via both the so called "Global VCMDQ
registers
Quoted 8 lines
Page0" and by "VINTF logical VCMDQ registers Page 0". And they are aliased.
The difference
being(from Nicolin's explanation here[0]):"
The global page0 is programmable at any time so long asCMDQV_ENis enabled.Quoted 7 lines
The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
".IIUC, for now a Linux Guest kernel will only use "VINTF logical VCMDQ registers Page 0".
Yes.
Quoted 3 lines
The question is whether QEMU should also support accesses via the "Global VCMDQ registers Page 0" if some other guest implementation chooses to use that path.
Quoted 6 lines
Yes.
The point is that we can't reject guest's access to the global MMIO space. So, QEMU has to support both. But guest OS is allowed to do a partial map/alloc, so global MMIO page isn't identical to the VINTF logical page in this case.
Quoted 5 lines
Are there any benefits for a guest implementation making use of the "Global VCMDQ registers Page 0"?
If not, the VINTF-based access path seems preferable from a hardware isolation point of view.
Yea. That's why Linux was implemented in that way.
Quoted 3 lines
In that case, would it make sense to restrict
usage to the VINTF logical page and recommend that guests use that interface?
I don't think QEMU can recommend that... Linux kernel made a good example but it doesn't make a lot of sense to recommend or force all guest SWs to follow.
From hardware emulation perspective, QEMU should just re-act like a real HW. It's okay to make it unfunctional v.s. unpredictable. But it shouldn't reject any MMIO access, since the HW wouldn't.
Nicolin
Re: [PATCH v3 15/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
In reply to Shameer KolothumQuoted 8 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (
VI_VCMDQ) registers at 0x30000/0x40000The
VI_VCMDQregister ranges are an alias of the global VCMDQ registers
I would recommend to use the same terminology anywhere, either VINTF or VI and stick to it.
Why do we need emulation for VI_VCMDQ. I thought this was mmapped?
Explicitly state we only support 2 VCMDQs
Quoted 125 lines
and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl
IOMMU_HW_QUEUE_ALLOC.Add read side emulation for both global VCMDQ and
VI_VCMDQregister ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.
VI_VCMDQaccesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 178 ++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.c | 77 +++++++++++++++++ 2 files changed, 255 insertions(+)hw/arm/tegra241-cmdqv.h +91
-0@@ -48,6 +48,14 @@ typedef struct Tegra241CMDQV {uint32_t vintf_sid_match[16 ];uint32_t vintf_sid_replace[16 ];uint32_t vintf_cmdq_err_map[4 ]; +uint32_t vcmdq_cons_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_prod_indx[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_config[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_status[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerror[TEGRA241_CMDQV_MAX_CMDQ ]; +uint32_t vcmdq_gerrorn[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_base[TEGRA241_CMDQV_MAX_CMDQ ]; +uint64_t vcmdq_cons_indx_base[TEGRA241_CMDQV_MAX_CMDQ ]; } Tegra241CMDQV;/* Global CMDQV MMIO registers (offset 0x00000) */ @@ -141,6 +149,176 @@ A_VINTFi_LVCMDQ_ERR_MAP_(0, 0)/* Omitting [0][1~2] as not being directly called */ A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) +/* +* VCMDQ register windows. +* +* Page 0 @ 0x10000: VCMDQ control and status registers +* Page 1 @ 0x20000: VCMDQ base and DRAM address registers + */ +#define A_VCMDQi_CONS_INDX (i) \ + REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VCMDQi_CONS_INDX(0) +A_VCMDQi_CONS_INDX(1) + +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4 + +#define A_VCMDQi_PROD_INDX (i) \ + REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VCMDQi_PROD_INDX(0) +A_VCMDQi_PROD_INDX(1) + +#define A_VCMDQi_CONFIG (i) \ + REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VCMDQi_CONFIG(0) +A_VCMDQi_CONFIG(1) + +#define A_VCMDQi_STATUS (i) \ + REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VCMDQi_STATUS(0) +A_VCMDQi_STATUS(1) + +#define A_VCMDQi_GERROR (i) \ + REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERROR(0) +A_VCMDQi_GERROR(1) + +#define A_VCMDQi_GERRORN (i) \ + REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERRORN(0) +A_VCMDQi_GERRORN(1) + +#define A_VCMDQi_BASE_L (i) \ + REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VCMDQi_BASE_L(0) +A_VCMDQi_BASE_L(1) + +#define A_VCMDQi_BASE_H (i) \ + REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VCMDQi_BASE_H(0) +A_VCMDQi_BASE_H(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \
hw/arm/tegra241-cmdqv.h A_VINTFi_LVCMDQ_ERR_MAP_(0, 0) · line 244
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +10
-0+ FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VCMDQi_CONS_INDX_BASE_DRAM_H (1 ) + +/* +* VI_VCMDQ register windows (VCMDQs mapped via VINTF). +* +* Page 0 @ 0x30000: VI_VCMDQ control and status registers +* Page 1 @ 0x40000: VI_VCMDQ base and DRAM address registers
I don't get why it is needed.
Quoted 172 lines
hw/arm/tegra241-cmdqv.h +77
-0+*/ +#define A_VI_VCMDQi_CONS_INDX (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VI_VCMDQi_CONS_INDX(0) +A_VI_VCMDQi_CONS_INDX(1) + +#define A_VI_VCMDQi_PROD_INDX (i) \ + REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VI_VCMDQi_PROD_INDX(0) +A_VI_VCMDQi_PROD_INDX(1) + +#define A_VI_VCMDQi_CONFIG (i) \ + REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VI_VCMDQi_CONFIG(0) +A_VI_VCMDQi_CONFIG(1) + +#define A_VI_VCMDQi_STATUS (i) \ + REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VI_VCMDQi_STATUS(0) +A_VI_VCMDQi_STATUS(1) + +#define A_VI_VCMDQi_GERROR (i) \ + REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERROR(0) +A_VI_VCMDQi_GERROR(1) + +#define A_VI_VCMDQi_GERRORN (i) \ + REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERRORN(0) +A_VI_VCMDQi_GERRORN(1) + +#define A_VI_VCMDQi_BASE_L (i) \ + REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VI_VCMDQi_BASE_L(0) +A_VI_VCMDQi_BASE_L(1) + +#define A_VI_VCMDQi_BASE_H (i) \ + REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VI_VCMDQi_BASE_H(0) +A_VI_VCMDQi_BASE_H(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +70
-0@@ -14,6 +14,46 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +/* +* Read a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* All VCMDQ accesses return cached registers. +*/ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0, +int index) +{ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +return cmdqv->vcmdq_cons_indx[index]; +case A_VCMDQ0_PROD_INDX : +return cmdqv->vcmdq_prod_indx[index]; +case A_VCMDQ0_CONFIG : +return cmdqv->vcmdq_config[index]; +case A_VCMDQ0_STATUS : +return cmdqv->vcmdq_status[index]; +case A_VCMDQ0_GERROR : +return cmdqv->vcmdq_gerror[index]; +case A_VCMDQ0_GERRORN : +return cmdqv->vcmdq_gerrorn[index]; +case A_VCMDQ0_BASE_L : +return cmdqv->vcmdq_base[index]; +case A_VCMDQ0_BASE_H : +return cmdqv->vcmdq_base[index] >>32 ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +return cmdqv->vcmdq_cons_indx_base[index]; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : +return cmdqv->vcmdq_cons_indx_base[index] >>32 ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset0); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) {int i;@@ -42,6 +82,7 @@ static uint64_t tegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv, hwaddr offset)static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) { qemu_log_mask(LOG_UNIMP,@@ -67,6 +108,42 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN: +/* +* VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ +* layout as the global VCMDQ registers, but are based at 0x30000 +* instead of 0x10000. +* +* Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent +* global VCMDQ offset, then fall through to reuse the common VCMDQ +* decoding logic below. +*/ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN: +/* +* Decode a per-VCMDQ register access. +* +* The hardware supports up to 128 identical VCMDQ instances; we +* currently expose TEGRA241_CMDQV_MAX_CMDQ (= 2). Each VCMDQ +* occupies a 0x80-byte window starting at 0x10000. +* +* The MMIO offset is decoded to extract the VCMDQ index and normalized +* to the corresponding VCMDQ0_* register by subtracting index * 0x80. +* +* A single helper then services all VCMDQs, with @index selecting the +* instance. +*/ + index = (offset -0x10000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset - index *0x80 , index); + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 149
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+/* Same decode logic as A_VI_VCMDQx_CONS_INDX case above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H: +/* Same decode logic as A_VCMDQx_CONS_INDX case above */ + index = (offset -0x20000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset - index *0x80 , index);default :qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , __func__, offset);
Thanks
Eric
PATCH v3 16/32 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
This is the write side counterpart of the VCMDQ read emulation.
Add write handling for global VCMDQ and VI_VCMDQ register windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.
VI_VCMDQ registers are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.
Writes are backed by cached register state only; no hardware queue mapping is performed yet.
--- hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)
hw/arm/tegra241-cmdqv.c +83 -0
@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +/* +* Write a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +*/ +static void +tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index, +uint64_t value,unsigned size) +{ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : + cmdqv->vcmdq_cons_indx[index] = value; +return ; +case A_VCMDQ0_PROD_INDX : + cmdqv->vcmdq_prod_indx[index] = (uint32_t )value; +return ; +case A_VCMDQ0_CONFIG : +if (value &R_VCMDQ0_CONFIG_CMDQ_EN_MASK ) { + cmdqv->vcmdq_status[index] |=R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + }else { + cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + } + cmdqv->vcmdq_config[index] = (uint32_t )value; +return ; +case A_VCMDQ0_GERRORN : + cmdqv->vcmdq_gerrorn[index] = (uint32_t )value; +return ; +case A_VCMDQ0_BASE_L : +if (size ==8 ) { + cmdqv->vcmdq_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_BASE_H : + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +if (size ==8 ) { + cmdqv->vcmdq_cons_indx_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset0); +return ; + } +} +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset,uint64_t value) {
@@ -187,6 +251,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) { qemu_log_mask(LOG_UNIMP,
@@ -213,6 +278,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:tegra241_cmdqv_write_vintf (cmdqv, offset, value);break ; + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN: +/* Same decoding as read() case: See comments above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN: + index = (offset -0x10000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size); +break ; + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H: +/* Same decoding as read() case: See comments above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H: + index = (offset -0x20000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size); +break ;default :qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset);
Signature · 1 line
2.43.0
Re: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes
Quoted 54 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
This is the write side counterpart of the VCMDQ read emulation.
Add write handling for global VCMDQ and
VI_VCMDQregister windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.
VI_VCMDQregisters are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.Writes are backed by cached register state only; no hardware queue mapping is performed yet.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)hw/arm/tegra241-cmdqv.c +26
-0@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +/* +* Write a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +*/ +static void +tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index, +uint64_t value,unsigned size) +{ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : + cmdqv->vcmdq_cons_indx[index] = value; +return ; +case A_VCMDQ0_PROD_INDX : + cmdqv->vcmdq_prod_indx[index] = (uint32_t )value; +return ; +case A_VCMDQ0_CONFIG : +if (value &R_VCMDQ0_CONFIG_CMDQ_EN_MASK ) { + cmdqv->vcmdq_status[index] |=R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + }else { + cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + } + cmdqv->vcmdq_config[index] = (uint32_t )value; +return ; +case A_VCMDQ0_GERRORN :
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ cmdqv->vcmdq_gerrorn[index] = (uint32_t )value; +return ; +case A_VCMDQ0_BASE_L : +if (size ==8 ) { + cmdqv->vcmdq_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + }
If we compare to smmuv3 std command queue API, we miss some checks. See smmu_writel()
if (!smmu_cmdq_base_writable(s, reg_sec_sid)) {
for CONS
if (!smmu_cmdq_disabled_stable(s, reg_sec_sid)) {
In the spec I have access to I failed to find any details about equivalent restrictions
Can you check if those write accesses are always allowed?
Thanks
Eric
Quoted 64 lines
hw/arm/tegra241-cmdqv.c +47
-0+return ; +case A_VCMDQ0_BASE_H : + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +if (size ==8 ) { + cmdqv->vcmdq_cons_indx_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset0); +return ; + } +} +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset,uint64_t value) {@@ -187,6 +251,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) { qemu_log_mask(LOG_UNIMP,@@ -213,6 +278,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:tegra241_cmdqv_write_vintf (cmdqv, offset, value);break ; + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN: +/* Same decoding as read() case: See comments above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN: + index = (offset -0x10000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size); +break ; + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H: +/* Same decoding as read() case: See comments above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H: + index = (offset -0x20000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size); +break ;default :qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset);
RE: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes
Hi Eric,
Quoted 93 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:32
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 16/32] hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writesExternal email: Use caution opening links or attachments
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 26 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
This is the write side counterpart of the VCMDQ read emulation.
Add write handling for global VCMDQ and
VI_VCMDQregister windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.
VI_VCMDQregisters are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.Writes are backed by cached register state only; no hardware queue mapping is performed yet.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+)hw/arm/tegra241-cmdqv.c index renamed from hw/arm/tegra241-cmdqv.c +0
-0
d2e6938e44..e1f1562c44100644hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,70 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 12 lines
hw/arm/tegra241-cmdqv.c +9
-0} } +/* +* Write a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always +refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +*/ +static void +tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,int index,
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 11 lines
hw/arm/tegra241-cmdqv.c +11
-0+ uint64_t value, unsigned size) { + switch (offset0) { +case A_VCMDQ0_CONS_INDX : + cmdqv->vcmdq_cons_indx[index] = value; +return ; +case A_VCMDQ0_PROD_INDX : + cmdqv->vcmdq_prod_indx[index] = (uint32_t )value; +return ; +case A_VCMDQ0_CONFIG : +if (value &R_VCMDQ0_CONFIG_CMDQ_EN_MASK ) { + cmdqv->vcmdq_status[index] |=
R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 175
Quoted 2 lines
hw/arm/tegra241-cmdqv.c +2
-0+ }else { + cmdqv->vcmdq_status[index] &=~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK;
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ } + cmdqv->vcmdq_config[index] = (uint32_t )value; +return ; +case A_VCMDQ0_GERRORN :hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ cmdqv->vcmdq_gerrorn[index] = (uint32_t )value; +return ; +case A_VCMDQ0_BASE_L : +if (size ==8 ) { + cmdqv->vcmdq_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + }If we compare to smmuv3 std command queue API, we miss some checks. See
smmu_writel()
if (!smmu_cmdq_base_writable(s,reg_sec_sid)) { for CONS if (!smmu_cmdq_disabled_stable(s,reg_sec_sid)) {
Quoted 4 lines
In the spec I have access to I failed to find any details about equivalent restrictions
Can you check if those write accesses are always allowed?
I couldn't find any explicit restriction in the spec either. The spec details the "Enabling the Virtual CMDQ" sequence as:
- Program the VIRT_CMDQ_BASE register
- Initialize CMDQ_ registers
- Program the VIRT_CMDQ_CONFIG to enable the CMDQ
From this, looks like, the expected behaviour is that BASE is programmed only when the VCMDQ is in a disabled state.
We can add a defensive check by reading VCMDQ_CONFIG directly from vintf_ptr before allowing a BASE write. If CMDQ_EN is set, we reject the write. This will ensure Guest has disabled the VCMDQ before reprogramming BASE register.
For v4(that I just send out), this part is unchanged. If required, we can add the above check in the next respin.
Thanks,
Shameer
PATCH v3 17/32 hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 45 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.h +3 -0
@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) *
@@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;
hw/arm/tegra241-cmdqv.c +42 -2
82 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 4 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real
Again I fail to understand the guest POV. If the goal is to let the guest have a direct access to some vcmdqs why do we need to emulate the CMDQ regs?
Eric
Quoted 132 lines
hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +3
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +42
-2@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static bool +tegra241_cmdqv_munmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +if (!cmdqv->vintf_page0) { +return true; + } + +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. * @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, }static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, -uint64_t value) +uint64_t value,Error **errp) {int i;@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset, cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value;break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
In reply to Shameer KolothumQuoted 7 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
Quoted 8 lines
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How do we transfer the state of the cmdq from one to the other?
Thanks
Eric
Quoted 121 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +3
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +42
-2@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static bool +tegra241_cmdqv_munmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +if (!cmdqv->vintf_page0) { +return true; + } + +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. * @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, }static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, -uint64_t value) +uint64_t value,Error **errp) {int i;@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset, cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value;break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 32 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 12 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.
Quoted 6 lines
Quoted 3 lines
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How do we transfer the state of the cmdq from one to the other?
Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.
So when the mapping between a global VCMDQ and a logical VCMDQ is created through the HW_QUEUE ioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.
I will double check this.
Thanks,
Shameer
Quoted 138 lines
Thanks
Eric
Quoted 6 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.h line 18
Quoted 8 lines
2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +0
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3
-0#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2
-0} } +static bool +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error**errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+{ +if (!cmdqv->vintf_page0) { +return true; + }hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV*cmdqv, Error **errp)
hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
Quoted 22 lines
hw/arm/tegra241-cmdqv.c +17
-0+{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu- viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. *
hw/arm/tegra241-cmdqv.c +0
-0@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV*cmdqv, hwaddr offset0, int index,
hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV · line 251
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0} static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddroffset,
hw/arm/tegra241-cmdqv.c static void
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1
-1- uint64_t value) + uint64_t value, Error **errp) { int i;@@ -227,8 +260,10 @@ static voidtegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 11 lines
hw/arm/tegra241-cmdqv.c +2
-0cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-0unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =value;
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-1break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 336
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4
-0__func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 34 lines · Shameer Kolothum Thodi
Quoted 34 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 15 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
Quoted 7 lines
Quoted 3 lines
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How do we transfer the state of the cmdq from one to the other?
Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.
Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of smmuv3_cmdq_consume()), no?
I don't see it done at the moment.
By the way, do we want support of VCMDQs in fully emulated mode. I guess it would be sensible?
Thanks
Eric
Quoted 146 lines
So when the mapping between a global VCMDQ and a logical VCMDQ is created through the
HW_QUEUEioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.I will double check this.
Thanks,
ShameerQuoted 136 lines
Thanks
Eric
Quoted 5 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.h line 18
Quoted 8 lines
2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +0
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3
-0#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2
-0} } +static bool +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error**errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+{ +if (!cmdqv->vintf_page0) { +return true; + }hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV*cmdqv, Error **errp)
hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
Quoted 22 lines
hw/arm/tegra241-cmdqv.c +17
-0+{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu- viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. *
hw/arm/tegra241-cmdqv.c +0
-0@@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV*cmdqv, hwaddr offset0, int index,
hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV · line 251
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0} static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddroffset,
hw/arm/tegra241-cmdqv.c static void
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1
-1- uint64_t value) + uint64_t value, Error **errp) { int i;@@ -227,8 +260,10 @@ static voidtegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-0cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-0unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =value;
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-1break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 336
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4
-0__func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 64 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTFPage0
Quoted 18 lines
Quoted 18 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 12 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How
do
Quoted 5 lines
Quoted 1 line
we transfer the state of the cmdq from one to the other?
Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.
Quoted 7 lines
Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of
smmuv3_cmdq_consume()), no?
I don't see it done at the moment.By the way, do we want support of VCMDQs in fully emulated mode. I guess it would be sensible?
Yes, command handling is not implemented now.
Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then forward the supported invalidation commands back to the host again.
The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.
Thanks,
Shameer
Quoted 153 lines
Thanks
Eric
Quoted 49 lines
So when the mapping between a global VCMDQ and a logical VCMDQ is created through the
HW_QUEUEioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.I will double check this.
Thanks,
ShameerQuoted 39 lines
Thanks
Eric
Quoted 5 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.h line 18
Quoted 8 lines
2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +0
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3
-0#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
Quoted 5 lines
Quoted 5 lines
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2
-0} } +static bool +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv,Error
Quoted 30 lines
Quoted 30 lines
**errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+{ +if (!cmdqv->vintf_page0) { +return true; + }hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV*cmdqv, Error **errp)
Quoted 14 lines
hw/arm/tegra241-cmdqv.c +13
-0+{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu- viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + }Quoted 8 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +4
-0+ +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. *
hw/arm/tegra241-cmdqv.c +0
-0@@ -216,7 +249,7 @@tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
Quoted 4 lines
Quoted 4 lines
*cmdqv, hwaddr offset0, int index,
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0} static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv,hwaddr
Quoted 30 lines
Quoted 30 lines
offset,
hw/arm/tegra241-cmdqv.c static void
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1
-1- uint64_t value) + uint64_t value, Error **errp) { int i;@@ -227,8 +260,10 @@ static voidtegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-0cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-0unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =value;
Quoted 8 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-1break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void*opaque,
Quoted 12 lines
Quoted 12 lines
hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void · line 336
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4
-0__func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 70 lines · Shameer Kolothum Thodi
Quoted 61 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 4 lines · Shameer Kolothum Thodi
Quoted 4 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu-Quoted 8 lines
Quoted 8 lines
[email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTFPage0
Quoted 18 lines
Quoted 18 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 12 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page. How
do
Quoted 5 lines
Quoted 1 line
we transfer the state of the cmdq from one to the other?
Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.
Quoted 7 lines
Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of
smmuv3_cmdq_consume()), no?
I don't see it done at the moment.By the way, do we want support of VCMDQs in fully emulated mode. I guess it would be sensible?
Yes, command handling is not implemented now.
Quoted 7 lines
Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then forward the supported invalidation commands back to the host again.
The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.
OK but really the whole user model is really confusing. One starts using a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are not. Also when the guest is using the global vcmdq, it can access the registers but they actually do not trigger any commands. This really needs to be clarified.
Then I understand we want to focus on accel mode but above points need to be clatified. I just wanted to mention that vcmdq should logically also work in fully emulated mode (and you seemed to implement a mix actually when you have emulated access to vi vcmdq) as it replaces the SMMU standard cmdq, if I understand correctly
Eric
Quoted 156 lines
Thanks,
ShameerQuoted 152 lines
Thanks
Eric
Quoted 48 lines
So when the mapping between a global VCMDQ and a logical VCMDQ is created through the
HW_QUEUEioctl, we should sync the state between the Global VCMDQ Page0 and the VINTF0 Logical VCMDQ Page0.I will double check this.
Thanks,
ShameerQuoted 39 lines
Thanks
Eric
Quoted 5 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.h line 18
Quoted 8 lines
2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +0
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3
-0#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
Quoted 5 lines
Quoted 5 lines
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2
-0} } +static bool +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv,Error
Quoted 30 lines
Quoted 30 lines
**errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+{ +if (!cmdqv->vintf_page0) { +return true; + }hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV*cmdqv, Error **errp)
Quoted 14 lines
hw/arm/tegra241-cmdqv.c +13
-0+{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu- viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + }Quoted 8 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +4
-0+ +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. *
hw/arm/tegra241-cmdqv.c +0
-0@@ -216,7 +249,7 @@tegra241_cmdqv_write_vcmdq(Tegra241CMDQV
Quoted 4 lines
Quoted 4 lines
*cmdqv, hwaddr offset0, int index,
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0} static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv,hwaddr
Quoted 30 lines
Quoted 30 lines
offset,
hw/arm/tegra241-cmdqv.c static void
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +1
-1- uint64_t value) + uint64_t value, Error **errp) { int i;@@ -227,8 +260,10 @@ static voidtegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-0cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque,
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-0unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, · line 312
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0cmdqv->cmdq_alloc_map[(offset - A_CMDQ_ALLOC_MAP_0) / 4] =value;
Quoted 8 lines
Quoted 8 lines
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +1
-1break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void*opaque,
Quoted 12 lines
Quoted 12 lines
hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void · line 336
Quoted 1 line
hw/arm/tegra241-cmdqv.c +0
-0qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%"PRIx64 "\n",
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +4
-0__func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 107 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:19
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTFPage0
Quoted 5 lines
Quoted 5 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 7 lines
Quoted 7 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 6 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];Nicolin
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Chen <[email protected]>; Nathan Chen <[email protected]>;
Matt
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Ochs <[email protected]>; Jiandi An <[email protected]>; Jason
Gunthorpe
Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTFPage0
Quoted 8 lines
Quoted 8 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 2 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association
between
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
VCMDQs and a VINTF is only established after
HW_QUEUEallocation.Prior
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
to that, the mapped Page0 does not back any real VCMDQ state.
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page.
How
Quoted 6 lines
Quoted 6 lines
do
Quoted 5 lines
Quoted 1 line
we transfer the state of the cmdq from one to the other?
Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.
Quoted 6 lines
Quoted 6 lines
Also assuming the global VCMDQs are accessible and thus used, I guess we shall properly handle the actual commands (equivalent of
smmuv3_cmdq_consume()), no?
I don't see it done at the moment.By the way, do we want support of VCMDQs in fully emulated mode. I
guess
Quoted 5 lines
Quoted 1 line
it would be sensible?
Yes, command handling is not implemented now.
Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then
forward
Quoted 5 lines
the supported invalidation commands back to the host again.
The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.
Quoted 6 lines
OK but really the whole user model is really confusing. One starts using a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are not. Also when the guest is using the global vcmdq, it can access the registers but they actually do not trigger any commands. This really needs to be clarified.
Yes, this could use better documentation. I can clarify the model in the commit message and code comments.
One option we discussed earlier to avoid the confusion was to restrict usage to the "VINTF logical VCMDQ" interface and return an error if the guest accesses the "Global VCMDQ registers".
However, since this is still MMIO space, the thinking was that it may be better to allow the access even if it is effectively non-functional, rather than making the behaviour unpredictable.
See the discussion:
https://lore.kernel.org/qemu-devel/aa8cJszkOKGLUYdD@Asurada-Nvidia/
Quoted 5 lines
Then I understand we want to focus on accel mode but above points need to be clatified. I just wanted to mention that vcmdq should logically also work in fully emulated mode (and you seemed to implement a mix actually when you have emulated access to vi vcmdq) as it replaces the SMMU standard cmdq, if I understand correctly
IIUC, in the current Linux implementation VCMDQ does not replace the standard SMMU CMDQ from the guest perspective. This is also why we do not expose the HYP_OWN bit to the guest.
When HYP_OWN is not set, guest still uses the normal SMMU CMDQ. CMDQ is only used as a fast-path for a small set of invalidation commands that can be sent directly to the host hardware.
See:
static bool tegra241_guest_vcmdq_supports_cmd(struct arm_smmu_cmdq_ent *ent)
{
switch (ent->opcode) {
case CMDQ_OP_TLBI_NH_ASID:
case CMDQ_OP_TLBI_NH_VA:
case CMDQ_OP_ATC_INV:
return true;
default:
return false;
}
}So VCMDQ is not a replacement for the SMMU CMDQ in the guest. It is only used to accelerate these invalidation commands.
Thanks,
Shameer
A message that is not in the archive (sent off-list or not yet received): [email protected]
RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 191 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 March 2026 15:11
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:19
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTFPage0
Quoted 5 lines
Quoted 5 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 7 lines
Quoted 7 lines · Shameer Kolothum Thodi
Quoted 6 lines
Quoted 6 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 10:05
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];Nicolin
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Chen <[email protected]>; Nathan Chen <[email protected]>;
Matt
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Ochs <[email protected]>; Jiandi An <[email protected]>; Jason
Gunthorpe
Quoted 17 lines
Quoted 17 lines
Quoted 4 lines
Quoted 4 lines
<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTFPage0
Quoted 12 lines
Quoted 12 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 6 lines · Shameer Kolothum Thodi
Quoted 6 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 07:55
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected];Quoted 7 lines
Quoted 7 lines
Nicolin
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Chen <[email protected]>; Nathan Chen <[email protected]>;
Matt
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Ochs <[email protected]>; Jiandi An <[email protected]>; Jason
Gunthorpe
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant
Jaju
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
<[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmapVINTF
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Page0
Quoted 9 lines
Quoted 9 lines
for CMDQV
External email: Use caution opening links or attachments
Quoted 3 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while
the
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes
a
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region
during
Quoted 9 lines
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
vIOMMU allocation. However, the logical-to-physical association
between
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
VCMDQs and a VINTF is only established after
HW_QUEUEallocation.Prior
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
to that, the mapped Page0 does not back any real VCMDQ state.
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation
support.
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
So at some point we transition from something that is purely emulated (page 1 global cmdq) to something that is mmapped on a host page.
How
Quoted 7 lines
Quoted 7 lines
do
Quoted 5 lines
Quoted 1 line
we transfer the state of the cmdq from one to the other?
Right. If a guest uses both the "Global VCMDQ registers Page0" and the "VINTF0 Logical VCMDQ registers Page0" interchangeably (and I see nothing in the spec that forbids this), then we need to keep the two views in sync.
Also assuming the global VCMDQs are accessible and thus used, I guess
we
Quoted 49 lines
Quoted 17 lines
Quoted 5 lines
Quoted 5 lines
shall properly handle the actual commands (equivalent of
smmuv3_cmdq_consume()), no?
I don't see it done at the moment.By the way, do we want support of VCMDQs in fully emulated mode. I
guess
Quoted 5 lines
Quoted 1 line
it would be sensible?
Yes, command handling is not implemented now.
Since this is part of the accelerated SMMUv3 feature, I am not sure it makes sense to implement full command handling in QEMU and then
forward
Quoted 5 lines
the supported invalidation commands back to the host again.
The idea here is to accelerate the CMDQ processing through the hardware. I am not sure a guest would have a real use case for VCMDQ in a fully emulated mode.
Quoted 6 lines
OK but really the whole user model is really confusing. One starts using a "global" vcmdq and then switches to a logical one.
On real HW I guess they are synced. But on emulation, currently they are not. Also when the guest is using the global vcmdq, it can access the registers but they actually do not trigger any commands. This really needs to be clarified.Yes, this could use better documentation. I can clarify the model in the commit message and code comments.
One option we discussed earlier to avoid the confusion was to restrict usage to the "VINTF logical VCMDQ" interface and return an error if the guest accesses the "Global VCMDQ registers".
However, since this is still MMIO space, the thinking was that it may be better to allow the access even if it is effectively non-functional, rather than making the behaviour unpredictable.
See the discussion:
https://lore.kernel.org/qemu-devel/aa8cJszkOKGLUYdD@Asurada-Nvidia/Quoted 5 lines
Then I understand we want to focus on accel mode but above points need to be clatified. I just wanted to mention that vcmdq should logically also work in fully emulated mode (and you seemed to implement a mix actually when you have emulated access to vi vcmdq) as it replaces the SMMU standard cmdq, if I understand correctly
IIUC, in the current Linux implementation VCMDQ does not replace the standard SMMU CMDQ from the guest perspective. This is also why we do not expose the
HYP_OWNbit to the guest.
WhenHYP_OWNis not set, guest still uses the normal SMMU CMDQ. CMDQ is only used as a fast-path for a small set of invalidation commands that can be sent directly to the host hardware.Quoted 3 lines
See:
static bool tegra241_guest_vcmdq_supports_cmd(struct
arm_smmu_cmdq_ent*ent)Quoted 11 lines
{ switch (ent->opcode) { case CMDQ_OP_TLBI_NH_ASID: case CMDQ_OP_TLBI_NH_VA: case CMDQ_OP_ATC_INV: return true; default: return false; } }Quoted 2 lines
So VCMDQ is not a replacement for the SMMU CMDQ in the guest. It is only used to accelerate these invalidation commands.
I understand. But guest works in
HYP_OWNmode as it sets theHYP_OWNbit. So resetting it looks like a hack to me, not documented in any way in the spec. Besides, astegra241_guest_vcmdq_supports_cmd()forces to send on the cmdqv cmdq only legal cmds I don't see any problem leaving the bit to 1. What is important is that the host does the check (and the host setsHYP_OWNto 0 for that cmdq). I did the trial to comment
Quoted 3 lines
//value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;and it seems to work properly
Ah.. that’s interesting. .I was under the impression that we are toggling the HYP_OWN so that guest does sets the tegra241_guest_vcmdq_supports_cmd() correctly and delegates the supported cmds through that queue.
See,
tegra241_cmdqv_init_structures()
tegra241_vintf_alloc_lvcmdq()
tegra241_vcmdq_alloc_smmu_cmdq()
{
....
if (!vcmdq->vintf->hyp_own)
cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;However, it looks like the above happens before tegra241_vintf_hw_init(), which sets HYP_OWN. That may explain why toggling the bit here does not make any difference.
I need to double check why we did the toggling in the QEMU then.
@Nicolin, do you remember?
Thanks,
Shameer
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 23 lines · Shameer Kolothum Thodi
Quoted 9 lines
I understand. But guest works in
HYP_OWNmode as it sets theHYP_OWNbit. So resetting it looks like a hack to me, not documented in any way in the spec. Besides, astegra241_guest_vcmdq_supports_cmd()forces to send on the cmdqv cmdq only legal cmds I don't see any problem leaving the bit to 1. What is important is that the host does the check (and the host setsHYP_OWNto 0 for that cmdq). I did the trial to comment//value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK;and it seems to work properly
Ah.. that’s interesting. .I was under the impression that we are toggling the
HYP_OWNso that guest does sets thetegra241_guest_vcmdq_supports_cmd()correctly and delegates the supported cmds through that queue.See,
tegra241_cmdqv_init_structures() tegra241_vintf_alloc_lvcmdq() tegra241_vcmdq_alloc_smmu_cmdq() { .... if (!vcmdq->vintf->hyp_own) cmdq->supports_cmd = tegra241_guest_vcmdq_supports_cmd;
Quoted 3 lines
However, it looks like the above happens before
tegra241_vintf_hw_init(), which setsHYP_OWN. That may explain why toggling the bit here does not make any difference.
Well, looks like we exposed a kernel bug...
It seems that the sequence of setting hyp_own and supports_cmd got inverted during a rework when I got the driver upstream.
It turned out that both host and guest kernels sets supports_cmd to tegra241_guest_vcmdq_supports_cmd().. IOW, the host is sending non- invalidation commands to the SMMU CMDQ rather than the HYP_OWNed VCMDQ, although this is completely harmless and probably does not affect host performance at all since non-invalidation commands are not that frequently issued.
This explains why Eric's experiment didn't fail. I'll submit a fix.
With that being said, in QEMU we do need to wire HYP_OWN bit to 0. This is not decided by the spec, but by the host SW implementation:
550 /*
551 * Note that HYP_OWN bit is wired to zero when running in guest kernel, 552 * whether enabling it here or not, as !HYP_OWN cmdq HWs only support a 553 * restricted set of supported commands. 554 */ 555 regval = FIELD_PREP(VINTF_HYP_OWN, hyp_own) | 556 FIELD_PREP(VINTF_VMID, vintf->vsmmu.vmid); 557 writel(regval, REG_VINTF(vintf, CONFIG)); 558 559 ret = vintf_write_config(vintf, regval | VINTF_EN); 560 if (ret) 561 return ret; 562 /* 563 * As being mentioned above, HYP_OWN bit is wired to zero for a guest 564 * kernel, so read it back from HW to ensure that reflects in hyp_own 565 */
566 vintf->hyp_own = !!(VINTF_HYP_OWN & readl(REG_VINTF(vintf, CONFIG)));
This should have been mentioned in the uAPI header. I'll submit a patch to add this line near IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.
One thing we need to get our head around is that a guest hardware is different than the host hardware:
- Host HW is backed by a VINTF (HYP_OWN=1)
- Guest HW is backed by a VINTF (HYP_OWN=0)
This is a fixed configuration, decided by the hardware and the spec also.
Thus, the guest version of the VINTF must align with its underlying hardware, so QEMU cannot allow setting HYP_OWN=1.
Otherwise, guest OS would have no idea this is a guest version of hardware and would issue unsupported commands that will fail.
Nicolin
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
In reply to Shameer KolothumQuoted 11 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.
So what does happen if the guest attempts to access the VI CMDQ before its mmapping.
At the moment MMIO accesses are valid as VI_VCMDQ0 MMIOs are handled in the same MMIO region as the Config page.
Is it the expected behavior? I guess you should signal an error and maaybe signal an interrupt if it is not masked for this VI?
Spec says:"While the SW can program the virtual CMDQ(s) directly using the direct VCMDQ aperture (and not though the Virtual Interface), it is required hat the VCMDQ be allocated to a VI before it is used to send commands to the SMMU
Note the terminology used here: direct VCMDQ aparture (not Global)
Thanks
Eric
Quoted 125 lines
When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +3
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +42
-2@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static bool +tegra241_cmdqv_munmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +if (!cmdqv->vintf_page0) { +return true; + } + +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (cmdqv->vintf_page0) { +return true; + } + +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VINTF_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. * @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, }static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, -uint64_t value) +uint64_t value,Error **errp) {int i;@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset, cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value;break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
In reply to Shameer KolothumQuoted 68 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +3
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U << TEGRA241_CMDQV_NUM_CMDQ_LOG2)#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +16
-0@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static bool +tegra241_cmdqv_munmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +if (!cmdqv->vintf_page0) { +return true; + } + +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 179
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+{ + IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + + if (cmdqv->vintf_page0) { + return true; + } + + if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu->viommu_id, + VINTF_PAGE_SIZE, + cmdqv->cmdqv_data.out_vintf_mmap_offset,
If I am not wrong the mmap_offset is only available after iommufd_backend_alloc_hw_queue(). So I would suggest to first introduce the patch that does the iommufd_backend_alloc_hw_queue (ie. 19/32) and then this one
Eric
Quoted 58 lines
hw/arm/tegra241-cmdqv.c +16
-2+ &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. * @@ -216,7 +249,7 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, }static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, -uint64_t value) +uint64_t value,Error **errp) {int i;@@ -227,8 +260,10 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset, cmdqv->vintf_config = value;if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { +tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }break ;@@ -251,6 +286,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;int index;if (offset >=TEGRA241_CMDQV_IO_LEN ) {@@ -276,7 +312,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value;break ; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: -tegra241_cmdqv_write_vintf (cmdqv, offset, value); +tegra241_cmdqv_write_vintf (cmdqv, offset, value, &local_err);break ; case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ1_GERRORN:/* Same decoding as read() case: See comments above */ @@ -300,6 +336,10 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
RE: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Hi Eric,
Quoted 91 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 13:59
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 17/32] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Quoted 16 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.When VINTF is enabled, mmap the kernel provided Page0 region and unmap it when VINTF is disabled. This prepares the VINTF mapping in advance of subsequent patches that add VCMDQ allocation support.
Quoted 5 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 3 +++ hw/arm/tegra241-cmdqv.c | 44+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.h line 18
Quoted 8 lines
2 files changed, 45 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.h +0
-0@@ -18,6 +18,8 @@#define TEGRA241_CMDQV_MAX_CMDQ (1U <<
TEGRA241_CMDQV_NUM_CMDQ_LOG2)hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 20 lines
hw/arm/tegra241-cmdqv.h +3
-0#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 4 +#define VINTF_PAGE_SIZE 0x10000 +/* * Tegra241 CMDQV MMIO layout (64KB pages) * @@ -34,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0;/* Register Cache */ uint32_t config;hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,39 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +2
-0} } +static bool +tegra241_cmdqv_munmap_vintf_page0(Tegra241CMDQV *cmdqv, Error**errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+{ +if (!cmdqv->vintf_page0) { +return true; + }hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +if (munmap (cmdqv->vintf_page0,VINTF_PAGE_SIZE ) <0 ) { +error_setg_errno (errp, errno,"Failed to unmap VINTF page0" ); +return false; + } + cmdqv->vintf_page0 =NULL ; +return true; +} + +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv,Error **errp)
Quoted 15 lines
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 179
Quoted 11 lines
hw/arm/tegra241-cmdqv.c +10
-0+{ + IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + + if (cmdqv->vintf_page0) { + return true; + } + + if (!iommufd_backend_viommu_mmap(viommu->iommufd, viommu- viommu_id, + VINTF_PAGE_SIZE, + cmdqv->cmdqv_data.out_vintf_mmap_offset,If I am not wrong the
mmap_offsetis only available afteriommufd_backend_alloc_hw_queue(). So I would suggest to first introduce the patch that does theiommufd_backend_alloc_hw_queue(ie. 19/32) and then this one
The out_vintf_mmap_offset is actually populated during IOMMU_VIOMMU_ALLOC (via iommufd_backend_alloc_viommu()), not IOMMU_HW_QUEUE_ALLOC. It is stored in cmdqv->cmdqv_data.out_vintf_mmap_offset at viommu allocation time and is therefore already available. iommufd_backend_alloc_hw_queue() only returns the hw_queue_id,
not the mmap offset.
Thanks,
Shameer
PATCH v3 18/32 system/physmem: Add address_space_is_ram() helper
In reply to Shameer KolothumIntroduce address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.
--- include/system/memory.h | 10 ++++++++++ system/physmem.c | 11 +++++++++++ 2 files changed, 21 insertions(+)
include/system/memory.h +10 -0
@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,*/ bool address_space_is_io (AddressSpace *as,hwaddr addr); +/** +* address_space_is_ram: check whether a guest physical address whithin +* an address space is RAM. +* +* @as: #AddressSpace to be accessed +* @addr: address within that address space +*/ + +bool address_space_is_ram (AddressSpace *as,hwaddr addr); +/* address_space_map: map a physical memory region into a host virtual address * * May map a subset of the requested range, given by and returned in @plen.
system/physmem.c +11 -0
@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)return !(memory_region_is_ram (mr) ||memory_region_is_romd (mr)); } +bool address_space_is_ram (AddressSpace *as,hwaddr addr) +{ +MemoryRegion *mr; + +RCU_READ_LOCK_GUARD (); + mr =address_space_translate (as, addr, &addr,NULL , false, +MEMTXATTRS_UNSPECIFIED ); + +return memory_region_is_ram (mr); +} + static hwaddr flatview_extend_translation(FlatView *fv, hwaddr addr, hwaddr target_len,
Signature · 1 line
2.43.0
PATCH v3 19/32 hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.
VCMDQ_EN is part of the VCMDQ_CONFIG register, which is accessed through the VINTF Page0 region. This region is mapped directly into the guest address space (introduced in a subsequent patch), so QEMU does not trap writes to VCMDQ_CONFIG.
Since VCMDQ_EN writes are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.
If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.
Writes with invalid addresses are ignored.
All allocated VCMDQs are freed when CMDQV or VINTF is disabled.
--- hw/arm/tegra241-cmdqv.h | 11 +++++++ hw/arm/tegra241-cmdqv.c | 71 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 79 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.h +11 -0
@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ ];void *vintf_page0;/* Register Cache */
@@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +static inline bool tegra241_cmdq_enabled (Tegra241CMDQV *cmdq) +{ +return cmdq->status &R_STATUS_CMDQV_ENABLED_MASK ; +} + +static inline bool tegra241_vintf_enabled (Tegra241CMDQV *cmdq) +{ +return cmdq->vintf_status &R_VINTF0_STATUS_ENABLE_OK_MASK ; +} +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */
hw/arm/tegra241-cmdqv.c +68 -3
118 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
Quoted 74 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.
VCMDQ_ENis part of theVCMDQ_CONFIGregister, which is accessed through the VINTF Page0 region. This region is mapped directly into the guest address space (introduced in a subsequent patch), so QEMU does not trap writes toVCMDQ_CONFIG.Since
VCMDQ_ENwrites are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.
Writes with invalid addresses are ignored.
All allocated VCMDQs are freed when CMDQV or VINTF is disabled.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 11 +++++++ hw/arm/tegra241-cmdqv.c | 71 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 79 insertions(+), 3 deletions(-)hw/arm/tegra241-cmdqv.h +11
-0@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ ];void *vintf_page0;/* Register Cache */ @@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +static inline bool tegra241_cmdq_enabled (Tegra241CMDQV *cmdq) +{ +return cmdq->status &R_STATUS_CMDQV_ENABLED_MASK ; +} + +static inline bool tegra241_vintf_enabled (Tegra241CMDQV *cmdq) +{ +return cmdq->vintf_status &R_VINTF0_STATUS_ENABLE_OK_MASK ; +} +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +8
-0@@ -151,6 +151,67 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static void tegra241_cmdqv_free_vcmdq (Tegra241CMDQV *cmdqv,int index) +{ +SMMUv3AccelState *accel = cmdqv->s_accel; +IOMMUFDViommu *viommu = accel->viommu; +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; + +if (!vcmdq) { +return ;
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 171
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ } +iommufd_backend_free_id (viommu->iommufd, vcmdq->hw_queue_id); +g_free (vcmdq); + cmdqv->vcmdq[index] =NULL ; +} + +static void tegra241_cmdqv_free_all_vcmdq (Tegra241CMDQV *cmdqv) +{ +/* Free in the reverse order to avoid "resource busy" error */ +for (int i = (TEGRA241_CMDQV_MAX_CMDQ -1 ); i >=0 ; i--) {
Is it future proof to restrict the number of queues to 2? Shouldn't it be configurable?
Eric
Quoted 103 lines
hw/arm/tegra241-cmdqv.c +50
-3+tegra241_cmdqv_free_vcmdq (cmdqv, i); + } +} + +static bool tegra241_cmdqv_setup_vcmdq (Tegra241CMDQV *cmdqv,int index, +Error **errp) +{ +SMMUv3AccelState *accel = cmdqv->s_accel; +uint64_t base_mask = (uint64_t )R_VCMDQ0_BASE_L_ADDR_MASK | + (uint64_t )R_VCMDQ0_BASE_H_ADDR_MASK <<32 ; +uint64_t addr = cmdqv->vcmdq_base[index] & base_mask; +uint64_t log2 = cmdqv->vcmdq_base[index] &R_VCMDQ0_BASE_L_LOG2SIZE_MASK ; +uint64_t size =1ULL << (log2 +4 ); +IOMMUFDViommu *viommu = accel->viommu; +IOMMUFDHWqueue *hw_queue; +uint32_t hw_queue_id; + +/* Ignore any invalid address. This may come as part of reset etc */ +if (!address_space_is_ram (&address_space_memory, addr) || + !address_space_is_ram (&address_space_memory, addr + size -1 )) { +return true; + } + +if (!tegra241_cmdq_enabled (cmdqv) || !tegra241_vintf_enabled (cmdqv)) { +return true; + } + +tegra241_cmdqv_free_vcmdq (cmdqv, index); + +if (!iommufd_backend_alloc_hw_queue (viommu->iommufd, viommu->viommu_id, +IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV , + index, addr, size, &hw_queue_id, + errp)) { +return false; + } + hw_queue =g_new (IOMMUFDHWqueue,1 ); + hw_queue->hw_queue_id = hw_queue_id; + hw_queue->viommu = viommu; + cmdqv->vcmdq[index] = hw_queue; + +return true; +} +static bool tegra241_cmdqv_munmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) {@@ -192,7 +253,7 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)*/ static void tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index, -uint64_t value,unsigned size) +uint64_t value,unsigned size,Error **errp) {switch (offset0) {case A_VCMDQ0_CONS_INDX :@@ -220,11 +281,13 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | (value &0xffffffffULL ); } +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp);return ;case A_VCMDQ0_BASE_H : cmdqv->vcmdq_base[index] = (cmdqv->vcmdq_base[index] &0xffffffffULL ) | ((uint64_t )value <<32 ); +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp);return ;case A_VCMDQ0_CONS_INDX_BASE_DRAM_L :if (size ==8 ) {@@ -263,6 +326,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_free_all_vcmdq (cmdqv);tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; }@@ -302,6 +366,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,if (value &R_CONFIG_CMDQV_EN_MASK ) { cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; }else { +tegra241_cmdqv_free_all_vcmdq (cmdqv); cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; }break ;@@ -321,7 +386,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VCMDQ0_CONS_INDX ... A_VCMDQ1_GERRORN: index = (offset -0x10000 ) /0x80 ;tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, - size); + size, &local_err);break ; case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ1_CONS_INDX_BASE_DRAM_H:/* Same decoding as read() case: See comments above */ @@ -330,7 +395,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VCMDQ0_BASE_L ... A_VCMDQ1_CONS_INDX_BASE_DRAM_H: index = (offset -0x20000 ) /0x80 ;tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, - size); + size, &local_err);break ;default : qemu_log_mask(LOG_UNIMP, "%s unhandled write access at 0x%" PRIx64 "\n",
RE: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
Quoted 101 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 14:34
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 19/32] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programmingExternal email: Use caution opening links or attachments
Quoted 20 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add support for allocating IOMMUFD hardware queues when the guest programs the VCMDQ BASE registers.
VCMDQ_ENis part of theVCMDQ_CONFIGregister, which is accessed through the VINTF Page0 region. This region is mapped directly into the guest address space (introduced in a subsequent patch), so QEMU does not trap writes toVCMDQ_CONFIG.Since
VCMDQ_ENwrites are not trapped, QEMU cannot allocate the hardware queue based on that bit. Instead, allocate the IOMMUFD hardware queue when the guest writes a VCMDQ BASE register with a valid RAM-backed address and when CMDQV and VINTF are enabled.If a hardware queue was previously allocated for the same VCMDQ, free it before reallocation.
Writes with invalid addresses are ignored.
Quoted 7 lines
All allocated VCMDQs are freed when CMDQV or VINTF is disabled.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.h | 11 +++++++ hw/arm/tegra241-cmdqv.c | 71+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void
Quoted 36 lines
2 files changed, 79 insertions(+), 3 deletions(-)hw/arm/tegra241-cmdqv.h +11
-0@@ -36,6 +36,7 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ ];void *vintf_page0;/* Register Cache */ @@ -322,6 +323,16 @@ A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(1) A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(0) A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(1) +static inline bool tegra241_cmdq_enabled (Tegra241CMDQV *cmdq) +{ +return cmdq->status &R_STATUS_CMDQV_ENABLED_MASK ; +} + +static inline bool tegra241_vintf_enabled (Tegra241CMDQV *cmdq) +{ +return cmdq->vintf_status &R_VINTF0_STATUS_ENABLE_OK_MASK ; +} +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_get_ops (void );#endif /* HW_ARM_TEGRA241_CMDQV_H */ hw/arm/tegra241-cmdqv.c +0
-0@@ -151,6 +151,67 @@ static uint64_t tegra241_cmdqv_read(void*opaque, hwaddr offset, unsigned size)
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void · line 154
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +1
-0} } +static void tegra241_cmdqv_free_vcmdq(Tegra241CMDQV *cmdqv, intindex)
Quoted 11 lines
hw/arm/tegra241-cmdqv.c +11
-0+{ +SMMUv3AccelState *accel = cmdqv->s_accel; +IOMMUFDViommu *viommu = accel->viommu; +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; + +if (!vcmdq) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, vcmdq->hw_queue_id); +g_free (vcmdq); + cmdqv->vcmdq[index] =NULL ;
Quoted 8 lines
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 171
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+} + +static void tegra241_cmdqv_free_all_vcmdq (Tegra241CMDQV *cmdqv) +{ +/* Free in the reverse order to avoid "resource busy" error */ +for (int i = (TEGRA241_CMDQV_MAX_CMDQ -1 ); i >=0 ; i--) {Is it future proof to restrict the number of queues to 2? Shouldn't it be configurable?
On current platforms the host reports log2vcmdqs = 1 (2 VCMDQs). I think, for the initial implementation we can keep the number of queues to 2. We can make it configurable if the requirement comes up in the future.
Thanks,
Shameer
PATCH v3 20/32 hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing
In reply to Shameer KolothumWhen a VCMDQ is allocated and VINTF page0 has been mmap'ed from the kernel, access the VCMDQ registers directly through the VINTF page0 backing instead of using QEMU's cached register state.
VINTF page0 provides the backing memory region for VCMDQ registers once a hardware queue is created. In that case, reads and writes should reflect the live backing state.
If a VCMDQ is not allocated, or if VINTF page0 is not available, continue to use the cached register values maintained by QEMU.
--- hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +46 -1
@@ -14,17 +14,44 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static inline uint32_t *tegra241_cmdqv_vintf_ptr (Tegra241CMDQV *cmdqv, +int index,hwaddr offset0) +{ +if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) { +return NULL ; + } + +return (uint32_t *)(cmdqv->vintf_page0 + (index *0x80 ) + + (offset0 -0x10000 )); +}/* * Read a VCMDQ register using VCMDQ0_* offsets. * * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. * -* All VCMDQ accesses return cached registers. +* If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly +* from the VINTF page0 backing. Otherwise, fall back to cached state. */ static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index) { +uint32_t *ptr =tegra241_cmdqv_vintf_ptr (cmdqv, index, offset0); + +if (ptr) { +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_STATUS : +case A_VCMDQ0_GERROR : +case A_VCMDQ0_GERRORN : +return *ptr; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX :return cmdqv->vcmdq_cons_indx[index];
@@ -250,11 +277,29 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)* * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly +* to the VINTF page0 backing. Otherwise, update cached state. */ static void tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index,uint64_t value,unsigned size,Error **errp) { +uint32_t *ptr =tegra241_cmdqv_vintf_ptr (cmdqv, index, offset0); + +if (ptr) { +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_GERRORN : + *ptr = (uint32_t )value; +return ; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX : cmdqv->vcmdq_cons_indx[index] = value;
Signature · 1 line
2.43.0
Re: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing
Quoted 10 lines · Shameer Kolothum
When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the kernel, access the VCMDQ registers directly through the VINTF page0 backing instead of using QEMU's cached register state.
VINTF page0 provides the backing memory region for VCMDQ registers once a hardware queue is created. In that case, reads and writes should reflect the live backing state.
If a VCMDQ is not allocated, or if VINTF page0 is not available, continue to use the cached register values maintained by QEMU.
But if both coexist shouldn't you also sync emulated/mmaped VINTF CMDQV. Somehow this means that you have 3 states of the same CMDQV: the global/direct one, the emulated VI CMDQV and the mmapped VI CMDQV (host backed).
Is this understanding correct?
Thanks
Eric
Quoted 86 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 47 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.c +46
-1@@ -14,17 +14,44 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static inline uint32_t *tegra241_cmdqv_vintf_ptr (Tegra241CMDQV *cmdqv, +int index,hwaddr offset0) +{ +if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) { +return NULL ; + } + +return (uint32_t *)(cmdqv->vintf_page0 + (index *0x80 ) + + (offset0 -0x10000 )); +}/* * Read a VCMDQ register using VCMDQ0_* offsets. * * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. * -* All VCMDQ accesses return cached registers. +* If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly +* from the VINTF page0 backing. Otherwise, fall back to cached state. */ static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index) { +uint32_t *ptr =tegra241_cmdqv_vintf_ptr (cmdqv, index, offset0); + +if (ptr) { +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_STATUS : +case A_VCMDQ0_GERROR : +case A_VCMDQ0_GERRORN : +return *ptr; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX :return cmdqv->vcmdq_cons_indx[index];@@ -250,11 +277,29 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)* * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly +* to the VINTF page0 backing. Otherwise, update cached state. */ static void tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index,uint64_t value,unsigned size,Error **errp) { +uint32_t *ptr =tegra241_cmdqv_vintf_ptr (cmdqv, index, offset0); + +if (ptr) { +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_GERRORN : + *ptr = (uint32_t )value; +return ; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX : cmdqv->vcmdq_cons_indx[index] = value;
RE: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing
Quoted 28 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 14:53
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 20/32] hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backingExternal email: Use caution opening links or attachments
Quoted 8 lines · Shameer Kolothum
When a VCMDQ is allocated and VINTF page0 has been mmap'ed from the kernel, access the VCMDQ registers directly through the VINTF page0 backing instead of using QEMU's cached register state.
VINTF page0 provides the backing memory region for VCMDQ registers once a hardware queue is created. In that case, reads and writes should reflect the live backing state.
Quoted 8 lines
Quoted 2 lines
If a VCMDQ is not allocated, or if VINTF page0 is not available, continue to use the cached register values maintained by QEMU.
But if both coexist shouldn't you also sync emulated/mmaped VINTF CMDQV. Somehow this means that you have 3 states of the same CMDQV: the global/direct one, the emulated VI CMDQV and the mmapped VI CMDQV (host
backed).
Is this understanding correct?
Yes. However, in the current implementation the VCMDQ is functionally valid only if QEMU successfully sets up the VINTF page0 mmap and creates a backing VCMDQ through the HW_QUEUE ioctl.
All other cases it just uses the cached registers for MMIO read/write.
Maybe we should expose such failures to the guest, by injecting an error or by setting STATUS/CONFIG to indicate that the VCMDQ is not functional.
Regarding keeping the Global VCMDQ and the VI CMDQ in sync always, this would only be possible if we do not map (tegra241_cmdqv_guest_map_vintf_page0) the VINTF Page0 into guest memory. Once it is mapped, accesses to the VI CMDQ (control/status registers) no longer trap to QEMU.
However, not mapping it is not advisable as it would hurt performance, since every CONS/PROD index update would trap to QEMU.
Thanks,
Shameer
Quoted 95 lines
Thanks
Eric
Quoted 4 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 47++++++++++++++++++++++++++++++++++++++++-
hw/arm/tegra241-cmdqv.c line 17
Quoted 11 lines
1 file changed, 46 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.c +1
-0@@ -14,17 +14,44 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV*cmdqv,
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9
-0+ int index, hwaddr offset0) +{ +if (!cmdqv->vcmdq[index] || !cmdqv->vintf_page0) { +return NULL ; + } + +return (uint32_t *)(cmdqv->vintf_page0 + (index *0x80 ) + + (offset0 -0x10000 )); +}/* hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +2
-1* Read a VCMDQ register using VCMDQ0_* offsets. * * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. * -* All VCMDQ accesses return cached registers. +* If the VCMDQ is allocated and VINTF page0 is mmap'ed, read directly +* from the VINTF page0 backing. Otherwise, fall back to cached state. */ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,hwaddr offset0,
hw/arm/tegra241-cmdqv.c static bool
Quoted 22 lines
hw/arm/tegra241-cmdqv.c +16
-0int index) { +uint32_t *ptr =tegra241_cmdqv_vintf_ptr (cmdqv, index, offset0); + +if (ptr) { +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_STATUS : +case A_VCMDQ0_GERROR : +case A_VCMDQ0_GERRORN : +return *ptr; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX :return cmdqv->vcmdq_cons_indx[index];@@ -250,11 +277,29 @@ static bool
tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +3
-0* * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* If the VCMDQ is allocated and VINTF page0 is mmap'ed, write directly +* to the VINTF page0 backing. Otherwise, update cached state. */ static void tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,int index,
Quoted 20 lines
hw/arm/tegra241-cmdqv.c +15
-0uint64_t value, unsigned size, Error **errp) { + uint32_t *ptr = tegra241_cmdqv_vintf_ptr(cmdqv, index, offset0); + +if (ptr) { +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_GERRORN : + *ptr = (uint32_t )value; +return ; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX : cmdqv->vcmdq_cons_indx[index] = value;
PATCH v3 21/32 hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
VINTF page0 is backed by host memory returned by the kernel via mmap. Instead of trapping and emulating accesses to this region, map it directly into the guest-visible MMIO space.
The VINTF page0 MMIO region is created lazily when the first VCMDQ hardware queue is allocated. When CMDQV or VINTF is disabled, the region is removed.
--- hw/arm/tegra241-cmdqv.h | 1 + hw/arm/tegra241-cmdqv.c | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 36 insertions(+)
hw/arm/tegra241-cmdqv.h +1 -0
@@ -38,6 +38,7 @@ typedef struct Tegra241CMDQV {qemu_irq irq;IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ ];void *vintf_page0; +MemoryRegion *mr_vintf_page0;/* Register Cache */ uint32_t config;
hw/arm/tegra241-cmdqv.c +35 -0
@@ -178,6 +178,38 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static void tegra241_cmdqv_guest_unmap_vintf_page0 (Tegra241CMDQV *cmdqv) +{ +if (!cmdqv->mr_vintf_page0) { +return ; + } + +memory_region_del_subregion (&cmdqv->mmio_cmdqv, cmdqv->mr_vintf_page0); +object_unparent (OBJECT (cmdqv->mr_vintf_page0)); +g_free (cmdqv->mr_vintf_page0); + cmdqv->mr_vintf_page0 =NULL ; +} + +static void tegra241_cmdqv_guest_map_vintf_page0 (Tegra241CMDQV *cmdqv) +{ +char *name; + +if (cmdqv->mr_vintf_page0) { +return ; + } + + name =g_strdup_printf ("%s vintf-page0" , +memory_region_name (&cmdqv->mmio_cmdqv)); + cmdqv->mr_vintf_page0 =g_malloc0 (sizeof (*cmdqv->mr_vintf_page0)); +memory_region_init_ram_device_ptr (cmdqv->mr_vintf_page0, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,VINTF_PAGE_SIZE , + cmdqv->vintf_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + cmdqv->mr_vintf_page0,1 ); +g_free (name); +} +static void tegra241_cmdqv_free_vcmdq (Tegra241CMDQV *cmdqv,int index) {SMMUv3AccelState *accel = cmdqv->s_accel;
@@ -236,6 +268,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, hw_queue->viommu = viommu; cmdqv->vcmdq[index] = hw_queue; +tegra241_cmdqv_guest_map_vintf_page0 (cmdqv);return true; }
@@ -371,6 +404,7 @@ static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr offset,tegra241_cmdqv_mmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; }else { +tegra241_cmdqv_guest_unmap_vintf_page0 (cmdqv);tegra241_cmdqv_free_all_vcmdq (cmdqv);tegra241_cmdqv_munmap_vintf_page0 (cmdqv, errp); cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ;
@@ -411,6 +445,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,if (value &R_CONFIG_CMDQV_EN_MASK ) { cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; }else { +tegra241_cmdqv_guest_unmap_vintf_page0 (cmdqv);tegra241_cmdqv_free_all_vcmdq (cmdqv); cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; }
Signature · 1 line
2.43.0
PATCH v3 22/32 hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
In reply to Shameer KolothumAllocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.
--- hw/arm/smmuv3-accel.h | 2 ++ hw/arm/tegra241-cmdqv.h | 1 + hw/arm/smmuv3-accel.c | 10 ++++++++- hw/arm/tegra241-cmdqv.c | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 59 insertions(+), 1 deletion(-)
hw/arm/smmuv3-accel.h +2 -0
@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {uint32_t *out_viommu_id, Error **errp);void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);void (*reset)(SMMUv3State *s); } SMMUv3AccelCmdqvOps;
hw/arm/tegra241-cmdqv.h +1 -0
@@ -37,6 +37,7 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_cmdqv;qemu_irq irq;IOMMUFDHWqueue *vcmdq[TEGRA241_CMDQV_MAX_CMDQ ]; +IOMMUFDVeventq *veventq;void *vintf_page0;MemoryRegion *mr_vintf_page0;
hw/arm/smmuv3-accel.c +9 -1
@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, goto free_bypass_hwpt; } +if (cmdqv_ops && !cmdqv_ops->alloc_veventq (s, errp)) { + goto free_veventq; + } +/* Attach a HWPT based on SMMUv3 GBPA.ABORT value */ hwpt_id =smmuv3_accel_gbpa_hwpt (s, accel);if (!host_iommu_device_iommufd_attach_hwpt (idev, hwpt_id, errp)) { - goto free_veventq; + goto free_cmdqv_veventq; }return true; +free_cmdqv_veventq: +if (cmdqv_ops && cmdqv_ops->free_veventq) { + cmdqv_ops->free_veventq (s); + } free_veventq:smmuv3_accel_free_veventq (accel); free_bypass_hwpt:
hw/arm/tegra241-cmdqv.c +47 -0
59 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
Quoted 16 lines · Shameer Kolothum
Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 2 ++ hw/arm/tegra241-cmdqv.h | 1 + hw/arm/smmuv3-accel.c | 10 ++++++++- hw/arm/tegra241-cmdqv.c | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 59 insertions(+), 1 deletion(-)hw/arm/smmuv3-accel.h +0
-0
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelCmdqvOps { · line 32
Quoted 5 lines
hw/arm/smmuv3-accel.h +2
-0@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {uint32_t *out_viommu_id, Error **errp);void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);
As I replied in v2, this should really depend on viommu and should be simply added to tegra241_cmdqv_alloc/free_viommu().
The alloc_viommu is an override of the standard viommu type, so it might be useful to have an op.
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 582
Quoted 12 lines
hw/arm/smmuv3-accel.c +4
-0@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, goto free_bypass_hwpt; } +if (cmdqv_ops && !cmdqv_ops->alloc_veventq (s, errp)) { + goto free_veventq; + } +
But, this veventq op is completely in parallel to the standard one. There is no need for the smmuv3-accel to initiate the allocation?
Thanks
Nicolin
RE: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
Quoted 29 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and freeQuoted 9 lines · Shameer Kolothum
Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 2 ++ hw/arm/tegra241-cmdqv.h | 1 + hw/arm/smmuv3-accel.c | 10 ++++++++- hw/arm/tegra241-cmdqv.c | 47+++++++++++++++++++++++++++++++++++++++++
Quoted 2 lines
4 files changed, 59 insertions(+), 1 deletion(-)
Quoted 13 lines
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelCmdqvOps { · line 32
Quoted 10 lines
hw/arm/smmuv3-accel.h +2
-0@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {uint32_t *out_viommu_id, Error **errp);void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);As I replied in v2, this should really depend on viommu and should be simply added to tegra241_cmdqv_alloc/free_viommu().
Yes, I remember that. That was actually my initial plan, which is why this was not part of the original callbacks in the ops struct.
However, I then realised that we are storing the viommu pointer in veventq,
veventq->viommu = accel->viommu;
And that struct viommu is only allocated after alloc_viommu() returns.
I didn't find an easy way to resolve that. Need to take another look.
Quoted 13 lines
The
alloc_viommuis an override of the standard viommu type, so it might be useful to have an op.hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,
Quoted 5 lines
hw/arm/smmuv3-accel.c +0
-0@@ -576,13 +576,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
Quoted 3 lines
hw/arm/smmuv3-accel.c +0
-0goto free_bypass_hwpt; }
Quoted 7 lines
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 582
Quoted 4 lines
hw/arm/smmuv3-accel.c +4
-0+if (cmdqv_ops && !cmdqv_ops->alloc_veventq (s, errp)) { + goto free_veventq; + } +But, this veventq op is completely in parallel to the standard one. There is no need for the smmuv3-accel to initiate the allocation?
If we can resolve the above dependency, then yes it can be tucked inside .
Thanks,
Shameer
Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
Quoted 46 lines · Shameer Kolothum Thodi
Quoted 29 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 March 2026 17:24
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 22/32] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and freeQuoted 9 lines · Shameer Kolothum
Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 2 ++ hw/arm/tegra241-cmdqv.h | 1 + hw/arm/smmuv3-accel.c | 10 ++++++++- hw/arm/tegra241-cmdqv.c | 47+++++++++++++++++++++++++++++++++++++++++
Quoted 2 lines
4 files changed, 59 insertions(+), 1 deletion(-)Quoted 13 lines
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelCmdqvOps { · line 32
Quoted 10 lines
hw/arm/smmuv3-accel.h +2
-0@@ -28,6 +28,8 @@ typedef struct SMMUv3AccelCmdqvOps {uint32_t *out_viommu_id, Error **errp);void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);As I replied in v2, this should really depend on viommu and should be simply added to tegra241_cmdqv_alloc/free_viommu().
Yes, I remember that. That was actually my initial plan, which is why this was not part of the original callbacks in the ops struct.
Quoted 8 lines
However, I then realised that we are storing the viommu pointer in veventq,
veventq->viommu=accel->viommu;And that struct viommu is only allocated after
alloc_viommu()returns.I didn't find an easy way to resolve that. Need to take another look.
Looks like we only needed iommufd pointer v.s. accel->viommu? This should allow us to merge alloc_veventq into alloc_viommu.
Alternatively, as we discussed on the other side, we could drop the alloc/free_viommu. Then, have a pair of viommu_init/destroy to have the veventq part (and any viommu specific routine in the future).
I feel okay with either approach. So, pick one that works the best.
Nicolin
PATCH v3 23/32 hw/arm/smmuv3-accel: Introduce common helper for veventq read
In reply to Shameer KolothumMove the vEVENTQ read and validation logic into a common helper.
The helper performs the read(), checks for overflow and short reads, validates the sequence number, and updates the sequence state.
This helper can be reused in the subsequent patch for Tegra241 CMDQV vEVENTQ support.
Error handling is slightly adjusted: instead of reporting errors directly in the read handler, the helper now returns errors via Error **. Sequence gaps are reported as warnings.
--- hw/arm/smmuv3-accel.h | 2 ++ hw/arm/smmuv3-accel-stubs.c | 6 ++++ hw/arm/smmuv3-accel.c | 67 ++++++++++++++++++++++--------------- 3 files changed, 48 insertions(+), 27 deletions(-)
hw/arm/smmuv3-accel.h +2 -0
@@ -70,6 +70,8 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev, Error **errp);void smmuv3_accel_idr_override (SMMUv3State *s);bool smmuv3_accel_alloc_veventq (SMMUv3State *s,Error **errp); +bool smmuv3_accel_event_read_validate (IOMMUFDVeventq *veventq,uint32_t type, +void *buf,size_t size,Error **errp);void smmuv3_accel_reset (SMMUv3State *s);#endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +6 -0
@@ -42,6 +42,12 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)return true; } +bool smmuv3_accel_event_read_validate (IOMMUFDVeventq *veventq,uint32_t type, +void *buf,size_t size,Error **errp) +{ +return true; +} +void smmuv3_accel_idr_override (SMMUv3State *s) { }
hw/arm/smmuv3-accel.c +40 -27
87 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v3 24/32 hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
In reply to Shameer KolothumInstall an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.
The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().
--- hw/arm/tegra241-cmdqv.c | 58 +++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 3 +++ 2 files changed, 61 insertions(+)
hw/arm/tegra241-cmdqv.c +58 -0
@@ -9,8 +9,10 @@#include "qemu/osdep.h" #include "qemu/log.h" +#include "trace.h" #include "hw/arm/smmuv3.h" +#include "hw/core/irq.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h"
@@ -487,6 +489,43 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, } } +static void tegra241_cmdqv_event_read (void *opaque) +{ +Tegra241CMDQV *cmdqv = opaque; +IOMMUFDVeventq *veventq = cmdqv->veventq; +struct { +struct iommufd_vevent_header hdr; +struct iommu_vevent_tegra241_cmdqv vevent; + } buf; +Error *local_err; + +if (!smmuv3_accel_event_read_validate (veventq, +IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV , + &buf,sizeof (buf), &local_err)) { +warn_report_err_once (local_err); +return ; + } + +if (buf.vevent.lvcmdq_err_map[0 ] || buf.vevent.lvcmdq_err_map[1 ]) { + cmdqv->vintf_cmdq_err_map[0 ] = + buf.vevent.lvcmdq_err_map[0 ] &0xffffffff ; + cmdqv->vintf_cmdq_err_map[1 ] = + (buf.vevent.lvcmdq_err_map[0 ] >>32 ) &0xffffffff ; + cmdqv->vintf_cmdq_err_map[2 ] = + buf.vevent.lvcmdq_err_map[1 ] &0xffffffff ; + cmdqv->vintf_cmdq_err_map[3 ] = + (buf.vevent.lvcmdq_err_map[1 ] >>32 ) &0xffffffff ; +for (int i =0 ; i <4 ; i++) { + cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i]; + } + cmdqv->vi_err_map[0 ] |=0x1 ; +qemu_irq_pulse (cmdqv->irq); +trace_tegra241_cmdqv_err_map ( + cmdqv->vintf_cmdq_err_map[3 ], cmdqv->vintf_cmdq_err_map[2 ], + cmdqv->vintf_cmdq_err_map[1 ], cmdqv->vintf_cmdq_err_map[0 ]); + } +} +static void tegra241_cmdqv_free_veventq (SMMUv3State *s) {SMMUv3AccelState *accel = s->s_accel;
@@ -496,6 +535,7 @@ static void tegra241_cmdqv_free_veventq(SMMUv3State *s)if (!veventq) {return ; } +qemu_set_fd_handler (veventq->veventq_fd,NULL ,NULL ,NULL );close (veventq->veventq_fd);iommufd_backend_free_id (veventq->viommu->iommufd, veventq->veventq_id);g_free (veventq);
@@ -510,6 +550,7 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)IOMMUFDVeventq *veventq;uint32_t veventq_id;uint32_t veventq_fd; +int flags;if (cmdqv->veventq) {return true;
@@ -523,13 +564,30 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)return false; } + flags =fcntl (veventq_fd,F_GETFL ); +if (flags <0 ) { +error_setg (errp,"Failed to get flags for vEVENTQ fd" ); + goto free_veventq; + } +if (fcntl (veventq_fd,F_SETFL ,O_NONBLOCK | flags) <0 ) { +error_setg (errp,"Failed to set O_NONBLOCK on vEVENTQ fd" ); + goto free_veventq; + } + veventq =g_new (IOMMUFDVeventq,1 ); veventq->veventq_id = veventq_id; veventq->veventq_fd = veventq_fd; veventq->viommu = accel->viommu; cmdqv->veventq = veventq; +/* Set up event handler for veventq fd */ +qemu_set_fd_handler (veventq_fd, tegra241_cmdqv_event_read,NULL , cmdqv);return true; + +free_veventq: +close (veventq_fd); +iommufd_backend_free_id (viommu->iommufd, veventq_id); +return false; }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
hw/arm/trace-events +3 -0
@@ -72,6 +72,9 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x" +# tegra241-cmdqv +tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X" + # strongarm.c strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d" strongarm_ssp_read_underrun(void) "SSP rx underrun"
Signature · 1 line
2.43.0
PATCH v3 25/32 hw/arm/tegra241-cmdqv: Add reset handler
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.
CMDQV gets initialized early during guest boot, hence the handler verifies that at least one cold-plugged device is attached to the associated vIOMMU before proceeding. This is required to retrieve host CMDQV info and to validate it against the QEMU implementation support.
--- hw/arm/tegra241-cmdqv.h | 2 ++ hw/arm/smmuv3-accel.c | 12 +++++++++- hw/arm/tegra241-cmdqv.c | 50 +++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 1 + 4 files changed, 64 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.h +2 -0
@@ -72,6 +72,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) FIELD(CONFIG, CONS_DRAM_EN, 20, 1) +#define V_CONFIG_RESET 0x00020403 + REG32(PARAM, 0x4) FIELD(PARAM, CMDQV_VER, 0, 4)FIELD (PARAM ,CMDQV_NUM_CMDQ_LOG2 ,4 ,4 )
hw/arm/smmuv3-accel.c +11 -1
@@ -774,7 +774,11 @@ static void smmuv3_accel_unset_iommu_device(PCIBus *bus, void *opaque,QLIST_REMOVE (accel_dev, next);trace_smmuv3_accel_unset_iommu_device (devfn, idev->devid); -if (QLIST_EMPTY (&accel->device_list)) { +/* +* Keep the vIOMMU alive when CMDQV is present, as the vIOMMU to host +* SMMUv3 association cannot be changed via device hot-plug. +*/ +if (QLIST_EMPTY (&accel->device_list) && !accel->cmdqv) {smmuv3_accel_free_viommu (accel); } }
@@ -966,6 +970,12 @@ void smmuv3_accel_reset(SMMUv3State *s)/* Attach a HWPT based on GBPA reset value */ smmuv3_accel_attach_gbpa_hwpt (s,NULL ); +if (s->cmdqv ==ON_OFF_AUTO_ON &&QLIST_EMPTY (&accel->device_list)) { +error_report ("cmdqv=on: requires at least one cold-plugged " +"vfio-pci device" ); +exit (1 ); + } +if (accel->cmdqv_ops && accel->cmdqv_ops->reset) { accel->cmdqv_ops->reset (s); }
hw/arm/tegra241-cmdqv.c +50 -0
64 lines not on this page · raw message
hw/arm/trace-events +1 -0
@@ -74,6 +74,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS # tegra241-cmdqv tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X" +tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X" # strongarm.c strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
PATCH v3 26/32 hw/arm/tegra241-cmdqv: Limit queue size based on backend page size
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
CMDQV HW reads guest queue memory in its host physical address setup via IOMUUFD. This requires the guest queue memory isn't only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQV size that the host can safely back with physically contiguous memory. Allowing a CMDQV larger than the host page size could cause the hardware to DMA across page boundaries leading to faults.
Limit IDR1.CMDQS so the guest cannot configure a CMDQV that exceeds the host’s contiguous backing.
--- hw/arm/tegra241-cmdqv.c | 43 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
hw/arm/tegra241-cmdqv.c +43 -0
@@ -11,10 +11,14 @@#include "qemu/error-report.h" #include "qemu/log.h" #include "trace.h" +#include <math.h> #include "hw/arm/smmuv3.h" #include "hw/core/irq.h" #include "smmuv3-accel.h" +#include "smmuv3-internal.h" +#include "system/ramblock.h" +#include "system/ramlist.h" #include "tegra241-cmdqv.h" static inline uint32_t *tegra241_cmdqv_vintf_ptr(Tegra241CMDQV *cmdqv,
@@ -618,9 +622,38 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,return true; } +static size_t tegra241_cmdqv_min_ram_pagesize (void ) +{ +RAMBlock *rb; +size_t pg, min_pg =SIZE_MAX ; + +RAMBLOCK_FOREACH (rb) { +MemoryRegion *mr = rb->mr; + +/* Only consider real RAM regions */ +if (!mr || !memory_region_is_ram (mr)) { +continue ; + } + +/* Skip RAM regions that are not backed by a memory-backend */ +if (!object_dynamic_cast (mr->owner,TYPE_MEMORY_BACKEND )) { +continue ; + } + + pg =qemu_ram_pagesize (rb); +if (pg && pg < min_pg) { + min_pg = pg; + } + } + +return (min_pg ==SIZE_MAX ) ?qemu_real_host_page_size () : min_pg; +} +static void tegra241_cmdqv_init_regs (SMMUv3State *s,Tegra241CMDQV *cmdqv) {int i; +size_t pgsize; +uint32_t val; cmdqv->config =V_CONFIG_RESET ; cmdqv->param =
@@ -652,6 +685,16 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv) cmdqv->vcmdq_base[i] =0 ; cmdqv->vcmdq_cons_indx_base[i] =0 ; } + +/* +* CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the +* queue fits entirely within the smallest backend page size, ensuring +* the command queue is physically contiguous in host memory. +*/ + pgsize =tegra241_cmdqv_min_ram_pagesize (); + val =FIELD_EX32 (s->idr[1 ],IDR1 ,CMDQS ); + s->idr[1 ] =FIELD_DP32 (s->idr[1 ],IDR1 ,CMDQS ,MIN (log2 (pgsize) -4 , val)); +return ; }
Signature · 1 line
2.43.0
PATCH v3 27/32 tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier change
In reply to Shameer KolothumThe IORT SMMUv3 node Identifier field will be derived from a new per-device "identifier" property instead of relying on enumeration order.
Add the affected IORT blobs to allowed-diff list for bios-table tests.
--- tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++ 1 file changed, 4 insertions(+)
tests/qtest/bios-tables-test-allowed-diff.h +4 -0
@@ -1 +1,5 @@/* List of comma-separated changed AML files to ignore */ +"tests/data/acpi/aarch64/virt/IORT.its_off" , +"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m" , +"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy" , +"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
Signature · 1 line
2.43.0
Re: [PATCH v3 27/32] tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier change
Quoted 7 lines · Shameer Kolothum
The IORT SMMUv3 node Identifier field will be derived from a new per-device "identifier" property instead of relying on enumeration order.
Add the affected IORT blobs to allowed-diff list for bios-table tests.
Signed-off-by Shameer Kolothum <[email protected]>
Eric
Quoted 14 lines
--- tests/qtest/bios-tables-test-allowed-diff.h | 4 ++++ 1 file changed, 4 insertions(+)tests/qtest/bios-tables-test-allowed-diff.h +4
-0@@ -1 +1,5 @@/* List of comma-separated changed AML files to ignore */ +"tests/data/acpi/aarch64/virt/IORT.its_off" , +"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m" , +"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy" , +"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
PATCH v3 28/32 hw/arm/smmuv3: Add per-device identifier property
In reply to Shameer KolothumAdd an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.
No functional change intended.
--- include/hw/arm/smmuv3.h | 1 + hw/arm/smmuv3.c | 2 ++ hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ 4 files changed, 9 insertions(+), 1 deletion(-)
include/hw/arm/smmuv3.h +1 -0
@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;
hw/arm/smmuv3.c +2 -0
@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +/* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */ +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiIortSMMUv3Dev { int irq; hwaddr base; + uint8_t id; GArray *rc_smmu_idmaps; /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */ size_t offset;
@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms) &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );
@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */ /* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */
hw/arm/virt.c +3 -0
@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev, } } +static uint8_t smmuv3_dev_id;static void virt_machine_device_pre_plug_cb (HotplugHandler *hotplug_dev,DeviceState *dev,Error **errp) {
@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, OBJECT(vms->sysmem), NULL);object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , smmuv3_dev_id++, +NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;
Signature · 1 line
2.43.0
Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property
Shameer,
Quoted 9 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.
No functional change intended.
besides changes the IORT table
Quoted 69 lines
Reviewed-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- include/hw/arm/smmuv3.h | 1 + hw/arm/smmuv3.c | 2 ++ hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ 4 files changed, 9 insertions(+), 1 deletion(-)include/hw/arm/smmuv3.h +1
-0@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;hw/arm/smmuv3.c +2
-0@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +/* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */ +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),hw/arm/virt-acpi-build.c +3
-1@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiIortSMMUv3Dev { int irq; hwaddr base; + uint8_t id; GArray *rc_smmu_idmaps; /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */ size_t offset;@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms) &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */ /* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */ hw/arm/virt.c +0
-0
hw/arm/virt.c static void virt_memory_plug(HotplugHandler *hotplug_dev, · line 3141
Quoted 4 lines
hw/arm/virt.c +1
-0@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev, } } +static uint8_t smmuv3_dev_id;
nit: I would rather put that at the top of the file
hw/arm/virt.c static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 3204
Quoted 12 lines
hw/arm/virt.c +2
-0static void virt_machine_device_pre_plug_cb (HotplugHandler *hotplug_dev,DeviceState *dev,Error **errp) {@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, OBJECT(vms->sysmem), NULL);object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , smmuv3_dev_id++, +NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;
Eric
Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property
In reply to Shameer KolothumQuoted 65 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.
No functional change intended.
Reviewed-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- include/hw/arm/smmuv3.h | 1 + hw/arm/smmuv3.c | 2 ++ hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ 4 files changed, 9 insertions(+), 1 deletion(-)include/hw/arm/smmuv3.h +1
-0@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;hw/arm/smmuv3.c +2
-0@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +/* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */ +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),hw/arm/virt-acpi-build.c +2
-0@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiIortSMMUv3Dev { int irq; hwaddr base; + uint8_t id; GArray *rc_smmu_idmaps; /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */ size_t offset;@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray *sdev_blob, VirtMachineState *vms) &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );
hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 635
Quoted 5 lines
hw/arm/virt-acpi-build.c +1
-1@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */
Hum actually I see that previously the id scope was shared by all nodes, including ITS, SMMU, RC.
So now can have its and smmu node having both id=0. And this is why you have a change in the IORT table for its=off test.
Is it legal?
IORT spec says:
Unique identifier for this node that can be used to locate it in the parent table. This identifier enables other ACPI tables and DSDT objects to locate this node. This field serves in the same capacity as the _UID object associated with ACPI device objects. Note IMPLEMENTATION NOTE: In the simplest scheme, the Identifier might be set to the node’s index in the array of IORT nodes in the parent IORT table. Other schemes are also possible and permitted.
So I don't think this is valid
Eric
Quoted 24 lines
hw/arm/virt-acpi-build.c +0
-0/* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */ hw/arm/virt.c +3
-0@@ -3138,6 +3138,7 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev, } } +static uint8_t smmuv3_dev_id;static void virt_machine_device_pre_plug_cb (HotplugHandler *hotplug_dev,DeviceState *dev,Error **errp) {@@ -3196,6 +3197,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, OBJECT(vms->sysmem), NULL);object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , smmuv3_dev_id++, +NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;
RE: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier property
Quoted 94 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 09 March 2026 18:23
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 28/32] hw/arm/smmuv3: Add per-device identifier propertyExternal email: Use caution opening links or attachments
Quoted 36 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on device enumeration order and provides a stable per-device identifier. A subsequent patch will use the same identifier when generating the DSDT description for Tegra241 CMDQV, ensuring that the IORT and DSDT entries refer to the same SMMUv3 instance.
No functional change intended.
Reviewed-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- include/hw/arm/smmuv3.h | 1 + hw/arm/smmuv3.c | 2 ++ hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ 4 files changed, 9 insertions(+), 1 deletion(-)include/hw/arm/smmuv3.h +1
-0@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;hw/arm/smmuv3.c +0
-0hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2112
Quoted 4 lines
hw/arm/smmuv3.c +1
-0@@ -2109,6 +2109,8 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING("stage", SMMUv3State, stage), +/* Identifier used for ACPI IORT SMMUv3 (and DSDT for CMDQV) generation */
hw/arm/virt-acpi-build.c static int iort_idmap_compare(gconstpointer a,
Quoted 9 lines
hw/arm/smmuv3.c +1
-0+DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),hw/arm/virt-acpi-build.c +0
-0@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a,gconstpointer b)
hw/arm/virt-acpi-build.c static int populate_smmuv3_dev(GArray
Quoted 8 lines
hw/arm/virt-acpi-build.c +1
-0typedef struct AcpiIortSMMUv3Dev { int irq; hwaddr base; + uint8_t id; GArray *rc_smmu_idmaps; /* Offset of the SMMUv3 IORT Node relative to the start of the IORT */ size_t offset;@@ -404,6 +405,7 @@ static int populate_smmuv3_dev(GArray
*sdev_blob, VirtMachineState *vms)hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker,
Quoted 8 lines
hw/arm/virt-acpi-build.c +1
-0&error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );@@ -630,7 +632,7 @@ build_iort(GArray *table_data, BIOSLinker *linker,VirtMachineState *vms)
hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 635
Quoted 5 lines
hw/arm/virt-acpi-build.c +1
-1(ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */
Quoted 4 lines
Hum actually I see that previously the id scope was shared by all nodes, including ITS, SMMU, RC.
So now can have its and smmu node having both id=0. And this is why you have a change in the IORT table for its=off test.
Hmm... I had that doubt as well. But I read it as a unique identifier for each type of IORT node.
Quoted 12 lines
Is it legal?
IORT spec says:
Unique identifier for this node that can be used to locate it in the parent table. This identifier enables other ACPI tables and DSDT objects to locate this node. This field serves in the same capacity as the
_UIDobject associated with ACPI device objects. Note IMPLEMENTATION NOTE: In the simplest scheme, the Identifier might be set to the node’s index in the array of IORT nodes in the parent IORT table. Other schemes are also possible and permitted.
Quoted 1 line
So I don't think this is valid
I think we need another way to sort out the identifier then.
Thanks,
Shameer
PATCH v3 29/32 tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier change
In reply to Shameer KolothumUpdate the reference IORT blobs after identifier change for SMMUv3 nodes. This affects the aarch64 'virt' IORT tests.
IORT diff is the identifier change in SMMUv3 nodes:
/*
* Intel ACPI Component Architecture
* AML/ASL+ Disassembler version 20230628 (64-bit version)
* Copyright (c) 2000 - 2023 Intel Corporation
*
- * Disassembly of tests/data/acpi/aarch64/virt/IORT.its_off, Wed Feb 25 16:36:44 2026
+ * Disassembly of /tmp/aml-KVO1K3, Wed Feb 25 16:36:44 2026
*
* ACPI Data Table [IORT]
*
* Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue (in hex) */
[000h 0000 004h] Signature : "IORT" [IO Remapping Table]
[004h 0004 004h] Table Length : 000000AC
[008h 0008 001h] Revision : 05 -[009h 0009 001h] Checksum : 95 +[009h 0009 001h] Checksum : 96
[00Ah 0010 006h] Oem ID : "BOCHS "
[010h 0016 008h] Oem Table ID : "BXPC "
[018h 0024 004h] Oem Revision : 00000001
[01Ch 0028 004h] Asl Compiler ID : "BXPC"
[020h 0032 004h] Asl Compiler Revision : 00000001
[024h 0036 004h] Node Count : 00000002
[028h 0040 004h] Node Offset : 00000030
[02Ch 0044 004h] Reserved : 00000000
...
[074h 0116 001h] Type : 02
[075h 0117 002h] Length : 0038
[077h 0119 001h] Revision : 03
-[078h 0120 004h] Identifier : 00000001
+[078h 0120 004h] Identifier : 00000000
[07Ch 0124 004h] Mapping Count : 00000001
[080h 0128 004h] Mapping Offset : 00000024
[084h 0132 008h] Memory Properties : [IORT Memory Access Properties]
[084h 0132 004h] Cache Coherency : 00000001
[088h 0136 001h] Hints (decoded below) : 00
Transient : 0
Write Allocate : 0
Read Allocate : 0
Override : 0
[089h 0137 002h] Reserved : 0000
[08Bh 0139 001h] Memory Flags (decoded below) : 03
Coherency : 1
Device Attribute : 1
[08Ch 0140 004h] ATS Attribute : 00000000
[090h 0144 004h] PCI Segment Number : 00000000
[094h 0148 001h] Memory Size Limit : 40
[095h 0149 002h] PASID Capabilities : 0000
[097h 0151 001h] Reserved : 00
[098h 0152 004h] Input base : 00000000
[09Ch 0156 004h] ID Count : 000000FF
[0wA0h 0160 004h] Output Base : 00000000
[0A4h 0164 004h] Output Reference : 00000030
[0A8h 0168 004h] Flags (decoded below) : 00000000
Single Mapping : 0
...
--- tests/qtest/bios-tables-test-allowed-diff.h | 4 ---- tests/data/acpi/aarch64/virt/IORT.its_off | Bin 172 -> 172 bytes tests/data/acpi/aarch64/virt/IORT.msi_gicv2m | Bin 172 -> 172 bytes tests/data/acpi/aarch64/virt/IORT.smmuv3-dev | Bin 260 -> 260 bytes tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes 5 files changed, 4 deletions(-)
tests/qtest/bios-tables-test-allowed-diff.h +0 -4
@@ -1,5 +1 @@/* List of comma-separated changed AML files to ignore */ -"tests/data/acpi/aarch64/virt/IORT.its_off" , -"tests/data/acpi/aarch64/virt/IORT.msi_gicv2m" , -"tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy" , -"tests/data/acpi/aarch64/virt/IORT.smmuv3-dev",
tests/data/acpi/aarch64/virt/IORT.its_off binary +0 -0
tests/data/acpi/aarch64/virt/IORT.msi_gicv2m binary +0 -0
tests/data/acpi/aarch64/virt/IORT.smmuv3-dev binary +0 -0
tests/data/acpi/aarch64/virt/IORT.smmuv3-legacy binary +0 -0
Signature · 1 line
2.43.0
PATCH v3 30/32 hw/arm/smmuv3-accel: Introduce helper to query CMDQV type
In reply to Shameer KolothumIntroduce a SMMUv3AccelCmdqvType enum and a helper to query the CMDQV implementation type associated with an accelerated SMMUv3 instance.
A subsequent patch will use this helper when generating the Tegra241 CMDQV DSDT.
--- hw/arm/smmuv3-accel.h | 7 +++++++ hw/arm/smmuv3-accel-stubs.c | 5 +++++ hw/arm/smmuv3-accel.c | 12 ++++++++++++ hw/arm/tegra241-cmdqv.c | 6 ++++++ 4 files changed, 30 insertions(+)
hw/arm/smmuv3-accel.h +7 -0
@@ -15,6 +15,11 @@#include <linux/iommufd.h> #endif +typedef enum SMMUv3AccelCmdqvType { +SMMUV3_CMDQV_NONE =0 , +SMMUV3_CMDQV_TEGRA241 , +}SMMUv3AccelCmdqvType ; +/* * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to * support multiple VCMDQs with virtualization capabilities.
@@ -30,6 +35,7 @@ typedef struct SMMUv3AccelCmdqvOps {void (*free_viommu)(SMMUv3State *s);bool (*alloc_veventq)(SMMUv3State *s,Error **errp);void (*free_veventq)(SMMUv3State *s); +SMMUv3AccelCmdqvType (*get_type)(void);void (*reset)(SMMUv3State *s); } SMMUv3AccelCmdqvOps;
@@ -73,5 +79,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);bool smmuv3_accel_event_read_validate (IOMMUFDVeventq *veventq,uint32_t type,void *buf,size_t size,Error **errp);void smmuv3_accel_reset (SMMUv3State *s); +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type (Object *obj);#endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3-accel-stubs.c +5 -0
@@ -55,3 +55,8 @@ void smmuv3_accel_idr_override(SMMUv3State *s)void smmuv3_accel_reset (SMMUv3State *s) { } + +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type (Object *obj) +{ +return SMMUV3_CMDQV_NONE ; +}
hw/arm/smmuv3-accel.c +12 -0
@@ -998,6 +998,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)address_space_init (shared_as_sysmem, &root,"smmuv3-accel-as-sysmem" ); } +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type (Object *obj) +{ +SMMUv3State *s =ARM_SMMUV3 (obj); +SMMUv3AccelState *accel = s->s_accel; + +if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) { +return SMMUV3_CMDQV_NONE ; + } + +return accel->cmdqv_ops->get_type (); +} +bool smmuv3_accel_init (SMMUv3State *s,Error **errp) {SMMUState *bs =ARM_SMMU (s);
hw/arm/tegra241-cmdqv.c +6 -0
@@ -736,6 +736,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)return true; } +static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type (void ) +{ +return SMMUV3_CMDQV_TEGRA241 ; +}; +static bool tegra241_cmdqv_probe (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,Error **errp) {
@@ -778,6 +783,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = { .free_viommu = tegra241_cmdqv_free_viommu, .alloc_veventq = tegra241_cmdqv_alloc_veventq, .free_veventq = tegra241_cmdqv_free_veventq, + .get_type = tegra241_cmdqv_get_type, .reset = tegra241_cmdqv_reset, };
Signature · 1 line
2.43.0
Re: [PATCH v3 30/32] hw/arm/smmuv3-accel: Introduce helper to query CMDQV type
Quoted 8 lines · Shameer Kolothum
Introduce a SMMUv3AccelCmdqvType enum and a helper to query the CMDQV implementation type associated with an accelerated SMMUv3 instance.
A subsequent patch will use this helper when generating the Tegra241 CMDQV DSDT.
Signed-off-by Shameer Kolothum <[email protected]>
Eric
Quoted 98 lines
--- hw/arm/smmuv3-accel.h | 7 +++++++ hw/arm/smmuv3-accel-stubs.c | 5 +++++ hw/arm/smmuv3-accel.c | 12 ++++++++++++ hw/arm/tegra241-cmdqv.c | 6 ++++++ 4 files changed, 30 insertions(+)hw/arm/smmuv3-accel.h +7
-0@@ -15,6 +15,11 @@#include <linux/iommufd.h> #endif +typedef enum SMMUv3AccelCmdqvType { +SMMUV3_CMDQV_NONE =0 , +SMMUV3_CMDQV_TEGRA241 , +}SMMUv3AccelCmdqvType ; +/* * CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to * support multiple VCMDQs with virtualization capabilities. @@ -30,6 +35,7 @@ typedef struct SMMUv3AccelCmdqvOps {void (*free_viommu)(SMMUv3State *s);bool (*alloc_veventq)(SMMUv3State *s,Error **errp);void (*free_veventq)(SMMUv3State *s); +SMMUv3AccelCmdqvType (*get_type)(void);void (*reset)(SMMUv3State *s); } SMMUv3AccelCmdqvOps;@@ -73,5 +79,6 @@ bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp);bool smmuv3_accel_event_read_validate (IOMMUFDVeventq *veventq,uint32_t type,void *buf,size_t size,Error **errp);void smmuv3_accel_reset (SMMUv3State *s); +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type (Object *obj);#endif /* HW_ARM_SMMUV3_ACCEL_H */ hw/arm/smmuv3-accel-stubs.c +5
-0@@ -55,3 +55,8 @@ void smmuv3_accel_idr_override(SMMUv3State *s)void smmuv3_accel_reset (SMMUv3State *s) { } + +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type (Object *obj) +{ +return SMMUV3_CMDQV_NONE ; +}hw/arm/smmuv3-accel.c +12
-0@@ -998,6 +998,18 @@ static void smmuv3_accel_as_init(SMMUv3State *s)address_space_init (shared_as_sysmem, &root,"smmuv3-accel-as-sysmem" ); } +SMMUv3AccelCmdqvType smmuv3_accel_cmdqv_type (Object *obj) +{ +SMMUv3State *s =ARM_SMMUV3 (obj); +SMMUv3AccelState *accel = s->s_accel; + +if (!accel || !accel->cmdqv_ops || !accel->cmdqv_ops->get_type) { +return SMMUV3_CMDQV_NONE ; + } + +return accel->cmdqv_ops->get_type (); +} +bool smmuv3_accel_init (SMMUv3State *s,Error **errp) {SMMUState *bs =ARM_SMMU (s);hw/arm/tegra241-cmdqv.c +6
-0@@ -736,6 +736,11 @@ static bool tegra241_cmdqv_init(SMMUv3State *s, Error **errp)return true; } +static SMMUv3AccelCmdqvType tegra241_cmdqv_get_type (void ) +{ +return SMMUV3_CMDQV_TEGRA241 ; +}; +static bool tegra241_cmdqv_probe (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,Error **errp) {@@ -778,6 +783,7 @@ static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops = { .free_viommu = tegra241_cmdqv_free_viommu, .alloc_veventq = tegra241_cmdqv_alloc_veventq, .free_veventq = tegra241_cmdqv_free_veventq, + .get_type = tegra241_cmdqv_get_type, .reset = tegra241_cmdqv_reset, };
PATCH v3 31/32 hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.
The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.
Because virt-acpi-build.c now includes CONFIG_DEVICES via the Tegra241 CMDQV header, the Meson file entry is updated to build it as part of arm_ss instead of arm_common_ss
--- hw/arm/virt-acpi-build.c | 52 ++++++++++++++++++++++++++++++++++++++++ hw/arm/trace-events | 1 + 2 files changed, 53 insertions(+)
hw/arm/virt-acpi-build.c +52 -0
@@ -65,6 +65,9 @@#include "target/arm/cpu.h" #include "target/arm/multiprocessing.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" +#define ARM_SPI_BASE 32#define ACPI_BUILD_TABLE_SIZE 0x20000
@@ -1113,6 +1116,51 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,build_fadt (table_data, linker, &fadt, vms->oem_id, vms->oem_table_id); } +static void acpi_dsdt_add_tegra241_cmdqv (Aml *scope,VirtMachineState *vms) +{ +for (int i =0 ; i < vms->smmuv3_devices->len; i++) { +Object *obj =OBJECT (g_ptr_array_index (vms->smmuv3_devices, i)); +PlatformBusDevice *pbus; +Aml *dev, *crs, *addr; +SysBusDevice *sbdev; +hwaddr base; +uint32_t id; +int irq; + +if (smmuv3_accel_cmdqv_type (obj) !=SMMUV3_CMDQV_TEGRA241 ) { +continue ; + } + id =object_property_get_uint (obj,"identifier" , &error_abort); + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + base += vms->memmap[VIRT_PLATFORM_BUS ].base; + irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + irq +=ARM_SPI_BASE ; + + dev =aml_device ("CV%.02u" , id); +aml_append (dev,aml_name_decl ("_HID" ,aml_string ("NVDA200C" ))); +aml_append (dev,aml_name_decl ("_UID" ,aml_int (id))); +aml_append (dev,aml_name_decl ("_CCA" ,aml_int (1 ))); + + crs =aml_resource_template (); + addr =aml_qword_memory (AML_POS_DECODE ,AML_MIN_FIXED ,AML_MAX_FIXED , +AML_CACHEABLE ,AML_READ_WRITE ,0x0 , base, + base +TEGRA241_CMDQV_IO_LEN -0x1 ,0x0 , +TEGRA241_CMDQV_IO_LEN ); +aml_append (crs, addr); +aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_EDGE , +AML_ACTIVE_HIGH ,AML_EXCLUSIVE , + (uint32_t *)&irq,1 )); +aml_append (dev,aml_name_decl ("_CRS" , crs)); + +aml_append (scope, dev); + +trace_virt_acpi_dsdt_tegra241_cmdqv (id, base, irq); + } +} +/* DSDT */ static void build_dsdt (GArray *table_data,BIOSLinker *linker,VirtMachineState *vms)
@@ -1177,6 +1225,10 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_dsdt_add_tpm (scope, vms);#endif +if (!vms->legacy_smmuv3_present) { +acpi_dsdt_add_tegra241_cmdqv (scope, vms); + } +aml_append (dsdt, scope); pci0_scope =aml_scope ("\\_SB.PCI0" );
hw/arm/trace-events +1 -0
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s" # virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." +virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d" # smmu-common.c smmu_add_mr(const char *name) "%s"
Signature · 1 line
2.43.0
PATCH v3 32/32 hw/arm/smmuv3: Add cmdqv property for SMMUv3 device
In reply to Shameer KolothumIntroduce a “cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
--- hw/arm/smmuv3.c | 7 +++++++ 1 file changed, 7 insertions(+)
hw/arm/smmuv3.c +7 -0
@@ -1967,6 +1967,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)error_setg (errp,"ssidsize can only be set if accel=on" );return false; } +if (s->cmdqv ==ON_OFF_AUTO_ON ) { +error_setg (errp,"cmdqv can only be enabled if accel=on" ); +return false; + }return true; }
@@ -2119,6 +2123,7 @@ static const Property smmuv3_properties[] = {DEFINE_PROP_BOOL ("ats" , SMMUv3State, ats, false),DEFINE_PROP_UINT8 ("oas" , SMMUv3State, oas,44 ),DEFINE_PROP_UINT8 ("ssidsize" , SMMUv3State, ssidsize,0 ), + DEFINE_PROP_ON_OFF_AUTO("cmdqv", SMMUv3State, cmdqv, ON_OFF_AUTO_AUTO), };static void smmuv3_instance_init (Object *obj)
@@ -2158,6 +2163,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)"Valid range is 0-20, where 0 disables SubstreamID support. " "Defaults to 0. A value greater than 0 is required to enable " "PASID support."); +object_class_property_set_description (klass,"cmdqv" , +"Enable/disable CMDQ-Virtualisation support (for accel=on)" ); } static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
Hi Shameer,
Quoted 25 lines · Shameer Kolothum
Hi,
Changes from v2:
https://lore.kernel.org/qemu-devel/[email protected]/- Addressed feedback on v2 and picked up R-by tags. Thanks!
- Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto"cmdqv". - Introduced a probe() callback to detect and initialize CMDQV support during device attach (patch #7). - Since CMDQV initialization is now deferred to device attach time, added a helper to link any unmapped MMIO/IRQ resources (patch #12). - Moved VINTF enable handling to the CONFIG ENABLE bit write path instead of any read/write access path (patch #17). Hopefully this won't break any corner cases. - Added checks for VINTF and CMDQV ENABLE bits before allocating vCMDQ HW queues, and free any allocated resources if disabled (patch #19). - Introduced a common helper for vEVENTQ read and validation (patch #23). - Updated bios-tables test IORT blobs for SMMUv3 device identifier changes (patch #29). - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0], using tegra241-cmdqv-stubs.c (patch #6).
Quoted 2 lines
Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcmdq-v3
with this version I still get those traces:
iommufd_backend_map_dma iommufd=28 ioas=2 iova=0xc0b0000 size=0x10000 addr=0xffff0bd30000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?
qemu-system-aarch64: warning: vfio_container_dma_map(0xaaaea11e2870, 0xc0b0000, 0x10000, 0xffff0bd30000) = -14 (Bad address)
Eric
Quoted 107 lines
I have sanity tested this on an NVIDIA GRACE platform and will continue with additional tests.
Feedback and testing are very welcome.
Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/---
Background(from RFCv1):
https://lore.kernel.org/qemu-devel/[email protected]/Thanks to Nicolin for the initial patches and testing on which this is based.
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
VCMDQ support is integrated via the
IOMMU_HW_QUEUE_ALLOCmechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].---
Nicolin Chen (15):
backends/iommufd: Updateiommufd_backend_get_device_infobackends/iommufd: Updateiommufd_backend_alloc_viommuto allow user
ptr
backends/iommufd: Introduceiommufd_backend_alloc_hw_queuebackends/iommufd: Introduceiommufd_backend_viommu_mmaphw/arm/tegra241-cmdqv: Implement CMDQV init hw/arm/tegra241-cmdqv: Implement CMDQV vIOMMU alloc/free hw/arm/tegra241-cmdqv: Emulate global CMDQV registers hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: Map VINTF page0 into guest MMIO space hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTShameer Kolothum (17):
hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle hw/arm/virt: Store SMMUv3 device objects in VirtMachineState. hw/arm/virt-acpi-build: Use stored SMMUv3 devices for IORT build hw/arm/tegra241-cmdqv: Probe host Tegra241 CMDQV support hw/arm/virt: Link SMMUv3 CMDQV resources to platform bus system/physmem: Addaddress_space_is_ram()helper hw/arm/tegra241-cmdqv: Use mmap'ed VINTF page0 as VCMDQ backing hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free hw/arm/smmuv3-accel: Introduce common helper for veventq read hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors tests/qtest/bios-tables-test: Prepare for IORT SMMUv3 node identifier
change
hw/arm/smmuv3: Add per-device identifier property tests/qtest/bios-tables-test: Update IORT blobs for SMMUv3 identifier
change
hw/arm/smmuv3-accel: Introduce helper to query CMDQV type hw/arm/smmuv3: Add cmdqv property for SMMUv3 devicehw/arm/smmuv3-accel.h | 29 + hw/arm/tegra241-cmdqv.h | 342 ++++++++ include/hw/arm/smmuv3.h | 3 + include/hw/arm/virt.h | 1 + include/system/iommufd.h | 16 + include/system/memory.h | 10 + backends/iommufd.c | 64 ++ hw/arm/smmuv3-accel-stubs.c | 11 + hw/arm/smmuv3-accel.c | 196 ++++- hw/arm/smmuv3.c | 9 + hw/arm/tegra241-cmdqv-stubs.c | 18 + hw/arm/tegra241-cmdqv.c | 793 ++++++++++++++++++ hw/arm/virt-acpi-build.c | 126 ++- hw/arm/virt.c | 27 + hw/vfio/iommufd.c | 4 +- system/physmem.c | 11 + backends/trace-events | 4 +- hw/arm/Kconfig | 5 + hw/arm/meson.build | 2 + hw/arm/trace-events | 5 + tests/data/acpi/aarch64/virt/IORT.its_off | Bin 172 -> 172 bytes tests/data/acpi/aarch64/virt/IORT.msi_gicv2m | Bin 172 -> 172 bytes tests/data/acpi/aarch64/virt/IORT.smmuv3-dev | Bin 260 -> 260 bytes .../data/acpi/aarch64/virt/IORT.smmuv3-legacy | Bin 192 -> 192 bytes 24 files changed, 1596 insertions(+), 80 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.h create mode 100644 hw/arm/tegra241-cmdqv-stubs.c create mode 100644 hw/arm/tegra241-cmdqv.c
Quoted 57 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 March 2026 18:25
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3External email: Use caution opening links or attachments
Hi Shameer,
Quoted 5 lines · Shameer Kolothum
Hi,
Changes from v2:
skolothumtho
Quoted 22 lines
@nvidia.com/
- Addressed feedback on v2 and picked up R-by tags. Thanks!
- Renamed the property from "tegra241-cmdqv" to a generic OnOffAuto"cmdqv". - Introduced a probe() callback to detect and initialize CMDQV support during device attach (patch #7). - Since CMDQV initialization is now deferred to device attach time, added a helper to link any unmapped MMIO/IRQ resources (patch #12). - Moved VINTF enable handling to the CONFIG ENABLE bit write path instead of any read/write access path (patch #17). Hopefully this won't break any corner cases. - Added checks for VINTF and CMDQV ENABLE bits before allocating vCMDQ HW queues, and free any allocated resources if disabled (patch #19). - Introduced a common helper for vEVENTQ read and validation (patch #23). - Updated bios-tables test IORT blobs for SMMUv3 device identifier changes (patch #29). - Rebasing on top of Philippe's CONFIG_DEVICES cleanup series [0], using tegra241-cmdqv-stubs.c (patch #6).Quoted 2 lines
Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v8-vcm
Quoted 1 line
dq-v3
with this version I still get those traces:
Quoted 7 lines
iommufd_backend_map_dmaiommufd=28 ioas=2 iova=0xc0b0000 size=0x10000
addr=0xffff0bd30000 readonly=0 (-1)
qemu-system-aarch64: warning:IOMMU_IOAS_MAPfailed: Bad address, PCI BAR?
qemu-system-aarch64: warning:vfio_container_dma_map(0xaaaea11e2870, 0xc0b0000, 0x10000, 0xffff0bd30000)= -14 (Bad address)
Yes. Please see my reply to Nicolin on v2:
https://lore.kernel.org/qemu-devel/CH3PR12MB7548C2DCA4D321D7FDE23754AB6CA@CH3PR12MB7548.namprd12.prod.outlook.com/
I do not have a way to suppress that warning currently. It should be safe to ignore, similar to the behaviour before the dma-buf support series for vfio-pci PCI BAR mappings.
Thanks,
Shameer
Quoted 3 lines · Shameer Kolothum Thodi
I do not have a way to suppress that warning currently. It should be safe to ignore, similar to the behaviour before the dma-buf support series for vfio-pci PCI BAR mappings.
qemu should have the concept of memory regions that are not available for DMA peer to peer operation - ie are not mapped to the iommu.
Now that the kernel strictly enforces this it is essential qemu itself keep track if it thinks p2p should work or not.
Jason
Quoted 21 lines
-----Original Message-----
From: Jason Gunthorpe <[email protected]>
Sent: 11 March 2026 20:00
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; qemu- [email protected]; [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>; [email protected]
Subject: Re: [PATCH v3 00/32] hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3Quoted 3 lines · Shameer Kolothum Thodi
I do not have a way to suppress that warning currently. It should be safe to ignore, similar to the behaviour before the dma-buf support series for vfio-pci PCI BAR mappings.
Quoted 5 lines
qemu should have the concept of memory regions that are not available for DMA peer to peer operation - ie are not mapped to the iommu.
Now that the kernel strictly enforces this it is essential qemu itself keep track if it thinks p2p should work or not.
We could introduce something like the below to address this. I will include it in the next respin.
Thanks,
Shameer
--- include/system/memory.h | 2 ++ hw/arm/tegra241-cmdqv.c | 1 + hw/vfio/listener.c | 5 +++++ 3 files changed, 8 insertions(+)
include/system/memory.h +2 -0
@@ -874,6 +874,8 @@ struct MemoryRegion {/* For devices designed to perform re-entrant IO into their own IO MRs */ bool disable_reentrancy_guard; +/* RAM device region that does not require IOMMU mapping for P2P */ +bool ram_device_skip_iommu_map; };struct IOMMUMemoryRegion {
hw/arm/tegra241-cmdqv.c +1 -0
@@ -212,6 +212,7 @@ static void tegra241_cmdqv_guest_map_vintf_page0(Tegra241CMDQV *cmdqv)memory_region_owner (&cmdqv->mmio_cmdqv), name,VINTF_PAGE_SIZE , cmdqv->vintf_page0); + cmdqv->mr_vintf_page0->ram_device_skip_iommu_map = true;memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , cmdqv->mr_vintf_page0,1 );g_free (name);
hw/vfio/listener.c +5 -0
@@ -614,6 +614,11 @@ void vfio_container_region_add(VFIOContainer *bcontainer, } } +if (memory_region_is_ram_device (section->mr) && + section->mr->ram_device_skip_iommu_map) { +return ; + } + ret =vfio_container_dma_map (bcontainer, iova,int128_get64 (llsize), vaddr, section->readonly, section->mr);if (ret) {
--
2.43.0