hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3
Subsystems (2)
v2 cover hw/arm/virt: Introduce Tegra241 CMDQV support for accelerated SMMUv3
Hi,
Changes from RFCv1:
https://lore.kernel.org/qemu-devel/[email protected]/
- Rebased on top of latest master with SMMUv3 accel support.
- RFC tag dropped as the SMMUv3 accel series is now applied.
- Addressed feedback on RFC and collected tags. Thanks!
- Introduces an ops interface to factor CMDQV specific behavior out of the base implementation(patch #5)
- Split read/write implementation to multiple patches(10-12)
- Only map the VINTF page0 MMIO region to guest(patch #15)
- Add per-device identifier property to SMMUv3 dev so that both Tegra DSDT and SMMUv3 IORT uses the same identifier (patch #22)
- TODO: There was a suggestion to factor out common code for vEVNTQ read side function. I have left it as is for now.
Please find the complete branch here:
https://github.com/shamiali2008/qemu-master/tree/master-veventq-v4-vcmdq-v2
I have sanity tested this on an NVIDIA GRACE platform and will continue with additional tests.
Feedback and testing are very welcome.
Thanks,
Shameer
---
Background(from RFCv1):
Thanks to Nicolin for the initial patches and testing on which this is based.
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)
Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].
Summary of QEMU changes:
- Integrated into the existing SMMUv3 accel path via a "tegra241-cmdqv" property.
- Support for allocating vIOMMU objects of type IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.
- Mapping and emulation of the CMDQV MMIO register layout.
- VCMDQ/VINTF read/write handling and queue allocation using IOMMUFD APIs.
- Reset and initialisation hooks, including checks for at least one cold-plugged device.
- CMDQV hardware reads guest queue memory using host physical addresses provided through IOMMUFD, which requires that the VCMDQ buffer be physically contiguous not only in guest PA space but also in host PA space. When Tegra241 CMDQV is enabled, QEMU must therefore only expose a CMDQV size that the host can reliably back with contiguous physical memory. Because of this constraint, it is suggested to use huge pages to back the guest RAM.
- ACPI DSDT node generation for CMDQV devices on the virt machine.
---
Nicolin Chen (14):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Implement CMDQV init and vIOMMU allocation hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV hw/arm/tegra241-cmdqv: Emulate global CMDQV registers hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Shameer Kolothum (10):
hw/arm/smmuv3-accel: Introduce CMDQV ops interface hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle system/physmem: Add address_space_is_ram() helper hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev hw/arm/smmuv3: Add per-device identifier poroperty hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
backends/iommufd.c | 65 ++++ backends/trace-events | 4 +- hw/arm/Kconfig | 5 + hw/arm/meson.build | 3 +- hw/arm/smmuv3-accel.c | 73 +++- hw/arm/smmuv3-accel.h | 24 ++ hw/arm/smmuv3.c | 15 + hw/arm/tegra241-cmdqv.c | 778 +++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 340 +++++++++++++++++ hw/arm/trace-events | 5 + hw/arm/virt-acpi-build.c | 121 ++++-- hw/arm/virt.c | 3 + hw/vfio/iommufd.c | 4 +- include/hw/arm/smmuv3.h | 3 + include/system/iommufd.h | 17 + include/system/memory.h | 10 + system/physmem.c | 11 + 17 files changed, 1447 insertions(+), 34 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.h
Signature · 1 line
2.43.0
PATCH v2 1/24 backends/iommufd: Update iommufd_backend_get_device_info
From: Nicolin Chen <[email protected]>
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.
--- backends/iommufd.c | 7 +++++++ hw/arm/smmuv3-accel.c | 2 +- hw/vfio/iommufd.c | 4 ++-- 3 files changed, 10 insertions(+), 3 deletions(-)
backends/iommufd.c +7 -0
@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,return true; } +/* +* @type can carry a desired HW info type defined in the uapi headers. If caller +* doesn't have one, indicating it wants the default type, then @type should be +* zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT). +*/ bool iommufd_backend_get_device_info (IOMMUFDBackend *be,uint32_t devid,uint32_t *type,void *data,uint32_t len,uint64_t *caps,uint8_t *max_pasid_log2,Error **errp) {struct iommu_hw_info info = { + .flags = (*type) ?IOMMU_HW_INFO_FLAG_INPUT_TYPE :0 , .size =sizeof (info), .dev_id = devid, .data_len = len, .data_uptr = (uintptr_t )data, + .in_data_type = *type, };if (ioctl (be->fd,IOMMU_GET_HW_INFO , &info)) {
hw/arm/smmuv3-accel.c +1 -1
@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) { struct iommu_hw_info_arm_smmuv3 info; - uint32_t data_type; + uint32_t data_type = 0; uint64_t caps; if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +2 -2
@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,ERRP_GUARD ();IOMMUFDBackend *iommufd = vbasedev->iommufd;VFIOContainer *bcontainer =VFIO_IOMMU (container); -uint32_t type, flags =0 ; +uint32_t type =0 , flags =0 ;uint64_t hw_caps;VendorCaps caps;VFIOIOASHwpt *hwpt;
@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,HostIOMMUDeviceIOMMUFD *idev;HostIOMMUDeviceCaps *caps = &hiod->caps;VendorCaps *vendor_caps = &caps->vendor_caps; -enum iommu_hw_info_type type; +enum iommu_hw_info_type type =0 ;uint8_t max_pasid_log2;uint64_t hw_caps;
Signature · 1 line
2.43.0
Re: [PATCH v2 01/24] backends/iommufd: Update iommufd_backend_get_device_info
Hi Shameer, Nicolin,
Quoted 50 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via
IOMMU_GET_HW_INFO. Updateiommufd_backend_get_device_info()to setIOMMU_HW_INFO_FLAG_INPUT_TYPEwhen a non-zero type is supplied, and adjust all callers to pass a type value explicitly initialised to zero (IOMMU_HW_INFO_TYPE_DEFAULT) when no specific type is requested.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 7 +++++++ hw/arm/smmuv3-accel.c | 2 +- hw/vfio/iommufd.c | 4 ++-- 3 files changed, 10 insertions(+), 3 deletions(-)backends/iommufd.c +7
-0@@ -387,16 +387,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,return true; } +/* +* @type can carry a desired HW info type defined in the uapi headers. If caller +* doesn't have one, indicating it wants the default type, then @type should be +* zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT). +*/ bool iommufd_backend_get_device_info (IOMMUFDBackend *be,uint32_t devid,uint32_t *type,void *data,uint32_t len,uint64_t *caps,uint8_t *max_pasid_log2,Error **errp) {struct iommu_hw_info info = { + .flags = (*type) ?IOMMU_HW_INFO_FLAG_INPUT_TYPE :0 , .size =sizeof (info), .dev_id = devid, .data_len = len, .data_uptr = (uintptr_t )data, + .in_data_type = *type, };if (ioctl (be->fd,IOMMU_GET_HW_INFO , &info)) {hw/arm/smmuv3-accel.c +0
-0
hw/arm/smmuv3-accel.c smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 130
Quoted 5 lines
hw/arm/smmuv3-accel.c +1
-1@@ -127,7 +127,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) {struct iommu_hw_info_arm_smmuv3 info; -uint32_t data_type; +uint32_t data_type =0 ;
I would use IOMMU_HW_INFO_TYPE_DEFAULT explicitly.
Quoted 17 lines
hw/arm/smmuv3-accel.c +0
-0uint64_t caps; if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,hw/vfio/iommufd.c +1
-1@@ -349,7 +349,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,ERRP_GUARD ();IOMMUFDBackend *iommufd = vbasedev->iommufd;VFIOContainer *bcontainer =VFIO_IOMMU (container); -uint32_t type, flags =0 ; +uint32_t type =0 , flags =0 ;uint64_t hw_caps;VendorCaps caps;VFIOIOASHwpt *hwpt;
hw/vfio/iommufd.c static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, · line 941
Quoted 5 lines
hw/vfio/iommufd.c +1
-1@@ -938,7 +938,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,HostIOMMUDeviceIOMMUFD *idev;HostIOMMUDeviceCaps *caps = &hiod->caps;VendorCaps *vendor_caps = &caps->vendor_caps; -enum iommu_hw_info_type type; +enum iommu_hw_info_type type =0 ;
here also
With that
Thanks
Eric
Quoted 3 lines
hw/vfio/iommufd.c +0
-0uint8_t max_pasid_log2;uint64_t hw_caps;
PATCH v2 2/24 backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.
Update the caller accordingly.
--- backends/iommufd.c | 4 ++++ backends/trace-events | 2 +- hw/arm/smmuv3-accel.c | 4 ++-- include/system/iommufd.h | 1 + 4 files changed, 8 insertions(+), 3 deletions(-)
backends/iommufd.c +4 -0
@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t data_len,uint32_t *out_viommu_id,Error **errp) {int ret;
@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, .type = viommu_type, .dev_id = dev_id, .hwpt_id = hwpt_id, + .data_len = data_len, + .data_uptr = (uintptr_t)data_ptr, }; ret =ioctl (be->fd,IOMMU_VIOMMU_ALLOC , &alloc_viommu);trace_iommufd_backend_alloc_viommu (be->fd, dev_id, viommu_type, hwpt_id, + data_len, (uintptr_t )data_ptr, alloc_viommu.out_viommu_id, ret);if (ret) {error_setg_errno (errp, errno,"IOMMU_VIOMMU_ALLOC failed" );
backends/trace-events +1 -1
@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (% iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)" iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)" iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)" -iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)" +iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"
hw/arm/smmuv3-accel.c +2 -2
@@ -531,8 +531,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,IOMMUFDViommu *viommu;if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , - s2_hwpt_id, &viommu_id, errp)) { +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, +NULL ,0 , &viommu_id, errp)) {return false; }
include/system/iommufd.h +1 -0
@@ -87,6 +87,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id, Error **errp);bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t data_len,uint32_t *out_hwpt,Error **errp); bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
Signature · 1 line
2.43.0
Re: [PATCH v2 02/24] backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr
Quoted 51 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
The updated IOMMUFD
VIOMMU_ALLOCuAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extendiommufd_backend_alloc_viommu()to pass a user pointer and size to the kernel.Update the caller accordingly.
Signed-off-by Nicolin Chen <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 4 ++++ backends/trace-events | 2 +- hw/arm/smmuv3-accel.c | 4 ++-- include/system/iommufd.h | 1 + 4 files changed, 8 insertions(+), 3 deletions(-)backends/iommufd.c +4
-0@@ -460,6 +460,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t data_len,uint32_t *out_viommu_id,Error **errp) {int ret;@@ -468,11 +469,14 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, .type = viommu_type, .dev_id = dev_id, .hwpt_id = hwpt_id, + .data_len = data_len, + .data_uptr = (uintptr_t)data_ptr, }; ret =ioctl (be->fd,IOMMU_VIOMMU_ALLOC , &alloc_viommu);trace_iommufd_backend_alloc_viommu (be->fd, dev_id, viommu_type, hwpt_id, + data_len, (uintptr_t )data_ptr, alloc_viommu.out_viommu_id, ret);if (ret) {error_setg_errno (errp, errno,"IOMMU_VIOMMU_ALLOC failed" );backends/trace-events +0
-0
backends/trace-events iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (% · line 24
Quoted 5 lines
backends/trace-events +1
-1@@ -21,7 +21,7 @@ iommufd_backend_free_id(int iommufd, uint32_t id, int ret) " iommufd=%d id=%d (% iommufd_backend_set_dirty(int iommufd, uint32_t hwpt_id, bool start, int ret) " iommufd=%d hwpt=%u enable=%d (%d)" iommufd_backend_get_dirty_bitmap(int iommufd, uint32_t hwpt_id, uint64_t iova, uint64_t size, uint64_t flags, uint64_t page_size, int ret) " iommufd=%d hwpt=%u iova=0x%"PRIx64" size=0x%"PRIx64" flags=0x%"PRIx64" page_size=0x%"PRIx64" (%d)" iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, uint32_t entry_len, uint32_t entry_num, uint32_t done_num, uint64_t data_ptr, int ret) " iommufd=%d id=%u data_type=%u entry_len=%u entry_num=%u done_num=%u data_ptr=0x%"PRIx64" (%d)" -iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)" +iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)"
nit: I would put the data_ptr first and then the data length to match most prevalent pattern. Also use hexa format for the length?
Besides
Eric
Quoted 30 lines
backends/trace-events +0
-0iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"hw/arm/smmuv3-accel.c +2
-2@@ -531,8 +531,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,IOMMUFDViommu *viommu;if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , - s2_hwpt_id, &viommu_id, errp)) { +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, +NULL ,0 , &viommu_id, errp)) {return false; }include/system/iommufd.h +1
-0@@ -87,6 +87,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id, Error **errp);bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t data_len,uint32_t *out_hwpt,Error **errp); bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
PATCH v2 3/24 backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add a helper to allocate an iommufd backed HW queue for a vIOMMU.
While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.
--- backends/iommufd.c | 32 ++++++++++++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 12 ++++++++++++ 3 files changed, 45 insertions(+)
backends/iommufd.c +32 -0
@@ -546,6 +546,38 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +enum iommu_hw_queue_type queue_type, +uint32_t index,uint64_t addr, +uint64_t length,uint32_t *out_hw_queue_id, +Error **errp) +{ +int ret; +struct iommu_hw_queue_alloc alloc_hw_queue = { + .size =sizeof (alloc_hw_queue), + .flags =0 , + .viommu_id = viommu_id, + .type = queue_type, + .index = index, + .nesting_parent_iova = addr, + .length = length, + }; + + ret =ioctl (be->fd,IOMMU_HW_QUEUE_ALLOC , &alloc_hw_queue); + +trace_iommufd_backend_alloc_hw_queue (be->fd, viommu_id, queue_type, + index, addr, length, + alloc_hw_queue.out_hw_queue_id, ret); +if (ret) { +error_setg_errno (errp, errno,"IOMMU_HW_QUEUE_ALLOC failed" ); +return false; + } + +g_assert (out_hw_queue_id); + *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id; +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {
backends/trace-events +1 -0
@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" +iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)" # igvm-cfg.c igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +12 -0
@@ -63,6 +63,12 @@ typedef struct IOMMUFDVeventq {uint32_t veventq_fd; } IOMMUFDVeventq; +/* HW queue object for a vIOMMU-specific HW-accelerated queue */ +typedef struct IOMMUFDHWqueue { +IOMMUFDViommu *viommu; +uint32_t hw_queue_id; +}IOMMUFDHWqueue ; +bool iommufd_backend_connect (IOMMUFDBackend *be,Error **errp);void iommufd_backend_disconnect (IOMMUFDBackend *be);
@@ -99,6 +105,12 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,uint32_t *out_veventq_id, uint32_t *out_veventq_fd, Error **errp); +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +enum iommu_hw_queue_type queue_type, +uint32_t index,uint64_t addr, +uint64_t length,uint32_t *out_hw_queue_id, +Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Signature · 1 line
2.43.0
Re: [PATCH v2 03/24] backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
Quoted 7 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add a helper to allocate an iommufd backed HW queue for a vIOMMU.
While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.
Quoted 7 lines
Signed-off-by Nicolin Chen <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 32 ++++++++++++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 12 ++++++++++++
Could you please add to your .gitconfig :
[diff]
orderFile = /path/to/qemu/scripts/git.orderfile
Quoted 75 lines
3 files changed, 45 insertions(+)backends/iommufd.c +32
-0@@ -546,6 +546,38 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +enum iommu_hw_queue_type queue_type, +uint32_t index,uint64_t addr, +uint64_t length,uint32_t *out_hw_queue_id, +Error **errp) +{ +int ret; +struct iommu_hw_queue_alloc alloc_hw_queue = { + .size =sizeof (alloc_hw_queue), + .flags =0 , + .viommu_id = viommu_id, + .type = queue_type, + .index = index, + .nesting_parent_iova = addr, + .length = length, + }; + + ret =ioctl (be->fd,IOMMU_HW_QUEUE_ALLOC , &alloc_hw_queue); + +trace_iommufd_backend_alloc_hw_queue (be->fd, viommu_id, queue_type, + index, addr, length, + alloc_hw_queue.out_hw_queue_id, ret); +if (ret) { +error_setg_errno (errp, errno,"IOMMU_HW_QUEUE_ALLOC failed" ); +return false; + } + +g_assert (out_hw_queue_id); + *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id; +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {backends/trace-events +1
-0@@ -24,6 +24,7 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t data_len, uint64_t data_ptr, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u data_len=%u data_ptr=0x%"PRIx64" viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" +iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)" # igvm-cfg.c igvm_reset_enter(int type) "type=%u"include/system/iommufd.h +6
-0@@ -63,6 +63,12 @@ typedef struct IOMMUFDVeventq {uint32_t veventq_fd; } IOMMUFDVeventq; +/* HW queue object for a vIOMMU-specific HW-accelerated queue */ +typedef struct IOMMUFDHWqueue { +IOMMUFDViommu *viommu; +uint32_t hw_queue_id; +}IOMMUFDHWqueue ; +bool iommufd_backend_connect (IOMMUFDBackend *be,Error **errp);void iommufd_backend_disconnect (IOMMUFDBackend *be);
include/system/iommufd.h bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id, · line 109
Quoted 5 lines
include/system/iommufd.h +2
-0@@ -99,6 +105,12 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,uint32_t *out_veventq_id, uint32_t *out_veventq_fd, Error **errp); +bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, + enum iommu_hw_queue_type queue_type,
Please use a uint32_t for the queue_type parameter. 'enum iommu_hw_queue_type' is a Linux-only type and build will break on other platform.
Thanks,
C.
Quoted 7 lines
include/system/iommufd.h +4
-0+ uint32_t index, uint64_t addr, + uint64_t length, uint32_t *out_hw_queue_id, + Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
PATCH v2 4/24 backends/iommufd: Introduce iommufd_backend_viommu_mmap
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.
The caller is responsible for unmapping the returned region.
--- backends/iommufd.c | 22 ++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 4 ++++ 3 files changed, 27 insertions(+)
backends/iommufd.c +22 -0
@@ -578,6 +578,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +/* +* Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance. +* The caller is responsible for unmapping the mapped region. +*/ +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp) +{ +g_assert (viommu_id); +g_assert (out_ptr); + + *out_ptr =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , be->fd, + offset); +trace_iommufd_backend_viommu_mmap (be->fd, viommu_id, size, offset); +if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"IOMMUFD vIOMMU mmap failed" ); +return false; + } + +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {
backends/trace-events +1 -0
@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)" +iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64 # igvm-cfg.c igvm_reset_enter(int type) "type=%u"
include/system/iommufd.h +4 -0
@@ -111,6 +111,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, uint64_t length, uint32_t *out_hw_queue_id, Error **errp); +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Signature · 1 line
2.43.0
Re: [PATCH v2 04/24] backends/iommufd: Introduce iommufd_backend_viommu_mmap
Quoted 22 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.
The caller is responsible for unmapping the returned region.
Signed-off-by Nicolin Chen <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 22 ++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 4 ++++ 3 files changed, 27 insertions(+)backends/iommufd.c +0
-0
backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, · line 587
Quoted 10 lines
backends/iommufd.c +7
-0@@ -578,6 +578,28 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +/* +* Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance. +* The caller is responsible for unmapping the mapped region. +*/ +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp)
Looking at this again, there is nothing really related to iommufd. This could simply end up as a static helper in tegra241-cmdqv.c
Eric
Quoted 45 lines
backends/iommufd.c +15
-0+{ +g_assert (viommu_id); +g_assert (out_ptr); + + *out_ptr =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , be->fd, + offset); +trace_iommufd_backend_viommu_mmap (be->fd, viommu_id, size, offset); +if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"IOMMUFD vIOMMU mmap failed" ); +return false; + } + +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {backends/trace-events +1
-0@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t queue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t queue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)" +iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64 # igvm-cfg.c igvm_reset_enter(int type) "type=%u"include/system/iommufd.h +4
-0@@ -111,6 +111,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, uint64_t length, uint32_t *out_hw_queue_id, Error **errp); +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
RE: [PATCH v2 04/24] backends/iommufd: Introduce iommufd_backend_viommu_mmap
Quoted 51 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 13 February 2026 10:19
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 04/24] backends/iommufd: Introduceiommufd_backend_viommu_mmapExternal email: Use caution opening links or attachments
Quoted 3 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add a backend helper to mmap hardware MMIO regions exposed via
iommufd for
backends/iommufd.c bool
Quoted 19 lines
a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.
The caller is responsible for unmapping the returned region.
Signed-off-by Nicolin Chen <[email protected]>Reviewed-by Eric Auger <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 22 ++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 4 ++++ 3 files changed, 27 insertions(+)backends/iommufd.c +0
-0@@ -578,6 +578,28 @@ booliommufd_backend_alloc_hw_queue(IOMMUFDBackend *be,
uint32_tviommu_id,backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
Quoted 5 lines
backends/iommufd.c +2
-0return true; } +/* +* Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance.
Quoted 8 lines
backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
Quoted 3 lines
backends/iommufd.c +3
-0+* The caller is responsible for unmapping the mapped region. +*/ +bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t
viommu_id,backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, · line 587
Quoted 2 lines
backends/iommufd.c +2
-0+ uint64_t size, off_t offset, void **out_ptr, + Error **errp)Looking at this again, there is nothing really related to iommufd. This could simply end up as a static helper in
tegra241-cmdqv.c
I had that thought as well. However, this operates on the IOMMUFDBackend fd and is not tegra specific. It performs an mmap on the backend fd associated with a vIOMMU instance, which could potentially be reused by others.
Keeping it in backends/iommufd.c also centralizes fd related operations (including tracing) for the IOMMUFD backend.
Thanks,
Shameer
Quoted 64 lines
backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
Quoted 5 lines
backends/iommufd.c +5
-0+{ +g_assert (viommu_id); +g_assert (out_ptr); + + *out_ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED,
be->fd,backends/iommufd.c bool · line 594
Quoted 2 lines
backends/iommufd.c +2
-0+ offset); + trace_iommufd_backend_viommu_mmap(be->fd, viommu_id, size,offset);
backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
Quoted 9 lines
backends/iommufd.c +8
-0+if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"IOMMUFD vIOMMU mmap failed" ); +return false; + } + +return true; +} +bool host_iommu_device_iommufd_attach_hwpt(HostIOMMUDeviceIOMMUFD *idev,
backends/trace-events iommufd_backend_alloc_viommu(int iommufd,
Quoted 7 lines
backends/iommufd.c +0
-0uint32_t hwpt_id, Error **errp) {backends/trace-events +0
-0@@ -25,6 +25,7 @@ iommufd_backend_alloc_viommu(int iommufd,
uint32_tdev_id,uint32_ttype, uint32backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 25
Quoted 1 line
backends/trace-events +0
-0iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t
viommu_id,uint64_tvirt_id,uint32_tvdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)"backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 26
Quoted 1 line
backends/trace-events +0
-0iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id,
uint32_ttype,uint32_tveventq_id,uint32_tveventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)"backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 27
Quoted 1 line
backends/trace-events +0
-0iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id,
uint32_tqueue_type,uint32_tindex,uint64_taddr,uint64_tsize,uint32_tqueue_id, int ret) " iommufd=%d viommu_id=%u queue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" queue_id=%u (%d)"backends/trace-events iommufd_backend_alloc_viommu(int iommufd, · line 28
Quoted 1 line
backends/trace-events +1
-0+iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id,
uint64_tsize,uint64_toffset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64include/system/iommufd.h bool
Quoted 8 lines
backends/trace-events +0
-0# igvm-cfg.c igvm_reset_enter(int type) "type=%u"include/system/iommufd.h +0
-0@@ -111,6 +111,10 @@ booliommufd_backend_alloc_hw_queue(IOMMUFDBackend *be,
uint32_tviommu_id,include/system/iommufd.h bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
Quoted 4 lines
include/system/iommufd.h +1
-0uint64_t length, uint32_t *out_hw_queue_id, Error **errp); +bool iommufd_backend_viommu_mmap(IOMMUFDBackend *be, uint32_t
viommu_id,include/system/iommufd.h bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,
Quoted 4 lines
include/system/iommufd.h +3
-0+ uint64_t size, off_t offset, void **out_ptr, + Error **errp); + bool iommufd_backend_set_dirty_tracking(IOMMUFDBackend *be,
uint32_thwpt_id,include/system/iommufd.h bool · line 120
Quoted 2 lines
include/system/iommufd.h +0
-0bool start, Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,
uint32_thwpt_id,
PATCH v2 5/24 hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer KolothumCMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
hw/arm/smmuv3-accel.h +19 -0
@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to +* support multiple VCMDQs with virtualization capabilities. +* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { +bool (*init)(SMMUv3State *s,Error **errp); +bool (*alloc_viommu)(SMMUv3State *s, +HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id, +Error **errp); +void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s); +void (*reset)(SMMUv3State *s); +}SMMUv3AccelCmdqvOps ; +/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object. * Holds bypass and abort proxy HWPT IDs used for device attachment.
@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv; } SMMUv3AccelState; typedef struct SMMUS1Hwpt {
Signature · 1 line
2.43.0
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 2 lines · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
Quoted 18 lines
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +0
-0
hw/arm/smmuv3-accel.h line 21
Quoted 6 lines
hw/arm/smmuv3-accel.h +3
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to +* support multiple VCMDQs with virtualization capabilities.
Ditto
hw/arm/smmuv3-accel.h line 30
Quoted 10 lines
hw/arm/smmuv3-accel.h +9
-0+* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { + bool (*init)(SMMUv3State *s, Error **errp); + bool (*alloc_viommu)(SMMUv3State *s, + HostIOMMUDeviceIOMMUFD *idev, + uint32_t *out_viommu_id, + Error **errp); + void (*free_viommu)(SMMUv3State *s);
I don't see much value of having alloc/free_viommu..
Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.
hw/arm/smmuv3-accel.h line 32
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);
alloc_/free_veventq can be moved into alloc/free_viommu functions.
hw/arm/smmuv3-accel.h line 34
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+void (*reset)(SMMUv3State *s); +} SMMUv3AccelCmdqvOps;
Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49
Quoted 9 lines
hw/arm/smmuv3-accel.h +2
-0/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object. * Holds bypass and abort proxy HWPT IDs used for device attachment. @@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;
Having two pointers for the same extension feels redundant. Maybe merge them with:
typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV struct iommu_viommu_tegra241_cmdqv viommu_data;
} SMMUv3AccelCmdqv;
...
SMMUv3AccelState {
...
SMMUv3AccelCmdqv *cmdqv;
};?
Nicolin
RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 16 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 6 lines
Quoted 1 line · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that
enables
Quoted 1 line
virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.
Quoted 40 lines
Quoted 19 lines
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +0
-0hw/arm/smmuv3-accel.h line 20
Quoted 5 lines
hw/arm/smmuv3-accel.h +2
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
hw/arm/smmuv3-accel.h line 21
Quoted 1 line
hw/arm/smmuv3-accel.h +1
-0+* support multiple VCMDQs with virtualization capabilities. Ditto
hw/arm/smmuv3-accel.h line 30
Quoted 10 lines
hw/arm/smmuv3-accel.h +9
-0+* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { + bool (*init)(SMMUv3State *s, Error **errp); + bool (*alloc_viommu)(SMMUv3State *s, + HostIOMMUDeviceIOMMUFD *idev, + uint32_t *out_viommu_id, + Error **errp); + void (*free_viommu)(SMMUv3State *s);
Quoted 4 lines
I don't see much value of having alloc/free_viommu..
Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.
That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.
Quoted 11 lines
hw/arm/smmuv3-accel.h line 32
Quoted 3 lines
hw/arm/smmuv3-accel.h +2
-0+bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);alloc_/free_veventq can be moved into alloc/free_viommu functions.
hw/arm/smmuv3-accel.h line 34
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+void (*reset)(SMMUv3State *s); +} SMMUv3AccelCmdqvOps;Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?
That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.
I'm open to suggestions, and let's wait to hear from others as well.
Quoted 19 lines
hw/arm/smmuv3-accel.h line 37
Quoted 2 lines
hw/arm/smmuv3-accel.h +0
-0/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49
Quoted 7 lines
hw/arm/smmuv3-accel.h +2
-0* Holds bypass and abort proxy HWPT IDs used for device attachment. @@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;Having two pointers for the same extension feels redundant. Maybe merge them with:
typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV structiommu_viommu_tegra241_cmdqvviommu_data;
} SMMUv3AccelCmdqv;
Quoted 8 lines
...
SMMUv3AccelState { ... SMMUv3AccelCmdqv *cmdqv; };?
We could fold the private data and ops into a single struct and store that in SMMUv3AccelState. However, if we want to keep this generic, it may be better to keep them separate.
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 11 lines · Shameer Kolothum Thodi
Quoted 6 lines · Shameer Kolothum
Quoted 1 line
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that
enables
Quoted 1 line
virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.
That might be a different naming which we have no idea about yet :-/
Quoted 7 lines
Quoted 6 lines
hw/arm/smmuv3-accel.h line 34
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+void (*reset)(SMMUv3State *s); +} SMMUv3AccelCmdqvOps;Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?
Quoted 6 lines
That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.
I'm open to suggestions, and let's wait to hear from others as well.
Well, I'll vote for adding that when someday there is second CMDQV. There are only a couple of CMDQ-V functions required to export for SMMUv3 code to call at this moment, which isn't so bad.
Having and naming a structure exclusively for NVIDIA CMDQ-V doesn't make a lot of sense to me. And if we really want an abstraction ops maybe "secondary_cmdq" in the kernel driver could be a good choice?
Nicolin
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 22 lines · Nicolin Chen
Quoted 9 lines · Shameer Kolothum Thodi
Quoted 5 lines · Shameer Kolothum
Quoted 1 line
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that
enables
Quoted 1 line
virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.
That might be a different naming which we have no idea about yet :-/
Quoted 10 lines
Quoted 5 lines
hw/arm/smmuv3-accel.h line 34
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+void (*reset)(SMMUv3State *s); +} SMMUv3AccelCmdqvOps;Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?
That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.
Quoted 8 lines
Quoted 1 line
I'm open to suggestions, and let's wait to hear from others as well.
Well, I'll vote for adding that when someday there is second CMDQV. There are only a couple of CMDQ-V functions required to export for SMMUv3 code to call at this moment, which isn't so bad.
Having and naming a structure exclusively for NVIDIA CMDQ-V doesn't make a lot of sense to me. And if we really want an abstraction ops maybe "
secondary_cmdq" in the kernel driver could be a good choice?
As mentionned earlier mu syggestion was mainly to address specialization with known implementations: accel smmu with vcmdq versus without. And i think we already have some implementation override in that matter.
Eric
Quoted 3 lines
Nicolin
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer Kolothum ThodiQuoted 61 lines · Shameer Kolothum Thodi
Quoted 15 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceQuoted 5 lines
Quoted 1 line · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that
enables
Quoted 1 line
virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.
Quoted 36 lines
Quoted 18 lines
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +0
-0hw/arm/smmuv3-accel.h line 20
Quoted 5 lines
hw/arm/smmuv3-accel.h +2
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to
hw/arm/smmuv3-accel.h line 21
Quoted 1 line
hw/arm/smmuv3-accel.h +1
-0+* support multiple VCMDQs with virtualization capabilities. Ditto
Quoted 9 lines
hw/arm/smmuv3-accel.h +9
-0+* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { + bool (*init)(SMMUv3State *s, Error **errp); + bool (*alloc_viommu)(SMMUv3State *s, + HostIOMMUDeviceIOMMUFD *idev, + uint32_t *out_viommu_id, + Error **errp); + void (*free_viommu)(SMMUv3State *s);
Quoted 7 lines
Quoted 4 lines
I don't see much value of having alloc/free_viommu..
Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.
That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.
My initial thought was callbacks to specialize implementation for accel smmu without vcmdq versus accel smmu with vcmdqs. Effectively we could later imagine to have other implementations with vcmdqs but it is difficult to guess the specialization needs at that stage. I think in the original series you already add the first kind of differentiation.
At least reset and init need to be specialized.
Maybe alloc_viommu does not need to because the differentiation only happens on the type arg. But originally you where also turning off
s->tegra241_cmdqv if IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV failed so I had the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric
Quoted 49 lines
Quoted 9 lines
hw/arm/smmuv3-accel.h line 32
Quoted 3 lines
hw/arm/smmuv3-accel.h +2
-0+bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);alloc_/free_veventq can be moved into alloc/free_viommu functions.
hw/arm/smmuv3-accel.h line 34
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+void (*reset)(SMMUv3State *s); +} SMMUv3AccelCmdqvOps;Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is exclusive to Tegra241CMDQV?
That's true for now, as there is only one implementation today. If this abstraction feels over engineered at this stage, we can simplify or defer it (and fall back to calling the tegra241-specific functions directly, as in the RFC) and revisit once there is a clearer need.
I'm open to suggestions, and let's wait to hear from others as well.
Quoted 18 lines
hw/arm/smmuv3-accel.h line 36
Quoted 2 lines
hw/arm/smmuv3-accel.h +0
-0/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object.
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49
Quoted 7 lines
hw/arm/smmuv3-accel.h +2
-0* Holds bypass and abort proxy HWPT IDs used for device attachment. @@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;Having two pointers for the same extension feels redundant. Maybe merge them with:
typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV structiommu_viommu_tegra241_cmdqvviommu_data;
} SMMUv3AccelCmdqv;Quoted 8 lines
...
SMMUv3AccelState { ... SMMUv3AccelCmdqv *cmdqv; };?
We could fold the private data and ops into a single struct and store that in SMMUv3AccelState. However, if we want to keep this generic, it may be better to keep them separate.
Thanks,
Shameer
RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Hi Eric,
Quoted 97 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:42
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceExternal email: Use caution opening links or attachments
Quoted 13 lines · Shameer Kolothum Thodi
Quoted 13 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQVops
Quoted 33 lines
Quoted 7 lines
interface
Quoted 1 line · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that
enables
Quoted 1 line
virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.
Quoted 21 lines
Quoted 21 lines
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +0
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES Quoted 2 lines
Quoted 2 lines
hw/arm/smmuv3-accel.h line 20
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3
Quoted 13 lines
Quoted 13 lines
to
hw/arm/smmuv3-accel.h line 21
Quoted 1 line
hw/arm/smmuv3-accel.h +1
-0+* support multiple VCMDQs with virtualization capabilities. Ditto
Quoted 9 lines
hw/arm/smmuv3-accel.h +9
-0+* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { + bool (*init)(SMMUv3State *s, Error **errp); + bool (*alloc_viommu)(SMMUv3State *s, + HostIOMMUDeviceIOMMUFD *idev, + uint32_t *out_viommu_id, + Error **errp); + void (*free_viommu)(SMMUv3State *s);Quoted 7 lines
Quoted 4 lines
I don't see much value of having alloc/free_viommu..
Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.
That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.
My initial thought was callbacks to specialize implementation for accel smmu without vcmdq versus accel smmu with vcmdqs. Effectively we could later imagine to have other implementations with vcmdqs but it is difficult to guess the specialization needs at that stage. I think in the original series you already add the first kind of differentiation.
At least reset and init need to be specialized.
Quoted 7 lines
Maybe
alloc_viommudoes not need to because the differentiation only happens on the type arg. But originally you where also turning off
s->tegra241_cmdqvifIOMMU_VIOMMU_TYPE_TEGRA241_CMDQVfailed so I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric
There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 100 lines · Shameer Kolothum Thodi
Hi Eric,
Quoted 98 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:42
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceExternal email: Use caution opening links or attachments
Quoted 4 lines · Shameer Kolothum Thodi
Quoted 4 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 19:42
To: Shameer Kolothum Thodi <[email protected]>Quoted 8 lines
Quoted 8 lines
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQVops
Quoted 33 lines
Quoted 7 lines
interface
Quoted 1 line · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that
enables
Quoted 1 line
virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
The idea of this ops based callback is to make it more generic and not NVIDIA specific to start with. This leaves room in case ARM introduces a standardised version of something similar in the future, or if another vendor provides a similar extension.
Quoted 21 lines
Quoted 21 lines
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +0
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES Quoted 2 lines
Quoted 2 lines
hw/arm/smmuv3-accel.h line 20
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3
Quoted 13 lines
Quoted 13 lines
to
hw/arm/smmuv3-accel.h line 21
Quoted 1 line
hw/arm/smmuv3-accel.h +1
-0+* support multiple VCMDQs with virtualization capabilities. Ditto
Quoted 9 lines
hw/arm/smmuv3-accel.h +9
-0+* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { + bool (*init)(SMMUv3State *s, Error **errp); + bool (*alloc_viommu)(SMMUv3State *s, + HostIOMMUDeviceIOMMUFD *idev, + uint32_t *out_viommu_id, + Error **errp); + void (*free_viommu)(SMMUv3State *s);Quoted 7 lines
Quoted 4 lines
I don't see much value of having alloc/free_viommu..
Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.
That is right. But again, this is mainly to keep the interface generic. IIUC, Eric had suggested, using an ops or object style abstraction to separate extension specific behaviour from the core code.
My initial thought was callbacks to specialize implementation for accel smmu without vcmdq versus accel smmu with vcmdqs. Effectively we could later imagine to have other implementations with vcmdqs but it is difficult to guess the specialization needs at that stage. I think in the original series you already add the first kind of differentiation.
At least reset and init need to be specialized.
Maybe
alloc_viommudoes not need to because the differentiation only happens on the type arg. But originally you where also turning off
Quoted 6 lines
Quoted 4 lines
s->tegra241_cmdqvifIOMMU_VIOMMU_TYPE_TEGRA241_CMDQVfailed so I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. EricThere is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
Thanks
Eric
Quoted 4 lines
Thanks,
Shameer
RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 18 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceQuoted 3 lines
Quoted 3 lines
I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. Eric
Quoted 6 lines
Quoted 2 lines
There is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
It is not an either/or case.
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();
if (accel && cmdqv)
cmdqv_init();
So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 30 lines · Shameer Kolothum Thodi
Quoted 14 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceQuoted 8 lines
Quoted 5 lines
Quoted 3 lines
I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. EricThere is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
It is not an either/or case.
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();
Quoted 8 lines
if (accel && cmdqv)
cmdqv_init();So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
Well I am concerned by the lack of scalability overall. Until now this was more or less generic. Now we may have several implementors coming with their own implementations of even a vendor will come with several
generations: today it is tegra241_cmdq, tomorrow we will have tegraxxx_cmdq, arm_cmdq, ... How to get prepared for this mess? Will we have a property for each one?
Eric
Quoted 3 lines
Thanks,
Shameer
RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 61 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 15:29
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceExternal email: Use caution opening links or attachments
Quoted 8 lines · Shameer Kolothum Thodi
Quoted 8 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; NathanChen
Quoted 5 lines
Quoted 5 lines
<[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQVops
Quoted 17 lines
Quoted 9 lines
interface
Quoted 5 lines
Quoted 3 lines
I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. EricThere is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
It is not an either/or case.
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();Quoted 8 lines
if (accel && cmdqv)
cmdqv_init();So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
Quoted 6 lines
Well I am concerned by the lack of scalability overall. Until now this was more or less generic. Now we may have several implementors coming with their own implementations of even a vendor will come with several
generations: today it istegra241_cmdq, tomorrow we will havetegraxxx_cmdq,arm_cmdq, ... How to get prepared for this mess? Will we have a property for each one?
Agree. But at the same time, it is hard to design without knowing the details of other implementations. As a middle path I would propose to keep the ops but just have the additional init/reset callbacks there. This gives an opportunity to extend it in future without over engineering now.
For having separate property for every implementation, that looks like the way it is going to be now. Unless kernel provides us a way to probe the extension type.
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer Kolothum ThodiQuoted 30 lines · Shameer Kolothum Thodi
Quoted 14 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceQuoted 8 lines
Quoted 5 lines
Quoted 3 lines
I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. EricThere is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
It is not an either/or case.
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();
Quoted 8 lines
if (accel && cmdqv)
cmdqv_init();So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?
Eric
Quoted 3 lines
Thanks,
Shameer
RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 58 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 15:35
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceExternal email: Use caution opening links or attachments
Quoted 8 lines · Shameer Kolothum Thodi
Quoted 8 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; NathanChen
Quoted 5 lines
Quoted 5 lines
<[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQVops
Quoted 23 lines
Quoted 9 lines
interface
Quoted 5 lines
Quoted 3 lines
I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. EricThere is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
It is not an either/or case.
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();if (accel && cmdqv)
cmdqv_init();So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
Quoted 6 lines
Quoted 2 lines
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?
Looking at it again, I think we can, by calling IOMMU_GET_HW_INFO with all supported CMDQ types and check which one succeeds.
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Hi Shameer,
Quoted 56 lines · Shameer Kolothum Thodi
Quoted 56 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 15:35
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceExternal email: Use caution opening links or attachments
Quoted 7 lines · Shameer Kolothum Thodi
Quoted 7 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 14:57
To: Shameer Kolothum Thodi <[email protected]>; Nicolin Chen <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; NathanChen
Quoted 5 lines
Quoted 5 lines
<[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQVops
Quoted 23 lines
Quoted 9 lines
interface
Quoted 5 lines
Quoted 3 lines
I had
the impression you reverted to a fallback impl. If there is specialization, I still think ops are relevant. EricThere is no fallback implementation. So just to confirm, you are suggesting to remove the ops completely or just keep the ops for reset/init only?
Depends on the code to come next. If your reset and init really differs between normal accel SMMU and vcmdq enhanced accel SMMU, to me it is still relevant to have separate implementations in different callbacks.
It is not an either/or case.
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();if (accel && cmdqv)
cmdqv_init();So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?
Looking at it again, I think we can, by calling
IOMMU_GET_HW_INFOwith all supported CMDQ types and check which one succeeds.
then I think we should endeavour doing that. Regarding the ops, if you feel that reset/init can be managed through a simple add-on code I am fine with it. Maybe Cédric or Peter will have another opinion though ;-)
Eric
Quoted 5 lines
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 11 lines · Eric Auger
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
The vCMDQ path does not replace the normal accelerated SMMU init/reset. Instead, it requires additional init/reset steps on top of the regular accel flow.
if (accel)
accel_init();if (accel && cmdqv)
cmdqv_init();So the vCMDQ logic is an extension of the accelerated mode, not a fallback or alternative implementation.
Yes. And I think it makes more sense by moving cmdqv_init() into accel_init(), completely transparent to the common smmuv3 module maybe?
Quoted 11 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
do we have any way to introspect from the kernel what cmdq type we have, so that we could have a cmdqv option with ON/OFF/AUTO value instead of duplication cmdq option for each vendor?
Looking at it again, I think we can, by calling
IOMMU_GET_HW_INFOwith all supported CMDQ types and check which one succeeds.then I think we should endeavour doing that. Regarding the ops, if you feel that reset/init can be managed through a simple add-on code I am fine with it. Maybe Cédric or Peter will have another opinion though ;-)
I like that on/off/auto depending on the IOMMU_GET_HW_INFO return.
Thanks
Nicolin
RE: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
Quoted 28 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 12 February 2026 17:23
To: Eric Auger <[email protected]>
Cc: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interfaceQuoted 1 line · Eric Auger
Quoted 1 line
Quoted 1 line
Quoted 1 line
The vCMDQ path does not replace the normal accelerated SMMU
init/reset.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Instead, it requires additional init/reset steps on top of the regular accel
flow.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
if (accel)
accel_init();if (accel && cmdqv)
cmdqv_init();
Quoted 7 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
So the vCMDQ logic is an extension of the accelerated mode, not a
fallback
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
or alternative implementation.
Yes. And I think it makes more sense by moving
cmdqv_init()intoaccel_init(), completely transparent to the common smmuv3 module maybe?
I have attempted that, IIRC. There is a catch there that the pci iommu_ops needs to be set earlier(before smu_common realize) and we can only call cmdqv_init() after the base SMMUv3 sysbus_init_mmio() and smmu_init_irq().
I need to take another if we can make that work or not.
Quoted 9 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
I think the later patches in the series make this clearer. Please let me know if you still feel the ops necessary.
do we have any way to introspect from the kernel what cmdq type we
have,
Quoted 1 line
Quoted 1 line
Quoted 1 line
so that we could have a cmdqv option with ON/OFF/AUTO value instead
of
Quoted 2 lines
Quoted 2 lines
Quoted 1 line
duplication cmdq option for each vendor?
Looking at it again, I think we can, by calling
IOMMU_GET_HW_INFOwith
Quoted 8 lines
all supported
Quoted 5 lines
Quoted 1 line
CMDQ types and check which one succeeds.
then I think we should endeavour doing that. Regarding the ops, if you feel that reset/init can be managed through a simple add-on code I am fine with it. Maybe Cédric or Peter will have another opinion though ;-)
I like that on/off/auto depending on the
IOMMU_GET_HW_INFOreturn.
Yeah. We need to do this.
Thanks,
Shameer
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer KolothumQuoted 2 lines · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).
Let's mention "NVIDIA", noting it's a non-standard extension.
Quoted 18 lines
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +0
-0
hw/arm/smmuv3-accel.h line 21
Quoted 6 lines
hw/arm/smmuv3-accel.h +3
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to +* support multiple VCMDQs with virtualization capabilities.
Ditto
hw/arm/smmuv3-accel.h line 30
Quoted 9 lines
hw/arm/smmuv3-accel.h +9
-0+* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { + bool (*init)(SMMUv3State *s, Error **errp); + bool (*alloc_viommu)(SMMUv3State *s, + HostIOMMUDeviceIOMMUFD *idev, + uint32_t *out_viommu_id, + Error **errp); + void (*free_viommu)(SMMUv3State *s);
I don't see much value of having alloc/free_viommu..
Type and data structure are the only difference from the standard viommu routine, and both are already in the uAPI header.
hw/arm/smmuv3-accel.h line 32
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s);
alloc_/free_veventq can be moved into alloc/free_viommu functions.
hw/arm/smmuv3-accel.h line 34
Quoted 2 lines
hw/arm/smmuv3-accel.h +2
-0+void (*reset)(SMMUv3State *s); +} SMMUv3AccelCmdqvOps;
Overall, an ops structure feels unnecessary to me. Maybe init and reset are somewhat plausible. But nobody else would reuse this ops structure that is defined exclusively for Tegra241CMDQV?
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49
Quoted 9 lines
hw/arm/smmuv3-accel.h +2
-0/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object. * Holds bypass and abort proxy HWPT IDs used for device attachment. @@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;
Having two pointers for the same extension feels redundant. Maybe merge them with:
typedef struct SMMUv3AccelCmdqv {
void *private; // points to Tegra241CMDQV struct iommu_viommu_tegra241_cmdqv viommu_data;
} SMMUv3AccelCmdqv;
...
SMMUv3AccelState {
...
SMMUv3AccelCmdqv *cmdqv;
};?
Nicolin
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer KolothumQuoted 45 lines · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +17
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to +* support multiple VCMDQs with virtualization capabilities. +* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { +bool (*init)(SMMUv3State *s,Error **errp); +bool (*alloc_viommu)(SMMUv3State *s, +HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id, +Error **errp); +void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s); +void (*reset)(SMMUv3State *s); +}SMMUv3AccelCmdqvOps ; +/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object. * Holds bypass and abort proxy HWPT IDs used for device attachment.
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49
Quoted 5 lines
hw/arm/smmuv3-accel.h +2
-0@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops; +void *cmdqv;
effectively looks duplucate, check !cmdqv_ops ?
Eric
Quoted 3 lines
hw/arm/smmuv3-accel.h +0
-0} SMMUv3AccelState; typedef struct SMMUS1Hwpt {
Re: [PATCH v2 05/24] hw/arm/smmuv3-accel: Introduce CMDQV ops interface
In reply to Shameer KolothumQuoted 45 lines · Shameer Kolothum
CMDQ-Virtualization (CMDQV) is a hardware extension to SMMUv3 that enables virtualization of multiple command queues (VCMDQs).
CMDQV support is a specialization of the IOMMUFD backed accelerated SMMUv3 path. Introduce an ops interface to factor CMDQV specific initialization and CMDQV vIOMMU/vEVENTQ allocation behavior out of the base implementation. The ops pointer and associated state are stored in the accelerated SMMUv3 state.
No functional change
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)hw/arm/smmuv3-accel.h +17
-0@@ -16,6 +16,23 @@#endif #include CONFIG_DEVICES +/* +* CMDQ-Virtualization (CMDQV) hardware support, extends the SMMUv3 to +* support multiple VCMDQs with virtualization capabilities. +* CMDQV specific behavior is factored behind this ops interface. +*/ +typedef struct SMMUv3AccelCmdqvOps { +bool (*init)(SMMUv3State *s,Error **errp); +bool (*alloc_viommu)(SMMUv3State *s, +HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id, +Error **errp); +void (*free_viommu)(SMMUv3State *s); +bool (*alloc_veventq)(SMMUv3State *s,Error **errp); +void (*free_veventq)(SMMUv3State *s); +void (*reset)(SMMUv3State *s); +}SMMUv3AccelCmdqvOps ; +/* * Represents an accelerated SMMU instance backed by an iommufd vIOMMU object. * Holds bypass and abort proxy HWPT IDs used for device attachment.
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 48
Quoted 4 lines
hw/arm/smmuv3-accel.h +1
-0@@ -28,6 +45,8 @@ typedef struct SMMUv3AccelState {uint32_t bypass_hwpt_id;uint32_t abort_hwpt_id; QLIST_HEAD(, SMMUv3AccelDevice) device_list; +const SMMUv3AccelCmdqvOps *cmdqv_ops;
in my original thought this was not AccelCmdqOps but AccelOps. This should be squashed with the patch that turns existing accel impl into those ops implementation.
hw/arm/smmuv3-accel.h typedef struct SMMUv3AccelState { · line 49
Quoted 1 line
hw/arm/smmuv3-accel.h +1
-0+void *cmdqv;
Sorry I misread the purpose of that field. This aims at storing cmdqv data. But until this is actually used, I think it is useless to introduce it.
Eric
Quoted 3 lines
hw/arm/smmuv3-accel.h +0
-0} SMMUv3AccelState; typedef struct SMMUS1Hwpt {
PATCH v2 6/24 hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
In reply to Shameer KolothumIntroduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV initialization, vIOMMU/vEVENTQ allocation and reset handling.
Functional CMDQV support is added in follow-up patches.
--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 23 ++++++++++++++++ 4 files changed, 89 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.h
hw/arm/Kconfig +5 -0
@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3
@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config ARM_SMMUV3 bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL bool
hw/arm/meson.build +1 -0
@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) +arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
hw/arm/tegra241-cmdqv.c new file +60 -0
60 lines not on this page · raw message
hw/arm/tegra241-cmdqv.h new file +23 -0
@@ -0,0 +1,23 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_TEGRA241_CMDQV_H +#define HW_TEGRA241_CMDQV_H + +#include CONFIG_DEVICES + +#ifdef CONFIG_TEGRA241_CMDQV +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void ); +#else +static inline const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void ) +{ +return NULL ; +} +#endif +#endif /* HW_TEGRA241_CMDQV_H */
Signature · 1 line
2.43.0
Re: [PATCH v2 06/24] hw/arm/tegra241-cmdqv: Add Tegra241 CMDQV ops backend stub
Hi Shameer,
Quoted 126 lines · Shameer Kolothum
Introduce a Tegra241 CMDQV backend that plugs into the SMMUv3 accelerated CMDQV ops interface.
This patch wires up the Tegra241 CMDQV backend and provides a stub implementation for CMDQV initialization, vIOMMU/vEVENTQ allocation and reset handling.
Functional CMDQV support is added in follow-up patches.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 23 ++++++++++++++++ 4 files changed, 89 insertions(+) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.hhw/arm/Kconfig +5
-0@@ -626,6 +626,10 @@ config FSL_IMX8MP_EVK depends on TCG select FSL_IMX8MP +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3@@ -633,6 +637,7 @@ config ARM_SMMUV3_ACCEL config ARM_SMMUV3 bool select ARM_SMMUV3_ACCEL if IOMMUFD + imply TEGRA241_CMDQV config FSL_IMX6UL boolhw/arm/meson.build +1
-0@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) +arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(hw/arm/tegra241-cmdqv.c new file +60
-0@@ -0,0 +1,60 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static void tegra241_cmdqv_free_veventq (SMMUv3State *s) +{ +} + +static bool tegra241_cmdqv_alloc_veventq (SMMUv3State *s,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static void tegra241_cmdqv_free_viommu (SMMUv3State *s) +{ +} + +static bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static void tegra241_cmdqv_reset (SMMUv3State *s) +{ +} + +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) +{ +error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); +return false; +} + +static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops_impl = { + .init = tegra241_cmdqv_init, + .alloc_viommu = tegra241_cmdqv_alloc_viommu, + .free_viommu = tegra241_cmdqv_free_viommu, + .alloc_veventq = tegra241_cmdqv_alloc_veventq, + .free_veventq = tegra241_cmdqv_free_veventq, + .reset = tegra241_cmdqv_reset, +}; + +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void ) +{ +return &tegra241_cmdqv_ops_impl; +}hw/arm/tegra241-cmdqv.h new file +0
-0
hw/arm/tegra241-cmdqv.h line 10
Quoted 10 lines
hw/arm/tegra241-cmdqv.h new file +10
-0@@ -0,0 +1,23 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_TEGRA241_CMDQV_H
nit: to match existing pattern this should be HW_ARM_TEGRA241_CMDQV_H
Quoted 13 lines
hw/arm/tegra241-cmdqv.h new file +13
-0+#define HW_TEGRA241_CMDQV_H + +#include CONFIG_DEVICES + +#ifdef CONFIG_TEGRA241_CMDQV +const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void ); +#else +static inline const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void ) +{ +return NULL ; +} +#endif +#endif /* HW_TEGRA241_CMDQV_H */
PATCH v2 7/24 hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
In reply to Shameer KolothumIntegrate CMDQV ops support into the accelerated SMMUv3 path. When CMDQV is enabled, the backend ops are used to initialize CMDQV state and to allocate the vIOMMU instance. The current implementation connects the Tegra241 CMDQV backend, but does not enable functional support yet.
No functional changes intended.
--- hw/arm/smmuv3-accel.c | 65 ++++++++++++++++++++++++++++++++++++----- hw/arm/smmuv3-accel.h | 5 ++++ hw/arm/smmuv3.c | 1 + include/hw/arm/smmuv3.h | 2 ++ 4 files changed, 66 insertions(+), 7 deletions(-)
hw/arm/smmuv3-accel.c +58 -7
@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem
@@ -520,6 +521,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) {SMMUv3AccelState *accel = s->s_accel; +const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;struct iommu_hwpt_arm_smmuv3 bypass_data = { .ste = {SMMU_STE_CFG_BYPASS |SMMU_STE_VALID ,0x0ULL }, };
@@ -530,12 +532,24 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,uint32_t viommu_id, hwpt_id;IOMMUFDViommu *viommu; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, -NULL ,0 , &viommu_id, errp)) { +if (cmdqv_ops && (!cmdqv_ops->alloc_viommu || !cmdqv_ops->alloc_veventq)) { +error_setg (errp,"CMDQV vIOMMU allocation not supported" );return false; } +if (cmdqv_ops) { +if (!cmdqv_ops->alloc_viommu (s, idev, &viommu_id, errp)) { +return false; + } + }else { +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , + s2_hwpt_id,NULL ,0 , &viommu_id, + errp)) { +return false; + } + } + viommu =g_new0 (IOMMUFDViommu,1 ); viommu->viommu_id = viommu_id; viommu->s2_hwpt_id = s2_hwpt_id;
@@ -565,13 +579,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, goto free_bypass_hwpt; } +if (cmdqv_ops && !cmdqv_ops->alloc_veventq (s, errp)) { + goto free_veventq; + } +/* Attach a HWPT based on SMMUv3 GBPA.ABORT value */ hwpt_id =smmuv3_accel_gbpa_hwpt (s, accel);if (!host_iommu_device_iommufd_attach_hwpt (idev, hwpt_id, errp)) { - goto free_veventq; + goto free_cmdqv_veventq; }return true; +free_cmdqv_veventq: +if (cmdqv_ops && cmdqv_ops->free_veventq) { + cmdqv_ops->free_veventq (s); + } free_veventq:smmuv3_accel_free_veventq (accel); free_bypass_hwpt:
@@ -579,7 +601,11 @@ free_bypass_hwpt: free_abort_hwpt:iommufd_backend_free_id (idev->iommufd, accel->abort_hwpt_id); free_viommu: -iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); +if (cmdqv_ops && cmdqv_ops->free_viommu) { + cmdqv_ops->free_viommu (s); + }else { +iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); + }g_free (viommu); accel->viommu =NULL ;return false;
@@ -865,8 +891,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)void smmuv3_accel_reset (SMMUv3State *s) { -/* Attach a HWPT based on GBPA reset value */ -smmuv3_accel_attach_gbpa_hwpt (s,NULL ); +SMMUv3AccelState *accel = s->s_accel; + +if (!accel) { +return ; + } +/* Attach a HWPT based on GBPA reset value */ +smmuv3_accel_attach_gbpa_hwpt (s,NULL ); + +if (accel->cmdqv_ops && accel->cmdqv_ops->reset) { + accel->cmdqv_ops->reset (s); + } }static void smmuv3_accel_as_init (SMMUv3State *s)
@@ -886,6 +921,22 @@ static void smmuv3_accel_as_init(SMMUv3State *s)address_space_init (shared_as_sysmem, &root,"smmuv3-accel-as-sysmem" ); } +bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp) +{ +SMMUv3AccelState *accel = s->s_accel; + +if (!accel || !s->tegra241_cmdqv) { +return true; + } + + accel->cmdqv_ops =tegra241_cmdqv_ops (); +if (!accel->cmdqv_ops || !accel->cmdqv_ops->init) { +error_setg (errp,"CMDQV support not available" ); +return false; + } +return accel->cmdqv_ops->init (s, errp); +} +void smmuv3_accel_init (SMMUv3State *s) {SMMUState *bs =ARM_SMMU (s);
hw/arm/smmuv3-accel.h +5 -0
@@ -74,6 +74,7 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,void smmuv3_accel_idr_override (SMMUv3State *s);bool smmuv3_accel_alloc_veventq (SMMUv3State *s,Error **errp);void smmuv3_accel_reset (SMMUv3State *s); +bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp);#else static inline void smmuv3_accel_init (SMMUv3State *s) {
@@ -110,6 +111,10 @@ static inline bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)static inline void smmuv3_accel_reset (SMMUv3State *s) { } +static bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp) +{ +return true; +}#endif #endif /* HW_ARM_SMMUV3_ACCEL_H */
hw/arm/smmuv3.c +1 -0
@@ -2034,6 +2034,7 @@ static void smmu_realize(DeviceState *d, Error **errp)smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +smmuv3_accel_cmdqv_init (s, errp); }static const VMStateDescription vmstate_smmuv3_queue = {
include/hw/arm/smmuv3.h +2 -0
@@ -73,6 +73,8 @@ struct SMMUv3State {bool ats;uint8_t oas;uint8_t ssidsize; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +bool tegra241_cmdqv; }; typedef enum {
Signature · 1 line
2.43.0
Re: [PATCH v2 07/24] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
Hi Shameer,
Quoted 46 lines · Shameer Kolothum
Integrate CMDQV ops support into the accelerated SMMUv3 path. When CMDQV is enabled, the backend ops are used to initialize CMDQV state and to allocate the vIOMMU instance. The current implementation connects the Tegra241 CMDQV backend, but does not enable functional support yet.
No functional changes intended.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.c | 65 ++++++++++++++++++++++++++++++++++++----- hw/arm/smmuv3-accel.h | 5 ++++ hw/arm/smmuv3.c | 1 + include/hw/arm/smmuv3.h | 2 ++ 4 files changed, 66 insertions(+), 7 deletions(-)hw/arm/smmuv3-accel.c +4
-3@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem @@ -520,6 +521,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) {SMMUv3AccelState *accel = s->s_accel; +const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;struct iommu_hwpt_arm_smmuv3 bypass_data = { .ste = {SMMU_STE_CFG_BYPASS |SMMU_STE_VALID ,0x0ULL }, };@@ -530,12 +532,24 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,uint32_t viommu_id, hwpt_id;IOMMUFDViommu *viommu; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, -NULL ,0 , &viommu_id, errp)) { +if (cmdqv_ops && (!cmdqv_ops->alloc_viommu || !cmdqv_ops->alloc_veventq)) { +error_setg (errp,"CMDQV vIOMMU allocation not supported" );return false; }
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 548
Quoted 10 lines
hw/arm/smmuv3-accel.c +9
-0+if (cmdqv_ops) { +if (!cmdqv_ops->alloc_viommu (s, idev, &viommu_id, errp)) { +return false; + } + }else { +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , + s2_hwpt_id,NULL ,0 , &viommu_id, + errp)) {
this is not add-on code but rather a switch. Depending on the cmdq type,
sizeof(cmdqv->cmdqv_data) varies. Wondering if ops does not makes sense here either.
Quoted 64 lines
hw/arm/smmuv3-accel.c +29
-4+return false; + } + } + viommu =g_new0 (IOMMUFDViommu,1 ); viommu->viommu_id = viommu_id; viommu->s2_hwpt_id = s2_hwpt_id;@@ -565,13 +579,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, goto free_bypass_hwpt; } +if (cmdqv_ops && !cmdqv_ops->alloc_veventq (s, errp)) { + goto free_veventq; + } +/* Attach a HWPT based on SMMUv3 GBPA.ABORT value */ hwpt_id =smmuv3_accel_gbpa_hwpt (s, accel);if (!host_iommu_device_iommufd_attach_hwpt (idev, hwpt_id, errp)) { - goto free_veventq; + goto free_cmdqv_veventq; }return true; +free_cmdqv_veventq: +if (cmdqv_ops && cmdqv_ops->free_veventq) { + cmdqv_ops->free_veventq (s); + } free_veventq:smmuv3_accel_free_veventq (accel); free_bypass_hwpt:@@ -579,7 +601,11 @@ free_bypass_hwpt: free_abort_hwpt:iommufd_backend_free_id (idev->iommufd, accel->abort_hwpt_id); free_viommu: -iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); +if (cmdqv_ops && cmdqv_ops->free_viommu) { + cmdqv_ops->free_viommu (s); + }else { +iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); + }g_free (viommu); accel->viommu =NULL ;return false;@@ -865,8 +891,17 @@ bool smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)void smmuv3_accel_reset (SMMUv3State *s) { -/* Attach a HWPT based on GBPA reset value */ -smmuv3_accel_attach_gbpa_hwpt (s,NULL ); +SMMUv3AccelState *accel = s->s_accel; + +if (!accel) { +return ; + } +/* Attach a HWPT based on GBPA reset value */ +smmuv3_accel_attach_gbpa_hwpt (s,NULL ); + +if (accel->cmdqv_ops && accel->cmdqv_ops->reset) { + accel->cmdqv_ops->reset (s); + } }static void smmuv3_accel_as_init (SMMUv3State *s)
hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State *s) · line 928
Quoted 8 lines
hw/arm/smmuv3-accel.c +5
-0@@ -886,6 +921,22 @@ static void smmuv3_accel_as_init(SMMUv3State *s)address_space_init (shared_as_sysmem, &root,"smmuv3-accel-as-sysmem" ); } +bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp) +{ +SMMUv3AccelState *accel = s->s_accel; + +if (!accel || !s->tegra241_cmdqv) {
isn't tegra241_cmdqv also only accelerated. Why do we need to have both checks?
Quoted 54 lines
hw/arm/smmuv3-accel.c +11
-0+return true; + } + + accel->cmdqv_ops =tegra241_cmdqv_ops (); +if (!accel->cmdqv_ops || !accel->cmdqv_ops->init) { +error_setg (errp,"CMDQV support not available" ); +return false; + } +return accel->cmdqv_ops->init (s, errp); +} +void smmuv3_accel_init (SMMUv3State *s) {SMMUState *bs =ARM_SMMU (s);hw/arm/smmuv3-accel.h +5
-0@@ -74,6 +74,7 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s, void *cmd, SMMUDevice *sdev,void smmuv3_accel_idr_override (SMMUv3State *s);bool smmuv3_accel_alloc_veventq (SMMUv3State *s,Error **errp);void smmuv3_accel_reset (SMMUv3State *s); +bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp);#else static inline void smmuv3_accel_init (SMMUv3State *s) {@@ -110,6 +111,10 @@ static inline bool smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)static inline void smmuv3_accel_reset (SMMUv3State *s) { } +static bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp) +{ +return true; +}#endif #endif /* HW_ARM_SMMUV3_ACCEL_H */ hw/arm/smmuv3.c +1
-0@@ -2034,6 +2034,7 @@ static void smmu_realize(DeviceState *d, Error **errp)smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +smmuv3_accel_cmdqv_init (s, errp); }static const VMStateDescription vmstate_smmuv3_queue = {include/hw/arm/smmuv3.h +0
-0
include/hw/arm/smmuv3.h struct SMMUv3State { · line 77
Quoted 5 lines
include/hw/arm/smmuv3.h +2
-0@@ -73,6 +73,8 @@ struct SMMUv3State {bool ats;uint8_t oas;uint8_t ssidsize; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +bool tegra241_cmdqv;
so I really advocate for auto detection and I think in that context I still think a selection of ops depending on the type of the cmdqv may be more future proof.
Also to me this should be within struct SMMUv3AccelState if cmdqv only works with accel. Or does emulation of cmdqv make sense?
Thanks
Eric
Quoted 3 lines
include/hw/arm/smmuv3.h +0
-0}; typedef enum {
RE: [PATCH v2 07/24] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycle
Quoted 84 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 17:38
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 07/24] hw/arm/smmuv3-accel: Wire CMDQV ops into accel lifecycleExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 1 line · Shameer Kolothum
Integrate CMDQV ops support into the accelerated SMMUv3 path. When
CMDQV
Quoted 9 lines
is enabled, the backend ops are used to initialize CMDQV state and to allocate the vIOMMU instance. The current implementation connects the Tegra241 CMDQV backend, but does not enable functional support yet.
No functional changes intended.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3-accel.c | 65++++++++++++++++++++++++++++++++++++-----
Quoted 10 lines
hw/arm/smmuv3-accel.h | 5 ++++ hw/arm/smmuv3.c | 1 + include/hw/arm/smmuv3.h | 2 ++ 4 files changed, 66 insertions(+), 7 deletions(-)hw/arm/smmuv3-accel.c +0
-0hw/arm/smmuv3-accel.c line 24
Quoted 7 lines
hw/arm/smmuv3-accel.c +1
-0@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and
shared_as_sysmemhw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,
Quoted 1 line
hw/arm/smmuv3-accel.c +0
-0@@ -520,6 +521,7 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,
Quoted 8 lines
hw/arm/smmuv3-accel.c +1
-0Error **errp) {SMMUv3AccelState *accel = s->s_accel; +const SMMUv3AccelCmdqvOps *cmdqv_ops = accel->cmdqv_ops;struct iommu_hwpt_arm_smmuv3 bypass_data = { .ste = {SMMU_STE_CFG_BYPASS |SMMU_STE_VALID ,0x0ULL }, };@@ -530,12 +532,24 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
Quoted 11 lines
hw/arm/smmuv3-accel.c +2
-3uint32_t viommu_id, hwpt_id;IOMMUFDViommu *viommu; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, -NULL ,0 , &viommu_id, errp)) { +if (cmdqv_ops && (!cmdqv_ops->alloc_viommu || !cmdqv_ops- alloc_veventq)) { +error_setg (errp,"CMDQV vIOMMU allocation not supported" );return false; }hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 548
Quoted 10 lines
hw/arm/smmuv3-accel.c +9
-0+if (cmdqv_ops) { +if (!cmdqv_ops->alloc_viommu (s, idev, &viommu_id, errp)) { +return false; + } + }else { +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , + s2_hwpt_id,NULL ,0 , &viommu_id, + errp)) {
Quoted 1 line
this is not add-on code but rather a switch. Depending on the cmdq type,
Yeah. This is not add-on but a switch.
Quoted 74 lines
sizeof(cmdqv->cmdqv_data)varies. Wondering if ops does not makes sense here either.hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,
Quoted 8 lines
hw/arm/smmuv3-accel.c +4
-0+return false; + } + } + viommu =g_new0 (IOMMUFDViommu,1 ); viommu->viommu_id = viommu_id;viommu->s2_hwpt_id = s2_hwpt_id;hw/arm/smmuv3-accel.c +0
-0@@ -565,13 +579,21 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/smmuv3-accel.c bool
Quoted 36 lines
hw/arm/smmuv3-accel.c +14
-2goto free_bypass_hwpt; } +if (cmdqv_ops && !cmdqv_ops->alloc_veventq (s, errp)) { + goto free_veventq; + } +/* Attach a HWPT based on SMMUv3 GBPA.ABORT value */ hwpt_id =smmuv3_accel_gbpa_hwpt (s, accel);if (!host_iommu_device_iommufd_attach_hwpt (idev, hwpt_id, errp)) { - goto free_veventq; + goto free_cmdqv_veventq; }return true; +free_cmdqv_veventq: +if (cmdqv_ops && cmdqv_ops->free_veventq) { + cmdqv_ops->free_veventq (s); + } free_veventq:smmuv3_accel_free_veventq (accel); free_bypass_hwpt:@@ -579,7 +601,11 @@ free_bypass_hwpt: free_abort_hwpt:iommufd_backend_free_id (idev->iommufd, accel->abort_hwpt_id); free_viommu: -iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); +if (cmdqv_ops && cmdqv_ops->free_viommu) { + cmdqv_ops->free_viommu (s); + }else { +iommufd_backend_free_id (idev->iommufd, viommu->viommu_id); + }g_free (viommu); accel->viommu =NULL ;return false;@@ -865,8 +891,17 @@ bool
smmuv3_accel_attach_gbpa_hwpt(SMMUv3State *s, Error **errp)hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State
Quoted 20 lines
hw/arm/smmuv3-accel.c +11
-2void smmuv3_accel_reset (SMMUv3State *s) { -/* Attach a HWPT based on GBPA reset value */ -smmuv3_accel_attach_gbpa_hwpt (s,NULL ); +SMMUv3AccelState *accel = s->s_accel; + +if (!accel) { +return ; + } +/* Attach a HWPT based on GBPA reset value */ +smmuv3_accel_attach_gbpa_hwpt (s,NULL ); + +if (accel->cmdqv_ops && accel->cmdqv_ops->reset) { + accel->cmdqv_ops->reset (s); + } }static void smmuv3_accel_as_init (SMMUv3State *s)@@ -886,6 +921,22 @@ static void smmuv3_accel_as_init(SMMUv3State*s)
hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State · line 921
Quoted 1 line
hw/arm/smmuv3-accel.c +0
-0address_space_init(shared_as_sysmem, &root, "smmuv3-accel-as-sysmem");
Quoted 2 lines
hw/arm/smmuv3-accel.c +0
-0}
Quoted 7 lines
hw/arm/smmuv3-accel.c static void smmuv3_accel_as_init(SMMUv3State *s) · line 928
Quoted 5 lines
hw/arm/smmuv3-accel.c +5
-0+bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp) +{ +SMMUv3AccelState *accel = s->s_accel; + +if (!accel || !s->tegra241_cmdqv) {isn't
tegra241_cmdqvalso only accelerated. Why do we need to have both checks?
True. accel check is not required.
Quoted 51 lines
hw/arm/smmuv3-accel.h bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s,
Quoted 19 lines
hw/arm/smmuv3-accel.c +11
-0+return true; + } + + accel->cmdqv_ops =tegra241_cmdqv_ops (); +if (!accel->cmdqv_ops || !accel->cmdqv_ops->init) { +error_setg (errp,"CMDQV support not available" ); +return false; + } +return accel->cmdqv_ops->init (s, errp); +} +void smmuv3_accel_init (SMMUv3State *s) {SMMUState *bs =ARM_SMMU (s);hw/arm/smmuv3-accel.h +0
-0@@ -74,6 +74,7 @@ bool smmuv3_accel_issue_inv_cmd(SMMUv3State *s,void *cmd, SMMUDevice *sdev,
hw/arm/smmuv3-accel.h static inline bool
Quoted 8 lines
hw/arm/smmuv3-accel.h +1
-0void smmuv3_accel_idr_override (SMMUv3State *s);bool smmuv3_accel_alloc_veventq (SMMUv3State *s,Error **errp);void smmuv3_accel_reset (SMMUv3State *s); +bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp);#else static inline void smmuv3_accel_init (SMMUv3State *s) {@@ -110,6 +111,10 @@ static inline bool
smmuv3_accel_alloc_veventq(SMMUv3State *s, Error **errp)hw/arm/smmuv3.c static void smmu_realize(DeviceState *d, Error
Quoted 15 lines
hw/arm/smmuv3-accel.h +4
-0static inline void smmuv3_accel_reset (SMMUv3State *s) { } +static bool smmuv3_accel_cmdqv_init (SMMUv3State *s,Error **errp) +{ +return true; +}#endif #endif /* HW_ARM_SMMUV3_ACCEL_H */ hw/arm/smmuv3.c +0
-0@@ -2034,6 +2034,7 @@ static void smmu_realize(DeviceState *d, Error**errp)
Quoted 6 lines
hw/arm/smmuv3.c +1
-0smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +smmuv3_accel_cmdqv_init (s, errp); }
Quoted 16 lines
include/hw/arm/smmuv3.h struct SMMUv3State { · line 77
Quoted 11 lines
hw/arm/smmuv3.c +0
-0static const VMStateDescription vmstate_smmuv3_queue = {include/hw/arm/smmuv3.h +2
-0@@ -73,6 +73,8 @@ struct SMMUv3State {bool ats;uint8_t oas;uint8_t ssidsize; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +bool tegra241_cmdqv;so I really advocate for auto detection and I think in that context I still think a selection of ops depending on the type of the cmdqv may be more future proof.
Also to me this should be within struct SMMUv3AccelState if cmdqv only works with accel. Or does emulation of cmdqv make sense?
Let me see if I can move all cmdqv related stuff to SMMUv3AccelState or not.
Emulation makes no sense as the whole purpose of this extension is to accelerate the performance.
Thanks,
Shameer
PATCH v2 8/24 hw/arm/tegra241-cmdqv: Implement CMDQV init and vIOMMU allocation
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add initial Tegra241 CMDQV support by wiring up initialization and IOMMUFD backed vIOMMU allocation. The CMDQV MMIO region and IRQ are registered, while register handling remains a stub and will be completed in subsequent patches.
--- hw/arm/tegra241-cmdqv.c | 55 ++++++++++++++++++++++++++++++++++++++--- hw/arm/tegra241-cmdqv.h | 19 ++++++++++++++ 2 files changed, 70 insertions(+), 4 deletions(-)
hw/arm/tegra241-cmdqv.c +51 -4
@@ -13,6 +13,16 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} +static void tegra241_cmdqv_free_veventq (SMMUv3State *s) { }
@@ -25,24 +35,61 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { +SMMUv3AccelState *accel = s->s_accel; +IOMMUFDViommu *viommu = accel->viommu; + +if (!viommu) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, viommu->viommu_id); }static bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV , + idev->hwpt_id, &cmdqv->cmdqv_data, +sizeof (cmdqv->cmdqv_data), out_viommu_id, + errp)) { +error_append_hint (errp,"Tegra241 CMDQV support unavailable" ); +return false; + } +return true; }static void tegra241_cmdqv_reset (SMMUv3State *s) { } +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +SMMUv3AccelState *accel = s->s_accel; +Tegra241CMDQV *cmdqv; + +if (!accel) { +error_setg (errp,"Tegra241 CMDQV requires SMMUv3 acceleration" ); +return false; + } + + cmdqv =g_new0 (Tegra241CMDQV,1 ); +memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->s_accel = accel; + accel->cmdqv = cmdqv; +return true; }static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops_impl = {
hw/arm/tegra241-cmdqv.h +19 -0
@@ -10,8 +10,27 @@#ifndef HW_TEGRA241_CMDQV_H #define HW_TEGRA241_CMDQV_H +#include "smmuv3-accel.h" #include CONFIG_DEVICES +/* +* Tegra241 CMDQV MMIO layout (64KB pages): +* +* 0x00000: Global CMDQV registers +* 0x10000: Global VCMDQ registers, page 0 +* 0x20000: Global VCMDQ registers, page 1 +* 0x30000: VINTF0 logical VCMDQ registers, page 0 +* 0x40000: VINTF0 logical VCMDQ registers, page 1 +*/ +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3AccelState *s_accel; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; +#ifdef CONFIG_TEGRA241_CMDQV const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void );#else
Signature · 1 line
2.43.0
Re: [PATCH v2 08/24] hw/arm/tegra241-cmdqv: Implement CMDQV init and vIOMMU allocation
Hi Shameer,
Quoted 6 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add initial Tegra241 CMDQV support by wiring up initialization and IOMMUFD backed vIOMMU allocation. The CMDQV MMIO region and IRQ are registered, while register handling remains a stub and will be completed in subsequent patches.
Explain what the MMIO region is made of (see below). Also describe what the interrupt is used for. what kind of interrupt is it? Also a work about the state (data)?
Quoted 100 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 55 ++++++++++++++++++++++++++++++++++++++--- hw/arm/tegra241-cmdqv.h | 19 ++++++++++++++ 2 files changed, 70 insertions(+), 4 deletions(-)hw/arm/tegra241-cmdqv.c +51
-4@@ -13,6 +13,16 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} +static void tegra241_cmdqv_free_veventq (SMMUv3State *s) { }@@ -25,24 +35,61 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)static void tegra241_cmdqv_free_viommu (SMMUv3State *s) { +SMMUv3AccelState *accel = s->s_accel; +IOMMUFDViommu *viommu = accel->viommu; + +if (!viommu) { +return ; + } +iommufd_backend_free_id (viommu->iommufd, viommu->viommu_id); }static bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +Tegra241CMDQV *cmdqv = s->s_accel->cmdqv; + +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV , + idev->hwpt_id, &cmdqv->cmdqv_data, +sizeof (cmdqv->cmdqv_data), out_viommu_id, + errp)) { +error_append_hint (errp,"Tegra241 CMDQV support unavailable" ); +return false; + } +return true; }static void tegra241_cmdqv_reset (SMMUv3State *s) { } +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; +static bool tegra241_cmdqv_init (SMMUv3State *s,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +SMMUv3AccelState *accel = s->s_accel; +Tegra241CMDQV *cmdqv; + +if (!accel) { +error_setg (errp,"Tegra241 CMDQV requires SMMUv3 acceleration" ); +return false; + } + + cmdqv =g_new0 (Tegra241CMDQV,1 ); +memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->s_accel = accel; + accel->cmdqv = cmdqv; +return true; }static const SMMUv3AccelCmdqvOps tegra241_cmdqv_ops_impl = {hw/arm/tegra241-cmdqv.h +0
-0
hw/arm/tegra241-cmdqv.h line 19
Quoted 10 lines
hw/arm/tegra241-cmdqv.h +5
-0@@ -10,8 +10,27 @@#ifndef HW_TEGRA241_CMDQV_H #define HW_TEGRA241_CMDQV_H +#include "smmuv3-accel.h" #include CONFIG_DEVICES +/* +* Tegra241 CMDQV MMIO layout (64KB pages): +* +* 0x00000: Global CMDQV registers
I would use the spec terminology, something like
TEGRA241_CMDQV_CFG TEGRA241_CMDQV_VCMDQ0TEGRA241_CMDQV_VCMDQ1 TEGRA241_CMDQV_VINTF0TEGRA241_CMDQV_VINTF1 Eric
Quoted 17 lines
hw/arm/tegra241-cmdqv.h +14
-0+* 0x10000: Global VCMDQ registers, page 0 +* 0x20000: Global VCMDQ registers, page 1 +* 0x30000: VINTF0 logical VCMDQ registers, page 0 +* 0x40000: VINTF0 logical VCMDQ registers, page 1 +*/ +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3AccelState *s_accel; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; +#ifdef CONFIG_TEGRA241_CMDQV const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void );#else
PATCH v2 9/24 hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages as follows:
0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0 (control/status) 0x20000: Global VCMDQ registers, Page1 (configuration) 0x30000: VINTF0 logical VCMDQ registers, Page0 (control/status) 0x40000: VINTF0 logical VCMDQ registers, Page1 (configuration)
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after HW_QUEUE allocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.
Prepare the VINTF0 Page0 mapping by mmapping the kernel provided region when CMDQV is accessed, while keeping all CMDQV MMIO accesses trapped in QEMU. The mapping is not yet exposed to the guest address space. A subsequent patch will install the RAM backed MMIO subregion for VINTF page0 once HW_QUEUE allocation completes.
--- hw/arm/tegra241-cmdqv.c | 38 ++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 3 +++ 2 files changed, 41 insertions(+)
hw/arm/tegra241-cmdqv.c +38 -0
@@ -13,14 +13,52 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (!viommu) { +return true; + } + +g_assert (!cmdqv->vintf_page0); +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VINTF_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) { +return false; + } + +return true; +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vintf_page0) { +if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) { +error_report_err (local_err); + local_err =NULL ; + } + } +return 0 ; }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vintf_page0) { +if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) { +error_report_err (local_err); + local_err =NULL ; + } + } }static void tegra241_cmdqv_free_veventq (SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +3 -0
@@ -29,8 +29,11 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0; } Tegra241CMDQV; +#define VINTF_REG_PAGE_SIZE 0x10000 +#ifdef CONFIG_TEGRA241_CMDQV const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void );#else
Signature · 1 line
2.43.0
Re: [PATCH v2 09/24] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Hi Shameer,
Quoted 11 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV extends SMMUv3 with support for virtual command queues (VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is split into 64KB pages as follows:
0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0 (control/status)0x20000: Global VCMDQ registers, Page1 (configuration)0x30000: VINTF0 logical VCMDQ registers, Page0 (control/status)0x40000: VINTF0 logical VCMDQ registers, Page1 (configuration)
this shall rather be introduced along with the whole mmio region. In this patch you focus on rather VINTF0 page0
Quoted 5 lines
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
Maybe you shall simply name it VINTF
Quoted 11 lines
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.Prepare the VINTF0 Page0 mapping by mmapping the kernel provided region when CMDQV is accessed, while keeping all CMDQV MMIO accesses trapped in QEMU. The mapping is not yet exposed to the guest address space. A subsequent patch will install the RAM backed MMIO subregion for VINTF page0 once
HW_QUEUEallocation completes.
So we are likely to mmap a page whose content is undefined until HW_QUEUE alloc, is that correct? In that case, why don't we do that only after HW_QUEUE alloc? Please can you clarify?
Quoted 19 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 38 ++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 3 +++ 2 files changed, 41 insertions(+)hw/arm/tegra241-cmdqv.c +3
-0@@ -13,14 +13,52 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
hw/arm/tegra241-cmdqv.c line 28
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ +if (!viommu) { +return true; + } + +g_assert (!cmdqv->vintf_page0); +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VINTF_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) {
So I would rather directly implement iommufd_backend_viommu_mmap() here.
hw/arm/tegra241-cmdqv.c line 38
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8
-0+return false; + } + +return true; +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;
init is not needed since you don't test it.
hw/arm/tegra241-cmdqv.c line 43
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5
-0+ +if (!cmdqv->vintf_page0) { +if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) { +error_report_err (local_err); + local_err =NULL ;
not needed either
hw/arm/tegra241-cmdqv.c line 54
Quoted 11 lines
hw/arm/tegra241-cmdqv.c +5
-0+ } + } +return 0 ; }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;
same here
Quoted 15 lines
hw/arm/tegra241-cmdqv.c +7
-0+ +if (!cmdqv->vintf_page0) { +if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) { +error_report_err (local_err); + local_err =NULL ; + } + } }static void tegra241_cmdqv_free_veventq (SMMUv3State *s)hw/arm/tegra241-cmdqv.h +0
-0
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35
Quoted 7 lines
hw/arm/tegra241-cmdqv.h +2
-0@@ -29,8 +29,11 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0; } Tegra241CMDQV; +#define VINTF_REG_PAGE_SIZE 0x10000
can't we rename that into VINTF_SIZE. REG does not mean much. Before you talked about Control/Status vs Configuration. Besides, I am not sure having a define brings much in that case.
Thanks
Eric
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +1
-0+#ifdef CONFIG_TEGRA241_CMDQV const SMMUv3AccelCmdqvOps *tegra241_cmdqv_ops (void );#else
RE: [PATCH v2 09/24] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQV
Quoted 30 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 13 February 2026 10:39
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 09/24] hw/arm/tegra241-cmdqv: mmap VINTF Page0 for CMDQVExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 3 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV extends SMMUv3 with support for virtual command
queues
Quoted 1 line
(VCMDQs) exposed via a CMDQV MMIO region. The CMDQV MMIO space is
split
Quoted 2 lines
into 64KB pages as follows:
Quoted 7 lines
Quoted 5 lines
0x00000: Global CMDQV registers
0x10000: Global VCMDQ registers, Page0 (control/status)0x20000: Global VCMDQ registers, Page1 (configuration)0x30000: VINTF0 logical VCMDQ registers, Page0 (control/status)0x40000: VINTF0 logical VCMDQ registers, Page1 (configuration)this shall rather be introduced along with the whole mmio region. In this patch you focus on rather VINTF0 page0
Ok.
Quoted 18 lines
Quoted 5 lines
Global VCMDQ pages provide a VM wide view of all VCMDQs, while the VINTF pages expose a logical view local to a given VINTF. Although real hardware may support multiple VINTFs, the kernel currently exposes a single VINTF per VM.
Maybe you shall simply name it VINTF
Quoted 11 lines
The kernel provides an mmap offset for the VINTF Page0 region during vIOMMU allocation. However, the logical-to-physical association between VCMDQs and a VINTF is only established after
HW_QUEUEallocation. Prior to that, the mapped Page0 does not back any real VCMDQ state.Prepare the VINTF0 Page0 mapping by mmapping the kernel provided region when CMDQV is accessed, while keeping all CMDQV MMIO accesses trapped in QEMU. The mapping is not yet exposed to the guest address space. A subsequent patch will install the RAM backed MMIO subregion for VINTF page0 once
HW_QUEUEallocation completes.
Quoted 3 lines
So we are likely to mmap a page whose content is undefined until
HW_QUEUEalloc, is that correct? In that case, why don't we do that only afterHW_QUEUEalloc? Please can you clarify?
Correct. Though we mmap it here, as mentioned in commit msg, it is not accessed until we do HW_QUEUE alloc(patch #16). Could delay this as well I suppose.
Quoted 19 lines
Quoted 5 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 38++++++++++++++++++++++++++++++++++++++
hw/arm/tegra241-cmdqv.c line 16
Quoted 12 lines
hw/arm/tegra241-cmdqv.h | 3 +++ 2 files changed, 41 insertions(+)hw/arm/tegra241-cmdqv.c +1
-0@@ -13,14 +13,52 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV*cmdqv, Error **errp)
Quoted 15 lines
hw/arm/tegra241-cmdqv.c line 28
Quoted 13 lines
hw/arm/tegra241-cmdqv.c +12
-0+{ +IOMMUFDViommu *viommu = cmdqv->s_accel->viommu; + +if (!viommu) { +return true; + } + +g_assert (!cmdqv->vintf_page0); +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu- viommu_id, +VINTF_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vintf_page0, errp)) {So I would rather directly implement
iommufd_backend_viommu_mmap()here.
I have explained the rationale for making this a common helper in the thread where we introduce it. Please let me know if that reasoning makes sense or not.
Quoted 52 lines
hw/arm/tegra241-cmdqv.c
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +6
-0+return false; + } + +return true; +} + static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset,unsigned size)
hw/arm/tegra241-cmdqv.c line 38
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +2
-0{ +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;init is not needed since you don't test it.
hw/arm/tegra241-cmdqv.c line 43
Quoted 5 lines
hw/arm/tegra241-cmdqv.c +5
-0+ +if (!cmdqv->vintf_page0) { +if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) { +error_report_err (local_err); + local_err =NULL ;not needed either
hw/arm/tegra241-cmdqv.c
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +3
-0+ } + } +return 0 ; } static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_tvalue,
hw/arm/tegra241-cmdqv.c line 54
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +2
-0unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ;same here
Quoted 21 lines
hw/arm/tegra241-cmdqv.c +7
-0+ +if (!cmdqv->vintf_page0) { +if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) { +error_report_err (local_err); + local_err =NULL ; + } + } }static void
tegra241_cmdqv_free_veventq(SMMUv3State *s)hw/arm/tegra241-cmdqv.h +1
-0@@ -29,8 +29,11 @@ typedef struct Tegra241CMDQV {SMMUv3AccelState *s_accel;MemoryRegion mmio_cmdqv;qemu_irq irq; +void *vintf_page0; } Tegra241CMDQV;
Quoted 4 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+#define VINTF_REG_PAGE_SIZE 0x10000can't we rename that into
VINTF_SIZE. REG does not mean much. Before you talked about Control/Status vs Configuration. Besides, I am not sure having a define brings much in that case.
Ok.
Thanks,
Shameer
PATCH v2 10/24 hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV defines a set of global control and status registers used to configure virtual command queue allocation and interrupt behavior.
Add read/write emulation for the global CMDQV register page (offset 0x00000), backed by a simple register cache. This includes CONFIG, PARAM, STATUS, VI error and interrupt maps, CMDQ allocation map and the VINTF0 related registers defined in the global CMDQV register space.
--- hw/arm/tegra241-cmdqv.c | 102 +++++++++++++++++++++++++++++++++++++++- hw/arm/tegra241-cmdqv.h | 86 +++++++++++++++++++++++++++++++++ 2 files changed, 187 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +101 -1
@@ -8,6 +8,7 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h"
@@ -32,6 +33,25 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)return true; } +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
@@ -44,7 +64,57 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } -return 0 ; +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return 0 ; + } + +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} + +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value) +{ +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } } static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
@@ -59,6 +129,36 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, local_err =NULL ; } } + +if (offset >=TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } }static void tegra241_cmdqv_free_veventq (SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +86 -0
100 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
Quoted 6 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +6
-0+static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG :
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 51
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ;
I wonder if we'd still need to implement VINTF0_SID_MATCH/REPLACE registers as it's a part of HW spec. They don't need to associate with anything but register cache alone to support MMIO.
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value) +{ +switch (offset) {
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 111
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + }
To emulate the HW perfectly, VINTF_EN = 0 might have to disable the VCMDQ feature, i.e. deallocate HW_QUEUEs and munmap page0.
Otherwise, guest could continue using the vcmdq "hardware" after disabling it, which doesn't make sense.
This means that HW_QUEUE allocations would have to be delayed, until VINTF_EN=1 and corresponding VCMDQ_EN=1.
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 136
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +8
-0@@ -59,6 +129,36 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + }
We'd need to do something about this as well. VINTF/VCMDQ should not be functional when CMDQV_EN=0.
Thanks
Nicolin
RE: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
Quoted 33 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 23:19
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registersQuoted 1 line · Shameer Kolothum
+static
uint64_ttegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,hwaddr offset)
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9
-0+{ + int i; + + switch (offset) { + case A_VINTF0_CONFIG: + return cmdqv->vintf_config; + case A_VINTF0_STATUS: + return cmdqv->vintf_status; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ...
A_VINTF0_LVCMDQ_ERR_MAP_3:hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"
Quoted 7 lines
PRIx64 "\n",
Review
Quoted 2 lines
(quoted diff) +2
-0+ __func__, offset); + return 0;I wonder if we'd still need to implement VINTF0_SID_MATCH/REPLACE registers as it's a part of HW spec. They don't need to associate with anything but register cache alone to support MMIO.
Ok. But I was thinking Guest should not be allowed to configure it as it belongs to hypervisor, right?
Quoted 20 lines
Quoted 2 lines
+static void tegra241_cmdqv_write_vintf(Tegra241CMDQV *cmdqv, hwaddr
offset,
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 111
Quoted 13 lines
hw/arm/tegra241-cmdqv.c +13
-0+ uint64_t value) +{ +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + }To emulate the HW perfectly,
VINTF_EN= 0 might have to disable the VCMDQ feature, i.e. deallocateHW_QUEUEsand munmap page0.
Quoted 5 lines
Otherwise, guest could continue using the vcmdq "hardware" after disabling it, which doesn't make sense.
This means that
HW_QUEUEallocations would have to be delayed, until VINTF_EN=1 and corresponding VCMDQ_EN=1.
Ok. Will add that.
Quoted 11 lines
Review static void tegra241_cmdqv_write(void *opaque,
Quoted 1 line
(quoted diff) +0
-0@@ -59,6 +129,36 @@ static void tegra241_cmdqv_write(void *opaque,hwaddr offset,
uint64_tvalue,hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, · line 147
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + }
Quoted 2 lines
We'd need to do something about this as well. VINTF/VCMDQ should not be functional when CMDQV_EN=0.
Ok.
Thanks,
Shameer
Re: [PATCH v2 10/24] hw/arm/tegra241-cmdqv: Emulate global CMDQV registers
Quoted 21 lines · Shameer Kolothum Thodi
Quoted 21 lines · Shameer Kolothum
Quoted 1 line
+static
uint64_ttegra241_cmdqv_read_vintf(Tegra241CMDQV *cmdqv,hwaddr offset)
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9
-0+{ + int i; + + switch (offset) { + case A_VINTF0_CONFIG: + return cmdqv->vintf_config; + case A_VINTF0_STATUS: + return cmdqv->vintf_status; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ...
A_VINTF0_LVCMDQ_ERR_MAP_3:hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : + qemu_log_mask(LOG_UNIMP, "%s unhandled read access at 0x%"PRIx64 "\n",
Quoted 2 lines
(quoted diff) +2
-0+ __func__, offset); + return 0;
Quoted 6 lines
Quoted 3 lines
I wonder if we'd still need to implement VINTF0_SID_MATCH/REPLACE registers as it's a part of HW spec. They don't need to associate with anything but register cache alone to support MMIO.
Ok. But I was thinking Guest should not be allowed to configure it as it belongs to hypervisor, right?
That's correct for the physical registers. But I think guest could still MMIO the VM registers? So, what we'd need is just two arrays in the register cache list to support read/write.
Nicolin
PATCH v2 11/24 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register reads
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV exposes per-VCMDQ register windows through two MMIO views:
-Global VCMDQ registers at 0x10000/0x20000 -VINTF VCMDQ (VI_VCMDQ) registers at 0x30000/0x40000
The VI_VCMDQ register ranges are an alias of the global VCMDQ registers and are only meaningful when a VCMDQ is mapped to a VINTF via ioctl IOMMU_HW_QUEUE_ALLOC.
Add read side emulation for both global VCMDQ and VI_VCMDQ register ranges. MMIO accesses are decoded to extract the VCMDQ instance index and normalized to a VCMDQ0_* register offset, allowing a single helper to service all VCMDQ instances.
VI_VCMDQ accesses are translated to their equivalent global VCMDQ offsets and reuse the same decoding path. All VCMDQ reads are currently served from cached register state.
--- hw/arm/tegra241-cmdqv.c | 75 +++++++++++++++ hw/arm/tegra241-cmdqv.h | 197 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 272 insertions(+)
hw/arm/tegra241-cmdqv.c +75 -0
@@ -33,6 +33,46 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)return true; } +/* +* Read a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +* +* All VCMDQ accesses are currently trapped. Use cached registers +*/ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0, +int index) +{ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +return cmdqv->vcmdq_cons_indx[index]; +case A_VCMDQ0_PROD_INDX : +return cmdqv->vcmdq_prod_indx[index]; +case A_VCMDQ0_CONFIG : +return cmdqv->vcmdq_config[index]; +case A_VCMDQ0_STATUS : +return cmdqv->vcmdq_status[index]; +case A_VCMDQ0_GERROR : +return cmdqv->vcmdq_gerror[index]; +case A_VCMDQ0_GERRORN : +return cmdqv->vcmdq_gerrorn[index]; +case A_VCMDQ0_BASE_L : +return cmdqv->vcmdq_base[index]; +case A_VCMDQ0_BASE_H : +return cmdqv->vcmdq_base[index] >>32 ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +return cmdqv->vcmdq_cons_indx_base[index]; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : +return cmdqv->vcmdq_cons_indx_base[index] >>32 ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset0); +return 0 ; + } +} +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) {int i;
@@ -56,6 +96,7 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;Error *local_err =NULL ; +int index;if (!cmdqv->vintf_page0) {if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) {
@@ -88,6 +129,40 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN: +/* +* VI_VCMDQ registers (VINTF logical view) have the same per-VCMDQ +* layout as the global VCMDQ registers, but are based at 0x30000 +* instead of 0x10000. +* +* Subtract 0x20000 to translate a VI_VCMDQ offset into the equivalent +* global VCMDQ offset, then fall through to reuse the common VCMDQ +* decoding logic below. +*/ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: +/* +* Decode a per-VCMDQ register access. +* +* VCMDQs are 128 identical instances, each occupying a 0x80-byte window +* starting at 0x10000. The MMIO offset is decoded to extract the VCMDQ +* index, and the per-instance offset is normalized to a VCMDQ0_* +* register (@offset0 = offset - 0x80 * index). +* +* A single helper then services all VCMDQs, with @index selecting the +* instance. +*/ + index = (offset -0x10000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index); + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* Same as A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN case above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* Same as A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN case above */ + index = (offset -0x20000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index);default :qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , __func__, offset);
hw/arm/tegra241-cmdqv.h +197 -0
210 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v2 12/24 hw/arm/tegra241-cmdqv: Emulate global and VINTF VCMDQ register writes
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
This is the write side counterpart of the VCMDQ read emulation.
Add write handling for global VCMDQ and VI_VCMDQ register windows. Per-VCMDQ accesses are decoded into a VCMDQ index and normalized to VCMDQ0_* offsets, reusing the same layout assumptions as the read path.
VI_VCMDQ registers are treated as a logical alias of the global VCMDQ registers and share the same decoding logic.
Writes are backed by cached register state only; no hardware queue mapping is performed yet.
--- hw/arm/tegra241-cmdqv.c | 84 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+)
hw/arm/tegra241-cmdqv.c +84 -0
@@ -170,6 +170,71 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +/* +* Write a VCMDQ register using VCMDQ0_* offsets. +* +* The caller normalizes the MMIO offset such that @offset0 always refers +* to a VCMDQ0_* register, while @index selects the VCMDQ instance. +*/ +static void +tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index, +uint64_t value,unsigned size) +{ + +switch (offset0) { +case A_VCMDQ0_CONS_INDX : + cmdqv->vcmdq_cons_indx[index] = value; +return ; +case A_VCMDQ0_PROD_INDX : + cmdqv->vcmdq_prod_indx[index] = (uint32_t )value; +return ; +case A_VCMDQ0_CONFIG : +if (value &R_VCMDQ0_CONFIG_CMDQ_EN_MASK ) { + cmdqv->vcmdq_status[index] |=R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + }else { + cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + } + cmdqv->vcmdq_config[index] = (uint32_t )value; +return ; +case A_VCMDQ0_GERRORN : + cmdqv->vcmdq_gerrorn[index] = (uint32_t )value; +return ; +case A_VCMDQ0_BASE_L : +if (size ==8 ) { + cmdqv->vcmdq_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_BASE_H : + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +if (size ==8 ) { + cmdqv->vcmdq_cons_indx_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset0); +return ; + } +} +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset,uint64_t value) {
@@ -197,6 +262,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;Error *local_err =NULL ; +int index;if (!cmdqv->vintf_page0) {if (!tegra241_cmdqv_mmap_vintf_page0 (cmdqv, &local_err)) {
@@ -230,6 +296,24 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3:tegra241_cmdqv_write_vintf (cmdqv, offset, value);break ; + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN: +/* Same decoding as read() case: See comments above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: + index = (offset -0x10000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size); +break ; + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* Same decoding as read() case: See comments above */ + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: + index = (offset -0x20000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size); +break ;default :qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset);
Signature · 1 line
2.43.0
PATCH v2 13/24 system/physmem: Add address_space_is_ram() helper
In reply to Shameer KolothumIntroduce address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.
--- include/system/memory.h | 10 ++++++++++ system/physmem.c | 11 +++++++++++ 2 files changed, 21 insertions(+)
include/system/memory.h +10 -0
@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,*/ bool address_space_is_io (AddressSpace *as,hwaddr addr); +/** +* address_space_is_ram: check whether a guest physical address whithin +* an address space is RAM. +* +* @as: #AddressSpace to be accessed +* @addr: address within that address space +*/ + +bool address_space_is_ram (AddressSpace *as,hwaddr addr); +/* address_space_map: map a physical memory region into a host virtual address * * May map a subset of the requested range, given by and returned in @plen.
system/physmem.c +11 -0
@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)return !(memory_region_is_ram (mr) ||memory_region_is_romd (mr)); } +bool address_space_is_ram (AddressSpace *as,hwaddr addr) +{ +MemoryRegion *mr; + +RCU_READ_LOCK_GUARD (); + mr =address_space_translate (as, addr, &addr,NULL , false, +MEMTXATTRS_UNSPECIFIED ); + +return memory_region_is_ram (mr); +} + static hwaddr flatview_extend_translation(FlatView *fv, hwaddr addr, hwaddr target_len,
Signature · 1 line
2.43.0
Re: [PATCH v2 13/24] system/physmem: Add address_space_is_ram() helper
Quoted 5 lines · Shameer Kolothum
Introduce
address_space_is_ram(), a helper to determine whether a guest physical address resolves to a RAM-backed MemoryRegion within an AddressSpace.Signed-off-by Shameer Kolothum <[email protected]>
Eric
Quoted 48 lines
--- include/system/memory.h | 10 ++++++++++ system/physmem.c | 11 +++++++++++ 2 files changed, 21 insertions(+)include/system/memory.h +10
-0@@ -2900,6 +2900,16 @@ bool address_space_access_valid(AddressSpace *as, hwaddr addr, hwaddr len,*/ bool address_space_is_io (AddressSpace *as,hwaddr addr); +/** +* address_space_is_ram: check whether a guest physical address whithin +* an address space is RAM. +* +* @as: #AddressSpace to be accessed +* @addr: address within that address space +*/ + +bool address_space_is_ram (AddressSpace *as,hwaddr addr); +/* address_space_map: map a physical memory region into a host virtual address * * May map a subset of the requested range, given by and returned in @plen. system/physmem.c +11
-0@@ -3646,6 +3646,17 @@ bool address_space_is_io(AddressSpace *as, hwaddr addr)return !(memory_region_is_ram (mr) ||memory_region_is_romd (mr)); } +bool address_space_is_ram (AddressSpace *as,hwaddr addr) +{ +MemoryRegion *mr; + +RCU_READ_LOCK_GUARD (); + mr =address_space_translate (as, addr, &addr,NULL , false, +MEMTXATTRS_UNSPECIFIED ); + +return memory_region_is_ram (mr); +} + static hwaddr flatview_extend_translation(FlatView *fv, hwaddr addr, hwaddr target_len,
PATCH v2 14/24 hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add support for allocating IOMMUFD hardware queues when VCMDQ base registers are programmed by the guest.
When a VCMDQ BASE register is written with a valid RAM-backed address, allocate a corresponding IOMMUFD hardware queue for the CMDQV device. Any previously allocated queue for the VCMDQ is freed before reallocation. Writes with invalid addresses (e.g. during reset) are ignored.
--- hw/arm/tegra241-cmdqv.c | 51 ++++++++++++++++++++++++++++++++++++++--- hw/arm/tegra241-cmdqv.h | 1 + 2 files changed, 49 insertions(+), 3 deletions(-)
hw/arm/tegra241-cmdqv.c +48 -3
@@ -170,6 +170,45 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static bool tegra241_cmdqv_setup_vcmdq (Tegra241CMDQV *cmdqv,int index, +Error **errp) +{ +SMMUv3AccelState *accel = cmdqv->s_accel; +uint64_t base_mask = (uint64_t )R_VCMDQ0_BASE_L_ADDR_MASK | + (uint64_t )R_VCMDQ0_BASE_H_ADDR_MASK <<32 ; +uint64_t addr = cmdqv->vcmdq_base[index] & base_mask; +uint64_t log2 = cmdqv->vcmdq_base[index] &R_VCMDQ0_BASE_L_LOG2SIZE_MASK ; +uint64_t size =1ULL << (log2 +4 ); +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; +IOMMUFDViommu *viommu = accel->viommu; +IOMMUFDHWqueue *hw_queue; +uint32_t hw_queue_id; + +/* Ignore any invalid address. This may come as part of reset etc */ +if (!address_space_is_ram (&address_space_memory, addr)) { +return true; + } + +if (vcmdq) { +iommufd_backend_free_id (viommu->iommufd, vcmdq->hw_queue_id); + cmdqv->vcmdq[index] =NULL ; +g_free (vcmdq); + } + +if (!iommufd_backend_alloc_hw_queue (viommu->iommufd, viommu->viommu_id, +IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV , + index, addr, size, &hw_queue_id, + errp)) { +return false; + } + hw_queue =g_new (IOMMUFDHWqueue,1 ); + hw_queue->hw_queue_id = hw_queue_id; + hw_queue->viommu = viommu; + cmdqv->vcmdq[index] = hw_queue; + +return true; +} +/* * Write a VCMDQ register using VCMDQ0_* offsets. *
@@ -178,7 +217,7 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)*/ static void tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index, -uint64_t value,unsigned size) +uint64_t value,unsigned size,Error **errp) {switch (offset0) {
@@ -207,11 +246,13 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | (value &0xffffffffULL ); } +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp);return ;case A_VCMDQ0_BASE_H : cmdqv->vcmdq_base[index] = (cmdqv->vcmdq_base[index] &0xffffffffULL ) | ((uint64_t )value <<32 ); +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp);return ;case A_VCMDQ0_CONS_INDX_BASE_DRAM_L :if (size ==8 ) {
@@ -303,7 +344,7 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: index = (offset -0x10000 ) /0x80 ;tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, - size); + size, &local_err);break ; case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H:/* Same decoding as read() case: See comments above */
@@ -312,12 +353,16 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: index = (offset -0x20000 ) /0x80 ;tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, - size); + size, &local_err);break ;default :qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , __func__, offset); } + +if (local_err) { +error_report_err (local_err); + } }static void tegra241_cmdqv_free_veventq (SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +1 -0
@@ -31,6 +31,7 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_cmdqv;qemu_irq irq;void *vintf_page0; +IOMMUFDHWqueue *vcmdq[128 ];/* Register Cache */ uint32_t config;
Signature · 1 line
2.43.0
Re: [PATCH v2 14/24] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
Quoted 7 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add support for allocating IOMMUFD hardware queues when VCMDQ base registers are programmed by the guest.
When a VCMDQ BASE register is written with a valid RAM-backed address, allocate a corresponding IOMMUFD hardware queue for the CMDQV device.
Let's mention that this is because QEMU doesn't trap VCMDQ_EN bit, which is a part of VCMDQ_CONFIG register accessed via the mmap'd page0.
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7
-0+static bool tegra241_cmdqv_setup_vcmdq (Tegra241CMDQV *cmdqv,int index, +Error **errp) +{ +SMMUv3AccelState *accel = cmdqv->s_accel; +uint64_t base_mask = (uint64_t )R_VCMDQ0_BASE_L_ADDR_MASK | + (uint64_t )R_VCMDQ0_BASE_H_ADDR_MASK <<32 ; +uint64_t addr = cmdqv->vcmdq_base[index] & base_mask;
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+uint64_t log2 = cmdqv->vcmdq_base[index] &R_VCMDQ0_BASE_L_LOG2SIZE_MASK ; +uint64_t size =1ULL << (log2 +4 ); +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; +IOMMUFDViommu *viommu = accel->viommu; +IOMMUFDHWqueue *hw_queue; +uint32_t hw_queue_id; + +/* Ignore any invalid address. This may come as part of reset etc */ +if (!address_space_is_ram (&address_space_memory, addr)) { +return true;
We should probably check addr and size - 1 + addr?
Nicolin
RE: [PATCH v2 14/24] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programming
Quoted 49 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 23:46
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 14/24] hw/arm/tegra241-cmdqv: Allocate HW VCMDQs on base register programmingQuoted 7 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add support for allocating IOMMUFD hardware queues when VCMDQ base registers are programmed by the guest.
When a VCMDQ BASE register is written with a valid RAM-backed address, allocate a corresponding IOMMUFD hardware queue for the CMDQV device.
Let's mention that this is because QEMU doesn't trap
VCMDQ_ENbit, which is a part ofVCMDQ_CONFIGregister accessed via the mmap'd page0.Quoted 1 line
+static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int
index,
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7
-0+ Error **errp) +{ +SMMUv3AccelState *accel = cmdqv->s_accel; +uint64_t base_mask = (uint64_t )R_VCMDQ0_BASE_L_ADDR_MASK | + (uint64_t )R_VCMDQ0_BASE_H_ADDR_MASK <<32 ; +uint64_t addr = cmdqv->vcmdq_base[index] & base_mask; + uint64_t log2 = cmdqv->vcmdq_base[index] &
R_VCMDQ0_BASE_L_LOG2SIZE_MASK;hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 189
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9
-0+uint64_t size =1ULL << (log2 +4 ); +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; +IOMMUFDViommu *viommu = accel->viommu; +IOMMUFDHWqueue *hw_queue; +uint32_t hw_queue_id; + +/* Ignore any invalid address. This may come as part of reset etc */ +if (!address_space_is_ram (&address_space_memory, addr)) { +return true;
Quoted 1 line
We should probably check addr and size - 1 + addr?
Right. Will add.
Thanks,
Shameer
PATCH v2 15/24 hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
VINTF page0 is guest accessible and backed by host memory, so it can be mapped directly instead of being trapped and emulated. Map the VINTF0 page0 MMIO region lazily on first vCMDQ setup.
--- hw/arm/tegra241-cmdqv.c | 21 +++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 2 ++ 2 files changed, 23 insertions(+)
hw/arm/tegra241-cmdqv.c +21 -0
@@ -170,6 +170,26 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +static void tegra241_cmdqv_map_vintf_page0 (Tegra241CMDQV *cmdqv) +{ +char *name; + +if (cmdqv->vintf_page0_mapped) { +return ; + } + + name =g_strdup_printf ("%s vintf-page0" , +memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page0, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,VINTF_REG_PAGE_SIZE , + cmdqv->vintf_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + &cmdqv->mmio_vintf_page0,1 ); +g_free (name); + cmdqv->vintf_page0_mapped = true; +} +static bool tegra241_cmdqv_setup_vcmdq (Tegra241CMDQV *cmdqv,int index,Error **errp) {
@@ -206,6 +226,7 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, hw_queue->viommu = viommu; cmdqv->vcmdq[index] = hw_queue; +tegra241_cmdqv_map_vintf_page0 (cmdqv);return true; }
hw/arm/tegra241-cmdqv.h +2 -0
@@ -31,6 +31,8 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_cmdqv;qemu_irq irq;void *vintf_page0; +MemoryRegion mmio_vintf_page0; +bool vintf_page0_mapped;IOMMUFDHWqueue *vcmdq[128 ];/* Register Cache */
Signature · 1 line
2.43.0
Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 34
Quoted 9 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.h +1
-0@@ -31,6 +31,8 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_cmdqv;qemu_irq irq;void *vintf_page0; +MemoryRegion mmio_vintf_page0;
mr_vintf_page0
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+bool vintf_page0_mapped;
Why do we need another boolean while having vintf_page0 pointer already?
Nicolin
RE: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region
Quoted 29 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 09 February 2026 23:50
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO regionhw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 34
Quoted 9 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.h +1
-0@@ -31,6 +31,8 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_cmdqv;qemu_irq irq;void *vintf_page0; +MemoryRegion mmio_vintf_page0;
mr_vintf_page0
Quoted 4 lines
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 35
Quoted 1 line
hw/arm/tegra241-cmdqv.h +1
-0+bool vintf_page0_mapped;Why do we need another boolean while having
vintf_page0pointer already?
My original idea for this flag was not to track the vintf_page0 mmap itself, but to indicate whether we have allocated a HW queue, since that is when vintf page0 is mapped to the guest.
Looking at this again, the flag may not be necessary. I will double check the flow and remove it.
Shameer
Thanks,
Shameer
Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region
In reply to Shameer KolothumQuoted 6 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +6
-0+static void tegra241_cmdqv_map_vintf_page0 (Tegra241CMDQV *cmdqv) +{ +char *name; + +if (cmdqv->vintf_page0_mapped) { +return ;
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 188
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ } + + name =g_strdup_printf ("%s vintf-page0" , +memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page0, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,VINTF_REG_PAGE_SIZE , + cmdqv->vintf_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + &cmdqv->mmio_vintf_page0,1 );
I wonder if you see this:
[ 0.853123] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for vcmdq0 [ 0.854739] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for vcmdq1
iommufd_backend_alloc_vdev iommufd=142 dev_id=1 viommu_id=4 virt_id=0x200 vdev_id=8 (0) iommufd_cdev_attach_ioas_hwpt [iommufd=142] Successfully attached device 0009:01:00.0 (143) to id=6 smmuv3_accel_install_ste vSID=0x200 ste type=bypass hwpt_id=0x6iommufd_viommu_alloc_eventq iommufd=142 viommu_id=4 type=1 veventq_id=9 veventq_fd=153 (0) iommufd_backend_alloc_hw_queue iommufd=142 viommu_id=4 queue_type=1 index=0 addr=0xe2000000 size=0x800000 queue_id=10 (0) vfio_listener_region_add_ram region_add [ram] 0xc0b0000 - 0xc0bffff [0xfffff7f00000] iommufd_backend_map_dma iommufd=142 ioas=2 iova=0xc0b0000 size=0x10000 addr=0xfffff7f00000 readonly=0 (-1)
qemu-system-aarch64: warning: IOMMU_IOAS_MAP failed: Bad address, PCI BAR?
[Switching to Thread 0xffffdd7aee80 (LWP 52426)]
Thread 38 "qemu-system-aar" hit Breakpoint 2, vfio_container_region_add (bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at ../hw/vfio/listener.c:620
620 error_setg(&err, "vfio_container_dma_map(%p, 0x%"HWADDR_PRIx", "
(gdb) bt
#0 vfio_container_region_add (bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at ../hw/vfio/listener.c:620 #1 0x0000aaaaab8a9290 in vfio_listener_region_add (listener=0xaaaaaf1220d0, section=0xffffdd7ade00) at ../hw/vfio/listener.c:492 #2 0x0000aaaaab3035f8 in address_space_update_topology_pass (as=0xaaaaaefa6de0, old_view=0xffffec5171b0, new_view=0xffffc8001410, adding=true) at ../system/memory.c:1030 #3 0x0000aaaaab303974 in address_space_set_flatview (as=0xaaaaaefa6de0) at ../system/memory.c:1106 #4 0x0000aaaaab303bd4 in memory_region_transaction_commit () at ../system/memory.c:1158 #5 0x0000aaaaab3084e0 in memory_region_update_container_subregions (subregion=0xaaaaaefa7b00) at ../system/memory.c:2653 #6 0x0000aaaaab3085a0 in memory_region_add_subregion_common (mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00) at ../system/memory.c:2668 #7 0x0000aaaaab308644 in memory_region_add_subregion_overlap (mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00, priority=1) at ../system/memory.c:2685 #8 0x0000aaaaab570f68 in tegra241_cmdqv_map_vintf_page0 (cmdqv=0xaaaaaefa79c0) at ../hw/arm/tegra241-cmdqv.c:275 #9 0x0000aaaaab571188 in tegra241_cmdqv_setup_vcmdq (cmdqv=0xaaaaaefa79c0, index=0, errp=0xffffdd7ae0e8) at ../hw/arm/tegra241-cmdqv.c:317
#10 0x0000aaaaab571550 in tegra241_cmdqv_write_vcmdq (cmdqv=0xaaaaaefa79c0, offset0=131072, index=0, value=3791650835, size=4, errp=0xffffdd7ae0e8) at ../hw/arm/tegra241-cmdqv.c:378 #11 0x0000aaaaab571a60 in tegra241_cmdqv_write (opaque=0xaaaaaefa79c0, offset=131072, value=3791650835, size=4) at ../hw/arm/tegra241-cmdqv.c:484
Nicolin
RE: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO region
Quoted 40 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 13 February 2026 23:07
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 15/24] hw/arm/tegra241-cmdqv: map VINTF page0 MMIO regionQuoted 5 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +5
-0+static void tegra241_cmdqv_map_vintf_page0 (Tegra241CMDQV *cmdqv) +{ +char *name; + +if (cmdqv->vintf_page0_mapped) {hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size)
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+return ; + } + + name =g_strdup_printf ("%s vintf-page0" , +memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page0, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,VINTF_REG_PAGE_SIZE , + cmdqv->vintf_page0); + memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,
0x30000,Quoted 1 line
+
&cmdqv->mmio_vintf_page0, 1);I wonder if you see this:
[ 0.853123] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries for vcmdq0 [ 0.854739] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 524288 entries
Quoted 16 lines
for vcmdq1
iommufd_backend_alloc_vdeviommufd=142 dev_id=1 viommu_id=4 virt_id=0x200 vdev_id=8 (0)iommufd_cdev_attach_ioas_hwpt[iommufd=142] Successfully attached device 0009:01:00.0 (143) to id=6smmuv3_accel_install_stevSID=0x200 ste type=bypass hwpt_id=0x6iommufd_viommu_alloc_eventqiommufd=142 viommu_id=4 type=1 veventq_id=9 veventq_fd=153 (0)iommufd_backend_alloc_hw_queueiommufd=142 viommu_id=4 queue_type=1 index=0 addr=0xe2000000 size=0x800000 queue_id=10 (0)vfio_listener_region_add_ramregion_add[ram]0xc0b0000-0xc0bffff[0xfffff7f00000]iommufd_backend_map_dmaiommufd=142 ioas=2 iova=0xc0b0000 size=0x10000 addr=0xfffff7f00000 readonly=0 (-1)
qemu-system-aarch64: warning:IOMMU_IOAS_MAPfailed: Bad address, PCI BAR?
Yes. Eric has reported this earlier here:
https://lore.kernel.org/qemu-devel/CH3PR12MB7548771111FF5AC450CB8721AB8EA@CH3PR12MB7548.namprd12.prod.outlook.com/
IIUC, we are using memory_region_init_ram_device_ptr() to avoid trapping accesses to this MMIO space. However, this marks the region as RAM, and when it is added via memory_region_add_subregion_overlap(), the VFIO listener attempts to DMA MAP it. But, the QEMU/kernel does not provide
dma-buf support for this register space.
This region represents VINTF page0 registers and is accessed by the CPU only. My understanding is, there is no expected device DMA or P2P use case targeting it, so an IOMMU_IOAS_MAP for this region should not be required and can be ignored. Please let me know if that is not the case.
Thanks,
Shameer
Quoted 40 lines
[Switching to Thread
0xffffdd7aee80(LWP 52426)]Thread 38 "qemu-system-aar" hit Breakpoint 2,
vfio_container_region_add(bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at../hw/vfio/listener.c:620
620 error_setg(&err, "vfio_container_dma_map(%p,
0x%"HWADDR_PRIx", "
(gdb) bt
#0vfio_container_region_add(bcontainer=0xaaaaaf1220a0, section=0xffffdd7ade00, cpr_remap=false) at../hw/vfio/listener.c:620#10x0000aaaaab8a9290invfio_listener_region_add(listener=0xaaaaaf1220d0, section=0xffffdd7ade00) at../hw/vfio/listener.c:492#20x0000aaaaab3035f8inaddress_space_update_topology_pass(as=0xaaaaaefa6de0, old_view=0xffffec5171b0, new_view=0xffffc8001410, adding=true) at../system/memory.c:1030
#30x0000aaaaab303974inaddress_space_set_flatview(as=0xaaaaaefa6de0) at../system/memory.c:1106
#40x0000aaaaab303bd4inmemory_region_transaction_commit() at../system/memory.c:1158
#50x0000aaaaab3084e0inmemory_region_update_container_subregions(subregion=0xaaaaaefa7b00) at../system/memory.c:2653
#60x0000aaaaab3085a0inmemory_region_add_subregion_common(mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00) at../system/memory.c:2668
#70x0000aaaaab308644inmemory_region_add_subregion_overlap(mr=0xaaaaaefa79e0, offset=196608, subregion=0xaaaaaefa7b00, priority=1) at../system/memory.c:2685
#80x0000aaaaab570f68integra241_cmdqv_map_vintf_page0(cmdqv=0xaaaaaefa79c0) at../hw/arm/tegra241-cmdqv.c:275
#90x0000aaaaab571188integra241_cmdqv_setup_vcmdq(cmdqv=0xaaaaaefa79c0, index=0, errp=0xffffdd7ae0e8) at../hw/arm/tegra241-cmdqv.c:317
#100x0000aaaaab571550integra241_cmdqv_write_vcmdq(cmdqv=0xaaaaaefa79c0, offset0=131072, index=0, value=3791650835, size=4, errp=0xffffdd7ae0e8) at../hw/arm/tegra241-cmdqv.c:378#110x0000aaaaab571a60integra241_cmdqv_write(opaque=0xaaaaaefa79c0, offset=131072, value=3791650835, size=4) at../hw/arm/tegra241-cmdqv.c:484Nicolin
PATCH v2 16/24 hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers
In reply to Shameer KolothumWhen a VCMDQ is mapped and VINTF page0 is available, read and write VCMDQ registers directly via the VINTF page0 backing instead of using cached register values.
--- hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +42 -2
@@ -38,12 +38,32 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)* * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. -* -* All VCMDQ accesses are currently trapped. Use cached registers */ static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset0,int index) { + +/* +* If this VCMDQ is mapped and VINTF page0 is available, read directly +* from the VINTF page0 backing. Otherwise, fall back to cached state. +*/ +if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) { +uint64_t off = (index *0x80 ) + (offset0 -0x10000 ); +uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off); + +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_STATUS : +case A_VCMDQ0_GERROR : +case A_VCMDQ0_GERRORN : +return *ptr; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX :return cmdqv->vcmdq_cons_indx[index];
@@ -241,6 +261,26 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, uint64_t value, unsigned size, Error **errp) { +/* +* If this VCMDQ is mapped and VINTF page0 is available, write directly +* to the VINTF page0 backing. Otherwise, update cached state. +*/ +if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) { +uint64_t off = (index *0x80 ) + (offset0 -0x10000 ); +uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off); + +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_GERRORN : + *ptr = (uint32_t )value; +return ; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX : cmdqv->vcmdq_cons_indx[index] = value;
Signature · 1 line
2.43.0
Re: [PATCH v2 16/24] hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers
Hi Shameer,
Quoted 3 lines · Shameer Kolothum
When a VCMDQ is mapped and VINTF page0 is available, read and write VCMDQ registers directly via the VINTF page0 backing instead of using cached register values.
At this stage of the reading I do not really understand the dance between mmap and not mmapped. Please could you elaborate?
Quoted 11 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 44 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 42 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.c +0
-0
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 42
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +0
-2@@ -38,12 +38,32 @@ static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)* * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. -* -* All VCMDQ accesses are currently trapped. Use cached registers */ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0,
s/offset0/offset?
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 52
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8
-0int index) { + +/* +* If this VCMDQ is mapped and VINTF page0 is available, read directly +* from the VINTF page0 backing. Otherwise, fall back to cached state. +*/ +if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) { +uint64_t off = (index *0x80 ) + (offset0 -0x10000 ); +uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);
nit: you could introduce a static helper that return ptr from cmdqv, index, offset0 or returns NULL if the VINTF is not mapped. resuable in write().
Quoted 34 lines
hw/arm/tegra241-cmdqv.c +27
-0+ +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_STATUS : +case A_VCMDQ0_GERROR : +case A_VCMDQ0_GERRORN : +return *ptr; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX :return cmdqv->vcmdq_cons_indx[index];@@ -241,6 +261,26 @@ tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, uint64_t value, unsigned size, Error **errp) { +/* +* If this VCMDQ is mapped and VINTF page0 is available, write directly +* to the VINTF page0 backing. Otherwise, update cached state. +*/ +if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) { +uint64_t off = (index *0x80 ) + (offset0 -0x10000 ); +uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off); + +switch (offset0) { +case A_VCMDQ0_CONS_INDX : +case A_VCMDQ0_PROD_INDX : +case A_VCMDQ0_CONFIG : +case A_VCMDQ0_GERRORN :
hw/arm/tegra241-cmdqv.c tegra241_cmdqv_write_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset0, int index, · line 286
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +7
-0+ *ptr = (uint32_t )value; +return ; +default : +break ; + } + } +switch (offset0) {case A_VCMDQ0_CONS_INDX : cmdqv->vcmdq_cons_indx[index] = value;
Thanks
Eric
RE: [PATCH v2 16/24] hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registers
Quoted 21 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 13 February 2026 10:33
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 16/24] hw/arm/tegra241-cmdqv: Use VINTF page0 for mapped VCMDQ registersExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 7 lines
Quoted 3 lines · Shameer Kolothum
When a VCMDQ is mapped and VINTF page0 is available, read and write VCMDQ registers directly via the VINTF page0 backing instead of using cached register values.
At this stage of the reading I do not really understand the dance between mmap and not mmapped. Please could you elaborate?
That’s fair. It is a bit ambiguous, and I will improve it next time.
This patch is basically trying to address what Nicolin explained and suggested here:
https://lore.kernel.org/qemu-devel/aXkTzRM695oTWFaD@Asurada-Nvidia/
In short:
a) if vcmdq is indexed to a mapped global vcmdq, read/write mmio_vintf_page0.
b) if vcmdq is indexed to an unmapped global vcmdq, read/write the value in the register array cached by QEMU.
Quoted 20 lines
Quoted 3 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 44+++++++++++++++++++++++++++++++++++++++--
hw/arm/tegra241-cmdqv.c static bool
Quoted 7 lines
1 file changed, 42 insertions(+), 2 deletions(-)hw/arm/tegra241-cmdqv.c +0
-0@@ -38,12 +38,32 @@ static bool
tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp)
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +0
-2* * The caller normalizes the MMIO offset such that @offset0 always refers * to a VCMDQ0_* register, while @index selects the VCMDQ instance. -* -* All VCMDQ accesses are currently trapped. Use cached registers */ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv,
Quoted 2 lines
hwaddr offset0,
s/offset0/offset?
Hmm.. I did this because in previous review there was some confusion w.r.t this. There was a suggestion to differentiate this normalized VCMDQ0_* offset from the top level MMIO offset used in tegra241_cmdqv_read()/write().
https://lore.kernel.org/qemu-devel/aXe8QgtdyBmXm5iS@Asurada-Nvidia/
Quoted 10 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_mmap_vintf_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 52
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +8
-0int index) { + +/* +* If this VCMDQ is mapped and VINTF page0 is available, read directly +* from the VINTF page0 backing. Otherwise, fall back to cached state. +*/ +if (cmdqv->vcmdq[index] && cmdqv->vintf_page0_mapped) { +uint64_t off = (index *0x80 ) + (offset0 -0x10000 ); +uint32_t *ptr = (uint32_t *)(cmdqv->vintf_page0 + off);
Quoted 3 lines
nit: you could introduce a static helper that return ptr from cmdqv, index, offset0 or returns NULL if the VINTF is not mapped. resuable in
write().
Ok.
Thanks,
Shameer
PATCH v2 17/24 hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
In reply to Shameer KolothumAllocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.
--- hw/arm/tegra241-cmdqv.c | 39 +++++++++++++++++++++++++++++++++++++-- hw/arm/tegra241-cmdqv.h | 1 + 2 files changed, 38 insertions(+), 2 deletions(-)
hw/arm/tegra241-cmdqv.c +37 -2
@@ -428,12 +428,47 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,static void tegra241_cmdqv_free_veventq (SMMUv3State *s) { +SMMUv3AccelState *accel = s->s_accel; +Tegra241CMDQV *cmdqv = accel->cmdqv; +IOMMUFDVeventq *veventq = cmdqv->veventq; + +if (!veventq) { +return ; + } +close (veventq->veventq_fd); +iommufd_backend_free_id (accel->viommu->iommufd, veventq->veventq_id); +g_free (veventq); + cmdqv->veventq =NULL ; }static bool tegra241_cmdqv_alloc_veventq (SMMUv3State *s,Error **errp) { -error_setg (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); -return false; +SMMUv3AccelState *accel = s->s_accel; +IOMMUFDViommu *viommu = accel->viommu; +Tegra241CMDQV *cmdqv = accel->cmdqv; +IOMMUFDVeventq *veventq; +uint32_t veventq_id; +uint32_t veventq_fd; + +if (cmdqv->veventq) { +return true; + } + +if (!iommufd_backend_alloc_veventq (viommu->iommufd, viommu->viommu_id, +IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV , +1 <<16 , &veventq_id, &veventq_fd, + errp)) { +error_append_hint (errp,"Tegra241 CMDQV: failed to alloc veventq" ); +return false; + } + + veventq =g_new (IOMMUFDVeventq,1 ); + veventq->veventq_id = veventq_id; + veventq->veventq_fd = veventq_fd; + veventq->viommu = viommu; + cmdqv->veventq = veventq; + +return true; }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +1 -0
@@ -34,6 +34,7 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_vintf_page0;bool vintf_page0_mapped;IOMMUFDHWqueue *vcmdq[128 ]; +IOMMUFDVeventq *veventq;/* Register Cache */ uint32_t config;
Signature · 1 line
2.43.0
Re: [PATCH v2 17/24] hw/arm/tegra241-cmdqv: Add vEVENTQ allocation and free
Quoted 4 lines · Shameer Kolothum
Allocate a CMDQV specific vEVENTQ via IOMMUFD, and add the corresponding teardown path to free the vEVENTQ during cleanup.
Signed-off-by Shameer Kolothum <[email protected]>
PATCH v2 18/24 hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
In reply to Shameer KolothumInstall an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.
The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().
--- hw/arm/tegra241-cmdqv.c | 83 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 2 + hw/arm/trace-events | 3 ++ 3 files changed, 88 insertions(+)
hw/arm/tegra241-cmdqv.c +83 -0
@@ -8,12 +8,76 @@*/ #include "qemu/osdep.h" +#include "qemu/error-report.h" #include "qemu/log.h" +#include "trace.h" #include "hw/arm/smmuv3.h" +#include "hw/core/irq.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static void tegra241_cmdqv_event_read (void *opaque) +{ +Tegra241CMDQV *cmdqv = opaque; +struct { +struct iommufd_vevent_header hdr; +struct iommu_vevent_tegra241_cmdqv vevent; + } buf; +uint32_t last_seq = cmdqv->last_event_seq; +ssize_t bytes; + + bytes =read (cmdqv->veventq->veventq_fd, &buf,sizeof (buf)); +if (bytes <=0 ) { +if (errno ==EAGAIN || errno ==EINTR ) { +return ; + } +error_report_once ("Tegra241 CMDQV: vEVENTQ: read failed (%m)" ); +return ; + } + +if (bytes ==sizeof (buf.hdr) && + (buf.hdr.flags &IOMMU_VEVENTQ_FLAG_LOST_EVENTS )) { +error_report_once ("Tegra241 CMDQV: vEVENTQ has lost events" ); +return ; + } + +if (bytes <sizeof (buf)) { +error_report_once ("Tegra241 `CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)" , + bytes,sizeof (buf)); + cmdqv->event_start = false; +return ; + } + +/* Check sequence in hdr for lost events if any */ +if (cmdqv->event_start && (buf.hdr.sequence - last_seq !=1 )) { +error_report_once ("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)" , + buf.hdr.sequence - last_seq -1 ); + } + +if (buf.vevent.lvcmdq_err_map[0 ] || buf.vevent.lvcmdq_err_map[1 ]) { + cmdqv->vintf_cmdq_err_map[0 ] = + buf.vevent.lvcmdq_err_map[0 ] &0xffffffff ; + cmdqv->vintf_cmdq_err_map[1 ] = + (buf.vevent.lvcmdq_err_map[0 ] >>32 ) &0xffffffff ; + cmdqv->vintf_cmdq_err_map[2 ] = + buf.vevent.lvcmdq_err_map[1 ] &0xffffffff ; + cmdqv->vintf_cmdq_err_map[3 ] = + (buf.vevent.lvcmdq_err_map[1 ] >>32 ) &0xffffffff ; +for (int i =0 ; i <4 ; i++) { + cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i]; + } + cmdqv->vi_err_map[0 ] |=0x1 ; +qemu_irq_pulse (cmdqv->irq); +trace_tegra241_cmdqv_err_map ( + cmdqv->vintf_cmdq_err_map[3 ], cmdqv->vintf_cmdq_err_map[2 ], + cmdqv->vintf_cmdq_err_map[1 ], cmdqv->vintf_cmdq_err_map[0 ]); + } + + cmdqv->last_event_seq = buf.hdr.sequence; + cmdqv->event_start = true; +} +static bool tegra241_cmdqv_mmap_vintf_page0 (Tegra241CMDQV *cmdqv,Error **errp) {IOMMUFDViommu *viommu = cmdqv->s_accel->viommu;
@@ -435,6 +499,7 @@ static void tegra241_cmdqv_free_veventq(SMMUv3State *s)if (!veventq) {return ; } +qemu_set_fd_handler (veventq->veventq_fd,NULL ,NULL ,NULL );close (veventq->veventq_fd);iommufd_backend_free_id (accel->viommu->iommufd, veventq->veventq_id);g_free (veventq);
@@ -449,6 +514,7 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)IOMMUFDVeventq *veventq;uint32_t veventq_id;uint32_t veventq_fd; +int flags;if (cmdqv->veventq) {return true;
@@ -462,13 +528,30 @@ static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp)return false; } + flags =fcntl (veventq_fd,F_GETFL ); +if (flags <0 ) { +error_setg (errp,"Failed to get flags for vEVENTQ fd" ); + goto free_veventq; + } +if (fcntl (veventq_fd,F_SETFL ,O_NONBLOCK | flags) <0 ) { +error_setg (errp,"Failed to set O_NONBLOCK on vEVENTQ fd" ); + goto free_veventq; + } + veventq =g_new (IOMMUFDVeventq,1 ); veventq->veventq_id = veventq_id; veventq->veventq_fd = veventq_fd; veventq->viommu = viommu; cmdqv->veventq = veventq; +/* Set up event handler for veventq fd */ +qemu_set_fd_handler (veventq_fd, tegra241_cmdqv_event_read,NULL , s);return true; + +free_veventq: +close (veventq_fd); +iommufd_backend_free_id (viommu->iommufd, veventq_id); +return false; }static void tegra241_cmdqv_free_viommu (SMMUv3State *s)
hw/arm/tegra241-cmdqv.h +2 -0
@@ -35,6 +35,8 @@ typedef struct Tegra241CMDQV {bool vintf_page0_mapped;IOMMUFDHWqueue *vcmdq[128 ];IOMMUFDVeventq *veventq; +uint32_t last_event_seq; +bool event_start;/* Register Cache */ uint32_t config;
hw/arm/trace-events +3 -0
@@ -72,6 +72,9 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x" +# tegra241-cmdqv +tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X" + # strongarm.c strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d" strongarm_ssp_read_underrun(void) "SSP rx underrun"
Signature · 1 line
2.43.0
Re: [PATCH v2 18/24] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
Quoted 2 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +2
-0+static void tegra241_cmdqv_event_read (void *opaque) +{
[...]
Quoted 17 lines
hw/arm/tegra241-cmdqv.c +17
-0+ bytes =read (cmdqv->veventq->veventq_fd, &buf,sizeof (buf)); +if (bytes <=0 ) { +if (errno ==EAGAIN || errno ==EINTR ) { +return ; + } +error_report_once ("Tegra241 CMDQV: vEVENTQ: read failed (%m)" ); +return ; + } + +if (bytes ==sizeof (buf.hdr) && + (buf.hdr.flags &IOMMU_VEVENTQ_FLAG_LOST_EVENTS )) { +error_report_once ("Tegra241 CMDQV: vEVENTQ has lost events" ); +return ; + } + +if (bytes <sizeof (buf)) { + error_report_once("Tegra241 `CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)",
hw/arm/tegra241-cmdqv.c line 56
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ bytes, sizeof(buf)); + cmdqv->event_start = false; +return ; + } + +/* Check sequence in hdr for lost events if any */ +if (cmdqv->event_start && (buf.hdr.sequence - last_seq !=1 )) { +error_report_once ("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)" , + buf.hdr.sequence - last_seq -1 ); + }
It shares the exact same routine with smmuv3_accel_event_read(). Can we put these into a common helper?
I also suggested in the veventq series that event_start/last_seq could be moved into the common structure.
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_alloc_veventq(SMMUv3State *s, Error **errp) · line 548
Quoted 8 lines
veventq = g_new(IOMMUFDVeventq, 1); veventq->veventq_id = veventq_id; veventq->veventq_fd = veventq_fd; veventq->viommu = viommu; cmdqv->veventq = veventq;hw/arm/tegra241-cmdqv.c +2
-0+/* Set up event handler for veventq fd */ +qemu_set_fd_handler (veventq_fd, tegra241_cmdqv_event_read,NULL , s);
Should pass in "cmdqv" instead of "s".
Nicolin
RE: [PATCH v2 18/24] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
Quoted 46 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 13 February 2026 23:29
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 18/24] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errorsQuoted 2 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +2
-0+static void tegra241_cmdqv_event_read (void *opaque) +{[...]
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +7
-0+ bytes =read (cmdqv->veventq->veventq_fd, &buf,sizeof (buf)); +if (bytes <=0 ) { +if (errno ==EAGAIN || errno ==EINTR ) { +return ; + } +error_report_once ("Tegra241 CMDQV: vEVENTQ: read failed (%m)" ); +return ;hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ } + +if (bytes ==sizeof (buf.hdr) && + (buf.hdr.flags &IOMMU_VEVENTQ_FLAG_LOST_EVENTS )) { +error_report_once ("Tegra241 CMDQV: vEVENTQ has lost events" ); +return ; + } + +if (bytes <sizeof (buf)) { + error_report_once("Tegra241 `CMDQV: vEVENTQ: incomplete read(%zd/%zd bytes)",
hw/arm/tegra241-cmdqv.c
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+ bytes, sizeof(buf)); + cmdqv->event_start = false; +return ; + } + +/* Check sequence in hdr for lost events if any */ +if (cmdqv->event_start && (buf.hdr.sequence - last_seq !=1 )) { + error_report_once("Tegra241 CMDQV: vEVENTQ: detected lost %u
Quoted 6 lines
event(s)",
Review
Quoted 2 lines
(quoted diff) +2
-0+ buf.hdr.sequence - last_seq - 1); + }It shares the exact same routine with
smmuv3_accel_event_read(). Can we put these into a common helper?
Yes. I think you had the same comment in RFC as well. I have noted that as a TODO in the cover letter for this. Will do.
Quoted 2 lines
I also suggested in the veventq series that event_start/last_seq could be moved into the common structure.
Yes. I see that.
Quoted 11 lines
Review
Quoted 8 lines
veventq = g_new(IOMMUFDVeventq, 1); veventq->veventq_id = veventq_id; veventq->veventq_fd = veventq_fd; veventq->viommu = viommu; cmdqv->veventq = veventq;(quoted diff) +2
-0+ /* Set up event handler for veventq fd */ + qemu_set_fd_handler(veventq_fd, tegra241_cmdqv_event_read, NULL,s);
Should pass in "cmdqv" instead of "s".
Ah..Right.
Thanks,
Shameer
PATCH v2 19/24 hw/arm/tegra241-cmdqv: Add reset handler
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.
CMDQV gets initialized early during guest boot, hence the handler verifies that at least one cold-plugged device is attached to the associated vIOMMU before proceeding. This is required to retrieve host CMDQV info and to validate it against the QEMU implementation support.
--- hw/arm/smmuv3-accel.c | 6 ++- hw/arm/tegra241-cmdqv.c | 107 ++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 6 +++ hw/arm/trace-events | 1 + 4 files changed, 119 insertions(+), 1 deletion(-)
hw/arm/smmuv3-accel.c +5 -1
@@ -707,7 +707,11 @@ static void smmuv3_accel_unset_iommu_device(PCIBus *bus, void *opaque,QLIST_REMOVE (accel_dev, next);trace_smmuv3_accel_unset_iommu_device (devfn, idev->devid); -if (QLIST_EMPTY (&accel->device_list)) { +/* +* Keep the vIOMMU alive when CMDQV is present, as the vIOMMU to host +* SMMUv3 association cannot be changed via device hot-plug. +*/ +if (QLIST_EMPTY (&accel->device_list) && !accel->cmdqv) {smmuv3_accel_free_viommu (accel); } }
hw/arm/tegra241-cmdqv.c +107 -0
115 lines not on this page · raw message
hw/arm/tegra241-cmdqv.h +6 -0
@@ -25,6 +25,10 @@*/ #define TEGRA241_CMDQV_IO_LEN 0x50000 +#define TEGRA241_CMDQV_VERSION 0x1 +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 0x1 +#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 0x4 + typedef struct Tegra241CMDQV { struct iommu_viommu_tegra241_cmdqv cmdqv_data; SMMUv3AccelState *s_accel;
@@ -67,6 +71,8 @@ FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) FIELD(CONFIG, CONS_DRAM_EN, 20, 1) +#define V_CONFIG_RESET 0x00020403 + REG32(PARAM, 0x4) FIELD(PARAM, CMDQV_VER, 0, 4)FIELD (PARAM ,CMDQV_NUM_CMDQ_LOG2 ,4 ,4 )
hw/arm/trace-events +1 -0
@@ -74,6 +74,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS # tegra241-cmdqv tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X" +tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X" # strongarm.c strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
PATCH v2 20/24 hw/arm/tegra241-cmdqv: Limit queue size based on backend page size
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
CMDQV HW reads guest queue memory in its host physical address setup via IOMUUFD. This requires the guest queue memory isn't only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQV size that the host can safely back with physically contiguous memory. Allowing a CMDQV larger than the host page size could cause the hardware to DMA across page boundaries leading to faults.
Limit IDR1.CMDQS so the guest cannot configure a CMDQV that exceeds the host’s contiguous backing.
--- hw/arm/tegra241-cmdqv.c | 43 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
hw/arm/tegra241-cmdqv.c +43 -0
@@ -11,10 +11,14 @@#include "qemu/error-report.h" #include "qemu/log.h" #include "trace.h" +#include <math.h> #include "hw/arm/smmuv3.h" #include "hw/core/irq.h" #include "smmuv3-accel.h" +#include "smmuv3-internal.h" +#include "system/ramblock.h" +#include "system/ramlist.h" #include "tegra241-cmdqv.h" static void tegra241_cmdqv_event_read (void *opaque)
@@ -582,6 +586,33 @@ tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,return true; } +static size_t tegra241_cmdqv_min_ram_pagesize (void ) +{ +RAMBlock *rb; +size_t pg, min_pg =SIZE_MAX ; + +RAMBLOCK_FOREACH (rb) { +MemoryRegion *mr = rb->mr; + +/* Only consider real RAM regions */ +if (!mr || !memory_region_is_ram (mr)) { +continue ; + } + +/* Skip RAM regions that are not backed by a memory-backend */ +if (!object_dynamic_cast (mr->owner,TYPE_MEMORY_BACKEND )) { +continue ; + } + + pg =qemu_ram_pagesize (rb); +if (pg && pg < min_pg) { + min_pg = pg; + } + } + +return (min_pg ==SIZE_MAX ) ?qemu_real_host_page_size () : min_pg; +} +static void tegra241_cmdqv_init_regs (SMMUv3State *s,Tegra241CMDQV *cmdqv) {SMMUv3AccelState *s_accel = s->s_accel;
@@ -589,7 +620,9 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)struct iommu_hw_info_tegra241_cmdqv cmdqv_info;SMMUv3AccelDevice *accel_dev;Error *local_err =NULL ; +size_t pgsize;uint64_t caps; +uint32_t val;int i;if (QLIST_EMPTY (&s_accel->device_list)) {
@@ -660,6 +693,16 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv) cmdqv->vcmdq_base[i] =0 ; cmdqv->vcmdq_cons_indx_base[i] =0 ; } + +/* +* CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the +* queue fits entirely within the smallest backend page size, ensuring +* the command queue is physically contiguous in host memory. +*/ + pgsize =tegra241_cmdqv_min_ram_pagesize (); + val =FIELD_EX32 (s->idr[1 ],IDR1 ,CMDQS ); + s->idr[1 ] =FIELD_DP32 (s->idr[1 ],IDR1 ,CMDQS ,MIN (log2 (pgsize) -4 , val)); +return ; out_err:
Signature · 1 line
2.43.0
PATCH v2 21/24 virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev
In reply to Shameer KolothumRename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.
No functional changes intended.
--- hw/arm/virt-acpi-build.c | 44 ++++++++++++++++++++++------------------ 1 file changed, 24 insertions(+), 20 deletions(-)
hw/arm/virt-acpi-build.c +24 -20
@@ -339,24 +339,28 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)return idmap_a->input_base - idmap_b->input_base; } -typedef struct AcpiIortSMMUv3Dev { +typedef struct AcpiSMMUv3Dev {int irq;hwaddr base; + +/* +* IORT-only fields. +* These are used when building IORT SMMUv3 nodes. +*/ GArray *rc_smmu_idmaps; -/* Offset of the SMMUv3 IORT Node relative to the start of the IORT */ -size_t offset; +size_t offset;/* Offset of the SMMUv3 node within the IORT table */ bool accel;bool ats; -}AcpiIortSMMUv3Dev ; +}AcpiSMMUv3Dev ;/* -* Populate the struct AcpiIortSMMUv3Dev for the legacy SMMUv3 and +* Populate the struct AcpiSMMUv3Dev for the legacy SMMUv3 and * return the total number of associated idmaps. */ static int populate_smmuv3_legacy_dev (GArray *sdev_blob) {VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev = {0 }; +AcpiSMMUv3Dev sdev = {0 }; sdev.rc_smmu_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping)); object_child_foreach_recursive(object_get_root(), iort_host_bridges,
@@ -376,8 +380,8 @@ static int populate_smmuv3_legacy_dev(GArray *sdev_blob)static int smmuv3_dev_idmap_compare (gconstpointer a,gconstpointer b) { -AcpiIortSMMUv3Dev *sdev_a = (AcpiIortSMMUv3Dev *)a; -AcpiIortSMMUv3Dev *sdev_b = (AcpiIortSMMUv3Dev *)b; +AcpiSMMUv3Dev *sdev_a = (AcpiSMMUv3Dev *)a; +AcpiSMMUv3Dev *sdev_b = (AcpiSMMUv3Dev *)b;AcpiIortIdMapping *map_a = &g_array_index (sdev_a->rc_smmu_idmaps, AcpiIortIdMapping,0 ); AcpiIortIdMapping *map_b = &g_array_index(sdev_b->rc_smmu_idmaps,
@@ -388,7 +392,7 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)static int iort_smmuv3_devices (Object *obj,void *opaque) {VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev = {0 }; +AcpiSMMUv3Dev sdev = {0 };GArray *sdev_blob = opaque;AcpiIortIdMapping idmap;PlatformBusDevice *pbus;
@@ -421,7 +425,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) }/* -* Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and +* Populate the struct AcpiSMMUv3Dev for all SMMUv3 devices and * return the total number of idmaps. */ static int populate_smmuv3_dev (GArray *sdev_blob)
@@ -442,10 +446,10 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) {AcpiIortIdMapping *idmap;AcpiIortIdMapping next_range = {0 }; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;for (int i =0 ; i < smmuv3_devs->len; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Based on the RID ranges that are directed to the SMMU, determine the * bypassed RID ranges, i.e., the ones that are directed to the ITS
@@ -479,7 +483,7 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)static void build_iort_rmr_nodes (GArray *table_data,GArray *smmuv3_devices,uint32_t *id) { -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;AcpiIortIdMapping *idmap;int i;
@@ -487,7 +491,7 @@ build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)uint16_t rmr_len;int bdf; - sdev = &g_array_index (smmuv3_devices, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devices, AcpiSMMUv3Dev, i);if (!sdev->accel) {continue ; }
@@ -544,13 +548,13 @@ static void build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) {int i, nb_nodes, rc_mapping_count; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;size_t node_size;bool ats_needed = false;int num_smmus =0 ;uint32_t id =0 ;int rc_smmu_idmaps_len =0 ; -GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiIortSMMUv3Dev)); +GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiSMMUv3Dev));GArray *rc_its_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping));AcpiTable table = { .sig ="IORT" , .rev =5 , .oem_id = vms->oem_id,
@@ -581,7 +585,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }/* Calculate RMR nodes required. One per SMMUv3 with accelerated mode */ for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);if (sdev->ats) { ats_needed = true; }
@@ -620,7 +624,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);int smmu_mapping_count, offset_to_id_array;int irq = sdev->irq;
@@ -699,7 +703,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)AcpiIortIdMapping *range;for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Map RIDs (input) from RC to SMMUv3 nodes: RC -> SMMUv3.
@@ -742,7 +746,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_table_end (linker, &table);g_array_free (rc_its_idmaps, true);for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);g_array_free (sdev->rc_smmu_idmaps, true); }g_array_free (smmuv3_devs, true);
Signature · 1 line
2.43.0
Re: [PATCH v2 21/24] virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev
Hi Shameer,
Quoted 25 lines · Shameer Kolothum
Rename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.
No functional changes intended.
Reviewed-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/virt-acpi-build.c | 44 ++++++++++++++++++++++------------------ 1 file changed, 24 insertions(+), 20 deletions(-)hw/arm/virt-acpi-build.c +2
-1@@ -339,24 +339,28 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)return idmap_a->input_base - idmap_b->input_base; } -typedef struct AcpiIortSMMUv3Dev { +typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; +
hw/arm/virt-acpi-build.c static int iort_idmap_compare(gconstpointer a, gconstpointer b) · line 353
Quoted 10 lines
hw/arm/virt-acpi-build.c +5
-2+/* +* IORT-only fields. +* These are used when building IORT SMMUv3 nodes. +*/ GArray *rc_smmu_idmaps; -/* Offset of the SMMUv3 IORT Node relative to the start of the IORT */ -size_t offset; +size_t offset;/* Offset of the SMMUv3 node within the IORT table */ bool accel;bool ats;
accel and ats seem to be only used by IORT too. So it is a bit weird to have a so-called generic datatype which is half iort specific. Why don't use keepAcpiIortSMMUv3Dev and flat encapsulate another object inside with irq and base and id? Eric
Quoted 128 lines
hw/arm/virt-acpi-build.c +17
-17-} AcpiIortSMMUv3Dev; +} AcpiSMMUv3Dev;/* -* Populate the struct AcpiIortSMMUv3Dev for the legacy SMMUv3 and +* Populate the struct AcpiSMMUv3Dev for the legacy SMMUv3 and * return the total number of associated idmaps. */ static int populate_smmuv3_legacy_dev (GArray *sdev_blob) {VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev = {0 }; +AcpiSMMUv3Dev sdev = {0 }; sdev.rc_smmu_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping)); object_child_foreach_recursive(object_get_root(), iort_host_bridges,@@ -376,8 +380,8 @@ static int populate_smmuv3_legacy_dev(GArray *sdev_blob)static int smmuv3_dev_idmap_compare (gconstpointer a,gconstpointer b) { -AcpiIortSMMUv3Dev *sdev_a = (AcpiIortSMMUv3Dev *)a; -AcpiIortSMMUv3Dev *sdev_b = (AcpiIortSMMUv3Dev *)b; +AcpiSMMUv3Dev *sdev_a = (AcpiSMMUv3Dev *)a; +AcpiSMMUv3Dev *sdev_b = (AcpiSMMUv3Dev *)b;AcpiIortIdMapping *map_a = &g_array_index (sdev_a->rc_smmu_idmaps, AcpiIortIdMapping,0 ); AcpiIortIdMapping *map_b = &g_array_index(sdev_b->rc_smmu_idmaps,@@ -388,7 +392,7 @@ static int smmuv3_dev_idmap_compare(gconstpointer a, gconstpointer b)static int iort_smmuv3_devices (Object *obj,void *opaque) {VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev = {0 }; +AcpiSMMUv3Dev sdev = {0 };GArray *sdev_blob = opaque;AcpiIortIdMapping idmap;PlatformBusDevice *pbus;@@ -421,7 +425,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) }/* -* Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and +* Populate the struct AcpiSMMUv3Dev for all SMMUv3 devices and * return the total number of idmaps. */ static int populate_smmuv3_dev (GArray *sdev_blob)@@ -442,10 +446,10 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) {AcpiIortIdMapping *idmap;AcpiIortIdMapping next_range = {0 }; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;for (int i =0 ; i < smmuv3_devs->len; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Based on the RID ranges that are directed to the SMMU, determine the * bypassed RID ranges, i.e., the ones that are directed to the ITS @@ -479,7 +483,7 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)static void build_iort_rmr_nodes (GArray *table_data,GArray *smmuv3_devices,uint32_t *id) { -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;AcpiIortIdMapping *idmap;int i;@@ -487,7 +491,7 @@ build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)uint16_t rmr_len;int bdf; - sdev = &g_array_index (smmuv3_devices, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devices, AcpiSMMUv3Dev, i);if (!sdev->accel) {continue ; }@@ -544,13 +548,13 @@ static void build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) {int i, nb_nodes, rc_mapping_count; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;size_t node_size;bool ats_needed = false;int num_smmus =0 ;uint32_t id =0 ;int rc_smmu_idmaps_len =0 ; -GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiIortSMMUv3Dev)); +GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiSMMUv3Dev));GArray *rc_its_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping));AcpiTable table = { .sig ="IORT" , .rev =5 , .oem_id = vms->oem_id,@@ -581,7 +585,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }/* Calculate RMR nodes required. One per SMMUv3 with accelerated mode */ for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);if (sdev->ats) { ats_needed = true; }@@ -620,7 +624,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);int smmu_mapping_count, offset_to_id_array;int irq = sdev->irq;@@ -699,7 +703,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)AcpiIortIdMapping *range;for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Map RIDs (input) from RC to SMMUv3 nodes: RC -> SMMUv3. @@ -742,7 +746,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_table_end (linker, &table);g_array_free (rc_its_idmaps, true);for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);g_array_free (sdev->rc_smmu_idmaps, true); }g_array_free (smmuv3_devs, true);
PATCH v2 22/24 hw/arm/smmuv3: Add per-device identifier poroperty
In reply to Shameer KolothumAdd an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
--- hw/arm/smmuv3.c | 1 + hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ include/hw/arm/smmuv3.h | 1 + 4 files changed, 8 insertions(+), 1 deletion(-)
hw/arm/smmuv3.c +1 -0
@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),
hw/arm/virt-acpi-build.c +3 -1
@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; + uint8_t id; /* * IORT-only fields.
@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );
@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */ /* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */
hw/arm/virt.c +3 -0
@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, (vms->legacy_smmuv3_present) ? "iommu=smmuv3" : "virtio-iommu"); }else if (vms->iommu ==VIRT_IOMMU_NONE ) { +static uint8_t id; +/* The new SMMUv3 device is specific to the PCI bus */ object_property_set_bool (OBJECT (dev),"smmu_per_bus" , true,NULL );object_property_set_link (OBJECT (dev),"memory" ,OBJECT (vms->sysmem),NULL );object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , id++,NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;
include/hw/arm/smmuv3.h +1 -0
@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;
Signature · 1 line
2.43.0
Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
Quoted 10 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
Signed-off-by Shameer Kolothum <[email protected]>
With nits:
hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2117
Quoted 9 lines
hw/arm/smmuv3.c +1
-0@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), + DEFINE_PROP_UINT8("identifier", SMMUv3State, identifier, 0),
Perhaps add a line of comments for where we use it.
hw/arm/virt.c static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, · line 3062
Quoted 9 lines
hw/arm/virt.c +1
-0@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, (vms->legacy_smmuv3_present) ? "iommu=smmuv3" : "virtio-iommu"); }else if (vms->iommu ==VIRT_IOMMU_NONE ) { +static uint8_t id;
Local static feels odd. Perhaps make it global?
Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
In reply to Shameer KolothumHi,
Quoted 2 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
typo in the title.
I don't understand what does mandate that change? We already have an identifier set in the IORT table. The rationale behind that patch needs to be clarified
Thanks
Eric
Quoted 88 lines
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3.c | 1 + hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ include/hw/arm/smmuv3.h | 1 + 4 files changed, 8 insertions(+), 1 deletion(-)hw/arm/smmuv3.c +1
-0@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),hw/arm/virt-acpi-build.c +3
-1@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; + uint8_t id; /* * IORT-only fields.@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */ /* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */ hw/arm/virt.c +3
-0@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, (vms->legacy_smmuv3_present) ? "iommu=smmuv3" : "virtio-iommu"); }else if (vms->iommu ==VIRT_IOMMU_NONE ) { +static uint8_t id; +/* The new SMMUv3 device is specific to the PCI bus */ object_property_set_bool (OBJECT (dev),"smmu_per_bus" , true,NULL );object_property_set_link (OBJECT (dev),"memory" ,OBJECT (vms->sysmem),NULL );object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , id++,NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;include/hw/arm/smmuv3.h +1
-0@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;
Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
In reply to Shameer KolothumQuoted 8 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
I think an important info which is missing here is you need the id used in IORT and DSDT to be the same, is that correct
By the way why can't you reuse the QOM id. I think Dan said there is one for all QOM objects.
Thanks
Eric
Quoted 82 lines
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3.c | 1 + hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ include/hw/arm/smmuv3.h | 1 + 4 files changed, 8 insertions(+), 1 deletion(-)hw/arm/smmuv3.c +1
-0@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),hw/arm/virt-acpi-build.c +3
-1@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; + uint8_t id; /* * IORT-only fields.@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */ /* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */ hw/arm/virt.c +3
-0@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, (vms->legacy_smmuv3_present) ? "iommu=smmuv3" : "virtio-iommu"); }else if (vms->iommu ==VIRT_IOMMU_NONE ) { +static uint8_t id; +/* The new SMMUv3 device is specific to the PCI bus */ object_property_set_bool (OBJECT (dev),"smmu_per_bus" , true,NULL );object_property_set_link (OBJECT (dev),"memory" ,OBJECT (vms->sysmem),NULL );object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , id++,NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;include/hw/arm/smmuv3.h +1
-0@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;
RE: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
Quoted 23 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 18:30
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poropertyExternal email: Use caution opening links or attachments
Quoted 3 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
Quoted 7 lines
Quoted 5 lines
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
I think an important info which is missing here is you need the id used in IORT and DSDT to be the same, is that correct
Yes. That is correct. I think I have that explanation in next patch. I will mention it here as well.
Quoted 3 lines
By the way why can't you reuse the QOM id. I think Dan said there is one for all QOM objects.
Yes. QOM device has a global id. But I am not sure we can use that here. For one it is a "char *" and normally user or management layers will use That,
eg:
-device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1-pxb1,
Thanks,
Shameer
Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
Quoted 31 lines · Shameer Kolothum Thodi
Quoted 22 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 18:30
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poropertyExternal email: Use caution opening links or attachments
Quoted 3 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
Quoted 7 lines
Quoted 5 lines
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
I think an important info which is missing here is you need the id used in IORT and DSDT to be the same, is that correct
Yes. That is correct. I think I have that explanation in next patch. I will mention it here as well.
Quoted 7 lines
Quoted 2 lines
By the way why can't you reuse the QOM id. I think Dan said there is one for all QOM objects.
Yes. QOM device has a global id. But I am not sure we can use that here. For one it is a "char *" and normally user or management layers will use That,
eg:
-device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1-pxb1,
oh yes you're correct. Forget about it
Eric
Quoted 4 lines
Thanks,
Shameer
Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
In reply to Shameer KolothumQuoted 50 lines · Shameer Kolothum
Add an "identifier" property to the SMMUv3 device and use it when building the ACPI IORT SMMUv3 node Identifier field.
This avoids relying on enumeration order and provides a stable per-device identifier. This is useful when we add support for Tegra241 CMDQV DSDT in subsequent patch.
No functional change intended.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3.c | 1 + hw/arm/virt-acpi-build.c | 4 +++- hw/arm/virt.c | 3 +++ include/hw/arm/smmuv3.h | 1 + 4 files changed, 8 insertions(+), 1 deletion(-)hw/arm/smmuv3.c +1
-0@@ -2114,6 +2114,7 @@ static const Property smmuv3_properties[] = {* Defaults to stage 1 */ DEFINE_PROP_STRING ("stage" , SMMUv3State, stage), +DEFINE_PROP_UINT8 ("identifier" , SMMUv3State, identifier,0 ),DEFINE_PROP_BOOL ("accel" , SMMUv3State, accel, false),/* GPA of MSI doorbell, for SMMUv3 accel use. */ DEFINE_PROP_UINT64("msi-gpa", SMMUv3State, msi_gpa, 0),hw/arm/virt-acpi-build.c +2
-0@@ -342,6 +342,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b) typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; + uint8_t id; /* * IORT-only fields.@@ -407,6 +408,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) bus =PCI_BUS (object_property_get_link (obj,"primary-bus" , &error_abort)); sdev.accel =object_property_get_bool (obj,"accel" , &error_abort); sdev.ats =object_property_get_bool (obj,"ats" , &error_abort); + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); sbdev =SYS_BUS_DEVICE (obj); sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,0 );
hw/arm/virt-acpi-build.c build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) · line 647
Quoted 5 lines
hw/arm/virt-acpi-build.c +1
-1@@ -642,7 +644,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) (ID_MAPPING_ENTRY_SIZE * smmu_mapping_count);build_append_int_noprefix (table_data, node_size,2 );/* Length */ build_append_int_noprefix (table_data,4 ,1 );/* Revision */ -build_append_int_noprefix (table_data, id++,4 );/* Identifier */ +build_append_int_noprefix (table_data, sdev->id,4 );/* Identifier */
Would that keep the same ordering for existing qtests?
Eric
Quoted 35 lines
hw/arm/virt-acpi-build.c +0
-0/* Number of ID mappings */ build_append_int_noprefix (table_data, smmu_mapping_count,4 );/* Reference to ID Array */ hw/arm/virt.c +3
-0@@ -3059,12 +3059,15 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, (vms->legacy_smmuv3_present) ? "iommu=smmuv3" : "virtio-iommu"); }else if (vms->iommu ==VIRT_IOMMU_NONE ) { +static uint8_t id; +/* The new SMMUv3 device is specific to the PCI bus */ object_property_set_bool (OBJECT (dev),"smmu_per_bus" , true,NULL );object_property_set_link (OBJECT (dev),"memory" ,OBJECT (vms->sysmem),NULL );object_property_set_link (OBJECT (dev),"secure-memory" ,OBJECT (vms->secure_sysmem),NULL ); +object_property_set_uint (OBJECT (dev),"identifier" , id++,NULL ); }if (object_property_get_bool (OBJECT (dev),"accel" , &error_abort)) {hwaddr db_start =0 ;include/hw/arm/smmuv3.h +1
-0@@ -63,6 +63,7 @@ struct SMMUv3State {qemu_irq irq[4 ];QemuMutex mutex;char *stage; +uint8_t identifier;/* SMMU has HW accelerator support for nested S1 + s2 */ bool accel;
RE: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
Quoted 16 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 February 2026 18:49
To: Shameer Kolothum Thodi <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 22/24] hw/arm/smmuv3: Add per-device identifier poroperty
Quoted 5 lines
Review
Quoted 4 lines
build_append_int_noprefix(table_data, 4, 1); /* Revision */(quoted diff) +2
-1- build_append_int_noprefix(table_data, id++, 4); /* Identifier */ + build_append_int_noprefix(table_data, sdev->id, 4); /* + Identifier */Would that keep the same ordering for existing qtests?
Hmm.. I need to check that.
Thanks,
Shameer
PATCH v2 23/24 hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.
The SMMUv3 device identifier is used as the ACPI _UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.
Because virt-acpi-build.c now includes CONFIG_DEVICES via the Tegra241 CMDQV header, the Meson file entry is updated to build it as part of arm_ss instead of arm_common_ss
--- hw/arm/meson.build | 2 +- hw/arm/trace-events | 1 + hw/arm/virt-acpi-build.c | 73 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+), 1 deletion(-)
hw/arm/meson.build +1 -1
@@ -1,7 +1,7 @@ arm_ss = ss.source_set() arm_common_ss = ss.source_set() arm_common_ss.add(when: 'CONFIG_ARM_VIRT', if_true: files('virt.c')) -arm_common_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c')) +arm_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c')) arm_common_ss.add(when: 'CONFIG_DIGIC', if_true: files('digic_boards.c')) arm_common_ss.add(when: 'CONFIG_EMCRAFT_SF2', if_true: files('msf2-som.c')) arm_common_ss.add(when: 'CONFIG_HIGHBANK', if_true: files('highbank.c'))
hw/arm/trace-events +1 -0
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s" # virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." +virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d" # smmu-common.c smmu_add_mr(const char *name) "%s"
hw/arm/virt-acpi-build.c +73 -0
91 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [PATCH v2 23/24] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 84 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv.
The SMMUv3 device identifier is used as the ACPI
_UID. This matches the Identifier field of the corresponding SMMUv3 IORT node, allowing the CMDQV DSDT device to be correctly associated with its SMMU.Because
virt-acpi-build.cnow includesCONFIG_DEVICESvia the Tegra241 CMDQV header, the Meson file entry is updated to build it as part ofarm_ssinstead ofarm_common_ssSigned-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/meson.build | 2 +- hw/arm/trace-events | 1 + hw/arm/virt-acpi-build.c | 73 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 75 insertions(+), 1 deletion(-)hw/arm/meson.build +1
-1@@ -1,7 +1,7 @@ arm_ss = ss.source_set() arm_common_ss = ss.source_set() arm_common_ss.add(when: 'CONFIG_ARM_VIRT', if_true: files('virt.c')) -arm_common_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c')) +arm_ss.add(when: 'CONFIG_ACPI', if_true: files('virt-acpi-build.c')) arm_common_ss.add(when: 'CONFIG_DIGIC', if_true: files('digic_boards.c')) arm_common_ss.add(when: 'CONFIG_EMCRAFT_SF2', if_true: files('msf2-som.c')) arm_common_ss.add(when: 'CONFIG_HIGHBANK', if_true: files('highbank.c'))hw/arm/trace-events +1
-0@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s" # virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." +virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d" # smmu-common.c smmu_add_mr(const char *name) "%s"hw/arm/virt-acpi-build.c +23
-0@@ -65,6 +65,8 @@#include "target/arm/cpu.h" #include "target/arm/multiprocessing.h" +#include "tegra241-cmdqv.h" +#define ARM_SPI_BASE 32#define ACPI_BUILD_TABLE_SIZE 0x20000@@ -1121,6 +1123,75 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,build_fadt (table_data, linker, &fadt, vms->oem_id, vms->oem_table_id); } +static int smmuv3_cmdqv_devices (Object *obj,void *opaque) +{ +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + } + + sdev.id =object_property_get_uint (obj,"identifier" , &error_abort); + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base;
hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1156
Quoted 10 lines
hw/arm/virt-acpi-build.c +10
-0+ sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ; +} + +static void acpi_dsdt_add_tegra241_cmdqv (Aml *scope,VirtMachineState *vms) +{ +GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiSMMUv3Dev));
wonder if it wouldn't be possible to compute and store the blob in vms once and reuse that for both iort and dsdt?
Eric
Quoted 50 lines
hw/arm/virt-acpi-build.c +40
-0+int i; + +if (vms->legacy_smmuv3_present) { +return ; + } + +object_child_foreach_recursive (object_get_root (), smmuv3_cmdqv_devices, + smmuv3_devs); + +for (i =0 ; i < smmuv3_devs->len; i++) { +AcpiSMMUv3Dev *sdev; +Aml *dev, *crs, *addr; + + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i); + + dev =aml_device ("CV%.02u" , sdev->id); +aml_append (dev,aml_name_decl ("_HID" ,aml_string ("NVDA200C" ))); +aml_append (dev,aml_name_decl ("_UID" ,aml_int (sdev->id))); +aml_append (dev,aml_name_decl ("_CCA" ,aml_int (1 ))); + + crs =aml_resource_template (); + addr =aml_qword_memory (AML_POS_DECODE ,AML_MIN_FIXED ,AML_MAX_FIXED , +AML_CACHEABLE ,AML_READ_WRITE ,0x0 , sdev->base, + sdev->base +TEGRA241_CMDQV_IO_LEN -0x1 ,0x0 , +TEGRA241_CMDQV_IO_LEN ); +aml_append (crs, addr); +aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_EDGE , +AML_ACTIVE_HIGH ,AML_EXCLUSIVE , + (uint32_t *)&sdev->irq,1 )); +aml_append (dev,aml_name_decl ("_CRS" , crs)); + +aml_append (scope, dev); + +trace_virt_acpi_dsdt_tegra241_cmdqv (sdev->id, sdev->base, sdev->irq); + } +g_array_free (smmuv3_devs, true); +} +/* DSDT */ static void build_dsdt (GArray *table_data,BIOSLinker *linker,VirtMachineState *vms)@@ -1185,6 +1256,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_dsdt_add_tpm (scope, vms);#endif +acpi_dsdt_add_tegra241_cmdqv (scope, vms); +aml_append (dsdt, scope); pci0_scope =aml_scope ("\\_SB.PCI0" );
PATCH v2 24/24 hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
In reply to Shameer KolothumIntroduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
--- hw/arm/smmuv3.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
hw/arm/smmuv3.c +13 -0
@@ -1956,6 +1956,12 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp) }#endif +#ifndef CONFIG_TEGRA241_CMDQV +if (s->tegra241_cmdqv) { +error_setg (errp,"tegra241_cmdqv=on support not compiled in" ); +return false; + } +#endif if (!s->accel) {if (!s->ril) {error_setg (errp,"ril can only be disabled if accel=on" );
@@ -1973,6 +1979,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)error_setg (errp,"ssidsize can only be set if accel=on" );return false; } +if (s->tegra241_cmdqv) { +error_setg (errp,"tegra241_cmdqv can only be enabled if accel=on" ); +return false; + }return true; }
@@ -2123,6 +2133,7 @@ static const Property smmuv3_properties[] = {DEFINE_PROP_BOOL ("ats" , SMMUv3State, ats, false),DEFINE_PROP_UINT8 ("oas" , SMMUv3State, oas,44 ),DEFINE_PROP_UINT8 ("ssidsize" , SMMUv3State, ssidsize,0 ), + DEFINE_PROP_BOOL("tegra241-cmdqv", SMMUv3State, tegra241_cmdqv, false), };static void smmuv3_instance_init (Object *obj)
@@ -2162,6 +2173,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data)"Valid range is 0-20, where 0 disables SubstreamID support. " "Defaults to 0. A value greater than 0 is required to enable " "PASID support."); +object_class_property_set_description (klass,"tegra241-cmdqv" , +"Enable/disable Tegra241 CMDQ-Virtualisation support (for accel=on)" ); } static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
Quoted 2 lines · Shameer Kolothum
Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
Hi, do we expect that in the future there will be some other option named {newplatform}-cmdqv?
I'm asking mainly from libvirt PoV where current RFC patch exports this option only as cmdqv.
Pavel
RE: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
Quoted 17 lines
-----Original Message-----
From: Pavel Hrdina <[email protected]>
Sent: 13 February 2026 11:55
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 deviceOn Fri, Feb 06, 2026 at 02:48:23PM +0000, Shameer Kolothum wrote:
Quoted 8 lines
Quoted 2 lines
Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
Hi, do we expect that in the future there will be some other option named
{newplatform}-cmdqv?I'm asking mainly from libvirt PoV where current RFC patch exports this option only as
cmdqv.
Yes, with the current design each implementation would expose its own xxx-cmdqv property.
However, please see the discussion:
https://lore.kernel.org/qemu-devel/CH3PR12MB75484A150D6DCD9D3C3C66F3AB60A@CH3PR12MB7548.namprd12.prod.outlook.com/
I am looking into whether we can instead have a single cmdqv property and probe the kernel to determine the specific implementation type.
Thanks,
Shameer
Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
Quoted 26 lines · Shameer Kolothum Thodi
Quoted 17 lines
-----Original Message-----
From: Pavel Hrdina <[email protected]>
Sent: 13 February 2026 11:55
To: Shameer Kolothum Thodi <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jiandi An <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [PATCH v2 24/24] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 deviceOn Fri, Feb 06, 2026 at 02:48:23PM +0000, Shameer Kolothum wrote:
Quoted 8 lines
Quoted 2 lines
Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
Hi, do we expect that in the future there will be some other option named
{newplatform}-cmdqv?I'm asking mainly from libvirt PoV where current RFC patch exports this option only as
cmdqv.
Quoted 8 lines
Yes, with the current design each implementation would expose its own xxx-cmdqv property.
However, please see the discussion:
https://lore.kernel.org/qemu-devel/CH3PR12MB75484A150D6DCD9D3C3C66F3AB60A@CH3PR12MB7548.namprd12.prod.outlook.com/I am looking into whether we can instead have a single cmdqv property and probe the kernel to determine the specific implementation type.
Thanks for the details, yeah that looks better.
I was also thinking about using cmdqv=tegra241 in addition to 'off' and 'auto' if it would be possible for host to support multiple implementations at the same time. Libvirt would probably prefer setting specific implementation instead of using 'auto' to make sure that the VM will not change.
Pavel