hw/arm: Introduce Tegra241 CMDQV support for accelerated SMMUv3
Subsystems (2)
cover hw/arm: Introduce Tegra241 CMDQV support for accelerated SMMUv3
Hi,
This RFC series adds initial support for NVIDIA Tegra241 CMDQV (Command Queue Virtualisation), an extension to ARM SMMUv3 that provides hardware accelerated virtual command queues (VCMDQs) for guests. CMDQV allows guests to issue SMMU invalidation commands directly to hardware without VM exits, significantly reducing TLBI overhead.
Thanks to Nicolin for the initial patches and testing on which this RFC is based.
This is based on v6[0] of the SMMUv3 accel series, which is still under review, though nearing convergence. This is sent as an RFC, with the goal of gathering early feedback on the CMDQV design and its integration with the SMMUv3 acceleration path.
Background:
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)
Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
VCMDQ support is integrated via the IOMMU_HW_QUEUE_ALLOC mechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].
Summary of QEMU changes:
- Integrated into the existing SMMUv3 accel path via a "tegra241-cmdqv" property.
- Support for allocating vIOMMU objects of type IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.
- Mapping and emulation of the CMDQV MMIO register layout.
- VCMDQ/VINTF read/write handling and queue allocation using IOMMUFD APIs.
- Reset and initialisation hooks, including checks for at least one cold-plugged device.
- CMDQV hardware reads guest queue memory using host physical addresses provided through IOMMUFD, which requires that the VCMDQ buffer be physically contiguous not only in guest PA space but also in host PA space. When Tegra241 CMDQV is enabled, QEMU must therefore only expose a CMDQV size that the host can reliably back with contiguous physical memory. Because of this constraint, it is suggested to use huge pages to back the guest RAM.
- ACPI DSDT node generation for CMDQV devices on the virt machine.
These patches have been sanity tested on NVIDIA Grace platforms.
ToDo / revisit:
- Prevent hot-unplug of the last device associated with vIOMMU as this might allow associating a different host SMMU/CMDQV.
- Locking requirements around error event propagation.
Feedback and testing are very welcome.
Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
Nicolin Chen (12):
backends/iommufd: Update iommufd_backend_get_device_info backends/iommufd: Update iommufd_backend_alloc_viommu to allow user
ptr
backends/iommufd: Introduce iommufd_backend_alloc_hw_queue backends/iommufd: Introduce iommufd_backend_viommu_mmap hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation
support
hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest hw/arm/tegra241-cmdqv: Add read emulation support for registers system/physmem: Add helper to check whether a guest PA maps to RAM hw/arm/tegra241-cmdqv:: Add write emulation for registers hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Shameer Kolothum (4):
hw/arm/tegra241-cmdqv: Allocate vEVENTQ object hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
backends/iommufd.c | 65 ++++ backends/trace-events | 2 + hw/arm/Kconfig | 5 + hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 16 +- hw/arm/smmuv3.c | 18 + hw/arm/tegra241-cmdqv.c | 759 ++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 337 +++++++++++++++++ hw/arm/trace-events | 5 + hw/arm/virt-acpi-build.c | 110 +++++- hw/vfio/iommufd.c | 6 +- include/exec/cpu-common.h | 2 + include/hw/arm/smmuv3.h | 3 + include/hw/arm/virt.h | 2 + include/system/iommufd.h | 16 + system/physmem.c | 12 + 16 files changed, 1332 insertions(+), 27 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.h
Signature · 1 line
2.43.0
RFC PATCH 1/16 backends/iommufd: Update iommufd_backend_get_device_info
From: Nicolin Chen <[email protected]>
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via IOMMU_GET_HW_INFO. Update iommufd_backend_get_device_info() to set IOMMU_HW_INFO_FLAG_INPUT_TYPE when a non-zero type is supplied, and adjust all callers to pass an explicitly initialised type value.
--- backends/iommufd.c | 7 +++++++ hw/arm/smmuv3-accel.c | 2 +- hw/vfio/iommufd.c | 6 ++---- 3 files changed, 10 insertions(+), 5 deletions(-)
backends/iommufd.c +7 -0
@@ -386,16 +386,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,return true; } +/* +* @type can carry a desired HW info type defined in the uapi headers. If caller +* doesn't have one, indicating it wants the default type, then @type should be +* zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT). +*/ bool iommufd_backend_get_device_info (IOMMUFDBackend *be,uint32_t devid,uint32_t *type,void *data,uint32_t len,uint64_t *caps,uint8_t *max_pasid_log2,Error **errp) {struct iommu_hw_info info = { + .flags = (*type) ?IOMMU_HW_INFO_FLAG_INPUT_TYPE :0 , .size =sizeof (info), .dev_id = devid, .data_len = len, .data_uptr = (uintptr_t )data, + .in_data_type = *type, };if (ioctl (be->fd,IOMMU_GET_HW_INFO , &info)) {
hw/arm/smmuv3-accel.c +1 -1
@@ -115,7 +115,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) { struct iommu_hw_info_arm_smmuv3 info; - uint32_t data_type; + uint32_t data_type = 0; uint64_t caps; if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,
hw/vfio/iommufd.c +2 -4
@@ -306,7 +306,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,ERRP_GUARD ();IOMMUFDBackend *iommufd = vbasedev->iommufd;VFIOContainer *bcontainer =VFIO_IOMMU (container); -uint32_t type, flags =0 ; +uint32_t type =0 , flags =0 ;uint64_t hw_caps;VFIOIOASHwpt *hwpt;uint32_t hwpt_id;
@@ -631,8 +631,6 @@ skip_ioas_alloc: bcontainer->initialized = true; found_container: - vbasedev->cpr.ioas_id = container->ioas_id; - ret =ioctl (devfd,VFIO_DEVICE_GET_INFO , &dev_info);if (ret) {error_setg_errno (errp, errno,"error getting device info" );
@@ -889,7 +887,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,HostIOMMUDeviceIOMMUFD *idev;HostIOMMUDeviceCaps *caps = &hiod->caps;VendorCaps *vendor_caps = &caps->vendor_caps; -enum iommu_hw_info_type type; +enum iommu_hw_info_type type =0 ;uint8_t max_pasid_log2;uint64_t hw_caps;
Signature · 1 line
2.43.0
Re: [RFC PATCH 01/16] backends/iommufd: Update iommufd_backend_get_device_info
Quoted 5 lines · Shameer Kolothum
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via
IOMMU_GET_HW_INFO. Updateiommufd_backend_get_device_info()to setIOMMU_HW_INFO_FLAG_INPUT_TYPEwhen a non-zero type is supplied, and adjust all callers to pass an explicitly initialised type value.
Let's mention IOMMU_HW_INFO_TYPE_DEFAULT=0 v.s. just "initialized".
Quoted 2 lines
+++
b/hw/vfio/iommufd.c(quoted diff) +0
-0@@ -306,7 +306,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,
[...]
hw/vfio/iommufd.c skip_ioas_alloc: · line 636
Quoted 8 lines
bcontainer->initialized = true;
found_container:hw/vfio/iommufd.c +0
-2- vbasedev->cpr.ioas_id = container->ioas_id; - ret =ioctl (devfd,VFIO_DEVICE_GET_INFO , &dev_info);if (ret) {error_setg_errno (errp, errno,"error getting device info" );
I might have missed some details but this seems unrelated?
Nicolin
RE: [RFC PATCH 01/16] backends/iommufd: Update iommufd_backend_get_device_info
Quoted 20 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 18:40
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 01/16] backends/iommufd: Updateiommufd_backend_get_device_infoQuoted 4 lines · Shameer Kolothum
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via
IOMMU_GET_HW_INFO. Updateiommufd_backend_get_device_info()to set
Quoted 5 lines
IOMMU_HW_INFO_FLAG_INPUT_TYPEQuoted 2 lines
when a non-zero type is supplied, and adjust all callers to pass an explicitly initialised type value.
Let's mention IOMMU_HW_INFO_TYPE_DEFAULT=0 v.s. just "initialized".
Ok.
Quoted 7 lines
Quoted 4 lines
+++
b/hw/vfio/iommufd.c(quoted diff) +0
-0@@ -306,7 +306,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,[...]
Quoted 2 lines
bcontainer->initialized = true;
Quoted 8 lines
hw/vfio/iommufd.c skip_ioas_alloc: · line 636
Quoted 6 lines
found_container:hw/vfio/iommufd.c +0
-2- vbasedev->cpr.ioas_id = container->ioas_id; - ret =ioctl (devfd,VFIO_DEVICE_GET_INFO , &dev_info);if (ret) {error_setg_errno (errp, errno,"error getting device info" );I might have missed some details but this seems unrelated?
Yes, that is likely an artifact from a careless rebase on my side. Thanks for catching it.
Shameer
Re: [RFC PATCH 01/16] backends/iommufd: Update iommufd_backend_get_device_info
In reply to Shameer KolothumHi Shameer,
Quoted 1 line · Shameer Kolothum
From: Nicolin Chen <[email protected]>
I would suggest to be more explicit in the commit title like:
Convey input type in iommufd_backend_get_device_info()
Quoted 20 lines
The updated IOMMUFD uAPI introduces the ability for userspace to request a specific hardware info data type via
IOMMU_GET_HW_INFO. Updateiommufd_backend_get_device_info()to setIOMMU_HW_INFO_FLAG_INPUT_TYPEwhen a non-zero type is supplied, and adjust all callers to pass an explicitly initialised type value.Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 7 +++++++ hw/arm/smmuv3-accel.c | 2 +- hw/vfio/iommufd.c | 6 ++---- 3 files changed, 10 insertions(+), 5 deletions(-)backends/iommufd.c +0
-0
backends/iommufd.c bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, · line 392
Quoted 7 lines
backends/iommufd.c +4
-0@@ -386,16 +386,23 @@ bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be,return true; } +/* +* @type can carry a desired HW info type defined in the uapi headers. If caller +* doesn't have one, indicating it wants the default type, then @type should be +* zeroed (i.e. IOMMU_HW_INFO_TYPE_DEFAULT).
the kernel uapi doc is a bit ambiguous as it is said:
* @flags: Must be 0
while a flag value does exist.
Quoted 43 lines
backends/iommufd.c +3
-0+*/ bool iommufd_backend_get_device_info (IOMMUFDBackend *be,uint32_t devid,uint32_t *type,void *data,uint32_t len,uint64_t *caps,uint8_t *max_pasid_log2,Error **errp) {struct iommu_hw_info info = { + .flags = (*type) ?IOMMU_HW_INFO_FLAG_INPUT_TYPE :0 , .size =sizeof (info), .dev_id = devid, .data_len = len, .data_uptr = (uintptr_t )data, + .in_data_type = *type, };if (ioctl (be->fd,IOMMU_GET_HW_INFO , &info)) {hw/arm/smmuv3-accel.c +1
-1@@ -115,7 +115,7 @@ smmuv3_accel_hw_compatible(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, Error **errp) { struct iommu_hw_info_arm_smmuv3 info; - uint32_t data_type; + uint32_t data_type = 0; uint64_t caps; if (!iommufd_backend_get_device_info(idev->iommufd, idev->devid, &data_type,hw/vfio/iommufd.c +1
-1@@ -306,7 +306,7 @@ static bool iommufd_cdev_autodomains_get(VFIODevice *vbasedev,ERRP_GUARD ();IOMMUFDBackend *iommufd = vbasedev->iommufd;VFIOContainer *bcontainer =VFIO_IOMMU (container); -uint32_t type, flags =0 ; +uint32_t type =0 , flags =0 ;uint64_t hw_caps;VFIOIOASHwpt *hwpt;uint32_t hwpt_id;
hw/vfio/iommufd.c skip_ioas_alloc: · line 635
Quoted 5 lines
hw/vfio/iommufd.c +0
-2@@ -631,8 +631,6 @@ skip_ioas_alloc: bcontainer->initialized = true; found_container: - vbasedev->cpr.ioas_id = container->ioas_id; -
indeed looks unrelated to that patch
hw/vfio/iommufd.c static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque, · line 893
Quoted 12 lines
hw/vfio/iommufd.c +1
-1ret =ioctl (devfd,VFIO_DEVICE_GET_INFO , &dev_info);if (ret) {error_setg_errno (errp, errno,"error getting device info" );@@ -889,7 +887,7 @@ static bool hiod_iommufd_vfio_realize(HostIOMMUDevice *hiod, void *opaque,HostIOMMUDeviceIOMMUFD *idev;HostIOMMUDeviceCaps *caps = &hiod->caps;VendorCaps *vendor_caps = &caps->vendor_caps; -enum iommu_hw_info_type type; +enum iommu_hw_info_type type =0 ;uint8_t max_pasid_log2;uint64_t hw_caps;
Besides
Thanks
Eric
RFC PATCH 2/16 backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
The updated IOMMUFD VIOMMU_ALLOC uAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extend iommufd_backend_alloc_viommu() to pass a user pointer and size to the kernel.
Update the caller accordingly.
--- backends/iommufd.c | 3 +++ hw/arm/smmuv3-accel.c | 4 ++-- include/system/iommufd.h | 1 + 3 files changed, 6 insertions(+), 2 deletions(-)
backends/iommufd.c +3 -0
@@ -459,6 +459,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t len,uint32_t *out_viommu_id,Error **errp) {int ret;
@@ -467,6 +468,8 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, .type = viommu_type, .dev_id = dev_id, .hwpt_id = hwpt_id, + .data_len = len, + .data_uptr = (uintptr_t)data_ptr, }; ret =ioctl (be->fd,IOMMU_VIOMMU_ALLOC , &alloc_viommu);
hw/arm/smmuv3-accel.c +2 -2
@@ -503,8 +503,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,SMMUv3AccelState *accel;if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , - s2_hwpt_id, &viommu_id, errp)) { +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, +NULL ,0 , &viommu_id, errp)) {return false; }
include/system/iommufd.h +1 -0
@@ -87,6 +87,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id, Error **errp);bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t len,uint32_t *out_hwpt,Error **errp); bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
Signature · 1 line
2.43.0
Re: [RFC PATCH 02/16] backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr
backends/iommufd.c bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, · line 473
Quoted 13 lines · Shameer Kolothum
bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be,
uint32_tdev_id,uint32_t viommu_type, uint32_t hwpt_id,backends/iommufd.c +3
-0+ void *data_ptr, uint32_t len, uint32_t *out_viommu_id, Error **errp) { int ret;@@ -467,6 +468,8 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, .type = viommu_type, .dev_id = dev_id, .hwpt_id = hwpt_id, + .data_len = len, + .data_uptr = (uintptr_t)data_ptr, };
Perhaps we should update the trace as well, given that we do print data_len/ptr for iommufd_backend_alloc_hwpt().
Nicolin
Re: [RFC PATCH 02/16] backends/iommufd: Update iommufd_backend_alloc_viommu to allow user ptr
In reply to Shameer KolothumQuoted 10 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
The updated IOMMUFD
VIOMMU_ALLOCuAPI allows userspace to provide a data buffer when creating a vIOMMU (e.g. for Tegra241 CMDQV). Extendiommufd_backend_alloc_viommu()to pass a user pointer and size to the kernel.Update the caller accordingly.
Signed-off-by Nicolin Chen <[email protected]>
with or without trace update:
Eric
Quoted 55 lines
Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 3 +++ hw/arm/smmuv3-accel.c | 4 ++-- include/system/iommufd.h | 1 + 3 files changed, 6 insertions(+), 2 deletions(-)backends/iommufd.c +3
-0@@ -459,6 +459,7 @@ bool iommufd_backend_invalidate_cache(IOMMUFDBackend *be, uint32_t id,bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t len,uint32_t *out_viommu_id,Error **errp) {int ret;@@ -467,6 +468,8 @@ bool iommufd_backend_alloc_viommu(IOMMUFDBackend *be, uint32_t dev_id, .type = viommu_type, .dev_id = dev_id, .hwpt_id = hwpt_id, + .data_len = len, + .data_uptr = (uintptr_t)data_ptr, }; ret =ioctl (be->fd,IOMMU_VIOMMU_ALLOC , &alloc_viommu);hw/arm/smmuv3-accel.c +2
-2@@ -503,8 +503,8 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,SMMUv3AccelState *accel;if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, -IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , - s2_hwpt_id, &viommu_id, errp)) { +IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id, +NULL ,0 , &viommu_id, errp)) {return false; }include/system/iommufd.h +1
-0@@ -87,6 +87,7 @@ bool iommufd_backend_alloc_hwpt(IOMMUFDBackend *be, uint32_t dev_id, Error **errp);bool iommufd_backend_alloc_viommu (IOMMUFDBackend *be,uint32_t dev_id,uint32_t viommu_type,uint32_t hwpt_id, +void *data_ptr,uint32_t len,uint32_t *out_hwpt,Error **errp); bool iommufd_backend_alloc_vdev(IOMMUFDBackend *be, uint32_t dev_id,
RFC PATCH 3/16 backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add a helper to allocate an iommufd backed HW queue for a vIOMMU.
While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.
--- backends/iommufd.c | 31 +++++++++++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 11 +++++++++++ 3 files changed, 43 insertions(+)
backends/iommufd.c +31 -0
@@ -544,6 +544,37 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +uint32_t data_type,uint32_t index, +uint64_t addr,uint64_t length, +uint32_t *out_hw_queue_id,Error **errp) +{ +int ret; +struct iommu_hw_queue_alloc alloc_hw_queue = { + .size =sizeof (alloc_hw_queue), + .flags =0 , + .viommu_id = viommu_id, + .type = data_type, + .index = index, + .nesting_parent_iova = addr, + .length = length, + }; + + ret =ioctl (be->fd,IOMMU_HW_QUEUE_ALLOC , &alloc_hw_queue); + +trace_iommufd_backend_alloc_hw_queue (be->fd, viommu_id, data_type, + index, addr, length, + alloc_hw_queue.out_hw_queue_id, ret); +if (ret) { +error_setg_errno (errp, errno,"IOMMU_HW_QUEUE_ALLOC failed" ); +return false; + } + +g_assert (out_hw_queue_id); + *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id; +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {
backends/trace-events +1 -0
@@ -24,3 +24,4 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" +iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t vqueue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t vqueue_id, int ret) " iommufd=%d viommu_id=%u vqueue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" vqueue_id=%u (%d)"
include/system/iommufd.h +11 -0
@@ -63,6 +63,12 @@ typedef struct IOMMUFDVeventq {uint32_t veventq_fd; } IOMMUFDVeventq; +/* HW queue object for a vIOMMU-specific HW-accelerated queue */ +typedef struct IOMMUFDHWqueue { +IOMMUFDViommu *viommu; +uint32_t hw_queue_id; +}IOMMUFDHWqueue ; +bool iommufd_backend_connect (IOMMUFDBackend *be,Error **errp);void iommufd_backend_disconnect (IOMMUFDBackend *be);
@@ -99,6 +105,11 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,uint32_t *out_veventq_id, uint32_t *out_veventq_fd, Error **errp); +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +uint32_t data_type,uint32_t index, +uint64_t addr,uint64_t length, +uint32_t *out_hw_queue_id,Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Signature · 1 line
2.43.0
Re: [RFC PATCH 03/16] backends/iommufd: Introduce iommufd_backend_alloc_hw_queue
Hi Shameer,
Quoted 20 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add a helper to allocate an iommufd backed HW queue for a vIOMMU.
While at it, define a struct IOMMUFDHWqueue for use by vendor implementations.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- backends/iommufd.c | 31 +++++++++++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 11 +++++++++++ 3 files changed, 43 insertions(+)backends/iommufd.c +0
-0
backends/iommufd.c bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id, · line 548
Quoted 5 lines
backends/iommufd.c +2
-0@@ -544,6 +544,37 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, + uint32_t data_type, uint32_t index,
this is named type in the uapi
Quoted 37 lines
backends/iommufd.c +29
-0+ uint64_t addr, uint64_t length, + uint32_t *out_hw_queue_id, Error **errp) +{ + int ret; +struct iommu_hw_queue_alloc alloc_hw_queue = { + .size =sizeof (alloc_hw_queue), + .flags =0 , + .viommu_id = viommu_id, + .type = data_type, + .index = index, + .nesting_parent_iova = addr, + .length = length, + }; + + ret =ioctl (be->fd,IOMMU_HW_QUEUE_ALLOC , &alloc_hw_queue); + +trace_iommufd_backend_alloc_hw_queue (be->fd, viommu_id, data_type, + index, addr, length, + alloc_hw_queue.out_hw_queue_id, ret); +if (ret) { +error_setg_errno (errp, errno,"IOMMU_HW_QUEUE_ALLOC failed" ); +return false; + } + +g_assert (out_hw_queue_id); + *out_hw_queue_id = alloc_hw_queue.out_hw_queue_id; +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {backends/trace-events +0
-0
backends/trace-events iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u · line 27
Quoted 4 lines
backends/trace-events +1
-0@@ -24,3 +24,4 @@ iommufd_backend_invalidate_cache(int iommufd, uint32_t id, uint32_t data_type, u iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32_t hwpt_id, uint32_t viommu_id, int ret) " iommufd=%d type=%u dev_id=%u hwpt_id=%u viommu_id=%u (%d)" iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" +iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t vqueue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t vqueue_id, int ret) " iommufd=%d viommu_id=%u vqueue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" vqueue_id=%u (%d)"
prefer using length instead of size to stick to the uapi terms
Quoted 19 lines
include/system/iommufd.h +6
-0@@ -63,6 +63,12 @@ typedef struct IOMMUFDVeventq {uint32_t veventq_fd; } IOMMUFDVeventq; +/* HW queue object for a vIOMMU-specific HW-accelerated queue */ +typedef struct IOMMUFDHWqueue { +IOMMUFDViommu *viommu; +uint32_t hw_queue_id; +}IOMMUFDHWqueue ; +bool iommufd_backend_connect (IOMMUFDBackend *be,Error **errp);void iommufd_backend_disconnect (IOMMUFDBackend *be);@@ -99,6 +105,11 @@ bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id, uint32_t *out_veventq_id,
include/system/iommufd.h bool iommufd_backend_alloc_veventq(IOMMUFDBackend *be, uint32_t viommu_id, · line 115
Quoted 10 lines
include/system/iommufd.h +5
-0uint32_t *out_veventq_fd, Error **errp); +bool iommufd_backend_alloc_hw_queue (IOMMUFDBackend *be,uint32_t viommu_id, +uint32_t data_type,uint32_t index, +uint64_t addr,uint64_t length, +uint32_t *out_hw_queue_id,Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Besides
Eric
RFC PATCH 4/16 backends/iommufd: Introduce iommufd_backend_viommu_mmap
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.
The caller is responsible for unmapping the returned region.
--- backends/iommufd.c | 24 ++++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 4 ++++ 3 files changed, 29 insertions(+)
backends/iommufd.c +24 -0
@@ -575,6 +575,30 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +/* +* Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance. +* The caller is responsible for unmapping the mapped region. +*/ +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp) +{ +g_assert (viommu_id); +g_assert (out_ptr); + + *out_ptr =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , be->fd, + offset); +if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"failed to mmap (size=0x%" PRIx64 +" offset=0x%" PRIx64") for viommu (id=%d)" , + size, offset, viommu_id); +return false; + } + +trace_iommufd_backend_viommu_mmap (be->fd, viommu_id, size, offset); +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {
backends/trace-events +1 -0
@@ -25,3 +25,4 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t vqueue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t vqueue_id, int ret) " iommufd=%d viommu_id=%u vqueue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" vqueue_id=%u (%d)" +iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64
include/system/iommufd.h +4 -0
@@ -110,6 +110,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, uint64_t addr, uint64_t length, uint32_t *out_hw_queue_id, Error **errp); +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
Signature · 1 line
2.43.0
Re: [RFC PATCH 04/16] backends/iommufd: Introduce iommufd_backend_viommu_mmap
Quoted 7 lines · Shameer Kolothum
backends/iommufd.c +7
-0+bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp) +{ +g_assert (viommu_id); +g_assert (out_ptr); +
backends/iommufd.c bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, · line 598
Quoted 10 lines
backends/iommufd.c +10
-0+ *out_ptr =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , be->fd, + offset); +if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"failed to mmap (size=0x%" PRIx64 +" offset=0x%" PRIx64") for viommu (id=%d)" , + size, offset, viommu_id); +return false; + } + +trace_iommufd_backend_viommu_mmap (be->fd, viommu_id, size, offset);
Like the other two APIs, maybe move the trace next to mmap() and simplify the error_setg_errno()?
Nicolin
Re: [RFC PATCH 04/16] backends/iommufd: Introduce iommufd_backend_viommu_mmap
In reply to Shameer KolothumQuoted 10 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add a backend helper to mmap hardware MMIO regions exposed via iommufd for a vIOMMU instance. This allows user space to access HW-accelerated MMIO pages provided by the vIOMMU.
The caller is responsible for unmapping the returned region.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>
Eric
Quoted 65 lines
--- backends/iommufd.c | 24 ++++++++++++++++++++++++ backends/trace-events | 1 + include/system/iommufd.h | 4 ++++ 3 files changed, 29 insertions(+)backends/iommufd.c +24
-0@@ -575,6 +575,30 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id,return true; } +/* +* Helper to mmap HW MMIO regions exposed via iommufd for a vIOMMU instance. +* The caller is responsible for unmapping the mapped region. +*/ +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp) +{ +g_assert (viommu_id); +g_assert (out_ptr); + + *out_ptr =mmap (NULL , size,PROT_READ |PROT_WRITE ,MAP_SHARED , be->fd, + offset); +if (*out_ptr ==MAP_FAILED ) { +error_setg_errno (errp, errno,"failed to mmap (size=0x%" PRIx64 +" offset=0x%" PRIx64") for viommu (id=%d)" , + size, offset, viommu_id); +return false; + } + +trace_iommufd_backend_viommu_mmap (be->fd, viommu_id, size, offset); +return true; +} +bool host_iommu_device_iommufd_attach_hwpt (HostIOMMUDeviceIOMMUFD *idev,uint32_t hwpt_id,Error **errp) {backends/trace-events +1
-0@@ -25,3 +25,4 @@ iommufd_backend_alloc_viommu(int iommufd, uint32_t dev_id, uint32_t type, uint32 iommufd_backend_alloc_vdev(int iommufd, uint32_t dev_id, uint32_t viommu_id, uint64_t virt_id, uint32_t vdev_id, int ret) " iommufd=%d dev_id=%u viommu_id=%u virt_id=0x%"PRIx64" vdev_id=%u (%d)" iommufd_viommu_alloc_eventq(int iommufd, uint32_t viommu_id, uint32_t type, uint32_t veventq_id, uint32_t veventq_fd, int ret) " iommufd=%d viommu_id=%u type=%u veventq_id=%u veventq_fd=%u (%d)" iommufd_backend_alloc_hw_queue(int iommufd, uint32_t viommu_id, uint32_t vqueue_type, uint32_t index, uint64_t addr, uint64_t size, uint32_t vqueue_id, int ret) " iommufd=%d viommu_id=%u vqueue_type=%u index=%u addr=0x%"PRIx64" size=0x%"PRIx64" vqueue_id=%u (%d)" +iommufd_backend_viommu_mmap(int iommufd, uint32_t viommu_id, uint64_t size, uint64_t offset) " iommufd=%d viommu_id=%u size=0x%"PRIx64" offset=0x%"PRIx64include/system/iommufd.h +4
-0@@ -110,6 +110,10 @@ bool iommufd_backend_alloc_hw_queue(IOMMUFDBackend *be, uint32_t viommu_id, uint64_t addr, uint64_t length, uint32_t *out_hw_queue_id, Error **errp); +bool iommufd_backend_viommu_mmap (IOMMUFDBackend *be,uint32_t viommu_id, +uint64_t size,off_t offset,void **out_ptr, +Error **errp); +bool iommufd_backend_set_dirty_tracking (IOMMUFDBackend *be,uint32_t hwpt_id,bool start,Error **errp); bool iommufd_backend_get_dirty_bitmap(IOMMUFDBackend *be, uint32_t hwpt_id,
RFC PATCH 5/16 hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Introduce initial support for NVIDIA Tegra241 CMDQ-Virtualisation (CMDQV), an extension to SMMUv3 providing virtualizable hardware command queues. This adds the basic MMIO handling, and integration hooks in the SMMUv3 accelerated path. When enabled, the SMMUv3 backend allocates a Tegra241 specific vIOMMU object via IOMMUFD and exposes a CMDQV MMIO region and IRQ to the guest.
The "tegra241-cmdqv" property isn't user visible yet and it will be introduced in a later patch once all the supporting pieces are ready.
--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 10 +++++-- hw/arm/smmuv3.c | 4 +++ hw/arm/tegra241-cmdqv.c | 65 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 40 +++++++++++++++++++++++++ include/hw/arm/smmuv3.h | 3 ++ 7 files changed, 126 insertions(+), 2 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.h
hw/arm/Kconfig +5 -0
@@ -37,6 +37,7 @@ config ARM_VIRT select VIRTIO_MEM_SUPPORTED select ACPI_CXL select ACPI_HMAT + select TEGRA241_CMDQV config CUBIEBOARD bool
@@ -634,6 +635,10 @@ config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3 && IOMMUFD +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config FSL_IMX6UL bool default y
hw/arm/meson.build +1 -0
@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) +arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(
hw/arm/smmuv3-accel.c +8 -2
@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem
@@ -499,10 +500,15 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, .ste = { SMMU_STE_VALID, 0x0ULL }, };uint32_t s2_hwpt_id = idev->hwpt_id; -uint32_t viommu_id, hwpt_id; +uint32_t viommu_id =0 , hwpt_id;SMMUv3AccelState *accel; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +if (s->tegra241_cmdqv && !tegra241_cmdqv_alloc_viommu (s, idev, &viommu_id, + errp)) { +return false; + } + +if (!viommu_id && !iommufd_backend_alloc_viommu (idev->iommufd, idev->devid,IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id,NULL ,0 , &viommu_id, errp)) {return false;
hw/arm/smmuv3.c +4 -0
@@ -36,6 +36,7 @@#include "smmuv3-accel.h" #include "smmuv3-internal.h" #include "smmu-internal.h" +#include "tegra241-cmdqv.h" #define PTW_RECORD_FAULT (ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \ (cfg)->record_faults) || \
@@ -2017,6 +2018,9 @@ static void smmu_realize(DeviceState *d, Error **errp)smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +if (s->tegra241_cmdqv) { +tegra241_cmdqv_init (s); + } }static const VMStateDescription vmstate_smmuv3_queue = {
hw/arm/tegra241-cmdqv.c new file +65 -0
65 lines not on this page · raw message
hw/arm/tegra241-cmdqv.h new file +40 -0
@@ -0,0 +1,40 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_TEGRA241_CMDQV_H +#define HW_TEGRA241_CMDQV_H + +#include CONFIG_DEVICES + +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3State *smmu; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; + +#ifdef CONFIG_TEGRA241_CMDQV +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp); +void tegra241_cmdqv_init (SMMUv3State *s); +#else +static inline void tegra241_cmdqv_init (SMMUv3State *s) +{ +} +static inline bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +return true; +} +#endif + +#endif /* HW_TEGRA241_CMDQV_H */
include/hw/arm/smmuv3.h +3 -0
@@ -73,6 +73,9 @@ struct SMMUv3State {bool ats;uint8_t oas;bool pasid; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +struct Tegra241CMDQV *cmdqv; +bool tegra241_cmdqv; }; typedef enum {
Signature · 1 line
2.43.0
Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
hw/arm/tegra241-cmdqv.c line 56
Quoted 8 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c new file +8
-0+void tegra241_cmdqv_init (SMMUv3State *s) +{ +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +Tegra241CMDQV *cmdqv; + +if (!s->tegra241_cmdqv) { +return ; + }
Maybe g_assert?
hw/arm/tegra241-cmdqv.h line 19
Quoted 3 lines
hw/arm/tegra241-cmdqv.h new file +3
-0+typedef struct Tegra241CMDQV { + struct iommu_viommu_tegra241_cmdqv cmdqv_data; + SMMUv3State *smmu;
I see all the cmdqv functions want "smmu->s_accel", so maybe store "s_accel" instead?
hw/arm/tegra241-cmdqv.h line 36
Quoted 13 lines
hw/arm/tegra241-cmdqv.h new file +13
-0+#ifdef CONFIG_TEGRA241_CMDQV +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp); +void tegra241_cmdqv_init (SMMUv3State *s); +#else +static inline void tegra241_cmdqv_init (SMMUv3State *s) +{ +} +static inline bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +return true;
Should it return false?
include/hw/arm/smmuv3.h struct SMMUv3State { · line 78
Quoted 10 lines
index
2d4970fe19..8e56e480a0100644include/hw/arm/smmuv3.h +3
-0@@ -73,6 +73,9 @@ struct SMMUv3State {bool ats;uint8_t oas;bool pasid; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +struct Tegra241CMDQV *cmdqv; +bool tegra241_cmdqv;
tegra241_cmdqv is a Property, so it has to stay with SMMUv3State.
But "struct Tegra241CMDQV *cmdqv" might be in the SMMUv3AccelState?
Nicolin
RE: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
Quoted 16 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 19:07
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation supportOn Wed, Dec 10, 2025 at 01:37:26PM +0000, Shameer Kolothum wrote:
Quoted 10 lines
hw/arm/tegra241-cmdqv.c line 56
Quoted 8 lines
hw/arm/tegra241-cmdqv.c new file +8
-0+void tegra241_cmdqv_init (SMMUv3State *s) +{ +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +Tegra241CMDQV *cmdqv; + +if (!s->tegra241_cmdqv) { +return ; + }Maybe
g_assert?
Sure.
Quoted 7 lines
hw/arm/tegra241-cmdqv.h line 19
Quoted 4 lines
hw/arm/tegra241-cmdqv.h new file +3
-0+typedef struct Tegra241CMDQV { + struct iommu_viommu_tegra241_cmdqv cmdqv_data; + SMMUv3State *smmu;I see all the cmdqv functions want "
smmu->s_accel", so maybe store "s_accel" instead?
Yes. But see below.
Quoted 12 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+#ifdef CONFIG_TEGRA241_CMDQV +bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.h
Quoted 8 lines
hw/arm/tegra241-cmdqv.h new file +8
-0+ uint32_t *out_viommu_id, Error **errp); +void tegra241_cmdqv_init (SMMUv3State *s); +#else +static inline void tegra241_cmdqv_init (SMMUv3State *s) +{ +} +static inline bool +tegra241_cmdqv_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
Quoted 5 lines
hw/arm/tegra241-cmdqv.c line 46
Quoted 3 lines
hw/arm/tegra241-cmdqv.c new file +3
-0+ uint32_t *out_viommu_id, Error **errp) +{ +return true;Should it return false?
Hmm..it should be, given how it is invoked here.
Quoted 14 lines
include/hw/arm/smmuv3.h struct SMMUv3State { · line 78
Quoted 11 lines
index
2d4970fe19..8e56e480a0100644include/hw/arm/smmuv3.h +3
-0@@ -73,6 +73,9 @@ struct SMMUv3State {bool ats;uint8_t oas;bool pasid; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +struct Tegra241CMDQV *cmdqv; +bool tegra241_cmdqv;
tegra241_cmdqvis a Property, so it has to stay with SMMUv3State.
Quoted 1 line
But "struct Tegra241CMDQV *cmdqv" might be in the SMMUv3AccelState?
I have considered this. Currently, SMMUv3AccelState is allocated in smmuv3_accel_alloc_viommu(), which happens later than tegra241_cmdqv_init().
Changing this would require some rework. I will look into whether we can address that as part of the next respin of the SMMUv3 accel series to make this cleaner.
Thanks,
Shameer
Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
In reply to Shameer KolothumQuoted 75 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Introduce initial support for NVIDIA Tegra241 CMDQ-Virtualisation (CMDQV), an extension to SMMUv3 providing virtualizable hardware command queues. This adds the basic MMIO handling, and integration hooks in the SMMUv3 accelerated path. When enabled, the SMMUv3 backend allocates a Tegra241 specific vIOMMU object via IOMMUFD and exposes a CMDQV MMIO region and IRQ to the guest.
The "tegra241-cmdqv" property isn't user visible yet and it will be introduced in a later patch once all the supporting pieces are ready.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 10 +++++-- hw/arm/smmuv3.c | 4 +++ hw/arm/tegra241-cmdqv.c | 65 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 40 +++++++++++++++++++++++++ include/hw/arm/smmuv3.h | 3 ++ 7 files changed, 126 insertions(+), 2 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.hhw/arm/Kconfig +5
-0@@ -37,6 +37,7 @@ config ARM_VIRT select VIRTIO_MEM_SUPPORTED select ACPI_CXL select ACPI_HMAT + select TEGRA241_CMDQV config CUBIEBOARD bool@@ -634,6 +635,10 @@ config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3 && IOMMUFD +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config FSL_IMX6UL bool default yhw/arm/meson.build +1
-0@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) +arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(hw/arm/smmuv3-accel.c +1
-0@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem @@ -499,10 +500,15 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, .ste = { SMMU_STE_VALID, 0x0ULL },
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 508
Quoted 10 lines
hw/arm/smmuv3-accel.c +4
-2};uint32_t s2_hwpt_id = idev->hwpt_id; -uint32_t viommu_id, hwpt_id; +uint32_t viommu_id =0 , hwpt_id;SMMUv3AccelState *accel; - if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, +if (s->tegra241_cmdqv && !tegra241_cmdqv_alloc_viommu (s, idev, &viommu_id, + errp)) { +return false;
I am confused. In tegra241_cmdqv_alloc_viommu() it returns false if alloc_viommu fails. but you seem to reset s->tegra241_cmdqv as if you would fall back to non cmdqv setup. What do you try do, fallback or execute either tegra241 code or default code. Or maybe I misunderstand the uapi call sequence?
I would not reset a property value in general under the hood. If the end user has set up this option, I guess he expects it to be enabled when he queries it back, no?
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 513
Quoted 5 lines
hw/arm/smmuv3-accel.c +3
-0+ } + +if (!viommu_id && !iommufd_backend_alloc_viommu (idev->iommufd, idev->devid,IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id,NULL ,0 , &viommu_id, errp)) {
If this is a specialization of alloc_viommu code, it generally points to a class or ops interface. You may face such kind of comments later on so better to justify that choice or adopt a new one ;-)
same for init which is overloaded compared to original implementation.
Quoted 145 lines
hw/arm/smmuv3-accel.c +0
-0return false;hw/arm/smmuv3.c +4
-0@@ -36,6 +36,7 @@#include "smmuv3-accel.h" #include "smmuv3-internal.h" #include "smmu-internal.h" +#include "tegra241-cmdqv.h" #define PTW_RECORD_FAULT (ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \ (cfg)->record_faults) || \@@ -2017,6 +2018,9 @@ static void smmu_realize(DeviceState *d, Error **errp)smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +if (s->tegra241_cmdqv) { +tegra241_cmdqv_init (s); + } }static const VMStateDescription vmstate_smmuv3_queue = {hw/arm/tegra241-cmdqv.c new file +65
-0@@ -0,0 +1,65 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} + +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; + +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +Tegra241CMDQV *cmdqv = s->cmdqv; + +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV , + idev->hwpt_id, &cmdqv->cmdqv_data, +sizeof (cmdqv->cmdqv_data), out_viommu_id, + errp)) { +error_append_hint (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); + s->tegra241_cmdqv = false; +return false; + } +return true; +} + +void tegra241_cmdqv_init (SMMUv3State *s) +{ +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +Tegra241CMDQV *cmdqv; + +if (!s->tegra241_cmdqv) { +return ; + } + + cmdqv =g_new0 (Tegra241CMDQV,1 ); +memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->smmu = s; + s->cmdqv = cmdqv; +}hw/arm/tegra241-cmdqv.h new file +40
-0@@ -0,0 +1,40 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_TEGRA241_CMDQV_H +#define HW_TEGRA241_CMDQV_H + +#include CONFIG_DEVICES + +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3State *smmu; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; + +#ifdef CONFIG_TEGRA241_CMDQV +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp); +void tegra241_cmdqv_init (SMMUv3State *s); +#else +static inline void tegra241_cmdqv_init (SMMUv3State *s) +{ +} +static inline bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +return true; +} +#endif + +#endif /* HW_TEGRA241_CMDQV_H */ include/hw/arm/smmuv3.h +0
-0
include/hw/arm/smmuv3.h struct SMMUv3State { · line 81
Quoted 9 lines
include/hw/arm/smmuv3.h +3
-0@@ -73,6 +73,9 @@ struct SMMUv3State {bool ats;uint8_t oas;bool pasid; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +struct Tegra241CMDQV *cmdqv; +bool tegra241_cmdqv; }; typedef enum {
Thanks
Eric
RE: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
Hi Eric,
Quoted 112 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 January 2026 15:29
To: Shameer Kolothum <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation supportExternal email: Use caution opening links or attachments
Quoted 3 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Introduce initial support for NVIDIA Tegra241 CMDQ-Virtualisation
(CMDQV),
Quoted 1 line
an extension to SMMUv3 providing virtualizable hardware command
queues.
Quoted 3 lines
This adds the basic MMIO handling, and integration hooks in the SMMUv3 accelerated path. When enabled, the SMMUv3 backend allocates a Tegra241 specific vIOMMU object via IOMMUFD and exposes a CMDQV MMIO region
and
Quoted 5 lines
IRQ to the guest.
The "tegra241-cmdqv" property isn't user visible yet and it will be introduced in a later patch once all the supporting pieces are ready.
Quoted 8 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 10 +++++-- hw/arm/smmuv3.c | 4 +++ hw/arm/tegra241-cmdqv.c | 65+++++++++++++++++++++++++++++++++++++++++
hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP',
Quoted 34 lines
hw/arm/tegra241-cmdqv.h | 40 +++++++++++++++++++++++++ include/hw/arm/smmuv3.h | 3 ++ 7 files changed, 126 insertions(+), 2 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.hhw/arm/Kconfig +5
-0@@ -37,6 +37,7 @@ config ARM_VIRT select VIRTIO_MEM_SUPPORTED select ACPI_CXL select ACPI_HMAT + select TEGRA241_CMDQV config CUBIEBOARD bool@@ -634,6 +635,10 @@ config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3 && IOMMUFD +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config FSL_IMX6UL bool default yhw/arm/meson.build +0
-0@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP',
if_true: files('fsl-imx8mp.c'))hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', · line 86
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true:files('imx8mp-evk.c'))
hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', · line 88
Quoted 2 lines
hw/arm/meson.build +0
-0arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true:files('smmuv3-accel.c'))
hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', · line 89
Quoted 1 line
hw/arm/meson.build +1
-0+arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-
cmdqv.c'))hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', · line 90
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-
imx6ul.c', 'mcimx6ul-evk.c'))hw/arm/meson.build arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', · line 91
Quoted 1 line
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true:
files('nrf51_soc.c'))hw/arm/smmuv3-accel.c line 24
Quoted 13 lines
hw/arm/meson.build +0
-0arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(hw/arm/smmuv3-accel.c +1
-0@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and
shared_as_sysmemhw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s,
Quoted 1 line
hw/arm/smmuv3-accel.c +0
-0@@ -499,10 +500,15 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Quoted 9 lines
hw/arm/smmuv3-accel.c +2
-2.ste = { SMMU_STE_VALID, 0x0ULL }, }; uint32_t s2_hwpt_id = idev->hwpt_id; - uint32_t viommu_id, hwpt_id; + uint32_t viommu_id = 0, hwpt_id; SMMUv3AccelState *accel; - if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + if (s->tegra241_cmdqv && !tegra241_cmdqv_alloc_viommu(s, idev,
Quoted 8 lines
&viommu_id,hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, · line 508
Quoted 2 lines
hw/arm/smmuv3-accel.c +2
-0+ errp)) { +return false;I am confused. In
tegra241_cmdqv_alloc_viommu()it returns false ifalloc_viommufails. but you seem to resets->tegra241_cmdqvas if you would fall back to non cmdqv setup. What do you try do, fallback or execute either tegra241 code or default code. Or maybe I misunderstand the uapi call sequence?
No fallback intended. Currently, if the user has enabled tegra241_cmdqv and tegra241_cmdqv_alloc_viommu() fails, we fail the device init. Sorry about that to reset s->tegra241_cmdqv and !viommu_id logic, , that was a left over logic from previous internal branch I had.
@Nicolin, is there any such requirement for a fallback in this case?
Quoted 3 lines
I would not reset a property value in general under the hood. If the end user has set up this option, I guess he expects it to be enabled when he queries it back, no?
Yes, that’s the idea.
Quoted 11 lines
hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Quoted 3 lines
hw/arm/smmuv3-accel.c +3
-0+ } + + if (!viommu_id && !iommufd_backend_alloc_viommu(idev->iommufd,
idev->devid,hw/arm/smmuv3-accel.c smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, · line 513
Quoted 2 lines
hw/arm/smmuv3-accel.c +0
-0IOMMU_VIOMMU_TYPE_ARM_SMMUV3, s2_hwpt_id, NULL, 0, &viommu_id, errp)) {If this is a specialization of
alloc_viommucode, it generally points to a class or ops interface. You may face such kind of comments later on so better to justify that choice or adopt a new one ;-)
same for init which is overloaded compared to original implementation.
I agree. I will take another look at this. Also, this is where any help would be really appreciated 😊. Please let me know if you have some ideas on how we can improve this.
Thanks,
Shameer
Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
Quoted 12 lines · Shameer Kolothum
Quoted 11 lines
Review
Quoted 2 lines
(quoted diff) +1
-1- if (!iommufd_backend_alloc_viommu(idev->iommufd, idev->devid, + if (s->tegra241_cmdqv && !tegra241_cmdqv_alloc_viommu(s, idev,
&viommu_id,Review
Quoted 2 lines
(quoted diff) +2
-0+ errp)) { + return false;I am confused. In
tegra241_cmdqv_alloc_viommu()it returns false ifalloc_viommufails. but you seem to resets->tegra241_cmdqvas if you would fall back to non cmdqv setup. What do you try do, fallback or execute either tegra241 code or default code. Or maybe I misunderstand the uapi call sequence?
Quoted 6 lines
No fallback intended. Currently, if the user has enabled
tegra241_cmdqvandtegra241_cmdqv_alloc_viommu()fails, we fail the device init. Sorry about that to resets->tegra241_cmdqvand !viommu_id logic, , that was a left over logic from previous internal branch I had.@Nicolin, is there any such requirement for a fallback in this case?
Likely no. I agree that we should do in the cleaner way.
Nicolin
Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
In reply to Shameer KolothumHi Shameer,
Quoted 200 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Introduce initial support for NVIDIA Tegra241 CMDQ-Virtualisation (CMDQV), an extension to SMMUv3 providing virtualizable hardware command queues. This adds the basic MMIO handling, and integration hooks in the SMMUv3 accelerated path. When enabled, the SMMUv3 backend allocates a Tegra241 specific vIOMMU object via IOMMUFD and exposes a CMDQV MMIO region and IRQ to the guest.
The "tegra241-cmdqv" property isn't user visible yet and it will be introduced in a later patch once all the supporting pieces are ready.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 10 +++++-- hw/arm/smmuv3.c | 4 +++ hw/arm/tegra241-cmdqv.c | 65 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 40 +++++++++++++++++++++++++ include/hw/arm/smmuv3.h | 3 ++ 7 files changed, 126 insertions(+), 2 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.hhw/arm/Kconfig +5
-0@@ -37,6 +37,7 @@ config ARM_VIRT select VIRTIO_MEM_SUPPORTED select ACPI_CXL select ACPI_HMAT + select TEGRA241_CMDQV config CUBIEBOARD bool@@ -634,6 +635,10 @@ config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3 && IOMMUFD +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config FSL_IMX6UL bool default yhw/arm/meson.build +1
-0@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) +arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(hw/arm/smmuv3-accel.c +8
-2@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem @@ -499,10 +500,15 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, .ste = { SMMU_STE_VALID, 0x0ULL }, };uint32_t s2_hwpt_id = idev->hwpt_id; -uint32_t viommu_id, hwpt_id; +uint32_t viommu_id =0 , hwpt_id;SMMUv3AccelState *accel; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +if (s->tegra241_cmdqv && !tegra241_cmdqv_alloc_viommu (s, idev, &viommu_id, + errp)) { +return false; + } + +if (!viommu_id && !iommufd_backend_alloc_viommu (idev->iommufd, idev->devid,IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id,NULL ,0 , &viommu_id, errp)) {return false;hw/arm/smmuv3.c +4
-0@@ -36,6 +36,7 @@#include "smmuv3-accel.h" #include "smmuv3-internal.h" #include "smmu-internal.h" +#include "tegra241-cmdqv.h" #define PTW_RECORD_FAULT (ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \ (cfg)->record_faults) || \@@ -2017,6 +2018,9 @@ static void smmu_realize(DeviceState *d, Error **errp)smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +if (s->tegra241_cmdqv) { +tegra241_cmdqv_init (s); + } }static const VMStateDescription vmstate_smmuv3_queue = {hw/arm/tegra241-cmdqv.c new file +65
-0@@ -0,0 +1,65 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} + +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; + +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +Tegra241CMDQV *cmdqv = s->cmdqv; + +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV , + idev->hwpt_id, &cmdqv->cmdqv_data, +sizeof (cmdqv->cmdqv_data), out_viommu_id, + errp)) { +error_append_hint (errp,"NVIDIA Tegra241 CMDQV is unsupported" ); + s->tegra241_cmdqv = false; +return false; + } +return true; +} + +void tegra241_cmdqv_init (SMMUv3State *s) +{ +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +Tegra241CMDQV *cmdqv; + +if (!s->tegra241_cmdqv) { +return ; + } + + cmdqv =g_new0 (Tegra241CMDQV,1 ); +memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->smmu = s; + s->cmdqv = cmdqv; +}hw/arm/tegra241-cmdqv.h new file +10
-0@@ -0,0 +1,40 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_TEGRA241_CMDQV_H
hw/arm/tegra241-cmdqv.h line 20
Quoted 10 lines
hw/arm/tegra241-cmdqv.h new file +10
-0+#define HW_TEGRA241_CMDQV_H + +#include CONFIG_DEVICES + +#define TEGRA241_CMDQV_IO_LEN 0x50000 + +typedef struct Tegra241CMDQV { + struct iommu_viommu_tegra241_cmdqv cmdqv_data; + SMMUv3State *smmu; + MemoryRegion mmio_cmdqv;
please use a _mr suffix. It eases the review of subsequent patches
Eric
Quoted 34 lines
hw/arm/tegra241-cmdqv.h new file +20
-0+ qemu_irq irq; +} Tegra241CMDQV; + +#ifdef CONFIG_TEGRA241_CMDQV +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp); +void tegra241_cmdqv_init (SMMUv3State *s); +#else +static inline void tegra241_cmdqv_init (SMMUv3State *s) +{ +} +static inline bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +return true; +} +#endif + +#endif /* HW_TEGRA241_CMDQV_H */ include/hw/arm/smmuv3.h +3
-0@@ -73,6 +73,9 @@ struct SMMUv3State {bool ats;uint8_t oas;bool pasid; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +struct Tegra241CMDQV *cmdqv; +bool tegra241_cmdqv; }; typedef enum {
Re: [RFC PATCH 05/16] hw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation support
In reply to Shameer KolothumQuoted 3 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Introduce initial support for NVIDIA Tegra241 CMDQ-Virtualisation (CMDQV),
well this is rather a skeleton and at this stage it does not introduce any support
Quoted 5 lines
an extension to SMMUv3 providing virtualizable hardware command queues. This adds the basic MMIO handling, and integration hooks in the SMMUv3 accelerated path. When enabled, the SMMUv3 backend allocates a Tegra241 specific vIOMMU object via IOMMUFD and exposes a CMDQV MMIO region and IRQ to the guest.
I would elaborate on the nature of region and interrupt here.
Quoted 3 lines
The "tegra241-cmdqv" property isn't user visible yet and it will be introduced in a later patch once all the supporting pieces are ready.
I would suggest "tegra241_cmdqv will be set by an SMMU property that does not exist yet. As such it is currently unset."
Quoted 108 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/Kconfig | 5 ++++ hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 10 +++++-- hw/arm/smmuv3.c | 4 +++ hw/arm/tegra241-cmdqv.c | 65 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 40 +++++++++++++++++++++++++ include/hw/arm/smmuv3.h | 3 ++ 7 files changed, 126 insertions(+), 2 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.hhw/arm/Kconfig +5
-0@@ -37,6 +37,7 @@ config ARM_VIRT select VIRTIO_MEM_SUPPORTED select ACPI_CXL select ACPI_HMAT + select TEGRA241_CMDQV config CUBIEBOARD bool@@ -634,6 +635,10 @@ config ARM_SMMUV3_ACCEL bool depends on ARM_SMMUV3 && IOMMUFD +config TEGRA241_CMDQV + bool + depends on ARM_SMMUV3_ACCEL + config FSL_IMX6UL bool default yhw/arm/meson.build +1
-0@@ -86,6 +86,7 @@ arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP', if_true: files('fsl-imx8mp.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX8MP_EVK', if_true: files('imx8mp-evk.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3', if_true: files('smmuv3.c')) arm_ss.add(when: 'CONFIG_ARM_SMMUV3_ACCEL', if_true: files('smmuv3-accel.c')) +arm_ss.add(when: 'CONFIG_TEGRA241_CMDQV', if_true: files('tegra241-cmdqv.c')) arm_common_ss.add(when: 'CONFIG_FSL_IMX6UL', if_true: files('fsl-imx6ul.c', 'mcimx6ul-evk.c')) arm_common_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_soc.c')) arm_common_ss.add(when: 'CONFIG_XEN', if_true: files(hw/arm/smmuv3-accel.c +8
-2@@ -18,6 +18,7 @@#include "smmuv3-internal.h" #include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" /* * The root region aliases the global system memory, and shared_as_sysmem @@ -499,10 +500,15 @@ smmuv3_accel_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev, .ste = { SMMU_STE_VALID, 0x0ULL }, };uint32_t s2_hwpt_id = idev->hwpt_id; -uint32_t viommu_id, hwpt_id; +uint32_t viommu_id =0 , hwpt_id;SMMUv3AccelState *accel; -if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +if (s->tegra241_cmdqv && !tegra241_cmdqv_alloc_viommu (s, idev, &viommu_id, + errp)) { +return false; + } + +if (!viommu_id && !iommufd_backend_alloc_viommu (idev->iommufd, idev->devid,IOMMU_VIOMMU_TYPE_ARM_SMMUV3 , s2_hwpt_id,NULL ,0 , &viommu_id, errp)) {return false;hw/arm/smmuv3.c +4
-0@@ -36,6 +36,7 @@#include "smmuv3-accel.h" #include "smmuv3-internal.h" #include "smmu-internal.h" +#include "tegra241-cmdqv.h" #define PTW_RECORD_FAULT (ptw_info, cfg) (((ptw_info).stage == SMMU_STAGE_1 && \ (cfg)->record_faults) || \@@ -2017,6 +2018,9 @@ static void smmu_realize(DeviceState *d, Error **errp)smmu_init_irq (s, dev);smmuv3_init_id_regs (s); +if (s->tegra241_cmdqv) { +tegra241_cmdqv_init (s); + } }static const VMStateDescription vmstate_smmuv3_queue = {hw/arm/tegra241-cmdqv.c new file +0
-0
hw/arm/tegra241-cmdqv.c line 2
Quoted 2 lines
hw/arm/tegra241-cmdqv.c new file +2
-0@@ -0,0 +1,65 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION
nit: s/2025/2026
Quoted 30 lines
hw/arm/tegra241-cmdqv.c new file +30
-0+* NVIDIA Tegra241 CMDQ-Virtualization extension for SMMUv3 +* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#include "qemu/osdep.h" + +#include "hw/arm/smmuv3.h" +#include "smmuv3-accel.h" +#include "tegra241-cmdqv.h" + +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) +{ +return 0 ; +} + +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value, +unsigned size) +{ +} + +static const MemoryRegionOps mmio_cmdqv_ops = { + .read = tegra241_cmdqv_read, + .write = tegra241_cmdqv_write, + .endianness =DEVICE_LITTLE_ENDIAN , +}; + +bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
hw/arm/tegra241-cmdqv.c line 42
Quoted 10 lines
hw/arm/tegra241-cmdqv.c new file +10
-0+ uint32_t *out_viommu_id, Error **errp) +{ + Tegra241CMDQV *cmdqv = s->cmdqv; + +if (!iommufd_backend_alloc_viommu (idev->iommufd, idev->devid, +IOMMU_VIOMMU_TYPE_TEGRA241_CMDQV , + idev->hwpt_id, &cmdqv->cmdqv_data, +sizeof (cmdqv->cmdqv_data), out_viommu_id, + errp)) { +error_append_hint (errp,"NVIDIA Tegra241 CMDQV is unsupported" );
are we sure this is the case here and not a radom failure of other source?
Quoted 29 lines
hw/arm/tegra241-cmdqv.c new file +23
-0+ s->tegra241_cmdqv = false; +return false; + } +return true; +} + +void tegra241_cmdqv_init (SMMUv3State *s) +{ +SysBusDevice *sbd =SYS_BUS_DEVICE (OBJECT (s)); +Tegra241CMDQV *cmdqv; + +if (!s->tegra241_cmdqv) { +return ; + } + + cmdqv =g_new0 (Tegra241CMDQV,1 ); +memory_region_init_io (&cmdqv->mmio_cmdqv,OBJECT (s), &mmio_cmdqv_ops, cmdqv, +"tegra241-cmdqv" ,TEGRA241_CMDQV_IO_LEN ); +sysbus_init_mmio (sbd, &cmdqv->mmio_cmdqv); +sysbus_init_irq (sbd, &cmdqv->irq); + cmdqv->smmu = s; + s->cmdqv = cmdqv; +}hw/arm/tegra241-cmdqv.h new file +0
-0
hw/arm/tegra241-cmdqv.h line 3
Quoted 3 lines
hw/arm/tegra241-cmdqv.h new file +3
-0@@ -0,0 +1,40 @@ +/* +* Copyright (C) 2025, NVIDIA CORPORATION +* NVIDIA Tegra241 CMDQ-Virtualiisation extension for SMMUv3
virtualization
hw/arm/tegra241-cmdqv.h line 15
Quoted 12 lines
hw/arm/tegra241-cmdqv.h new file +12
-0+* +* Written by Nicolin Chen, Shameer Kolothum +* +* SPDX-License-Identifier: GPL-2.0-or-later +*/ + +#ifndef HW_TEGRA241_CMDQV_H +#define HW_TEGRA241_CMDQV_H + +#include CONFIG_DEVICES + +#define TEGRA241_CMDQV_IO_LEN 0x50000
can you explain the size
Quoted 39 lines
hw/arm/tegra241-cmdqv.h new file +25
-0+ +typedef struct Tegra241CMDQV { +struct iommu_viommu_tegra241_cmdqv cmdqv_data; +SMMUv3State *smmu; +MemoryRegion mmio_cmdqv; +qemu_irq irq; +}Tegra241CMDQV ; + +#ifdef CONFIG_TEGRA241_CMDQV +bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp); +void tegra241_cmdqv_init (SMMUv3State *s); +#else +static inline void tegra241_cmdqv_init (SMMUv3State *s) +{ +} +static inline bool +tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev, +uint32_t *out_viommu_id,Error **errp) +{ +return true; +} +#endif + +#endif /* HW_TEGRA241_CMDQV_H */ include/hw/arm/smmuv3.h +3
-0@@ -73,6 +73,9 @@ struct SMMUv3State {bool ats;uint8_t oas;bool pasid; +/* Support for NVIDIA Tegra241 SMMU CMDQV extension */ +struct Tegra241CMDQV *cmdqv; +bool tegra241_cmdqv; }; typedef enum {
RFC PATCH 6/16 hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1: configuration registers (queue GPA/size) that must be trapped by the VMM and translated before programming the HW queue.
This patch implements the Page0 handling in QEMU. Using the vintf offset returned by IOMMUFD during VIOMMU allocation, QEMU maps Page0 into guest physical address space and exposes it via two guest MMIO windows:
- 0x10000 :VCMDQ register
- 0x30000 :VINTF register
The mapping is lazily initialized on first read/write.
--- hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 5 ++++ 2 files changed, 65 insertions(+)
hw/arm/tegra241-cmdqv.c +60 -0
@@ -13,14 +13,74 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_init_vcmdq_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu; +char *name; + +if (!s_accel) { +return true; + } + + viommu = &s_accel->viommu; +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VCMDQ_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vcmdq_page0, errp)) { + cmdqv->vcmdq_page0 =NULL ; +return false; + } + + name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vcmdq_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x10000 , + &cmdqv->mmio_vcmdq_page,1 ); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + &cmdqv->mmio_vintf_page,1 ); +g_free (name); + +return true; +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vcmdq_page0) { +tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err); +if (local_err) { +error_report_err (local_err); + local_err =NULL ; + } + } +return 0 ; }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vcmdq_page0) { +tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err); +if (local_err) { +error_report_err (local_err); + local_err =NULL ; + } + } }static const MemoryRegionOps mmio_cmdqv_ops = {
hw/arm/tegra241-cmdqv.h +5 -0
@@ -19,8 +19,13 @@ typedef struct Tegra241CMDQV {SMMUv3State *smmu;MemoryRegion mmio_cmdqv;qemu_irq irq; +MemoryRegion mmio_vcmdq_page; +MemoryRegion mmio_vintf_page; +void *vcmdq_page0; } Tegra241CMDQV; +#define VCMDQ_REG_PAGE_SIZE 0x10000 +#ifdef CONFIG_TEGRA241_CMDQV bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp);
Signature · 1 line
2.43.0
Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
Quoted 8 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1: configuration registers (queue GPA/size) that must be trapped by the VMM and translated before programming the HW queue.
Quoted 4 lines
This patch implements the Page0 handling in QEMU. Using the vintf offset returned by IOMMUFD during VIOMMU allocation, QEMU maps Page0 into guest physical address space and exposes it via two guest MMIO windows:
-0x10000:VCMDQ register
global VCMDQ MMIO pages.
Quoted 1 line
-
0x30000:VINTF register
private VINTF MMIO pages
hw/arm/tegra241-cmdqv.c line 25
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+static bool tegra241_cmdqv_init_vcmdq_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu; +char *name; + +if (!s_accel) { +return true; + }
g_assert?
The entire thing can't work without s_accel, so returning true doesn't seem to make sense.
hw/arm/tegra241-cmdqv.c line 32
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+ viommu = &s_accel->viommu; +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VCMDQ_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vcmdq_page0, errp)) { + cmdqv->vcmdq_page0 =NULL ;
We probably shouldn't nuke the vcmdq_page0.
And I think we should add g_assert(!cmdqv->vcmdq_page0) too. It would be a bug if we pass in a valid page0 pointer.
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+ name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vcmdq_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0);
hw/arm/tegra241-cmdqv.c line 49
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x10000 , + &cmdqv->mmio_vcmdq_page,1 ); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + &cmdqv->mmio_vintf_page,1 );
Let's add some comments here (maybe something similar in commit log also):
/*
* Each VM can only own one VINTF exposed by the kernel via a VIOMMU object.
* And all available VCMDQs are already preallocated in the VINTF. Thus, the
* global VCMDQ MMIO page0 and the private VINTF MMIO page0 are effectively
* the same, i.e. cmdqv->vcmdq_page0.
*/
Nicolin
Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
In reply to Shameer KolothumHi Shameer,
Quoted 53 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1: configuration registers (queue GPA/size) that must be trapped by the VMM and translated before programming the HW queue.This patch implements the Page0 handling in QEMU. Using the vintf offset returned by IOMMUFD during VIOMMU allocation, QEMU maps Page0 into guest physical address space and exposes it via two guest MMIO windows:
-0x10000:VCMDQ register
-0x30000:VINTF registerThe mapping is lazily initialized on first read/write.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 5 ++++ 2 files changed, 65 insertions(+)hw/arm/tegra241-cmdqv.c +22
-0@@ -13,14 +13,74 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_init_vcmdq_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu; +char *name; + +if (!s_accel) { +return true; + } + + viommu = &s_accel->viommu; +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VCMDQ_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vcmdq_page0, errp)) { + cmdqv->vcmdq_page0 =NULL ; +return false; + } + + name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv->mmio_cmdqv)); + memory_region_init_ram_device_ptr(&cmdqv->mmio_vcmdq_page,
hw/arm/tegra241-cmdqv.c line 47
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+memory_region_owner (&cmdqv->mmio_cmdqv), + name, 0x10000, cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x10000 , + &cmdqv->mmio_vcmdq_page,1 ); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0);
I don't get why we need/have 2 RAM devices pointing to the same @ptr = cmdqv->vcmdq_page0. Is 0x10000 ~ VCMDQ_REG_PAGE_SIZE?
The names of the MRs are quite confusing: If my understanding is correct we have;
cmdqv->mmio_cmdqv (0x50000 sized) which as the container for the 2 subregions. Then within that one we have 2 subregions, one at offset 0x10000 (mmio_vcmdq_page ), one at offset 0x30000 (cmdqv->mmio_vintf_page).
I have difficulties to link that with the commit message
"Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1:configuration registers ../.."
Those 2 pages, are they part of mmio_vcmdq_page?
Then you talk about 0x30000 :VINTF register I guess this is the second cmdqv->mmio_vintf_page
Well I am confused at this stage of the reading.
Also without any spec, this is difficult to understand. Is there any public doc?
Quoted 44 lines
hw/arm/tegra241-cmdqv.c +28
-0+memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + &cmdqv->mmio_vintf_page,1 ); +g_free (name); + +return true; +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vcmdq_page0) { +tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err); +if (local_err) { +error_report_err (local_err); + local_err =NULL ; + } + } +return 0 ; }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vcmdq_page0) { +tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err); +if (local_err) { +error_report_err (local_err); + local_err =NULL ; + } + } }static const MemoryRegionOps mmio_cmdqv_ops = {hw/arm/tegra241-cmdqv.h +0
-0
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 22
Quoted 4 lines
hw/arm/tegra241-cmdqv.h +1
-0@@ -19,8 +19,13 @@ typedef struct Tegra241CMDQV {SMMUv3State *smmu;MemoryRegion mmio_cmdqv;qemu_irq irq; +MemoryRegion mmio_vcmdq_page;
please also use _mr suffix as in the rest of the code
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 31
Quoted 9 lines
hw/arm/tegra241-cmdqv.h +4
-0+MemoryRegion mmio_vintf_page; +void *vcmdq_page0; } Tegra241CMDQV; +#define VCMDQ_REG_PAGE_SIZE 0x10000 +#ifdef CONFIG_TEGRA241_CMDQV bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp);
Thanks
Eric
RE: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
Hi Eric,
Quoted 88 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 26 January 2026 13:49
To: Shameer Kolothum <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guestExternal email: Use caution opening links or attachments
Hi Shameer,
Quoted 14 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1: configuration registers (queue GPA/size) that must be trapped by the VMM and translated before programming the HW queue.This patch implements the Page0 handling in QEMU. Using the vintf offset returned by IOMMUFD during VIOMMU allocation, QEMU maps Page0 into guest physical address space and exposes it via two guest MMIO windows:
-0x10000:VCMDQ register
-0x30000:VINTF registerQuoted 6 lines
The mapping is lazily initialized on first read/write.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 60+++++++++++++++++++++++++++++++++++++++++
hw/arm/tegra241-cmdqv.c line 16
Quoted 12 lines
hw/arm/tegra241-cmdqv.h | 5 ++++ 2 files changed, 65 insertions(+)hw/arm/tegra241-cmdqv.c +1
-0@@ -13,14 +13,74 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv,Error **errp)
Quoted 16 lines
hw/arm/tegra241-cmdqv.c +15
-0+{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu; +char *name; + +if (!s_accel) { +return true; + } + + viommu = &s_accel->viommu; +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu- viommu_id, +VCMDQ_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vcmdq_page0, errp)) {hw/arm/tegra241-cmdqv.c
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +9
-0+ cmdqv->vcmdq_page0 =NULL ; +return false; + } + + name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv- mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vcmdq_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); + memory_region_add_subregion_overlap(&cmdqv->mmio_cmdqv,
0x10000,hw/arm/tegra241-cmdqv.c line 47
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +7
-0+ &cmdqv->mmio_vcmdq_page, 1); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv- mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0);
Quoted 2 lines
I don't get why we need/have 2 RAM devices pointing to the same @ptr =
cmdqv->vcmdq_page0. Is0x10000~VCMDQ_REG_PAGE_SIZE?
Looking at this again, I am now a bit unsure as well why we need these memory_region_init_ram_device_ptr() calls here.
I will go back and discuss with Nicolin and come back. I think we need to document the memory layout better here. That will avoid the confusion.
(Also, will check if we have a public link for the spec or not).
Thanks,
Shameer
Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
In reply to Eric AugerQuoted 13 lines · Eric Auger
hw/arm/tegra241-cmdqv.c line 47
Quoted 12 lines
hw/arm/tegra241-cmdqv.c +12
-0+ name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vcmdq_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x10000 , + &cmdqv->mmio_vcmdq_page,1 ); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0);
Quoted 2 lines
I don't get why we need/have 2 RAM devices pointing to the same @ptr = cmdqv->vcmdq_page0. Is
0x10000~VCMDQ_REG_PAGE_SIZE?
The first one is for "vcmdq" and the second one is for "vintf". Explaining below...
Quoted 6 lines
The names of the MRs are quite confusing: If my understanding is correct we have;
cmdqv->mmio_cmdqv(0x50000sized) which as the container for the 2 subregions. Then within that one we have 2 subregions, one at offset0x10000(mmio_vcmdq_page), one at offset0x30000(cmdqv->mmio_vintf_page).
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c?h=v6.19-rc7#n19
It's defined in the kernel driver (yea, we should clarify in the QEMU code as well).
So, the MMIO regions look like this:
1st 64 KB page -- Global CMDQV registers
2nd 64 KB page -- Global VCMDQ registers Page0 (mmap) 3rd 64 KB page -- Global VCMDQ registers Page1 (trap) 4th 64 KB page -- VINTF0 Logical VCMDQ registers Page0 (mmap) 5th 64 KB page -- VINTF0 Logical VCMDQ registers Page1 (trap)
In real hardware, there will be 6th 64KB and beyond, for VINTF1 and others. But, we're omitting here in QEMU as only VINTF0 will be supported -- kernel only exposes one VINTF per VM as well. (Yes, we should clarify this too.)
Quoted 7 lines
I have difficulties to link that with the commit message
"Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1:configuration registers ../.."
Those 2 pages, are they part ofmmio_vcmdq_page?
Not exactly. Both the global VCMDQ region and VINTF region have their own pages (at 0x10000 and 0x30000).
Here, it duplicates the mapping to 0x10000 (Global VCMDQ page0) and 0x30000 (VINTF0 page0) for simplification, because we only support VINTF0 in this case.
There is a little catch in this implementation. The real physical mapping between a global VCMDQ and a logical VCMDQ happens when QEMU calls HW_QUEUE ioctl. So, the mmap'd page0 doesn't have any real VCMDQ backing up the emulated VCMDQ. So, perhaps QEMU should trap the page0 and delay the memory_region_init_ram_device_ptr() until the HW_QUEUE ioctl is done?
There might be also a corner case: when the kernel exposes two physical VCMDQs, but the guest OS only uses one, i.e. QEMU only allocates one HW_QUEUE for VCMDQ0 but doesn't allocate VCMDQ1. In such a case, the VTINF0 page0 should be able to control the logical VCMDQ0 only, while the global page0 should control both.
We are getting away this corner case with any guest OS running Linux kernel because it only accesses VTINF pages. But likely we should do something about it..
Quoted 5 lines
Then you talk about
0x30000:VINTF register I guess this is the secondcmdqv->mmio_vintf_pageWell I am confused at this stage of the reading.
Also without any spec, this is difficult to understand. Is there any public doc?
It seems that Red Hat can get the doc under NDA..
Thanks
Nicolin
Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
Hi Nicolin,
Quoted 29 lines · Nicolin Chen
Quoted 12 lines · Eric Auger
hw/arm/tegra241-cmdqv.c line 47
Quoted 12 lines
hw/arm/tegra241-cmdqv.c +12
-0+ name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vcmdq_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x10000 , + &cmdqv->mmio_vcmdq_page,1 ); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0);Quoted 2 lines
I don't get why we need/have 2 RAM devices pointing to the same @ptr = cmdqv->vcmdq_page0. Is
0x10000~VCMDQ_REG_PAGE_SIZE?The first one is for "vcmdq" and the second one is for "vintf". Explaining below...
Quoted 6 lines
The names of the MRs are quite confusing: If my understanding is correct we have;
cmdqv->mmio_cmdqv(0x50000sized) which as the container for the 2 subregions. Then within that one we have 2 subregions, one at offset0x10000(mmio_vcmdq_page), one at offset0x30000(cmdqv->mmio_vintf_page).https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/iommu/arm/arm-smmu-v3/tegra241-cmdqv.c?h=v6.19-rc7#n19
It's defined in the kernel driver (yea, we should clarify in the QEMU code as well).
Quoted 6 lines
So, the MMIO regions look like this:
1st 64 KB page -- Global CMDQV registers
2nd 64 KB page -- Global VCMDQ registers Page0 (mmap) 3rd 64 KB page -- Global VCMDQ registers Page1 (trap) 4th 64 KB page -- VINTF0 Logical VCMDQ registers Page0 (mmap) 5th 64 KB page -- VINTF0 Logical VCMDQ registers Page1 (trap)
This kind of layout representation is really helpful
Quoted 5 lines
In real hardware, there will be 6th 64KB and beyond, for VINTF1 and others. But, we're omitting here in QEMU as only VINTF0 will be supported -- kernel only exposes one VINTF per VM as well. (Yes, we should clarify this too.)
OK
Quoted 8 lines
Quoted 8 lines
I have difficulties to link that with the commit message
"Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1:configuration registers ../.."
Those 2 pages, are they part ofmmio_vcmdq_page?
Quoted 6 lines
Not exactly. Both the global VCMDQ region and VINTF region have their own pages (at
0x10000and0x30000).Here, it duplicates the mapping to
0x10000(Global VCMDQ page0) and0x30000(VINTF0 page0) for simplification, because we only support VINTF0 in this case.
so Global VCMDQ registers Page0 and VINTF0 Logical VCMDQ registers Page0 are basically the same?
I would recommend to use cmdqv->mmio_vcmdq_page0 and cmdqv->mmio_vintf_page0 to avoid any misunderstanding
Quoted 7 lines
There is a little catch in this implementation. The real physical mapping between a global VCMDQ and a logical VCMDQ happens when QEMU calls
HW_QUEUEioctl. So, the mmap'd page0 doesn't have any real VCMDQ backing up the emulated VCMDQ. So, perhaps QEMU should trap the page0 and delay thememory_region_init_ram_device_ptr()until theHW_QUEUEioctl is done?
might be safer indeed.
Quoted 6 lines
There might be also a corner case: when the kernel exposes two physical VCMDQs, but the guest OS only uses one, i.e. QEMU only allocates one
HW_QUEUEfor VCMDQ0 but doesn't allocate VCMDQ1. In such a case, the VTINF0 page0 should be able to control the logical VCMDQ0 only, while the global page0 should control both.
you lost me. Need to look at the kernel or spec ;-)
Quoted 11 lines
We are getting away this corner case with any guest OS running Linux kernel because it only accesses VTINF pages. But likely we should do something about it..
Quoted 5 lines
Then you talk about
0x30000:VINTF register I guess this is the secondcmdqv->mmio_vintf_pageWell I am confused at this stage of the reading.
Also without any spec, this is difficult to understand. Is there any public doc?
It seems that Red Hat can get the doc under NDA..
OK thanks
Eric
Quoted 4 lines
Thanks
Nicolin
Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
Quoted 8 lines · Eric Auger
Quoted 7 lines · Nicolin Chen
On Mon, Jan 26, 2026 at 02:48:55PM +0100, Eric Auger wrote:
So, the MMIO regions look like this:
1st 64 KB page -- Global CMDQV registers
2nd 64 KB page -- Global VCMDQ registers Page0 (mmap) 3rd 64 KB page -- Global VCMDQ registers Page1 (trap) 4th 64 KB page -- VINTF0 Logical VCMDQ registers Page0 (mmap) 5th 64 KB page -- VINTF0 Logical VCMDQ registers Page1 (trap)
[...]
Quoted 10 lines
Quoted 10 lines
Quoted 7 lines
I have difficulties to link that with the commit message
"Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1:configuration registers ../.."
Those 2 pages, are they part ofmmio_vcmdq_page?Not exactly. Both the global VCMDQ region and VINTF region have their own pages (at
0x10000and0x30000).
Quoted 6 lines
Quoted 3 lines
Here, it duplicates the mapping to
0x10000(Global VCMDQ page0) and0x30000(VINTF0 page0) for simplification, because we only support VINTF0 in this case.so Global VCMDQ registers Page0 and VINTF0 Logical VCMDQ registers Page0 are basically the same?
Not exactly the same.
The global page0 is programmable at any time so long as CMDQV_EN is enabled.
The logical page0 is programmable only when SW allocates and maps global vcmdq(s) to a VINTF. "logical" also means "local" to that VINTF.
Quoted 2 lines
I would recommend to use
cmdqv->mmio_vcmdq_page0andcmdqv->mmio_vintf_page0to avoid any misunderstanding
Yea, that makes sense to me.
Quoted 8 lines
Quoted 6 lines
There is a little catch in this implementation. The real physical mapping between a global VCMDQ and a logical VCMDQ happens when QEMU calls
HW_QUEUEioctl. So, the mmap'd page0 doesn't have any real VCMDQ backing up the emulated VCMDQ. So, perhaps QEMU should trap the page0 and delay thememory_region_init_ram_device_ptr()until theHW_QUEUEioctl is done?might be safer indeed.
Quoted 1 line
Quoted 7 lines
Quoted 5 lines
There might be also a corner case: when the kernel exposes two physical VCMDQs, but the guest OS only uses one, i.e. QEMU only allocates one
HW_QUEUEfor VCMDQ0 but doesn't allocate VCMDQ1. In such a case, the VTINF0 page0 should be able to control the logical VCMDQ0 only, while the global page0 should control both.you lost me. Need to look at the kernel or spec ;-)
That was for supporting a partial local vcmdq mapping with QEMU, as guest OS is allowed to do so from HW simulation perspective.
E.g.
A VTINF exposed by the kernel supports maximum 2 VCMDQs. IOW, VM owns 2 global vcmdqs. And mmio_vcmdq_page0 is supposed to have the access to both vcmdqs.
Kernel only exposes physical VINTF's page0 via mmap, for security reason, which starts with 0 logical vcmdq, i.e. no access to any global vcmdq via mmio_vcmdq_page0 or mmio_vintf_page0.
A guest SW only allocates and maps one global vcmdq to the VINTF, which means only one HW_QUEUE ioctl is invoked so that the kernel only maps one global vcmdq accordingly. Then, both page0 only has the access to one global vcmdq via its logical mapping.
In such a case, memory_region_init_ram_device_ptr() to 0x10000 for mmio_vcmdq_page0 is basically wrong, since it's supposed to have access to both global vcmdqs.
So, this makes things uneasy to support.
What we could likely do:
- only mmap mmio_vintf_page0 to 0x30000 (logical page0)
- drop mmio_vcmdq_page0 and trap 0x10000 (global page0)
a) if vcmdq is indexed to a mapped global vcmdq, read/write mmio_vintf_page0.
b) if vcmdq is indexed to an unmapped global vcmdq, read/write the value in the register array cached by QEMU.
This would likely slow down the perf if guest OS is using a LVCMDQ via the global page0. But functional wise it should be okay.
Nicolin
Re: [RFC PATCH 06/16] hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest
In reply to Shameer KolothumQuoted 8 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Tegra241 CMDQV assigns each VINTF a 128KB MMIO region split into two 64 KB pages:
- Page0: guest accessible control/status registers for all VCMDQs
- Page1: configuration registers (queue GPA/size) that must be trapped by the VMM and translated before programming the HW queue.
Quoted 5 lines
This patch implements the Page0 handling in QEMU. Using the vintf offset returned by IOMMUFD during VIOMMU allocation, QEMU maps Page0 into guest physical address space and exposes it via two guest MMIO windows:
-0x10000:VCMDQ register
-0x30000:VINTF register
I would recommend to have a drawing showing the mr container, the subregions, their offsets, and content/nature of each region
Eric
Quoted 107 lines
The mapping is lazily initialized on first read/write.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 60 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 5 ++++ 2 files changed, 65 insertions(+)hw/arm/tegra241-cmdqv.c +60
-0@@ -13,14 +13,74 @@#include "smmuv3-accel.h" #include "tegra241-cmdqv.h" +static bool tegra241_cmdqv_init_vcmdq_page0 (Tegra241CMDQV *cmdqv,Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu; +char *name; + +if (!s_accel) { +return true; + } + + viommu = &s_accel->viommu; +if (!iommufd_backend_viommu_mmap (viommu->iommufd, viommu->viommu_id, +VCMDQ_REG_PAGE_SIZE , + cmdqv->cmdqv_data.out_vintf_mmap_offset, + &cmdqv->vcmdq_page0, errp)) { + cmdqv->vcmdq_page0 =NULL ; +return false; + } + + name =g_strdup_printf ("%s vcmdq" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vcmdq_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x10000 , + &cmdqv->mmio_vcmdq_page,1 ); +g_free (name); + + name =g_strdup_printf ("%s vintf" ,memory_region_name (&cmdqv->mmio_cmdqv)); +memory_region_init_ram_device_ptr (&cmdqv->mmio_vintf_page, +memory_region_owner (&cmdqv->mmio_cmdqv), + name,0x10000 , cmdqv->vcmdq_page0); +memory_region_add_subregion_overlap (&cmdqv->mmio_cmdqv,0x30000 , + &cmdqv->mmio_vintf_page,1 ); +g_free (name); + +return true; +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vcmdq_page0) { +tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err); +if (local_err) { +error_report_err (local_err); + local_err =NULL ; + } + } +return 0 ; }static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) { +Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque; +Error *local_err =NULL ; + +if (!cmdqv->vcmdq_page0) { +tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err); +if (local_err) { +error_report_err (local_err); + local_err =NULL ; + } + } }static const MemoryRegionOps mmio_cmdqv_ops = {hw/arm/tegra241-cmdqv.h +5
-0@@ -19,8 +19,13 @@ typedef struct Tegra241CMDQV {SMMUv3State *smmu;MemoryRegion mmio_cmdqv;qemu_irq irq; +MemoryRegion mmio_vcmdq_page; +MemoryRegion mmio_vintf_page; +void *vcmdq_page0; } Tegra241CMDQV; +#define VCMDQ_REG_PAGE_SIZE 0x10000 +#ifdef CONFIG_TEGRA241_CMDQV bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp);
RFC PATCH 7/16 hw/arm/tegra241-cmdqv: Add read emulation support for registers
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Implement read support for Tegra241 CMDQV register blocks, including VINTF and per VCMDQ register regions. The patch decodes offsets, extracts queue indices, and returns the corresponding cached register state.
Subsequent patch will add write support.
--- hw/arm/tegra241-cmdqv.c | 144 +++++++++++++++++++- hw/arm/tegra241-cmdqv.h | 282 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 425 insertions(+), 1 deletion(-)
hw/arm/tegra241-cmdqv.c +143 -1
@@ -8,6 +8,7 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h"
@@ -52,10 +53,94 @@ static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp)return true; } +/* Note that offset aligns down to 0x1000 */ +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i]; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} + +/* Note that offset aligns down to 0x10000 */ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset, +int index) +{ +uint32_t *ptr =NULL ; +uint64_t off; + +/* +* Each VCMDQ instance occupies a 128 byte region (0x80). +* The hardware layout is: +* vcmdq_page0 + (index * 0x80) + (offset - 0x10000) +*/ +if (cmdqv->vcmdq_page0) { + off = (0x80 * index) + (offset -0x10000 ); + ptr = (uint32_t *)(cmdqv->vcmdq_page0 + off); + } + +switch (offset) { +case A_VCMDQ0_CONS_INDX : +if (ptr) { + cmdqv->vcmdq_cons_indx[index] = *ptr; + } +return cmdqv->vcmdq_cons_indx[index]; +case A_VCMDQ0_PROD_INDX : +if (ptr) { + cmdqv->vcmdq_prod_indx[index] = *ptr; + } +return cmdqv->vcmdq_prod_indx[index]; +case A_VCMDQ0_CONFIG : +if (ptr) { + cmdqv->vcmdq_config[index] = *ptr; + } +return cmdqv->vcmdq_config[index]; +case A_VCMDQ0_STATUS : +if (ptr) { + cmdqv->vcmdq_status[index] = *ptr; + } +return cmdqv->vcmdq_status[index]; +case A_VCMDQ0_GERROR : +if (ptr) { + cmdqv->vcmdq_gerror[index] = *ptr; + } +return cmdqv->vcmdq_gerror[index]; +case A_VCMDQ0_GERRORN : +if (ptr) { + cmdqv->vcmdq_gerrorn[index] = *ptr; + } +return cmdqv->vcmdq_gerrorn[index]; +case A_VCMDQ0_BASE_L : +return cmdqv->vcmdq_base[index]; +case A_VCMDQ0_BASE_H : +return cmdqv->vcmdq_base[index] >>32 ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +return cmdqv->vcmdq_cons_indx_base[index]; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : +return cmdqv->vcmdq_cons_indx_base[index] >>32 ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +}static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;Error *local_err =NULL ; +int index;if (!cmdqv->vcmdq_page0) {tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err);
@@ -65,7 +150,64 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } -return 0 ; +if (offset >TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return 0 ; + } + +/* Fallback to cached register values */ +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ]; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: +/* +* Align offset down to 0x10000 while extracting the index: +* VCMDQ0_CONS_INDX (0x10000) => 0x10000, 0 +* VCMDQ1_CONS_INDX (0x10080) => 0x10000, 1 +* VCMDQ2_CONS_INDX (0x10100) => 0x10000, 2 +* ... +* VCMDQ127_CONS_INDX (0x13f80) => 0x10000, 127 +*/ + index = (offset -0x10000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index); + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* +* Align offset down to 0x20000 while extracting the index: +* VCMDQ0_BASE_L (0x20000) => 0x20000, 0 +* VCMDQ1_BASE_L (0x20080) => 0x20000, 1 +* VCMDQ2_BASE_L (0x20100) => 0x20000, 2 +* ... +* VCMDQ127_BASE_L (0x23f80) => 0x20000, 127 +*/ + index = (offset -0x20000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } } static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,
hw/arm/tegra241-cmdqv.h +282 -0
298 lines not on this page · raw message
Signature · 1 line
2.43.0
Re: [RFC PATCH 07/16] hw/arm/tegra241-cmdqv: Add read emulation support for registers
Quoted 13 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +13
-0+/* Note that offset aligns down to 0x10000 */ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset, +int index) +{ +uint32_t *ptr =NULL ; +uint64_t off; + +/* +* Each VCMDQ instance occupies a 128 byte region (0x80). +* The hardware layout is: +* vcmdq_page0 + (index * 0x80) + (offset - 0x10000) +*/ +if (cmdqv->vcmdq_page0) {
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 98
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ off = (0x80 * index) + (offset -0x10000 ); + ptr = (uint32_t *)(cmdqv->vcmdq_page0 + off); + } + +switch (offset) { +case A_VCMDQ0_CONS_INDX : +if (ptr) { + cmdqv->vcmdq_cons_indx[index] = *ptr; + } +return cmdqv->vcmdq_cons_indx[index];
I forgot the details. But this seems a deadcode:
+ if (ptr) {
+ cmdqv->vcmdq_cons_indx[index] = *ptr;
+ }
?
hw/arm/tegra241-cmdqv.h typedef struct Tegra241CMDQV { · line 123
Quoted 8 lines
hw/arm/tegra241-cmdqv.h +8
-0+/* i = [0, 0], j = [0, 3] */ +#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) + /* Omitting [0][1~2] as not being directly called */ +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 )
The indentations at 0 is odd related to 3. And should fix all the following macros too.
Nicolin
Re: [RFC PATCH 07/16] hw/arm/tegra241-cmdqv: Add read emulation support for registers
In reply to Shameer KolothumHi Shameer,
Quoted 44 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Implement read support for Tegra241 CMDQV register blocks, including VINTF and per VCMDQ register regions. The patch decodes offsets, extracts queue indices, and returns the corresponding cached register state.
Subsequent patch will add write support.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 144 +++++++++++++++++++- hw/arm/tegra241-cmdqv.h | 282 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 425 insertions(+), 1 deletion(-)hw/arm/tegra241-cmdqv.c +14
-0@@ -8,6 +8,7 @@*/ #include "qemu/osdep.h" +#include "qemu/log.h" #include "hw/arm/smmuv3.h" #include "smmuv3-accel.h" @@ -52,10 +53,94 @@ static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp)return true; } +/* Note that offset aligns down to 0x1000 */ +static uint64_t tegra241_cmdqv_read_vintf (Tegra241CMDQV *cmdqv,hwaddr offset) +{ +int i; + +switch (offset) { +case A_VINTF0_CONFIG : +return cmdqv->vintf_config; +case A_VINTF0_STATUS : +return cmdqv->vintf_status; + case A_VINTF0_LVCMDQ_ERR_MAP_0 ... A_VINTF0_LVCMDQ_ERR_MAP_3: + i = (offset -A_VINTF0_LVCMDQ_ERR_MAP_0 ) /4 ; +return cmdqv->vintf_cmdq_err_map[i];
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 78
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +} + +/* Note that offset aligns down to 0x10000 */ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset, +int index)
the offset is within the instance, right? Better to rename it differently from the main offset
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 81
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3
-0+{ +uint32_t *ptr =NULL ; +uint64_t off;
better initialize it?
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 90
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9
-0+ +/* +* Each VCMDQ instance occupies a 128 byte region (0x80). +* The hardware layout is: +* vcmdq_page0 + (index * 0x80) + (offset - 0x10000) +*/ +if (cmdqv->vcmdq_page0) { + off = (0x80 * index) + (offset -0x10000 ); + ptr = (uint32_t *)(cmdqv->vcmdq_page0 + off);
I am confused because I understood page0 was mmapped so that the guest could read it without VMM mediation?
Quoted 82 lines
hw/arm/tegra241-cmdqv.c +70
-1+ } + +switch (offset) { +case A_VCMDQ0_CONS_INDX : +if (ptr) { + cmdqv->vcmdq_cons_indx[index] = *ptr; + } +return cmdqv->vcmdq_cons_indx[index]; +case A_VCMDQ0_PROD_INDX : +if (ptr) { + cmdqv->vcmdq_prod_indx[index] = *ptr; + } +return cmdqv->vcmdq_prod_indx[index]; +case A_VCMDQ0_CONFIG : +if (ptr) { + cmdqv->vcmdq_config[index] = *ptr; + } +return cmdqv->vcmdq_config[index]; +case A_VCMDQ0_STATUS : +if (ptr) { + cmdqv->vcmdq_status[index] = *ptr; + } +return cmdqv->vcmdq_status[index]; +case A_VCMDQ0_GERROR : +if (ptr) { + cmdqv->vcmdq_gerror[index] = *ptr; + } +return cmdqv->vcmdq_gerror[index]; +case A_VCMDQ0_GERRORN : +if (ptr) { + cmdqv->vcmdq_gerrorn[index] = *ptr; + } +return cmdqv->vcmdq_gerrorn[index]; +case A_VCMDQ0_BASE_L : +return cmdqv->vcmdq_base[index]; +case A_VCMDQ0_BASE_H : +return cmdqv->vcmdq_base[index] >>32 ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +return cmdqv->vcmdq_cons_indx_base[index]; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : +return cmdqv->vcmdq_cons_indx_base[index] >>32 ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } +}static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;Error *local_err =NULL ; +int index;if (!cmdqv->vcmdq_page0) {tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err);@@ -65,7 +150,64 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } -return 0 ; +if (offset >TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return 0 ; + } + +/* Fallback to cached register values */ +switch (offset) { +case A_CONFIG : +return cmdqv->config; +case A_PARAM : +return cmdqv->param; +case A_STATUS : +return cmdqv->status; + case A_VI_ERR_MAP ... A_VI_ERR_MAP_1: +return cmdqv->vi_err_map[(offset -A_VI_ERR_MAP ) /4 ]; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: +return cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ]; + case A_CMDQ_ERR_MAP ... A_CMDQ_ERR_MAP_3: +return cmdqv->cmdq_err_map[(offset -A_CMDQ_ERR_MAP ) /4 ];
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 183
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+ case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127: +return cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ]; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +return tegra241_cmdqv_read_vintf (cmdqv, offset); + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: +/* +* Align offset down to 0x10000 while extracting the index:
this is a bit cryptic to me. Could you explain this in the commit msg
Quoted 334 lines
hw/arm/tegra241-cmdqv.c +27
-0+* VCMDQ0_CONS_INDX (0x10000) => 0x10000, 0 +* VCMDQ1_CONS_INDX (0x10080) => 0x10000, 1 +* VCMDQ2_CONS_INDX (0x10100) => 0x10000, 2 +* ... +* VCMDQ127_CONS_INDX (0x13f80) => 0x10000, 127 +*/ + index = (offset -0x10000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index); + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* +* Align offset down to 0x20000 while extracting the index: +* VCMDQ0_BASE_L (0x20000) => 0x20000, 0 +* VCMDQ1_BASE_L (0x20080) => 0x20000, 1 +* VCMDQ2_BASE_L (0x20100) => 0x20000, 2 +* ... +* VCMDQ127_BASE_L (0x23f80) => 0x20000, 127 +*/ + index = (offset -0x20000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index); +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled read access at 0x%" PRIx64"\n" , + __func__, offset); +return 0 ; + } } static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value,hw/arm/tegra241-cmdqv.h +282
-0@@ -10,6 +10,7 @@#ifndef HW_TEGRA241_CMDQV_H #define HW_TEGRA241_CMDQV_H +#include "hw/registerfields.h" #include CONFIG_DEVICES #define TEGRA241_CMDQV_IO_LEN 0x50000@@ -22,10 +23,291 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_vcmdq_page;MemoryRegion mmio_vintf_page;void *vcmdq_page0; +IOMMUFDHWqueue *vcmdq[128 ]; + +/* Register Cache */ +uint32_t config; +uint32_t param; +uint32_t status; +uint32_t vi_err_map[2 ]; +uint32_t vi_int_mask[2 ]; +uint32_t cmdq_err_map[4 ]; +uint32_t cmdq_alloc_map[128 ]; +uint32_t vintf_config; +uint32_t vintf_status; +uint32_t vintf_cmdq_err_map[4 ]; +uint32_t vcmdq_cons_indx[128 ]; +uint32_t vcmdq_prod_indx[128 ]; +uint32_t vcmdq_config[128 ]; +uint32_t vcmdq_status[128 ]; +uint32_t vcmdq_gerror[128 ]; +uint32_t vcmdq_gerrorn[128 ]; +uint64_t vcmdq_base[128 ]; +uint64_t vcmdq_cons_indx_base[128 ]; } Tegra241CMDQV; +/* MMIO Registers */ +REG32(CONFIG, 0x0) +FIELD(CONFIG, CMDQV_EN, 0, 1) +FIELD(CONFIG, CMDQV_PER_CMD_OFFSET, 1, 3) +FIELD(CONFIG, CMDQ_MAX_CLK_BATCH, 4, 8) +FIELD(CONFIG, CMDQ_MAX_CMD_BATCH, 12, 8) +FIELD(CONFIG, CONS_DRAM_EN, 20, 1) + +#define V_CONFIG_RESET 0x00020403 + +REG32(PARAM, 0x4) +FIELD(PARAM, CMDQV_VER, 0, 4) +FIELD(PARAM, CMDQV_NUM_CMDQ_LOG2, 4, 4) +FIELD(PARAM, CMDQV_NUM_VM_LOG2, 8, 4) +FIELD(PARAM, CMDQV_NUM_SID_PER_VM_LOG2, 12, 4) + +#define V_PARAM_RESET 0x00004011 + +REG32(STATUS, 0x8) +FIELD(STATUS, CMDQV_ENABLED, 0, 1) + +#define A_VI_ERR_MAP 0x14 +#define A_VI_ERR_MAP_1 0x18 +#define V_VI_ERR_MAP_NO_ERROR (0) +#define V_VI_ERR_MAP_ERROR (1) + +#define A_VI_INT_MASK 0x1c +#define A_VI_INT_MASK_1 0x20 +#define V_VI_INT_MASK_NOT_MASKED (0) +#define V_VI_INT_MASK_MASKED (1) + +#define A_CMDQ_ERR_MAP 0x24 +#define A_CMDQ_ERR_MAP_1 0x28 +#define A_CMDQ_ERR_MAP_2 0x2c +#define A_CMDQ_ERR_MAP_3 0x30 + +/* i = [0, 127] */ +#define A_CMDQ_ALLOC_MAP_ (i) \ + REG32(CMDQ_ALLOC_MAP_##i, 0x200 + i * 4) \ + FIELD(CMDQ_ALLOC_MAP_##i, ALLOC, 0, 1) \ + FIELD(CMDQ_ALLOC_MAP_##i, LVCMDQ, 1, 7) \ + FIELD(CMDQ_ALLOC_MAP_##i, VIRT_INTF_INDX, 15, 6) + +A_CMDQ_ALLOC_MAP_(0)cmdq_alloc_map +/* Omitting 1~126 as not being directly called */ +A_CMDQ_ALLOC_MAP_(127) + +/* i = [0, 0] */ +#define A_VINTFi_CONFIG (i) \ + REG32(VINTF##i##_CONFIG, 0x1000 + i * 0x100) \ + FIELD(VINTF##i##_CONFIG, ENABLE, 0, 1) \ + FIELD(VINTF##i##_CONFIG, VMID, 1, 16) \ + FIELD(VINTF##i##_CONFIG, HYP_OWN, 17, 1) + +A_VINTFi_CONFIG(0) + +#define A_VINTFi_STATUS (i) \ + REG32(VINTF##i##_STATUS, 0x1004 + i * 0x100) \ + FIELD(VINTF##i##_STATUS, ENABLE_OK, 0, 1) \ + FIELD(VINTF##i##_STATUS, STATUS, 1, 3) \ + FIELD(VINTF##i##_STATUS, VI_NUM_LVCMDQ, 16, 8) + + A_VINTFi_STATUS(0) + +#define V_VINTF_STATUS_NO_ERROR (0 << 1) +#define V_VINTF_STATUS_VCMDQ_EROR (1 << 1) + +/* i = [0, 0], j = [0, 3] */ +#define A_VINTFi_LVCMDQ_ERR_MAP_ (i, j) \ + REG32(VINTF##i##_LVCMDQ_ERR_MAP_##j, 0x10c0 + j * 4 + i * 0x100) \ + FIELD(VINTF##i##_LVCMDQ_ERR_MAP_##j, LVCMDQ_ERR_MAP, 0, 32) + +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,0 ) + /* Omitting [0][1~2] as not being directly called */ +A_VINTFi_LVCMDQ_ERR_MAP_ (0 ,3 ) + +/* VCMDQ registers -- starting from 0x10000 with size 64KB * 2 (0x20000) */ +#define VCMDQ_REG_OFFSET 0x10000#define VCMDQ_REG_PAGE_SIZE 0x10000 +#define A_VCMDQi_CONS_INDX (i) \ + REG32(VCMDQ##i##_CONS_INDX, 0x10000 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VCMDQi_CONS_INDX (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_CONS_INDX(127) + +#define V_VCMDQ_CONS_INDX_ERR_CERROR_NONE 0 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_OPCODE 1 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ABT 2 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ATC_INV_SYNC 3 +#define V_VCMDQ_CONS_INDX_ERR_CERROR_ILL_ACCESS 4 + +#define A_VCMDQi_PROD_INDX (i) \ + REG32(VCMDQ##i##_PROD_INDX, 0x10000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VCMDQi_PROD_INDX (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_PROD_INDX(127) + +#define A_VCMDQi_CONFIG (i) \ + REG32(VCMDQ##i##_CONFIG, 0x10000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VCMDQi_CONFIG (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_CONFIG(127) + +#define A_VCMDQi_STATUS (i) \ + REG32(VCMDQ##i##_STATUS, 0x10000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VCMDQi_STATUS (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_STATUS(127) + +#define A_VCMDQi_GERROR (i) \ + REG32(VCMDQ##i##_GERROR, 0x10000 + 0x10 + i * 0x80) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERROR (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_GERROR(127) + +#define A_VCMDQi_GERRORN (i) \ + REG32(VCMDQ##i##_GERRORN, 0x10000 + 0x14 + i * 0x80) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VCMDQi_GERRORN (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_GERRORN(127) + +#define A_VCMDQi_BASE_L (i) \ + REG32(VCMDQ##i##_BASE_L, 0x20000 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VCMDQi_BASE_L (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_BASE_L(127) + +#define A_VCMDQi_BASE_H (i) \ + REG32(VCMDQ##i##_BASE_H, 0x20000 + 0x4 + i * 0x80) \ + FIELD(VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VCMDQi_BASE_H (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_BASE_H(127) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x20000 + 0x8 + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VCMDQi_CONS_INDX_BASE_DRAM_L (0 ) + /* Omitting [1~126] as not being directly called */ + A_VCMDQi_CONS_INDX_BASE_DRAM_L(127) + +#define A_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x20000 + 0xc + i * 0x80) \ + FIELD(VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VCMDQi_CONS_INDX_BASE_DRAM_H (0 ) + /* Omitting [1~126] as not being directly called */ +A_VCMDQi_CONS_INDX_BASE_DRAM_H (127 ) + +/* +* VINTF VI_VCMDQ registers -- starting from 0x30000 with size 64KB * 2 +* (0x20000) + */ +#define A_VI_VCMDQi_CONS_INDX (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX, 0x30000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, RD, 0, 20) \ + FIELD(VI_VCMDQ##i##_CONS_INDX, ERR, 24, 7) + +A_VI_VCMDQi_CONS_INDX (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_CONS_INDX(127) + +#define A_VI_VCMDQi_PROD_INDX (i) \ + REG32(VI_VCMDQ##i##_PROD_INDX, 0x30000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_PROD_INDX, WR, 0, 20) + +A_VI_VCMDQi_PROD_INDX (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_PROD_INDX(127) + +#define A_VI_VCMDQi_CONFIG (i) \ + REG32(VI_VCMDQ##i##_CONFIG, 0x30000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONFIG, CMDQ_EN, 0, 1) + +A_VI_VCMDQi_CONFIG (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_CONFIG(127) + +#define A_VI_VCMDQi_STATUS (i) \ + REG32(VI_VCMDQ##i##_STATUS, 0x30000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_STATUS, CMDQ_EN_OK, 0, 1) + +A_VI_VCMDQi_STATUS (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_STATUS(127) + +#define A_VI_VCMDQi_GERROR (i) \ + REG32(VI_VCMDQ##i##_GERROR, 0x30000 + 0x10 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERROR, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERROR (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_GERROR(127) + +#define A_VI_VCMDQi_GERRORN (i) \ + REG32(VI_VCMDQ##i##_GERRORN, 0x30000 + 0x14 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_ERR, 0, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CONS_DRAM_WR_ABT_ERR, 1, 1) \ + FIELD(VI_VCMDQ##i##_GERRORN, CMDQ_INIT_ERR, 2, 1) + +A_VI_VCMDQi_GERRORN (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_GERRORN(127) + +#define A_VI_VCMDQi_BASE_L (i) \ + REG32(VI_VCMDQ##i##_BASE_L, 0x40000 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_L, LOG2SIZE, 0, 5) \ + FIELD(VI_VCMDQ##i##_BASE_L, ADDR, 5, 27) + +A_VI_VCMDQi_BASE_L (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_BASE_L(127) + +#define A_VI_VCMDQi_BASE_H (i) \ + REG32(VI_VCMDQ##i##_BASE_H, 0x40000 + 0x4 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_BASE_H, ADDR, 0, 16) + +A_VI_VCMDQi_BASE_H (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_BASE_H(127) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, 0x40000 + 0x8 + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_L, ADDR, 0, 32) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_CONS_INDX_BASE_DRAM_L(127) + +#define A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H (i) \ + REG32(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, 0x40000 + 0xc + i * 0x80) \ + FIELD(VI_VCMDQ##i##_CONS_INDX_BASE_DRAM_H, ADDR, 0, 16) + +A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H (0 ) + /* Omitting [1~126] as not being directly called */ + A_VI_VCMDQi_CONS_INDX_BASE_DRAM_H(127) +#ifdef CONFIG_TEGRA241_CMDQV bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,
Quoted 1 line
uint32_t *out_viommu_id, Error **errp);
Thanks
Eric
Re: [RFC PATCH 07/16] hw/arm/tegra241-cmdqv: Add read emulation support for registers
Hi Eric,
Quoted 6 lines · Eric Auger
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 78
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3
-0+/* Note that offset aligns down to 0x10000 */ +static uint64_t tegra241_cmdqv_read_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset, +int index)the offset is within the instance, right? Better to rename it differently from the main offset
In tegra241_cmdqv_read(), its offset is the register offset to the entire CMDQV. That can be A_VCMDQ0_CONS_INDX, A_VCMDQ1_CONS_INDX, until A_VCMDQ127_CONS_INDX.
Then, we align it down to 0x10000 (A_VCMDQ0_CONS_INDX) to pass it into this function, so this function can handle any VCMDQ via the A_VCMDQ0_* registers only, as "index" tells which VCMDQ instead.
Perhaps we could rename this "offset" to "offset0", and elaborate this with a better function comments, e.g.:
/*
* The caller striped away the VCMDQ index from its offset and passed that in
* via @offset0 so any VCMDQ can fit into A_VCMDQ0_CONS_INDX for simplicity. *
* On the other hand, @index will tell which VCMDQ is being accessed here. */
Quoted 4 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 81
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3
-0+{ +uint32_t *ptr =NULL ; +uint64_t off;better initialize it?
It's not used unless cmdqv->vcmdq_page0, so we should probably move it to the if statement instead.
Quoted 10 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_init_vcmdq_page0(Tegra241CMDQV *cmdqv, Error **errp) · line 90
Quoted 9 lines
hw/arm/tegra241-cmdqv.c +9
-0+ +/* +* Each VCMDQ instance occupies a 128 byte region (0x80). +* The hardware layout is: +* vcmdq_page0 + (index * 0x80) + (offset - 0x10000) +*/ +if (cmdqv->vcmdq_page0) { + off = (0x80 * index) + (offset -0x10000 ); + ptr = (uint32_t *)(cmdqv->vcmdq_page0 + off);
Quoted 2 lines
I am confused because I understood page0 was mmapped so that the guest could read it without VMM mediation?
Yes. I pointed it out too. This should be cleaned away:
https://lore.kernel.org/qemu-devel/aVLZv+nwUB7+ohuw@Asurada-Nvidia/
So, only return the cached regval unless !cmdqv_page0:
if (!cmdqv->vcmdq_page0) {
return cmdqv->vcmdq_cons_indx[index];
}Quoted 12 lines
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) · line 183
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +3
-0+ case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: +/* +* Align offset down to 0x10000 while extracting the index: this is a bit cryptic to me. Could you explain this in the commit msg
Quoted 8 lines
hw/arm/tegra241-cmdqv.c +8
-0+* VCMDQ0_CONS_INDX (0x10000) => 0x10000, 0 +* VCMDQ1_CONS_INDX (0x10080) => 0x10000, 1 +* VCMDQ2_CONS_INDX (0x10100) => 0x10000, 2 +* ... +* VCMDQ127_CONS_INDX (0x13f80) => 0x10000, 127 +*/ + index = (offset -0x10000 ) /0x80 ; +return tegra241_cmdqv_read_vcmdq (cmdqv, offset -0x80 * index, index);
As I mentioned above, it is to ease the tegra241_cmdqv_read_vcmdq() that only needs to handle A_VCMDQ0_* cases in the switch statement.
Yes, let's add in the commit message.
Thanks
Nicolin
RFC PATCH 8/16 system/physmem: Add helper to check whether a guest PA maps to RAM
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Introduce cpu_physical_memory_is_ram(), a helper that performs an address_space translation and returns whether the resolved MemoryRegion is backed by RAM.
This will be used by the upcoming Tegra241 CMDQV support to validate guest provided VCMDQ buffer addresses.
--- include/exec/cpu-common.h | 2 ++ system/physmem.c | 12 ++++++++++++ 2 files changed, 14 insertions(+)
include/exec/cpu-common.h +2 -0
@@ -148,6 +148,8 @@ void qemu_flush_coalesced_mmio_buffer(void);typedef int (RAMBlockIterFunc )(RAMBlock *rb,void *opaque); +bool cpu_physical_memory_is_ram (hwaddr phys_addr); +int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque);/* vl.c */
system/physmem.c +12 -0
@@ -4068,6 +4068,18 @@ int cpu_memory_rw_debug(CPUState *cpu, vaddr addr,return 0 ; } +bool cpu_physical_memory_is_ram (hwaddr phys_addr) +{ +MemoryRegion *mr; +hwaddr l =1 ; + +RCU_READ_LOCK_GUARD (); + mr =address_space_translate (&address_space_memory, phys_addr, &phys_addr, + &l, false,MEMTXATTRS_UNSPECIFIED ); + +return memory_region_is_ram (mr); +} +int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque) {RAMBlock *block;
Signature · 1 line
2.43.0
Re: [RFC PATCH 08/16] system/physmem: Add helper to check whether a guest PA maps to RAM
Quoted 34 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Introduce
cpu_physical_memory_is_ram(), a helper that performs anaddress_spacetranslation and returns whether the resolved MemoryRegion is backed by RAM.This will be used by the upcoming Tegra241 CMDQV support to validate guest provided VCMDQ buffer addresses.
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- include/exec/cpu-common.h | 2 ++ system/physmem.c | 12 ++++++++++++ 2 files changed, 14 insertions(+)include/exec/cpu-common.h +2
-0@@ -148,6 +148,8 @@ void qemu_flush_coalesced_mmio_buffer(void);typedef int (RAMBlockIterFunc )(RAMBlock *rb,void *opaque); +bool cpu_physical_memory_is_ram (hwaddr phys_addr); +int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque);/* vl.c */ system/physmem.c +0
-0
system/physmem.c int cpu_memory_rw_debug(CPUState *cpu, vaddr addr, · line 4071
Quoted 4 lines
system/physmem.c +1
-0@@ -4068,6 +4068,18 @@ int cpu_memory_rw_debug(CPUState *cpu, vaddr addr,return 0 ; } +bool cpu_physical_memory_is_ram (hwaddr phys_addr)
I guess this would need to match the new address_space_is_io()
Eric
Quoted 14 lines
system/physmem.c +11
-0+{ +MemoryRegion *mr; +hwaddr l =1 ; + +RCU_READ_LOCK_GUARD (); + mr =address_space_translate (&address_space_memory, phys_addr, &phys_addr, + &l, false,MEMTXATTRS_UNSPECIFIED ); + +return memory_region_is_ram (mr); +} +int qemu_ram_foreach_block (RAMBlockIterFunc func,void *opaque) {RAMBlock *block;
RFC PATCH 9/16 hw/arm/tegra241-cmdqv:: Add write emulation for registers
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Introduces write handling for VINTF and VCMDQ MMIO regions, including status/config updates, queue index tracking, and BASE_L/BASE_H processing. Writes to VCMDQ BASE_L/BASE_H trigger allocation or teardown of an IOMMUFD HW queue.
--- hw/arm/tegra241-cmdqv.c | 213 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 213 insertions(+)
hw/arm/tegra241-cmdqv.c +213 -0
@@ -210,11 +210,158 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +/* Note that offset aligns down to 0x1000 */ +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value,unsigned size) +{ +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } +} + +static bool tegra241_cmdqv_setup_vcmdq (Tegra241CMDQV *cmdqv,int index, +Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +uint64_t base_mask = (uint64_t )R_VCMDQ0_BASE_L_ADDR_MASK | + (uint64_t )R_VCMDQ0_BASE_H_ADDR_MASK <<32 ; +uint64_t addr = cmdqv->vcmdq_base[index] & base_mask; +uint64_t log2 = cmdqv->vcmdq_base[index] &R_VCMDQ0_BASE_L_LOG2SIZE_MASK ; +uint64_t size =1ULL << (log2 +4 ); +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; +IOMMUFDViommu *viommu; +IOMMUFDHWqueue *hw_queue; +uint32_t hw_queue_id; + +/* Ignore any invalid address. This may come as part of reset etc */ +if (!cpu_physical_memory_is_ram (addr)) { +return true; + } + +if (vcmdq) { +iommufd_backend_free_id (s_accel->viommu.iommufd, vcmdq->hw_queue_id); + cmdqv->vcmdq[index] =NULL ; +g_free (vcmdq); + } + + viommu = &s_accel->viommu; +if (!iommufd_backend_alloc_hw_queue (viommu->iommufd, viommu->viommu_id, +IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV , + index, addr, size, &hw_queue_id, + errp)) { +return false; + } + hw_queue =g_new (IOMMUFDHWqueue,1 ); + hw_queue->hw_queue_id = hw_queue_id; + hw_queue->viommu = viommu; + + cmdqv->vcmdq[index] = hw_queue; +return true; +} + +/* Note that offset aligns down to 0x10000 */ +static void +tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset,int index, +uint64_t value,unsigned size,Error **errp) +{ +uint32_t *ptr =NULL ; +uint64_t off; + +if (cmdqv->vcmdq_page0) { + off = (0x80 * index) + (offset -0x10000 ); + ptr = (uint32_t *)(cmdqv->vcmdq_page0 + off); + } + +switch (offset) { +case A_VCMDQ0_CONS_INDX : +if (ptr) { + *ptr = value; + } + cmdqv->vcmdq_cons_indx[index] = value; +return ; +case A_VCMDQ0_PROD_INDX : +if (ptr) { + *ptr = value; + } + cmdqv->vcmdq_prod_indx[index] = (uint32_t )value; +return ; +case A_VCMDQ0_CONFIG : +if (ptr) { + *ptr = (uint32_t )value; + }else { +if (value &R_VCMDQ0_CONFIG_CMDQ_EN_MASK ) { + cmdqv->vcmdq_status[index] |=R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + }else { + cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + } + } + cmdqv->vcmdq_config[index] = (uint32_t )value; +return ; +case A_VCMDQ0_GERRORN : +if (ptr) { + *ptr = (uint32_t )value; + } + cmdqv->vcmdq_gerrorn[index] = (uint32_t )value; +return ; +case A_VCMDQ0_BASE_L : +if (size ==8 ) { + cmdqv->vcmdq_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp); +return ; +case A_VCMDQ0_BASE_H : + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp); +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +if (size ==8 ) { + cmdqv->vcmdq_cons_indx_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } +} +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;Error *local_err =NULL ; +int index;if (!cmdqv->vcmdq_page0) {tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err);
@@ -223,6 +370,72 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, local_err =NULL ; } } + +if (offset >TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value, size); +break ; + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: +/* +* Align offset down to 0x10000 while extracting the index: +* VCMDQ0_CONS_INDX (0x10000) => 0x10000, 0 +* VCMDQ1_CONS_INDX (0x10080) => 0x10000, 1 +* VCMDQ2_CONS_INDX (0x10100) => 0x10000, 2 +* ... +* VCMDQ127_CONS_INDX (0x13f80) => 0x10000, 127 +*/ + index = (offset -0x10000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size, &local_err); +break ; + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* +* Align offset down to 0x20000 while extracting the index: +* VCMDQ0_BASE_L (0x20000) => 0x20000, 0 +* VCMDQ1_BASE_L (0x20080) => 0x20000, 1 +* VCMDQ2_BASE_L (0x20100) => 0x20000, 2 +* ... +* VCMDQ127_BASE_L (0x23f80) => 0x20000, 127 +*/ + index = (offset -0x20000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size, &local_err); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } + +if (local_err) { +error_report_err (local_err); + } }static const MemoryRegionOps mmio_cmdqv_ops = {
Signature · 1 line
2.43.0
Re: [RFC PATCH 09/16] hw/arm/tegra241-cmdqv:: Add write emulation for registers
Hi Shameer,
Quoted 6 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Introduces write handling for VINTF and VCMDQ MMIO regions, including status/config updates, queue index tracking, and BASE_L/BASE_H processing. Writes to VCMDQ BASE_L/BASE_H trigger allocation or teardown of an IOMMUFD HW queue.
I would strongly recommend to split read/write emulation patches into several patches. The code diff is huge and I am pretty sure you can split that into several functional changes. Then comes the question about to review that with a proper spec.
Eric
Quoted 243 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/tegra241-cmdqv.c | 213 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 213 insertions(+)hw/arm/tegra241-cmdqv.c +213
-0@@ -210,11 +210,158 @@ static uint64_t tegra241_cmdqv_read(void *opaque, hwaddr offset, unsigned size) } } +/* Note that offset aligns down to 0x1000 */ +static void tegra241_cmdqv_write_vintf (Tegra241CMDQV *cmdqv,hwaddr offset, +uint64_t value,unsigned size) +{ +switch (offset) { +case A_VINTF0_CONFIG : +/* Strip off HYP_OWN setting from guest kernel */ + value &= ~R_VINTF0_CONFIG_HYP_OWN_MASK ; + + cmdqv->vintf_config = value; +if (value &R_VINTF0_CONFIG_ENABLE_MASK ) { + cmdqv->vintf_status |=R_VINTF0_STATUS_ENABLE_OK_MASK ; + }else { + cmdqv->vintf_status &= ~R_VINTF0_STATUS_ENABLE_OK_MASK ; + } +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } +} + +static bool tegra241_cmdqv_setup_vcmdq (Tegra241CMDQV *cmdqv,int index, +Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +uint64_t base_mask = (uint64_t )R_VCMDQ0_BASE_L_ADDR_MASK | + (uint64_t )R_VCMDQ0_BASE_H_ADDR_MASK <<32 ; +uint64_t addr = cmdqv->vcmdq_base[index] & base_mask; +uint64_t log2 = cmdqv->vcmdq_base[index] &R_VCMDQ0_BASE_L_LOG2SIZE_MASK ; +uint64_t size =1ULL << (log2 +4 ); +IOMMUFDHWqueue *vcmdq = cmdqv->vcmdq[index]; +IOMMUFDViommu *viommu; +IOMMUFDHWqueue *hw_queue; +uint32_t hw_queue_id; + +/* Ignore any invalid address. This may come as part of reset etc */ +if (!cpu_physical_memory_is_ram (addr)) { +return true; + } + +if (vcmdq) { +iommufd_backend_free_id (s_accel->viommu.iommufd, vcmdq->hw_queue_id); + cmdqv->vcmdq[index] =NULL ; +g_free (vcmdq); + } + + viommu = &s_accel->viommu; +if (!iommufd_backend_alloc_hw_queue (viommu->iommufd, viommu->viommu_id, +IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV , + index, addr, size, &hw_queue_id, + errp)) { +return false; + } + hw_queue =g_new (IOMMUFDHWqueue,1 ); + hw_queue->hw_queue_id = hw_queue_id; + hw_queue->viommu = viommu; + + cmdqv->vcmdq[index] = hw_queue; +return true; +} + +/* Note that offset aligns down to 0x10000 */ +static void +tegra241_cmdqv_write_vcmdq (Tegra241CMDQV *cmdqv,hwaddr offset,int index, +uint64_t value,unsigned size,Error **errp) +{ +uint32_t *ptr =NULL ; +uint64_t off; + +if (cmdqv->vcmdq_page0) { + off = (0x80 * index) + (offset -0x10000 ); + ptr = (uint32_t *)(cmdqv->vcmdq_page0 + off); + } + +switch (offset) { +case A_VCMDQ0_CONS_INDX : +if (ptr) { + *ptr = value; + } + cmdqv->vcmdq_cons_indx[index] = value; +return ; +case A_VCMDQ0_PROD_INDX : +if (ptr) { + *ptr = value; + } + cmdqv->vcmdq_prod_indx[index] = (uint32_t )value; +return ; +case A_VCMDQ0_CONFIG : +if (ptr) { + *ptr = (uint32_t )value; + }else { +if (value &R_VCMDQ0_CONFIG_CMDQ_EN_MASK ) { + cmdqv->vcmdq_status[index] |=R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + }else { + cmdqv->vcmdq_status[index] &= ~R_VCMDQ0_STATUS_CMDQ_EN_OK_MASK ; + } + } + cmdqv->vcmdq_config[index] = (uint32_t )value; +return ; +case A_VCMDQ0_GERRORN : +if (ptr) { + *ptr = (uint32_t )value; + } + cmdqv->vcmdq_gerrorn[index] = (uint32_t )value; +return ; +case A_VCMDQ0_BASE_L : +if (size ==8 ) { + cmdqv->vcmdq_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp); +return ; +case A_VCMDQ0_BASE_H : + cmdqv->vcmdq_base[index] = + (cmdqv->vcmdq_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +tegra241_cmdqv_setup_vcmdq (cmdqv, index, errp); +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_L : +if (size ==8 ) { + cmdqv->vcmdq_cons_indx_base[index] = value; + }else if (size ==4 ) { + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffff00000000ULL ) | + (value &0xffffffffULL ); + } +return ; +case A_VCMDQ0_CONS_INDX_BASE_DRAM_H : + cmdqv->vcmdq_cons_indx_base[index] = + (cmdqv->vcmdq_cons_indx_base[index] &0xffffffffULL ) | + ((uint64_t )value <<32 ); +return ; +default : +qemu_log_mask (LOG_UNIMP , +"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); +return ; + } +} +static void tegra241_cmdqv_write (void *opaque,hwaddr offset,uint64_t value,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;Error *local_err =NULL ; +int index;if (!cmdqv->vcmdq_page0) {tegra241_cmdqv_init_vcmdq_page0 (cmdqv, &local_err);@@ -223,6 +370,72 @@ static void tegra241_cmdqv_write(void *opaque, hwaddr offset, uint64_t value, local_err =NULL ; } } + +if (offset >TEGRA241_CMDQV_IO_LEN ) { +qemu_log_mask (LOG_UNIMP , +"%s offset 0x%" PRIx64" off limit (0x50000)\n" , __func__, + offset); +return ; + } + +switch (offset) { +case A_CONFIG : + cmdqv->config = value; +if (value &R_CONFIG_CMDQV_EN_MASK ) { + cmdqv->status |=R_STATUS_CMDQV_ENABLED_MASK ; + }else { + cmdqv->status &= ~R_STATUS_CMDQV_ENABLED_MASK ; + } +break ; + case A_VI_INT_MASK ... A_VI_INT_MASK_1: + cmdqv->vi_int_mask[(offset -A_VI_INT_MASK ) /4 ] = value; +break ; + case A_CMDQ_ALLOC_MAP_0 ... A_CMDQ_ALLOC_MAP_127: + cmdqv->cmdq_alloc_map[(offset -A_CMDQ_ALLOC_MAP_0 ) /4 ] = value; +break ; + case A_VINTF0_CONFIG ... A_VINTF0_LVCMDQ_ERR_MAP_3: +tegra241_cmdqv_write_vintf (cmdqv, offset, value, size); +break ; + case A_VI_VCMDQ0_CONS_INDX ... A_VI_VCMDQ127_GERRORN: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_CONS_INDX ... A_VCMDQ127_GERRORN: +/* +* Align offset down to 0x10000 while extracting the index: +* VCMDQ0_CONS_INDX (0x10000) => 0x10000, 0 +* VCMDQ1_CONS_INDX (0x10080) => 0x10000, 1 +* VCMDQ2_CONS_INDX (0x10100) => 0x10000, 2 +* ... +* VCMDQ127_CONS_INDX (0x13f80) => 0x10000, 127 +*/ + index = (offset -0x10000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size, &local_err); +break ; + case A_VI_VCMDQ0_BASE_L ... A_VI_VCMDQ127_CONS_INDX_BASE_DRAM_H: + offset -=0x20000 ; +QEMU_FALLTHROUGH ; + case A_VCMDQ0_BASE_L ... A_VCMDQ127_CONS_INDX_BASE_DRAM_H: +/* +* Align offset down to 0x20000 while extracting the index: +* VCMDQ0_BASE_L (0x20000) => 0x20000, 0 +* VCMDQ1_BASE_L (0x20080) => 0x20000, 1 +* VCMDQ2_BASE_L (0x20100) => 0x20000, 2 +* ... +* VCMDQ127_BASE_L (0x23f80) => 0x20000, 127 +*/ + index = (offset -0x20000 ) /0x80 ; +tegra241_cmdqv_write_vcmdq (cmdqv, offset -0x80 * index, index, value, + size, &local_err); +break ; +default : +qemu_log_mask (LOG_UNIMP ,"%s unhandled write access at 0x%" PRIx64"\n" , + __func__, offset); + } + +if (local_err) { +error_report_err (local_err); + } }static const MemoryRegionOps mmio_cmdqv_ops = {
RFC PATCH 10/16 hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
In reply to Shameer KolothumAllocate a Tegra241 CMDQV type vEVENTQ object so that any host side errors related to the CMDQV can be received and propagated back to the guest.
Event read and propagation will be added in a later patch.
--- hw/arm/tegra241-cmdqv.c | 51 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 1 + 2 files changed, 52 insertions(+)
hw/arm/tegra241-cmdqv.c +51 -0
@@ -136,6 +136,52 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset,return 0 ; } } + +static void tegra241_cmdqv_free_veventq (Tegra241CMDQV *cmdqv) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu = &s_accel->viommu; +IOMMUFDVeventq *veventq = cmdqv->veventq; + +if (!veventq) { +return ; + } + +iommufd_backend_free_id (viommu->iommufd, veventq->veventq_id); +g_free (veventq); + cmdqv->veventq =NULL ; +} + +static bool tegra241_cmdqv_alloc_veventq (Tegra241CMDQV *cmdqv,Error **errp) +{ +SMMUv3State *smmu = cmdqv->smmu; +SMMUv3AccelState *s_accel = smmu->s_accel; +IOMMUFDViommu *viommu = &s_accel->viommu; +IOMMUFDVeventq *veventq; +uint32_t veventq_id; +uint32_t veventq_fd; + +if (cmdqv->veventq) { +return true; + } + +if (!iommufd_backend_alloc_veventq (viommu->iommufd, viommu->viommu_id, +IOMMU_VEVENTQ_TYPE_TEGRA241_CMDQV , +1 <<16 , &veventq_id, &veventq_fd, + errp)) { +error_append_hint (errp,"Tegra241 CMDQV: failed to alloc veventq" ); +return false; + } + + veventq =g_new (IOMMUFDVeventq,1 ); + veventq->veventq_id = veventq_id; + veventq->veventq_fd = veventq_fd; + veventq->viommu = viommu; + cmdqv->veventq = veventq; +return true; +} +static uint64_t tegra241_cmdqv_read (void *opaque,hwaddr offset,unsigned size) {Tegra241CMDQV *cmdqv = (Tegra241CMDQV *)opaque;
@@ -259,11 +305,16 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,g_free (vcmdq); } +if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } + viommu = &s_accel->viommu;if (!iommufd_backend_alloc_hw_queue (viommu->iommufd, viommu->viommu_id,IOMMU_HW_QUEUE_TYPE_TEGRA241_CMDQV , index, addr, size, &hw_queue_id, errp)) { +tegra241_cmdqv_free_veventq (cmdqv);return false; } hw_queue =g_new (IOMMUFDHWqueue,1 );
hw/arm/tegra241-cmdqv.h +1 -0
@@ -24,6 +24,7 @@ typedef struct Tegra241CMDQV {MemoryRegion mmio_vintf_page;void *vcmdq_page0;IOMMUFDHWqueue *vcmdq[128 ]; +IOMMUFDVeventq *veventq;/* Register Cache */ uint32_t config;
Signature · 1 line
2.43.0
Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 311
Quoted 8 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +4
-0@@ -259,11 +305,16 @@ static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,g_free (vcmdq); } +if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } +
I think this should be called in tegra241_cmdqv_alloc_viommu().
FWIW, vEVENTQ is crucial for CMDQV, because VINTF/VCMDQ will not work if guest OS doesn't recover the HW error correctly.
Nicolin
RE: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
Quoted 21 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 19:50
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ objectReview static bool
Quoted 1 line · Shameer Kolothum
(quoted diff) +0
-0@@ -259,11 +305,16 @@ static booltegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0g_free (vcmdq); }
Quoted 6 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 311
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } +I think this should be called in
tegra241_cmdqv_alloc_viommu().
Should/Can we? _alloc_viommu() will happen before the Guest boot. Unlike the SMMUv3 vEVENTQ for accel case, we restrict the hotplug use case when the tegra214-cmdqv is set. And as soon as Guest setup vCMDQ the vEVENTQ is enabled here. Do you see any issues with this approach?
Thanks,
Shameer
Thanks,
Shameer
Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
Quoted 18 lines · Shameer Kolothum
Quoted 18 lines
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 19:50
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ objectReview static bool
Quoted 1 line · Shameer Kolothum
(quoted diff) +0
-0@@ -259,11 +305,16 @@ static booltegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0g_free (vcmdq); }
Quoted 8 lines
Quoted 6 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 311
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } +I think this should be called in
tegra241_cmdqv_alloc_viommu().Should/Can we?
_alloc_viommu()will happen before the Guest boot.
I think we can. vEVENTQ only needs vIOMMU ID. It's per VINTF v.s. per VCMDQ (HW_QUEUE). So, it should follow vIOMMU.
Basically, we can allocate vIOMMU (VINTF) and its vEVENTQ even if the VINTF doesn't contain any LVCMDQs, which are allocated/mapped to the VINTF later via HW_QUEUE allocations.
Quoted 3 lines
Unlike the SMMUv3 vEVENTQ for accel case, we restrict the hotplug use case when the tegra214-cmdqv is set. And as soon as Guest setup vCMDQ the vEVENTQ is enabled here. Do you see any issues with this approach?
Once guest OS is booting, !tegra241_cmdqv_alloc_veventq will fail tegra241_cmdqv_setup_vcmdq while it cannot break the guest OS any more. Then, the whole thing wouldn't work although guest OS runs?
Nicolin
RE: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
Quoted 43 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 30 December 2025 17:54
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ objectQuoted 6 lines · Shameer Kolothum
Quoted 6 lines
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 19:50
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; JasonGunthorpe
Quoted 4 lines
Quoted 4 lines
<[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: AllocatevEVENTQ
Quoted 15 lines
Quoted 14 lines
object
Review static bool
Quoted 1 line · Shameer Kolothum
(quoted diff) +0
-0@@ -259,11 +305,16 @@ static booltegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 311
Quoted 7 lines
hw/arm/tegra241-cmdqv.c +4
-0g_free (vcmdq); } +if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } +I think this should be called in
tegra241_cmdqv_alloc_viommu().
Quoted 8 lines
Quoted 1 line
Should/Can we?
_alloc_viommu()will happen before the Guest boot.I think we can. vEVENTQ only needs vIOMMU ID. It's per VINTF v.s. per VCMDQ (
HW_QUEUE). So, it should follow vIOMMU.Basically, we can allocate vIOMMU (VINTF) and its vEVENTQ even if the VINTF doesn't contain any LVCMDQs, which are allocated/mapped to the VINTF later via
HW_QUEUEallocations.
But what happens when you receive an event from host during that time? Should we just ignore it? Or is that not possible at all from a hoist driver perspective? To me, it looks like the natural flow is setup vEVENTQ when Guest is ready to receive the events.
Quoted 8 lines
Quoted 2 lines
Unlike the SMMUv3 vEVENTQ for accel case, we restrict the hotplug use case when the tegra214-cmdqv is set. And as soon as Guest setup
vCMDQ
Quoted 1 line
the vEVENTQ is enabled here. Do you see any issues with this approach?
Once guest OS is booting, !tegra241_cmdqv_alloc_veventq will fail
tegra241_cmdqv_setup_vcmdqwhile it cannot break the guest OS any more. Then, the whole thing wouldn't work although guest OS runs?
If vEVENTQ is mandatory we could exit the Guest with a proper msg in case tegra241_cmdqv_alloc_veventq() fails, right?
Thanks,
Shameer
Quoted 1 line
Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
Quoted 18 lines · Shameer Kolothum
Quoted 18 lines
Quoted 6 lines
Quoted 6 lines · Shameer Kolothum
Review static bool
Quoted 1 line
(quoted diff) +0
-0@@ -259,11 +305,16 @@ static booltegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0g_free (vcmdq); }Quoted 8 lines
Quoted 6 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 311
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } +I think this should be called in
tegra241_cmdqv_alloc_viommu().Should/Can we?
_alloc_viommu()will happen before the Guest boot.I think we can. vEVENTQ only needs vIOMMU ID. It's per VINTF v.s. per VCMDQ (
HW_QUEUE). So, it should follow vIOMMU.
Quoted 8 lines
Quoted 3 lines
Basically, we can allocate vIOMMU (VINTF) and its vEVENTQ even if the VINTF doesn't contain any LVCMDQs, which are allocated/mapped to the VINTF later via
HW_QUEUEallocations.But what happens when you receive an event from host during that time? Should we just ignore it? Or is that not possible at all from a hoist driver perspective? To me, it looks like the natural flow is setup vEVENTQ when Guest is ready to receive the events.
I don't think that will happen. The IRQ register for VINTF/vIOMMU is LVCMDQ_ERR_MAP. If there is no HW_QUEUE allocated to the VINTF, there shouldn't be a vEVENT. Otherwise, it would be a kernel bug, IMHO.
Quoted 11 lines
Quoted 8 lines
Quoted 2 lines
Unlike the SMMUv3 vEVENTQ for accel case, we restrict the hotplug use case when the tegra214-cmdqv is set. And as soon as Guest setup
vCMDQ
Quoted 1 line
the vEVENTQ is enabled here. Do you see any issues with this approach?
Once guest OS is booting, !tegra241_cmdqv_alloc_veventq will fail
tegra241_cmdqv_setup_vcmdqwhile it cannot break the guest OS any more. Then, the whole thing wouldn't work although guest OS runs?If vEVENTQ is mandatory we could exit the Guest with a proper msg in case
tegra241_cmdqv_alloc_veventq()fails, right?
I don't know. But that doesn't sound right to me. In my mind, once the guest OS is booting, it could only stop with something like a blue screen..
Maybe QEMU experts can shed some light here.
Nicolin
RE: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ object
Quoted 44 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 30 December 2025 18:22
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 10/16] hw/arm/tegra241-cmdqv: Allocate vEVENTQ objectQuoted 1 line · Shameer Kolothum
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Wed, Dec 10, 2025 at 01:37:31PM +0000, Shameer Kolothum
Quoted 17 lines · wrote:
Quoted 17 lines
Quoted 5 lines
Quoted 5 lines
Review static bool
Quoted 1 line
(quoted diff) +0
-0@@ -259,11 +305,16 @@ static booltegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index,
Quoted 3 lines
hw/arm/tegra241-cmdqv.c +0
-0g_free (vcmdq); }Quoted 8 lines
Quoted 6 lines
hw/arm/tegra241-cmdqv.c static bool tegra241_cmdqv_setup_vcmdq(Tegra241CMDQV *cmdqv, int index, · line 311
Quoted 4 lines
hw/arm/tegra241-cmdqv.c +4
-0+if (!tegra241_cmdqv_alloc_veventq (cmdqv, errp)) { +return false; + } +I think this should be called in
tegra241_cmdqv_alloc_viommu().Should/Can we?
_alloc_viommu()will happen before the Guest boot.I think we can. vEVENTQ only needs vIOMMU ID. It's per VINTF v.s. per VCMDQ (
HW_QUEUE). So, it should follow vIOMMU.Quoted 8 lines
Quoted 3 lines
Basically, we can allocate vIOMMU (VINTF) and its vEVENTQ even if the VINTF doesn't contain any LVCMDQs, which are allocated/mapped to the VINTF later via
HW_QUEUEallocations.But what happens when you receive an event from host during that time? Should we just ignore it? Or is that not possible at all from a hoist driver perspective? To me, it looks like the natural flow is setup vEVENTQ when Guest is ready to receive the events.
Quoted 4 lines
I don't think that will happen. The IRQ register for VINTF/vIOMMU is
LVCMDQ_ERR_MAP. If there is noHW_QUEUEallocated to the VINTF, there shouldn't be a vEVENT. Otherwise, it would be a kernel bug, IMHO.
Ok. That is reassuring then. I will add that as a comment and move it.
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 2 lines
Unlike the SMMUv3 vEVENTQ for accel case, we restrict the hotplug use case when the tegra214-cmdqv is set. And as soon as Guest setup
vCMDQ
Quoted 1 line
the vEVENTQ is enabled here. Do you see any issues with this approach?
Once guest OS is booting, !tegra241_cmdqv_alloc_veventq will fail
tegra241_cmdqv_setup_vcmdqwhile it cannot break the guest OS any more. Then, the whole thing wouldn't work although guest OS runs?
Quoted 8 lines
Quoted 2 lines
If vEVENTQ is mandatory we could exit the Guest with a proper msg in case
tegra241_cmdqv_alloc_veventq()fails, right?I don't know. But that doesn't sound right to me. In my mind, once the guest OS is booting, it could only stop with something like a blue screen..
Maybe QEMU experts can shed some light here.
I think, generally it is a last resort option and should be avoided.
Thanks,
Shameer
RFC PATCH 11/16 hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
In reply to Shameer KolothumInstall an event handler on the CMDQV vEVENTQ fd to read and propagate host received CMDQV errors to the guest.
The handler runs in QEMU’s main loop, using a non-blocking fd registered via qemu_set_fd_handler().
--- hw/arm/tegra241-cmdqv.c | 80 +++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 2 ++ hw/arm/trace-events | 3 ++ 3 files changed, 85 insertions(+)
hw/arm/tegra241-cmdqv.c +80 -0
@@ -8,9 +8,12 @@*/ #include "qemu/osdep.h" +#include "qemu/error-report.h" #include "qemu/log.h" +#include "trace.h" #include "hw/arm/smmuv3.h" +#include "hw/irq.h" #include "smmuv3-accel.h" #include "tegra241-cmdqv.h"
@@ -137,6 +140,79 @@ static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset, } } +static void tegra241_cmdqv_event_read (void *opaque) +{ +Tegra241CMDQV *cmdqv = opaque; +struct { +struct iommufd_vevent_header hdr; +struct iommu_vevent_tegra241_cmdqv vevent; + } buf; +ssize_t readsz =sizeof (buf); +uint32_t last_seq = cmdqv->last_event_seq; +ssize_t bytes; + + bytes =read (cmdqv->veventq->veventq_fd, &buf, readsz); +if (bytes <=0 ) { +if (errno ==EAGAIN || errno ==EINTR ) { +return ; + } +error_report ("Tegra241 CMDQV: vEVENTQ: read failed (%s)" , +strerror (errno)); +return ; + } + +if (bytes < readsz) { +error_report ("Tegra241 CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)" , + bytes, readsz); +return ; + } + +if (buf.hdr.flags &IOMMU_VEVENTQ_FLAG_LOST_EVENTS ) { +error_report ("Tegra241 CMDQV: vEVENTQ has lost events" ); +return ; + } + +/* Check sequence in hdr for lost events if any */ +if (cmdqv->event_start) { +uint32_t expected = (last_seq ==INT_MAX ) ?0 : last_seq +1 ; + +if (buf.hdr.sequence != expected) { +uint32_t delta; + +if (buf.hdr.sequence >= last_seq) { + delta = buf.hdr.sequence - last_seq; + }else { +/* Handle wraparound from INT_MAX */ + delta = (INT_MAX - last_seq) + buf.hdr.sequence +1 ; + } +error_report ("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)" , + delta -1 ); + } + } + +if (buf.vevent.lvcmdq_err_map[0 ] || buf.vevent.lvcmdq_err_map[1 ]) { + cmdqv->vintf_cmdq_err_map[0 ] = + buf.vevent.lvcmdq_err_map[0 ] &0xffffffff ; + cmdqv->vintf_cmdq_err_map[1 ] = + (buf.vevent.lvcmdq_err_map[0 ] >>32 ) &0xffffffff ; + cmdqv->vintf_cmdq_err_map[2 ] = + buf.vevent.lvcmdq_err_map[1 ] &0xffffffff ; + cmdqv->vintf_cmdq_err_map[3 ] = + (buf.vevent.lvcmdq_err_map[1 ] >>32 ) &0xffffffff ; +for (int i =0 ; i <4 ; i++) { + cmdqv->cmdq_err_map[i] = cmdqv->vintf_cmdq_err_map[i]; + } + cmdqv->vi_err_map[0 ] |=0x1 ; +qemu_irq_pulse (cmdqv->irq); +trace_tegra241_cmdqv_err_map ( + cmdqv->vintf_cmdq_err_map[3 ], cmdqv->vintf_cmdq_err_map[2 ], + cmdqv->vintf_cmdq_err_map[1 ], cmdqv->vintf_cmdq_err_map[0 ]); + } + + cmdqv->last_event_seq = buf.hdr.sequence; + cmdqv->event_start = true; +} +static void tegra241_cmdqv_free_veventq (Tegra241CMDQV *cmdqv) {SMMUv3State *smmu = cmdqv->smmu;
@@ -179,6 +255,10 @@ static bool tegra241_cmdqv_alloc_veventq(Tegra241CMDQV *cmdqv, Error **errp) veventq->veventq_fd = veventq_fd; veventq->viommu = viommu; cmdqv->veventq = veventq; + +/* Set up event handler for veventq fd */ +fcntl (veventq_fd,F_SETFL ,O_NONBLOCK ); +qemu_set_fd_handler (veventq_fd, tegra241_cmdqv_event_read,NULL , cmdqv);return true; }
hw/arm/tegra241-cmdqv.h +2 -0
@@ -25,6 +25,8 @@ typedef struct Tegra241CMDQV {void *vcmdq_page0;IOMMUFDHWqueue *vcmdq[128 ];IOMMUFDVeventq *veventq; +uint32_t last_event_seq; +bool event_start;/* Register Cache */ uint32_t config;
hw/arm/trace-events +3 -0
@@ -72,6 +72,9 @@ smmuv3_accel_unset_iommu_device(int devfn, uint32_t devid) "devfn=0x%x (idev dev smmuv3_accel_translate_ste(uint32_t vsid, uint32_t hwpt_id, uint64_t ste_1, uint64_t ste_0) "vSID=0x%x hwpt_id=0x%x ste=%"PRIx64":%"PRIx64 smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vSID=0x%x ste type=%s hwpt_id=0x%x" +# tegra241-cmdqv +tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X" + # strongarm.c strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d" strongarm_ssp_read_underrun(void) "SSP rx underrun"
Signature · 1 line
2.43.0
Re: [RFC PATCH 11/16] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
Quoted 39 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +39
-0+static void tegra241_cmdqv_event_read (void *opaque) +{ +Tegra241CMDQV *cmdqv = opaque; +struct { +struct iommufd_vevent_header hdr; +struct iommu_vevent_tegra241_cmdqv vevent; + } buf; +ssize_t readsz =sizeof (buf); +uint32_t last_seq = cmdqv->last_event_seq; +ssize_t bytes; + + bytes =read (cmdqv->veventq->veventq_fd, &buf, readsz); +if (bytes <=0 ) { +if (errno ==EAGAIN || errno ==EINTR ) { +return ; + } +error_report ("Tegra241 CMDQV: vEVENTQ: read failed (%s)" , +strerror (errno)); +return ; + } + +if (bytes < readsz) { +error_report ("Tegra241 CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)" , + bytes, readsz); +return ; + } + +if (buf.hdr.flags &IOMMU_VEVENTQ_FLAG_LOST_EVENTS ) { +error_report ("Tegra241 CMDQV: vEVENTQ has lost events" ); +return ; + } + +/* Check sequence in hdr for lost events if any */ +if (cmdqv->event_start) { +uint32_t expected = (last_seq ==INT_MAX ) ?0 : last_seq +1 ; + +if (buf.hdr.sequence != expected) { +uint32_t delta; +
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset, · line 191
Quoted 10 lines
hw/arm/tegra241-cmdqv.c +10
-0+if (buf.hdr.sequence >= last_seq) { + delta = buf.hdr.sequence - last_seq; + }else { +/* Handle wraparound from INT_MAX */ + delta = (INT_MAX - last_seq) + buf.hdr.sequence +1 ; + } +error_report ("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)" , + delta -1 ); + } + }
This part looks quite similar to SMMUv3's vEVENTQ function. Maybe add a vEVENTQ helper in the iommufd backend?
Nicolin
Re: [RFC PATCH 11/16] hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
Quoted 44 lines · Nicolin Chen
Quoted 43 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +43
-0+static void tegra241_cmdqv_event_read (void *opaque) +{ +Tegra241CMDQV *cmdqv = opaque; +struct { +struct iommufd_vevent_header hdr; +struct iommu_vevent_tegra241_cmdqv vevent; + } buf; +ssize_t readsz =sizeof (buf); +uint32_t last_seq = cmdqv->last_event_seq; +ssize_t bytes; + + bytes =read (cmdqv->veventq->veventq_fd, &buf, readsz); +if (bytes <=0 ) { +if (errno ==EAGAIN || errno ==EINTR ) { +return ; + } +error_report ("Tegra241 CMDQV: vEVENTQ: read failed (%s)" , +strerror (errno)); +return ; + } + +if (bytes < readsz) { +error_report ("Tegra241 CMDQV: vEVENTQ: incomplete read (%zd/%zd bytes)" , + bytes, readsz); +return ; + } + +if (buf.hdr.flags &IOMMU_VEVENTQ_FLAG_LOST_EVENTS ) { +error_report ("Tegra241 CMDQV: vEVENTQ has lost events" ); +return ; + } + +/* Check sequence in hdr for lost events if any */ +if (cmdqv->event_start) { +uint32_t expected = (last_seq ==INT_MAX ) ?0 : last_seq +1 ; + +if (buf.hdr.sequence != expected) { +uint32_t delta; + +if (buf.hdr.sequence >= last_seq) { + delta = buf.hdr.sequence - last_seq; + }else { +/* Handle wraparound from INT_MAX */
Quoted 8 lines
hw/arm/tegra241-cmdqv.c static uint64_t tegra241_cmdqv_read_vcmdq(Tegra241CMDQV *cmdqv, hwaddr offset, · line 191
Quoted 6 lines
hw/arm/tegra241-cmdqv.c +6
-0+ delta = (INT_MAX - last_seq) + buf.hdr.sequence +1 ; + } +error_report ("Tegra241 CMDQV: vEVENTQ: detected lost %u event(s)" , + delta -1 ); + } + }This part looks quite similar to SMMUv3's vEVENTQ function. Maybe add a vEVENTQ helper in the iommufd backend?
+1
Eric
Quoted 3 lines
Nicolin
RFC PATCH 12/16 hw/arm/tegra241-cmdqv: Add reset handler
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Introduce a reset handler for the Tegra241 CMDQV and initialize its register state.
CMDQV gets initialized early during guest boot, hence the handler verifies that at least one cold-plugged device is attached to the associated vIOMMU before proceeding. This is required to retrieve host CMDQV info and to validate it against the QEMU implementation support.
--- hw/arm/smmuv3.c | 1 + hw/arm/tegra241-cmdqv.c | 105 ++++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 7 +++ hw/arm/trace-events | 1 + 4 files changed, 114 insertions(+)
hw/arm/smmuv3.c +1 -0
@@ -1943,6 +1943,7 @@ static void smmu_reset_exit(Object *obj, ResetType type)smmuv3_reset (s);smmuv3_accel_reset (s); +tegra241_cmdqv_reset (s); }static bool smmu_validate_property (SMMUv3State *s,Error **errp)
hw/arm/tegra241-cmdqv.c +105 -0
111 lines not on this page · raw message
hw/arm/tegra241-cmdqv.h +7 -0
@@ -13,6 +13,9 @@#include "hw/registerfields.h" #include CONFIG_DEVICES +#define TEGRA241_CMDQV_VERSION 0x1 +#define TEGRA241_CMDQV_NUM_CMDQ_LOG2 0x1 +#define TEGRA241_CMDQV_NUM_SID_PER_VM_LOG2 0x4#define TEGRA241_CMDQV_IO_LEN 0x50000 typedef struct Tegra241CMDQV {
@@ -314,11 +317,15 @@ A_VINTFi_CONFIG(0)#ifdef CONFIG_TEGRA241_CMDQV bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp); +void tegra241_cmdqv_reset (SMMUv3State *s);void tegra241_cmdqv_init (SMMUv3State *s);#else static inline void tegra241_cmdqv_init (SMMUv3State *s) { } +static inline void tegra241_cmdqv_reset (SMMUv3State *s) +{ +}static inline bool tegra241_cmdqv_alloc_viommu (SMMUv3State *s,HostIOMMUDeviceIOMMUFD *idev,uint32_t *out_viommu_id,Error **errp)
hw/arm/trace-events +1 -0
@@ -74,6 +74,7 @@ smmuv3_accel_install_ste(uint32_t vsid, const char * type, uint32_t hwpt_id) "vS # tegra241-cmdqv tegra241_cmdqv_err_map(uint32_t map3, uint32_t map2, uint32_t map1, uint32_t map0) "hw irq received. error (hex) maps: %04X:%04X:%04X:%04X" +tegra241_cmdqv_init_regs(uint32_t param) "hw info received. param: 0x%04X" # strongarm.c strongarm_uart_update_parameters(const char *label, int speed, char parity, int data_bits, int stop_bits) "%s speed=%d parity=%c data=%d stop=%d"
Signature · 1 line
2.43.0
RFC PATCH 13/16 hw/arm/tegra241-cmdqv: Limit queue size based on backend page size
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
CMDQV HW reads guest queue memory in its host physical address setup via IOMUUFD. This requires the guest queue memory isn't only contiguous in guest PA space but also in host PA space. With Tegra241 CMDQV enabled, we must only advertise a CMDQV size that the host can safely back with physically contiguous memory. Allowing a CMDQV larger than the host page size could cause the hardware to DMA across page boundaries leading to faults.
Limit IDR1.CMDQS so the guest cannot configure a CMDQV that exceeds the host’s contiguous backing.
--- hw/arm/tegra241-cmdqv.c | 43 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
hw/arm/tegra241-cmdqv.c +43 -0
@@ -11,10 +11,14 @@#include "qemu/error-report.h" #include "qemu/log.h" #include "trace.h" +#include <math.h> #include "hw/arm/smmuv3.h" #include "hw/irq.h" #include "smmuv3-accel.h" +#include "smmuv3-internal.h" +#include "system/ramblock.h" +#include "exec/ramlist.h" #include "tegra241-cmdqv.h" static bool tegra241_cmdqv_init_vcmdq_page0 (Tegra241CMDQV *cmdqv,Error **errp)
@@ -592,6 +596,33 @@ bool tegra241_cmdqv_alloc_viommu(SMMUv3State *s, HostIOMMUDeviceIOMMUFD *idev,return true; } +static size_t tegra241_cmdqv_min_ram_pagesize (void ) +{ +RAMBlock *rb; +size_t pg, min_pg =SIZE_MAX ; + +RAMBLOCK_FOREACH (rb) { +MemoryRegion *mr = rb->mr; + +/* Only consider real RAM regions */ +if (!mr || !memory_region_is_ram (mr)) { +continue ; + } + +/* Skip RAM regions that are not backed by a memory-backend */ +if (!object_dynamic_cast (mr->owner,TYPE_MEMORY_BACKEND )) { +continue ; + } + + pg =qemu_ram_pagesize (rb); +if (pg && pg < min_pg) { + min_pg = pg; + } + } + +return (min_pg ==SIZE_MAX ) ?qemu_real_host_page_size () : min_pg; +} +static void tegra241_cmdqv_init_regs (SMMUv3State *s,Tegra241CMDQV *cmdqv) {SMMUv3AccelState *s_accel = s->s_accel;
@@ -599,7 +630,9 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv)struct iommu_hw_info_tegra241_cmdqv cmdqv_info;SMMUv3AccelDevice *accel_dev;Error *local_err =NULL ; +size_t pgsize;uint64_t caps; +uint32_t val;int i;if (QLIST_EMPTY (&s_accel->device_list)) {
@@ -670,6 +703,16 @@ static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv) cmdqv->vcmdq_base[i] =0 ; cmdqv->vcmdq_cons_indx_base[i] =0 ; } + +/* +* CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the +* queue fits entirely within the smallest backend page size. +* FIXME: Migration support requires this to be taken care. +*/ + pgsize =tegra241_cmdqv_min_ram_pagesize (); + val =FIELD_EX32 (s->idr[1 ],IDR1 ,CMDQS ); + s->idr[1 ] =FIELD_DP32 (s->idr[1 ],IDR1 ,CMDQS ,MIN (log2 (pgsize) -4 , val)); +return ; out_err:
Signature · 1 line
2.43.0
Re: [RFC PATCH 13/16] hw/arm/tegra241-cmdqv: Limit queue size based on backend page size
hw/arm/tegra241-cmdqv.c static void tegra241_cmdqv_init_regs(SMMUv3State *s, Tegra241CMDQV *cmdqv) · line 714
Quoted 8 lines · Shameer Kolothum
hw/arm/tegra241-cmdqv.c +8
-0+/* +* CMDQ must not cross a physical RAM backend page. Adjust CMDQS so the +* queue fits entirely within the smallest backend page size. +* FIXME: Migration support requires this to be taken care. +*/ + pgsize =tegra241_cmdqv_min_ram_pagesize (); + val =FIELD_EX32 (s->idr[1 ],IDR1 ,CMDQS ); + s->idr[1 ] =FIELD_DP32 (s->idr[1 ],IDR1 ,CMDQS ,MIN (log2 (pgsize) -4 , val));
Let's mention it is for the physical contiguity of the Q memory.
Nicolin
RFC PATCH 14/16 virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev
In reply to Shameer KolothumRename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.
No functional changes intended.
--- hw/arm/virt-acpi-build.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-)
hw/arm/virt-acpi-build.c +18 -18
@@ -339,7 +339,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)return idmap_a->input_base - idmap_b->input_base; } -typedef struct AcpiIortSMMUv3Dev { +typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; GArray *rc_smmu_idmaps;
@@ -347,16 +347,16 @@ typedef struct AcpiIortSMMUv3Dev {size_t offset;bool accel;bool ats; -} AcpiIortSMMUv3Dev; +} AcpiSMMUv3Dev;/* -* Populate the struct AcpiIortSMMUv3Dev for the legacy SMMUv3 and +* Populate the struct AcpiSMMUv3Dev for the legacy SMMUv3 and * return the total number of associated idmaps. */ static int populate_smmuv3_legacy_dev (GArray *sdev_blob) {VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev; +AcpiSMMUv3Dev sdev; sdev.rc_smmu_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping)); object_child_foreach_recursive(object_get_root(), iort_host_bridges,
@@ -376,8 +376,8 @@ static int populate_smmuv3_legacy_dev(GArray *sdev_blob)static int smmuv3_dev_idmap_compare (gconstpointer a,gconstpointer b) { -AcpiIortSMMUv3Dev *sdev_a = (AcpiIortSMMUv3Dev *)a; -AcpiIortSMMUv3Dev *sdev_b = (AcpiIortSMMUv3Dev *)b; +AcpiSMMUv3Dev *sdev_a = (AcpiSMMUv3Dev *)a; +AcpiSMMUv3Dev *sdev_b = (AcpiSMMUv3Dev *)b;AcpiIortIdMapping *map_a = &g_array_index (sdev_a->rc_smmu_idmaps, AcpiIortIdMapping,0 ); AcpiIortIdMapping *map_b = &g_array_index(sdev_b->rc_smmu_idmaps,
@@ -391,7 +391,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)GArray *sdev_blob = opaque;AcpiIortIdMapping idmap;PlatformBusDevice *pbus; -AcpiIortSMMUv3Dev sdev; +AcpiSMMUv3Dev sdev;int min_bus, max_bus;SysBusDevice *sbdev;PCIBus *bus;
@@ -421,7 +421,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) }/* -* Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and +* Populate the struct AcpiSMMUv3Dev for all SMMUv3 devices and * return the total number of idmaps. */ static int populate_smmuv3_dev (GArray *sdev_blob)
@@ -442,10 +442,10 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) {AcpiIortIdMapping *idmap;AcpiIortIdMapping next_range = {0 }; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;for (int i =0 ; i < smmuv3_devs->len; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Based on the RID ranges that are directed to the SMMU, determine the * bypassed RID ranges, i.e., the ones that are directed to the ITS
@@ -479,7 +479,7 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)static void build_iort_rmr_nodes (GArray *table_data,GArray *smmuv3_devices,uint32_t *id) { -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;AcpiIortIdMapping *idmap;int i;
@@ -487,7 +487,7 @@ build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)uint16_t rmr_len;int bdf; - sdev = &g_array_index (smmuv3_devices, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devices, AcpiSMMUv3Dev, i);if (!sdev->accel) {continue ; }
@@ -544,13 +544,13 @@ static void build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) {int i, nb_nodes, rc_mapping_count; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;size_t node_size;bool ats_needed = false;int num_smmus =0 ;uint32_t id =0 ;int rc_smmu_idmaps_len =0 ; -GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiIortSMMUv3Dev)); +GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiSMMUv3Dev));GArray *rc_its_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping));AcpiTable table = { .sig ="IORT" , .rev =5 , .oem_id = vms->oem_id,
@@ -581,7 +581,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }/* Calculate RMR nodes required. One per SMMUv3 with accelerated mode */ for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);if (sdev->ats) { ats_needed = true; }
@@ -620,7 +620,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);int smmu_mapping_count, offset_to_id_array;int irq = sdev->irq;
@@ -699,7 +699,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)AcpiIortIdMapping *range;for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Map RIDs (input) from RC to SMMUv3 nodes: RC -> SMMUv3.
@@ -742,7 +742,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_table_end (linker, &table);g_array_free (rc_its_idmaps, true);for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);g_array_free (sdev->rc_smmu_idmaps, true); }g_array_free (smmuv3_devs, true);
Signature · 1 line
2.43.0
Re: [RFC PATCH 14/16] virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev
Quoted 7 lines · Shameer Kolothum
Rename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.
No functional changes intended.
Signed-off-by Shameer Kolothum <[email protected]>
Re: [RFC PATCH 14/16] virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev
In reply to Shameer KolothumQuoted 16 lines · Shameer Kolothum
Rename struct AcpiIortSMMUv3Dev to AcpiSMMUv3Dev so that it is not specific to IORT. Subsequent Tegra241 CMDQV support patch will use the same struct to build CMDQV DSDT support as well.
No functional changes intended.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/virt-acpi-build.c | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-)hw/arm/virt-acpi-build.c +0
-0
hw/arm/virt-acpi-build.c static int iort_idmap_compare(gconstpointer a, gconstpointer b) · line 345
Quoted 8 lines
hw/arm/virt-acpi-build.c +1
-1@@ -339,7 +339,7 @@ static int iort_idmap_compare(gconstpointer a, gconstpointer b)return idmap_a->input_base - idmap_b->input_base; } -typedef struct AcpiIortSMMUv3Dev { +typedef struct AcpiSMMUv3Dev { int irq; hwaddr base; GArray *rc_smmu_idmaps;
is the concept of idmap also used in DSDT?
hw/arm/virt-acpi-build.c typedef struct AcpiIortSMMUv3Dev { · line 347
Quoted 2 lines
hw/arm/virt-acpi-build.c +0
-0@@ -347,16 +347,16 @@ typedef struct AcpiIortSMMUv3Dev {size_t offset;
also above offset there is
/* Offset of the SMMUv3 IORT Node relative to the start of the IORT */
Eric
Quoted 130 lines
hw/arm/virt-acpi-build.c +17
-17bool accel;bool ats; -} AcpiIortSMMUv3Dev; +} AcpiSMMUv3Dev;/* -* Populate the struct AcpiIortSMMUv3Dev for the legacy SMMUv3 and +* Populate the struct AcpiSMMUv3Dev for the legacy SMMUv3 and * return the total number of associated idmaps. */ static int populate_smmuv3_legacy_dev (GArray *sdev_blob) {VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); -AcpiIortSMMUv3Dev sdev; +AcpiSMMUv3Dev sdev; sdev.rc_smmu_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping)); object_child_foreach_recursive(object_get_root(), iort_host_bridges,@@ -376,8 +376,8 @@ static int populate_smmuv3_legacy_dev(GArray *sdev_blob)static int smmuv3_dev_idmap_compare (gconstpointer a,gconstpointer b) { -AcpiIortSMMUv3Dev *sdev_a = (AcpiIortSMMUv3Dev *)a; -AcpiIortSMMUv3Dev *sdev_b = (AcpiIortSMMUv3Dev *)b; +AcpiSMMUv3Dev *sdev_a = (AcpiSMMUv3Dev *)a; +AcpiSMMUv3Dev *sdev_b = (AcpiSMMUv3Dev *)b;AcpiIortIdMapping *map_a = &g_array_index (sdev_a->rc_smmu_idmaps, AcpiIortIdMapping,0 ); AcpiIortIdMapping *map_b = &g_array_index(sdev_b->rc_smmu_idmaps,@@ -391,7 +391,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque)GArray *sdev_blob = opaque;AcpiIortIdMapping idmap;PlatformBusDevice *pbus; -AcpiIortSMMUv3Dev sdev; +AcpiSMMUv3Dev sdev;int min_bus, max_bus;SysBusDevice *sbdev;PCIBus *bus;@@ -421,7 +421,7 @@ static int iort_smmuv3_devices(Object *obj, void *opaque) }/* -* Populate the struct AcpiIortSMMUv3Dev for all SMMUv3 devices and +* Populate the struct AcpiSMMUv3Dev for all SMMUv3 devices and * return the total number of idmaps. */ static int populate_smmuv3_dev (GArray *sdev_blob)@@ -442,10 +442,10 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs) {AcpiIortIdMapping *idmap;AcpiIortIdMapping next_range = {0 }; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;for (int i =0 ; i < smmuv3_devs->len; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Based on the RID ranges that are directed to the SMMU, determine the * bypassed RID ranges, i.e., the ones that are directed to the ITS @@ -479,7 +479,7 @@ static void create_rc_its_idmaps(GArray *its_idmaps, GArray *smmuv3_devs)static void build_iort_rmr_nodes (GArray *table_data,GArray *smmuv3_devices,uint32_t *id) { -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;AcpiIortIdMapping *idmap;int i;@@ -487,7 +487,7 @@ build_iort_rmr_nodes(GArray *table_data, GArray *smmuv3_devices, uint32_t *id)uint16_t rmr_len;int bdf; - sdev = &g_array_index (smmuv3_devices, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devices, AcpiSMMUv3Dev, i);if (!sdev->accel) {continue ; }@@ -544,13 +544,13 @@ static void build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) {int i, nb_nodes, rc_mapping_count; -AcpiIortSMMUv3Dev *sdev; +AcpiSMMUv3Dev *sdev;size_t node_size;bool ats_needed = false;int num_smmus =0 ;uint32_t id =0 ;int rc_smmu_idmaps_len =0 ; -GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiIortSMMUv3Dev)); +GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiSMMUv3Dev));GArray *rc_its_idmaps =g_array_new (false, true,sizeof (AcpiIortIdMapping));AcpiTable table = { .sig ="IORT" , .rev =5 , .oem_id = vms->oem_id,@@ -581,7 +581,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }/* Calculate RMR nodes required. One per SMMUv3 with accelerated mode */ for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);if (sdev->ats) { ats_needed = true; }@@ -620,7 +620,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) }for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);int smmu_mapping_count, offset_to_id_array;int irq = sdev->irq;@@ -699,7 +699,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)AcpiIortIdMapping *range;for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);/* * Map RIDs (input) from RC to SMMUv3 nodes: RC -> SMMUv3. @@ -742,7 +742,7 @@ build_iort(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_table_end (linker, &table);g_array_free (rc_its_idmaps, true);for (i =0 ; i < num_smmus; i++) { - sdev = &g_array_index (smmuv3_devs, AcpiIortSMMUv3Dev, i); + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i);g_array_free (sdev->rc_smmu_idmaps, true); }g_array_free (smmuv3_devs, true);
RFC PATCH 15/16 hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
In reply to Shameer KolothumFrom: Nicolin Chen <[email protected]>
Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv=on.
--- hw/arm/trace-events | 1 + hw/arm/virt-acpi-build.c | 74 ++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 77 insertions(+)
hw/arm/trace-events +1 -0
@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s" # virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." +virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d" # smmu-common.c smmu_add_mr(const char *name) "%s"
hw/arm/virt-acpi-build.c +74 -0
86 lines not on this page · raw message
include/hw/arm/virt.h +2 -0
@@ -46,6 +46,8 @@#define NUM_VIRTIO_TRANSPORTS 32#define NUM_SMMU_IRQS 4 +#define TEGRA241_CMDQV_IO_LEN 0x50000 +/* See Linux kernel arch/arm64/include/asm/pvclock-abi.h */ #define PVTIME_SIZE_PER_CPU 64
Signature · 1 line
2.43.0
Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 15 lines · Shameer Kolothum
hw/arm/virt-acpi-build.c +15
-0+static int smmuv3_cmdqv_devices (Object *obj,void *opaque) +{ +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + }
hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1142
Quoted 10 lines
hw/arm/virt-acpi-build.c +10
-0+ + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; + sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ;
This is pre-building SMMU's IORT nodes right? Maybe a different naming? And can be shared with the existing iort_smmuv3_devices?
We do so, because we need to link CMDQV's DSDT node to the SMMU's IORT node but it is created in build_iort() that is called after build_dsdt().
Let's explain why we have this pre-building.
Nicolin
RE: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 36 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 20:11
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTQuoted 20 lines · Shameer Kolothum
hw/arm/virt-acpi-build.c +20
-0+static int smmuv3_cmdqv_devices (Object *obj,void *opaque) +{ +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + } + + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base;
Quoted 8 lines
hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1142
Quoted 5 lines
hw/arm/virt-acpi-build.c +5
-0+ sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ;This is pre-building SMMU's IORT nodes right? Maybe a different naming? And can be shared with the existing
iort_smmuv3_devices?
Not really, if you are referring about this patch here,
https://github.com/NVIDIA/QEMU/commit/cc3b65e6a49a9b7addf44b377d4ef1de99bfee3f
I didn't find a clean way to store the pre-built smmuv3_devs other than placing them in struct AcpiBuildTables. It's not clear we gain much by restructuring things to populate and store them separately. At best, it might slightly improve boot time, and if that becomes important we can always add it as an optimization later.
This patch enumerates SMMUv3 accel instances with CMDQV separately, uses that information to build the DSDT, and then frees the device array.
Quoted 4 lines
We do so, because we need to link CMDQV's DSDT node to the SMMU's IORT node but it is created in
build_iort()that is called afterbuild_dsdt().
Hmm..not sure I get that. Does this mean IORT has a link to DSDT? I can't find one.. Maybe I missed. Please let me know.
Thanks,
Shameer
Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 45 lines · Shameer Kolothum
Quoted 36 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 20:11
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTQuoted 20 lines · Shameer Kolothum
hw/arm/virt-acpi-build.c +20
-0+static int smmuv3_cmdqv_devices (Object *obj,void *opaque) +{ +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + } + + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base;Quoted 8 lines
hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1142
Quoted 5 lines
hw/arm/virt-acpi-build.c +5
-0+ sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ;This is pre-building SMMU's IORT nodes right? Maybe a different naming? And can be shared with the existing
iort_smmuv3_devices?
Quoted 2 lines
Not really, if you are referring about this patch here,
https://github.com/NVIDIA/QEMU/commit/cc3b65e6a49a9b7addf44b377d4ef1de99bfee3f
Yes. I am talking about this. But you are actually pre-building an ACPI SMMU device array here. So, my question wasn't accurate..
Quoted 8 lines
I didn’t find a clean way to store the pre-built
smmuv3_devsother than placing them in struct AcpiBuildTables. It’s not clear we gain much by restructuring things to populate and store them separately. At best, it might slightly improve boot time, and if that becomes important we can always add it as an optimization later.This patch enumerates SMMUv3 accel instances with CMDQV separately, uses that information to build the DSDT, and then frees the device array.
Oh, I missed the g_array_free() in acpi_dsdt_add_tegra241_cmdqv().
Let's have a note inline and copy to the commit message too.
Quoted 6 lines
Quoted 3 lines
We do so, because we need to link CMDQV's DSDT node to the SMMU's IORT node but it is created in
build_iort()that is called afterbuild_dsdt().Hmm..not sure I get that. Does this mean IORT has a link to DSDT? I can't find one.. Maybe I missed. Please let me know.
The _UID field of the DSDT node links to the Identifier field of an SMMU's IORT node. E.g.
[DSDT] // Qemu builds DSDT first
CMDQV0 {
_UID = 0;
}[IORT] // Then builds IORT
SMMU0 {
Identifier = 0;
}If we are sure that the SMMU sequence is fixed every time we rebuild the sdev array so that IORT.Identifier matches with DSDT._UID, we'd be fine. That being said, we should probably make a node inline too.
Otherwise, it would be safer to pre-build the sdev array and use the same array in both DSDT and IORT.
Thanks
Nicolin
Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 80 lines · Nicolin Chen
Quoted 45 lines · Shameer Kolothum
Quoted 36 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 20:11
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTQuoted 20 lines · Shameer Kolothum
hw/arm/virt-acpi-build.c +20
-0+static int smmuv3_cmdqv_devices (Object *obj,void *opaque) +{ +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + } + + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base;Quoted 8 lines
hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1142
Quoted 5 lines
hw/arm/virt-acpi-build.c +5
-0+ sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ;This is pre-building SMMU's IORT nodes right? Maybe a different naming? And can be shared with the existing
iort_smmuv3_devices?Quoted 2 lines
Not really, if you are referring about this patch here,
https://github.com/NVIDIA/QEMU/commit/cc3b65e6a49a9b7addf44b377d4ef1de99bfee3fYes. I am talking about this. But you are actually pre-building an ACPI SMMU device array here. So, my question wasn't accurate..
Quoted 8 lines
I didn’t find a clean way to store the pre-built
smmuv3_devsother than placing them in struct AcpiBuildTables. It’s not clear we gain much by restructuring things to populate and store them separately. At best, it might slightly improve boot time, and if that becomes important we can always add it as an optimization later.This patch enumerates SMMUv3 accel instances with CMDQV separately, uses that information to build the DSDT, and then frees the device array.
Oh, I missed the
g_array_free()inacpi_dsdt_add_tegra241_cmdqv().Let's have a note inline and copy to the commit message too.
Quoted 6 lines
Quoted 3 lines
We do so, because we need to link CMDQV's DSDT node to the SMMU's IORT node but it is created in
build_iort()that is called afterbuild_dsdt().Hmm..not sure I get that. Does this mean IORT has a link to DSDT? I can't find one.. Maybe I missed. Please let me know.
The
_UIDfield of the DSDT node links to the Identifier field of an SMMU's IORT node. E.g.[DSDT] // Qemu builds DSDT first
CMDQV0 { _UID = 0; }
Quoted 8 lines
[IORT] // Then builds IORT
SMMU0 { Identifier = 0; }If we are sure that the SMMU sequence is fixed every time we rebuild the sdev array so that IORT.Identifier matches with DSDT._UID, we'd be fine. That being said, we should probably make a node inline too.
Typo: node->note
Quoted 5 lines
Otherwise, it would be safer to pre-build the sdev array and use the same array in both DSDT and IORT.
Thanks
Nicolin
RE: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 81 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 30 December 2025 18:24
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTQuoted 45 lines · Nicolin Chen
Quoted 44 lines · Shameer Kolothum
Quoted 43 lines
-----Original Message-----
From: Nicolin Chen <[email protected]>
Sent: 29 December 2025 20:11
To: Shameer Kolothum <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTQuoted 14 lines · Shameer Kolothum
hw/arm/virt-acpi-build.c +14
-0+static int smmuv3_cmdqv_devices (Object *obj,void *opaque) { +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + }hw/arm/virt-acpi-build.c static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker, · line 1142
Quoted 10 lines
hw/arm/virt-acpi-build.c +10
-0+ + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; + sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ;This is pre-building SMMU's IORT nodes right? Maybe a different naming? And can be shared with the existing
iort_smmuv3_devices?Quoted 2 lines
Quoted 2 lines
Not really, if you are referring about this patch here,
https://github.com/NVIDIA/QEMU/commit/cc3b65e6a49a9b7addf44b377d
4ef1Quoted 16 lines
Quoted 1 line
de99bfee3fYes. I am talking about this. But you are actually pre-building an ACPI SMMU device array here. So, my question wasn't accurate..
Quoted 8 lines
I didn't find a clean way to store the pre-built
smmuv3_devsother than placing them in struct AcpiBuildTables. It's not clear we gain much by restructuring things to populate and store them separately. At best, it might slightly improve boot time, and if that becomes important we can always add it as an optimization later.This patch enumerates SMMUv3 accel instances with CMDQV separately, uses that information to build the DSDT, and then frees the device array.
Oh, I missed the
g_array_free()inacpi_dsdt_add_tegra241_cmdqv().
Quoted 26 lines
Quoted 7 lines
Let's have a note inline and copy to the commit message too.
Quoted 5 lines
Quoted 3 lines
We do so, because we need to link CMDQV's DSDT node to the SMMU's IORT node but it is created in
build_iort()that is called afterbuild_dsdt().Hmm..not sure I get that. Does this mean IORT has a link to DSDT? I can't
find one..
Quoted 18 lines
Quoted 1 line
Maybe I missed. Please let me know.
The
_UIDfield of the DSDT node links to the Identifier field of an SMMU's IORT node. E.g.[DSDT] // Qemu builds DSDT first
CMDQV0 { _UID = 0; }[IORT] // Then builds IORT
SMMU0 { Identifier = 0; }If we are sure that the SMMU sequence is fixed every time we rebuild the sdev array so that IORT.Identifier matches with DSDT._UID, we'd be fine. That being said, we should probably make a node inline too.
I see that the IORT node Identifier is used in the kernel SMMUv3 driver acpi_smmu_dsdt_probe_tegra241_cmdqv() to retrieve the corresponding ACPI device. This appears to be the relationship you were referring to.
However, I don't think this holds for the current series as is. It will likely fail if we have SMMUv3 accel devices both with and without tegra241-cmdqv enabled, since the "identifier" will not match in that case. I will fix that.
Regarding QEMU keeping the same sequence across both iterations, I couldn't find any guarantee that object_child_foreach_recursive() does that. Need to check.
Thanks,
Shameer
Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
In reply to Shameer KolothumQuoted 4 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv=on.
Pls can you give a point to the DSDT spec to understand why this is needed and what needs to be added.
Quoted 118 lines
Signed-off-by Nicolin Chen <[email protected]>Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/trace-events | 1 + hw/arm/virt-acpi-build.c | 74 ++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 77 insertions(+)hw/arm/trace-events +1
-0@@ -9,6 +9,7 @@ omap1_lpg_led(const char *onoff) "omap1 LPG: LED is %s" # virt-acpi-build.c virt_acpi_setup(void) "No fw cfg or ACPI disabled. Bailing out." +virt_acpi_dsdt_tegra241_cmdqv(int smmu_id, uint64_t base, uint32_t irq) "DSDT: add cmdqv node for (id=%d), base=0x%" PRIx64 ", irq=%d" # smmu-common.c smmu_add_mr(const char *name) "%s"hw/arm/virt-acpi-build.c +74
-0@@ -1115,6 +1115,78 @@ static void build_fadt_rev6(GArray *table_data, BIOSLinker *linker,build_fadt (table_data, linker, &fadt, vms->oem_id, vms->oem_table_id); } +static int smmuv3_cmdqv_devices (Object *obj,void *opaque) +{ +VirtMachineState *vms =VIRT_MACHINE (qdev_get_machine ()); +GArray *sdev_blob = opaque; +PlatformBusDevice *pbus; +AcpiSMMUv3Dev sdev; +SysBusDevice *sbdev; + +if (!object_dynamic_cast (obj,TYPE_ARM_SMMUV3 )) { +return 0 ; + } + +if (!object_property_get_bool (obj,"tegra241-cmdqv" ,NULL )) { +return 0 ; + } + + pbus =PLATFORM_BUS_DEVICE (vms->platform_bus_dev); + sbdev =SYS_BUS_DEVICE (obj); + sdev.base =platform_bus_get_mmio_addr (pbus, sbdev,1 ); + sdev.base += vms->memmap[VIRT_PLATFORM_BUS ].base; + sdev.irq =platform_bus_get_irqn (pbus, sbdev,NUM_SMMU_IRQS ); + sdev.irq += vms->irqmap[VIRT_PLATFORM_BUS ]; + sdev.irq +=ARM_SPI_BASE ; +g_array_append_val (sdev_blob, sdev); +return 0 ; +} + +static void acpi_dsdt_add_tegra241_cmdqv (Aml *scope,VirtMachineState *vms) +{ +GArray *smmuv3_devs =g_array_new (false, true,sizeof (AcpiSMMUv3Dev)); +int i; + +if (vms->legacy_smmuv3_present) { +return ; + } + +object_child_foreach_recursive (object_get_root (), smmuv3_cmdqv_devices, + smmuv3_devs); + +for (i =0 ; i < smmuv3_devs->len; i++) { +uint32_t identifier = i; +AcpiSMMUv3Dev *sdev; +Aml *dev, *crs, *addr; + + sdev = &g_array_index (smmuv3_devs, AcpiSMMUv3Dev, i); + + dev =aml_device ("CV%.02u" , identifier); +aml_append (dev,aml_name_decl ("_HID" ,aml_string ("NVDA200C" ))); +if (vms->its) { + identifier++; + } +aml_append (dev,aml_name_decl ("_UID" ,aml_int (identifier))); +aml_append (dev,aml_name_decl ("_CCA" ,aml_int (1 ))); + + crs =aml_resource_template (); + addr =aml_qword_memory (AML_POS_DECODE ,AML_MIN_FIXED ,AML_MAX_FIXED , +AML_CACHEABLE ,AML_READ_WRITE ,0x0 , sdev->base, + sdev->base +TEGRA241_CMDQV_IO_LEN -0x1 ,0x0 , +TEGRA241_CMDQV_IO_LEN ); +aml_append (crs, addr); +aml_append (crs,aml_interrupt (AML_CONSUMER ,AML_EDGE , +AML_ACTIVE_HIGH ,AML_EXCLUSIVE , + (uint32_t *)&sdev->irq,1 )); +aml_append (dev,aml_name_decl ("_CRS" , crs)); + +aml_append (scope, dev); + +trace_virt_acpi_dsdt_tegra241_cmdqv (identifier, sdev->base, sdev->irq); + } +g_array_free (smmuv3_devs, true); +} +/* DSDT */ static void build_dsdt (GArray *table_data,BIOSLinker *linker,VirtMachineState *vms)@@ -1179,6 +1251,8 @@ build_dsdt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)acpi_dsdt_add_tpm (scope, vms);#endif +acpi_dsdt_add_tegra241_cmdqv (scope, vms); +aml_append (dsdt, scope); pci0_scope =aml_scope ("\\_SB.PCI0" );include/hw/arm/virt.h +0
-0
include/hw/arm/virt.h line 49
Quoted 4 lines
include/hw/arm/virt.h +1
-0@@ -46,6 +46,8 @@#define NUM_VIRTIO_TRANSPORTS 32#define NUM_SMMU_IRQS 4 +#define TEGRA241_CMDQV_IO_LEN 0x50000
isn't it already defined elsewhere, use the corresponding include?
Eric
Quoted 4 lines
include/hw/arm/virt.h +1
-0+/* See Linux kernel arch/arm64/include/asm/pvclock-abi.h */ #define PVTIME_SIZE_PER_CPU 64
Re: [RFC PATCH 15/16] hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDT
Quoted 8 lines · Eric Auger
Quoted 4 lines · Shameer Kolothum
From: Nicolin Chen <[email protected]>
Add ACPI DSDT support for Tegra241 CMDQV when the SMMUv3 instance is created with tegra241-cmdqv=on.
Pls can you give a point to the DSDT spec to understand why this is needed and what needs to be added.
This was an implementation decision by ARM that CMDQV node resides in DSDT v.s. IORT.
Kernel driver scans DSDT unconditionally:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c#n4572
NVIDIA also has a public DSDT:
https://github.com/NVIDIA/edk2-nvidia/blob/main/Silicon/NVIDIA/Drivers/ConfigurationManagerData/AcpiTableList/PlatformASLTablesLib/Dsdt_TH500.asl#L1005
I don't think there is a DSDT spec though...
Nicolin
RFC PATCH 16/16 hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
In reply to Shameer KolothumIntroduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
--- hw/arm/smmuv3.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
hw/arm/smmuv3.c +13 -0
@@ -1953,6 +1953,12 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)error_setg (errp,"accel=on support not compiled in" ); return false; } +#endif +#ifndef CONFIG_TEGRA241_CMDQ +if (s->tegra241_cmdqv) { +error_setg (errp,"tegra241_cmdqv=on support not compiled in" ); +return false; + }#endif if (!s->accel) {if (!s->ril) {
@@ -1971,6 +1977,10 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)error_setg (errp,"pasid can only be enabled if accel=on" );return false; } +if (s->tegra241_cmdqv) { +error_setg (errp,"tegra241_cmdqv can only be enabled if accel=on" ); +return false; + }return true; }
@@ -2109,6 +2119,7 @@ static const Property smmuv3_properties[] = {DEFINE_PROP_BOOL ("ats" , SMMUv3State, ats, false),DEFINE_PROP_UINT8 ("oas" , SMMUv3State, oas,44 ),DEFINE_PROP_BOOL ("pasid" , SMMUv3State, pasid, false), + DEFINE_PROP_BOOL("tegra241-cmdqv", SMMUv3State, tegra241_cmdqv, false), };static void smmuv3_instance_init (Object *obj)
@@ -2144,6 +2155,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data) "are 44 or 48 bits. Defaults to 44 bits");object_class_property_set_description (klass,"pasid" ,"Enable/disable PASID support (for accel=on)" ); +object_class_property_set_description (klass,"tegra241-cmdqv" , +"Enable/disable Tegra241 CMDQ-Virtualisation support (for accel=on)" ); } static int smmuv3_notify_flag_changed(IOMMUMemoryRegion *iommu,
Signature · 1 line
2.43.0
Re: [RFC PATCH 16/16] hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 device
Quoted 13 lines · Shameer Kolothum
Introduce a “tegra241-cmdqv” property to enable Tegra241 CMDQV support. This is only enabled for accelerated SMMUv3 devices.
Signed-off-by Shameer Kolothum <[email protected]>--- hw/arm/smmuv3.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)hw/arm/smmuv3.c +0
-0
hw/arm/smmuv3.c static bool smmu_validate_property(SMMUv3State *s, Error **errp) · line 1962
Quoted 10 lines
hw/arm/smmuv3.c +6
-0@@ -1953,6 +1953,12 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)error_setg (errp,"accel=on support not compiled in" ); return false; } +#endif +#ifndef CONFIG_TEGRA241_CMDQ +if (s->tegra241_cmdqv) { +error_setg (errp,"tegra241_cmdqv=on support not compiled in" ); +return false; + }#endif
Instead of adding this..
hw/arm/smmuv3.c static const Property smmuv3_properties[] = { · line 2122
Quoted 5 lines
hw/arm/smmuv3.c +1
-0@@ -2109,6 +2119,7 @@ static const Property smmuv3_properties[] = {DEFINE_PROP_BOOL ("ats" , SMMUv3State, ats, false),DEFINE_PROP_UINT8 ("oas" , SMMUv3State, oas,44 ),DEFINE_PROP_BOOL ("pasid" , SMMUv3State, pasid, false), + DEFINE_PROP_BOOL("tegra241-cmdqv", SMMUv3State, tegra241_cmdqv, false),
Could this be
+#if CONFIG_TEGRA241_CMDQ
+ DEFINE_PROP_BOOL("tegra241-cmdqv", SMMUv3State, tegra241_cmdqv, false),
+endif
?
hw/arm/smmuv3.c static void smmuv3_class_init(ObjectClass *klass, const void *data) · line 2159
Quoted 6 lines
hw/arm/smmuv3.c +2
-0@@ -2144,6 +2155,8 @@ static void smmuv3_class_init(ObjectClass *klass, const void *data) "are 44 or 48 bits. Defaults to 44 bits");object_class_property_set_description (klass,"pasid" ,"Enable/disable PASID support (for accel=on)" ); +object_class_property_set_description (klass,"tegra241-cmdqv" , +"Enable/disable Tegra241 CMDQ-Virtualisation support (for accel=on)" );
And here too.
The current approach looks okay to me though. So either way:
Hi Shameer,
Quoted 66 lines · Shameer Kolothum
Hi,
This RFC series adds initial support for NVIDIA Tegra241 CMDQV (Command Queue Virtualisation), an extension to ARM SMMUv3 that provides hardware accelerated virtual command queues (VCMDQs) for guests. CMDQV allows guests to issue SMMU invalidation commands directly to hardware without VM exits, significantly reducing TLBI overhead.
Thanks to Nicolin for the initial patches and testing on which this RFC is based.
This is based on v6[0] of the SMMUv3 accel series, which is still under review, though nearing convergence. This is sent as an RFC, with the goal of gathering early feedback on the CMDQV design and its integration with the SMMUv3 acceleration path.
Background:
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
VCMDQ support is integrated via the
IOMMU_HW_QUEUE_ALLOCmechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].Summary of QEMU changes:
- Integrated into the existing SMMUv3 accel path via a "tegra241-cmdqv" property.
- Support for allocating vIOMMU objects of typeIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.
- Mapping and emulation of the CMDQV MMIO register layout.
- VCMDQ/VINTF read/write handling and queue allocation using IOMMUFD APIs.
- Reset and initialisation hooks, including checks for at least one cold-plugged device.
- CMDQV hardware reads guest queue memory using host physical addresses provided through IOMMUFD, which requires that the VCMDQ buffer be physically contiguous not only in guest PA space but also in host PA space. When Tegra241 CMDQV is enabled, QEMU must therefore only expose a CMDQV size that the host can reliably back with contiguous physical memory. Because of this constraint, it is suggested to use huge pages to back the guest RAM.
- ACPI DSDT node generation for CMDQV devices on the virt machine.These patches have been sanity tested on NVIDIA Grace platforms.
ToDo / revisit:
- Prevent hot-unplug of the last device associated with vIOMMU as this might allow associating a different host SMMU/CMDQV.
- Locking requirements around error event propagation.
Quoted 6 lines
Feedback and testing are very welcome.
Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/
do you have a branch to share with all the bits?
Thanks
Eric
Quoted 43 lines
Nicolin Chen (12):
backends/iommufd: Updateiommufd_backend_get_device_infobackends/iommufd: Updateiommufd_backend_alloc_viommuto allow user
ptr
backends/iommufd: Introduceiommufd_backend_alloc_hw_queuebackends/iommufd: Introduceiommufd_backend_viommu_mmaphw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation
support
hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest hw/arm/tegra241-cmdqv: Add read emulation support for registers system/physmem: Add helper to check whether a guest PA maps to RAM hw/arm/tegra241-cmdqv:: Add write emulation for registers hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTShameer Kolothum (4):
hw/arm/tegra241-cmdqv: Allocate vEVENTQ object hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 devicebackends/iommufd.c | 65 ++++ backends/trace-events | 2 + hw/arm/Kconfig | 5 + hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 16 +- hw/arm/smmuv3.c | 18 + hw/arm/tegra241-cmdqv.c | 759 ++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 337 +++++++++++++++++ hw/arm/trace-events | 5 + hw/arm/virt-acpi-build.c | 110 +++++- hw/vfio/iommufd.c | 6 +- include/exec/cpu-common.h | 2 + include/hw/arm/smmuv3.h | 3 + include/hw/arm/virt.h | 2 + include/system/iommufd.h | 16 + system/physmem.c | 12 + 16 files changed, 1332 insertions(+), 27 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.h
Hi Eric,
Quoted 14 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 December 2025 17:55
To: Shameer Kolothum <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 00/16] hw/arm: Introduce Tegra241 CMDQV support for accelerated SMMUv3External email: Use caution opening links or attachments
[...]
Quoted 6 lines
Quoted 2 lines
[0] https://lore.kernel.org/qemu-devel/20251120132213.56581-1-
Quoted 1 line
do you have a branch to share with all the bits?
Here:
https://github.com/shamiali2008/qemu-master.git master-smmuv3-accel-v6-veventq-v2-vcmdq-rfcv1
Thanks,
Shameer
Shameer Kolothum
AppliedQuoted 43 lines
-----Original Message-----
From: Shameer Kolothum <[email protected]>
Sent: 12 December 2025 00:23
To: [email protected]; [email protected]; qemu- [email protected]
Cc: [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: RE: [RFC PATCH 00/16] hw/arm: Introduce Tegra241 CMDQV support for accelerated SMMUv3Hi Eric,
Quoted 4 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 11 December 2025 17:55
To: Shameer Kolothum <[email protected]>; qemu-Quoted 3 lines
[email protected]
Cc: [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>;Jason
Quoted 7 lines
Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 00/16] hw/arm: Introduce Tegra241 CMDQV support for accelerated SMMUv3External email: Use caution opening links or attachments
[...]
Quoted 7 lines
Quoted 2 lines
[0] https://lore.kernel.org/qemu-devel/20251120132213.56581-1-
Quoted 3 lines
Quoted 5 lines
Quoted 1 line
do you have a branch to share with all the bits?
Here:
https://github.com/shamiali2008/qemu-master.git master-smmuv3-accel- v6-veventq-v2-vcmdq-rfcv1
I just realised this needs a fix in patch #16,
hw/arm/smmuv3.c +1 -1
@@ -1954,7 +1954,7 @@ static bool smmu_validate_property(SMMUv3State *s, Error **errp)return false; }#endif -#ifndef CONFIG_TEGRA241_CMDQ +#ifndef CONFIG_TEGRA241_CMDQV if (s->tegra241_cmdqv) {error_setg (errp,"tegra241_cmdqv=on support not compiled in" );return false;
--
Pushed to above branch.
Thanks,
Shameer
Quoted 17 lines · Shameer Kolothum
Hi,
This RFC series adds initial support for NVIDIA Tegra241 CMDQV (Command Queue Virtualisation), an extension to ARM SMMUv3 that provides hardware accelerated virtual command queues (VCMDQs) for guests. CMDQV allows guests to issue SMMU invalidation commands directly to hardware without VM exits, significantly reducing TLBI overhead.
Thanks to Nicolin for the initial patches and testing on which this RFC is based.
This is based on v6[0] of the SMMUv3 accel series, which is still under review, though nearing convergence. This is sent as an RFC, with the goal of gathering early feedback on the CMDQV design and its integration with the SMMUv3 acceleration path.
Quoted 4 lines
Background:
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual interfaces" (VINTFs), each hosting up to 128 VCMDQs.
Can you add a reference to some specification documet please?
Quoted 6 lines
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)
Quoted 6 lines
Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does not support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
Do I understand correctly that this Tegra241 CMDQV aware driveris enabled by the CONFIG_TEGRA241_CMDQV on guest? Is it fully supported upstream?
Eric
Quoted 82 lines
VCMDQ support is integrated via the
IOMMU_HW_QUEUE_ALLOCmechanism, allowing QEMU to attach guest configured VCMDQ buffers to the underlying CMDQV hardware through IOMMUFD. The Linux kernel already supports the full CMDQV virtualisation model via IOMMUFD[0].Summary of QEMU changes:
- Integrated into the existing SMMUv3 accel path via a "tegra241-cmdqv" property.
- Support for allocating vIOMMU objects of typeIOMMU_VIOMMU_TYPE_TEGRA241_CMDQV.
- Mapping and emulation of the CMDQV MMIO register layout.
- VCMDQ/VINTF read/write handling and queue allocation using IOMMUFD APIs.
- Reset and initialisation hooks, including checks for at least one cold-plugged device.
- CMDQV hardware reads guest queue memory using host physical addresses provided through IOMMUFD, which requires that the VCMDQ buffer be physically contiguous not only in guest PA space but also in host PA space. When Tegra241 CMDQV is enabled, QEMU must therefore only expose a CMDQV size that the host can reliably back with contiguous physical memory. Because of this constraint, it is suggested to use huge pages to back the guest RAM.
- ACPI DSDT node generation for CMDQV devices on the virt machine.These patches have been sanity tested on NVIDIA Grace platforms.
ToDo / revisit:
- Prevent hot-unplug of the last device associated with vIOMMU as this might allow associating a different host SMMU/CMDQV.
- Locking requirements around error event propagation.Feedback and testing are very welcome.
Thanks,
Shameer
[0] https://lore.kernel.org/qemu-devel/[email protected]/
[1] https://lore.kernel.org/all/[email protected]/Nicolin Chen (12):
backends/iommufd: Updateiommufd_backend_get_device_infobackends/iommufd: Updateiommufd_backend_alloc_viommuto allow user
ptr
backends/iommufd: Introduceiommufd_backend_alloc_hw_queuebackends/iommufd: Introduceiommufd_backend_viommu_mmaphw/arm/tegra241-cmdqv: Add initial Tegra241 CMDQ-Virtualisation
support
hw/arm/tegra241-cmdqv: Map VINTF Page0 into guest hw/arm/tegra241-cmdqv: Add read emulation support for registers system/physmem: Add helper to check whether a guest PA maps to RAM hw/arm/tegra241-cmdqv:: Add write emulation for registers hw/arm/tegra241-cmdqv: Add reset handler hw/arm/tegra241-cmdqv: Limit queue size based on backend page size hw/arm/virt-acpi: Advertise Tegra241 CMDQV nodes in DSDTShameer Kolothum (4):
hw/arm/tegra241-cmdqv: Allocate vEVENTQ object hw/arm/tegra241-cmdqv: Read and propagate Tegra241 CMDQV errors
virt-acpi-build: Rename AcpiIortSMMUv3Dev to AcpiSMMUv3Dev hw/arm/smmuv3: Add tegra241-cmdqv property for SMMUv3 devicebackends/iommufd.c | 65 ++++ backends/trace-events | 2 + hw/arm/Kconfig | 5 + hw/arm/meson.build | 1 + hw/arm/smmuv3-accel.c | 16 +- hw/arm/smmuv3.c | 18 + hw/arm/tegra241-cmdqv.c | 759 ++++++++++++++++++++++++++++++++++++++ hw/arm/tegra241-cmdqv.h | 337 +++++++++++++++++ hw/arm/trace-events | 5 + hw/arm/virt-acpi-build.c | 110 +++++- hw/vfio/iommufd.c | 6 +- include/exec/cpu-common.h | 2 + include/hw/arm/smmuv3.h | 3 + include/hw/arm/virt.h | 2 + include/system/iommufd.h | 16 + system/physmem.c | 12 + 16 files changed, 1332 insertions(+), 27 deletions(-) create mode 100644 hw/arm/tegra241-cmdqv.c create mode 100644 hw/arm/tegra241-cmdqv.h
Quoted 50 lines
-----Original Message-----
From: Eric Auger <[email protected]>
Sent: 12 January 2026 15:44
To: Shameer Kolothum <[email protected]>; qemu- [email protected]; [email protected]
Cc: [email protected]; Nicolin Chen <[email protected]>; Nathan Chen <[email protected]>; Matt Ochs <[email protected]>; Jason Gunthorpe <[email protected]>; [email protected]; [email protected]; [email protected]; Krishnakant Jaju <[email protected]>
Subject: Re: [RFC PATCH 00/16] hw/arm: Introduce Tegra241 CMDQV support for accelerated SMMUv3External email: Use caution opening links or attachments
Quoted 12 lines · Shameer Kolothum
Hi,
This RFC series adds initial support for NVIDIA Tegra241 CMDQV (Command Queue Virtualisation), an extension to ARM SMMUv3 that provides hardware accelerated virtual command queues (VCMDQs) for guests. CMDQV allows guests to issue SMMU invalidation commands directly to hardware without VM exits, significantly reducing TLBI overhead.
Thanks to Nicolin for the initial patches and testing on which this RFC is based.
Quoted 8 lines
This is based on v6[0] of the SMMUv3 accel series, which is still under review, though nearing convergence. This is sent as an RFC, with the goal of gathering early feedback on the CMDQV design and its integration with the SMMUv3 acceleration path.
Background:
Tegra241 CMDQV extends SMMUv3 by allocating per-VM "virtual
interfaces"
Quoted 1 line
(VINTFs), each hosting up to 128 VCMDQs.
Can you add a reference to some specification documet please?
Quoted 7 lines
Each VINTF exposes two 64KB MMIO pages:
- Page0 – guest owned control and status registers (directly mapped into the VM)
- Page1 – queue configuration registers (trapped/emulated by QEMU)Unlike the standard SMMU CMDQ, a guest owned Tegra241 VCMDQ does
not
Quoted 8 lines
Quoted 5 lines
support the full command set. Only a subset, primarily invalidation related commands, is accepted by the CMDQV hardware. For this reason, a distinct CMDQV device must be exposed to the guest, and the guest OS must include a Tegra241 CMDQV aware driver to take advantage of the hardware acceleration.
Do I understand correctly that this Tegra241 CMDQV aware driveris enabled by the
CONFIG_TEGRA241_CMDQVon guest? Is it fully supported upstream?
Yes. With CONFIG_TEGRA241_CMDQV enabled, it should work.
Thanks,
Shameer