iommufd: Enable noiommu mode for cdev
Subsystems (6)
v5 cover iommufd: Enable noiommu mode for cdev
VFIO's unsafe_noiommu_mode has long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices under vfio_compat mode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.
In summary, we have:
|-------------------------+------+---------------| | Device access mode | VFIO | IOMMUFD | |-------------------------+------+---------------| | group /dev/vfio/$GROUP | Yes | Yes | |-------------------------+------+---------------| | cdev /dev/vfio/devices/ | No | This patch | |-------------------------+------+---------------|
Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort to mlock(), which is less robust than pin_user_pages() used for devices backed by a physical IOMMU. For example, mlock() does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on /proc/pagemap tricks or hardcoded values.
By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].
Furthermore, these devices will use the IOMMUFD-based ownership checking model for VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for an iommufd_access object as required in a previous attempt [3].
ChangeLog:
V5:
- Split CONFIG_VFIO_NOIOMMU into CONFIG_VFIO_GROUP_NOIOMMU and CONFIG_VFIO_CDEV_NOIOMMU so cdev noiommu is independent of VFIO_GROUP (Alex)
- Add CAP_SYS_RAWIO check for cdev open and bind under noiommu, security parity with group noiommu (Alex)
- Add IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) guard in iommufd_device_is_noiommu() to prevent noiommu bind when feature is disabled
- Add prep patch to tolerate NULL group for cdev noiommu devices when CONFIG_VFIO_GROUP_NOIOMMU is not set [7/9]
- Rename IOCTL to IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA to be more specific (Kevin)
- Simplify iommufd_device_is_noiommu, use iommufd_bind_noiommu helper (Kevin, Yi)
- Move IOMMU cap check under iommufd_bind_iommu() (Yi)
- Fix next_iova exceeding iopt_area_last_iova in GET_PA (Alex)
- Fix const hwpt, copyright date, typo in moved comment (Kevin)
- Add Reviewed-by tags
- Squash noiommu cdev selftest fix into selftest patch
- Drop DSA selftest patch
- Details in each patch changelog.
V4:
- Fix various corner cases pointed out by (Sashiko) Details in each patch changelog.
V3:
- Improve error handling [3/10] (Mostafa)
- Simplify vfio_device_is_noiommu logic and merged in [6/10] (Mostafa)
- Add comment to explain the design difference over the legacy noiommu
VFIO code.[1/10]
V2:
- Fix build dependency by adding IOMMU_SUPPORT in [8/11]
- Add an optimization to scan beyond the first page for a contiguous physical address range and return its length instead of a single page.[4/11]
Since RFC[4]:
- Abandoned dummy iommu driver approach as patch 1-3 absorbed the changes into iommufd.
[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-pci/[email protected]/
[3] https://lore.kernel.org/kvm/[email protected]/
[4] https://lore.kernel.org/linux-iommu/[email protected]/
Future cleanup: consolidate all CONFIG_IOMMUFD_NOIOMMU code (iopt_get_phys, iommufd_ioas_noiommu_get_pa, iommufd_noiommu_ops) into hwpt_noiommu.c to eliminate #ifdef guards from ioas.c and io_pagetable.c.
Signature · 1 line
2.43.0
PATCH v5 1/9 vfio: Rename VFIO_NOIOMMU to VFIO_GROUP_NOIOMMU
In preparation for adding cdev-based noiommu support under iommufd, rename CONFIG_VFIO_NOIOMMU to CONFIG_VFIO_GROUP_NOIOMMU to clearly scope it to the legacy group/container path. Also rename the helper vfio_device_is_noiommu() to vfio_device_is_group_noiommu() to match.
Add an explicit dependency on VFIO_CONTAINER or IOMMUFD_VFIO_CONTAINER since the group-based noiommu path is only meaningful when container support is enabled.
This is a pure rename with no functional change, laying the groundwork for a separate VFIO_CDEV_NOIOMMU config that enables noiommu mode through the iommufd cdev interface.
--- drivers/iommu/iommufd/vfio_compat.c | 4 ++-- drivers/vfio/Kconfig | 6 +++--- drivers/vfio/container.c | 6 +++--- drivers/vfio/group.c | 4 ++-- drivers/vfio/iommufd.c | 6 +++--- drivers/vfio/vfio.h | 12 ++++++------ drivers/vfio/vfio_main.c | 4 ++-- 7 files changed, 21 insertions(+), 21 deletions(-)
drivers/iommu/iommufd/vfio_compat.c +2 -2
@@ -286,7 +286,7 @@ static int iommufd_vfio_check_extension(struct iommufd_ctx *ictx,return !ictx->no_iommu_mode;case VFIO_NOIOMMU_IOMMU : -return IS_ENABLED (CONFIG_VFIO_NOIOMMU ); +return IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU );case VFIO_DMA_CC_IOMMU :return iommufd_vfio_cc_iommu (ictx);
@@ -318,7 +318,7 @@ static int iommufd_vfio_set_iommu(struct iommufd_ctx *ictx, unsigned long type)* other ioctls. We let them keep working but they mostly fail since no * IOAS should exist. */ -if (IS_ENABLED (CONFIG_VFIO_NOIOMMU ) && type ==VFIO_NOIOMMU_IOMMU && +if (IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) && type ==VFIO_NOIOMMU_IOMMU && no_iommu_mode) {if (!capable (CAP_SYS_RAWIO ))return -EPERM ;
drivers/vfio/Kconfig +3 -3
@@ -60,9 +60,9 @@ config VFIO_IOMMU_SPAPR_TCE default VFIO endif -config VFIO_NOIOMMU - bool "VFIO No-IOMMU support" - depends on VFIO_GROUP +config VFIO_GROUP_NOIOMMU + bool "VFIO group No-IOMMU support" + depends on VFIO_GROUP && (VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER) help VFIO is built on the ability to isolate devices using the IOMMU. Only with an IOMMU can userspace access to DMA capable devices be
drivers/vfio/container.c +3 -3
@@ -80,7 +80,7 @@ static const struct vfio_iommu_driver_ops vfio_noiommu_ops = {static bool vfio_iommu_driver_allowed (struct vfio_container *container,const struct vfio_iommu_driver *driver) { -if (!IS_ENABLED (CONFIG_VFIO_NOIOMMU )) +if (!IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ))return true;return container->noiommu == (driver->ops == &vfio_noiommu_ops); }
@@ -583,7 +583,7 @@ int __init vfio_container_init(void)return ret; } -if (IS_ENABLED (CONFIG_VFIO_NOIOMMU )) { +if (IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU )) { ret =vfio_register_iommu_driver (&vfio_noiommu_ops);if (ret) goto err_misc;
@@ -597,7 +597,7 @@ int __init vfio_container_init(void)void vfio_container_cleanup (void ) { -if (IS_ENABLED (CONFIG_VFIO_NOIOMMU )) +if (IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ))vfio_unregister_iommu_driver (&vfio_noiommu_ops);misc_deregister (&vfio_dev);mutex_destroy (&vfio.iommu_drivers_lock);
drivers/vfio/group.c +2 -2
@@ -137,7 +137,7 @@ static int vfio_group_ioctl_set_container(struct vfio_group *group, iommufd =iommufd_ctx_from_file (fd_file (f));if (!IS_ERR (iommufd)) { -if (IS_ENABLED (CONFIG_VFIO_NOIOMMU ) && +if (IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) && group->type ==VFIO_NO_IOMMU ) ret =iommufd_vfio_compat_set_no_iommu (iommufd);else
@@ -190,7 +190,7 @@ static int vfio_df_group_open(struct vfio_device_file *df)vfio_device_group_get_kvm_safe (device); df->iommufd = device->group->iommufd; -if (df->iommufd &&vfio_device_is_noiommu (device) && device->open_count ==0 ) { +if (df->iommufd &&vfio_device_is_group_noiommu (device) && device->open_count ==0 ) {/* * Require no compat ioas to be assigned to proceed. The basic * statement is that the user cannot have done something that
drivers/vfio/iommufd.c +3 -3
@@ -26,7 +26,7 @@ int vfio_df_iommufd_bind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock);/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_noiommu (vdev)) +if (vfio_device_is_group_noiommu (vdev))return 0 ;return vdev->ops->bind_iommufd (vdev, ictx, &df->devid);
@@ -41,7 +41,7 @@ int vfio_iommufd_compat_attach_ioas(struct vfio_device *vdev,lockdep_assert_held (&vdev->dev_set->lock);/* compat noiommu does not need to do ioas attach */ -if (vfio_device_is_noiommu (vdev)) +if (vfio_device_is_group_noiommu (vdev))return 0 ; ret =iommufd_vfio_compat_ioas_get_id (ictx, &ioas_id);
@@ -58,7 +58,7 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -if (vfio_device_is_noiommu (vdev)) +if (vfio_device_is_group_noiommu (vdev))return ;if (vdev->ops->unbind_iommufd)
drivers/vfio/vfio.h +6 -6
@@ -36,7 +36,7 @@ vfio_allocate_device_file(struct vfio_device *device);extern const struct file_operations vfio_device_fops; -#ifdef CONFIG_VFIO_NOIOMMU +#ifdef CONFIG_VFIO_GROUP_NOIOMMU extern bool vfio_noiommu__read_mostly ;#else enum { vfio_noiommu = false };
@@ -112,9 +112,9 @@ bool vfio_device_has_container(struct vfio_device *device);int __init vfio_group_init (void );void vfio_group_cleanup (void ); -static inline bool vfio_device_is_noiommu (struct vfio_device *vdev) +static inline bool vfio_device_is_group_noiommu (struct vfio_device *vdev) { -return IS_ENABLED (CONFIG_VFIO_NOIOMMU ) && +return IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) && vdev->group->type ==VFIO_NO_IOMMU ; }#else
@@ -188,7 +188,7 @@ static inline void vfio_group_cleanup(void) { } -static inline bool vfio_device_is_noiommu (struct vfio_device *vdev) +static inline bool vfio_device_is_group_noiommu (struct vfio_device *vdev) {return false; }
@@ -359,7 +359,7 @@ void vfio_init_device_cdev(struct vfio_device *device);static inline int vfio_device_add (struct vfio_device *device) {/* cdev does not support noiommu device */ -if (vfio_device_is_noiommu (device)) +if (vfio_device_is_group_noiommu (device))return device_add (&device->device);vfio_init_device_cdev (device);return cdev_device_add (&device->cdev, &device->device);
@@ -367,7 +367,7 @@ static inline int vfio_device_add(struct vfio_device *device)static inline void vfio_device_del (struct vfio_device *device) { -if (vfio_device_is_noiommu (device)) +if (vfio_device_is_group_noiommu (device))device_del (&device->device);else cdev_device_del (&device->cdev, &device->device);
drivers/vfio/vfio_main.c +2 -2
@@ -54,7 +54,7 @@ static struct vfio {int fs_count; } vfio; -#ifdef CONFIG_VFIO_NOIOMMU +#ifdef CONFIG_VFIO_GROUP_NOIOMMU bool vfio_noiommu__read_mostly ;module_param_named (enable_unsafe_noiommu_mode, vfio_noiommu, bool,S_IRUGO |S_IWUSR );
@@ -353,7 +353,7 @@ static int __vfio_register_dev(struct vfio_device *device,* restore cache coherency. It has to be checked here because it is only * valid for cases where we are using iommu groups. */ -if (type ==VFIO_IOMMU && !vfio_device_is_noiommu (device) && +if (type ==VFIO_IOMMU && !vfio_device_is_group_noiommu (device) && !device_iommu_capable (device->dev,IOMMU_CAP_CACHE_COHERENCY )) { ret = -EINVAL ; goto err_out;
Signature · 1 line
2.43.0
Re: [PATCH v5 1/9] vfio: Rename VFIO_NOIOMMU to VFIO_GROUP_NOIOMMU
Quoted 16 lines · Jacob Pan
In preparation for adding cdev-based noiommu support under iommufd, rename
CONFIG_VFIO_NOIOMMUtoCONFIG_VFIO_GROUP_NOIOMMUto clearly scope it to the legacy group/container path. Also rename the helpervfio_device_is_noiommu()tovfio_device_is_group_noiommu()to match.Add an explicit dependency on
VFIO_CONTAINERorIOMMUFD_VFIO_CONTAINERsince the group-based noiommu path is only meaningful when container support is enabled.This is a pure rename with no functional change, laying the groundwork for a separate
VFIO_CDEV_NOIOMMUconfig that enables noiommu mode through the iommufd cdev interface.Suggested-by Alex Williamson <[email protected]>Signed-off-by Jacob Pan <[email protected]>
Quoted 9 lines
--- drivers/iommu/iommufd/vfio_compat.c | 4 ++-- drivers/vfio/Kconfig | 6 +++--- drivers/vfio/container.c | 6 +++--- drivers/vfio/group.c | 4 ++-- drivers/vfio/iommufd.c | 6 +++--- drivers/vfio/vfio.h | 12 ++++++------ drivers/vfio/vfio_main.c | 4 ++-- 7 files changed, 21 insertions(+), 21 deletions(-)
Jason
PATCH v5 2/9 iommufd: Support a HWPT without an iommu driver for noiommu
In reply to Jacob PanFrom: Jason Gunthorpe <[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the dev_iommu_ops() and allow iommufd to call domain_alloc_paging_flags() and fail everything else.
This allows explicitly creating a HWPT under an IOAS.
A new Kconfig option IOMMUFD_NOIOMMU is introduced to differentiate from the VFIO group/container based noiommu mode.
---
v5:
- Use the new IOMMUFD_NOIOMMU Kconfig instead of VFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Make iommufd_noiommu_ops const
v3:
- Add comment to explain the design difference over the legacy noiommu VFIO code.
--- drivers/iommu/iommufd/Kconfig | 13 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 15 +++- drivers/iommu/iommufd/hwpt_noiommu.c | 102 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 2 + 5 files changed, 131 insertions(+), 2 deletions(-) create mode 100644 drivers/iommu/iommufd/hwpt_noiommu.c
drivers/iommu/iommufd/Kconfig +13 -0
@@ -16,6 +16,19 @@ config IOMMUFD If you don't know what to do here, say N. if IOMMUFD +config IOMMUFD_NOIOMMU + bool + depends on !GENERIC_ATOMIC64 # IOMMU_PT_AMDV1 requires cmpxchg64 + select GENERIC_PT + select IOMMU_PT + select IOMMU_PT_AMDV1 + help + Provides a SW-only IO page table for devices without hardware + IOMMU backing. This uses the AMDV1 page table format for + IOVA-to-PA lookups only, not for hardware DMA translation. + + Selected by VFIO_CDEV_NOIOMMU. Not intended to be enabled directly. + config IOMMUFD_VFIO_CONTAINER bool "IOMMUFD provides the VFIO container /dev/vfio/vfio" depends on VFIO_GROUP && !VFIO_CONTAINER
drivers/iommu/iommufd/Makefile +1 -0
@@ -10,6 +10,7 @@ iommufd-y := \ vfio_compat.o \ viommu.o +iommufd-$(CONFIG_IOMMUFD_NOIOMMU) += hwpt_noiommu.o iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o obj-$(CONFIG_IOMMUFD) += iommufd.o
drivers/iommu/iommufd/hw_pagetable.c +13 -2
@@ -8,6 +8,15 @@#include "../iommu-priv.h" #include "iommufd_private.h" +static const struct iommu_ops *get_iommu_ops (struct iommufd_device *idev) +{ +if (IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->igroup->group) +return &iommufd_noiommu_ops; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return dev_iommu_ops (idev->dev); +} +static void __iommufd_hwpt_destroy (struct iommufd_hw_pagetable *hwpt) {if (hwpt->domain)
@@ -114,11 +123,13 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas,IOMMU_HWPT_ALLOC_DIRTY_TRACKING |IOMMU_HWPT_FAULT_ID_VALID |IOMMU_HWPT_ALLOC_PASID ; -const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_hw_pagetable *hwpt;int rc; +if (!ops) +return ERR_PTR (-ENODEV );lockdep_assert_held (&ioas->mutex);if ((flags || user_data) && !ops->domain_alloc_paging_flags)
@@ -229,7 +240,7 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, struct iommufd_device *idev, u32 flags, const struct iommu_user_data *user_data) { - const struct iommu_ops *ops = dev_iommu_ops(idev->dev); + const struct iommu_ops *ops = get_iommu_ops(idev);struct iommufd_hwpt_nested *hwpt_nested;struct iommufd_hw_pagetable *hwpt;int rc;
drivers/iommu/iommufd/hwpt_noiommu.c new file +102 -0
102 lines not on this page · raw message
drivers/iommu/iommufd/iommufd_private.h +2 -0
@@ -464,6 +464,8 @@ static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx,refcount_dec (&hwpt->obj.users); } +extern const struct iommu_ops iommufd_noiommu_ops; +struct iommufd_attach ;struct iommufd_group {
Signature · 1 line
2.43.0
Re: [PATCH v5 2/9] iommufd: Support a HWPT without an iommu driver for noiommu
Quoted 90 lines · Jacob Pan
From: Jason Gunthorpe<[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the
dev_iommu_ops()and allow iommufd to calldomain_alloc_paging_flags()and fail everything else.This allows explicitly creating a HWPT under an IOAS.
A new
KconfigoptionIOMMUFD_NOIOMMUis introduced to differentiate from the VFIO group/container based noiommu mode.Signed-off-by Jason Gunthorpe<[email protected]>Signed-off-by Jacob Pan<[email protected]>---
v5:
- Use the newIOMMUFD_NOIOMMUKconfiginstead ofVFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Makeiommufd_noiommu_opsconst
v3:
- Add comment to explain the design difference over the legacy noiommu VFIO code.--- drivers/iommu/iommufd/Kconfig | 13 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 15 +++- drivers/iommu/iommufd/hwpt_noiommu.c | 102 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 2 + 5 files changed, 131 insertions(+), 2 deletions(-)create mode 100644
drivers/iommu/iommufd/hwpt_noiommu.cdrivers/iommu/iommufd/Kconfig +13
-0@@ -16,6 +16,19 @@ config IOMMUFD If you don't know what to do here, say N. if IOMMUFD +config IOMMUFD_NOIOMMU + bool + depends on !GENERIC_ATOMIC64 # IOMMU_PT_AMDV1 requires cmpxchg64 + select GENERIC_PT + select IOMMU_PT + select IOMMU_PT_AMDV1 + help + Provides a SW-only IO page table for devices without hardware + IOMMU backing. This uses the AMDV1 page table format for + IOVA-to-PA lookups only, not for hardware DMA translation. + + Selected by VFIO_CDEV_NOIOMMU. Not intended to be enabled directly. + config IOMMUFD_VFIO_CONTAINER bool "IOMMUFD provides the VFIO container /dev/vfio/vfio" depends on VFIO_GROUP && !VFIO_CONTAINERdrivers/iommu/iommufd/Makefile +1
-0@@ -10,6 +10,7 @@ iommufd-y := \ vfio_compat.o \ viommu.o +iommufd-$(CONFIG_IOMMUFD_NOIOMMU) += hwpt_noiommu.o iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o obj-$(CONFIG_IOMMUFD) += iommufd.odrivers/iommu/iommufd/hw_pagetable.c +9
-0@@ -8,6 +8,15 @@#include "../iommu-priv.h" #include "iommufd_private.h" +static const struct iommu_ops *get_iommu_ops (struct iommufd_device *idev) +{ +if (IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->igroup->group) +return &iommufd_noiommu_ops; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return dev_iommu_ops (idev->dev); +} +static void __iommufd_hwpt_destroy (struct iommufd_hw_pagetable *hwpt) {if (hwpt->domain)@@ -114,11 +123,13 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, IOMMU_HWPT_ALLOC_DIRTY_TRACKING |
drivers/iommu/iommufd/hw_pagetable.c iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, · line 132
Quoted 10 lines
drivers/iommu/iommufd/hw_pagetable.c +3
-1IOMMU_HWPT_FAULT_ID_VALID |IOMMU_HWPT_ALLOC_PASID ; -const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_hw_pagetable *hwpt;int rc; +if (!ops) +return ERR_PTR (-ENODEV );
Nit: it's unnecessary to add this check. get_iommu_ops() will WARN if ops is NULL. The previous code assumes that ops is always valid and does not return here. If you want to change this behavior, it’s better to explain the reasoning in the commit message.
Otherwise it looks good to me.
Thanks,
baolu
Re: [PATCH v5 2/9] iommufd: Support a HWPT without an iommu driver for noiommu
Hi Baolu,
Quoted 102 lines · On Wed, 13 May 2026 14:58:40 +0800 Baolu Lu <[email protected]> wrote:
Quoted 101 lines · Jacob Pan
From: Jason Gunthorpe<[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the
dev_iommu_ops()and allow iommufd to calldomain_alloc_paging_flags()and fail everything else.This allows explicitly creating a HWPT under an IOAS.
A new
KconfigoptionIOMMUFD_NOIOMMUis introduced to differentiate from the VFIO group/container based noiommu mode.Signed-off-by Jason Gunthorpe<[email protected]>Signed-off-by Jacob Pan<[email protected]>---
v5:
- Use the newIOMMUFD_NOIOMMUKconfiginstead ofVFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Makeiommufd_noiommu_opsconst
v3:
- Add comment to explain the design difference over the legacy noiommu VFIO code.--- drivers/iommu/iommufd/Kconfig | 13 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 15 +++- drivers/iommu/iommufd/hwpt_noiommu.c | 102++++++++++++++++++++++++
drivers/iommu/iommufd/iommufd_private.h|
2 + 5 files changed, 131 insertions(+), 2 deletions(-)
create mode 100644drivers/iommu/iommufd/hwpt_noiommu.c(quoted diff) +0
-0b/drivers/iommu/iommufd/Kconfig index
455bac0351f2..74d6ea5b5b3b100644 --- a/drivers/iommu/iommufd/Kconfig
+++ b/drivers/iommu/iommufd/Kconfigdrivers/iommu/iommufd/Kconfig +3
-0@@ -16,6 +16,19 @@ config IOMMUFD If you don't know what to do here, say N. if IOMMUFD +config IOMMUFD_NOIOMMU + bool + depends on !GENERIC_ATOMIC64 # IOMMU_PT_AMDV1 requirescmpxchg64
drivers/iommu/iommufd/Kconfig +5
-0+ select GENERIC_PT + select IOMMU_PT + select IOMMU_PT_AMDV1 + help + Provides a SW-only IO page table for devices withouthardware
(quoted diff) +2
-0+ IOMMU backing. This uses the AMDV1 page table format for + IOVA-to-PA lookups only, not for hardware DMAtranslation. +
(quoted diff) +1
-0+ Selected by VFIO_CDEV_NOIOMMU. Not intended to beenabled directly. +
configIOMMUFD_VFIO_CONTAINER
bool "IOMMUFD provides the VFIO container/dev/vfio/vfio" depends onVFIO_GROUP&& !VFIO_CONTAINER(quoted diff) +0
-0b/drivers/iommu/iommufd/Makefile index
71d692c9a8f4..67207914bb6e100644 --- a/drivers/iommu/iommufd/Makefile
+++ b/drivers/iommu/iommufd/Makefiledrivers/iommu/iommufd/Makefile +1
-0@@ -10,6 +10,7 @@ iommufd-y := \ vfio_compat.o \ viommu.o +iommufd-$(CONFIG_IOMMUFD_NOIOMMU) += hwpt_noiommu.o iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o obj-$(CONFIG_IOMMUFD) += iommufd.o(quoted diff) +0
-0
b/drivers/iommu/iommufd/hw_pagetable.cindexfe789c2dc0c9..0ae14cd3fc72100644 ---a/drivers/iommu/iommufd/hw_pagetable.c+++b/drivers/iommu/iommufd/hw_pagetable.c@@ -8,6 +8,15 @@
#include "../iommu-priv.h"
#include "iommufd_private.h"+static const struct
iommu_ops*get_iommu_ops(structiommufd_device*idev) +{(quoted diff) +1
-0+ if (IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) &&!idev->igroup->group)
drivers/iommu/iommufd/hw_pagetable.c +6
-0+return &iommufd_noiommu_ops; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return dev_iommu_ops (idev->dev); +} + static void __iommufd_hwpt_destroy(struct iommufd_hw_pagetable*hwpt) {
if (hwpt->domain)drivers/iommu/iommufd/hw_pagetable.c +1
-1@@ -114,11 +123,13 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, IOMMU_HWPT_ALLOC_DIRTY_TRACKING |IOMMU_HWPT_FAULT_ID_VALID |IOMMU_HWPT_ALLOC_PASID ; -const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_hw_pagetable *hwpt;int rc;
Quoted 8 lines
drivers/iommu/iommufd/hw_pagetable.c iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, · line 132
Quoted 2 lines
drivers/iommu/iommufd/hw_pagetable.c +2
-0+if (!ops) +return ERR_PTR (-ENODEV );Nit: it's unnecessary to add this check.
get_iommu_ops()will WARN if ops is NULL. The previous code assumes that ops is always valid and does not return here. If you want to change this behavior, it’s better to explain the reasoning in the commit message.
I am fine with removing it, just extra defensive to prevent null pointer deref.
Quoted 6 lines
Otherwise it looks good to me.
Reviewed-by Lu Baolu <[email protected]>Thanks,
baolu
Thanks,
Jacob
Re: [PATCH v5 2/9] iommufd: Support a HWPT without an iommu driver for noiommu
In reply to Jacob PanQuoted 24 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the
dev_iommu_ops()and allow iommufd to calldomain_alloc_paging_flags()and fail everything else.This allows explicitly creating a HWPT under an IOAS.
A new
KconfigoptionIOMMUFD_NOIOMMUis introduced to differentiate from the VFIO group/container based noiommu mode.Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- Use the newIOMMUFD_NOIOMMUKconfiginstead ofVFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Makeiommufd_noiommu_opsconst
v3:
- Add comment to explain the design difference over the legacy noiommu VFIO code.
---
Quoted 8 lines
drivers/iommu/iommufd/Kconfig | 13 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 15 +++- drivers/iommu/iommufd/hwpt_noiommu.c | 102 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 2 + 5 files changed, 131 insertions(+), 2 deletions(-)create mode 100644
drivers/iommu/iommufd/hwpt_noiommu.c
Re: [PATCH v5 2/9] iommufd: Support a HWPT without an iommu driver for noiommu
In reply to Jacob PanQuoted 118 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the
dev_iommu_ops()and allow iommufd to calldomain_alloc_paging_flags()and fail everything else.This allows explicitly creating a HWPT under an IOAS.
A new
KconfigoptionIOMMUFD_NOIOMMUis introduced to differentiate from the VFIO group/container based noiommu mode.Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- Use the newIOMMUFD_NOIOMMUKconfiginstead ofVFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Makeiommufd_noiommu_opsconst
v3:
- Add comment to explain the design difference over the legacy noiommu VFIO code.--- drivers/iommu/iommufd/Kconfig | 13 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 15 +++- drivers/iommu/iommufd/hwpt_noiommu.c | 102 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 2 + 5 files changed, 131 insertions(+), 2 deletions(-)create mode 100644
drivers/iommu/iommufd/hwpt_noiommu.cdrivers/iommu/iommufd/Kconfig +13
-0@@ -16,6 +16,19 @@ config IOMMUFD If you don't know what to do here, say N. if IOMMUFD +config IOMMUFD_NOIOMMU + bool + depends on !GENERIC_ATOMIC64 # IOMMU_PT_AMDV1 requires cmpxchg64 + select GENERIC_PT + select IOMMU_PT + select IOMMU_PT_AMDV1 + help + Provides a SW-only IO page table for devices without hardware + IOMMU backing. This uses the AMDV1 page table format for + IOVA-to-PA lookups only, not for hardware DMA translation. + + Selected by VFIO_CDEV_NOIOMMU. Not intended to be enabled directly. + config IOMMUFD_VFIO_CONTAINER bool "IOMMUFD provides the VFIO container /dev/vfio/vfio" depends on VFIO_GROUP && !VFIO_CONTAINERdrivers/iommu/iommufd/Makefile +1
-0@@ -10,6 +10,7 @@ iommufd-y := \ vfio_compat.o \ viommu.o +iommufd-$(CONFIG_IOMMUFD_NOIOMMU) += hwpt_noiommu.o iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o obj-$(CONFIG_IOMMUFD) += iommufd.odrivers/iommu/iommufd/hw_pagetable.c +13
-2@@ -8,6 +8,15 @@#include "../iommu-priv.h" #include "iommufd_private.h" +static const struct iommu_ops *get_iommu_ops (struct iommufd_device *idev) +{ +if (IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->igroup->group) +return &iommufd_noiommu_ops; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return dev_iommu_ops (idev->dev); +} +static void __iommufd_hwpt_destroy (struct iommufd_hw_pagetable *hwpt) {if (hwpt->domain)@@ -114,11 +123,13 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas,IOMMU_HWPT_ALLOC_DIRTY_TRACKING |IOMMU_HWPT_FAULT_ID_VALID |IOMMU_HWPT_ALLOC_PASID ; -const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_hw_pagetable *hwpt;int rc; +if (!ops) +return ERR_PTR (-ENODEV );lockdep_assert_held (&ioas->mutex);if ((flags || user_data) && !ops->domain_alloc_paging_flags)@@ -229,7 +240,7 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, struct iommufd_device *idev, u32 flags, const struct iommu_user_data *user_data) { - const struct iommu_ops *ops = dev_iommu_ops(idev->dev); + const struct iommu_ops *ops = get_iommu_ops(idev);struct iommufd_hwpt_nested *hwpt_nested;struct iommufd_hw_pagetable *hwpt;int rc;drivers/iommu/iommufd/hwpt_noiommu.c new file +0
-0
drivers/iommu/iommufd/hwpt_noiommu.c line 7
Quoted 7 lines
drivers/iommu/iommufd/hwpt_noiommu.c new file +7
-0@@ -0,0 +1,102 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES +*/ +#include <linux/iommu.h> +#include <linux/generic_pt/iommu.h> +#include "iommufd_private.h"
you missed the comments in the below link except for the kconfig comment. :(
Quoted 108 lines
drivers/iommu/iommufd/hwpt_noiommu.c new file +95
-0+ +static const struct iommu_domain_ops noiommu_amdv1_ops; + +struct noiommu_domain { +union { +struct iommu_domain domain; +struct pt_iommu_amdv1 amdv1; + }; +spinlock_t lock; +}; +PT_IOMMU_CHECK_DOMAIN(struct noiommu_domain, amdv1.iommu, domain); + +static void noiommu_change_top (struct pt_iommu *iommu_table, +phys_addr_t top_paddr,unsigned int top_level) +{ +} + +static spinlock_t *noiommu_get_top_lock (struct pt_iommu *iommupt) +{ +struct noiommu_domain *domain = + container_of(iommupt, struct noiommu_domain, amdv1.iommu); + +return &domain->lock; +} + +static const struct pt_iommu_driver_ops noiommu_driver_ops = { + .get_top_lock = noiommu_get_top_lock, + .change_top = noiommu_change_top, +}; + +static struct iommu_domain * +noiommu_alloc_paging_flags (struct device *dev,u32 flags, +const struct iommu_user_data *user_data) +{ +struct pt_iommu_amdv1_cfg cfg = {}; +struct noiommu_domain *dom; +int rc; + +if (flags || user_data) +return ERR_PTR (-EOPNOTSUPP ); + + cfg.common.hw_max_vasz_lg2 =64 ; + cfg.common.hw_max_oasz_lg2 =52 ; + cfg.starting_level =2 ; + cfg.common.features = + (BIT (PT_FEAT_DYNAMIC_TOP ) |BIT (PT_FEAT_AMDV1_ENCRYPT_TABLES ) | +BIT (PT_FEAT_AMDV1_FORCE_COHERENCE )); + + dom =kzalloc (sizeof (*dom),GFP_KERNEL ); +if (!dom) +return ERR_PTR (-ENOMEM ); + +spin_lock_init (&dom->lock); + dom->amdv1.iommu.nid =NUMA_NO_NODE ; + dom->amdv1.iommu.driver_ops = &noiommu_driver_ops; + dom->domain.ops = &noiommu_amdv1_ops; + +/* Use mock page table which is based on AMDV1 */ + rc =pt_iommu_amdv1_init (&dom->amdv1, &cfg,GFP_KERNEL ); +if (rc) { +kfree (dom); +return ERR_PTR (rc); + } + +return &dom->domain; +} + +static void noiommu_domain_free (struct iommu_domain *iommu_domain) +{ +struct noiommu_domain *domain = + container_of(iommu_domain, struct noiommu_domain, domain); + +pt_iommu_deinit (&domain->amdv1.iommu); +kfree (domain); +} + +/* +* AMDV1 is used as a SW-only page table for no-IOMMU mode, similar to the +* iommufd selftest mock page table. +* Unlike the VFIO group-container based no-IOMMU mode, where no container +* level APIs are supported, this allows IOAS and hwpt objects to exist +* without hardware IOMMU support. IOVAs are used only for IOVA-to-PA +* lookups not for hardware translation in DMA. +* +* This is only used with iommufd and cdev-based interfaces and does not +* apply to the VFIO group-container based noiommu mode. +*/ +static const struct iommu_domain_ops noiommu_amdv1_ops = { +IOMMU_PT_DOMAIN_OPS (amdv1), + .free = noiommu_domain_free, +}; + +const struct iommu_ops iommufd_noiommu_ops = { + .domain_alloc_paging_flags = noiommu_alloc_paging_flags, +};drivers/iommu/iommufd/iommufd_private.h +2
-0@@ -464,6 +464,8 @@ static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx,refcount_dec (&hwpt->obj.users); } +extern const struct iommu_ops iommufd_noiommu_ops; +struct iommufd_attach ;struct iommufd_group {
Re: [PATCH v5 2/9] iommufd: Support a HWPT without an iommu driver for noiommu
Hi Yi,
Quoted 124 lines · On Wed, 20 May 2026 15:19:13 +0800 Yi Liu <[email protected]> wrote:
Quoted 123 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the
dev_iommu_ops()and allow iommufd to calldomain_alloc_paging_flags()and fail everything else.This allows explicitly creating a HWPT under an IOAS.
A new
KconfigoptionIOMMUFD_NOIOMMUis introduced to differentiate from the VFIO group/container based noiommu mode.Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- Use the newIOMMUFD_NOIOMMUKconfiginstead ofVFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Makeiommufd_noiommu_opsconst
v3:
- Add comment to explain the design difference over the legacy noiommu VFIO code.--- drivers/iommu/iommufd/Kconfig | 13 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 15 +++- drivers/iommu/iommufd/hwpt_noiommu.c | 102++++++++++++++++++++++++
drivers/iommu/iommufd/iommufd_private.h|
2 + 5 files changed, 131 insertions(+), 2 deletions(-)
create mode 100644drivers/iommu/iommufd/hwpt_noiommu.c(quoted diff) +0
-0b/drivers/iommu/iommufd/Kconfig index
455bac0351f2..74d6ea5b5b3b100644 --- a/drivers/iommu/iommufd/Kconfig
+++ b/drivers/iommu/iommufd/Kconfigdrivers/iommu/iommufd/Kconfig +3
-0@@ -16,6 +16,19 @@ config IOMMUFD If you don't know what to do here, say N. if IOMMUFD +config IOMMUFD_NOIOMMU + bool + depends on !GENERIC_ATOMIC64 # IOMMU_PT_AMDV1 requirescmpxchg64
drivers/iommu/iommufd/Kconfig +5
-0+ select GENERIC_PT + select IOMMU_PT + select IOMMU_PT_AMDV1 + help + Provides a SW-only IO page table for devices withouthardware
(quoted diff) +2
-0+ IOMMU backing. This uses the AMDV1 page table format for + IOVA-to-PA lookups only, not for hardware DMAtranslation. +
(quoted diff) +1
-0+ Selected by VFIO_CDEV_NOIOMMU. Not intended to beenabled directly. +
configIOMMUFD_VFIO_CONTAINER
bool "IOMMUFD provides the VFIO container/dev/vfio/vfio" depends onVFIO_GROUP&& !VFIO_CONTAINER(quoted diff) +0
-0b/drivers/iommu/iommufd/Makefile index
71d692c9a8f4..67207914bb6e100644 --- a/drivers/iommu/iommufd/Makefile
+++ b/drivers/iommu/iommufd/Makefiledrivers/iommu/iommufd/Makefile +1
-0@@ -10,6 +10,7 @@ iommufd-y := \ vfio_compat.o \ viommu.o +iommufd-$(CONFIG_IOMMUFD_NOIOMMU) += hwpt_noiommu.o iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o obj-$(CONFIG_IOMMUFD) += iommufd.o(quoted diff) +0
-0
b/drivers/iommu/iommufd/hw_pagetable.cindexfe789c2dc0c9..0ae14cd3fc72100644 ---a/drivers/iommu/iommufd/hw_pagetable.c+++b/drivers/iommu/iommufd/hw_pagetable.c@@ -8,6 +8,15 @@
#include "../iommu-priv.h"
#include "iommufd_private.h"+static const struct
iommu_ops*get_iommu_ops(structiommufd_device*idev) +{(quoted diff) +1
-0+ if (IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) &&!idev->igroup->group)
drivers/iommu/iommufd/hw_pagetable.c +6
-0+return &iommufd_noiommu_ops; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return dev_iommu_ops (idev->dev); +} + static void __iommufd_hwpt_destroy(struct iommufd_hw_pagetable*hwpt) {
if (hwpt->domain)drivers/iommu/iommufd/hw_pagetable.c +3
-1@@ -114,11 +123,13 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, IOMMU_HWPT_ALLOC_DIRTY_TRACKING |IOMMU_HWPT_FAULT_ID_VALID |IOMMU_HWPT_ALLOC_PASID ; -const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_hw_pagetable *hwpt;int rc; +if (!ops) +return ERR_PTR (-ENODEV );lockdep_assert_held (&ioas->mutex); if ((flags || user_data) &&!ops->domain_alloc_paging_flags) @@ -229,7 +240,7 @@ iommufd_hwpt_nested_alloc(struct
iommufd_ctx*ictx, structiommufd_device*idev, u32 flags, const structiommu_user_data*user_data) {drivers/iommu/iommufd/hw_pagetable.c +1
-1-const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_nested *hwpt_nested;struct iommufd_hw_pagetable *hwpt;int rc;(quoted diff) +0
-0
b/drivers/iommu/iommufd/hwpt_noiommu.cnew file mode 100644 index 000000000000..b1efc4bca880drivers/iommu/iommufd/hwpt_noiommu.c new file +2
-0@@ -0,0 +1,102 @@ +// SPDX-License-Identifier: GPL-2.0-only +/*
Quoted 9 lines
drivers/iommu/iommufd/hwpt_noiommu.c line 7
Quoted 5 lines
drivers/iommu/iommufd/hwpt_noiommu.c new file +5
-0+* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES +*/ +#include <linux/iommu.h> +#include <linux/generic_pt/iommu.h> +#include "iommufd_private.h" you missed the comments in the below link except for the kconfig comment. :(
indeed, sorry about that. will make the following change in v6:
+++ b/drivers/iommu/iommufd/hwpt_noiommu.c
@@ -2,8 +2,8 @@
/*
* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES */
-#include <linux/iommu.h>
#include <linux/generic_pt/iommu.h>
+#include <linux/iommu.h>
#include "iommufd_private.h"
static const struct iommu_domain_ops noiommu_amdv1_ops;
@@ -62,7 +62,7 @@ noiommu_alloc_paging_flags(struct device *dev, u32 flags,
dom->amdv1.iommu.driver_ops = &noiommu_driver_ops;
dom->domain.ops = &noiommu_amdv1_ops;- /* Use mock page table which is based on AMDV1 */
+ /* Use SW-only page table which is based on AMDV1 */
rc = pt_iommu_amdv1_init(&dom->amdv1, &cfg, GFP_KERNEL);
if (rc) {
kfree(dom);@@ -82,15 +82,10 @@ static void noiommu_domain_free(struct iommu_domain *iommu_domain)
}
/*
- * AMDV1 is used as a SW-only page table for no-IOMMU mode, similar to the
- * iommufd selftest mock page table.
- * Unlike the VFIO group-container based no-IOMMU mode, where no container
- * level APIs are supported, this allows IOAS and hwpt objects to exist
- * without hardware IOMMU support. IOVAs are used only for IOVA-to-PA
- * lookups not for hardware translation in DMA.
- *
- * This is only used with iommufd and cdev-based interfaces and does not
- * apply to the VFIO group-container based noiommu mode.
+ * Domain ops for iommufd no-IOMMU mode. Uses AMDV1 format as a
+ * SW-only IOPT because it has the best multi-page size options
+ * of all the formats. IOVAs serve only for IOVA-to-PA lookups,
+ * not for hardware DMA translation.
*/
Quoted 117 lines
https://lore.kernel.org/linux-iommu/[email protected]/#t
Quoted 114 lines
drivers/iommu/iommufd/hwpt_noiommu.c new file +14
-0+ +static const struct iommu_domain_ops noiommu_amdv1_ops; + +struct noiommu_domain { +union { +struct iommu_domain domain; +struct pt_iommu_amdv1 amdv1; + }; +spinlock_t lock; +}; +PT_IOMMU_CHECK_DOMAIN(struct noiommu_domain, amdv1.iommu, domain); + +static void noiommu_change_top (struct pt_iommu *iommu_table, + phys_addr_t top_paddr, unsigned int
top_level) +{drivers/iommu/iommufd/hwpt_noiommu.c new file +6
-0+} + +static spinlock_t *noiommu_get_top_lock (struct pt_iommu *iommupt) +{ +struct noiommu_domain *domain = + container_of(iommupt, struct noiommu_domain,amdv1.iommu); +
drivers/iommu/iommufd/hwpt_noiommu.c new file +24
-0+return &domain->lock; +} + +static const struct pt_iommu_driver_ops noiommu_driver_ops = { + .get_top_lock = noiommu_get_top_lock, + .change_top = noiommu_change_top, +}; + +static struct iommu_domain * +noiommu_alloc_paging_flags (struct device *dev,u32 flags, +const struct iommu_user_data *user_data) +{ +struct pt_iommu_amdv1_cfg cfg = {}; +struct noiommu_domain *dom; +int rc; + +if (flags || user_data) +return ERR_PTR (-EOPNOTSUPP ); + + cfg.common.hw_max_vasz_lg2 =64 ; + cfg.common.hw_max_oasz_lg2 =52 ; + cfg.starting_level =2 ; + cfg.common.features = + (BIT(PT_FEAT_DYNAMIC_TOP) |
BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES)|drivers/iommu/iommufd/hwpt_noiommu.c new file +25
-0+ BIT(PT_FEAT_AMDV1_FORCE_COHERENCE)); + + dom =kzalloc (sizeof (*dom),GFP_KERNEL ); +if (!dom) +return ERR_PTR (-ENOMEM ); + +spin_lock_init (&dom->lock); + dom->amdv1.iommu.nid =NUMA_NO_NODE ; + dom->amdv1.iommu.driver_ops = &noiommu_driver_ops; + dom->domain.ops = &noiommu_amdv1_ops; + +/* Use mock page table which is based on AMDV1 */ + rc =pt_iommu_amdv1_init (&dom->amdv1, &cfg,GFP_KERNEL ); +if (rc) { +kfree (dom); +return ERR_PTR (rc); + } + +return &dom->domain; +} + +static void noiommu_domain_free (struct iommu_domain *iommu_domain) +{ +struct noiommu_domain *domain = + container_of(iommu_domain, struct noiommu_domain,domain); +
drivers/iommu/iommufd/hwpt_noiommu.c new file +6
-0+pt_iommu_deinit (&domain->amdv1.iommu); +kfree (domain); +} + +/* +* AMDV1 is used as a SW-only page table for no-IOMMU mode, similar to the
+ * iommufd selftest mock page table.
+ * Unlike the VFIO group-container based no-IOMMU mode, where no container
+ * level APIs are supported, this allows IOAS and hwpt objects to exist
+ * without hardware IOMMU support. IOVAs are used only for IOVA-to-PA
+ * lookups not for hardware translation in DMA.
+ *
+ * This is only used with iommufd and cdev-based interfaces and does not
+ * apply to the VFIO group-container based noiommu mode.
+ */drivers/iommu/iommufd/hwpt_noiommu.c new file +8
-0+static const struct iommu_domain_ops noiommu_amdv1_ops = { +IOMMU_PT_DOMAIN_OPS (amdv1), + .free = noiommu_domain_free, +}; + +const struct iommu_ops iommufd_noiommu_ops = { + .domain_alloc_paging_flags = noiommu_alloc_paging_flags, +};(quoted diff) +0
-0
b/drivers/iommu/iommufd/iommufd_private.hindex6ac1965199e9..2682b5baa6e9100644 ---a/drivers/iommu/iommufd/iommufd_private.h+++b/drivers/iommu/iommufd/iommufd_private.h@@ -464,6 +464,8 @@ static inline void iommufd_hw_pagetable_put(structiommufd_ctx*ictx,refcount_dec(&hwpt->obj.users); }drivers/iommu/iommufd/iommufd_private.h +2
-0+extern const struct iommu_ops iommufd_noiommu_ops; +struct iommufd_attach ;struct iommufd_group {
PATCH v5 3/9 iommufd: Move igroup allocation to a function
In reply to Jacob PanFrom: Jason Gunthorpe <[email protected]>
So it can be reused in the next patch which allows binding to noiommu device.
---
v5:
- Add NULL group to the error handling path of iommufd_group_setup_msi()
v3:
- New patch
--- drivers/iommu/iommufd/device.c | 43 +++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 16 deletions(-)
drivers/iommu/iommufd/device.c +27 -16
@@ -56,6 +56,30 @@ static bool iommufd_group_try_get(struct iommufd_group *igroup,return kref_get_unless_zero (&igroup->ref); } +static struct iommufd_group *iommufd_alloc_group (struct iommufd_ctx *ictx, +struct iommu_group *group) +{ +struct iommufd_group *new_igroup; + + new_igroup =kzalloc (sizeof (*new_igroup),GFP_KERNEL ); +if (!new_igroup) +return ERR_PTR (-ENOMEM ); + +kref_init (&new_igroup->ref); +mutex_init (&new_igroup->lock); +xa_init (&new_igroup->pasid_attach); + new_igroup->sw_msi_start =PHYS_ADDR_MAX ; +/* group reference moves into new_igroup */ + new_igroup->group = group; + +/* +* The ictx is not additionally refcounted here because all objects using +* an igroup must put it before their destroy completes. +*/ + new_igroup->ictx = ictx; +return new_igroup; +} +/* * iommufd needs to store some more data for each iommu_group, we keep a * parallel xarray indexed by iommu_group id to hold this instead of putting it
@@ -87,25 +111,12 @@ static struct iommufd_group *iommufd_get_group(struct iommufd_ctx *ictx, }xa_unlock (&ictx->groups); - new_igroup =kzalloc_obj (*new_igroup); -if (!new_igroup) { + new_igroup =iommufd_alloc_group (ictx, group); +if (IS_ERR (new_igroup)) {iommu_group_put (group); -return ERR_PTR (-ENOMEM ); +return new_igroup; } -kref_init (&new_igroup->ref); -mutex_init (&new_igroup->lock); -xa_init (&new_igroup->pasid_attach); - new_igroup->sw_msi_start =PHYS_ADDR_MAX ; -/* group reference moves into new_igroup */ - new_igroup->group = group; - -/* -* The ictx is not additionally refcounted here becase all objects using -* an igroup must put it before their destroy completes. -*/ - new_igroup->ictx = ictx; -/* * We dropped the lock so igroup is invalid. NULL is a safe and likely * value to assume for the xa_cmpxchg algorithm.
Signature · 1 line
2.43.0
Re: [PATCH v5 3/9] iommufd: Move igroup allocation to a function
Quoted 10 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
So it can be reused in the next patch which allows binding to noiommu device.
Reviewed-by Samiullah Khawaja <[email protected]>Reviewed-by Yi Liu <[email protected]>Reviewed-by Kevin Tian <[email protected]>Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>
Quoted 4 lines
---
v5:
- Add NULL group to the error handling path ofiommufd_group_setup_msi()
This logic is not visible in the patch's diff. Perhaps anything I overlooked?
Quoted 11 lines
v3:
- New patch--- drivers/iommu/iommufd/device.c | 43 +++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 16 deletions(-)drivers/iommu/iommufd/device.c +0
-0
drivers/iommu/iommufd/device.c static bool iommufd_group_try_get(struct iommufd_group *igroup, · line 64
Quoted 9 lines
drivers/iommu/iommufd/device.c +6
-0@@ -56,6 +56,30 @@ static bool iommufd_group_try_get(struct iommufd_group *igroup,return kref_get_unless_zero (&igroup->ref); } +static struct iommufd_group *iommufd_alloc_group (struct iommufd_ctx *ictx, +struct iommu_group *group) +{ +struct iommufd_group *new_igroup; + + new_igroup =kzalloc (sizeof (*new_igroup),GFP_KERNEL );
kzalloc_obj() is replaced with kzalloc(). Can you please explain the reason?
Quoted 40 lines
drivers/iommu/iommufd/device.c +21
-9+if (!new_igroup) +return ERR_PTR (-ENOMEM ); + +kref_init (&new_igroup->ref); +mutex_init (&new_igroup->lock); +xa_init (&new_igroup->pasid_attach); + new_igroup->sw_msi_start =PHYS_ADDR_MAX ; +/* group reference moves into new_igroup */ + new_igroup->group = group; + +/* +* The ictx is not additionally refcounted here because all objects using +* an igroup must put it before their destroy completes. +*/ + new_igroup->ictx = ictx; +return new_igroup; +} +/* * iommufd needs to store some more data for each iommu_group, we keep a * parallel xarray indexed by iommu_group id to hold this instead of putting it @@ -87,25 +111,12 @@ static struct iommufd_group *iommufd_get_group(struct iommufd_ctx *ictx, }xa_unlock (&ictx->groups); - new_igroup =kzalloc_obj (*new_igroup); -if (!new_igroup) { + new_igroup =iommufd_alloc_group (ictx, group); +if (IS_ERR (new_igroup)) {iommu_group_put (group); -return ERR_PTR (-ENOMEM ); +return new_igroup; } -kref_init (&new_igroup->ref); -mutex_init (&new_igroup->lock); -xa_init (&new_igroup->pasid_attach); - new_igroup->sw_msi_start =PHYS_ADDR_MAX ; -/* group reference moves into new_igroup */ - new_igroup->group = group;
drivers/iommu/iommufd/device.c static struct iommufd_group *iommufd_get_group(struct iommufd_ctx *ictx, · line 122
Quoted 10 lines
drivers/iommu/iommufd/device.c +0
-7- -/* -* The ictx is not additionally refcounted here becase all objects using -* an igroup must put it before their destroy completes. -*/ - new_igroup->ictx = ictx; -/* * We dropped the lock so igroup is invalid. NULL is a safe and likely * value to assume for the xa_cmpxchg algorithm.
Otherwise this patch looks good to me,
PATCH v5 4/9 iommufd: Allow binding to a noiommu device
In reply to Jacob PanFrom: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy IOMMU group for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
---
v5:
- simplify logic and rename iommufd_device_is_noiommu (Kevin, Yi)
- use a helper iommufd_bind_noiommu instead of open coding (Kevin)
- move IOMMU cap check under iommufd_bind_iommu() (Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches
--- drivers/iommu/iommufd/device.c | 148 ++++++++++++++++++++++++--------- 1 file changed, 109 insertions(+), 39 deletions(-)
drivers/iommu/iommufd/device.c +109 -39
@@ -23,6 +23,16 @@ struct iommufd_attach {struct xarray device_array; }; +/* +* A noiommu device has no IOMMU driver attached regardless of whether it +* enters via the cdev path (no iommu_group) or the group path (fake +* noiommu iommu_group). In both cases dev->iommu is NULL. +*/ +static bool iommufd_device_is_noiommu (struct iommufd_device *idev) +{ +return IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->dev->iommu; +} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =
@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); +if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree (igroup); }
@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct iommufd_object *obj)struct iommufd_device *idev = container_of(obj, struct iommufd_device, obj); -iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ; +if (!iommufd_device_is_noiommu (idev)) +iommu_device_release_dma_owner (idev->dev);iommufd_put_group (idev->igroup);if (!iommufd_selftest_is_mock_dev (idev->dev))iommufd_ctx_put (idev->ictx); } -/** -* iommufd_device_bind - Bind a physical device to an iommu fd -* @ictx: iommufd file descriptor -* @dev: Pointer to a physical device struct -* @id: Output ID number to return to userspace for this device -* -* A successful bind establishes an ownership over the device and returns -* struct iommufd_device pointer, otherwise returns error pointer. -* -* A driver using this API must set driver_managed_dma and must not touch -* the device until this routine succeeds and establishes ownership. -* -* Binding a PCI device places the entire RID under iommufd control. -* -* The caller must undo this with iommufd_device_unbind() -*/ -struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, -struct device *dev,u32 *id) +static int iommufd_bind_iommu (struct iommufd_device *idev) { -struct iommufd_device *idev; +struct iommufd_ctx *ictx = idev->ictx; +struct device *dev = idev->dev;struct iommufd_group *igroup;int rc;
@@ -238,11 +237,11 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,* to restore cache coherency. */ if (!device_iommu_capable (dev,IOMMU_CAP_CACHE_COHERENCY )) -return ERR_PTR (-EINVAL ); +return -EINVAL ; igroup =iommufd_get_group (ictx, dev);if (IS_ERR (igroup)) -return ERR_CAST (igroup); +return PTR_ERR (igroup);/* * For historical compat with VFIO the insecure interrupt path is
@@ -268,21 +267,80 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,if (rc) goto out_group_put; +/* igroup refcount moves into iommufd_device */ + idev->igroup = igroup; + idev->enforce_cache_coherency = +device_iommu_capable (dev,IOMMU_CAP_ENFORCE_CACHE_COHERENCY ); +return 0 ; + +out_group_put: +iommufd_put_group (igroup); +return rc; +} + +/* +* Noiommu devices have no real IOMMU group. Create a dummy igroup so that +* internal code paths that expect idev->igroup to be present still work. +* A NULL igroup->group distinguishes this from a real IOMMU-backed group. +*/ +static int iommufd_bind_noiommu (struct iommufd_device *idev) +{ +struct iommufd_group *igroup; + + igroup =iommufd_alloc_group (idev->ictx,NULL ); +if (IS_ERR (igroup)) +return PTR_ERR (igroup); + idev->igroup = igroup; +return 0 ; +} + +/** +* iommufd_device_bind - Bind a physical device to an iommu fd +* @ictx: iommufd file descriptor +* @dev: Pointer to a physical device struct +* @id: Output ID number to return to userspace for this device +* +* A successful bind establishes an ownership over the device and returns +* struct iommufd_device pointer, otherwise returns error pointer. +* +* A driver using this API must set driver_managed_dma and must not touch +* the device until this routine succeeds and establishes ownership. +* +* Binding a PCI device places the entire RID under iommufd control. +* +* The caller must undo this with iommufd_device_unbind() +*/ +struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, +struct device *dev,u32 *id) +{ +struct iommufd_device *idev; +int rc; + idev =iommufd_object_alloc (ictx, idev,IOMMUFD_OBJ_DEVICE ); -if (IS_ERR (idev)) { - rc =PTR_ERR (idev); - goto out_release_owner; - } +if (IS_ERR (idev)) +return idev; + idev->ictx = ictx; + idev->dev = dev; + +if (!iommufd_device_is_noiommu (idev)) { + rc =iommufd_bind_iommu (idev); +if (rc) + goto err_out; + }else { + rc =iommufd_bind_noiommu (idev); +if (rc) + goto err_out; + } + +/* +* Take a ctx reference after bind succeeds. This must happen here +* so that iommufd_device_destroy() can handle partial initialization +*/ if (!iommufd_selftest_is_mock_dev (dev))iommufd_ctx_get (ictx); - idev->dev = dev; - idev->enforce_cache_coherency = -device_iommu_capable (dev,IOMMU_CAP_ENFORCE_CACHE_COHERENCY );/* The calling driver is a user until iommufd_device_unbind() */ refcount_inc (&idev->obj.users); -/* igroup refcount moves into iommufd_device */ - idev->igroup = igroup;/* * If the caller fails after this success it must call
@@ -294,11 +352,14 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, *id = idev->obj.id;return idev; -out_release_owner: -iommu_device_release_dma_owner (dev); -out_group_put: -iommufd_put_group (igroup); +err_out: +/* +* iommufd_device_destroy() handles partially initialized idev, +* so iommufd_object_abort_and_destroy() is safe to call here. +*/ +iommufd_object_abort_and_destroy (ictx, &idev->obj);return ERR_PTR (rc); + }EXPORT_SYMBOL_NS_GPL (iommufd_device_bind,"IOMMUFD" );
@@ -512,6 +573,9 @@ static int iommufd_hwpt_attach_device(struct iommufd_hw_pagetable *hwpt,struct iommufd_attach_handle *handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;
@@ -559,6 +623,9 @@ static void iommufd_hwpt_detach_device(struct iommufd_hw_pagetable *hwpt, {struct iommufd_attach_handle *handle; +if (iommufd_device_is_noiommu (idev)) +return ; + handle =iommufd_device_get_attach_handle (idev, pasid);if (pasid ==IOMMU_NO_PASID )iommu_detach_group_handle (hwpt->domain, idev->igroup->group);
@@ -577,6 +644,9 @@ static int iommufd_hwpt_replace_device(struct iommufd_device *idev,struct iommufd_attach_handle *handle, *old_handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;
@@ -652,7 +722,7 @@ int iommufd_hw_pagetable_attach(struct iommufd_hw_pagetable *hwpt, goto err_release_devid; } -if (attach_resv) { +if (attach_resv && !iommufd_device_is_noiommu (idev)) { rc =iommufd_device_attach_reserved_iova (idev, hwpt_paging);if (rc) goto err_release_devid;
Signature · 1 line
2.43.0
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
Quoted 31 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy IOMMU group for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- simplify logic and renameiommufd_device_is_noiommu(Kevin, Yi)
- use a helperiommufd_bind_noiommuinstead of open coding (Kevin)
- move IOMMU cap check underiommufd_bind_iommu()(Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches--- drivers/iommu/iommufd/device.c | 148 ++++++++++++++++++++++++--------- 1 file changed, 109 insertions(+), 39 deletions(-)drivers/iommu/iommufd/device.c +0
-0@@ -23,6 +23,16 @@ struct iommufd_attach {struct xarray device_array;
drivers/iommu/iommufd/device.c struct iommufd_attach { · line 33
Quoted 10 lines
drivers/iommu/iommufd/device.c +8
-0}; +/* +* A noiommu device has no IOMMU driver attached regardless of whether it +* enters via the cdev path (no iommu_group) or the group path (fake +* noiommu iommu_group). In both cases dev->iommu is NULL. +*/ +static bool iommufd_device_is_noiommu (struct iommufd_device *idev) +{ +return IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->dev->iommu;
How about using device_iommu_mapped()? As I understand it, this is the standard way to determine whether a device is protected by an IOMMU.
Quoted 21 lines
drivers/iommu/iommufd/device.c +7
-3+} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); +if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree (igroup); }
drivers/iommu/iommufd/device.c void iommufd_device_destroy(struct iommufd_object *obj) · line 220
Quoted 6 lines
drivers/iommu/iommufd/device.c +2
-1@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct iommufd_object *obj)struct iommufd_device *idev = container_of(obj, struct iommufd_device, obj); -iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ;
I don't quite follow this logic. Is this check being added specifically for the new noiommu mode? Since the noiommu mode introduces the convention that "igroup->group == NULL" implies noiommu, there should be no cases where idev->igroup itself is NULL.
Quoted 23 lines
drivers/iommu/iommufd/device.c +2
-16+if (!iommufd_device_is_noiommu (idev)) +iommu_device_release_dma_owner (idev->dev);iommufd_put_group (idev->igroup);if (!iommufd_selftest_is_mock_dev (idev->dev))iommufd_ctx_put (idev->ictx); } -/** -* iommufd_device_bind - Bind a physical device to an iommu fd -* @ictx: iommufd file descriptor -* @dev: Pointer to a physical device struct -* @id: Output ID number to return to userspace for this device -* -* A successful bind establishes an ownership over the device and returns -* struct iommufd_device pointer, otherwise returns error pointer. -* -* A driver using this API must set driver_managed_dma and must not touch -* the device until this routine succeeds and establishes ownership. -* -* Binding a PCI device places the entire RID under iommufd control. -* -* The caller must undo this with iommufd_device_unbind() -*/
drivers/iommu/iommufd/device.c void iommufd_device_destroy(struct iommufd_object *obj) · line 234
Quoted 10 lines
drivers/iommu/iommufd/device.c +3
-3-struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, -struct device *dev,u32 *id) +static int iommufd_bind_iommu (struct iommufd_device *idev) { -struct iommufd_device *idev; +struct iommufd_ctx *ictx = idev->ictx; +struct device *dev = idev->dev;struct iommufd_group *igroup;int rc;
Thanks,
baolu
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
Hi Baolu,
Quoted 38 lines · On Wed, 13 May 2026 15:37:20 +0800 Baolu Lu <[email protected]> wrote:
Quoted 37 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy IOMMU group for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- simplify logic and renameiommufd_device_is_noiommu(Kevin,
Yi)
- use a helperiommufd_bind_noiommuinstead of open coding
(Kevin)
- move IOMMU cap check underiommufd_bind_iommu()(Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches--- drivers/iommu/iommufd/device.c | 148++++++++++++++++++++++++--------- 1 file changed, 109 insertions(+), 39 deletions(-)
(quoted diff) +0
-0
b/drivers/iommu/iommufd/device.cindexd03076fcf3c2..4d75720432cc100644 ---a/drivers/iommu/iommufd/device.c
+++b/drivers/iommu/iommufd/device.cdrivers/iommu/iommufd/device.c +0
-0@@ -23,6 +23,16 @@ struct iommufd_attach {struct xarray device_array; };
Quoted 15 lines
Quoted 11 lines
(quoted diff) +2
-0+/* + * A noiommu device has no IOMMU driver attached regardless ofwhether it
+ * enters via the cdev path (noiommu_group) or the group path (fake
+ * noiommuiommu_group). In both casesdev->iommuis NULL.
+ */drivers/iommu/iommufd/device.c +3
-0+static bool iommufd_device_is_noiommu (struct iommufd_device *idev) +{ + return IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) &&!idev->dev->iommu;
How about using
device_iommu_mapped()? As I understand it, this is the standard way to determine whether a device is protected by an IOMMU.
device_iommu_mapped() — checks dev->iommu_group != NULL
For noiommu via the cdev path, there's no iommu_group at all, so both would return the same thing. But for noiommu via the group path (fake noiommu group), the device does have an iommu_group (the fake one), but dev->iommu is still NULL.
I know today we only call this check in the cdev path but seems more robust not to rely on group.
Quoted 26 lines
Quoted 26 lines
drivers/iommu/iommufd/device.c +7
-3+} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); + if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree(igroup); }(quoted diff) +0
-0@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct
iommufd_object*obj) structiommufd_device*idev =
container_of(obj, struct iommufd_device, obj);
Quoted 9 lines
drivers/iommu/iommufd/device.c void iommufd_device_destroy(struct iommufd_object *obj) · line 220
Quoted 3 lines
drivers/iommu/iommufd/device.c +2
-1-iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ;I don't quite follow this logic. Is this check being added specifically for the new noiommu mode? Since the noiommu mode introduces the convention that "
igroup->group== NULL" implies noiommu, there should be no cases whereidev->igroupitself is NULL.
idev->igroup can be null on error path only, not limited to noiommu. It is the partially initialized idev case.
/*
* iommufd_device_destroy() handles partially initialized idev,
* so iommufd_object_abort_and_destroy() is safe to call here. */
How about add this comment?
/* igroup is NULL when destroy called during bind error cleanup */
Quoted 41 lines
Quoted 33 lines
drivers/iommu/iommufd/device.c +2
-7+if (!iommufd_device_is_noiommu (idev)) +iommu_device_release_dma_owner (idev->dev);iommufd_put_group (idev->igroup);if (!iommufd_selftest_is_mock_dev (idev->dev))iommufd_ctx_put (idev->ictx); } -/** -* iommufd_device_bind - Bind a physical device to an iommu fd -* @ictx: iommufd file descriptor -* @dev: Pointer to a physical device struct -* @id: Output ID number to return to userspace for this device -* -* A successful bind establishes an ownership over the device and returns
- * structiommufd_devicepointer, otherwise returns error pointer.
- *
- * A driver using this API must setdriver_managed_dmaand must not touch
- * the device until this routine succeeds and establishes ownership.
- *
- * Binding a PCI device places the entire RID under iommufd control.
- *
- * The caller must undo this withiommufd_device_unbind()
- */
-structiommufd_device*iommufd_device_bind(structiommufd_ctx*ictx,(quoted diff) +0
-1- struct device *dev, u32*id) +static int
iommufd_bind_iommu(struct iommufd_device *idev)
{drivers/iommu/iommufd/device.c void iommufd_device_destroy(struct iommufd_object *obj) · line 233
Quoted 5 lines
drivers/iommu/iommufd/device.c +2
-1-struct iommufd_device *idev; +struct iommufd_ctx *ictx = idev->ictx; +struct device *dev = idev->dev;struct iommufd_group *igroup;int rc;Thanks,
baolu
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
Quoted 41 lines · Jacob Pan
Quoted 26 lines
Quoted 26 lines
drivers/iommu/iommufd/device.c +7
-3+} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); + if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree(igroup); }(quoted diff) +0
-0@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct
iommufd_object*obj) structiommufd_device*idev =
container_of(obj, struct iommufd_device, obj);Quoted 8 lines
drivers/iommu/iommufd/device.c void iommufd_device_destroy(struct iommufd_object *obj) · line 220
Quoted 3 lines
drivers/iommu/iommufd/device.c +2
-1-iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ;I don't quite follow this logic. Is this check being added specifically for the new noiommu mode? Since the noiommu mode introduces the convention that "
igroup->group== NULL" implies noiommu, there should be no cases whereidev->igroupitself is NULL.
idev->igroupcan be null on error path only, not limited to noiommu. It is the partially initialized idev case.
/*
*iommufd_device_destroy()handles partially initialized idev,
* soiommufd_object_abort_and_destroy()is safe to call here. */
Quoted 2 lines
How about add this comment?
/* igroup is NULL when destroy called during bind error cleanup */
If it's not limited to noiommu, why not making it in a separated patch?
Thanks,
baolu
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
Hi Baolu,
Quoted 42 lines · On Thu, 14 May 2026 14:51:01 +0800 Baolu Lu <[email protected]> wrote:
Quoted 41 lines · Jacob Pan
Quoted 26 lines
Quoted 26 lines
drivers/iommu/iommufd/device.c +7
-3+} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); + if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree(igroup); }(quoted diff) +0
-0@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct
iommufd_object*obj) structiommufd_device*idev =
container_of(obj, struct iommufd_device, obj);Quoted 8 lines
drivers/iommu/iommufd/device.c void iommufd_device_destroy(struct iommufd_object *obj) · line 220
Quoted 3 lines
drivers/iommu/iommufd/device.c +2
-1-iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ;I don't quite follow this logic. Is this check being added specifically for the new noiommu mode? Since the noiommu mode introduces the convention that "
igroup->group== NULL" implies noiommu, there should be no cases whereidev->igroupitself is NULL.
idev->igroupcan be null on error path only, not limited to noiommu. It is the partially initialized idev case.
/*
*iommufd_device_destroy()handles partially initialized idev,
* soiommufd_object_abort_and_destroy()is safe to call here. */
Quoted 7 lines
Quoted 3 lines
How about add this comment?
/* igroup is NULL when destroy called during bind error cleanup
*/If it's not limited to noiommu, why not making it in a separated patch?
Because the error handling is restructured by this patch to use iommufd_object_abort_and_destroy() on any failure, which calls iommufd_device_destroy(). Since idev->igroup is now set inside iommufd_bind_iommu()/iommufd_bind_noiommu(), a failure before those succeed (or within them before the assignment) leaves idev->igroup == NULL. Hence the new null check applies to both iommu and noiommu but only needed because of this patch.
Before this patch, iommufd_device_destroy was only reachable after bind fully succeeded. i.e. no partial init idev.
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
In reply to Jacob PanQuoted 30 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy IOMMU group for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- simplify logic and renameiommufd_device_is_noiommu(Kevin, Yi)
- use a helperiommufd_bind_noiommuinstead of open coding (Kevin)
- move IOMMU cap check underiommufd_bind_iommu()(Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches--- drivers/iommu/iommufd/device.c | 148 ++++++++++++++++++++++++--------- 1 file changed, 109 insertions(+), 39 deletions(-)drivers/iommu/iommufd/device.c +0
-0
drivers/iommu/iommufd/device.c struct iommufd_attach { · line 29
Quoted 7 lines
drivers/iommu/iommufd/device.c +4
-0@@ -23,6 +23,16 @@ struct iommufd_attach {struct xarray device_array; }; +/* +* A noiommu device has no IOMMU driver attached regardless of whether it +* enters via the cdev path (no iommu_group) or the group path (fake +* noiommu iommu_group). In both cases dev->iommu is NULL.
Are you trying to identify the case where both group and cdev interfaces coexist? In such a scenario, the group path creates a fake iommu group while the cdev path creates a dummy igroup. Both operations need to be performed, but checking iommu_group might interfere with the cdev noiommu path logic.
Is that why you're checking idev->dev->iommu here instead? If so, could you please make this rationale more explicit in the comment?
At first glance, the comment is quite confusing. It's unclear why a cdev-path-only helper function needs to reference the group path at all. The last sentence in particular seems to suggest this helper covers both paths, which adds to the confusion.
Quoted 141 lines
drivers/iommu/iommufd/device.c +73
-29+*/ +static bool iommufd_device_is_noiommu (struct iommufd_device *idev) +{ +return IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->dev->iommu; +} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); +if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree (igroup); }@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct iommufd_object *obj)struct iommufd_device *idev = container_of(obj, struct iommufd_device, obj); -iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ; +if (!iommufd_device_is_noiommu (idev)) +iommu_device_release_dma_owner (idev->dev);iommufd_put_group (idev->igroup);if (!iommufd_selftest_is_mock_dev (idev->dev))iommufd_ctx_put (idev->ictx); } -/** -* iommufd_device_bind - Bind a physical device to an iommu fd -* @ictx: iommufd file descriptor -* @dev: Pointer to a physical device struct -* @id: Output ID number to return to userspace for this device -* -* A successful bind establishes an ownership over the device and returns -* struct iommufd_device pointer, otherwise returns error pointer. -* -* A driver using this API must set driver_managed_dma and must not touch -* the device until this routine succeeds and establishes ownership. -* -* Binding a PCI device places the entire RID under iommufd control. -* -* The caller must undo this with iommufd_device_unbind() -*/ -struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, -struct device *dev,u32 *id) +static int iommufd_bind_iommu (struct iommufd_device *idev) { -struct iommufd_device *idev; +struct iommufd_ctx *ictx = idev->ictx; +struct device *dev = idev->dev;struct iommufd_group *igroup;int rc;@@ -238,11 +237,11 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,* to restore cache coherency. */ if (!device_iommu_capable (dev,IOMMU_CAP_CACHE_COHERENCY )) -return ERR_PTR (-EINVAL ); +return -EINVAL ; igroup =iommufd_get_group (ictx, dev);if (IS_ERR (igroup)) -return ERR_CAST (igroup); +return PTR_ERR (igroup);/* * For historical compat with VFIO the insecure interrupt path is @@ -268,21 +267,80 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,if (rc) goto out_group_put; +/* igroup refcount moves into iommufd_device */ + idev->igroup = igroup; + idev->enforce_cache_coherency = +device_iommu_capable (dev,IOMMU_CAP_ENFORCE_CACHE_COHERENCY ); +return 0 ; + +out_group_put: +iommufd_put_group (igroup); +return rc; +} + +/* +* Noiommu devices have no real IOMMU group. Create a dummy igroup so that +* internal code paths that expect idev->igroup to be present still work. +* A NULL igroup->group distinguishes this from a real IOMMU-backed group. +*/ +static int iommufd_bind_noiommu (struct iommufd_device *idev) +{ +struct iommufd_group *igroup; + + igroup =iommufd_alloc_group (idev->ictx,NULL ); +if (IS_ERR (igroup)) +return PTR_ERR (igroup); + idev->igroup = igroup; +return 0 ; +} + +/** +* iommufd_device_bind - Bind a physical device to an iommu fd +* @ictx: iommufd file descriptor +* @dev: Pointer to a physical device struct +* @id: Output ID number to return to userspace for this device +* +* A successful bind establishes an ownership over the device and returns +* struct iommufd_device pointer, otherwise returns error pointer. +* +* A driver using this API must set driver_managed_dma and must not touch +* the device until this routine succeeds and establishes ownership. +* +* Binding a PCI device places the entire RID under iommufd control. +* +* The caller must undo this with iommufd_device_unbind() +*/ +struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, +struct device *dev,u32 *id) +{ +struct iommufd_device *idev; +int rc; + idev =iommufd_object_alloc (ictx, idev,IOMMUFD_OBJ_DEVICE ); -if (IS_ERR (idev)) { - rc =PTR_ERR (idev); - goto out_release_owner; - } +if (IS_ERR (idev)) +return idev; + idev->ictx = ictx; + idev->dev = dev;
drivers/iommu/iommufd/device.c struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, · line 334
Quoted 10 lines
drivers/iommu/iommufd/device.c +10
-0+ +if (!iommufd_device_is_noiommu (idev)) { + rc =iommufd_bind_iommu (idev); +if (rc) + goto err_out; + }else { + rc =iommufd_bind_noiommu (idev); +if (rc) + goto err_out; + }
how about moving the 'if (rc)'out of the two branches?
drivers/iommu/iommufd/device.c struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, · line 339
Quoted 5 lines
drivers/iommu/iommufd/device.c +5
-0+ +/* +* Take a ctx reference after bind succeeds. This must happen here +* so that iommufd_device_destroy() can handle partial initialization +*/
s/after bind succeeds/after bind_iommu/noiommu() succeeds. or more explicit, "idev->igroup is set." :)
Other parts LGTM, feel free to add my r-b tag after solving the above nits.
Quoted 70 lines
drivers/iommu/iommufd/device.c +17
-10if (!iommufd_selftest_is_mock_dev (dev))iommufd_ctx_get (ictx); - idev->dev = dev; - idev->enforce_cache_coherency = -device_iommu_capable (dev,IOMMU_CAP_ENFORCE_CACHE_COHERENCY );/* The calling driver is a user until iommufd_device_unbind() */ refcount_inc (&idev->obj.users); -/* igroup refcount moves into iommufd_device */ - idev->igroup = igroup;/* * If the caller fails after this success it must call @@ -294,11 +352,14 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, *id = idev->obj.id;return idev; -out_release_owner: -iommu_device_release_dma_owner (dev); -out_group_put: -iommufd_put_group (igroup); +err_out: +/* +* iommufd_device_destroy() handles partially initialized idev, +* so iommufd_object_abort_and_destroy() is safe to call here. +*/ +iommufd_object_abort_and_destroy (ictx, &idev->obj);return ERR_PTR (rc); + }EXPORT_SYMBOL_NS_GPL (iommufd_device_bind,"IOMMUFD" );@@ -512,6 +573,9 @@ static int iommufd_hwpt_attach_device(struct iommufd_hw_pagetable *hwpt,struct iommufd_attach_handle *handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;@@ -559,6 +623,9 @@ static void iommufd_hwpt_detach_device(struct iommufd_hw_pagetable *hwpt, {struct iommufd_attach_handle *handle; +if (iommufd_device_is_noiommu (idev)) +return ; + handle =iommufd_device_get_attach_handle (idev, pasid);if (pasid ==IOMMU_NO_PASID )iommu_detach_group_handle (hwpt->domain, idev->igroup->group);@@ -577,6 +644,9 @@ static int iommufd_hwpt_replace_device(struct iommufd_device *idev,struct iommufd_attach_handle *handle, *old_handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;@@ -652,7 +722,7 @@ int iommufd_hw_pagetable_attach(struct iommufd_hw_pagetable *hwpt, goto err_release_devid; } -if (attach_resv) { +if (attach_resv && !iommufd_device_is_noiommu (idev)) { rc =iommufd_device_attach_reserved_iova (idev, hwpt_paging);if (rc) goto err_release_devid;
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
Hi Yi,
Quoted 51 lines · On Wed, 20 May 2026 15:20:06 +0800 Yi Liu <[email protected]> wrote:
Quoted 39 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy IOMMU group for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- simplify logic and renameiommufd_device_is_noiommu(Kevin,
Yi)
- use a helperiommufd_bind_noiommuinstead of open coding
(Kevin)
- move IOMMU cap check underiommufd_bind_iommu()(Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches--- drivers/iommu/iommufd/device.c | 148++++++++++++++++++++++++--------- 1 file changed, 109 insertions(+), 39 deletions(-)
(quoted diff) +0
-0
b/drivers/iommu/iommufd/device.cindexd03076fcf3c2..4d75720432cc100644 ---a/drivers/iommu/iommufd/device.c
+++b/drivers/iommu/iommufd/device.cdrivers/iommu/iommufd/device.c +2
-0@@ -23,6 +23,16 @@ struct iommufd_attach {struct xarray device_array; }; +/* +* A noiommu device has no IOMMU driver attached regardless of Quoted 4 lines
whether it
+ * enters via the cdev path (noiommu_group) or the group path (fake
+ * noiommuiommu_group). In both casesdev->iommuis NULL.Are you trying to identify the case where both group and cdev interfaces coexist? In such a scenario, the group path creates a fake iommu group while the cdev path creates a dummy igroup. Both operations need to be performed, but checking
iommu_groupmight interfere with the cdev noiommu path logic.
Quoted 8 lines
Is that why you're checking
idev->dev->iommuhere instead? If so, could you please make this rationale more explicit in the comment?At first glance, the comment is quite confusing. It's unclear why a cdev-path-only helper function needs to reference the group path at all. The last sentence in particular seems to suggest this helper covers both paths, which adds to the confusion.
Yes, Baolu also commented on this asking why not use device_iommu_mapped. I will update the comment as follows:
/*
- * A noiommu device has no IOMMU driver attached regardless of whether it
- * enters via the cdev path (no iommu_group) or the group path (fake
- * noiommu iommu_group). In both cases dev->iommu is NULL.
+ * Detect a noiommu device for the cdev path. We check dev->iommu rather than
+ * using device_iommu_mapped() (which checks dev->iommu_group) because when
+ * both group and cdev interfaces coexist, the group path assigns a fake
+ * noiommu iommu_group to the device. That would cause device_iommu_mapped()
+ * to return true and hide the noiommu case from the cdev path. dev->iommu is
+ * reliably NULL when no IOMMU driver is managing the device. */
static bool iommufd_device_is_noiommu(struct iommufd_device *idev) {
Quoted 163 lines
Quoted 163 lines
+ */
drivers/iommu/iommufd/device.c +3
-0+static bool iommufd_device_is_noiommu (struct iommufd_device *idev) +{ + return IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) &&!idev->dev->iommu; +}
drivers/iommu/iommufd/device.c +6
-3+static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =@@ -30,9 +40,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); + if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree(igroup); }(quoted diff) +0
-0@@ -204,32 +216,19 @@ void iommufd_device_destroy(struct
iommufd_object*obj) structiommufd_device*idev =
container_of(obj, struct iommufd_device, obj);drivers/iommu/iommufd/device.c +4
-8-iommu_device_release_dma_owner (idev->dev); +if (!idev->igroup) +return ; +if (!iommufd_device_is_noiommu (idev)) +iommu_device_release_dma_owner (idev->dev);iommufd_put_group (idev->igroup);if (!iommufd_selftest_is_mock_dev (idev->dev))iommufd_ctx_put (idev->ictx); } -/** -* iommufd_device_bind - Bind a physical device to an iommu fd -* @ictx: iommufd file descriptor -* @dev: Pointer to a physical device struct -* @id: Output ID number to return to userspace for this device -* -* A successful bind establishes an ownership over the device and returns
- * structiommufd_devicepointer, otherwise returns error pointer.
- *
- * A driver using this API must setdriver_managed_dmaand must not touch
- * the device until this routine succeeds and establishes ownership.
- *
- * Binding a PCI device places the entire RID under iommufd control.
- *
- * The caller must undo this withiommufd_device_unbind()
- */
-structiommufd_device*iommufd_device_bind(structiommufd_ctx*ictx,(quoted diff) +0
-1- struct device *dev, u32*id) +static int
iommufd_bind_iommu(struct iommufd_device *idev)
{drivers/iommu/iommufd/device.c +4
-3-struct iommufd_device *idev; +struct iommufd_ctx *ictx = idev->ictx; +struct device *dev = idev->dev;struct iommufd_group *igroup;int rc;@@ -238,11 +237,11 @@ struct iommufd_device*iommufd_device_bind(struct iommufd_ctx *ictx, * to restore cache coherency. */ if (!device_iommu_capable (dev,IOMMU_CAP_CACHE_COHERENCY )) -return ERR_PTR (-EINVAL ); +return -EINVAL ; igroup =iommufd_get_group (ictx, dev);if (IS_ERR (igroup)) -return ERR_CAST (igroup); +return PTR_ERR (igroup);/* * For historical compat with VFIO the insecure interrupt
path is @@ -268,21 +267,80 @@ structiommufd_device*iommufd_device_bind(structiommufd_ctx*ictx, if (rc)
gotoout_group_put;drivers/iommu/iommufd/device.c +4
-0+/* igroup refcount moves into iommufd_device */ + idev->igroup = igroup; + idev->enforce_cache_coherency = + device_iommu_capable(dev,
IOMMU_CAP_ENFORCE_CACHE_COHERENCY);drivers/iommu/iommufd/device.c +9
-0+return 0 ; + +out_group_put: +iommufd_put_group (igroup); +return rc; +} + +/* +* Noiommu devices have no real IOMMU group. Create a dummy igroup so that
+ * internal code paths that expectidev->igroupto be present still work.
+ * A NULLigroup->groupdistinguishes this from a real IOMMU-backed group.
+ */drivers/iommu/iommufd/device.c +18
-0+static int iommufd_bind_noiommu (struct iommufd_device *idev) +{ +struct iommufd_group *igroup; + + igroup =iommufd_alloc_group (idev->ictx,NULL ); +if (IS_ERR (igroup)) +return PTR_ERR (igroup); + idev->igroup = igroup; +return 0 ; +} + +/** +* iommufd_device_bind - Bind a physical device to an iommu fd +* @ictx: iommufd file descriptor +* @dev: Pointer to a physical device struct +* @id: Output ID number to return to userspace for this device +* +* A successful bind establishes an ownership over the device and returns
+ * structiommufd_devicepointer, otherwise returns error pointer.
+ *
+ * A driver using this API must setdriver_managed_dmaand must not touch
+ * the device until this routine succeeds and establishes ownership.
+ *
+ * Binding a PCI device places the entire RID under iommufd control.
+ *
+ * The caller must undo this withiommufd_device_unbind()
+ */
+structiommufd_device*iommufd_device_bind(structiommufd_ctx*ictx,(quoted diff) +1
-0+ struct device *dev, u32*id) +{
drivers/iommu/iommufd/device.c +3
-0+struct iommufd_device *idev; +int rc; + idev = iommufd_object_alloc(ictx, idev,
IOMMUFD_OBJ_DEVICE);drivers/iommu/iommufd/device.c +8
-4-if (IS_ERR (idev)) { - rc =PTR_ERR (idev); - goto out_release_owner; - } +if (IS_ERR (idev)) +return idev; + idev->ictx = ictx; + idev->dev = dev; + +if (!iommufd_device_is_noiommu (idev)) { + rc =iommufd_bind_iommu (idev); +if (rc)
Quoted 9 lines
drivers/iommu/iommufd/device.c struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, · line 334
Quoted 6 lines
drivers/iommu/iommufd/device.c +6
-0+ goto err_out; + }else { + rc =iommufd_bind_noiommu (idev); +if (rc) + goto err_out; + }how about moving the 'if (rc)'out of the two branches?
Sounds good!
Quoted 11 lines
Review
Quoted 7 lines
drivers/iommu/iommufd/device.c +3
-0+ +/* +* Take a ctx reference after bind succeeds. This must happen here
(quoted diff) +1
-0+ * so that iommufd_device_destroy() can handle partialinitialization
(quoted diff) +1
-0+ */s/after bind succeeds/after bind_iommu/noiommu() succeeds. or more explicit, "
idev->igroupis set." :)
Quoted 4 lines
Other parts LGTM, feel free to add my r-b tag after solving the above nits.
Reviewed-by Yi Liu <[email protected]>
will do.
Thanks a lot!
Quoted 73 lines
Quoted 73 lines
if (!iommufd_selftest_is_mock_dev(dev)) iommufd_ctx_get(ictx);drivers/iommu/iommufd/device.c +0
-3- idev->dev = dev; - idev->enforce_cache_coherency = - device_iommu_capable(dev,
IOMMU_CAP_ENFORCE_CACHE_COHERENCY); /* The calling driver is a user untiliommufd_device_unbind()*/refcount_inc(&idev->obj.users);drivers/iommu/iommufd/device.c +4
-6-/* igroup refcount moves into iommufd_device */ - idev->igroup = igroup;/* * If the caller fails after this success it must call @@ -294,11 +352,14 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, *id = idev->obj.id;return idev; -out_release_owner: -iommu_device_release_dma_owner (dev); -out_group_put: -iommufd_put_group (igroup); +err_out: +/* +* iommufd_device_destroy() handles partially initialized idev, +* so iommufd_object_abort_and_destroy() is safe to call here.
drivers/iommu/iommufd/device.c +9
-0+*/ +iommufd_object_abort_and_destroy (ictx, &idev->obj);return ERR_PTR (rc); + }EXPORT_SYMBOL_NS_GPL (iommufd_device_bind,"IOMMUFD" );@@ -512,6 +573,9 @@ static int iommufd_hwpt_attach_device(struct iommufd_hw_pagetable *hwpt, struct iommufd_attach_handle *handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;@@ -559,6 +623,9 @@ static void iommufd_hwpt_detach_device(struct iommufd_hw_pagetable *hwpt, {struct iommufd_attach_handle *handle; +if (iommufd_device_is_noiommu (idev)) +return ; + handle =iommufd_device_get_attach_handle (idev, pasid);if (pasid ==IOMMU_NO_PASID ) iommu_detach_group_handle(hwpt->domain,
idev->igroup->group); @@ -577,6 +644,9 @@ static int iommufd_hwpt_replace_device(structiommufd_device*idev, structiommufd_attach_handle*handle,*old_handle; int rc;drivers/iommu/iommufd/device.c +4
-1+if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;@@ -652,7 +722,7 @@ int iommufd_hw_pagetable_attach(struct iommufd_hw_pagetable *hwpt, goto err_release_devid; } -if (attach_resv) { +if (attach_resv && !iommufd_device_is_noiommu (idev)) { rc =iommufd_device_attach_reserved_iova (idev, hwpt_paging);if (rc) goto err_release_devid;
Re: [PATCH v5 4/9] iommufd: Allow binding to a noiommu device
Quoted 77 lines · Jacob Pan
Hi Yi,
Quoted 51 lines · On Wed, 20 May 2026 15:20:06 +0800 Yi Liu <[email protected]> wrote:
Quoted 39 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy IOMMU group for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- simplify logic and renameiommufd_device_is_noiommu(Kevin,
Yi)
- use a helperiommufd_bind_noiommuinstead of open coding
(Kevin)
- move IOMMU cap check underiommufd_bind_iommu()(Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches--- drivers/iommu/iommufd/device.c | 148++++++++++++++++++++++++--------- 1 file changed, 109 insertions(+), 39 deletions(-)
(quoted diff) +0
-0
b/drivers/iommu/iommufd/device.cindexd03076fcf3c2..4d75720432cc100644 ---a/drivers/iommu/iommufd/device.c
+++b/drivers/iommu/iommufd/device.cdrivers/iommu/iommufd/device.c +2
-0@@ -23,6 +23,16 @@ struct iommufd_attach {struct xarray device_array; }; +/* +* A noiommu device has no IOMMU driver attached regardless of Quoted 4 lines
whether it
+ * enters via the cdev path (noiommu_group) or the group path (fake
+ * noiommuiommu_group). In both casesdev->iommuis NULL.Are you trying to identify the case where both group and cdev interfaces coexist? In such a scenario, the group path creates a fake iommu group while the cdev path creates a dummy igroup. Both operations need to be performed, but checking
iommu_groupmight interfere with the cdev noiommu path logic.Quoted 8 lines
Is that why you're checking
idev->dev->iommuhere instead? If so, could you please make this rationale more explicit in the comment?At first glance, the comment is quite confusing. It's unclear why a cdev-path-only helper function needs to reference the group path at all. The last sentence in particular seems to suggest this helper covers both paths, which adds to the confusion.
Yes, Baolu also commented on this asking why not use
device_iommu_mapped. I will update the comment as follows:
/*
- * A noiommu device has no IOMMU driver attached regardless of whether it
- * enters via the cdev path (noiommu_group) or the group path (fake
- * noiommuiommu_group). In both casesdev->iommuis NULL.
+ * Detect a noiommu device for the cdev path. We checkdev->iommurather than
+ * usingdevice_iommu_mapped()(which checksdev->iommu_group) because when
+ * both group and cdev interfaces coexist, the group path assigns a fake
Quoted 9 lines
+ * noiommu
iommu_groupto the device. That would causedevice_iommu_mapped()
+ * to return true and hide the noiommu case from the cdev path.dev->iommuis
+ * reliably NULL when no IOMMU driver is managing the device. */
static booliommufd_device_is_noiommu(struct iommufd_device *idev){
this is clear to me now. I was just confused by the comment. And then realized what you really mean.
PATCH v5 5/9 iommufd: Add an ioctl to query PA from IOVA for noiommu mode
In reply to Jacob PanTo support no-IOMMU mode where userspace drivers perform unsafe DMA using physical addresses, introduce a new API to retrieve the physical address of a user-allocated DMA buffer that has been mapped to an IOVA via IOAS. The mapping is backed by SW-only I/O page tables maintained by the generic IOMMUPT framework.
---
v5:
- Add header stubs for iopt_get_phys() and iommufd_ioas_noiommu_get_pa() to avoid ifdef at call sites (Kevin)
v4:
- Fix ioctl return type (Yi Liu)
v2:
- New patch
--- drivers/iommu/iommufd/io_pagetable.c | 62 +++++++++++++++++++++++++ drivers/iommu/iommufd/ioas.c | 30 ++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 18 +++++++ drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 25 ++++++++++ 5 files changed, 138 insertions(+)
drivers/iommu/iommufd/io_pagetable.c +62 -0
@@ -859,6 +859,68 @@ int iopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova,return iopt_unmap_iova_range (iopt, iova, iova_last, unmapped); } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length) +{ +struct iopt_area *area; +u64 tmp_length =0 ; +u64 tmp_paddr =0 ; +int rc =0 ; + +down_read (&iopt->iova_rwsem); + area =iopt_area_iter_first (iopt, iova, iova); +if (!area || !area->pages) { + rc = -ENOENT ; + goto unlock_exit; + } + +if (!area->storage_domain || + area->storage_domain->owner != &iommufd_noiommu_ops) { + rc = -EOPNOTSUPP ; + goto unlock_exit; + } + + *paddr =iommu_iova_to_phys (area->storage_domain, iova); +if (!*paddr) { + rc = -EINVAL ; + goto unlock_exit; + } + + tmp_length =PAGE_SIZE -offset_in_page (iova); + tmp_paddr = *paddr; +/* +* Scan the domain for the contiguous physical address length so that +* userspace search can be optimized for fewer ioctls. +*/ +while (iova <iopt_area_last_iova (area)) { +unsigned long next_iova; +u64 next_paddr; + +if (check_add_overflow (iova,PAGE_SIZE , &next_iova)) +break ; + +if (next_iova >iopt_area_last_iova (area)) +break ; + + next_paddr =iommu_iova_to_phys (area->storage_domain, next_iova); + +if (!next_paddr || next_paddr != tmp_paddr +PAGE_SIZE ) +break ; + + iova = next_iova; + tmp_paddr +=PAGE_SIZE ; + tmp_length +=PAGE_SIZE ; + } + *length = tmp_length; + +unlock_exit: +up_read (&iopt->iova_rwsem); + +return rc; +} +#endif +int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped) {/* If the IOVAs are empty then unmap all succeeds */
drivers/iommu/iommufd/ioas.c +30 -0
@@ -375,6 +375,36 @@ int iommufd_ioas_unmap(struct iommufd_ucmd *ucmd)return rc; } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +struct iommu_ioas_noiommu_get_pa *cmd = ucmd->cmd; +struct iommufd_ioas *ioas; +int rc; + +if (!capable (CAP_SYS_RAWIO )) +return -EPERM ; + +if (cmd->flags || cmd->__reserved) +return -EOPNOTSUPP ; + + ioas =iommufd_get_ioas (ucmd->ictx, cmd->ioas_id); +if (IS_ERR (ioas)) +return PTR_ERR (ioas); + + rc =iopt_get_phys (&ioas->iopt, cmd->iova, &cmd->out_phys, + &cmd->out_length); +if (rc) + goto out_put; + + rc =iommufd_ucmd_respond (ucmd,sizeof (*cmd)); +out_put: +iommufd_put_object (ucmd->ictx, &ioas->obj); + +return rc; +} +#endif +static void iommufd_release_all_iova_rwsem (struct iommufd_ctx *ictx,struct xarray *ioas_list) {
drivers/iommu/iommufd/iommufd_private.h +18 -0
@@ -118,6 +118,16 @@ int iopt_map_pages(struct io_pagetable *iopt, struct list_head *pages_list,int iopt_unmap_iova (struct io_pagetable *iopt,unsigned long iova,unsigned long length,unsigned long *unmapped);int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length); +#else +static inline int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova, +u64 *paddr,u64 *length) +{ +return -EOPNOTSUPP ; +} +#endif int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain,
@@ -346,6 +356,14 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd);int iommufd_ioas_change_process (struct iommufd_ucmd *ucmd);int iommufd_ioas_copy (struct iommufd_ucmd *ucmd);int iommufd_ioas_unmap (struct iommufd_ucmd *ucmd); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd); +#else +static inline int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +return -EOPNOTSUPP ; +} +#endif int iommufd_ioas_option (struct iommufd_ucmd *ucmd);int iommufd_option_rlimit_mode (struct iommu_option *cmd,struct iommufd_ctx *ictx);
drivers/iommu/iommufd/main.c +3 -0
@@ -424,6 +424,7 @@ union ucmd_buffer {struct iommu_ioas_alloc alloc;struct iommu_ioas_allow_iovas allow_iovas;struct iommu_ioas_copy ioas_copy; +struct iommu_ioas_noiommu_get_pa noiommu_get_pa;struct iommu_ioas_iova_ranges iova_ranges;struct iommu_ioas_map map;struct iommu_ioas_unmap unmap;
@@ -482,6 +483,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { IOCTL_OP(IOMMU_IOAS_MAP, iommufd_ioas_map, struct iommu_ioas_map, iova),IOCTL_OP (IOMMU_IOAS_MAP_FILE , iommufd_ioas_map_file, struct iommu_ioas_map_file, iova), + IOCTL_OP(IOMMU_IOAS_NOIOMMU_GET_PA, iommufd_ioas_noiommu_get_pa, struct iommu_ioas_noiommu_get_pa, + out_phys), IOCTL_OP(IOMMU_IOAS_UNMAP, iommufd_ioas_unmap, struct iommu_ioas_unmap, length), IOCTL_OP(IOMMU_OPTION, iommufd_option, struct iommu_option, val64),
include/uapi/linux/iommufd.h +25 -0
@@ -57,6 +57,7 @@ enum {IOMMUFD_CMD_IOAS_CHANGE_PROCESS =0x92 ,IOMMUFD_CMD_VEVENTQ_ALLOC =0x93 ,IOMMUFD_CMD_HW_QUEUE_ALLOC =0x94 , + IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA = 0x95, };/**
@@ -219,6 +220,30 @@ struct iommu_ioas_map { };#define IOMMU_IOAS_MAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP) +/** +* struct iommu_ioas_noiommu_get_pa - ioctl(IOMMU_IOAS_NOIOMMU_GET_PA) +* @size: sizeof(struct iommu_ioas_noiommu_get_pa) +* @flags: Reserved, must be 0 for now +* @ioas_id: IOAS ID to query IOVA to PA mapping from +* @__reserved: Must be 0 +* @iova: IOVA to query +* @out_length: Number of bytes contiguous physical address starting from phys +* @out_phys: Output physical address the IOVA maps to +* +* Query the physical address backing an IOVA range. The entire range must be +* mapped already. For noiommu devices doing unsafe DMA only. +*/ +struct iommu_ioas_noiommu_get_pa { +__u32 size; +__u32 flags; +__u32 ioas_id; +__u32 __reserved; +__aligned_u64 iova; +__aligned_u64 out_length; +__aligned_u64 out_phys; +}; +#define IOMMU_IOAS_NOIOMMU_GET_PA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA) +/** * struct iommu_ioas_map_file - ioctl(IOMMU_IOAS_MAP_FILE) * @size: sizeof(struct iommu_ioas_map_file)
Signature · 1 line
2.43.0
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
Quoted 9 lines · On Mon, 11 May 2026 11:41:10 -0700 Jacob Pan <[email protected]>…
To support no-IOMMU mode where userspace drivers perform unsafe DMA using physical addresses, introduce a new API to retrieve the physical address of a user-allocated DMA buffer that has been mapped to an IOVA via IOAS. The mapping is backed by SW-only I/O page tables maintained by the generic IOMMUPT framework.
Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>
Quoted 6 lines
---
v5:
- Add header stubs foriopt_get_phys()andiommufd_ioas_noiommu_get_pa()to avoid ifdef at call sites
(Kevin) v4:
- Fix ioctl return type (Yi Liu)
This is not the correct change log, I have made a mistake. The correct one should be:
- Fix next_iova exceeds iopt_area_last_iova (Alex)
- Rename IOCTL more specific to NOIOMMU, i.e. IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA (Kevin)
Sorry about that.
Quoted 233 lines
v2:
- New patch--- drivers/iommu/iommufd/io_pagetable.c | 62 +++++++++++++++++++++++++ drivers/iommu/iommufd/ioas.c | 30 ++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 18 +++++++ drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 25 ++++++++++ 5 files changed, 138 insertions(+)(quoted diff) +0
-0
b/drivers/iommu/iommufd/io_pagetable.cindex24d4917105d9..1ee7c8e6408c100644 ---a/drivers/iommu/iommufd/io_pagetable.c+++b/drivers/iommu/iommufd/io_pagetable.c@@ -859,6 +859,68 @@ int iopt_unmap_iova(structio_pagetable*iopt, unsigned long iova, returniopt_unmap_iova_range(iopt, iova, iova_last, unmapped); }(quoted diff) +2
-0+#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys(struct io_pagetable *iopt, unsigned long iova, u64*paddr,
drivers/iommu/iommufd/io_pagetable.c +30
-0+ u64 *length) +{ +struct iopt_area *area; +u64 tmp_length =0 ; +u64 tmp_paddr =0 ; +int rc =0 ; + +down_read (&iopt->iova_rwsem); + area =iopt_area_iter_first (iopt, iova, iova); +if (!area || !area->pages) { + rc = -ENOENT ; + goto unlock_exit; + } + +if (!area->storage_domain || + area->storage_domain->owner != &iommufd_noiommu_ops) { + rc = -EOPNOTSUPP ; + goto unlock_exit; + } + + *paddr =iommu_iova_to_phys (area->storage_domain, iova); +if (!*paddr) { + rc = -EINVAL ; + goto unlock_exit; + } + + tmp_length =PAGE_SIZE -offset_in_page (iova); + tmp_paddr = *paddr; +/* +* Scan the domain for the contiguous physical address length so that
drivers/iommu/iommufd/io_pagetable.c +13
-0+* userspace search can be optimized for fewer ioctls. +*/ +while (iova <iopt_area_last_iova (area)) { +unsigned long next_iova; +u64 next_paddr; + +if (check_add_overflow (iova,PAGE_SIZE , &next_iova)) +break ; + +if (next_iova >iopt_area_last_iova (area)) +break ; + + next_paddr =
iommu_iova_to_phys(area->storage_domain, next_iova); +(quoted diff) +1
-0+ if (!next_paddr || next_paddr != tmp_paddr +
PAGE_SIZE)drivers/iommu/iommufd/io_pagetable.c +15
-0+break ; + + iova = next_iova; + tmp_paddr +=PAGE_SIZE ; + tmp_length += PAGE_SIZE; + } + *length = tmp_length; + +unlock_exit: +up_read (&iopt->iova_rwsem); + +return rc; +} +#endif + int iopt_unmap_all(struct io_pagetable *iopt, unsigned long*unmapped) {
drivers/iommu/iommufd/io_pagetable.c +0
-0/* If the IOVAs are empty then unmap all succeeds */ (quoted diff) +0
-0
b/drivers/iommu/iommufd/ioas.cindexfed06c2b728e..666440e32c9e100644drivers/iommu/iommufd/ioas.c +30
-0@@ -375,6 +375,36 @@ int iommufd_ioas_unmap(struct iommufd_ucmd *ucmd)return rc; } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +struct iommu_ioas_noiommu_get_pa *cmd = ucmd->cmd; +struct iommufd_ioas *ioas; +int rc; + +if (!capable (CAP_SYS_RAWIO )) +return -EPERM ; + +if (cmd->flags || cmd->__reserved) +return -EOPNOTSUPP ; + + ioas =iommufd_get_ioas (ucmd->ictx, cmd->ioas_id); +if (IS_ERR (ioas)) +return PTR_ERR (ioas); + + rc =iopt_get_phys (&ioas->iopt, cmd->iova, &cmd->out_phys, + &cmd->out_length); +if (rc) + goto out_put; + + rc =iommufd_ucmd_respond (ucmd,sizeof (*cmd)); +out_put: +iommufd_put_object (ucmd->ictx, &ioas->obj); + +return rc; +} +#endif +static void iommufd_release_all_iova_rwsem (struct iommufd_ctx *ictx,struct xarray *ioas_list) {(quoted diff) +0
-0
b/drivers/iommu/iommufd/iommufd_private.hindex2682b5baa6e9..13f1506d8066100644 ---a/drivers/iommu/iommufd/iommufd_private.h+++b/drivers/iommu/iommufd/iommufd_private.h@@ -118,6 +118,16 @@ int iopt_map_pages(structio_pagetable*iopt, structlist_head*pages_list, intiopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova, unsigned long length, unsigned long *unmapped); intiopt_unmap_all(struct io_pagetable *iopt, unsigned long *unmapped); +#ifdefCONFIG_IOMMUFD_NOIOMMU+int iopt_get_phys(structio_pagetable*iopt, unsigned long iova, u64 *paddr,drivers/iommu/iommufd/iommufd_private.h +3
-0+ u64 *length); +#else +static inline int iopt_get_phys(struct io_pagetable *iopt, unsignedlong iova,
drivers/iommu/iommufd/iommufd_private.h +12
-0+ u64 *paddr, u64 *length) +{ +return -EOPNOTSUPP ; +} +#endif int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain,@@ -346,6 +356,14 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd); int iommufd_ioas_change_process(struct iommufd_ucmd *ucmd);int iommufd_ioas_copy (struct iommufd_ucmd *ucmd);int iommufd_ioas_unmap (struct iommufd_ucmd *ucmd); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd); +#else +static inline int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +return -EOPNOTSUPP ; +} +#endif int iommufd_ioas_option (struct iommufd_ucmd *ucmd);int iommufd_option_rlimit_mode (struct iommu_option *cmd,struct iommufd_ctx *ictx);(quoted diff) +0
-0
b/drivers/iommu/iommufd/main.cindex8c6d43601afb..3b4192d70570100644drivers/iommu/iommufd/main.c +3
-0@@ -424,6 +424,7 @@ union ucmd_buffer {struct iommu_ioas_alloc alloc;struct iommu_ioas_allow_iovas allow_iovas;struct iommu_ioas_copy ioas_copy; +struct iommu_ioas_noiommu_get_pa noiommu_get_pa;struct iommu_ioas_iova_ranges iova_ranges;struct iommu_ioas_map map;struct iommu_ioas_unmap unmap;@@ -482,6 +483,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { IOCTL_OP(IOMMU_IOAS_MAP, iommufd_ioas_map, struct iommu_ioas_map, iova), IOCTL_OP(IOMMU_IOAS_MAP_FILE, iommufd_ioas_map_file, struct iommu_ioas_map_file, iova), + IOCTL_OP(IOMMU_IOAS_NOIOMMU_GET_PA, iommufd_ioas_noiommu_get_pa, struct iommu_ioas_noiommu_get_pa, + out_phys), IOCTL_OP(IOMMU_IOAS_UNMAP, iommufd_ioas_unmap, struct iommu_ioas_unmap, length),IOCTL_OP(IOMMU_OPTION,
iommufd_option, structiommu_option,
val64), diff --gita/include/uapi/linux/iommufd.hb/include/uapi/linux/iommufd.hindexe998dfbd6960..7df366d161f1100644include/uapi/linux/iommufd.h +9
-0@@ -57,6 +57,7 @@ enum {IOMMUFD_CMD_IOAS_CHANGE_PROCESS =0x92 ,IOMMUFD_CMD_VEVENTQ_ALLOC =0x93 ,IOMMUFD_CMD_HW_QUEUE_ALLOC =0x94 , + IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA = 0x95, };/** @@ -219,6 +220,30 @@ struct iommu_ioas_map { };#define IOMMU_IOAS_MAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP) +/** +* struct iommu_ioas_noiommu_get_pa - ioctl(IOMMU_IOAS_NOIOMMU_GET_PA) +* @size: sizeof(struct iommu_ioas_noiommu_get_pa) +* @flags: Reserved, must be 0 for now +* @ioas_id: IOAS ID to query IOVA to PA mapping from +* @__reserved: Must be 0 +* @iova: IOVA to query +* @out_length: Number of bytes contiguous physical address starting from phys
+ * @out_phys: Output physical address the IOVA maps to
+ *
+ * Query the physical address backing an IOVA range. The entire range must be
+ * mapped already. For noiommu devices doing unsafe DMA only.
+ */include/uapi/linux/iommufd.h +10
-0+struct iommu_ioas_noiommu_get_pa { +__u32 size; +__u32 flags; +__u32 ioas_id; +__u32 __reserved; +__aligned_u64 iova; +__aligned_u64 out_length; +__aligned_u64 out_phys; +}; +#define IOMMU_IOAS_NOIOMMU_GET_PA _IO(IOMMUFD_TYPE,
IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA) +
/**
* structiommu_ioas_map_file-ioctl(IOMMU_IOAS_MAP_FILE)
* @size:sizeof(struct iommu_ioas_map_file)
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
In reply to Jacob PanQuoted 210 lines · Jacob Pan
To support no-IOMMU mode where userspace drivers perform unsafe DMA using physical addresses, introduce a new API to retrieve the physical address of a user-allocated DMA buffer that has been mapped to an IOVA via IOAS. The mapping is backed by SW-only I/O page tables maintained by the generic IOMMUPT framework.
Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v5:
- Add header stubs foriopt_get_phys()andiommufd_ioas_noiommu_get_pa()to avoid ifdef at call sites (Kevin)
v4:
- Fix ioctl return type (Yi Liu)
v2:
- New patch--- drivers/iommu/iommufd/io_pagetable.c | 62 +++++++++++++++++++++++++ drivers/iommu/iommufd/ioas.c | 30 ++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 18 +++++++ drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 25 ++++++++++ 5 files changed, 138 insertions(+)drivers/iommu/iommufd/io_pagetable.c +62
-0@@ -859,6 +859,68 @@ int iopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova,return iopt_unmap_iova_range (iopt, iova, iova_last, unmapped); } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length) +{ +struct iopt_area *area; +u64 tmp_length =0 ; +u64 tmp_paddr =0 ; +int rc =0 ; + +down_read (&iopt->iova_rwsem); + area =iopt_area_iter_first (iopt, iova, iova); +if (!area || !area->pages) { + rc = -ENOENT ; + goto unlock_exit; + } + +if (!area->storage_domain || + area->storage_domain->owner != &iommufd_noiommu_ops) { + rc = -EOPNOTSUPP ; + goto unlock_exit; + } + + *paddr =iommu_iova_to_phys (area->storage_domain, iova); +if (!*paddr) { + rc = -EINVAL ; + goto unlock_exit; + } + + tmp_length =PAGE_SIZE -offset_in_page (iova); + tmp_paddr = *paddr; +/* +* Scan the domain for the contiguous physical address length so that +* userspace search can be optimized for fewer ioctls. +*/ +while (iova <iopt_area_last_iova (area)) { +unsigned long next_iova; +u64 next_paddr; + +if (check_add_overflow (iova,PAGE_SIZE , &next_iova)) +break ; + +if (next_iova >iopt_area_last_iova (area)) +break ; + + next_paddr =iommu_iova_to_phys (area->storage_domain, next_iova); + +if (!next_paddr || next_paddr != tmp_paddr +PAGE_SIZE ) +break ; + + iova = next_iova; + tmp_paddr +=PAGE_SIZE ; + tmp_length +=PAGE_SIZE ; + } + *length = tmp_length; + +unlock_exit: +up_read (&iopt->iova_rwsem); + +return rc; +} +#endif +int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped) {/* If the IOVAs are empty then unmap all succeeds */ drivers/iommu/iommufd/ioas.c +30
-0@@ -375,6 +375,36 @@ int iommufd_ioas_unmap(struct iommufd_ucmd *ucmd)return rc; } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +struct iommu_ioas_noiommu_get_pa *cmd = ucmd->cmd; +struct iommufd_ioas *ioas; +int rc; + +if (!capable (CAP_SYS_RAWIO )) +return -EPERM ; + +if (cmd->flags || cmd->__reserved) +return -EOPNOTSUPP ; + + ioas =iommufd_get_ioas (ucmd->ictx, cmd->ioas_id); +if (IS_ERR (ioas)) +return PTR_ERR (ioas); + + rc =iopt_get_phys (&ioas->iopt, cmd->iova, &cmd->out_phys, + &cmd->out_length); +if (rc) + goto out_put; + + rc =iommufd_ucmd_respond (ucmd,sizeof (*cmd)); +out_put: +iommufd_put_object (ucmd->ictx, &ioas->obj); + +return rc; +} +#endif +static void iommufd_release_all_iova_rwsem (struct iommufd_ctx *ictx,struct xarray *ioas_list) {drivers/iommu/iommufd/iommufd_private.h +18
-0@@ -118,6 +118,16 @@ int iopt_map_pages(struct io_pagetable *iopt, struct list_head *pages_list,int iopt_unmap_iova (struct io_pagetable *iopt,unsigned long iova,unsigned long length,unsigned long *unmapped);int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length); +#else +static inline int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova, +u64 *paddr,u64 *length) +{ +return -EOPNOTSUPP ; +} +#endif int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain,@@ -346,6 +356,14 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd);int iommufd_ioas_change_process (struct iommufd_ucmd *ucmd);int iommufd_ioas_copy (struct iommufd_ucmd *ucmd);int iommufd_ioas_unmap (struct iommufd_ucmd *ucmd); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd); +#else +static inline int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +return -EOPNOTSUPP ; +} +#endif int iommufd_ioas_option (struct iommufd_ucmd *ucmd);int iommufd_option_rlimit_mode (struct iommu_option *cmd,struct iommufd_ctx *ictx);drivers/iommu/iommufd/main.c +3
-0@@ -424,6 +424,7 @@ union ucmd_buffer {struct iommu_ioas_alloc alloc;struct iommu_ioas_allow_iovas allow_iovas;struct iommu_ioas_copy ioas_copy; +struct iommu_ioas_noiommu_get_pa noiommu_get_pa;struct iommu_ioas_iova_ranges iova_ranges;struct iommu_ioas_map map;struct iommu_ioas_unmap unmap;@@ -482,6 +483,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { IOCTL_OP(IOMMU_IOAS_MAP, iommufd_ioas_map, struct iommu_ioas_map, iova),IOCTL_OP (IOMMU_IOAS_MAP_FILE , iommufd_ioas_map_file, struct iommu_ioas_map_file, iova), + IOCTL_OP(IOMMU_IOAS_NOIOMMU_GET_PA, iommufd_ioas_noiommu_get_pa, struct iommu_ioas_noiommu_get_pa, + out_phys), IOCTL_OP(IOMMU_IOAS_UNMAP, iommufd_ioas_unmap, struct iommu_ioas_unmap, length), IOCTL_OP(IOMMU_OPTION, iommufd_option, struct iommu_option, val64),include/uapi/linux/iommufd.h +1
-0@@ -57,6 +57,7 @@ enum {IOMMUFD_CMD_IOAS_CHANGE_PROCESS =0x92 ,IOMMUFD_CMD_VEVENTQ_ALLOC =0x93 ,IOMMUFD_CMD_HW_QUEUE_ALLOC =0x94 , + IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA = 0x95, };/** @@ -219,6 +220,30 @@ struct iommu_ioas_map { };
include/uapi/linux/iommufd.h struct iommu_ioas_map { · line 230
Quoted 10 lines
include/uapi/linux/iommufd.h +8
-0#define IOMMU_IOAS_MAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP) +/** +* struct iommu_ioas_noiommu_get_pa - ioctl(IOMMU_IOAS_NOIOMMU_GET_PA) +* @size: sizeof(struct iommu_ioas_noiommu_get_pa) +* @flags: Reserved, must be 0 for now +* @ioas_id: IOAS ID to query IOVA to PA mapping from +* @__reserved: Must be 0 +* @iova: IOVA to query +* @out_length: Number of bytes contiguous physical address starting from phys
Nit: Instead of making this behavior mandatory, would it be valuable to allocate a bit in @flags to toggle this behavior? For extremely large mappings (e.g., several GBs of contiguous hugepages), the loop to determine the contiguous physical addresses might take a long time. A very long scan could theoretically delay userspace DMA setup.
Quoted 9 lines
include/uapi/linux/iommufd.h +9
-0+* @out_phys: Output physical address the IOVA maps to +* +* Query the physical address backing an IOVA range. The entire range must be +* mapped already. For noiommu devices doing unsafe DMA only. +*/ +struct iommu_ioas_noiommu_get_pa { +__u32 size; +__u32 flags; +__u32 ioas_id;
include/uapi/linux/iommufd.h struct iommu_ioas_map { · line 249
Quoted 10 lines
include/uapi/linux/iommufd.h +7
-0+__u32 __reserved; +__aligned_u64 iova; +__aligned_u64 out_length; +__aligned_u64 out_phys; +}; +#define IOMMU_IOAS_NOIOMMU_GET_PA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA) +/** * struct iommu_ioas_map_file - ioctl(IOMMU_IOAS_MAP_FILE) * @size: sizeof(struct iommu_ioas_map_file)
Otherwise, this looks good to me,
Jason Gunthorpe
maintainerRe: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
Quoted 10 lines · Baolu Lu
Quoted 9 lines
#define
IOMMU_IOAS_MAP_IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP)
+/**
+ * structiommu_ioas_noiommu_get_pa-ioctl(IOMMU_IOAS_NOIOMMU_GET_PA)
+ * @size:sizeof(struct iommu_ioas_noiommu_get_pa)
+ * @flags: Reserved, must be 0 for now
+ * @ioas_id: IOAS ID to query IOVA to PA mapping from
+ * @__reserved: Must be 0
+ * @iova: IOVA to query
+ * @out_length: Number of bytes contiguous physical address starting from phys
Quoted 5 lines
Nit: Instead of making this behavior mandatory, would it be valuable to allocate a bit in @flags to toggle this behavior? For extremely large mappings (e.g., several GBs of contiguous hugepages), the loop to determine the contiguous physical addresses might take a long time. A very long scan could theoretically delay userspace DMA setup.
If we are worrying about that then a iova/length pair would be appropriate such that out_len doesn't exceed the length. Right now it stops at the area boundary IIRC.
Jason
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
Hi Jason,
Quoted 20 lines · On Wed, 13 May 2026 09:22:21 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 12 lines · Baolu Lu
Quoted 11 lines
#define
IOMMU_IOAS_MAP_IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP)
+/**
+ * structiommu_ioas_noiommu_get_pa-ioctl(IOMMU_IOAS_NOIOMMU_GET_PA)
+ * @size:sizeof(struct iommu_ioas_noiommu_get_pa)
+ * @flags: Reserved, must be 0 for now
+ * @ioas_id: IOAS ID to query IOVA to PA mapping from
+ * @__reserved: Must be 0
+ * @iova: IOVA to query
+ * @out_length: Number of bytes contiguous physical address starting from physQuoted 6 lines
Nit: Instead of making this behavior mandatory, would it be valuable to allocate a bit in @flags to toggle this behavior? For extremely large mappings (e.g., several GBs of contiguous hugepages), the loop to determine the contiguous physical addresses might take a long time. A very long scan could theoretically delay userspace DMA setup.
Quoted 4 lines
If we are worrying about that then a iova/length pair would be appropriate such that
out_lendoesn't exceed the length. Right now it stops at the area boundary IIRC.
Yes, it stops at area boundary. How about renaming out_length to length, then use it as both input (cap on the scan range) and output (actual contiguous bytes found)? When length == 0 on input, it could mean "no limit" to keep the current behavior as default.
Jason Gunthorpe
maintainerRe: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
Quoted 30 lines · Jacob Pan
Hi Jason,
Quoted 20 lines · On Wed, 13 May 2026 09:22:21 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 12 lines · Baolu Lu
Quoted 11 lines
#define
IOMMU_IOAS_MAP_IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP)
+/**
+ * structiommu_ioas_noiommu_get_pa-ioctl(IOMMU_IOAS_NOIOMMU_GET_PA)
+ * @size:sizeof(struct iommu_ioas_noiommu_get_pa)
+ * @flags: Reserved, must be 0 for now
+ * @ioas_id: IOAS ID to query IOVA to PA mapping from
+ * @__reserved: Must be 0
+ * @iova: IOVA to query
+ * @out_length: Number of bytes contiguous physical address starting from physQuoted 6 lines
Nit: Instead of making this behavior mandatory, would it be valuable to allocate a bit in @flags to toggle this behavior? For extremely large mappings (e.g., several GBs of contiguous hugepages), the loop to determine the contiguous physical addresses might take a long time. A very long scan could theoretically delay userspace DMA setup.
Quoted 4 lines
If we are worrying about that then a iova/length pair would be appropriate such that
out_lendoesn't exceed the length. Right now it stops at the area boundary IIRC.
Quoted 4 lines
Yes, it stops at area boundary. How about renaming
out_lengthto length, then use it as both input (cap on the scan range) and output (actual contiguous bytes found)? When length == 0 on input, it could mean "no limit" to keep the current behavior as default.
That sounds Ok, document it carefully it is tricky!
Jason
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
In reply to Jacob PanQuoted 8 lines · Jacob Pan
To support no-IOMMU mode where userspace drivers perform unsafe DMA using physical addresses, introduce a new API to retrieve the physical address of a user-allocated DMA buffer that has been mapped to an IOVA via IOAS. The mapping is backed by SW-only I/O page tables maintained by the generic IOMMUPT framework.
Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Jason Gunthorpe <[email protected]>
since this patch is under your ahtuorship, maybe Jason's s-o-b should be prefixed with a c-d-b tag.
Quoted 217 lines
Signed-off-by Jacob Pan <[email protected]>---
v5:
- Add header stubs foriopt_get_phys()andiommufd_ioas_noiommu_get_pa()to avoid ifdef at call sites (Kevin)
v4:
- Fix ioctl return type (Yi Liu)
v2:
- New patch--- drivers/iommu/iommufd/io_pagetable.c | 62 +++++++++++++++++++++++++ drivers/iommu/iommufd/ioas.c | 30 ++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 18 +++++++ drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 25 ++++++++++ 5 files changed, 138 insertions(+)drivers/iommu/iommufd/io_pagetable.c +62
-0@@ -859,6 +859,68 @@ int iopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova,return iopt_unmap_iova_range (iopt, iova, iova_last, unmapped); } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length) +{ +struct iopt_area *area; +u64 tmp_length =0 ; +u64 tmp_paddr =0 ; +int rc =0 ; + +down_read (&iopt->iova_rwsem); + area =iopt_area_iter_first (iopt, iova, iova); +if (!area || !area->pages) { + rc = -ENOENT ; + goto unlock_exit; + } + +if (!area->storage_domain || + area->storage_domain->owner != &iommufd_noiommu_ops) { + rc = -EOPNOTSUPP ; + goto unlock_exit; + } + + *paddr =iommu_iova_to_phys (area->storage_domain, iova); +if (!*paddr) { + rc = -EINVAL ; + goto unlock_exit; + } + + tmp_length =PAGE_SIZE -offset_in_page (iova); + tmp_paddr = *paddr; +/* +* Scan the domain for the contiguous physical address length so that +* userspace search can be optimized for fewer ioctls. +*/ +while (iova <iopt_area_last_iova (area)) { +unsigned long next_iova; +u64 next_paddr; + +if (check_add_overflow (iova,PAGE_SIZE , &next_iova)) +break ; + +if (next_iova >iopt_area_last_iova (area)) +break ; + + next_paddr =iommu_iova_to_phys (area->storage_domain, next_iova); + +if (!next_paddr || next_paddr != tmp_paddr +PAGE_SIZE ) +break ; + + iova = next_iova; + tmp_paddr +=PAGE_SIZE ; + tmp_length +=PAGE_SIZE ; + } + *length = tmp_length; + +unlock_exit: +up_read (&iopt->iova_rwsem); + +return rc; +} +#endif +int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped) {/* If the IOVAs are empty then unmap all succeeds */ drivers/iommu/iommufd/ioas.c +30
-0@@ -375,6 +375,36 @@ int iommufd_ioas_unmap(struct iommufd_ucmd *ucmd)return rc; } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +struct iommu_ioas_noiommu_get_pa *cmd = ucmd->cmd; +struct iommufd_ioas *ioas; +int rc; + +if (!capable (CAP_SYS_RAWIO )) +return -EPERM ; + +if (cmd->flags || cmd->__reserved) +return -EOPNOTSUPP ; + + ioas =iommufd_get_ioas (ucmd->ictx, cmd->ioas_id); +if (IS_ERR (ioas)) +return PTR_ERR (ioas); + + rc =iopt_get_phys (&ioas->iopt, cmd->iova, &cmd->out_phys, + &cmd->out_length); +if (rc) + goto out_put; + + rc =iommufd_ucmd_respond (ucmd,sizeof (*cmd)); +out_put: +iommufd_put_object (ucmd->ictx, &ioas->obj); + +return rc; +} +#endif +static void iommufd_release_all_iova_rwsem (struct iommufd_ctx *ictx,struct xarray *ioas_list) {drivers/iommu/iommufd/iommufd_private.h +18
-0@@ -118,6 +118,16 @@ int iopt_map_pages(struct io_pagetable *iopt, struct list_head *pages_list,int iopt_unmap_iova (struct io_pagetable *iopt,unsigned long iova,unsigned long length,unsigned long *unmapped);int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length); +#else +static inline int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova, +u64 *paddr,u64 *length) +{ +return -EOPNOTSUPP ; +} +#endif int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain,@@ -346,6 +356,14 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd);int iommufd_ioas_change_process (struct iommufd_ucmd *ucmd);int iommufd_ioas_copy (struct iommufd_ucmd *ucmd);int iommufd_ioas_unmap (struct iommufd_ucmd *ucmd); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd); +#else +static inline int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +return -EOPNOTSUPP ; +} +#endif int iommufd_ioas_option (struct iommufd_ucmd *ucmd);int iommufd_option_rlimit_mode (struct iommu_option *cmd,struct iommufd_ctx *ictx);drivers/iommu/iommufd/main.c +3
-0@@ -424,6 +424,7 @@ union ucmd_buffer {struct iommu_ioas_alloc alloc;struct iommu_ioas_allow_iovas allow_iovas;struct iommu_ioas_copy ioas_copy; +struct iommu_ioas_noiommu_get_pa noiommu_get_pa;struct iommu_ioas_iova_ranges iova_ranges;struct iommu_ioas_map map;struct iommu_ioas_unmap unmap;@@ -482,6 +483,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { IOCTL_OP(IOMMU_IOAS_MAP, iommufd_ioas_map, struct iommu_ioas_map, iova),IOCTL_OP (IOMMU_IOAS_MAP_FILE , iommufd_ioas_map_file, struct iommu_ioas_map_file, iova), + IOCTL_OP(IOMMU_IOAS_NOIOMMU_GET_PA, iommufd_ioas_noiommu_get_pa, struct iommu_ioas_noiommu_get_pa, + out_phys), IOCTL_OP(IOMMU_IOAS_UNMAP, iommufd_ioas_unmap, struct iommu_ioas_unmap, length), IOCTL_OP(IOMMU_OPTION, iommufd_option, struct iommu_option, val64),include/uapi/linux/iommufd.h +14
-0@@ -57,6 +57,7 @@ enum {IOMMUFD_CMD_IOAS_CHANGE_PROCESS =0x92 ,IOMMUFD_CMD_VEVENTQ_ALLOC =0x93 ,IOMMUFD_CMD_HW_QUEUE_ALLOC =0x94 , + IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA = 0x95, };/** @@ -219,6 +220,30 @@ struct iommu_ioas_map { };#define IOMMU_IOAS_MAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP) +/** +* struct iommu_ioas_noiommu_get_pa - ioctl(IOMMU_IOAS_NOIOMMU_GET_PA) +* @size: sizeof(struct iommu_ioas_noiommu_get_pa) +* @flags: Reserved, must be 0 for now +* @ioas_id: IOAS ID to query IOVA to PA mapping from +* @__reserved: Must be 0 +* @iova: IOVA to query +* @out_length: Number of bytes contiguous physical address starting from phys +* @out_phys: Output physical address the IOVA maps to +* +* Query the physical address backing an IOVA range. The entire range must be +* mapped already. For noiommu devices doing unsafe DMA only. +*/
include/uapi/linux/iommufd.h struct iommu_ioas_map { · line 245
Quoted 10 lines
include/uapi/linux/iommufd.h +10
-0+struct iommu_ioas_noiommu_get_pa { +__u32 size; +__u32 flags; +__u32 ioas_id; +__u32 __reserved; +__aligned_u64 iova; +__aligned_u64 out_length; +__aligned_u64 out_phys; +}; +#define IOMMU_IOAS_NOIOMMU_GET_PA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA)
can this work for normal vfio usage as well? e.g. vfio usages that open devices that have iommu procetction.
My concern is kernel does not prevent normal vfio usage from invoking this ioctl. If it works for normal vfio usage as well, may just remove the noiommu term from thie name. If not, may be
Quoted 4 lines
include/uapi/linux/iommufd.h +1
-0+/** * struct iommu_ioas_map_file - ioctl(IOMMU_IOAS_MAP_FILE) * @size: sizeof(struct iommu_ioas_map_file)
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
Quoted 15 lines · Yi Liu
Quoted 11 lines
include/uapi/linux/iommufd.h +10
-0+struct iommu_ioas_noiommu_get_pa { + __u32 size; + __u32 flags; + __u32 ioas_id; + __u32 __reserved; + __aligned_u64 iova; + __aligned_u64 out_length; + __aligned_u64 out_phys; +}; +#define IOMMU_IOAS_NOIOMMU_GET_PA _IO(IOMMUFD_TYPE,
IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA)can this work for normal vfio usage as well? e.g. vfio usages that open devices that have iommu procetction.
Quoted 3 lines
My concern is kernel does not prevent normal vfio usage from invoking this ioctl. If it works for normal vfio usage as well, may just remove the noiommu term from thie name. If not, may be
failed to finish typing. I meant if this ioctl does not work for normal vfio usage, mind pointing me where kernel fails it. :)
Jason Gunthorpe
maintainerRe: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
In reply to Yi LiuQuoted 2 lines · Yi Liu
can this work for normal vfio usage as well? e.g. vfio usages that open devices that have iommu procetction.
No, it must be blocked we cannot leak physical addresses to userspace.
Quoted 3 lines
My concern is kernel does not prevent normal vfio usage from invoking this ioctl. If it works for normal vfio usage as well, may just remove the noiommu term from thie name. If not, may be
This is a bug
Jason
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
Quoted 5 lines · Jason Gunthorpe
Quoted 2 lines · Yi Liu
can this work for normal vfio usage as well? e.g. vfio usages that open devices that have iommu procetction.
Quoted 7 lines
No, it must be blocked we cannot leak physical addresses to userspace.
Quoted 3 lines
My concern is kernel does not prevent normal vfio usage from invoking this ioctl. If it works for normal vfio usage as well, may just remove the noiommu term from thie name. If not, may be
This is a bug
got it. After a second look, I got where it gets failed.
+ if (!area->storage_domain ||
+ area->storage_domain->owner != &iommufd_noiommu_ops) {
+ rc = -EOPNOTSUPP;
+ goto unlock_exit;
+ }
Regards,
Yi Liu
Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode
In reply to Yi LiuHi Yi,
Quoted 21 lines · On Wed, 20 May 2026 15:20:15 +0800 Yi Liu <[email protected]> wrote:
From: Yi Liu <[email protected]>
To: Jacob Pan <[email protected]>, <[email protected]>, "[email protected]" <[email protected]>, Jason Gunthorpe <[email protected]>, Alex Williamson <[email protected]>, Joerg Roedel <[email protected]>, Mostafa Saleh <[email protected]>, David Matlack <[email protected]>, Robin Murphy <[email protected]>, Nicolin Chen <[email protected]>, "Tian, Kevin" <[email protected]> CC:
Saurabh Sengar <[email protected]>, <[email protected]>, <[email protected]>, Will Deacon <[email protected]>, Baolu Lu <[email protected]> Subject: Re: [PATCH v5 5/9] iommufd: Add an ioctl to query PA from IOVA for noiommu mode Date: Wed, 20 May 2026 15:20:15 +0800 User-Agent: Mozilla ThunderbirdQuoted 6 lines · Jacob Pan
To support no-IOMMU mode where userspace drivers perform unsafe DMA using physical addresses, introduce a new API to retrieve the physical address of a user-allocated DMA buffer that has been mapped to an IOVA via IOAS. The mapping is backed by SW-only I/O page tables maintained by the generic IOMMUPT framework.
Quoted 5 lines
Quoted 2 lines
Suggested-by Jason Gunthorpe <[email protected]>Signed-off-by Jason Gunthorpe <[email protected]>since this patch is under your ahtuorship, maybe Jason's s-o-b should be prefixed with a c-d-b tag.
Missed this before, yeah, will do
PATCH v5 6/9 vfio/group: Add VFIO_CDEV_NOIOMMU Kconfig and tolerate NULL group
In reply to Jacob PanAdd a new CONFIG_VFIO_CDEV_NOIOMMU option, independent of CONFIG_VFIO_GROUP, to support noiommu mode via the cdev interface.
Since CONFIG_VFIO_GROUP can be enabled while CONFIG_VFIO_GROUP_NOIOMMU is not, guard the noiommu group allocation in vfio_group_find_or_alloc() with IS_ENABLED(CONFIG_VFIO_GROUP_NOIOMMU) to prevent creating spurious /dev/vfio/noiommu-N group files when only cdev noiommu is configured.
For cdev noiommu devices that have no group, let vfio_device_set_group() return success with a NULL group pointer and add null guards in group functions that may be called during device lifecycle. These guards are contained within group.c and are dead code for IOMMU-enabled devices where device->group is always non-NULL.
--- drivers/vfio/Kconfig | 17 +++++++++++++++++ drivers/vfio/group.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 2 deletions(-)
drivers/vfio/Kconfig +17 -0
@@ -75,6 +75,23 @@ config VFIO_GROUP_NOIOMMU If you don't know what to do here, say N. +config VFIO_CDEV_NOIOMMU + bool "VFIO cdev No-IOMMU support" + depends on VFIO_DEVICE_CDEV + select IOMMUFD_NOIOMMU + help + VFIO cdev no-IOMMU mode enables device access via the cdev + interface without hardware IOMMU backing. This relies on + IOMMUFD_NOIOMMU to provide a SW-only IO page table for + IOVA-to-PA lookups. + + Use of this mode will result in an unsupportable kernel and + will therefore taint the kernel. Device assignment to virtual + machines is also not possible with this mode since there is + no IOMMU to provide DMA translation. + + If you don't know what to do here, say N. + config VFIO_VIRQFD bool select EVENTFD
drivers/vfio/group.c +29 -2
@@ -386,6 +386,9 @@ int vfio_device_block_group(struct vfio_device *device)struct vfio_group *group = device->group;int ret =0 ; +if (!group) +return 0 ; +mutex_lock (&group->group_lock);if (group->opened_file) { ret = -EBUSY ;
@@ -403,6 +406,9 @@ void vfio_device_unblock_group(struct vfio_device *device) {struct vfio_group *group = device->group; +if (!group) +return ; +mutex_lock (&group->group_lock); group->cdev_device_open_cnt--;mutex_unlock (&group->group_lock);
@@ -641,7 +647,8 @@ static struct vfio_group *vfio_group_find_or_alloc(struct device *dev)struct vfio_group *group; iommu_group =iommu_group_get (dev); -if (!iommu_group && vfio_noiommu) { +if (!iommu_group &&IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) && + vfio_noiommu) {/* * With noiommu enabled, create an IOMMU group for devices that * don't already have one, implying no IOMMU hardware/driver
@@ -686,8 +693,19 @@ int vfio_device_set_group(struct vfio_device *device,else group =vfio_noiommu_group_alloc (device->dev, type); -if (IS_ERR (group)) +if (IS_ERR (group)) { +/* +* Cdev noiommu devices don't need a vfio_group. When +* CONFIG_VFIO_GROUP_NOIOMMU is not set, the group alloc +* above returns -EINVAL for devices without an IOMMU. +* That's fine — a NULL group is expected and iommufd +* handles these devices directly. +*/ +if (IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) && + vfio_noiommu && !device->dev->iommu) +return 0 ;return PTR_ERR (group); + }/* Our reference on group is moved to the device */ device->group = group;
@@ -699,6 +717,9 @@ void vfio_device_remove_group(struct vfio_device *device)struct vfio_group *group = device->group;struct iommu_group *iommu_group; +if (!group) +return ; +if (group->type ==VFIO_NO_IOMMU || group->type ==VFIO_EMULATED_IOMMU )iommu_group_remove_device (device->dev);
@@ -742,6 +763,8 @@ void vfio_device_remove_group(struct vfio_device *device)void vfio_device_group_register (struct vfio_device *device) { +if (!device->group) +return ;mutex_lock (&device->group->device_lock);list_add (&device->group_next, &device->group->device_list);mutex_unlock (&device->group->device_lock);
@@ -749,6 +772,8 @@ void vfio_device_group_register(struct vfio_device *device)void vfio_device_group_unregister (struct vfio_device *device) { +if (!device->group) +return ;mutex_lock (&device->group->device_lock);list_del (&device->group_next);mutex_unlock (&device->group->device_lock);
@@ -786,6 +811,8 @@ void vfio_device_group_unuse_iommu(struct vfio_device *device)bool vfio_device_has_container (struct vfio_device *device) { +if (!device->group) +return false;return device->group->container; }
Signature · 1 line
2.43.0
Alex Williamson
maintainerRe: [PATCH v5 6/9] vfio/group: Add VFIO_CDEV_NOIOMMU Kconfig and tolerate NULL group
Quoted 25 lines · On Mon, 11 May 2026 11:41:11 -0700 Jacob Pan <[email protected]>…
Add a new
CONFIG_VFIO_CDEV_NOIOMMUoption, independent ofCONFIG_VFIO_GROUP, to support noiommu mode via the cdev interface.Since
CONFIG_VFIO_GROUPcan be enabled whileCONFIG_VFIO_GROUP_NOIOMMUis not, guard the noiommu group allocation invfio_group_find_or_alloc()withIS_ENABLED(CONFIG_VFIO_GROUP_NOIOMMU)to prevent creating spurious/dev/vfio/noiommu-Ngroup files when only cdev noiommu is configured.For cdev noiommu devices that have no group, let
vfio_device_set_group()return success with a NULL group pointer and add null guards in group functions that may be called during device lifecycle. These guards are contained withingroup.cand are dead code for IOMMU-enabled devices wheredevice->groupis always non-NULL.Signed-off-by Jacob Pan <[email protected]>--- drivers/vfio/Kconfig | 17 +++++++++++++++++ drivers/vfio/group.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 2 deletions(-)drivers/vfio/Kconfig +0
-0
drivers/vfio/Kconfig config VFIO_GROUP_NOIOMMU · line 81
Quoted 7 lines
drivers/vfio/Kconfig +4
-0@@ -75,6 +75,23 @@ config VFIO_GROUP_NOIOMMU If you don't know what to do here, say N. +config VFIO_CDEV_NOIOMMU + bool "VFIO cdev No-IOMMU support" + depends on VFIO_DEVICE_CDEV + select IOMMUFD_NOIOMMU
AIUI, config would warn but allow this to select IOMMUFD_NOIOMMU even if the dependency on !GENERIC_ATOMIC64 in the Kconfig is unmet. This should include that dependency as well. Thanks,
Alex
Quoted 108 lines
drivers/vfio/Kconfig +13
-0+ help + VFIO cdev no-IOMMU mode enables device access via the cdev + interface without hardware IOMMU backing. This relies on + IOMMUFD_NOIOMMU to provide a SW-only IO page table for + IOVA-to-PA lookups. + + Use of this mode will result in an unsupportable kernel and + will therefore taint the kernel. Device assignment to virtual + machines is also not possible with this mode since there is + no IOMMU to provide DMA translation. + + If you don't know what to do here, say N. + config VFIO_VIRQFD bool select EVENTFDdrivers/vfio/group.c +29
-2@@ -386,6 +386,9 @@ int vfio_device_block_group(struct vfio_device *device)struct vfio_group *group = device->group;int ret =0 ; +if (!group) +return 0 ; +mutex_lock (&group->group_lock);if (group->opened_file) { ret = -EBUSY ;@@ -403,6 +406,9 @@ void vfio_device_unblock_group(struct vfio_device *device) {struct vfio_group *group = device->group; +if (!group) +return ; +mutex_lock (&group->group_lock); group->cdev_device_open_cnt--;mutex_unlock (&group->group_lock);@@ -641,7 +647,8 @@ static struct vfio_group *vfio_group_find_or_alloc(struct device *dev)struct vfio_group *group; iommu_group =iommu_group_get (dev); -if (!iommu_group && vfio_noiommu) { +if (!iommu_group &&IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) && + vfio_noiommu) {/* * With noiommu enabled, create an IOMMU group for devices that * don't already have one, implying no IOMMU hardware/driver @@ -686,8 +693,19 @@ int vfio_device_set_group(struct vfio_device *device,else group =vfio_noiommu_group_alloc (device->dev, type); -if (IS_ERR (group)) +if (IS_ERR (group)) { +/* +* Cdev noiommu devices don't need a vfio_group. When +* CONFIG_VFIO_GROUP_NOIOMMU is not set, the group alloc +* above returns -EINVAL for devices without an IOMMU. +* That's fine — a NULL group is expected and iommufd +* handles these devices directly. +*/ +if (IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) && + vfio_noiommu && !device->dev->iommu) +return 0 ;return PTR_ERR (group); + }/* Our reference on group is moved to the device */ device->group = group;@@ -699,6 +717,9 @@ void vfio_device_remove_group(struct vfio_device *device)struct vfio_group *group = device->group;struct iommu_group *iommu_group; +if (!group) +return ; +if (group->type ==VFIO_NO_IOMMU || group->type ==VFIO_EMULATED_IOMMU )iommu_group_remove_device (device->dev);@@ -742,6 +763,8 @@ void vfio_device_remove_group(struct vfio_device *device)void vfio_device_group_register (struct vfio_device *device) { +if (!device->group) +return ;mutex_lock (&device->group->device_lock);list_add (&device->group_next, &device->group->device_list);mutex_unlock (&device->group->device_lock);@@ -749,6 +772,8 @@ void vfio_device_group_register(struct vfio_device *device)void vfio_device_group_unregister (struct vfio_device *device) { +if (!device->group) +return ;mutex_lock (&device->group->device_lock);list_del (&device->group_next);mutex_unlock (&device->group->device_lock);@@ -786,6 +811,8 @@ void vfio_device_group_unuse_iommu(struct vfio_device *device)bool vfio_device_has_container (struct vfio_device *device) { +if (!device->group) +return false;return device->group->container; }
Re: [PATCH v5 6/9] vfio/group: Add VFIO_CDEV_NOIOMMU Kconfig and tolerate NULL group
Hi Alex,
Quoted 34 lines · On Tue, 19 May 2026 21:45:16 -0600 Alex Williamson <[email protected]> wrote:
Quoted 31 lines · On Mon, 11 May 2026 11:41:11 -0700 Jacob Pan <[email protected]>…
Add a new
CONFIG_VFIO_CDEV_NOIOMMUoption, independent ofCONFIG_VFIO_GROUP, to support noiommu mode via the cdev interface.Since
CONFIG_VFIO_GROUPcan be enabled whileCONFIG_VFIO_GROUP_NOIOMMUis not, guard the noiommu group allocation invfio_group_find_or_alloc()withIS_ENABLED(CONFIG_VFIO_GROUP_NOIOMMU)to prevent creating spurious/dev/vfio/noiommu-Ngroup files when only cdev noiommu is configured.For cdev noiommu devices that have no group, let
vfio_device_set_group()return success with a NULL group pointer and add null guards in group functions that may be called during device lifecycle. These guards are contained withingroup.cand are dead code for IOMMU-enabled devices wheredevice->groupis always non-NULL.Signed-off-by Jacob Pan <[email protected]>--- drivers/vfio/Kconfig | 17 +++++++++++++++++ drivers/vfio/group.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 46 insertions(+), 2 deletions(-)drivers/vfio/Kconfig +0
-0@@ -75,6 +75,23 @@ config VFIO_GROUP_NOIOMMU If you don't know what to do here, say N.
Quoted 9 lines
drivers/vfio/Kconfig config VFIO_GROUP_NOIOMMU · line 81
Quoted 4 lines
drivers/vfio/Kconfig +4
-0+config VFIO_CDEV_NOIOMMU + bool "VFIO cdev No-IOMMU support" + depends on VFIO_DEVICE_CDEV + select IOMMUFD_NOIOMMUAIUI, config would warn but allow this to select
IOMMUFD_NOIOMMUeven if the dependency on !GENERIC_ATOMIC64 in theKconfigis unmet. This should include that dependency as well. Thanks,
Yes, will add
depends on !GENERIC_ATOMIC64
PATCH v5 7/9 vfio: Enable cdev noiommu mode under iommufd
In reply to Jacob PanNow that devices under noiommu mode can bind with IOMMUFD and perform IOAS operations, lift restrictions on cdev from VFIO side.
Remove the vfio_device_is_group_noiommu() early returns in vfio_df_iommufd_bind() and vfio_df_iommufd_unbind() so that both group and cdev noiommu devices go through the standard iommufd bind path. This is safe because iommufd_device_bind() now handles noiommu devices via its own iommufd_device_is_noiommu() check.
Add CAP_SYS_RAWIO checks for cdev open and bind under noiommu to maintain security parity with the group noiommu path.
No IOMMU cdevs are explicitly named with noiommu prefix. e.g.
/dev/vfio/
|-- devices
| `-- noiommu-vfio0
`-- vfio---
v5:
- Add Kconfig VFIO_CDEV_NOIOMMU to select IOMMUFD_NOIOMMU and its dependencies
- Add comment to explain vfio_noiommu conditional definition (Alex)
- Removed early return for group noiommu in bind/unbind
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Update unsafe_noiommu Kconfig help text (Kevin)
- Change dev_warn to dev_info for noiommu enabling msg (Kevin)
v4:
- Remove early return in iommufd_bind for noiommu (Alex)
v3:
- Consolidate into fewer patches
v2:
- removed unnecessary device->noiommu set in iommufd_vfio_compat_ioas_get_id()
--- drivers/vfio/Kconfig | 3 +-- drivers/vfio/device_cdev.c | 10 ++++++++++ drivers/vfio/iommufd.c | 7 ------- drivers/vfio/vfio.h | 22 ++++++++++++++-------- drivers/vfio/vfio_main.c | 25 ++++++++++++++++++++----- include/linux/vfio.h | 1 + 6 files changed, 46 insertions(+), 22 deletions(-)
drivers/vfio/Kconfig +1 -2
@@ -22,8 +22,7 @@ config VFIO_DEVICE_CDEV The VFIO device cdev is another way for userspace to get device access. Userspace gets device fd by opening device cdev under /dev/vfio/devices/vfioX, and then bind the device fd with an iommufd - to set up secure DMA context for device access. This interface does - not support noiommu. + to set up secure DMA context for device access. If you don't know what to do here, say N.
drivers/vfio/device_cdev.c +10 -0
@@ -27,6 +27,9 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep)struct vfio_device_file *df;int ret; +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ; +/* Paired with the put in vfio_device_fops_release() */ if (!vfio_device_try_get_registration (device))return -ENODEV ;
@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,if (df->group)return -EINVAL ; +/* +* CAP_SYS_RAWIO is already checked at cdev open, recheck here +* in case the fd was passed to a less privileged process. +*/ +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ; + ret =vfio_device_block_group (device);if (ret)return ret;
drivers/vfio/iommufd.c +0 -7
@@ -25,10 +25,6 @@ int vfio_df_iommufd_bind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_group_noiommu (vdev)) -return 0 ; -return vdev->ops->bind_iommufd (vdev, ictx, &df->devid); }
@@ -58,9 +54,6 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -if (vfio_device_is_group_noiommu (vdev)) -return ; -if (vdev->ops->unbind_iommufd) vdev->ops->unbind_iommufd (vdev); }
drivers/vfio/vfio.h +14 -8
@@ -36,7 +36,7 @@ vfio_allocate_device_file(struct vfio_device *device);extern const struct file_operations vfio_device_fops; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )extern bool vfio_noiommu__read_mostly ;#else enum { vfio_noiommu = false };
@@ -358,19 +358,13 @@ void vfio_init_device_cdev(struct vfio_device *device);static inline int vfio_device_add (struct vfio_device *device) { -/* cdev does not support noiommu device */ -if (vfio_device_is_group_noiommu (device)) -return device_add (&device->device);vfio_init_device_cdev (device);return cdev_device_add (&device->cdev, &device->device); }static inline void vfio_device_del (struct vfio_device *device) { -if (vfio_device_is_group_noiommu (device)) -device_del (&device->device); -else -cdev_device_del (&device->cdev, &device->device); +cdev_device_del (&device->cdev, &device->device); }int vfio_device_fops_cdev_open (struct inode *inode,struct file *filep);
@@ -420,6 +414,18 @@ static inline void vfio_cdev_cleanup(void) }#endif /* CONFIG_VFIO_DEVICE_CDEV */ +#if IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) +static inline bool vfio_device_is_cdev_noiommu (struct vfio_device *vdev) +{ +return vdev->noiommu; +} +#else +static inline bool vfio_device_is_cdev_noiommu (struct vfio_device *vdev) +{ +return false; +} +#endif +#if IS_ENABLED (CONFIG_VFIO_VIRQFD )int __init vfio_virqfd_init (void );void vfio_virqfd_exit (void );
drivers/vfio/vfio_main.c +20 -5
@@ -54,7 +54,7 @@ static struct vfio {int fs_count; } vfio; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )bool vfio_noiommu__read_mostly ;module_param_named (enable_unsafe_noiommu_mode, vfio_noiommu, bool,S_IRUGO |S_IWUSR );
@@ -321,6 +321,20 @@ static int vfio_init_device(struct vfio_device *device, struct device *dev,return ret; } +static int vfio_device_set_noiommu_and_name (struct vfio_device *device) +{ +if (IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) && vfio_noiommu && !device->dev->iommu) { + device->noiommu = true; +add_taint (TAINT_USER ,LOCKDEP_STILL_OK ); +dev_warn (device->dev, +"Adding kernel taint for vfio-noiommu cdev on device\n" ); + } + +/* Just to be safe, expose to user explicitly noiommu cdev node */ +return dev_set_name (&device->device,"%svfio%d" , + device->noiommu ?"noiommu-" :"" , device->index); +} +static int __vfio_register_dev (struct vfio_device *device,enum vfio_group_type type) {
@@ -340,20 +354,21 @@ static int __vfio_register_dev(struct vfio_device *device,if (!device->dev_set)vfio_assign_device_set (device, device); - ret =dev_set_name (&device->device,"vfio%d" , device->index); + ret =vfio_device_set_group (device, type);if (ret)return ret; - ret =vfio_device_set_group (device, type); + ret =vfio_device_set_noiommu_and_name (device);if (ret) -return ret; + goto err_out;/* * VFIO always sets IOMMU_CACHE because we offer no way for userspace to * restore cache coherency. It has to be checked here because it is only * valid for cases where we are using iommu groups. */ -if (type ==VFIO_IOMMU && !vfio_device_is_group_noiommu (device) && +if (type ==VFIO_IOMMU && !(vfio_device_is_group_noiommu (device) || +vfio_device_is_cdev_noiommu (device)) && !device_iommu_capable (device->dev,IOMMU_CAP_CACHE_COHERENCY )) { ret = -EINVAL ; goto err_out;
include/linux/vfio.h +1 -0
@@ -74,6 +74,7 @@ struct vfio_device { u8 iommufd_attached:1;#endif u8 cdev_opened:1; + u8 noiommu:1;/* * debug_root is a static property of the vfio_device * which must be set prior to registering the vfio_device.
Signature · 1 line
2.43.0
Jason Gunthorpe
maintainerRe: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
drivers/vfio/device_cdev.c long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, · line 121
Quoted 10 lines · Jacob Pan
drivers/vfio/device_cdev.c +6
-0@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,if (df->group)return -EINVAL ; +/* +* CAP_SYS_RAWIO is already checked at cdev open, recheck here +* in case the fd was passed to a less privileged process. +*/ +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ;
I don't think we should do this, an open only check is sufficient. It is entirely reasonable to design a userspace to drop SYS_RAWIO after it opens the FD to minimize retained privileges.
Jason
Re: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
Hi Jason,
Quoted 5 lines · On Tue, 19 May 2026 20:40:05 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 4 lines · Jacob Pan
drivers/vfio/device_cdev.c +0
-0@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, if (df->group) return -EINVAL;
Quoted 11 lines
drivers/vfio/device_cdev.c long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, · line 121
Quoted 7 lines
(quoted diff) +2
-0+ /* + * CAP_SYS_RAWIO is already checked at cdev open, recheckhere
drivers/vfio/device_cdev.c +4
-0+* in case the fd was passed to a less privileged process. +*/ +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ;I don't think we should do this, an open only check is sufficient. It is entirely reasonable to design a userspace to drop
SYS_RAWIOafter it opens the FD to minimize retained privileges.
right, will remove. VFIO group also only checks at open not during SET_CONTAINER.
Alex Williamson
maintainerRe: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
In reply to Jacob PanQuoted 117 lines · On Mon, 11 May 2026 11:41:12 -0700 Jacob Pan <[email protected]>…
Now that devices under noiommu mode can bind with IOMMUFD and perform IOAS operations, lift restrictions on cdev from VFIO side.
Remove the
vfio_device_is_group_noiommu()early returns invfio_df_iommufd_bind()andvfio_df_iommufd_unbind()so that both group and cdev noiommu devices go through the standard iommufd bind path. This is safe becauseiommufd_device_bind()now handles noiommu devices via its owniommufd_device_is_noiommu()check.Add
CAP_SYS_RAWIOchecks for cdev open and bind under noiommu to maintain security parity with the group noiommu path.No IOMMU cdevs are explicitly named with noiommu prefix. e.g.
/dev/vfio/ |-- devices | `-- noiommu-vfio0 `-- vfioSigned-off-by Jacob Pan <[email protected]>---
v5:
- AddKconfigVFIO_CDEV_NOIOMMUto selectIOMMUFD_NOIOMMUand its dependencies
- Add comment to explainvfio_noiommuconditional definition (Alex)
- Removed early return for group noiommu in bind/unbind
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Updateunsafe_noiommuKconfighelp text (Kevin)
- Changedev_warntodev_infofor noiommu enabling msg (Kevin)
v4:
- Remove early return iniommufd_bindfor noiommu (Alex)
v3:
- Consolidate into fewer patches
v2:
- removed unnecessarydevice->noiommuset iniommufd_vfio_compat_ioas_get_id()--- drivers/vfio/Kconfig | 3 +-- drivers/vfio/device_cdev.c | 10 ++++++++++ drivers/vfio/iommufd.c | 7 ------- drivers/vfio/vfio.h | 22 ++++++++++++++-------- drivers/vfio/vfio_main.c | 25 ++++++++++++++++++++----- include/linux/vfio.h | 1 + 6 files changed, 46 insertions(+), 22 deletions(-)drivers/vfio/Kconfig +1
-2@@ -22,8 +22,7 @@ config VFIO_DEVICE_CDEV The VFIO device cdev is another way for userspace to get device access. Userspace gets device fd by opening device cdev under /dev/vfio/devices/vfioX, and then bind the device fd with an iommufd - to set up secure DMA context for device access. This interface does - not support noiommu. + to set up secure DMA context for device access. If you don't know what to do here, say N.drivers/vfio/device_cdev.c +10
-0@@ -27,6 +27,9 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep)struct vfio_device_file *df;int ret; +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ; +/* Paired with the put in vfio_device_fops_release() */ if (!vfio_device_try_get_registration (device))return -ENODEV ;@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,if (df->group)return -EINVAL ; +/* +* CAP_SYS_RAWIO is already checked at cdev open, recheck here +* in case the fd was passed to a less privileged process. +*/ +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ; + ret =vfio_device_block_group (device);if (ret)return ret;drivers/vfio/iommufd.c +0
-7@@ -25,10 +25,6 @@ int vfio_df_iommufd_bind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_group_noiommu (vdev)) -return 0 ; -return vdev->ops->bind_iommufd (vdev, ictx, &df->devid); }@@ -58,9 +54,6 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -if (vfio_device_is_group_noiommu (vdev)) -return ; -if (vdev->ops->unbind_iommufd) vdev->ops->unbind_iommufd (vdev); }drivers/vfio/vfio.h +0
-0
drivers/vfio/vfio.h vfio_allocate_device_file(struct vfio_device *device); · line 39
Quoted 5 lines
drivers/vfio/vfio.h +1
-1@@ -36,7 +36,7 @@ vfio_allocate_device_file(struct vfio_device *device);extern const struct file_operations vfio_device_fops; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )
Have you considered what happens when these are y/n or n/y?
I think in the former case we can create cdev devices for group-noiommu devices that are not labeled noiommu, skip the CAP_SYS_RAWIO test, but will fail to bind. In the latter case, I think we fail to setup an iommufd_device and unbind will segfault.
We really don't need to support independently setting GROUP vs CDEV NOIOMMU, the suggestion was to try to get NOIOMMU from depending on VFIO_GROUP. We can do that other ways though and I think we can do it without the rename in patch 1 that will inevitably result in some lost config options for NOIOMMU on upgrade.
The Kconfig may get messy, perhaps something like:
config VFIO_NOIOMMU
bool "VFIO No-IOMMU support" depends on VFIO_GROUP || VFIO_DEVICE_CDEV depends on !VFIO_GROUP || VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER depends on !VFIO_DEVICE_CDEV || !GENERIC_ATOMIC64 select IOMMUFD_NOIOMMU if VFIO_DEVICE_CDEV
Sorry if the previous suggestion sent us astray, but the subtleties of independent support look tricky. Thanks,
Alex
Quoted 115 lines
drivers/vfio/vfio.h +13
-7extern bool vfio_noiommu__read_mostly ;#else enum { vfio_noiommu = false };@@ -358,19 +358,13 @@ void vfio_init_device_cdev(struct vfio_device *device);static inline int vfio_device_add (struct vfio_device *device) { -/* cdev does not support noiommu device */ -if (vfio_device_is_group_noiommu (device)) -return device_add (&device->device);vfio_init_device_cdev (device);return cdev_device_add (&device->cdev, &device->device); }static inline void vfio_device_del (struct vfio_device *device) { -if (vfio_device_is_group_noiommu (device)) -device_del (&device->device); -else -cdev_device_del (&device->cdev, &device->device); +cdev_device_del (&device->cdev, &device->device); }int vfio_device_fops_cdev_open (struct inode *inode,struct file *filep);@@ -420,6 +414,18 @@ static inline void vfio_cdev_cleanup(void) }#endif /* CONFIG_VFIO_DEVICE_CDEV */ +#if IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) +static inline bool vfio_device_is_cdev_noiommu (struct vfio_device *vdev) +{ +return vdev->noiommu; +} +#else +static inline bool vfio_device_is_cdev_noiommu (struct vfio_device *vdev) +{ +return false; +} +#endif +#if IS_ENABLED (CONFIG_VFIO_VIRQFD )int __init vfio_virqfd_init (void );void vfio_virqfd_exit (void );drivers/vfio/vfio_main.c +20
-5@@ -54,7 +54,7 @@ static struct vfio {int fs_count; } vfio; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )bool vfio_noiommu__read_mostly ;module_param_named (enable_unsafe_noiommu_mode, vfio_noiommu, bool,S_IRUGO |S_IWUSR );@@ -321,6 +321,20 @@ static int vfio_init_device(struct vfio_device *device, struct device *dev,return ret; } +static int vfio_device_set_noiommu_and_name (struct vfio_device *device) +{ +if (IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) && vfio_noiommu && !device->dev->iommu) { + device->noiommu = true; +add_taint (TAINT_USER ,LOCKDEP_STILL_OK ); +dev_warn (device->dev, +"Adding kernel taint for vfio-noiommu cdev on device\n" ); + } + +/* Just to be safe, expose to user explicitly noiommu cdev node */ +return dev_set_name (&device->device,"%svfio%d" , + device->noiommu ?"noiommu-" :"" , device->index); +} +static int __vfio_register_dev (struct vfio_device *device,enum vfio_group_type type) {@@ -340,20 +354,21 @@ static int __vfio_register_dev(struct vfio_device *device,if (!device->dev_set)vfio_assign_device_set (device, device); - ret =dev_set_name (&device->device,"vfio%d" , device->index); + ret =vfio_device_set_group (device, type);if (ret)return ret; - ret =vfio_device_set_group (device, type); + ret =vfio_device_set_noiommu_and_name (device);if (ret) -return ret; + goto err_out;/* * VFIO always sets IOMMU_CACHE because we offer no way for userspace to * restore cache coherency. It has to be checked here because it is only * valid for cases where we are using iommu groups. */ -if (type ==VFIO_IOMMU && !vfio_device_is_group_noiommu (device) && +if (type ==VFIO_IOMMU && !(vfio_device_is_group_noiommu (device) || +vfio_device_is_cdev_noiommu (device)) && !device_iommu_capable (device->dev,IOMMU_CAP_CACHE_COHERENCY )) { ret = -EINVAL ; goto err_out;include/linux/vfio.h +1
-0@@ -74,6 +74,7 @@ struct vfio_device { u8 iommufd_attached:1;#endif u8 cdev_opened:1; + u8 noiommu:1;/* * debug_root is a static property of the vfio_device * which must be set prior to registering the vfio_device.
Re: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
Quoted 142 lines · Alex Williamson
Quoted 117 lines · On Mon, 11 May 2026 11:41:12 -0700 Jacob Pan <[email protected]>…
Now that devices under noiommu mode can bind with IOMMUFD and perform IOAS operations, lift restrictions on cdev from VFIO side.
Remove the
vfio_device_is_group_noiommu()early returns invfio_df_iommufd_bind()andvfio_df_iommufd_unbind()so that both group and cdev noiommu devices go through the standard iommufd bind path. This is safe becauseiommufd_device_bind()now handles noiommu devices via its owniommufd_device_is_noiommu()check.Add
CAP_SYS_RAWIOchecks for cdev open and bind under noiommu to maintain security parity with the group noiommu path.No IOMMU cdevs are explicitly named with noiommu prefix. e.g.
/dev/vfio/ |-- devices | `-- noiommu-vfio0 `-- vfioSigned-off-by Jacob Pan <[email protected]>---
v5:
- AddKconfigVFIO_CDEV_NOIOMMUto selectIOMMUFD_NOIOMMUand its dependencies
- Add comment to explainvfio_noiommuconditional definition (Alex)
- Removed early return for group noiommu in bind/unbind
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Updateunsafe_noiommuKconfighelp text (Kevin)
- Changedev_warntodev_infofor noiommu enabling msg (Kevin)
v4:
- Remove early return iniommufd_bindfor noiommu (Alex)
v3:
- Consolidate into fewer patches
v2:
- removed unnecessarydevice->noiommuset iniommufd_vfio_compat_ioas_get_id()--- drivers/vfio/Kconfig | 3 +-- drivers/vfio/device_cdev.c | 10 ++++++++++ drivers/vfio/iommufd.c | 7 ------- drivers/vfio/vfio.h | 22 ++++++++++++++-------- drivers/vfio/vfio_main.c | 25 ++++++++++++++++++++----- include/linux/vfio.h | 1 + 6 files changed, 46 insertions(+), 22 deletions(-)drivers/vfio/Kconfig +1
-2@@ -22,8 +22,7 @@ config VFIO_DEVICE_CDEV The VFIO device cdev is another way for userspace to get device access. Userspace gets device fd by opening device cdev under /dev/vfio/devices/vfioX, and then bind the device fd with an iommufd - to set up secure DMA context for device access. This interface does - not support noiommu. + to set up secure DMA context for device access. If you don't know what to do here, say N.drivers/vfio/device_cdev.c +10
-0@@ -27,6 +27,9 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep)struct vfio_device_file *df;int ret; +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ; +/* Paired with the put in vfio_device_fops_release() */ if (!vfio_device_try_get_registration (device))return -ENODEV ;@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,if (df->group)return -EINVAL ; +/* +* CAP_SYS_RAWIO is already checked at cdev open, recheck here +* in case the fd was passed to a less privileged process. +*/ +if (device->noiommu && !capable (CAP_SYS_RAWIO )) +return -EPERM ; + ret =vfio_device_block_group (device);if (ret)return ret;drivers/vfio/iommufd.c +0
-7@@ -25,10 +25,6 @@ int vfio_df_iommufd_bind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_group_noiommu (vdev)) -return 0 ; -return vdev->ops->bind_iommufd (vdev, ictx, &df->devid); }@@ -58,9 +54,6 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -if (vfio_device_is_group_noiommu (vdev)) -return ; -if (vdev->ops->unbind_iommufd) vdev->ops->unbind_iommufd (vdev); }drivers/vfio/vfio.h +0
-0drivers/vfio/vfio.h vfio_allocate_device_file(struct vfio_device *device); · line 39
Quoted 5 lines
drivers/vfio/vfio.h +1
-1@@ -36,7 +36,7 @@ vfio_allocate_device_file(struct vfio_device *device);extern const struct file_operations vfio_device_fops; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )Have you considered what happens when these are y/n or n/y?
I think in the former case we can create cdev devices for group-noiommu devices that are not labeled noiommu, skip the
CAP_SYS_RAWIOtest, but will fail to bind. In the latter case, I think we fail to setup aniommufd_deviceand unbind will segfault.We really don't need to support independently setting GROUP vs CDEV NOIOMMU, the suggestion was to try to get NOIOMMU from depending on
VFIO_GROUP. We can do that other ways though and I think we can do it without the rename in patch 1 that will inevitably result in some lost config options for NOIOMMU on upgrade.The
Kconfigmay get messy, perhaps something like:config
VFIO_NOIOMMU
Quoted 8 lines
bool "VFIO No-IOMMU support" depends on VFIO_GROUP || VFIO_DEVICE_CDEV depends on !VFIO_GROUP || VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER depends on !VFIO_DEVICE_CDEV || !GENERIC_ATOMIC64 select IOMMUFD_NOIOMMU if VFIO_DEVICE_CDEVSorry if the previous suggestion sent us astray, but the subtleties of independent support look tricky. Thanks,
this also looks better to me. Less kconfigs. :)
just one nit: current VFIO_NOIOMMU only depends on VFIO_GROUP, maybe a
separate patch to extend it depends on 'VFIO_GROUP && (VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER)' first, then add the cdev path noiommu dependency.
Quoted 118 lines
Alex
Quoted 115 lines
drivers/vfio/vfio.h +13
-7extern bool vfio_noiommu__read_mostly ;#else enum { vfio_noiommu = false };@@ -358,19 +358,13 @@ void vfio_init_device_cdev(struct vfio_device *device);static inline int vfio_device_add (struct vfio_device *device) { -/* cdev does not support noiommu device */ -if (vfio_device_is_group_noiommu (device)) -return device_add (&device->device);vfio_init_device_cdev (device);return cdev_device_add (&device->cdev, &device->device); }static inline void vfio_device_del (struct vfio_device *device) { -if (vfio_device_is_group_noiommu (device)) -device_del (&device->device); -else -cdev_device_del (&device->cdev, &device->device); +cdev_device_del (&device->cdev, &device->device); }int vfio_device_fops_cdev_open (struct inode *inode,struct file *filep);@@ -420,6 +414,18 @@ static inline void vfio_cdev_cleanup(void) }#endif /* CONFIG_VFIO_DEVICE_CDEV */ +#if IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) +static inline bool vfio_device_is_cdev_noiommu (struct vfio_device *vdev) +{ +return vdev->noiommu; +} +#else +static inline bool vfio_device_is_cdev_noiommu (struct vfio_device *vdev) +{ +return false; +} +#endif +#if IS_ENABLED (CONFIG_VFIO_VIRQFD )int __init vfio_virqfd_init (void );void vfio_virqfd_exit (void );drivers/vfio/vfio_main.c +20
-5@@ -54,7 +54,7 @@ static struct vfio {int fs_count; } vfio; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )bool vfio_noiommu__read_mostly ;module_param_named (enable_unsafe_noiommu_mode, vfio_noiommu, bool,S_IRUGO |S_IWUSR );@@ -321,6 +321,20 @@ static int vfio_init_device(struct vfio_device *device, struct device *dev,return ret; } +static int vfio_device_set_noiommu_and_name (struct vfio_device *device) +{ +if (IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU ) && vfio_noiommu && !device->dev->iommu) { + device->noiommu = true; +add_taint (TAINT_USER ,LOCKDEP_STILL_OK ); +dev_warn (device->dev, +"Adding kernel taint for vfio-noiommu cdev on device\n" ); + } + +/* Just to be safe, expose to user explicitly noiommu cdev node */ +return dev_set_name (&device->device,"%svfio%d" , + device->noiommu ?"noiommu-" :"" , device->index); +} +static int __vfio_register_dev (struct vfio_device *device,enum vfio_group_type type) {@@ -340,20 +354,21 @@ static int __vfio_register_dev(struct vfio_device *device,if (!device->dev_set)vfio_assign_device_set (device, device); - ret =dev_set_name (&device->device,"vfio%d" , device->index); + ret =vfio_device_set_group (device, type);if (ret)return ret; - ret =vfio_device_set_group (device, type); + ret =vfio_device_set_noiommu_and_name (device);if (ret) -return ret; + goto err_out;/* * VFIO always sets IOMMU_CACHE because we offer no way for userspace to * restore cache coherency. It has to be checked here because it is only * valid for cases where we are using iommu groups. */ -if (type ==VFIO_IOMMU && !vfio_device_is_group_noiommu (device) && +if (type ==VFIO_IOMMU && !(vfio_device_is_group_noiommu (device) || +vfio_device_is_cdev_noiommu (device)) && !device_iommu_capable (device->dev,IOMMU_CAP_CACHE_COHERENCY )) { ret = -EINVAL ; goto err_out;include/linux/vfio.h +1
-0@@ -74,6 +74,7 @@ struct vfio_device { u8 iommufd_attached:1;#endif u8 cdev_opened:1; + u8 noiommu:1;/* * debug_root is a static property of the vfio_device * which must be set prior to registering the vfio_device.
Re: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
Hi Yi,
Quoted 155 lines · On Wed, 20 May 2026 15:20:26 +0800 Yi Liu <[email protected]> wrote:
Quoted 154 lines · Alex Williamson
Quoted 122 lines · On Mon, 11 May 2026 11:41:12 -0700 Jacob Pan <[email protected]>…
Now that devices under noiommu mode can bind with IOMMUFD and perform IOAS operations, lift restrictions on cdev from VFIO side.
Remove the
vfio_device_is_group_noiommu()early returns invfio_df_iommufd_bind()andvfio_df_iommufd_unbind()so that both group and cdev noiommu devices go through the standard iommufd bind path. This is safe becauseiommufd_device_bind()now handles noiommu devices via its owniommufd_device_is_noiommu()check.Add
CAP_SYS_RAWIOchecks for cdev open and bind under noiommu to maintain security parity with the group noiommu path.No IOMMU cdevs are explicitly named with noiommu prefix. e.g.
/dev/vfio/ |-- devices | `-- noiommu-vfio0 `-- vfioSigned-off-by Jacob Pan <[email protected]>---
v5:
- AddKconfigVFIO_CDEV_NOIOMMUto selectIOMMUFD_NOIOMMUand its dependencies
- Add comment to explainvfio_noiommuconditional definition
(Alex)
- Removed early return for group noiommu in bind/unbind
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Updateunsafe_noiommuKconfighelp text (Kevin)
- Changedev_warntodev_infofor noiommu enabling msg (Kevin)
v4:
- Remove early return iniommufd_bindfor noiommu (Alex)
v3:
- Consolidate into fewer patches
v2:
- removed unnecessarydevice->noiommuset iniommufd_vfio_compat_ioas_get_id()--- drivers/vfio/Kconfig | 3 +-- drivers/vfio/device_cdev.c | 10 ++++++++++ drivers/vfio/iommufd.c | 7 ------- drivers/vfio/vfio.h | 22 ++++++++++++++-------- drivers/vfio/vfio_main.c | 25 ++++++++++++++++++++----- include/linux/vfio.h | 1 + 6 files changed, 46 insertions(+), 22 deletions(-)drivers/vfio/Kconfig +0
-0@@ -22,8 +22,7 @@ config VFIO_DEVICE_CDEV The VFIO device cdev is another way for userspace toget device access. Userspace gets device fd by opening device cdev under
/dev/vfio/devices/vfioX, and then bind the device fd with an iommufddrivers/vfio/Kconfig +0
-1- to set up secure DMA context for device access. Thisinterface does
drivers/vfio/Kconfig +1
-1- not support noiommu. + to set up secure DMA context for device access. If you don't know what to do here, say N.(quoted diff) +0
-0
b/drivers/vfio/device_cdev.cindex54abf312cf04..46a808244398100644 ---a/drivers/vfio/device_cdev.c
+++b/drivers/vfio/device_cdev.c(quoted diff) +5
-0@@ -27,6 +27,9 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep) struct vfio_device_file *df; int ret; + if (device->noiommu && !capable(CAP_SYS_RAWIO)) + return -EPERM; + /* Paired with the put in vfio_device_fops_release() */ if (!vfio_device_try_get_registration(device)) return -ENODEV;@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, if (df->group) return -EINVAL; + /* + * CAP_SYS_RAWIO is already checked at cdev open, recheckhere
(quoted diff) +5
-0+ * in case the fd was passed to a less privileged process. + */ + if (device->noiommu && !capable(CAP_SYS_RAWIO)) + return -EPERM; + ret = vfio_device_block_group(device); if (ret) return ret;drivers/vfio/iommufd.c +0
-7@@ -25,10 +25,6 @@ int vfio_df_iommufd_bind(structvfio_device_file *df) lockdep_assert_held(&vdev->dev_set->lock); -/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_group_noiommu (vdev)) -return 0 ; -return vdev->ops->bind_iommufd (vdev, ictx, &df->devid); }@@ -58,9 +54,6 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df) lockdep_assert_held(&vdev->dev_set->lock); -if (vfio_device_is_group_noiommu (vdev)) -return ; -if (vdev->ops->unbind_iommufd) vdev->ops->unbind_iommufd (vdev); }drivers/vfio/vfio.h +0
-0drivers/vfio/vfio.h vfio_allocate_device_file(struct vfio_device · line 40
Quoted 6 lines
drivers/vfio/vfio.h +1
-1@@ -36,7 +36,7 @@ vfio_allocate_device_file(struct vfio_device *device);extern const struct file_operations vfio_device_fops; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )Have you considered what happens when these are y/n or n/y?
I think in the former case we can create cdev devices for group-noiommu devices that are not labeled noiommu, skip the
CAP_SYS_RAWIOtest, but will fail to bind. In the latter case, I think we fail to setup aniommufd_deviceand unbind will segfault.We really don't need to support independently setting GROUP vs CDEV NOIOMMU, the suggestion was to try to get NOIOMMU from depending on
VFIO_GROUP. We can do that other ways though and I think we can do it without the rename in patch 1 that will inevitably result in some lost config options for NOIOMMU on upgrade.The
Kconfigmay get messy, perhaps something like:config
VFIO_NOIOMMUbool "VFIO No-IOMMU support" depends on VFIO_GROUP || VFIO_DEVICE_CDEV depends on !VFIO_GROUP || VFIO_CONTAINER ||
IOMMUFD_VFIO_CONTAINERdepends on !VFIO_DEVICE_CDEV || !GENERIC_ATOMIC64 selectIOMMUFD_NOIOMMUifVFIO_DEVICE_CDEV
Quoted 5 lines
Quoted 2 lines
Sorry if the previous suggestion sent us astray, but the subtleties of independent support look tricky. Thanks,
this also looks better to me. Less kconfigs. :)
agreed, let me give it a try in v6.
Quoted 4 lines
just one nit: current
VFIO_NOIOMMUonly depends onVFIO_GROUP, maybe a separate patch to extend it depends on 'VFIO_GROUP&& (VFIO_CONTAINER||IOMMUFD_VFIO_CONTAINER)' first, then add the cdev path noiommu dependency.
I had this below in patch 1/9. will do in a separate patch. Thanks,
Jacob
drivers/vfio/Kconfig +3 -3
@@ -60,9 +60,9 @@ config VFIO_IOMMU_SPAPR_TCE default VFIO endif -config VFIO_NOIOMMU - bool "VFIO No-IOMMU support" - depends on VFIO_GROUP +config VFIO_GROUP_NOIOMMU + bool "VFIO group No-IOMMU support" + depends on VFIO_GROUP && (VFIO_CONTAINER ||
IOMMUFD_VFIO_CONTAINER) help
Re: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
Quoted 189 lines · Jacob Pan
Hi Yi,
Quoted 155 lines · On Wed, 20 May 2026 15:20:26 +0800 Yi Liu <[email protected]> wrote:
Quoted 154 lines · Alex Williamson
Quoted 122 lines · On Mon, 11 May 2026 11:41:12 -0700 Jacob Pan <[email protected]>…
Now that devices under noiommu mode can bind with IOMMUFD and perform IOAS operations, lift restrictions on cdev from VFIO side.
Remove the
vfio_device_is_group_noiommu()early returns invfio_df_iommufd_bind()andvfio_df_iommufd_unbind()so that both group and cdev noiommu devices go through the standard iommufd bind path. This is safe becauseiommufd_device_bind()now handles noiommu devices via its owniommufd_device_is_noiommu()check.Add
CAP_SYS_RAWIOchecks for cdev open and bind under noiommu to maintain security parity with the group noiommu path.No IOMMU cdevs are explicitly named with noiommu prefix. e.g.
/dev/vfio/ |-- devices | `-- noiommu-vfio0 `-- vfioSigned-off-by Jacob Pan <[email protected]>---
v5:
- AddKconfigVFIO_CDEV_NOIOMMUto selectIOMMUFD_NOIOMMUand its dependencies
- Add comment to explainvfio_noiommuconditional definition
(Alex)
- Removed early return for group noiommu in bind/unbind
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Updateunsafe_noiommuKconfighelp text (Kevin)
- Changedev_warntodev_infofor noiommu enabling msg (Kevin)
v4:
- Remove early return iniommufd_bindfor noiommu (Alex)
v3:
- Consolidate into fewer patches
v2:
- removed unnecessarydevice->noiommuset iniommufd_vfio_compat_ioas_get_id()--- drivers/vfio/Kconfig | 3 +-- drivers/vfio/device_cdev.c | 10 ++++++++++ drivers/vfio/iommufd.c | 7 ------- drivers/vfio/vfio.h | 22 ++++++++++++++-------- drivers/vfio/vfio_main.c | 25 ++++++++++++++++++++----- include/linux/vfio.h | 1 + 6 files changed, 46 insertions(+), 22 deletions(-)drivers/vfio/Kconfig +0
-0@@ -22,8 +22,7 @@ config VFIO_DEVICE_CDEV The VFIO device cdev is another way for userspace toget device access. Userspace gets device fd by opening device cdev under
/dev/vfio/devices/vfioX, and then bind the device fd with an iommufddrivers/vfio/Kconfig +0
-1- to set up secure DMA context for device access. Thisinterface does
drivers/vfio/Kconfig +1
-1- not support noiommu. + to set up secure DMA context for device access. If you don't know what to do here, say N.(quoted diff) +0
-0
b/drivers/vfio/device_cdev.cindex54abf312cf04..46a808244398100644 ---a/drivers/vfio/device_cdev.c
+++b/drivers/vfio/device_cdev.c(quoted diff) +5
-0@@ -27,6 +27,9 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep) struct vfio_device_file *df; int ret; + if (device->noiommu && !capable(CAP_SYS_RAWIO)) + return -EPERM; + /* Paired with the put in vfio_device_fops_release() */ if (!vfio_device_try_get_registration(device)) return -ENODEV;@@ -110,6 +113,13 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df, if (df->group) return -EINVAL; + /* + * CAP_SYS_RAWIO is already checked at cdev open, recheckhere
(quoted diff) +5
-0+ * in case the fd was passed to a less privileged process. + */ + if (device->noiommu && !capable(CAP_SYS_RAWIO)) + return -EPERM; + ret = vfio_device_block_group(device); if (ret) return ret;drivers/vfio/iommufd.c +0
-7@@ -25,10 +25,6 @@ int vfio_df_iommufd_bind(structvfio_device_file *df) lockdep_assert_held(&vdev->dev_set->lock); -/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_group_noiommu (vdev)) -return 0 ; -return vdev->ops->bind_iommufd (vdev, ictx, &df->devid); }@@ -58,9 +54,6 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df) lockdep_assert_held(&vdev->dev_set->lock); -if (vfio_device_is_group_noiommu (vdev)) -return ; -if (vdev->ops->unbind_iommufd) vdev->ops->unbind_iommufd (vdev); }drivers/vfio/vfio.h +0
-0drivers/vfio/vfio.h vfio_allocate_device_file(struct vfio_device · line 40
Quoted 6 lines
drivers/vfio/vfio.h +1
-1@@ -36,7 +36,7 @@ vfio_allocate_device_file(struct vfio_device *device);extern const struct file_operations vfio_device_fops; -#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED (CONFIG_VFIO_GROUP_NOIOMMU ) ||IS_ENABLED (CONFIG_VFIO_CDEV_NOIOMMU )Have you considered what happens when these are y/n or n/y?
I think in the former case we can create cdev devices for group-noiommu devices that are not labeled noiommu, skip the
CAP_SYS_RAWIOtest, but will fail to bind. In the latter case, I think we fail to setup aniommufd_deviceand unbind will segfault.We really don't need to support independently setting GROUP vs CDEV NOIOMMU, the suggestion was to try to get NOIOMMU from depending on
VFIO_GROUP. We can do that other ways though and I think we can do it without the rename in patch 1 that will inevitably result in some lost config options for NOIOMMU on upgrade.The
Kconfigmay get messy, perhaps something like:config
VFIO_NOIOMMUbool "VFIO No-IOMMU support" depends on VFIO_GROUP || VFIO_DEVICE_CDEV depends on !VFIO_GROUP || VFIO_CONTAINER ||
IOMMUFD_VFIO_CONTAINERdepends on !VFIO_DEVICE_CDEV || !GENERIC_ATOMIC64 selectIOMMUFD_NOIOMMUifVFIO_DEVICE_CDEVQuoted 5 lines
Quoted 2 lines
Sorry if the previous suggestion sent us astray, but the subtleties of independent support look tricky. Thanks,
this also looks better to me. Less kconfigs. :)
agreed, let me give it a try in v6.
Quoted 4 lines
just one nit: current
VFIO_NOIOMMUonly depends onVFIO_GROUP, maybe a separate patch to extend it depends on 'VFIO_GROUP&& (VFIO_CONTAINER||IOMMUFD_VFIO_CONTAINER)' first, then add the cdev path noiommu dependency.I had this below in patch 1/9. will do in a separate patch. Thanks,
Jacob
drivers/vfio/Kconfig +3
-3@@ -60,9 +60,9 @@ config VFIO_IOMMU_SPAPR_TCE default VFIO endif -config VFIO_NOIOMMU - bool "VFIO No-IOMMU support" - depends on VFIO_GROUP +config VFIO_GROUP_NOIOMMU + bool "VFIO group No-IOMMU support" + depends on VFIO_GROUP && (VFIO_CONTAINER ||
Quoted 1 line
IOMMUFD_VFIO_CONTAINER) help
got it. not sure if it needs to have a fix tag or not. @Alex?
Re: [PATCH v5 7/9] vfio: Enable cdev noiommu mode under iommufd
In reply to Yi LiuHi Yi,
Quoted 27 lines · On Wed, 20 May 2026 15:20:26 +0800 Yi Liu <[email protected]> wrote:
Quoted 27 lines
Quoted 4 lines
(quoted diff) +1
-1-#ifdef CONFIG_VFIO_GROUP_NOIOMMU +#if IS_ENABLED(CONFIG_VFIO_GROUP_NOIOMMU) ||
IS_ENABLED(CONFIG_VFIO_CDEV_NOIOMMU)Have you considered what happens when these are y/n or n/y?
I think in the former case we can create cdev devices for group-noiommu devices that are not labeled noiommu, skip the
CAP_SYS_RAWIOtest, but will fail to bind. In the latter case, I think we fail to setup aniommufd_deviceand unbind will segfault.We really don't need to support independently setting GROUP vs CDEV NOIOMMU, the suggestion was to try to get NOIOMMU from depending on
VFIO_GROUP. We can do that other ways though and I think we can do it without the rename in patch 1 that will inevitably result in some lost config options for NOIOMMU on upgrade.The
Kconfigmay get messy, perhaps something like:config
VFIO_NOIOMMUbool "VFIO No-IOMMU support" depends on VFIO_GROUP || VFIO_DEVICE_CDEV depends on !VFIO_GROUP || VFIO_CONTAINER ||
IOMMUFD_VFIO_CONTAINERdepends on !VFIO_DEVICE_CDEV || !GENERIC_ATOMIC64 selectIOMMUFD_NOIOMMUifVFIO_DEVICE_CDEV
Quoted 4 lines
Quoted 2 lines
Sorry if the previous suggestion sent us astray, but the subtleties of independent support look tricky. Thanks,
this also looks better to me. Less kconfigs. :)
Just to recap our discussions, we agreed with Alex's Kconfig and the breakdown cases are as follows:
Container = VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER
# GROUP Container CDEV NOIOMMU Notes - ----- --------- ---- ------- ----------------------- 1 y y n yes Group noiommu works 2 y n n no Blocked - no container 3 y y y yes Both paths work 4 y n y no Blocked - no container 5 n - y yes Cdev-only works 6 n - n no No access path
Note that we sacrifice #4. The tradeoff: case 4 sacrifices cdev noiommu to avoid the useless group node problem purely in Kconfig. The practical impact is small - if someone wants cdev noiommu without container, they just set GROUP=n. This greatly simplifies the code since NULL group checks are not needed.
OTOH, if we want #4, we can loosen the Kconfig:
depends on !VFIO_GROUP || VFIO_DEVICE_CDEV || \ VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER
but that would allow creating useless group nodes without a container backend. Enforcing VFIO_GROUP=n purely via Kconfig is not possible since it would cause a circular dependency (CONTAINER depends on GROUP).
I will send v6 shortly.
Thanks,
Jacob
PATCH v5 8/9 selftests/vfio: Add iommufd noiommu mode selftest for cdev
In reply to Jacob PanAdd comprehensive selftest for VFIO device operations with iommufd in noiommu mode. Tests cover:
- Device binding to iommufd
- IOAS (I/O Address Space) allocation, mapping with dummy IOVA
- Retrieve PA from dummy IOVA
- Device attach/detach operations as usual
---
v4:
- squash DSA specific selftest changes
v2:
- New selftest for generic noiommu bind/unbind
--- tools/testing/selftests/vfio/Makefile | 1 + .../lib/include/libvfio/vfio_pci_device.h | 16 + .../selftests/vfio/lib/vfio_pci_device.c | 5 +- .../vfio/vfio_iommufd_noiommu_test.c | 567 ++++++++++++++++++ 4 files changed, 587 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/vfio/vfio_iommufd_noiommu_test.c
tools/testing/selftests/vfio/Makefile +1 -0
@@ -9,6 +9,7 @@ CFLAGS = $(KHDR_INCLUDES) TEST_GEN_PROGS += vfio_dma_mapping_test TEST_GEN_PROGS += vfio_dma_mapping_mmio_test TEST_GEN_PROGS += vfio_iommufd_setup_test +TEST_GEN_PROGS += vfio_iommufd_noiommu_test TEST_GEN_PROGS += vfio_pci_device_test TEST_GEN_PROGS += vfio_pci_device_init_perf_test TEST_GEN_PROGS += vfio_pci_driver_test
tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h +16 -0
@@ -122,4 +122,20 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device,const char *vfio_pci_get_cdev_path (const char *bdf); +static inline bool vfio_pci_noiommu_mode_enabled (void ) +{ +char buf[8 ] = {}; +int fd, n; + + fd =open ("/sys/module/vfio/parameters/enable_unsafe_noiommu_mode" , +O_RDONLY ); +if (fd <0 ) +return false; + + n =read (fd, buf,sizeof (buf) -1 ); +close (fd); + +return n >0 && buf[0 ] =='Y' ; +} +#endif /* SELFTESTS_VFIO_LIB_INCLUDE_LIBVFIO_VFIO_PCI_DEVICE_H */
tools/testing/selftests/vfio/lib/vfio_pci_device.c +3 -2
@@ -308,8 +308,9 @@ const char *vfio_pci_get_cdev_path(const char *bdf)VFIO_ASSERT_NOT_NULL (dir,"Failed to open directory %s\n" , dir_path);while ((entry =readdir (dir)) !=NULL ) { -/* Find the file that starts with "vfio" */ -if (strncmp ("vfio" , entry->d_name,4 )) +/* Find the file that starts with "vfio" or "noiommu-vfio" */ +if (strncmp ("vfio" , entry->d_name,4 ) && +strncmp ("noiommu-vfio" , entry->d_name,12 ))continue ;snprintf (cdev_path,PATH_MAX ,"/dev/vfio/devices/%s" , entry->d_name);
tools/testing/selftests/vfio/vfio_iommufd_noiommu_test.c new file +567 -0
567 lines not on this page · raw message
Signature · 1 line
2.43.0
PATCH v5 9/9 Documentation: Update VFIO NOIOMMU mode
In reply to Jacob PanDocument the NOIOMMU mode with newly added cdev support under iommufd.
--- Documentation/driver-api/vfio.rst | 46 +++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-)
Documentation/driver-api/vfio.rst +44 -2
@@ -275,8 +275,6 @@ in a VFIO group. With CONFIG_VFIO_DEVICE_CDEV=y the user can now acquire a device fd by directly opening a character device /dev/vfio/devices/vfioX where "X" is the number allocated uniquely by VFIO for registered devices. -cdev interface does not support noiommu devices, so user should use -the legacy group interface if noiommu is wanted. The cdev only works with IOMMUFD. Both VFIO drivers and applications must adapt to the new cdev security model which requires using
@@ -370,6 +368,50 @@ IOMMUFD IOAS/HWPT to enable userspace DMA:: /* Other device operations as stated in "VFIO Usage Example" */ +VFIO NOIOMMU mode +------------------------------------------------------------------------------- +VFIO also supports a no-IOMMU mode, intended for usages where unsafe DMA can +be performed by userspace drivers w/o physical IOMMU protection. This mode +is controlled by the parameter: + +/sys/module/vfio/parameters/enable_unsafe_noiommu_mode + +Upon enabling this mode, with an assigned device, the user will be presented +with a VFIO group and device file, e.g.:: + + /dev/vfio/ + |-- devices + | `-- noiommu-vfio0 /* VFIO device cdev */ + |-- noiommu-0 /* VFIO group */ + `-- vfio + +The capabilities vary depending on the device programming interface and kernel +configuration used. The following table summarizes the differences: + ++-------------------+---------------------+----------------------+ +| Feature | VFIO group | VFIO device cdev | ++===================+=====================+======================+ +| VFIO device UAPI | Yes | Yes | ++-------------------+---------------------+----------------------+ +| VFIO container | No | No | ++-------------------+---------------------+----------------------+ +| IOMMUFD IOAS | No | Yes* | ++-------------------+---------------------+----------------------+ + +Note that the VFIO container case includes IOMMUFD provided VFIO compatibility +interfaces when either CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER is +enabled. + +* IOMMUFD UAPI is available for VFIO device cdev to pin and map user memory with + the ability to retrieve physical addresses for DMA command submission. + +A new IOMMUFD ioctl IOMMU_IOAS_NOIOMMU_GET_PA is added to retrieve the physical +address for a given IOVA. Although there is no physical DMA remapping hardware, +IOMMU_IOAS_MAP_FIXED_IOVA is still used to establish IOVA-to-PA mappings in the +software page table for later IOMMU_IOAS_NOIOMMU_GET_PA lookups. +tools/testing/selftests/vfio/vfio_iommufd_noiommu_test.c provides an example of +using this ioctl in no-IOMMU mode. + VFIO User API -------------------------------------------------------------------------------
Signature · 1 line
2.43.0
Re: [PATCH v5 9/9] Documentation: Update VFIO NOIOMMU mode
Quoted 31 lines · Jacob Pan
Document the NOIOMMU mode with newly added cdev support under iommufd.
Cc Jonathan Corbet <[email protected]>Signed-off-by Jacob Pan <[email protected]>--- Documentation/driver-api/vfio.rst | 46 +++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-)Documentation/driver-api/vfio.rst +6
-2@@ -275,8 +275,6 @@ in a VFIO group. With CONFIG_VFIO_DEVICE_CDEV=y the user can now acquire a device fd by directly opening a character device /dev/vfio/devices/vfioX where "X" is the number allocated uniquely by VFIO for registered devices. -cdev interface does not support noiommu devices, so user should use -the legacy group interface if noiommu is wanted. The cdev only works with IOMMUFD. Both VFIO drivers and applications must adapt to the new cdev security model which requires using@@ -370,6 +368,50 @@ IOMMUFD IOAS/HWPT to enable userspace DMA:: /* Other device operations as stated in "VFIO Usage Example" */ +VFIO NOIOMMU mode +------------------------------------------------------------------------------- +VFIO also supports a no-IOMMU mode, intended for usages where unsafe DMA can +be performed by userspace drivers w/o physical IOMMU protection. This mode +is controlled by the parameter: +
Documentation/driver-api/vfio.rst IOMMUFD IOAS/HWPT to enable userspace DMA:: · line 386
Quoted 10 lines
Documentation/driver-api/vfio.rst +10
-0+/sys/module/vfio/parameters/enable_unsafe_noiommu_mode + +Upon enabling this mode, with an assigned device, the user will be presented +with a VFIO group and device file, e.g.:: + + /dev/vfio/ + |-- devices + | `-- noiommu-vfio0 /* VFIO device cdev */ + |-- noiommu-0 /* VFIO group */ + `-- vfio
how about noiommu-vfioX and noiommu-Y? Just want to deliver the message that the two number is not 100% identical.
Documentation/driver-api/vfio.rst IOMMUFD IOAS/HWPT to enable userspace DMA:: · line 400
Quoted 13 lines
Documentation/driver-api/vfio.rst +13
-0+The capabilities vary depending on the device programming interface and kernel +configuration used. The following table summarizes the differences: + ++-------------------+---------------------+----------------------+ +| Feature | VFIO group | VFIO device cdev | ++===================+=====================+======================+ +| VFIO device UAPI | Yes | Yes | ++-------------------+---------------------+----------------------+ +| VFIO container | No | No | ++-------------------+---------------------+----------------------+ +| IOMMUFD IOAS | No | Yes* | ++-------------------+---------------------+----------------------+ +
not quite the above table. Why 'VFIO container' is "No" for VFIO group?
Quoted 17 lines
Documentation/driver-api/vfio.rst +14
-0+Note that the VFIO container case includes IOMMUFD provided VFIO compatibility +interfaces when either CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER is +enabled. + +* IOMMUFD UAPI is available for VFIO device cdev to pin and map user memory with + the ability to retrieve physical addresses for DMA command submission. + +A new IOMMUFD ioctl IOMMU_IOAS_NOIOMMU_GET_PA is added to retrieve the physical +address for a given IOVA. Although there is no physical DMA remapping hardware, +IOMMU_IOAS_MAP_FIXED_IOVA is still used to establish IOVA-to-PA mappings in the +software page table for later IOMMU_IOAS_NOIOMMU_GET_PA lookups. +tools/testing/selftests/vfio/vfio_iommufd_noiommu_test.c provides an example of +using this ioctl in no-IOMMU mode. + VFIO User API -------------------------------------------------------------------------------
Re: [PATCH v5 9/9] Documentation: Update VFIO NOIOMMU mode
Hi Yi,
Quoted 39 lines · On Wed, 20 May 2026 15:20:30 +0800 Yi Liu <[email protected]> wrote:
Quoted 38 lines · Jacob Pan
Document the NOIOMMU mode with newly added cdev support under iommufd.
Cc Jonathan Corbet <[email protected]>Signed-off-by Jacob Pan <[email protected]>--- Documentation/driver-api/vfio.rst | 46+++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-)
(quoted diff) +0
-0
b/Documentation/driver-api/vfio.rstindex2a21a42c9386..d97017d80b98100644 ---a/Documentation/driver-api/vfio.rst+++b/Documentation/driver-api/vfio.rst@@ -275,8 +275,6 @@ in a VFIO group. WithCONFIG_VFIO_DEVICE_CDEV=ythe user can now acquire a device fd by directly opening a character device/dev/vfio/devices/vfioXwhere "X" is the number allocated uniquely by VFIO for registered devices. -cdev interface does not support noiommu devices, so user should use -the legacy group interface if noiommu is wanted.
The cdev only works with IOMMUFD. Both VFIO drivers and
applications must adapt to the new cdev security model which requires using @@ -370,6 +368,50 @@ IOMMUFD IOAS/HWPT to enable userspace DMA::
/* Other device operations as stated in "VFIO Usage
Example" */Documentation/driver-api/vfio.rst +3
-0+VFIO NOIOMMU mode +------------------------------------------------------------------------------- +VFIO also supports a no-IOMMU mode, intended for usages whereunsafe DMA can +be performed by userspace drivers w/o physical IOMMU protection. This mode +is controlled by the parameter:
Documentation/driver-api/vfio.rst +4
-0+ +/sys/module/vfio/parameters/enable_unsafe_noiommu_mode + +Upon enabling this mode, with an assigned device, the user will bepresented +with a VFIO group and device file, e.g.::
+
Quoted 9 lines
Documentation/driver-api/vfio.rst IOMMUFD IOAS/HWPT to enable userspace DMA:: · line 386
Quoted 5 lines
Documentation/driver-api/vfio.rst +5
-0+ /dev/vfio/ + |-- devices + | `-- noiommu-vfio0 /* VFIO device cdev */ + |-- noiommu-0 /* VFIO group */ + `-- vfiohow about noiommu-vfioX and noiommu-Y? Just want to deliver the message that the two number is not 100% identical.
good point.
Quoted 14 lines
Quoted 5 lines
+The capabilities vary depending on the device programming interface and kernel +configuration used. The following table summarizes the differences: +
++-------------------+---------------------+----------------------+
+| Feature | VFIO group | VFIO device cdev |Quoted 8 lines
++===================+=====================+======================+
+| VFIO device UAPI | Yes | Yes |
++-------------------+---------------------+----------------------+
+| VFIO container | No | No |
++-------------------+---------------------+----------------------+
+| IOMMUFD IOAS | No | Yes* |
++-------------------+---------------------+----------------------+
+
Quoted 3 lines
not quite the above table. Why 'VFIO container' is "No" for VFIO group?
I meant VFIO group interface has no access to VFIO container UAPIs, such as DMA MAP/UNMAP.
how about I will add the following:
The capabilities vary depending on the device programming interface and kernel
-configuration used. The following table summarizes the differences:
+configuration used. The following table summarizes the differences ("Yes" means +the UAPI is accessible and functional in noiommu mode, "No" means the UAPI is +not supported):
Quoted 18 lines
Quoted 18 lines
+Note that the VFIO container case includes IOMMUFD provided VFIO compatibility +interfaces when either
CONFIG_VFIO_CONTAINERorCONFIG_IOMMUFD_VFIO_CONTAINERis +enabled.(quoted diff) +2
-0+ +* IOMMUFD UAPI is available for VFIO device cdev to pin and mapuser memory with
+ the ability to retrieve physical addresses for DMA command submission. +
+A new IOMMUFD ioctlIOMMU_IOAS_NOIOMMU_GET_PAis added to retrieve the physical +address for a given IOVA. Although there is no physical DMA remapping hardware, +IOMMU_IOAS_MAP_FIXED_IOVA is still used to establish IOVA-to-PA mappings in the +software page table for laterIOMMU_IOAS_NOIOMMU_GET_PAlookups.+tools/testing/selftests/vfio/vfio_iommufd_noiommu_test.cprovides an example of +using this ioctl in no-IOMMU mode. +
VFIO User API
-------------------------------------------------------------------------------
Re: [PATCH v5 9/9] Documentation: Update VFIO NOIOMMU mode
Quoted 78 lines · Jacob Pan
Hi Yi,
Quoted 39 lines · On Wed, 20 May 2026 15:20:30 +0800 Yi Liu <[email protected]> wrote:
Quoted 38 lines · Jacob Pan
Document the NOIOMMU mode with newly added cdev support under iommufd.
Cc Jonathan Corbet <[email protected]>Signed-off-by Jacob Pan <[email protected]>--- Documentation/driver-api/vfio.rst | 46+++++++++++++++++++++++++++++-- 1 file changed, 44 insertions(+), 2 deletions(-)
(quoted diff) +0
-0
b/Documentation/driver-api/vfio.rstindex2a21a42c9386..d97017d80b98100644 ---a/Documentation/driver-api/vfio.rst+++b/Documentation/driver-api/vfio.rst@@ -275,8 +275,6 @@ in a VFIO group. WithCONFIG_VFIO_DEVICE_CDEV=ythe user can now acquire a device fd by directly opening a character device/dev/vfio/devices/vfioXwhere "X" is the number allocated uniquely by VFIO for registered devices. -cdev interface does not support noiommu devices, so user should use -the legacy group interface if noiommu is wanted.
The cdev only works with IOMMUFD. Both VFIO drivers and
applications must adapt to the new cdev security model which requires using @@ -370,6 +368,50 @@ IOMMUFD IOAS/HWPT to enable userspace DMA::/* Other device operations as stated in "VFIO UsageExample" */
Documentation/driver-api/vfio.rst +3
-0+VFIO NOIOMMU mode +------------------------------------------------------------------------------- +VFIO also supports a no-IOMMU mode, intended for usages whereunsafe DMA can +be performed by userspace drivers w/o physical IOMMU protection. This mode +is controlled by the parameter:
Documentation/driver-api/vfio.rst +4
-0+ +/sys/module/vfio/parameters/enable_unsafe_noiommu_mode + +Upon enabling this mode, with an assigned device, the user will bepresented +with a VFIO group and device file, e.g.::
+Quoted 9 lines
Documentation/driver-api/vfio.rst IOMMUFD IOAS/HWPT to enable userspace DMA:: · line 386
Quoted 5 lines
Documentation/driver-api/vfio.rst +5
-0+ /dev/vfio/ + |-- devices + | `-- noiommu-vfio0 /* VFIO device cdev */ + |-- noiommu-0 /* VFIO group */ + `-- vfiohow about noiommu-vfioX and noiommu-Y? Just want to deliver the message that the two number is not 100% identical.
good point.
Quoted 8 lines
Quoted 8 lines
+The capabilities vary depending on the device programming interface and kernel +configuration used. The following table summarizes the differences: +
++-------------------+---------------------+----------------------+
+| Feature | VFIO group | VFIO device cdev |
++===================+=====================+======================+
+| VFIO device UAPI | Yes | Yes |
++-------------------+---------------------+----------------------+Quoted 9 lines
Quoted 5 lines
+| VFIO container | No | No |
++-------------------+---------------------+----------------------+
+| IOMMUFD IOAS | No | Yes* |
++-------------------+---------------------+----------------------+
+not quite the above table. Why 'VFIO container' is "No" for VFIO group?
I meant VFIO group interface has no access to VFIO container UAPIs, such as DMA MAP/UNMAP.
how about I will add the following:
The capabilities vary depending on the device programming interface and kernel
Review
Quoted 4 lines
(quoted diff) +3
-1-configuration used. The following table summarizes the differences: +configuration used. The following table summarizes the differences ("Yes" means +the UAPI is accessible and functional in noiommu mode, "No" means the UAPI is +not supported):
I see. this helps a lot. :)
Quoted 19 lines
Quoted 18 lines
Quoted 18 lines
+Note that the VFIO container case includes IOMMUFD provided VFIO compatibility +interfaces when either
CONFIG_VFIO_CONTAINERorCONFIG_IOMMUFD_VFIO_CONTAINERis +enabled.(quoted diff) +2
-0+ +* IOMMUFD UAPI is available for VFIO device cdev to pin and mapuser memory with
+ the ability to retrieve physical addresses for DMA command submission. +
+A new IOMMUFD ioctlIOMMU_IOAS_NOIOMMU_GET_PAis added to retrieve the physical +address for a given IOVA. Although there is no physical DMA remapping hardware, +IOMMU_IOAS_MAP_FIXED_IOVA is still used to establish IOVA-to-PA mappings in the +software page table for laterIOMMU_IOAS_NOIOMMU_GET_PAlookups.+tools/testing/selftests/vfio/vfio_iommufd_noiommu_test.cprovides an example of +using this ioctl in no-IOMMU mode. +
VFIO User API
-------------------------------------------------------------------------------
Jason Gunthorpe
maintainerQuoted 5 lines · Jacob Pan
VFIO's
unsafe_noiommu_modehas long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices undervfio_compatmode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.
I would like to try to take this for this cycle, how are we doing for a v6?
Alex, did you have any remarks?
Thanks,
Jason
Hi Jason,
Quoted 10 lines · On Tue, 19 May 2026 15:01:56 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 6 lines · Jacob Pan
VFIO's
unsafe_noiommu_modehas long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices undervfio_compatmode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.I would like to try to take this for this cycle, how are we doing for a v6?
I will send out v6 with Baolu's feedback in a couple of days.
Quoted 4 lines
Alex, did you have any remarks?
Thanks,
Jason