iommufd: Enable noiommu mode for cdev
Subsystems (4)
v10 cover iommufd: Enable noiommu mode for cdev
VFIO's unsafe_noiommu_mode has long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices under vfio_compat mode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.
In summary, we have:
|-------------------------+------+---------------| | Device access mode | VFIO | IOMMUFD | |-------------------------+------+---------------| | group /dev/vfio/$GROUP | Yes | Yes | |-------------------------+------+---------------| | cdev /dev/vfio/devices/ | No | This patch | |-------------------------+------+---------------|
Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort to mlock(), which is less robust than pin_user_pages() used for devices backed by a physical IOMMU. For example, mlock() does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on /proc/pagemap tricks or hardcoded values.
By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].
Furthermore, these devices will use the IOMMUFD-based ownership checking model for VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for an iommufd_access object as required in a previous attempt [3].
ChangeLog:
v10:
- Rebased to v7.2-rc2, no intended code change
v9:
- Leave device->device.devt unset for no-IOMMU dev so cdev_device_add() registers only the struct device and does not expose an unsupported cdev. (Alex, Sashiko)
- Clarify VFIO cdev no-IOMMU Kconfig limits in documentation
- Hold registration while checking cdev no-IOMMU access (Sashiko)
- Make no-IOMMU GET_PA length a real upper bound and reject zero length, avoiding an unbounded scan while holding IOAS locks. This matches the bounded-range semantics expected by the incoming iommu_iova_to_phys_length() helper.
- Guard replace path for noiommu device (Sashiko)
v8:
- Guard noiommu for vdevice viommu alloc (Kevin)
v7:
- Handle Sashiko reviews.
- Dropped selftest for now, will submit separately for v7.2 to use new lib helpers
v6: Undo CDEV-GROUP NOIOMMU split, use Kconfig to restrict unwanted
combo.
V5:
- Split CONFIG_VFIO_NOIOMMU into CONFIG_VFIO_GROUP_NOIOMMU and CONFIG_VFIO_CDEV_NOIOMMU so cdev noiommu is independent of VFIO_GROUP (Alex)
- Add CAP_SYS_RAWIO check for cdev open and bind under noiommu, security parity with group noiommu (Alex)
- Add IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) guard in iommufd_device_is_noiommu() to prevent noiommu bind when feature is disabled
- Add prep patch to tolerate NULL group for cdev noiommu devices when CONFIG_VFIO_GROUP_NOIOMMU is not set [7/9]
- Rename IOCTL to IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA to be more specific (Kevin)
- Simplify iommufd_device_is_noiommu, use iommufd_bind_noiommu helper (Kevin, Yi)
- Move IOMMU cap check under iommufd_bind_iommu() (Yi)
- Fix next_iova exceeding iopt_area_last_iova in GET_PA (Alex)
- Fix const hwpt, copyright date, typo in moved comment (Kevin)
- Add Reviewed-by tags
- Squash noiommu cdev selftest fix into selftest patch
- Drop DSA selftest patch
- Details in each patch changelog.
V4:
- Fix various corner cases pointed out by (Sashiko) Details in each patch changelog.
V3:
- Improve error handling [3/10] (Mostafa)
- Simplify vfio_device_is_noiommu logic and merged in [6/10] (Mostafa)
- Add comment to explain the design difference over the legacy noiommu
VFIO code.[1/10]
V2:
- Fix build dependency by adding IOMMU_SUPPORT in [8/11]
- Add an optimization to scan beyond the first page for a contiguous physical address range and return its length instead of a single page.[4/11]
Since RFC[4]:
- Abandoned dummy iommu driver approach as patch 1-3 absorbed the changes into iommufd.
[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-pci/[email protected]/
[3] https://lore.kernel.org/kvm/[email protected]/
[4] https://lore.kernel.org/linux-iommu/[email protected]/
Jacob Pan (3):
iommufd: Add an ioctl to query PA from IOVA for noiommu mode
vfio: Enable cdev noiommu mode under iommufd
Documentation: Update VFIO NOIOMMU mode
Jason Gunthorpe (3):
iommufd: Support a HWPT without an iommu driver for noiommu
iommufd: Move igroup allocation to a function
iommufd: Allow binding to a noiommu device
Documentation/driver-api/vfio.rst | 89 ++++++++++- drivers/iommu/iommufd/Kconfig | 12 ++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/device.c | 201 +++++++++++++++++------- drivers/iommu/iommufd/hw_pagetable.c | 19 ++- drivers/iommu/iommufd/hwpt_noiommu.c | 105 +++++++++++++ drivers/iommu/iommufd/io_pagetable.c | 78 +++++++++ drivers/iommu/iommufd/ioas.c | 36 +++++ drivers/iommu/iommufd/iommufd_private.h | 30 ++++ drivers/iommu/iommufd/main.c | 4 + drivers/iommu/iommufd/viommu.c | 14 +- drivers/vfio/Kconfig | 7 +- drivers/vfio/device_cdev.c | 9 ++ drivers/vfio/iommufd.c | 12 +- drivers/vfio/vfio.h | 23 ++- drivers/vfio/vfio_main.c | 26 ++- include/linux/vfio.h | 1 + include/uapi/linux/iommufd.h | 28 ++++ 18 files changed, 609 insertions(+), 86 deletions(-) create mode 100644 drivers/iommu/iommufd/hwpt_noiommu.c
Signature · 1 line
2.43.0
PATCH v10 1/6 iommufd: Support a HWPT without an iommu driver for noiommu
From: Jason Gunthorpe <[email protected]>
Create just a little part of a real iommu driver, enough to slot in under the dev_iommu_ops() and allow iommufd to call domain_alloc_paging_flags() and fail everything else.
This allows explicitly creating a HWPT under an IOAS.
A new Kconfig option IOMMUFD_NOIOMMU is introduced to differentiate from the VFIO group/container based noiommu mode.
---
V8:
- Guard vIOMMU and vDevice allocation paths for noiommu (Sashiko)
v7:
- Drain no-IOMMU generic-PT freelist (Sashiko)
- Import generic-PT IOMMU namespace (Sashiko)
v6: (Yi)
- Sort includes alphabetically (iommu.h after generic_pt/iommu.h)
- Fix comment: s/mock page table/SW-only page table/ to avoid confusion with selftest mock
- Rewrite noiommu_amdv1_ops comment: explain why AMDV1 format is chosen (multi-page size options), remove references to group-container mode distinction
v5:
- Use the new IOMMUFD_NOIOMMU Kconfig instead of VFIO_NOIOMMU
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Copyright date fix (Kevin)
v4:
- Make iommufd_noiommu_ops const
v3:
- Add comment to explain the design difference over the
legacy noiommu VFIO code.
--- drivers/iommu/iommufd/Kconfig | 12 +++ drivers/iommu/iommufd/Makefile | 1 + drivers/iommu/iommufd/hw_pagetable.c | 19 ++++- drivers/iommu/iommufd/hwpt_noiommu.c | 105 ++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 12 +++ drivers/iommu/iommufd/main.c | 1 + drivers/iommu/iommufd/viommu.c | 14 +++- 7 files changed, 158 insertions(+), 6 deletions(-) create mode 100644 drivers/iommu/iommufd/hwpt_noiommu.c
drivers/iommu/iommufd/Kconfig +12 -0
@@ -16,6 +16,18 @@ config IOMMUFD If you don't know what to do here, say N. if IOMMUFD +config IOMMUFD_NOIOMMU + bool + depends on !GENERIC_ATOMIC64 # IOMMU_PT_AMDV1 requires cmpxchg64 + select GENERIC_PT + select IOMMU_PT + select IOMMU_PT_AMDV1 + help + Provides a SW-only IO page table for devices without hardware + IOMMU backing. This uses the AMDV1 page table format for + IOVA-to-PA lookups only, not for hardware DMA translation. + To be selected by VFIO_NOIOMMU when VFIO_DEVICE_CDEV is enabled. + config IOMMUFD_VFIO_CONTAINER bool "IOMMUFD provides the VFIO container /dev/vfio/vfio" depends on VFIO_GROUP && !VFIO_CONTAINER
drivers/iommu/iommufd/Makefile +1 -0
@@ -10,6 +10,7 @@ iommufd-y := \ vfio_compat.o \ viommu.o +iommufd-$(CONFIG_IOMMUFD_NOIOMMU) += hwpt_noiommu.o iommufd-$(CONFIG_IOMMUFD_TEST) += selftest.o obj-$(CONFIG_IOMMUFD) += iommufd.o
drivers/iommu/iommufd/hw_pagetable.c +16 -3
@@ -8,6 +8,15 @@#include "../iommu-priv.h" #include "iommufd_private.h" +static const struct iommu_ops *get_iommu_ops (struct iommufd_device *idev) +{ +if (IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->igroup->group) +return &iommufd_noiommu_ops; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return dev_iommu_ops (idev->dev); +} +static void __iommufd_hwpt_destroy (struct iommufd_hw_pagetable *hwpt) {if (hwpt->domain)
@@ -114,11 +123,13 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas,IOMMU_HWPT_ALLOC_DIRTY_TRACKING |IOMMU_HWPT_FAULT_ID_VALID |IOMMU_HWPT_ALLOC_PASID ; -const struct iommu_ops *ops =dev_iommu_ops (idev->dev); +const struct iommu_ops *ops =get_iommu_ops (idev);struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_hw_pagetable *hwpt;int rc; +if (!ops) +return ERR_PTR (-ENODEV );lockdep_assert_held (&ioas->mutex);if ((flags || user_data) && !ops->domain_alloc_paging_flags)
@@ -229,7 +240,7 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx, struct iommufd_device *idev, u32 flags, const struct iommu_user_data *user_data) { - const struct iommu_ops *ops = dev_iommu_ops(idev->dev); + const struct iommu_ops *ops = get_iommu_ops(idev);struct iommufd_hwpt_nested *hwpt_nested;struct iommufd_hw_pagetable *hwpt;int rc;
@@ -389,10 +400,12 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) hwpt = &hwpt_nested->common; }else if (pt_obj->type ==IOMMUFD_OBJ_VIOMMU ) {struct iommufd_hwpt_nested *hwpt_nested; +struct iommu_device *iommu_dev;struct iommufd_viommu *viommu; viommu = container_of(pt_obj, struct iommufd_viommu, obj); -if (viommu->iommu_dev !=__iommu_get_iommu_dev (idev->dev)) { + iommu_dev =iommufd_device_get_iommu_dev (idev); +if (!iommu_dev || viommu->iommu_dev != iommu_dev) { rc = -EINVAL ; goto out_unlock; }
drivers/iommu/iommufd/hwpt_noiommu.c new file +105 -0
105 lines not on this page · raw message
drivers/iommu/iommufd/iommufd_private.h +12 -0
@@ -464,6 +464,8 @@ static inline void iommufd_hw_pagetable_put(struct iommufd_ctx *ictx,refcount_dec (&hwpt->obj.users); } +extern const struct iommu_ops iommufd_noiommu_ops; +struct iommufd_attach ;struct iommufd_group {
@@ -501,6 +503,16 @@ iommufd_get_device(struct iommufd_ucmd *ucmd, u32 id) struct iommufd_device, obj); } +static inline struct iommu_device * +iommufd_device_get_iommu_dev (struct iommufd_device *idev) +{ +if (IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->igroup->group) +return NULL ; +if (WARN_ON_ONCE (!idev->dev->iommu)) +return NULL ; +return __iommu_get_iommu_dev (idev->dev); +} +void iommufd_device_pre_destroy (struct iommufd_object *obj);void iommufd_device_destroy (struct iommufd_object *obj);int iommufd_get_hw_info (struct iommufd_ucmd *ucmd);
drivers/iommu/iommufd/main.c +1 -0
@@ -804,5 +804,6 @@ MODULE_ALIAS("devname:vfio/vfio");MODULE_IMPORT_NS ("IOMMUFD_INTERNAL" );MODULE_IMPORT_NS ("IOMMUFD" );MODULE_IMPORT_NS ("DMA_BUF" ); +MODULE_IMPORT_NS ("GENERIC_PT_IOMMU" );MODULE_DESCRIPTION ("I/O Address Space Management for passthrough devices" );MODULE_LICENSE ("GPL" );
drivers/iommu/iommufd/viommu.c +11 -3
@@ -25,6 +25,7 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *ucmd)struct iommufd_hwpt_paging *hwpt_paging;struct iommufd_viommu *viommu;struct iommufd_device *idev; +struct iommu_device *iommu_dev;const struct iommu_ops *ops;size_t viommu_size;int rc;
@@ -36,7 +37,12 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *ucmd)if (IS_ERR (idev))return PTR_ERR (idev); - ops =dev_iommu_ops (idev->dev); + iommu_dev =iommufd_device_get_iommu_dev (idev); +if (!iommu_dev) { + rc = -EOPNOTSUPP ; + goto out_put_idev; + } + ops = iommu_dev->ops;if (!ops->get_viommu_size || !ops->viommu_init) { rc = -EOPNOTSUPP ; goto out_put_idev;
@@ -87,7 +93,7 @@ int iommufd_viommu_alloc_ioctl(struct iommufd_ucmd *ucmd)* pluggable IOMMU instance (if exists) is responsible for refcounting * on its own. */ - viommu->iommu_dev =__iommu_get_iommu_dev (idev->dev); + viommu->iommu_dev = iommu_dev; rc = ops->viommu_init (viommu, hwpt_paging->common.domain, user_data.len ? &user_data :NULL );
@@ -146,6 +152,7 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd)struct iommufd_vdevice *vdev, *curr;size_t vdev_size =sizeof (*vdev);struct iommufd_viommu *viommu; +struct iommu_device *iommu_dev;struct iommufd_device *idev;u64 virt_id = cmd->virt_id;int rc =0 ;
@@ -164,7 +171,8 @@ int iommufd_vdevice_alloc_ioctl(struct iommufd_ucmd *ucmd) goto out_put_viommu; } -if (viommu->iommu_dev !=__iommu_get_iommu_dev (idev->dev)) { + iommu_dev =iommufd_device_get_iommu_dev (idev); +if (!iommu_dev || viommu->iommu_dev != iommu_dev) { rc = -EINVAL ; goto out_put_idev; }
Signature · 1 line
2.43.0
PATCH v10 2/6 iommufd: Move igroup allocation to a function
In reply to Jacob PanFrom: Jason Gunthorpe <[email protected]>
So it can be reused in the next patch which allows binding to noiommu device.
---
v9:
- Use kzalloc_obj() for igroup allocation
v5:
- Add NULL group to the error handling path of iommufd_group_setup_msi()
--- drivers/iommu/iommufd/device.c | 43 +++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 16 deletions(-)
drivers/iommu/iommufd/device.c +27 -16
@@ -56,6 +56,30 @@ static bool iommufd_group_try_get(struct iommufd_group *igroup,return kref_get_unless_zero (&igroup->ref); } +static struct iommufd_group *iommufd_alloc_group (struct iommufd_ctx *ictx, +struct iommu_group *group) +{ +struct iommufd_group *new_igroup; + + new_igroup =kzalloc_obj (*new_igroup,GFP_KERNEL ); +if (!new_igroup) +return ERR_PTR (-ENOMEM ); + +kref_init (&new_igroup->ref); +mutex_init (&new_igroup->lock); +xa_init (&new_igroup->pasid_attach); + new_igroup->sw_msi_start =PHYS_ADDR_MAX ; +/* group reference moves into new_igroup */ + new_igroup->group = group; + +/* +* The ictx is not additionally refcounted here because all objects using +* an igroup must put it before their destroy completes. +*/ + new_igroup->ictx = ictx; +return new_igroup; +} +/* * iommufd needs to store some more data for each iommu_group, we keep a * parallel xarray indexed by iommu_group id to hold this instead of putting it
@@ -87,25 +111,12 @@ static struct iommufd_group *iommufd_get_group(struct iommufd_ctx *ictx, }xa_unlock (&ictx->groups); - new_igroup =kzalloc_obj (*new_igroup); -if (!new_igroup) { + new_igroup =iommufd_alloc_group (ictx, group); +if (IS_ERR (new_igroup)) {iommu_group_put (group); -return ERR_PTR (-ENOMEM ); +return new_igroup; } -kref_init (&new_igroup->ref); -mutex_init (&new_igroup->lock); -xa_init (&new_igroup->pasid_attach); - new_igroup->sw_msi_start =PHYS_ADDR_MAX ; -/* group reference moves into new_igroup */ - new_igroup->group = group; - -/* -* The ictx is not additionally refcounted here becase all objects using -* an igroup must put it before their destroy completes. -*/ - new_igroup->ictx = ictx; -/* * We dropped the lock so igroup is invalid. NULL is a safe and likely * value to assume for the xa_cmpxchg algorithm.
Signature · 1 line
2.43.0
Re: [PATCH v10 2/6] iommufd: Move igroup allocation to a function
Quoted 5 lines · Jacob Pan
From: Jason Gunthorpe <[email protected]>
So it can be reused in the next patch which allows binding to noiommu device.
Quoted 7 lines
Reviewed-by Samiullah Khawaja <[email protected]>Reviewed-by Yi Liu <[email protected]>Reviewed-by Kevin Tian <[email protected]>Reviewed-by Lu Baolu <[email protected]>Signed-off-by Jason Gunthorpe <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
Thanks,
Praan
PATCH v10 3/6 iommufd: Allow binding to a noiommu device
In reply to Jacob PanFrom: Jason Gunthorpe <[email protected]>
Allow iommufd to bind devices without an IOMMU (noiommu mode) by creating a dummy igroup for such devices and skipping hwpt operations.
This enables noiommu devices to operate through the same iommufd API as IOMMU- capable devices.
---
v9:
- Guard replace path for noiommu device (Sashiko)
v7:
- Block get hw info for noiommu
v6:
- Expand iommufd_device_is_noiommu() comment to explain why dev->iommu is checked instead of device_iommu_mapped() (Yi & Baolu)
- Simplify bind error handling by factoring out duplicated rc check (Yi)
v5:
- simplify logic and rename iommufd_device_is_noiommu (Kevin, Yi)
- use a helper iommufd_bind_noiommu instead of open coding (Kevin)
- move IOMMU cap check under iommufd_bind_iommu() (Yi)
- reword comments for partial init (Yi)
- misc minor clean up
v4:
- Update the description of the module parameter (Alex)
v3:
- Consolidate into fewer patches
--- drivers/iommu/iommufd/device.c | 158 ++++++++++++++++++++++++--------- 1 file changed, 118 insertions(+), 40 deletions(-)
drivers/iommu/iommufd/device.c +118 -40
@@ -23,6 +23,19 @@ struct iommufd_attach {struct xarray device_array; }; +/* +* Detect a noiommu device for the cdev path. We check dev->iommu rather than +* using device_iommu_mapped() (which checks dev->iommu_group) because when +* both group and cdev interfaces coexist, the group path assigns a fake +* noiommu iommu_group to the device. That would cause device_iommu_mapped() +* to return true and hide the noiommu case from the cdev path. dev->iommu is +* reliably NULL when no IOMMU driver is managing the device. +*/ +static bool iommufd_device_is_noiommu (struct iommufd_device *idev) +{ +return IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && !idev->dev->iommu; +} +static void iommufd_group_release (struct kref *kref) { struct iommufd_group *igroup =
@@ -30,9 +43,11 @@ static void iommufd_group_release(struct kref *kref)WARN_ON (!xa_empty (&igroup->pasid_attach)); -xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), igroup, -NULL ,GFP_KERNEL ); -iommu_group_put (igroup->group); +if (igroup->group) { +xa_cmpxchg (&igroup->ictx->groups,iommu_group_id (igroup->group), + igroup,NULL ,GFP_KERNEL ); +iommu_group_put (igroup->group); + }mutex_destroy (&igroup->lock);kfree (igroup); }
@@ -204,32 +219,20 @@ void iommufd_device_destroy(struct iommufd_object *obj)struct iommufd_device *idev = container_of(obj, struct iommufd_device, obj); -iommu_device_release_dma_owner (idev->dev); +/* igroup is NULL when destroy called during bind error cleanup */ +if (!idev->igroup) +return ; +if (!iommufd_device_is_noiommu (idev)) +iommu_device_release_dma_owner (idev->dev);iommufd_put_group (idev->igroup);if (!iommufd_selftest_is_mock_dev (idev->dev))iommufd_ctx_put (idev->ictx); } -/** -* iommufd_device_bind - Bind a physical device to an iommu fd -* @ictx: iommufd file descriptor -* @dev: Pointer to a physical device struct -* @id: Output ID number to return to userspace for this device -* -* A successful bind establishes an ownership over the device and returns -* struct iommufd_device pointer, otherwise returns error pointer. -* -* A driver using this API must set driver_managed_dma and must not touch -* the device until this routine succeeds and establishes ownership. -* -* Binding a PCI device places the entire RID under iommufd control. -* -* The caller must undo this with iommufd_device_unbind() -*/ -struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, -struct device *dev,u32 *id) +static int iommufd_bind_iommu (struct iommufd_device *idev) { -struct iommufd_device *idev; +struct iommufd_ctx *ictx = idev->ictx; +struct device *dev = idev->dev;struct iommufd_group *igroup;int rc;
@@ -238,11 +241,11 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,* to restore cache coherency. */ if (!device_iommu_capable (dev,IOMMU_CAP_CACHE_COHERENCY )) -return ERR_PTR (-EINVAL ); +return -EINVAL ; igroup =iommufd_get_group (ictx, dev);if (IS_ERR (igroup)) -return ERR_CAST (igroup); +return PTR_ERR (igroup);/* * For historical compat with VFIO the insecure interrupt path is
@@ -268,21 +271,77 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx,if (rc) goto out_group_put; +/* igroup refcount moves into iommufd_device */ + idev->igroup = igroup; + idev->enforce_cache_coherency = +device_iommu_capable (dev,IOMMU_CAP_ENFORCE_CACHE_COHERENCY ); +return 0 ; + +out_group_put: +iommufd_put_group (igroup); +return rc; +} + +/* +* Noiommu devices have no real IOMMU group. Create a dummy igroup so that +* internal code paths that expect idev->igroup to be present still work. +* A NULL igroup->group distinguishes this from a real IOMMU-backed group. +*/ +static int iommufd_bind_noiommu (struct iommufd_device *idev) +{ +struct iommufd_group *igroup; + + igroup =iommufd_alloc_group (idev->ictx,NULL ); +if (IS_ERR (igroup)) +return PTR_ERR (igroup); + idev->igroup = igroup; +return 0 ; +} + +/** +* iommufd_device_bind - Bind a physical device to an iommu fd +* @ictx: iommufd file descriptor +* @dev: Pointer to a physical device struct +* @id: Output ID number to return to userspace for this device +* +* A successful bind establishes an ownership over the device and returns +* struct iommufd_device pointer, otherwise returns error pointer. +* +* A driver using this API must set driver_managed_dma and must not touch +* the device until this routine succeeds and establishes ownership. +* +* Binding a PCI device places the entire RID under iommufd control. +* +* The caller must undo this with iommufd_device_unbind() +*/ +struct iommufd_device *iommufd_device_bind (struct iommufd_ctx *ictx, +struct device *dev,u32 *id) +{ +struct iommufd_device *idev; +int rc; + idev =iommufd_object_alloc (ictx, idev,IOMMUFD_OBJ_DEVICE ); -if (IS_ERR (idev)) { - rc =PTR_ERR (idev); - goto out_release_owner; - } +if (IS_ERR (idev)) +return idev; + idev->ictx = ictx; + idev->dev = dev; + +if (!iommufd_device_is_noiommu (idev)) + rc =iommufd_bind_iommu (idev); +else + rc =iommufd_bind_noiommu (idev); +if (rc) + goto err_out; + +/* +* Take a ctx reference after bind succeeds. This must happen here +* so that iommufd_device_destroy() can handle partial initialization +*/ if (!iommufd_selftest_is_mock_dev (dev))iommufd_ctx_get (ictx); - idev->dev = dev; - idev->enforce_cache_coherency = -device_iommu_capable (dev,IOMMU_CAP_ENFORCE_CACHE_COHERENCY );/* The calling driver is a user until iommufd_device_unbind() */ refcount_inc (&idev->obj.users); -/* igroup refcount moves into iommufd_device */ - idev->igroup = igroup;/* * If the caller fails after this success it must call
@@ -294,11 +353,14 @@ struct iommufd_device *iommufd_device_bind(struct iommufd_ctx *ictx, *id = idev->obj.id;return idev; -out_release_owner: -iommu_device_release_dma_owner (dev); -out_group_put: -iommufd_put_group (igroup); +err_out: +/* +* iommufd_device_destroy() handles partially initialized idev, +* so iommufd_object_abort_and_destroy() is safe to call here. +*/ +iommufd_object_abort_and_destroy (ictx, &idev->obj);return ERR_PTR (rc); + }EXPORT_SYMBOL_NS_GPL (iommufd_device_bind,"IOMMUFD" );
@@ -512,6 +574,9 @@ static int iommufd_hwpt_attach_device(struct iommufd_hw_pagetable *hwpt,struct iommufd_attach_handle *handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;
@@ -559,6 +624,9 @@ static void iommufd_hwpt_detach_device(struct iommufd_hw_pagetable *hwpt, {struct iommufd_attach_handle *handle; +if (iommufd_device_is_noiommu (idev)) +return ; + handle =iommufd_device_get_attach_handle (idev, pasid);if (pasid ==IOMMU_NO_PASID )iommu_detach_group_handle (hwpt->domain, idev->igroup->group);
@@ -577,6 +645,9 @@ static int iommufd_hwpt_replace_device(struct iommufd_device *idev,struct iommufd_attach_handle *handle, *old_handle;int rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; +if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;
@@ -614,7 +685,8 @@ int iommufd_hw_pagetable_attach(struct iommufd_hw_pagetable *hwpt, struct iommufd_device *idev, ioasid_t pasid) {struct iommufd_hwpt_paging *hwpt_paging =find_hwpt_paging (hwpt); -bool attach_resv = hwpt_paging && pasid ==IOMMU_NO_PASID ; +bool attach_resv = hwpt_paging && pasid ==IOMMU_NO_PASID && + !iommufd_device_is_noiommu (idev);struct iommufd_group *igroup = idev->igroup;struct iommufd_hw_pagetable *old_hwpt;struct iommufd_attach *attach;
@@ -793,7 +865,8 @@ iommufd_device_do_replace(struct iommufd_device *idev, ioasid_t pasid, struct iommufd_hw_pagetable *hwpt) {struct iommufd_hwpt_paging *hwpt_paging =find_hwpt_paging (hwpt); -bool attach_resv = hwpt_paging && pasid ==IOMMU_NO_PASID ; +bool attach_resv = hwpt_paging && pasid ==IOMMU_NO_PASID && + !iommufd_device_is_noiommu (idev);struct iommufd_hwpt_paging *old_hwpt_paging;struct iommufd_group *igroup = idev->igroup;struct iommufd_hw_pagetable *old_hwpt;
@@ -1585,6 +1658,11 @@ int iommufd_get_hw_info(struct iommufd_ucmd *ucmd)if (IS_ERR (idev))return PTR_ERR (idev); +if (iommufd_device_is_noiommu (idev)) { + rc = -EOPNOTSUPP ; + goto out_put; + } + ops =dev_iommu_ops (idev->dev);if (ops->hw_info) { data = ops->hw_info (idev->dev, &data_len, &cmd->out_data_type);
Signature · 1 line
2.43.0
PATCH v10 4/6 iommufd: Add an ioctl to query PA from IOVA for noiommu mode
In reply to Jacob PanTo support no-IOMMU mode where userspace drivers perform unsafe DMA using physical addresses, introduce a new API to retrieve the physical address of a user-allocated DMA buffer that has been mapped to an IOVA via IOMMU_IOAS_MAP. The mapping is backed by SW-only I/O page tables maintained by the GENERIC_PT framework.
---
v10
-Rebased to v7.2-rc2, no intended code change.
v9
- Make no-IOMMU GET_PA length a real upper bound and reject zero length, avoiding an unbounded scan while holding IOAS locks. This matches the bounded-range semantics expected by the incoming iommu_iova_to_phys_length() helper.
v8:
- Fix comment on start IOVA range (Kevin)
v7:
- Fix commit message (Yi)
- Avoid duplicated tmp_length settting (yi)
- Handle race with dma-buf revoke pages (Sashiko)
v6:
- Limit search length (Baolu, Jason)
v5:
- Fix next_iova exceeds iopt_area_last_iova (Alex)
- Rename IOCTL more specific to NOIOMMU, i.e. IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA (Kevin)
- Add header stubs for iopt_get_phys()
v4:
- Fix ioctl return type (Yi Liu)
--- drivers/iommu/iommufd/io_pagetable.c | 78 +++++++++++++++++++++++++ drivers/iommu/iommufd/ioas.c | 36 ++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 18 ++++++ drivers/iommu/iommufd/main.c | 3 + include/uapi/linux/iommufd.h | 28 +++++++++ 5 files changed, 163 insertions(+)
drivers/iommu/iommufd/io_pagetable.c +78 -0
@@ -859,6 +859,84 @@ int iopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova,return iopt_unmap_iova_range (iopt, iova, iova_last, unmapped); } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length) +{ +struct iopt_area *area; +struct iopt_pages *pages; +u64 max_length = *length; +u64 tmp_length =0 ; +u64 tmp_paddr =0 ; +int rc =0 ; + +if (!max_length) +return -EINVAL ; + +down_read (&iopt->iova_rwsem); + area =iopt_area_iter_first (iopt, iova, iova); +if (!area || !area->pages) { + rc = -ENOENT ; + goto unlock_exit; + } + + pages = area->pages; +mutex_lock (&pages->mutex); +if (iopt_dmabuf_revoked (pages)) { + rc = -EINVAL ; + goto unlock_pages; + } + +if (!area->storage_domain || + area->storage_domain->owner != &iommufd_noiommu_ops) { + rc = -EOPNOTSUPP ; + goto unlock_pages; + } + + *paddr =iommu_iova_to_phys (area->storage_domain, iova); + tmp_length =min_t (u64,PAGE_SIZE -offset_in_page (iova), +iopt_area_last_iova (area) - iova +1 ); + tmp_paddr = *paddr; +/* +* Return the physically contiguous length, capped by the caller +* supplied range length. +*/ +while (iova <iopt_area_last_iova (area)) { +unsigned long next_iova; +u64 next_paddr; + +if (tmp_length >= max_length) +break ; + +if (check_add_overflow (iova,PAGE_SIZE , &next_iova)) +break ; + +if (next_iova >iopt_area_last_iova (area)) +break ; + + next_paddr =iommu_iova_to_phys (area->storage_domain, next_iova); + +if (!next_paddr || next_paddr != tmp_paddr +PAGE_SIZE ) +break ; + + iova = next_iova; + tmp_paddr +=PAGE_SIZE ; + tmp_length +=PAGE_SIZE ; + } + +if (tmp_length > max_length) + tmp_length = max_length; + *length = tmp_length; + +unlock_pages: +mutex_unlock (&pages->mutex); +unlock_exit: +up_read (&iopt->iova_rwsem); + +return rc; +} +#endif +int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped) {/* If the IOVAs are empty then unmap all succeeds */
drivers/iommu/iommufd/ioas.c +36 -0
@@ -375,6 +375,42 @@ int iommufd_ioas_unmap(struct iommufd_ucmd *ucmd)return rc; } +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +struct iommu_ioas_noiommu_get_pa *cmd = ucmd->cmd; +struct iommufd_ioas *ioas; +int rc; + +if (!capable (CAP_SYS_RAWIO )) +return -EPERM ; + +if (cmd->flags || cmd->__reserved) +return -EOPNOTSUPP ; + +if (!cmd->length) +return -EINVAL ; + +if (cmd->iova >=ULONG_MAX ) +return -EOVERFLOW ; + + ioas =iommufd_get_ioas (ucmd->ictx, cmd->ioas_id); +if (IS_ERR (ioas)) +return PTR_ERR (ioas); + + rc =iopt_get_phys (&ioas->iopt, cmd->iova, &cmd->out_phys, + &cmd->length); +if (rc) + goto out_put; + + rc =iommufd_ucmd_respond (ucmd,sizeof (*cmd)); +out_put: +iommufd_put_object (ucmd->ictx, &ioas->obj); + +return rc; +} +#endif +static void iommufd_release_all_iova_rwsem (struct iommufd_ctx *ictx,struct xarray *ioas_list) {
drivers/iommu/iommufd/iommufd_private.h +18 -0
@@ -118,6 +118,16 @@ int iopt_map_pages(struct io_pagetable *iopt, struct list_head *pages_list,int iopt_unmap_iova (struct io_pagetable *iopt,unsigned long iova,unsigned long length,unsigned long *unmapped);int iopt_unmap_all (struct io_pagetable *iopt,unsigned long *unmapped); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova,u64 *paddr, +u64 *length); +#else +static inline int iopt_get_phys (struct io_pagetable *iopt,unsigned long iova, +u64 *paddr,u64 *length) +{ +return -EOPNOTSUPP ; +} +#endif int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain,
@@ -346,6 +356,14 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd);int iommufd_ioas_change_process (struct iommufd_ucmd *ucmd);int iommufd_ioas_copy (struct iommufd_ucmd *ucmd);int iommufd_ioas_unmap (struct iommufd_ucmd *ucmd); +#ifdef CONFIG_IOMMUFD_NOIOMMU +int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd); +#else +static inline int iommufd_ioas_noiommu_get_pa (struct iommufd_ucmd *ucmd) +{ +return -EOPNOTSUPP ; +} +#endif int iommufd_ioas_option (struct iommufd_ucmd *ucmd);int iommufd_option_rlimit_mode (struct iommu_option *cmd,struct iommufd_ctx *ictx);
drivers/iommu/iommufd/main.c +3 -0
@@ -424,6 +424,7 @@ union ucmd_buffer {struct iommu_ioas_alloc alloc;struct iommu_ioas_allow_iovas allow_iovas;struct iommu_ioas_copy ioas_copy; +struct iommu_ioas_noiommu_get_pa noiommu_get_pa;struct iommu_ioas_iova_ranges iova_ranges;struct iommu_ioas_map map;struct iommu_ioas_unmap unmap;
@@ -482,6 +483,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { IOCTL_OP(IOMMU_IOAS_MAP, iommufd_ioas_map, struct iommu_ioas_map, iova),IOCTL_OP (IOMMU_IOAS_MAP_FILE , iommufd_ioas_map_file, struct iommu_ioas_map_file, iova), + IOCTL_OP(IOMMU_IOAS_NOIOMMU_GET_PA, iommufd_ioas_noiommu_get_pa, struct iommu_ioas_noiommu_get_pa, + out_phys), IOCTL_OP(IOMMU_IOAS_UNMAP, iommufd_ioas_unmap, struct iommu_ioas_unmap, length), IOCTL_OP(IOMMU_OPTION, iommufd_option, struct iommu_option, val64),
include/uapi/linux/iommufd.h +28 -0
@@ -57,6 +57,7 @@ enum {IOMMUFD_CMD_IOAS_CHANGE_PROCESS =0x92 ,IOMMUFD_CMD_VEVENTQ_ALLOC =0x93 ,IOMMUFD_CMD_HW_QUEUE_ALLOC =0x94 , + IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA = 0x95, };/**
@@ -219,6 +220,33 @@ struct iommu_ioas_map { };#define IOMMU_IOAS_MAP _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_MAP) +/** +* struct iommu_ioas_noiommu_get_pa - ioctl(IOMMU_IOAS_NOIOMMU_GET_PA) +* @size: sizeof(struct iommu_ioas_noiommu_get_pa) +* @flags: Reserved, must be 0 for now +* @ioas_id: IOAS ID to query IOVA to PA mapping from +* @__reserved: Must be 0 +* @iova: IOVA to query +* @length: On input, non-zero maximum number of bytes to query starting from +* @iova. On output, number of physically contiguous bytes starting +* from @out_phys, capped by the input length. +* @out_phys: Output physical address the IOVA maps to +* +* Query the physical address backing an IOVA range. The beginning of the +* range must be mapped already and length must be non-zero. For noiommu +* devices doing unsafe DMA only. +*/ +struct iommu_ioas_noiommu_get_pa { +__u32 size; +__u32 flags; +__u32 ioas_id; +__u32 __reserved; +__aligned_u64 iova; +__aligned_u64 length; +__aligned_u64 out_phys; +}; +#define IOMMU_IOAS_NOIOMMU_GET_PA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA) +/** * struct iommu_ioas_map_file - ioctl(IOMMU_IOAS_MAP_FILE) * @size: sizeof(struct iommu_ioas_map_file)
Signature · 1 line
2.43.0
PATCH v10 5/6 vfio: Enable cdev noiommu mode under iommufd
In reply to Jacob PanNow that devices under noiommu mode can bind with IOMMUFD and perform IOAS operations, lift restrictions on cdev from VFIO side. Use cases are documented in Documentation/driver-api/vfio.rst
---
v10:
- Rebased to v7.2-rc2, no intended code change
v9:
- Leave device->device.devt unset for no-IOMMU dev so cdev_device_add() registers only the struct device and does not expose an unsupported cdev. (Alex, Sashiko)
- Hold registration while checking cdev no-IOMMU access (Sashiko)
v8:
- Fix warning message (Kevin)
v7:
- Avoid treating emulated device as noiommu device (Sashiko)
- Keep platforms w/ GENERIC_ATOMIC64 to use VFIO group noiommu as before (Sashiko)
- Restore order of group & cdev init for noiommu (Yi)
- Consolidate noiommu helper for cdev & group (Yi)
v6:
- Revert back to unified VFIO_NOIOMMU Kconfig for both cdev and group. Use Kconfig dependency to restrict usages and avoid null group checks. (Alex & Yi)
- Add CAP_SYS_RAWIO checks for cdev open to maintain security parity with the group noiommu path. (Alex)
v5:
- Add Kconfig VFIO_CDEV_NOIOMMU to select IOMMUFD_NOIOMMU and its dependencies
- Add comment to explain vfio_noiommu conditional definition (Alex)
- Removed early return for group noiommu in bind/unbind
- Use consistent wording referring to VFIO noiommu mode (Kevin)
- Update unsafe_noiommu Kconfig help text (Kevin)
- Change dev_warn to dev_info for noiommu enabling msg (Kevin)
v4:
- Remove early return in iommufd_bind for noiommu (Alex)
v3:
- Consolidate into fewer patches
v2:
- removed unnecessary device->noiommu set in iommufd_vfio_compat_ioas_get_id()
--- drivers/vfio/Kconfig | 7 ++++--- drivers/vfio/device_cdev.c | 9 +++++++++ drivers/vfio/iommufd.c | 12 ++++++++---- drivers/vfio/vfio.h | 23 +++++++++-------------- drivers/vfio/vfio_main.c | 26 +++++++++++++++++++++++++- include/linux/vfio.h | 1 + 6 files changed, 56 insertions(+), 22 deletions(-)
drivers/vfio/Kconfig +4 -3
@@ -22,8 +22,7 @@ config VFIO_DEVICE_CDEV The VFIO device cdev is another way for userspace to get device access. Userspace gets device fd by opening device cdev under /dev/vfio/devices/vfioX, and then bind the device fd with an iommufd - to set up secure DMA context for device access. This interface does - not support noiommu. + to set up secure DMA context for device access. If you don't know what to do here, say N.
@@ -62,7 +61,9 @@ endif config VFIO_NOIOMMU bool "VFIO No-IOMMU support" - depends on VFIO_GROUP + depends on VFIO_GROUP || (VFIO_DEVICE_CDEV && !GENERIC_ATOMIC64) + depends on !VFIO_GROUP || VFIO_CONTAINER || IOMMUFD_VFIO_CONTAINER + select IOMMUFD_NOIOMMU if VFIO_DEVICE_CDEV && !GENERIC_ATOMIC64 help VFIO is built on the ability to isolate devices using the IOMMU. Only with an IOMMU can userspace access to DMA capable devices be
drivers/vfio/device_cdev.c +9 -0
@@ -11,6 +11,10 @@ static dev_t device_devt;void vfio_init_device_cdev (struct vfio_device *device) { +if (vfio_device_is_noiommu (device) && + !IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU )) +return ; + device->device.devt =MKDEV (MAJOR (device_devt), device->index);cdev_init (&device->cdev, &vfio_device_fops); device->cdev.owner =THIS_MODULE ;
@@ -31,6 +35,11 @@ int vfio_device_fops_cdev_open(struct inode *inode, struct file *filep)if (!vfio_device_try_get_registration (device))return -ENODEV ; +if (vfio_device_is_noiommu (device) && !capable (CAP_SYS_RAWIO )) { + ret = -EPERM ; + goto err_put_registration; + } + df =vfio_allocate_device_file (device);if (IS_ERR (df)) { ret =PTR_ERR (df);
drivers/vfio/iommufd.c +8 -4
@@ -25,8 +25,8 @@ int vfio_df_iommufd_bind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -/* Returns 0 to permit device opening under noiommu mode */ -if (vfio_device_is_noiommu (vdev)) +/* Group noiommu via iommufd compat needs no device binding */ +if (df->group &&vfio_device_is_noiommu (vdev))return 0 ;return vdev->ops->bind_iommufd (vdev, ictx, &df->devid);
@@ -40,7 +40,11 @@ int vfio_iommufd_compat_attach_ioas(struct vfio_device *vdev,lockdep_assert_held (&vdev->dev_set->lock); -/* compat noiommu does not need to do ioas attach */ +/* +* Compat noiommu does not need to do ioas attach. This helper is +* only called from the legacy group/iommufd compat path, so no +* explicit df->group check is needed. +*/ if (vfio_device_is_noiommu (vdev))return 0 ;
@@ -58,7 +62,7 @@ void vfio_df_iommufd_unbind(struct vfio_device_file *df)lockdep_assert_held (&vdev->dev_set->lock); -if (vfio_device_is_noiommu (vdev)) +if (df->group &&vfio_device_is_noiommu (vdev))return ;if (vdev->ops->unbind_iommufd)
drivers/vfio/vfio.h +9 -14
@@ -112,11 +112,6 @@ bool vfio_device_has_container(struct vfio_device *device);int __init vfio_group_init (void );void vfio_group_cleanup (void ); -static inline bool vfio_device_is_noiommu (struct vfio_device *vdev) -{ -return IS_ENABLED (CONFIG_VFIO_NOIOMMU ) && - vdev->group->type ==VFIO_NO_IOMMU ; -}#else struct vfio_group ;
@@ -188,11 +183,17 @@ static inline void vfio_group_cleanup(void) { } +#endif /* CONFIG_VFIO_GROUP */ +static inline bool vfio_device_is_noiommu (struct vfio_device *vdev) { -return false; +#if IS_ENABLED (CONFIG_VFIO_GROUP ) +if (vdev->group && vdev->group->type ==VFIO_NO_IOMMU ) +return true; +#endif + +return IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && vdev->noiommu; } -#endif /* CONFIG_VFIO_GROUP */ #if IS_ENABLED (CONFIG_VFIO_CONTAINER )/**
@@ -358,19 +359,13 @@ void vfio_init_device_cdev(struct vfio_device *device);static inline int vfio_device_add (struct vfio_device *device) { -/* cdev does not support noiommu device */ -if (vfio_device_is_noiommu (device)) -return device_add (&device->device);vfio_init_device_cdev (device);return cdev_device_add (&device->cdev, &device->device); }static inline void vfio_device_del (struct vfio_device *device) { -if (vfio_device_is_noiommu (device)) -device_del (&device->device); -else -cdev_device_del (&device->cdev, &device->device); +cdev_device_del (&device->cdev, &device->device); }int vfio_device_fops_cdev_open (struct inode *inode,struct file *filep);
drivers/vfio/vfio_main.c +25 -1
@@ -321,6 +321,24 @@ static int vfio_init_device(struct vfio_device *device, struct device *dev,return ret; } +static int vfio_device_set_noiommu_and_name (struct vfio_device *device,enum vfio_group_type type) +{ +if (IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU ) && vfio_noiommu && + !device->dev->iommu && type ==VFIO_IOMMU ) + device->noiommu = true; + +/* +* device->noiommu records no-IOMMU support for the standalone cdev +* interface. VFIO_NOIOMMU enables both group and cdev no-IOMMU; when +* cdev no-IOMMU is available, device->noiommu is set before +* vfio_device_set_group(), so the cdev is named noiommu-vfio%d up +* front. If IOMMUFD_NOIOMMU is unavailable, no-IOMMU devices are +* limited to the group interface and do not receive a device cdev. +*/ +return dev_set_name (&device->device,"%svfio%d" , + device->noiommu ?"noiommu-" :"" , device->index); +} +static int __vfio_register_dev (struct vfio_device *device,enum vfio_group_type type) {
@@ -340,7 +358,7 @@ static int __vfio_register_dev(struct vfio_device *device,if (!device->dev_set)vfio_assign_device_set (device, device); - ret =dev_set_name (&device->device,"vfio%d" , device->index); + ret =vfio_device_set_noiommu_and_name (device, type);if (ret)return ret;
@@ -348,6 +366,12 @@ static int __vfio_register_dev(struct vfio_device *device,if (ret)return ret; +if (vfio_device_is_noiommu (device) &&IS_ENABLED (CONFIG_IOMMUFD_NOIOMMU )) { +add_taint (TAINT_USER ,LOCKDEP_STILL_OK ); +dev_warn (device->dev, +"Adding kernel taint for vfio-noiommu cdev\n" ); + } +/* * VFIO always sets IOMMU_CACHE because we offer no way for userspace to * restore cache coherency. It has to be checked here because it is only
include/linux/vfio.h +1 -0
@@ -74,6 +74,7 @@ struct vfio_device { u8 iommufd_attached:1;#endif u8 cdev_opened:1; + u8 noiommu:1;/* * debug_root is a static property of the vfio_device * which must be set prior to registering the vfio_device.
Signature · 1 line
2.43.0
PATCH v10 6/6 Documentation: Update VFIO NOIOMMU mode
In reply to Jacob PanDocument the NOIOMMU mode with newly added cdev support under iommufd.
---
v9:
- Clarify VFIO cdev no-IOMMU Kconfig limits
V8:
- Remove reference about self test.
v7:
- Added Kconfig matrix
v6:
- Generalize device node names (noiommu-vfioX, noiommu-Y) in the tree example (Yi)
- Clarify table column descriptions for Yes/No meanings (Yi)
--- Documentation/driver-api/vfio.rst | 89 ++++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 2 deletions(-)
Documentation/driver-api/vfio.rst +87 -2
@@ -275,8 +275,6 @@ in a VFIO group. With CONFIG_VFIO_DEVICE_CDEV=y the user can now acquire a device fd by directly opening a character device /dev/vfio/devices/vfioX where "X" is the number allocated uniquely by VFIO for registered devices. -cdev interface does not support noiommu devices, so user should use -the legacy group interface if noiommu is wanted. The cdev only works with IOMMUFD. Both VFIO drivers and applications must adapt to the new cdev security model which requires using
@@ -370,6 +368,93 @@ IOMMUFD IOAS/HWPT to enable userspace DMA:: /* Other device operations as stated in "VFIO Usage Example" */ +VFIO NOIOMMU mode +------------------------------------------------------------------------------- +VFIO also supports a no-IOMMU mode, intended for usages where unsafe DMA can +be performed by userspace drivers w/o physical IOMMU protection. This mode +is controlled by the parameter: + +/sys/module/vfio/parameters/enable_unsafe_noiommu_mode + +Upon enabling this mode, with an assigned device, the user will be presented +with a VFIO group and device file, e.g.:: + + /dev/vfio/ + |-- devices + | `-- noiommu-vfioX /* VFIO device cdev */ + |-- noiommu-Y /* VFIO group */ + `-- vfio + +The capabilities vary depending on the device programming interface and kernel +configuration used. The following table summarizes the differences ("Yes" means +the UAPI is accessible and functional in noiommu mode, "No" means the UAPI is +not supported): + ++-------------------+---------------------+----------------------+ +| Feature | VFIO group | VFIO device cdev | ++===================+=====================+======================+ +| VFIO device UAPI | Yes | Yes | ++-------------------+---------------------+----------------------+ +| VFIO container | No | No | ++-------------------+---------------------+----------------------+ +| IOMMUFD IOAS | No | Yes* | ++-------------------+---------------------+----------------------+ + +Note that the VFIO container case includes IOMMUFD provided VFIO compatibility +interfaces when either CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER is +enabled. + +* IOMMUFD UAPI is available for VFIO device cdev to pin and map user memory with + the ability to retrieve physical addresses for DMA command submission. + +Kconfig Support Matrix +^^^^^^^^^^^^^^^^^^^^^^ + +The visibility of CONFIG_VFIO_NOIOMMU depends on the combination of +CONFIG_VFIO_GROUP, CONFIG_VFIO_DEVICE_CDEV, and whether a container backend +(CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER) is configured. The +Kconfig dependencies enforce the following constraints: + +- At least one access path (group or cdev) must be available. +- If VFIO_GROUP is enabled, a container backend is required; otherwise the + group node would be unusable in noiommu mode. +- The cdev noiommu path requires CONFIG_GENERIC_ATOMIC64=n because it depends + on CONFIG_IOMMUFD_NOIOMMU. + +The resulting support matrix: + ++------+-------+-----------+------+---------+---------------------------+ +| Case | GROUP | Container | CDEV | NOIOMMU | Notes | ++======+=======+===========+======+=========+===========================+ +| 1 | y | y | n | yes | Group noiommu works | ++------+-------+-----------+------+---------+---------------------------+ +| 2 | y | n | n | no | Blocked - no container | ++------+-------+-----------+------+---------+---------------------------+ +| 3 | y | y | y | yes | Both paths work [#cdev]_ | ++------+-------+-----------+------+---------+---------------------------+ +| 4 | y | n | y | no | Blocked - no container | ++------+-------+-----------+------+---------+---------------------------+ +| 5 | n | - | y | yes | Cdev-only works [#cdev]_ | ++------+-------+-----------+------+---------+---------------------------+ +| 6 | n | - | n | no | No access path | ++------+-------+-----------+------+---------+---------------------------+ + +Container = CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER (either +suffices). Case 4 is intentionally blocked: allowing NOIOMMU with GROUP +enabled but no container would create unusable group nodes. Users who want +cdev-only noiommu should set CONFIG_VFIO_GROUP=n (case 5). + +.. [#cdev] The cdev noiommu path requires CONFIG_GENERIC_ATOMIC64=n. When + CONFIG_VFIO_GROUP=y, CONFIG_VFIO_DEVICE_CDEV=y, and + CONFIG_GENERIC_ATOMIC64=y, CONFIG_VFIO_NOIOMMU remains selectable for the + group path, but no noiommu device cdev is registered. Cdev-only noiommu is + not selectable on those platforms. + +A new IOMMUFD ioctl IOMMU_IOAS_NOIOMMU_GET_PA is added to retrieve the physical +address for a given IOVA. Although there is no physical DMA remapping hardware, +IOMMU_IOAS_MAP_FIXED_IOVA is still used to establish IOVA-to-PA mappings in the +software page table for later IOMMU_IOAS_NOIOMMU_GET_PA lookups. + VFIO User API -------------------------------------------------------------------------------
Signature · 1 line
2.43.0
Re: [PATCH v10 6/6] Documentation: Update VFIO NOIOMMU mode
Quoted 17 lines · On Mon, 6 Jul 2026 11:48:34 -0700 Jacob Pan <[email protected]>…
Document the NOIOMMU mode with newly added cdev support under iommufd.
Cc Jonathan Corbet <[email protected]>Reviewed-by Yi Liu <[email protected]>Reviewed-by Kevin Tian <[email protected]>Signed-off-by Jacob Pan <[email protected]>---
v9:
- Clarify VFIO cdev no-IOMMUKconfiglimits
V8:
- Remove reference about self test.
v7:
- AddedKconfigmatrix
v6:
- Generalize device node names (noiommu-vfioX, noiommu-Y) in the tree example (Yi)
- Clarify table column descriptions for Yes/No meanings (Yi)
Quoted 3 lines
--- Documentation/driver-api/vfio.rst | 89 ++++++++++++++++++++++++++++++- 1 file changed, 87 insertions(+), 2 deletions(-)
Quoted 108 lines
Documentation/driver-api/vfio.rst +87
-2@@ -275,8 +275,6 @@ in a VFIO group. With CONFIG_VFIO_DEVICE_CDEV=y the user can now acquire a device fd by directly opening a character device /dev/vfio/devices/vfioX where "X" is the number allocated uniquely by VFIO for registered devices. -cdev interface does not support noiommu devices, so user should use -the legacy group interface if noiommu is wanted. The cdev only works with IOMMUFD. Both VFIO drivers and applications must adapt to the new cdev security model which requires using@@ -370,6 +368,93 @@ IOMMUFD IOAS/HWPT to enable userspace DMA:: /* Other device operations as stated in "VFIO Usage Example" */ +VFIO NOIOMMU mode +------------------------------------------------------------------------------- +VFIO also supports a no-IOMMU mode, intended for usages where unsafe DMA can +be performed by userspace drivers w/o physical IOMMU protection. This mode +is controlled by the parameter: + +/sys/module/vfio/parameters/enable_unsafe_noiommu_mode + +Upon enabling this mode, with an assigned device, the user will be presented +with a VFIO group and device file, e.g.:: + + /dev/vfio/ + |-- devices + | `-- noiommu-vfioX /* VFIO device cdev */ + |-- noiommu-Y /* VFIO group */ + `-- vfio + +The capabilities vary depending on the device programming interface and kernel +configuration used. The following table summarizes the differences ("Yes" means +the UAPI is accessible and functional in noiommu mode, "No" means the UAPI is +not supported): + ++-------------------+---------------------+----------------------+ +| Feature | VFIO group | VFIO device cdev | ++===================+=====================+======================+ +| VFIO device UAPI | Yes | Yes | ++-------------------+---------------------+----------------------+ +| VFIO container | No | No | ++-------------------+---------------------+----------------------+ +| IOMMUFD IOAS | No | Yes* | ++-------------------+---------------------+----------------------+ + +Note that the VFIO container case includes IOMMUFD provided VFIO compatibility +interfaces when either CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER is +enabled. + +* IOMMUFD UAPI is available for VFIO device cdev to pin and map user memory with + the ability to retrieve physical addresses for DMA command submission. + +Kconfig Support Matrix +^^^^^^^^^^^^^^^^^^^^^^ + +The visibility of CONFIG_VFIO_NOIOMMU depends on the combination of +CONFIG_VFIO_GROUP, CONFIG_VFIO_DEVICE_CDEV, and whether a container backend +(CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER) is configured. The +Kconfig dependencies enforce the following constraints: + +- At least one access path (group or cdev) must be available. +- If VFIO_GROUP is enabled, a container backend is required; otherwise the + group node would be unusable in noiommu mode. +- The cdev noiommu path requires CONFIG_GENERIC_ATOMIC64=n because it depends + on CONFIG_IOMMUFD_NOIOMMU. + +The resulting support matrix: + ++------+-------+-----------+------+---------+---------------------------+ +| Case | GROUP | Container | CDEV | NOIOMMU | Notes | ++======+=======+===========+======+=========+===========================+ +| 1 | y | y | n | yes | Group noiommu works | ++------+-------+-----------+------+---------+---------------------------+ +| 2 | y | n | n | no | Blocked - no container | ++------+-------+-----------+------+---------+---------------------------+ +| 3 | y | y | y | yes | Both paths work [#cdev]_ | ++------+-------+-----------+------+---------+---------------------------+ +| 4 | y | n | y | no | Blocked - no container | ++------+-------+-----------+------+---------+---------------------------+ +| 5 | n | - | y | yes | Cdev-only works [#cdev]_ | ++------+-------+-----------+------+---------+---------------------------+ +| 6 | n | - | n | no | No access path | ++------+-------+-----------+------+---------+---------------------------+ + +Container = CONFIG_VFIO_CONTAINER or CONFIG_IOMMUFD_VFIO_CONTAINER (either +suffices). Case 4 is intentionally blocked: allowing NOIOMMU with GROUP +enabled but no container would create unusable group nodes. Users who want +cdev-only noiommu should set CONFIG_VFIO_GROUP=n (case 5). + +.. [#cdev] The cdev noiommu path requires CONFIG_GENERIC_ATOMIC64=n. When + CONFIG_VFIO_GROUP=y, CONFIG_VFIO_DEVICE_CDEV=y, and + CONFIG_GENERIC_ATOMIC64=y, CONFIG_VFIO_NOIOMMU remains selectable for the + group path, but no noiommu device cdev is registered. Cdev-only noiommu is + not selectable on those platforms. + +A new IOMMUFD ioctl IOMMU_IOAS_NOIOMMU_GET_PA is added to retrieve the physical +address for a given IOVA. Although there is no physical DMA remapping hardware, +IOMMU_IOAS_MAP_FIXED_IOVA is still used to establish IOVA-to-PA mappings in the +software page table for later IOMMU_IOAS_NOIOMMU_GET_PA lookups. + VFIO User API -------------------------------------------------------------------------------
Jason Gunthorpe
maintainerQuoted 34 lines · Jacob Pan
VFIO's
unsafe_noiommu_modehas long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices undervfio_compatmode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.In summary, we have:
|-------------------------+------+---------------| | Device access mode | VFIO | IOMMUFD | |-------------------------+------+---------------| | group /dev/vfio/$GROUP | Yes | Yes | |-------------------------+------+---------------| | cdev /dev/vfio/devices/ | No | This patch | |-------------------------+------+---------------|Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort tomlock(), which is less robust thanpin_user_pages()used for devices backed by a physical IOMMU. For example,mlock()does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on/proc/pagemaptricks or hardcoded values.By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].
Quoted 7 lines
Furthermore, these devices will use the IOMMUFD-based ownership checking model for
VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for aniommufd_accessobject as required in a previous attempt [3].ChangeLog:
v10:
- Rebased to v7.2-rc2, no intended code change
My impression is we are good on this now, right? I would like to pick it up?
Jason
Alex Williamson
maintainerQuoted 39 lines · On Mon, 6 Jul 2026 16:54:03 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 38 lines · Jacob Pan
VFIO's
unsafe_noiommu_modehas long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices undervfio_compatmode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.In summary, we have:
|-------------------------+------+---------------| | Device access mode | VFIO | IOMMUFD | |-------------------------+------+---------------| | group /dev/vfio/$GROUP | Yes | Yes | |-------------------------+------+---------------| | cdev /dev/vfio/devices/ | No | This patch | |-------------------------+------+---------------|Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort tomlock(), which is less robust thanpin_user_pages()used for devices backed by a physical IOMMU. For example,mlock()does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on/proc/pagemaptricks or hardcoded values.By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].
Furthermore, these devices will use the IOMMUFD-based ownership checking model for
VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for aniommufd_accessobject as required in a previous attempt [3].
Quoted 6 lines
Quoted 3 lines
ChangeLog:
v10:
- Rebased to v7.2-rc2, no intended code changeMy impression is we are good on this now, right? I would like to pick it up?
There are a couple Sashiko findings. The PAGE_SIZE issue seems like a false positive. The AMD based page tables necessarily expose 4K, but pinning is obviously at PAGE_SIZE, so returning multiples of PAGE_SIZE is arguably correct.
The PASID issue seems more like a uAPI hygiene question, and it might be the correct answer to return error for values other than IOMMU_NO_PASID. Your call.
Fine otherwise from a vfio perspective. Thanks,
Alex
Hi Alex,
Quoted 59 lines · On Mon, 6 Jul 2026 15:51:24 -0600 Alex Williamson <[email protected]> wrote:
Quoted 44 lines · On Mon, 6 Jul 2026 16:54:03 -0300 Jason Gunthorpe <[email protected]> wrote:
Quoted 43 lines · Jacob Pan
VFIO's
unsafe_noiommu_modehas long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices undervfio_compatmode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.In summary, we have:
|-------------------------+------+---------------| | Device access mode | VFIO | IOMMUFD | |-------------------------+------+---------------| | group /dev/vfio/$GROUP | Yes | Yes | |-------------------------+------+---------------| | cdev /dev/vfio/devices/ | No | This patch | |-------------------------+------+---------------|Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort tomlock(), which is less robust thanpin_user_pages()used for devices backed by a physical IOMMU. For example,mlock()does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on/proc/pagemaptricks or hardcoded values.By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].
Furthermore, these devices will use the IOMMUFD-based ownership checking model for
VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for aniommufd_accessobject as required in a previous attempt [3].Quoted 6 lines
Quoted 3 lines
ChangeLog:
v10:
- Rebased to v7.2-rc2, no intended code changeMy impression is we are good on this now, right? I would like to pick it up?
There are a couple Sashiko findings. The
PAGE_SIZEissue seems like a false positive. The AMD based page tables necessarily expose 4K, but pinning is obviously atPAGE_SIZE, so returning multiples ofPAGE_SIZEis arguably correct.
Quoted 7 lines
The PASID issue seems more like a uAPI hygiene question, and it might be the correct answer to return error for values other than
IOMMU_NO_PASID. Your call.Fine otherwise from a vfio perspective. Thanks,
Alex
I agree this is worth rejecting explicitly.
No-IOMMU only supports allocating paging domain and cannot allocate a PASID-capable domain. The practical path I found is VFIO_DEVICE_ATTACH_IOMMUFD_PT with VFIO_DEVICE_ATTACH_PASID set, where userspace supplies the PASID at attach time.
Jason, I tested the change below. If you are okay with it, could you fold it in? Otherwise I can spin v11.
Date: Tue, 7 Jul 2026 11:33:26 -0700 Subject: [PATCH]
iommufd: Validate HWPT compatibility before noiommu attach
--- drivers/iommu/iommufd/device.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/iommu/iommufd/device.c +6 -6
@@ -574,9 +574,6 @@ static int iommufd_hwpt_attach_device(struct iommufd_hw_pagetable *hwpt,struct iommufd_attach_handle *handle;int rc; -if (iommufd_device_is_noiommu (idev)) -return 0 ; -if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;
@@ -584,6 +581,9 @@ static int iommufd_hwpt_attach_device(struct iommufd_hw_pagetable *hwpt,if (rc)return rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; + handle =kzalloc_obj (*handle);if (!handle)return -ENOMEM ;
@@ -645,9 +645,6 @@ static int iommufd_hwpt_replace_device(struct iommufd_device *idev,struct iommufd_attach_handle *handle, *old_handle;int rc; -if (iommufd_device_is_noiommu (idev)) -return 0 ; -if (!iommufd_hwpt_compatible_device (hwpt, idev))return -EINVAL ;
@@ -655,6 +652,9 @@ static int iommufd_hwpt_replace_device(struct iommufd_device *idev,if (rc)return rc; +if (iommufd_device_is_noiommu (idev)) +return 0 ; + old_handle =iommufd_device_get_attach_handle (idev, pasid); handle =kzalloc_obj (*handle);
Quoted 44 lines · Jacob Pan
VFIO's
unsafe_noiommu_modehas long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices undervfio_compatmode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.In summary, we have:
|-------------------------+------+---------------| | Device access mode | VFIO | IOMMUFD | |-------------------------+------+---------------| | group /dev/vfio/$GROUP | Yes | Yes | |-------------------------+------+---------------| | cdev /dev/vfio/devices/ | No | This patch | |-------------------------+------+---------------|Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort tomlock(), which is less robust thanpin_user_pages()used for devices backed by a physical IOMMU. For example,mlock()does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on/proc/pagemaptricks or hardcoded values.By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].
Furthermore, these devices will use the IOMMUFD-based ownership checking model for
VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for aniommufd_accessobject as required in a previous attempt [3].Signed-off-by Jacob Pan <[email protected]>Jacob Pan (3):
iommufd: Add an ioctl to query PA from IOVA for noiommu mode
vfio: Enable cdev noiommu mode under iommufd
Documentation: Update VFIO NOIOMMU mode
Quoted 4 lines
Jason Gunthorpe (3):
iommufd: Support a HWPT without an iommu driver for noiommu
iommufd: Move igroup allocation to a function
iommufd: Allow binding to a noiommu device
Applied to for-next
Thanks,
Jason