Skip to content

iommufd: Enable noiommu mode for cdev

Jacob Pan linux-iommu lkml 10 versions – Applied

Status from this message: Applied to for-next

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v10 Latest6 patches 6 repliesApplied
  2. v96 patches 11 repliesSuperseded
  3. v86 patches 4 repliesSuperseded
Earlier versions · 7
VersionPostedPatchesRepliesStatus
v76 patches9 repliesSuperseded
v67 patches18 repliesSuperseded
v59 patches38 repliesSuperseded
v410 patches29 repliesSuperseded
v310 patches8 repliesSuperseded
v211 patches18 repliesSuperseded
v111 patches6 repliesSuperseded

Changes in v10 from the cover letter

- Rebased to v7.2-rc2, no intended code change

Patches in v10 Applied

#Subject and filesSizeRepliesApplied
1/6iommufd: Support a HWPT without an iommu driver for noiommudrivers/iommu/iommufd/Kconfig drivers/iommu/iommufd/Makefile drivers/iommu/iommufd/hw_pagetable.c +4 more+158 -60 replies
2/6iommufd: Move igroup allocation to a functiondrivers/iommu/iommufd/device.c+27 -161 reply
3/6iommufd: Allow binding to a noiommu devicedrivers/iommu/iommufd/device.c+118 -400 replies
4/6iommufd: Add an ioctl to query PA from IOVA for noiommu modedrivers/iommu/iommufd/io_pagetable.c drivers/iommu/iommufd/ioas.c drivers/iommu/iommufd/iommufd_private.h +2 more+163 -00 replies
5/6vfio: Enable cdev noiommu mode under iommufddrivers/vfio/Kconfig drivers/vfio/device_cdev.c drivers/vfio/iommufd.c +3 more+56 -220 replies
6/6Documentation: Update VFIO NOIOMMU modeDocumentation/driver-api/vfio.rst+87 -21 reply

Reviews trailers per patch

All 6 patches have a Reviewed-by or Acked-by.

PatchLu BaoluSamiullah KhawajaKevin TianPranjal ShrivastavaYi LiuAlex Williamson
1/6R-b R-b R-b R-b
2/6R-b R-b R-b R-b R-b
3/6R-b R-b R-b R-b
4/6R-b R-b R-b R-b
5/6R-b R-b R-b R-b
6/6R-b R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 6 reviewers.

Read the v10 thread

Cover letter [PATCH v10 0/6] iommufd: Enable noiommu mode for cdev

VFIO's unsafe_noiommu_mode has long provided a way for userspace drivers to operate on platforms lacking a hardware IOMMU. Today, IOMMUFD also supports No-IOMMU mode for group-based devices under vfio_compat mode. However, IOMMUFD's native character device (cdev) does not yet support No-IOMMU mode, which is the purpose of this patch.

In summary, we have:

|-------------------------+------+---------------|
| Device access mode      | VFIO | IOMMUFD       |
|-------------------------+------+---------------|
| group /dev/vfio/$GROUP  | Yes  | Yes           |
|-------------------------+------+---------------|
| cdev /dev/vfio/devices/ | No   | This patch    |
|-------------------------+------+---------------|

Beyond enabling cdev for IOMMUFD, this patch also addresses the following deficiencies in the current No-IOMMU mode suggested by Jason[1]:
- Devices operating under No-IOMMU mode are limited to device-level UAPI access, without container or IOAS-level capabilities. Consequently, user-space drivers lack structured mechanisms for page pinning and often resort to mlock(), which is less robust than pin_user_pages() used for devices backed by a physical IOMMU. For example, mlock() does not prevent page migration.
- There is no architectural mechanism for obtaining physical addresses for DMA. As a workaround, user-space drivers frequently rely on /proc/pagemap tricks or hardcoded values.

By allowing noiommu device access to IOMMUFD IOAS and HWPT objects, this patch brings No-IOMMU mode closer to full citizenship within the IOMMU subsystem. In addition to addressing the two deficiencies mentioned above, the expectation is that it will also enable No-IOMMU devices to seamlessly participate in live update sessions via KHO [2].

Furthermore, these devices will use the IOMMUFD-based ownership checking model for VFIO_DEVICE_PCI_HOT_RESET, eliminating the need for an iommufd_access object as required in a previous attempt [3].

ChangeLog:
v10:
- Rebased to v7.2-rc2, no intended code change
v9:
- Leave device->device.devt unset for no-IOMMU dev so cdev_device_add() registers only the struct device and does not expose an unsupported cdev. (Alex, Sashiko)
- Clarify VFIO cdev no-IOMMU Kconfig limits in documentation
- Hold registration while checking cdev no-IOMMU access (Sashiko)
- Make no-IOMMU GET_PA length a real upper bound and reject zero length, avoiding an unbounded scan while holding IOAS locks. This matches the bounded-range semantics expected by the incoming iommu_iova_to_phys_length() helper.
- Guard replace path for noiommu device (Sashiko)
v8:
- Guard noiommu for vdevice viommu alloc (Kevin)
v7:
- Handle Sashiko reviews.
- Dropped selftest for now, will submit separately for v7.2 to use new lib helpers
v6: Undo CDEV-GROUP NOIOMMU split, use Kconfig to restrict unwanted
combo.
V5:
- Split CONFIG_VFIO_NOIOMMU into CONFIG_VFIO_GROUP_NOIOMMU and CONFIG_VFIO_CDEV_NOIOMMU so cdev noiommu is independent of VFIO_GROUP (Alex)
- Add CAP_SYS_RAWIO check for cdev open and bind under noiommu, security parity with group noiommu (Alex)
- Add IS_ENABLED(CONFIG_IOMMUFD_NOIOMMU) guard in iommufd_device_is_noiommu() to prevent noiommu bind when feature is disabled
- Add prep patch to tolerate NULL group for cdev noiommu devices when CONFIG_VFIO_GROUP_NOIOMMU is not set [7/9]
- Rename IOCTL to IOMMUFD_CMD_IOAS_NOIOMMU_GET_PA to be more specific (Kevin)
- Simplify iommufd_device_is_noiommu, use iommufd_bind_noiommu helper (Kevin, Yi)
- Move IOMMU cap check under iommufd_bind_iommu() (Yi)
- Fix next_iova exceeding iopt_area_last_iova in GET_PA (Alex)
- Fix const hwpt, copyright date, typo in moved comment (Kevin)
- Add Reviewed-by tags
- Squash noiommu cdev selftest fix into selftest patch
- Drop DSA selftest patch
- Details in each patch changelog.

V4:
- Fix various corner cases pointed out by (Sashiko) Details in each patch changelog.

V3:
- Improve error handling [3/10] (Mostafa)
- Simplify vfio_device_is_noiommu logic and merged in [6/10] (Mostafa)
- Add comment to explain the design difference over the legacy noiommu

    VFIO code.[1/10]

V2:
- Fix build dependency by adding IOMMU_SUPPORT in [8/11]
- Add an optimization to scan beyond the first page for a contiguous physical address range and return its length instead of a single page.[4/11]

Since RFC[4]:
- Abandoned dummy iommu driver approach as patch 1-3 absorbed the changes into iommufd.

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-pci/[email protected]/
[3] https://lore.kernel.org/kvm/[email protected]/
[4] https://lore.kernel.org/linux-iommu/[email protected]/

Jacob Pan (3):
iommufd: Add an ioctl to query PA from IOVA for noiommu mode
vfio: Enable cdev noiommu mode under iommufd
Documentation: Update VFIO NOIOMMU mode

Jason Gunthorpe (3):
iommufd: Support a HWPT without an iommu driver for noiommu
iommufd: Move igroup allocation to a function
iommufd: Allow binding to a noiommu device

 Documentation/driver-api/vfio.rst       |  89 ++++++++++-
 drivers/iommu/iommufd/Kconfig           |  12 ++
 drivers/iommu/iommufd/Makefile          |   1 +
 drivers/iommu/iommufd/device.c          | 201 +++++++++++++++++-------
 drivers/iommu/iommufd/hw_pagetable.c    |  19 ++-
 drivers/iommu/iommufd/hwpt_noiommu.c    | 105 +++++++++++++
 drivers/iommu/iommufd/io_pagetable.c    |  78 +++++++++
 drivers/iommu/iommufd/ioas.c            |  36 +++++
 drivers/iommu/iommufd/iommufd_private.h |  30 ++++
 drivers/iommu/iommufd/main.c            |   4 +
 drivers/iommu/iommufd/viommu.c          |  14 +-
 drivers/vfio/Kconfig                    |   7 +-
 drivers/vfio/device_cdev.c              |   9 ++
 drivers/vfio/iommufd.c                  |  12 +-
 drivers/vfio/vfio.h                     |  23 ++-
 drivers/vfio/vfio_main.c                |  26 ++-
 include/linux/vfio.h                    |   1 +
 include/uapi/linux/iommufd.h            |  28 ++++
 18 files changed, 609 insertions(+), 86 deletions(-)
 create mode 100644 drivers/iommu/iommufd/hwpt_noiommu.c
Signature · 1 line
2.43.0