vfio-pci support pasid attach/detach
Subsystems (8)
v9 cover vfio-pci support pasid attach/detach
This series introduces the PASID attach/detach user APIs (uAPIs) that allow userspace to attach or detach a device's PASID to or from a specified IOAS/hwpt. Currently, only the vfio-pci driver is enabled in this series.
Following this update, PASID-capable devices bound to vfio-pci can report PASID capabilities to userspace and virtual machines (VMs), facilitating PASID use cases such as Shared Virtual Addressing (SVA). In discussions about reporting the virtual PASID (vPASID) to VMs [1], it was agreed that the userspace virtual machine monitor (VMM) will synthesize the vPASID capability. The VMM must identify a suitable location to insert the vPASID capability, including handling hidden bits for certain devices. However, this responsibility lies with userspace and is not the focus of this series.
This series begins by adding helpers for PASID attachment in the vfio core, then extends the device character device (cdev) attach/detach ioctls to support PASID attach/detach operations. At the conclusion of this series, the IOMMU_GET_HW_INFO ioctl is extended to report PCI PASID capabilities to userspace. Userspace should verify this capability before utilizing any PASID-related uAPIs provided by VFIO, as agreed in [2]. This series depends on the iommufd PASID attach/detach series [3].
The complete code is available at [4] and has been tested with a modified QEMU branch [5].
[1] https://lore.kernel.org/kvm/BN9PR11MB5276318969A212AD0649C7BE8CBE2@BN9PR11MB5276.namprd11.prod.outlook.com/
[2] https://lore.kernel.org/kvm/[email protected]/
[3] https://lore.kernel.org/linux-iommu/[email protected]/
[4] https://github.com/yiliu1765/iommufd/tree/iommufd_pasid
[5] https://github.com/yiliu1765/qemu/tree/wip/zhenzhong/iommufd_nesting_rfcv2-test-pasid
Change log:
v9:
- Remove unnecessary parentheses in flag check (Nic)
- Use the MOCK_PASID_WIDTH in the place of 20 is used in patch 05 (Nic)
- Drop duplicated pci.h include (Nic)
v8: https://lore.kernel.org/kvm/[email protected]/
- Rebased on top of the latest iommufd series, mainly using the latest kAPI for pasid attach
v7: https://lore.kernel.org/kvm/[email protected]/#t
- Add Alex's and Kevin's r-b on vfio patches
- Minor tweaks on patch 04 (Kevin)
v6: https://lore.kernel.org/kvm/[email protected]/
- Drop the vfio_copy_user_data() generalization as it is not totally clear what it would cover. (Alex)
- Reworked the patch 03 of v5 a bit. e.g. lift the pasid_{at|de}tach_ioas op test before the second user data copy; make 'if (xend > minsz)' to be 'if (xend)' and remove the comment accordingly. This is because we don't generalize the user data copy now, so xend is either 0 or non-zero, no need to check against minsz.
- Make the IOMMU_GET_HW_INFO report out_max_pasid_log2 by checking the dev->iommu->max_pasids. This is because iommu drivers enables PASID as long as it supports. So checking it is enough. Also, it is more friendly to non-PCI PASID supports compared with reading the PCI config space to check if PASID is enabled.
- Add selftest coverage for reporting max_pasid_log2 in IOMMU_HW_INFO ioctl.
v5: https://lore.kernel.org/kvm/[email protected]/
- Fix a wrong return value (Alex)
- Fix the policy of setting the xend array per flag extension (Alex)
- A separate patch to generalize the code of copy user data (Alex)
v4: https://lore.kernel.org/kvm/[email protected]/
- Add acked-by for the ida patch from Matthew
- Add r-b from Kevin and Jason on patch 01, 02 and 04 of v3
- Add common code to copy user data for the user struct with new fields
- Extend the VFIO_DEVICE_[AT|DE]TACH_IOMMUFD_PT to support pasid, patch 03 is updated per this change. Hence drop r-b of it. (Kevin, Alex)
- Add t-b from Zhangfei for patch 4 of v3
- Nits from Vasant
v3: https://lore.kernel.org/linux-iommu/[email protected]/
- Misc enhancement on patch 01 of v2 (Alex, Jason)
- Add Jason's r-b to patch 03 of v2
- Drop the logic that report PASID via VFIO_DEVICE_FEATURE ioctl
- Extend IOMMU_GET_HW_INFO to report PASID support (Kevin, Jason, Alex)
v2: https://lore.kernel.org/kvm/[email protected]/
- Use IDA to track if PASID is attached or not in VFIO. (Jason)
- Fix the issue of calling pasid_at[de]tach_ioas callback unconditionally (Alex)
- Fix the wrong data copy in vfio_df_ioctl_pasid_detach_pt() (Zhenzhong)
- Minor tweaks in comments (Kevin)
v1: https://lore.kernel.org/kvm/[email protected]/
- Report PASID capability via VFIO_DEVICE_FEATURE (Alex)
rfc: https://lore.kernel.org/linux-iommu/[email protected]/
Regards,
Yi Liu
Yi Liu (5):
ida: Add ida_find_first_range()
vfio-iommufd: Support pasid [at|de]tach for physical VFIO devices
vfio: VFIO_DEVICE_[AT|DE]TACH_IOMMUFD_PT support pasid
iommufd: Extend IOMMU_GET_HW_INFO to report PASID capability iommufd/selftest: Add coverage for reporting max_pasid_log2 via
IOMMU_HW_INFO
drivers/iommu/iommufd/device.c | 34 ++++++++- drivers/pci/ats.c | 33 +++++++++ drivers/vfio/device_cdev.c | 60 +++++++++++++--- drivers/vfio/iommufd.c | 50 +++++++++++++ drivers/vfio/pci/vfio_pci.c | 2 + include/linux/idr.h | 11 +++ include/linux/pci-ats.h | 3 + include/linux/vfio.h | 14 ++++ include/uapi/linux/iommufd.h | 14 +++- include/uapi/linux/vfio.h | 29 +++++--- lib/idr.c | 67 ++++++++++++++++++ lib/test_ida.c | 70 +++++++++++++++++++ tools/testing/selftests/iommu/iommufd.c | 18 +++++ .../selftests/iommu/iommufd_fail_nth.c | 3 +- tools/testing/selftests/iommu/iommufd_utils.h | 17 +++-- 15 files changed, 400 insertions(+), 25 deletions(-)
Signature · 1 line
2.34.1
PATCH v9 1/5 ida: Add ida_find_first_range()
There is no helpers for user to check if a given ID is allocated or not, neither a helper to loop all the allocated IDs in an IDA and do something for cleanup. With the two needs, a helper to get the lowest allocated ID of a range and two variants based on it.
Caller can check if a given ID is allocated or not by:
bool ida_exists(struct ida *ida, unsigned int id)
Caller can iterate all allocated IDs by:
int id;
while ((id = ida_find_first(&pasid_ida)) >= 0) {
//anything to do with the allocated ID
ida_free(pasid_ida, pasid);
}--- include/linux/idr.h | 11 +++++++ lib/idr.c | 67 +++++++++++++++++++++++++++++++++++++++++++ lib/test_ida.c | 70 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 148 insertions(+)
include/linux/idr.h +11 -0
@@ -257,6 +257,7 @@ struct ida {int ida_alloc_range (struct ida *,unsigned int min,unsigned int max,gfp_t );void ida_free (struct ida *,unsigned int id);void ida_destroy (struct ida *ida); +int ida_find_first_range (struct ida *ida,unsigned int min,unsigned int max);/** * ida_alloc() - Allocate an unused ID.
@@ -328,4 +329,14 @@ static inline bool ida_is_empty(const struct ida *ida) {return xa_empty (&ida->xa); } + +static inline bool ida_exists (struct ida *ida,unsigned int id) +{ +return ida_find_first_range (ida, id, id) == id; +} + +static inline int ida_find_first (struct ida *ida) +{ +return ida_find_first_range (ida,0 , ~0 ); +}#endif /* __IDR_H__ */
lib/idr.c +67 -0
73 lines not on this page · raw message
lib/test_ida.c +70 -0
82 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v9 2/5 vfio-iommufd: Support pasid [at|de]tach for physical VFIO devices
In reply to Yi LiuThis adds pasid_at|de]tach_ioas ops for attaching hwpt to pasid of a device and the helpers for it. For now, only vfio-pci supports pasid attach/detach.
--- drivers/vfio/iommufd.c | 50 +++++++++++++++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci.c | 2 ++ include/linux/vfio.h | 14 +++++++++++ 3 files changed, 66 insertions(+)
drivers/vfio/iommufd.c +50 -0
@@ -119,14 +119,22 @@ int vfio_iommufd_physical_bind(struct vfio_device *vdev,if (IS_ERR (idev))return PTR_ERR (idev); vdev->iommufd_device = idev; +ida_init (&vdev->pasids);return 0 ; }EXPORT_SYMBOL_GPL (vfio_iommufd_physical_bind);void vfio_iommufd_physical_unbind (struct vfio_device *vdev) { +int pasid; +lockdep_assert_held (&vdev->dev_set->lock); +while ((pasid =ida_find_first (&vdev->pasids)) >=0 ) { +iommufd_device_detach (vdev->iommufd_device, pasid); +ida_free (&vdev->pasids, pasid); + } +if (vdev->iommufd_attached) {iommufd_device_detach (vdev->iommufd_device,IOMMU_NO_PASID ); vdev->iommufd_attached = false;
@@ -170,6 +178,48 @@ void vfio_iommufd_physical_detach_ioas(struct vfio_device *vdev) }EXPORT_SYMBOL_GPL (vfio_iommufd_physical_detach_ioas); +int vfio_iommufd_physical_pasid_attach_ioas (struct vfio_device *vdev, +u32 pasid,u32 *pt_id) +{ +int rc; + +lockdep_assert_held (&vdev->dev_set->lock); + +if (WARN_ON (!vdev->iommufd_device)) +return -EINVAL ; + +if (ida_exists (&vdev->pasids, pasid)) +return iommufd_device_replace (vdev->iommufd_device, + pasid, pt_id); + + rc =ida_alloc_range (&vdev->pasids, pasid, pasid,GFP_KERNEL ); +if (rc <0 ) +return rc; + + rc =iommufd_device_attach (vdev->iommufd_device, pasid, pt_id); +if (rc) +ida_free (&vdev->pasids, pasid); + +return rc; +} +EXPORT_SYMBOL_GPL (vfio_iommufd_physical_pasid_attach_ioas); + +void vfio_iommufd_physical_pasid_detach_ioas (struct vfio_device *vdev, +u32 pasid) +{ +lockdep_assert_held (&vdev->dev_set->lock); + +if (WARN_ON (!vdev->iommufd_device)) +return ; + +if (!ida_exists (&vdev->pasids, pasid)) +return ; + +iommufd_device_detach (vdev->iommufd_device, pasid); +ida_free (&vdev->pasids, pasid); +} +EXPORT_SYMBOL_GPL (vfio_iommufd_physical_pasid_detach_ioas); +/* * The emulated standard ops mean that vfio_device is going to use the * "mdev path" and will call vfio_pin_pages()/vfio_dma_rw(). Drivers using this
drivers/vfio/pci/vfio_pci.c +2 -0
@@ -144,6 +144,8 @@ static const struct vfio_device_ops vfio_pci_ops = { .unbind_iommufd = vfio_iommufd_physical_unbind, .attach_ioas = vfio_iommufd_physical_attach_ioas, .detach_ioas = vfio_iommufd_physical_detach_ioas, + .pasid_attach_ioas = vfio_iommufd_physical_pasid_attach_ioas, + .pasid_detach_ioas = vfio_iommufd_physical_pasid_detach_ioas, };static int vfio_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id)
include/linux/vfio.h +14 -0
@@ -67,6 +67,7 @@ struct vfio_device {struct inode *inode;#if IS_ENABLED (CONFIG_IOMMUFD )struct iommufd_device *iommufd_device; +struct ida pasids; u8 iommufd_attached:1;#endif u8 cdev_opened:1;
@@ -91,6 +92,8 @@ struct vfio_device {* bound iommufd. Undo in unbind_iommufd if @detach_ioas is not * called. * @detach_ioas: Opposite of attach_ioas +* @pasid_attach_ioas: The pasid variation of attach_ioas +* @pasid_detach_ioas: Opposite of pasid_attach_ioas * @open_device: Called when the first file descriptor is opened for this device * @close_device: Opposite of open_device * @read: Perform read(2) on device file descriptor
@@ -115,6 +118,9 @@ struct vfio_device_ops {void (*unbind_iommufd)(struct vfio_device *vdev);int (*attach_ioas)(struct vfio_device *vdev,u32 *pt_id);void (*detach_ioas)(struct vfio_device *vdev); +int (*pasid_attach_ioas)(struct vfio_device *vdev,u32 pasid, +u32 *pt_id); +void (*pasid_detach_ioas)(struct vfio_device *vdev,u32 pasid);int (*open_device)(struct vfio_device *vdev);void (*close_device)(struct vfio_device *vdev); ssize_t (*read)(struct vfio_device *vdev, char __user *buf,
@@ -139,6 +145,10 @@ int vfio_iommufd_physical_bind(struct vfio_device *vdev,void vfio_iommufd_physical_unbind (struct vfio_device *vdev);int vfio_iommufd_physical_attach_ioas (struct vfio_device *vdev,u32 *pt_id);void vfio_iommufd_physical_detach_ioas (struct vfio_device *vdev); +int vfio_iommufd_physical_pasid_attach_ioas (struct vfio_device *vdev, +u32 pasid,u32 *pt_id); +void vfio_iommufd_physical_pasid_detach_ioas (struct vfio_device *vdev, +u32 pasid);int vfio_iommufd_emulated_bind (struct vfio_device *vdev,struct iommufd_ctx *ictx,u32 *out_device_id);void vfio_iommufd_emulated_unbind (struct vfio_device *vdev);
@@ -166,6 +176,10 @@ vfio_iommufd_get_dev_id(struct vfio_device *vdev, struct iommufd_ctx *ictx) ((int (*)(struct vfio_device *vdev, u32 *pt_id)) NULL)#define vfio_iommufd_physical_detach_ioas \ ((void (*)(struct vfio_device *vdev)) NULL) +#define vfio_iommufd_physical_pasid_attach_ioas \ + ((int (*)(struct vfio_device *vdev, u32 pasid, u32 *pt_id)) NULL) +#define vfio_iommufd_physical_pasid_detach_ioas \ + ((void (*)(struct vfio_device *vdev, u32 pasid)) NULL)#define vfio_iommufd_emulated_bind \ ((int (*)(struct vfio_device *vdev, struct iommufd_ctx *ictx, \ u32 *out_device_id)) NULL)
Signature · 1 line
2.34.1
PATCH v9 3/5 vfio: VFIO_DEVICE_[AT|DE]TACH_IOMMUFD_PT support pasid
In reply to Yi LiuThis extends the VFIO_DEVICE_[AT|DE]TACH_IOMMUFD_PT ioctls to attach/detach a given pasid of a vfio device to/from an IOAS/HWPT.
--- drivers/vfio/device_cdev.c | 60 +++++++++++++++++++++++++++++++++----- include/uapi/linux/vfio.h | 29 +++++++++++------- 2 files changed, 71 insertions(+), 18 deletions(-)
drivers/vfio/device_cdev.c +52 -8
@@ -162,9 +162,9 @@ void vfio_df_unbind_iommufd(struct vfio_device_file *df)int vfio_df_ioctl_attach_pt (struct vfio_device_file *df,struct vfio_device_attach_iommufd_pt __user *arg) { -struct vfio_device *device = df->device;struct vfio_device_attach_iommufd_pt attach; -unsigned long minsz; +struct vfio_device *device = df->device; +unsigned long minsz, xend =0 ;int ret; minsz = offsetofend(struct vfio_device_attach_iommufd_pt, pt_id);
@@ -172,11 +172,34 @@ int vfio_df_ioctl_attach_pt(struct vfio_device_file *df,if (copy_from_user (&attach, arg, minsz))return -EFAULT ; -if (attach.argsz < minsz || attach.flags) +if (attach.argsz < minsz)return -EINVAL ; +if (attach.flags & ~VFIO_DEVICE_ATTACH_PASID ) +return -EINVAL ; + +if (attach.flags &VFIO_DEVICE_ATTACH_PASID ) { +if (!device->ops->pasid_attach_ioas) +return -EOPNOTSUPP ; + xend = offsetofend(struct vfio_device_attach_iommufd_pt, pasid); + } + +if (xend) { +if (attach.argsz < xend) +return -EINVAL ; + +if (copy_from_user ((void *)&attach + minsz, + (void __user *)arg + minsz, xend - minsz)) +return -EFAULT ; + } +mutex_lock (&device->dev_set->lock); - ret = device->ops->attach_ioas (device, &attach.pt_id); +if (attach.flags &VFIO_DEVICE_ATTACH_PASID ) + ret = device->ops->pasid_attach_ioas (device, + attach.pasid, + &attach.pt_id); +else + ret = device->ops->attach_ioas (device, &attach.pt_id);if (ret) goto out_unlock;
@@ -198,20 +221,41 @@ int vfio_df_ioctl_attach_pt(struct vfio_device_file *df,int vfio_df_ioctl_detach_pt (struct vfio_device_file *df,struct vfio_device_detach_iommufd_pt __user *arg) { -struct vfio_device *device = df->device;struct vfio_device_detach_iommufd_pt detach; -unsigned long minsz; +struct vfio_device *device = df->device; +unsigned long minsz, xend =0 ; minsz = offsetofend(struct vfio_device_detach_iommufd_pt, flags);if (copy_from_user (&detach, arg, minsz))return -EFAULT ; -if (detach.argsz < minsz || detach.flags) +if (detach.argsz < minsz)return -EINVAL ; +if (detach.flags & ~VFIO_DEVICE_DETACH_PASID ) +return -EINVAL ; + +if (detach.flags &VFIO_DEVICE_DETACH_PASID ) { +if (!device->ops->pasid_detach_ioas) +return -EOPNOTSUPP ; + xend = offsetofend(struct vfio_device_detach_iommufd_pt, pasid); + } + +if (xend) { +if (detach.argsz < xend) +return -EINVAL ; + +if (copy_from_user ((void *)&detach + minsz, + (void __user *)arg + minsz, xend - minsz)) +return -EFAULT ; + } +mutex_lock (&device->dev_set->lock); - device->ops->detach_ioas (device); +if (detach.flags &VFIO_DEVICE_DETACH_PASID ) + device->ops->pasid_detach_ioas (device, detach.pasid); +else + device->ops->detach_ioas (device);mutex_unlock (&device->dev_set->lock);return 0 ;
include/uapi/linux/vfio.h +19 -10
@@ -931,29 +931,34 @@ struct vfio_device_bind_iommufd {* VFIO_DEVICE_ATTACH_IOMMUFD_PT - _IOW(VFIO_TYPE, VFIO_BASE + 19, * struct vfio_device_attach_iommufd_pt) * @argsz: User filled size of this data. -* @flags: Must be 0. +* @flags: Flags for attach. * @pt_id: Input the target id which can represent an ioas or a hwpt * allocated via iommufd subsystem. * Output the input ioas id or the attached hwpt id which could * be the specified hwpt itself or a hwpt automatically created * for the specified ioas by kernel during the attachment. +* @pasid: The pasid to be attached, only meaningful when +* VFIO_DEVICE_ATTACH_PASID is set in @flags * * Associate the device with an address space within the bound iommufd. * Undo by VFIO_DEVICE_DETACH_IOMMUFD_PT or device fd close. This is only * allowed on cdev fds. * -* If a vfio device is currently attached to a valid hw_pagetable, without doing -* a VFIO_DEVICE_DETACH_IOMMUFD_PT, a second VFIO_DEVICE_ATTACH_IOMMUFD_PT ioctl -* passing in another hw_pagetable (hwpt) id is allowed. This action, also known -* as a hw_pagetable replacement, will replace the device's currently attached -* hw_pagetable with a new hw_pagetable corresponding to the given pt_id. +* If a vfio device or a pasid of this device is currently attached to a valid +* hw_pagetable (hwpt), without doing a VFIO_DEVICE_DETACH_IOMMUFD_PT, a second +* VFIO_DEVICE_ATTACH_IOMMUFD_PT ioctl passing in another hwpt id is allowed. +* This action, also known as a hw_pagetable replacement, will replace the +* currently attached hwpt of the device or the pasid of this device with a new +* hwpt corresponding to the given pt_id. * * Return: 0 on success, -errno on failure. */ struct vfio_device_attach_iommufd_pt {__u32 argsz;__u32 flags; +#define VFIO_DEVICE_ATTACH_PASID (1 << 0)__u32 pt_id; +__u32 pasid; };#define VFIO_DEVICE_ATTACH_IOMMUFD_PT _IO(VFIO_TYPE, VFIO_BASE + 19)
@@ -962,17 +967,21 @@ struct vfio_device_attach_iommufd_pt {* VFIO_DEVICE_DETACH_IOMMUFD_PT - _IOW(VFIO_TYPE, VFIO_BASE + 20, * struct vfio_device_detach_iommufd_pt) * @argsz: User filled size of this data. -* @flags: Must be 0. +* @flags: Flags for detach. +* @pasid: The pasid to be detached, only meaningful when +* VFIO_DEVICE_DETACH_PASID is set in @flags * -* Remove the association of the device and its current associated address -* space. After it, the device should be in a blocking DMA state. This is only -* allowed on cdev fds. +* Remove the association of the device or a pasid of the device and its current +* associated address space. After it, the device or the pasid should be in a +* blocking DMA state. This is only allowed on cdev fds. * * Return: 0 on success, -errno on failure. */ struct vfio_device_detach_iommufd_pt {__u32 argsz;__u32 flags; +#define VFIO_DEVICE_DETACH_PASID (1 << 0) +__u32 pasid; };#define VFIO_DEVICE_DETACH_IOMMUFD_PT _IO(VFIO_TYPE, VFIO_BASE + 20)
Signature · 1 line
2.34.1
PATCH v9 4/5 iommufd: Extend IOMMU_GET_HW_INFO to report PASID capability
In reply to Yi LiuPASID usage requires PASID support in both device and IOMMU. Since the iommu drivers always enable the PASID capability for the device if it is supported, this extends the IOMMU_GET_HW_INFO to report the PASID capability to userspace. Also, enhances the selftest accordingly.
--- drivers/iommu/iommufd/device.c | 34 +++++++++++++++++++++++++++++++++- drivers/pci/ats.c | 33 +++++++++++++++++++++++++++++++++ include/linux/pci-ats.h | 3 +++ include/uapi/linux/iommufd.h | 14 +++++++++++++- 4 files changed, 82 insertions(+), 2 deletions(-)
drivers/iommu/iommufd/device.c +33 -1
@@ -3,6 +3,7 @@*/ #include <linux/iommu.h> #include <linux/iommufd.h> +#include <linux/pci-ats.h> #include <linux/slab.h> #include <uapi/linux/iommufd.h>
@@ -1455,7 +1456,8 @@ int iommufd_get_hw_info(struct iommufd_ucmd *ucmd)void *data;int rc; -if (cmd->flags || cmd->__reserved) +if (cmd->flags || cmd->__reserved[0 ] || cmd->__reserved[1 ] || + cmd->__reserved[2 ])return -EOPNOTSUPP ; idev =iommufd_get_device (ucmd, cmd->dev_id);
@@ -1512,6 +1514,36 @@ int iommufd_get_hw_info(struct iommufd_ucmd *ucmd)if (device_iommu_capable (idev->dev,IOMMU_CAP_DIRTY_TRACKING )) cmd->out_capabilities |=IOMMU_HW_CAP_DIRTY_TRACKING ; + cmd->out_max_pasid_log2 =0 ; +/* +* Currently, all iommu drivers enable PASID in the probe_device() +* op if iommu and device supports it. So the max_pasids stored in +* dev->iommu indicates both PASID support and enable status. A +* non-zero dev->iommu->max_pasids means PASID is supported and +* enabled. The iommufd only reports PASID capability to userspace +* if it's enabled. +*/ +if (idev->dev->iommu->max_pasids) { + cmd->out_max_pasid_log2 =ilog2 (idev->dev->iommu->max_pasids); + +if (dev_is_pci (idev->dev)) { +struct pci_dev *pdev =to_pci_dev (idev->dev); +int ctrl; + + ctrl =pci_pasid_status (pdev); + +WARN_ON_ONCE (ctrl <0 || + !(ctrl &PCI_PASID_CTRL_ENABLE )); + +if (ctrl &PCI_PASID_CTRL_EXEC ) + cmd->out_capabilities |= +IOMMU_HW_CAP_PCI_PASID_EXEC ; +if (ctrl &PCI_PASID_CTRL_PRIV ) + cmd->out_capabilities |= +IOMMU_HW_CAP_PCI_PASID_PRIV ; + } + } + rc =iommufd_ucmd_respond (ucmd,sizeof (*cmd)); out_free:kfree (data);
drivers/pci/ats.c +33 -0
@@ -538,4 +538,37 @@ int pci_max_pasids(struct pci_dev *pdev) return (1 << FIELD_GET(PCI_PASID_CAP_WIDTH, supported)); }EXPORT_SYMBOL_GPL (pci_max_pasids); + +/** +* pci_pasid_status - Check the PASID status +* @pdev: PCI device structure +* +* Returns a negative value when no PASID capability is present. +* Otherwise the value of the control register is returned. +* Status reported are: +* +* PCI_PASID_CTRL_ENABLE - PASID enabled +* PCI_PASID_CTRL_EXEC - Execute permission enabled +* PCI_PASID_CTRL_PRIV - Privileged mode enabled +*/ +int pci_pasid_status (struct pci_dev *pdev) +{ +int pasid; +u16 ctrl; + +if (pdev->is_virtfn) + pdev =pci_physfn (pdev); + + pasid = pdev->pasid_cap; +if (!pasid) +return -EINVAL ; + +pci_read_config_word (pdev, pasid +PCI_PASID_CTRL , &ctrl); + + ctrl &=PCI_PASID_CTRL_ENABLE |PCI_PASID_CTRL_EXEC | +PCI_PASID_CTRL_PRIV ; + +return ctrl; +} +EXPORT_SYMBOL_GPL (pci_pasid_status);#endif /* CONFIG_PCI_PASID */
include/linux/pci-ats.h +3 -0
@@ -42,6 +42,7 @@ int pci_enable_pasid(struct pci_dev *pdev, int features);void pci_disable_pasid (struct pci_dev *pdev);int pci_pasid_features (struct pci_dev *pdev);int pci_max_pasids (struct pci_dev *pdev); +int pci_pasid_status (struct pci_dev *pdev);#else /* CONFIG_PCI_PASID */ static inline int pci_enable_pasid (struct pci_dev *pdev,int features) {return -EINVAL ; }
@@ -50,6 +51,8 @@ static inline int pci_pasid_features(struct pci_dev *pdev) {return -EINVAL ; }static inline int pci_max_pasids (struct pci_dev *pdev) {return -EINVAL ; } +static inline int pci_pasid_status (struct pci_dev *pdev) +{return -EINVAL ; }#endif /* CONFIG_PCI_PASID */ #endif /* LINUX_PCI_ATS_H */
include/uapi/linux/iommufd.h +13 -1
@@ -612,9 +612,17 @@ enum iommu_hw_info_type {* IOMMU_HWPT_GET_DIRTY_BITMAP * IOMMU_HWPT_SET_DIRTY_TRACKING * +* @IOMMU_HW_CAP_PASID_EXEC: Execute Permission Supported, user ignores it +* when the struct iommu_hw_info::out_max_pasid_log2 +* is zero. +* @IOMMU_HW_CAP_PASID_PRIV: Privileged Mode Supported, user ignores it +* when the struct iommu_hw_info::out_max_pasid_log2 +* is zero. */ enum iommufd_hw_capabilities {IOMMU_HW_CAP_DIRTY_TRACKING =1 <<0 , +IOMMU_HW_CAP_PCI_PASID_EXEC =1 <<1 , +IOMMU_HW_CAP_PCI_PASID_PRIV =1 <<2 , };/**
@@ -630,6 +638,9 @@ enum iommufd_hw_capabilities {* iommu_hw_info_type. * @out_capabilities: Output the generic iommu capability info type as defined * in the enum iommu_hw_capabilities. +* @out_max_pasid_log2: Output the width of PASIDs. 0 means no PASID support. +* PCI devices turn to out_capabilities to check if the +* specific capabilities is supported or not. * @__reserved: Must be 0 * * Query an iommu type specific hardware information data from an iommu behind
@@ -653,7 +664,8 @@ struct iommu_hw_info {__u32 data_len;__aligned_u64 data_uptr;__u32 out_data_type; -__u32 __reserved; +__u8 out_max_pasid_log2; +__u8 __reserved[3 ];__aligned_u64 out_capabilities; };#define IOMMU_GET_HW_INFO _IO(IOMMUFD_TYPE, IOMMUFD_CMD_GET_HW_INFO)
Signature · 1 line
2.34.1
PATCH v9 5/5 iommufd/selftest: Add coverage for reporting max_pasid_log2 via IOMMU_HW_INFO
In reply to Yi LiuIOMMU_HW_INFO is extended to report max_pasid_log2, hence add coverage for it.
--- tools/testing/selftests/iommu/iommufd.c | 18 ++++++++++++++++++ .../testing/selftests/iommu/iommufd_fail_nth.c | 3 ++- tools/testing/selftests/iommu/iommufd_utils.h | 17 +++++++++++++---- 3 files changed, 33 insertions(+), 5 deletions(-)
tools/testing/selftests/iommu/iommufd.c +18 -0
@@ -342,12 +342,14 @@ FIXTURE(iommufd_ioas)uint32_t hwpt_id;uint32_t device_id;uint64_t base_iova; +uint32_t device_pasid_id; };FIXTURE_VARIANT (iommufd_ioas) {unsigned int mock_domains;unsigned int memory_limit; +bool pasid_capable; };FIXTURE_SETUP (iommufd_ioas)
@@ -372,6 +374,12 @@ FIXTURE_SETUP(iommufd_ioas) IOMMU_TEST_DEV_CACHE_DEFAULT); self->base_iova =MOCK_APERTURE_START ; } + +if (variant->pasid_capable) +test_cmd_mock_domain_flags (self->ioas_id, +MOCK_FLAGS_DEVICE_PASID , +NULL ,NULL , + &self->device_pasid_id); }FIXTURE_TEARDOWN (iommufd_ioas)
@@ -387,6 +395,7 @@ FIXTURE_VARIANT_ADD(iommufd_ioas, no_domain)FIXTURE_VARIANT_ADD (iommufd_ioas, mock_domain) { .mock_domains = 1, + .pasid_capable = true, };FIXTURE_VARIANT_ADD (iommufd_ioas, two_mock_domain)
@@ -752,6 +761,8 @@ TEST_F(iommufd_ioas, get_hw_info) } buffer_smaller;if (self->device_id) { +uint8_t max_pasid =0 ; +/* Provide a zero-size user_buffer */ test_cmd_get_hw_info (self->device_id,NULL ,0 );/* Provide a user_buffer with exact size */
@@ -766,6 +777,13 @@ TEST_F(iommufd_ioas, get_hw_info)* the fields within the size range still gets updated. */ test_cmd_get_hw_info (self->device_id, &buffer_smaller,sizeof (buffer_smaller)); +test_cmd_get_hw_info_pasid (self->device_id, &max_pasid); +ASSERT_EQ (0 , max_pasid); +if (variant->pasid_capable) { +test_cmd_get_hw_info_pasid (self->device_pasid_id, + &max_pasid); +ASSERT_EQ (MOCK_PASID_WIDTH , max_pasid); + } }else {test_err_get_hw_info (ENOENT , self->device_id, &buffer_exact,sizeof (buffer_exact));
tools/testing/selftests/iommu/iommufd_fail_nth.c +2 -1
@@ -666,7 +666,8 @@ TEST_FAIL_NTH(basic_fail_nth, device) &self->stdev_id, NULL, &idev_id))return -1 ; -if (_test_cmd_get_hw_info (self->fd, idev_id, &info,sizeof (info),NULL )) +if (_test_cmd_get_hw_info (self->fd, idev_id, &info, +sizeof (info),NULL ,NULL ))return -1 ; if (_test_cmd_hwpt_alloc(self->fd, idev_id, ioas_id, 0,
tools/testing/selftests/iommu/iommufd_utils.h +13 -4
@@ -758,7 +758,8 @@ static void teardown_iommufd(int fd, struct __test_metadata *_metadata)/* @data can be NULL */ static int _test_cmd_get_hw_info (int fd,__u32 device_id,void *data, -size_t data_len,uint32_t *capabilities) +size_t data_len,uint32_t *capabilities, +uint8_t *max_pasid) {struct iommu_test_hw_info *info = (struct iommu_test_hw_info *)data;struct iommu_hw_info cmd = {
@@ -803,6 +804,9 @@ static int _test_cmd_get_hw_info(int fd, __u32 device_id, void *data,assert (!info->flags); } +if (max_pasid) + *max_pasid = cmd.out_max_pasid_log2; +if (capabilities) *capabilities = cmd.out_capabilities;
@@ -811,14 +815,19 @@ static int _test_cmd_get_hw_info(int fd, __u32 device_id, void *data,#define test_cmd_get_hw_info (device_id, data, data_len) \ ASSERT_EQ(0, _test_cmd_get_hw_info(self->fd, device_id, data, \ - data_len, NULL)) + data_len, NULL, NULL))#define test_err_get_hw_info (_errno, device_id, data, data_len) \ EXPECT_ERRNO(_errno, _test_cmd_get_hw_info(self->fd, device_id, data, \ - data_len, NULL)) + data_len, NULL, NULL))#define test_cmd_get_hw_capabilities (device_id, caps, mask) \ - ASSERT_EQ(0, _test_cmd_get_hw_info(self->fd, device_id, NULL, 0, &caps)) + ASSERT_EQ(0, _test_cmd_get_hw_info(self->fd, device_id, NULL, \ + 0, &caps, NULL)) + +#define test_cmd_get_hw_info_pasid (device_id, max_pasid) \ + ASSERT_EQ(0, _test_cmd_get_hw_info(self->fd, device_id, NULL, \ + 0, NULL, max_pasid))static int _test_ioctl_fault_alloc (int fd,__u32 *fault_id,__u32 *fault_fd) {
Signature · 1 line
2.34.1
Re: [PATCH v9 5/5] iommufd/selftest: Add coverage for reporting max_pasid_log2 via IOMMU_HW_INFO
Quoted 5 lines · Yi Liu
IOMMU_HW_INFOis extended to reportmax_pasid_log2, hence add coverage for it.Reviewed-by Nicolin Chen <[email protected]>Signed-off-by Yi Liu <[email protected]>
It seems that I was CCed only in two patches of this v9. So, I have to reply to this one.
For the whole series,
With an integration of Yi's QEMU patches, I am able to detect the PASID cap to run vSVA cases in a VM enabling vSMMU.
Selftest also runs correctly.
Thanks
Nicolin
Quoted 7 lines · Yi Liu
Yi Liu (5):
ida: Addida_find_first_range()
vfio-iommufd: Support pasid [at|de]tach for physical VFIO devices
vfio: VFIO_DEVICE_[AT|DE]TACH_IOMMUFD_PT support pasid
iommufd: ExtendIOMMU_GET_HW_INFOto report PASID capability iommufd/selftest: Add coverage for reportingmax_pasid_log2via
IOMMU_HW_INFO
Applied to iommufd for-next, thanks
Jason