Skip to contentSkip to conversation

Add set_dev_data and unset_dev_data support

PATCH New Nicolin Chen 1 participant 5 messages Jul 27, 2023

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v5 cover Add set_dev_data and unset_dev_data support

Changelog
v5:
* Renamed ioctls to IOMMU_SET_DEV_DATA and IOMMU_UNSET_DEV_DATA
* Renamed data structs to iommu_set_dev_data and IOMMU_UNSET_DEV_DATA
* Added missing TEST_LENGTH for those two new data structures
v4:
https://lore.kernel.org/all/[email protected]/
* Rebased on top of v6.4-rc1, and iommufd_nesting-v2 (candidate)
* Dropped WARN_ON in the destroy()
v3:
https://lore.kernel.org/all/[email protected]/
* Reverted the data in VFIO BIND ioctl to a set of new iommufd ioctls
* Replaced the iommu_device_data_size array with a dev_user_data_len variable in the iommu_ops structure.
* Added has_dev_data check and locking protection
* Added selftest coverage
v2:
https://lore.kernel.org/all/[email protected]/
* Integrated the uAPI into VFIO_DEVICE_BIND_IOMMUFD call
* Renamed the previous set_rid_user to set_dev_data, to decouple from the PCI regime.
v1:
https://lore.kernel.org/all/[email protected]/

This is a pair of new uAPI/ops for user space to set an iommu specific device data for a passthrough device. This is primarily used by SMMUv3 driver for now, to link the vSID and the pSID of a device that's behind the SMMU. The link (lookup table) will be used to verify any ATC_INV command from the user space for that device, and then replace the SID field (virtual SID) with the corresponding physical SID.

This series is available on Github:
https://github.com/nicolinc/iommufd/commits/set_dev_data-v5

Thanks!
Nicolin

Nicolin Chen (4):
iommu: Add set/unset_dev_user_data ops
iommufd: Add IOMMUFD_CMD_SET_DEV_DATA and IOMMUFD_CMD_UNSET_DEV_DATA iommufd/selftest: Add IOMMU_TEST_OP_DEV_CHECK_DATA iommufd/selftest: Add coverage for IOMMU_SET/UNSET_DEV_DATA

 drivers/iommu/iommufd/device.c                |  2 +
 drivers/iommu/iommufd/iommufd_private.h       |  1 +
 drivers/iommu/iommufd/iommufd_test.h          | 15 ++++
 drivers/iommu/iommufd/main.c                  | 85 +++++++++++++++++++
 drivers/iommu/iommufd/selftest.c              | 42 +++++++++
 include/linux/iommu.h                         | 13 +++
 include/uapi/linux/iommufd.h                  | 32 +++++++
 tools/testing/selftests/iommu/iommufd.c       | 22 +++++
 tools/testing/selftests/iommu/iommufd_utils.h | 59 +++++++++++++
 9 files changed, 271 insertions(+)
Signature · 1 line
2.41.0
#1 lore raw

PATCH v5 1/4 iommu: Add set/unset_dev_user_data ops

The device behind an IOMMU might be used in the user space by a VM. So, it might have some user space data. For example, a device behind an SMMU has a static stream ID. In a virtualization use case, both a host environment and a guest environment have their own Stream IDs. A link (a lookup table) between the physical Stream ID and the virtual (user) Stream ID is needed when the host handles the user cache invalidation commands.

Add a pair of new ops to allow user space to forward user_data of a device via iommufd, and a new dev_user_data_len for data structure sanity done by the iommufd core.

---
 include/linux/iommu.h | 13 +++++++++++++
 1 file changed, 13 insertions(+)
include/linux/iommu.h +13 -0
index e4835230d5f0..231920efab84 100644
@@ -281,6 +281,15 @@ union iommu_domain_user_data {
  * @remove_dev_pasid: Remove any translation configurations of a specific
  *                    pasid, so that any DMA transactions with this pasid
  *                    will be blocked by the hardware.
+ * @set/unset_dev_user_data: set/unset an iommu specific device data from user
+ *                           space. The user device data info will be used by
+ *                           the driver to take care of user space requests.
+ *                           The device data structure must be defined in
+ *                           include/uapi/linux/iommufd.h.
+ * @dev_user_data_len: Length of the device data from user space (in bytes),
+ *                     simply the "sizeof" the data structure defined in the
+ *                     include/uapi/linux/iommufd.h. This is used by iommufd
+ *                     core to run a data length validation.
  * @hw_info_type: One of enum iommu_hw_info_type defined in
  *                include/uapi/linux/iommufd.h. It is used to tag the type
  *                of data returned by .hw_info callback. The drivers that
@@ -326,6 +335,10 @@ struct iommu_ops {
 	int (*def_domain_type)(struct device *dev);
 	void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid);
 
+	int (*set_dev_user_data)(struct device *dev, const void *user_data);
+	void (*unset_dev_user_data)(struct device *dev);
+	size_t dev_user_data_len;
+
 	const struct iommu_domain_ops *default_domain_ops;
 	enum iommu_hw_info_type hw_info_type;
 	unsigned long pgsize_bitmap;
Signature · 1 line
2.41.0
#2 lore raw

PATCH v5 2/4 iommufd: Add IOMMUFD_CMD_SET_DEV_DATA and IOMMUFD_CMD_UNSET_DEV_DATA

In reply to Nicolin Chen

Add a new pair of ioctls to allow user space to set and unset its iommu- specific device data at the iommu that a passthrough device is behind.

On platforms with SMMUv3, this new uAPIs will be used to forward a user space virtual Stream ID of a passthrough device to link to its physical Stream ID and log into a lookup table, in order for the host kernel to later run sanity on ATC invalidation requests from the user space, with ATC_INV commands that have SID fields (virtual Stream IDs).

---
 drivers/iommu/iommufd/device.c          |  2 +
 drivers/iommu/iommufd/iommufd_private.h |  1 +
 drivers/iommu/iommufd/main.c            | 85 +++++++++++++++++++++++++
 include/uapi/linux/iommufd.h            | 32 ++++++++++
 4 files changed, 120 insertions(+)
drivers/iommu/iommufd/device.c +2 -0
index 7f238c583b61..37a234371645 100644
@@ -136,6 +136,8 @@ void iommufd_device_destroy(struct iommufd_object *obj)
 	struct iommufd_device *idev =
 		container_of(obj, struct iommufd_device, obj);
 
+	if (idev->has_user_data)
+		dev_iommu_ops(idev->dev)->unset_dev_user_data(idev->dev);
 	iommu_device_release_dma_owner(idev->dev);
 	iommufd_put_group(idev->igroup);
 	if (!iommufd_selftest_is_mock_dev(idev->dev))
drivers/iommu/iommufd/iommufd_private.h +1 -0
index 9ae6edea697f..b8abf664d3f8 100644
@@ -313,6 +313,7 @@ struct iommufd_device {
 	/* always the physical device */
 	struct device *dev;
 	bool enforce_cache_coherency;
+	bool has_user_data;
 };
 
 static inline struct iommufd_device *
drivers/iommu/iommufd/main.c +85 -0

103 lines not on this page · raw message

include/uapi/linux/iommufd.h +32 -0
index 034da283cd3a..ede822e5acbb 100644
@@ -49,6 +49,8 @@ enum {
 	IOMMUFD_CMD_GET_HW_INFO,
 	IOMMUFD_CMD_RESV_IOVA_RANGES,
 	IOMMUFD_CMD_HWPT_INVALIDATE,
+	IOMMUFD_CMD_SET_DEV_DATA,
+	IOMMUFD_CMD_UNSET_DEV_DATA,
 };
 
 /**
@@ -508,4 +510,34 @@ struct iommu_hwpt_invalidate {
 	__aligned_u64 data_uptr;
 };
 #define IOMMU_HWPT_INVALIDATE _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_INVALIDATE)
+
+/**
+ * struct iommu_set_dev_data - ioctl(IOMMU_SET_DEV_DATA)
+ * @size: sizeof(struct iommu_set_dev_data)
+ * @dev_id: The device to set an iommu specific device data
+ * @data_uptr: User pointer of the device user data
+ * @data_len: Length of the device user data
+ *
+ * The device data must be unset using ioctl(IOMMU_UNSET_DEV_DATA), before
+ * another ioctl(IOMMU_SET_DEV_DATA) call or before the device itself gets
+ * unbind'd from the iommufd context.
+ */
+struct iommu_set_dev_data {
+	__u32 size;
+	__u32 dev_id;
+	__aligned_u64 data_uptr;
+	__u32 data_len;
+};
+#define IOMMU_SET_DEV_DATA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_SET_DEV_DATA)
+
+/**
+ * struct iommu_unset_dev_data - ioctl(IOMMU_UNSET_DEV_DATA)
+ * @size: sizeof(struct iommu_unset_dev_data)
+ * @dev_id: The device to unset its device user data
+ */
+struct iommu_unset_dev_data {
+	__u32 size;
+	__u32 dev_id;
+};
+#define IOMMU_UNSET_DEV_DATA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_UNSET_DEV_DATA)
 #endif
Signature · 1 line
2.41.0
#3 lore raw

PATCH v5 3/4 iommufd/selftest: Add IOMMU_TEST_OP_DEV_CHECK_DATA

In reply to Nicolin Chen

Add mock_domain_set/unset_dev_user_data and iommufd_test_dev_check_data to allow testing on IOMMUFD_CMD_SET/UNSET_DEV_DATA ioctls from the user space selftest via IOMMU_TEST_OP_DEV_CHECK_DATA.

---
 drivers/iommu/iommufd/iommufd_test.h | 15 ++++++++++
 drivers/iommu/iommufd/selftest.c     | 42 ++++++++++++++++++++++++++++
 2 files changed, 57 insertions(+)
drivers/iommu/iommufd/iommufd_test.h +15 -0
index 9768d0c9e347..b191d7417719 100644
@@ -22,6 +22,7 @@ enum {
 	IOMMU_TEST_OP_DEV_ADD_RESERVED,
 	IOMMU_TEST_OP_DEV_DEL_RESERVED,
 	IOMMU_TEST_OP_MD_CHECK_IOTLB,
+	IOMMU_TEST_OP_DEV_CHECK_DATA,
 };
 
 enum {
@@ -105,6 +106,9 @@ struct iommu_test_cmd {
 		struct {
 			__u32 iotlb;
 		} check_iotlb;
+		struct {
+			__u32 val;
+		} check_dev_data;
 	};
 	__u32 last;
 };
@@ -119,6 +123,17 @@ struct iommu_test_hw_info {
 	__u32 test_reg;
 };
 
+#define IOMMU_DEVICE_DATA_SELFTEST	0xdadbeef
+
+/**
+ * struct iommu_test_device_data
+ *
+ * @val: Should be set to IOMMU_DEVICE_DATA_SELFTEST or unset to 0x0
+ */
+struct iommu_test_device_data {
+	__u32 val;
+};
+
 /* Should not be equal to any defined value in enum iommu_hwpt_type */
 #define IOMMU_HWPT_TYPE_SELFTTEST		0xdead
 
drivers/iommu/iommufd/selftest.c +42 -0

72 lines not on this page · raw message

Signature · 1 line
2.41.0
#4 lore raw

PATCH v5 4/4 iommufd/selftest: Add coverage for IOMMU_SET/UNSET_DEV_DATA

In reply to Nicolin Chen

Add a new IOMMU_TEST_OP_DEV_CHECK_DATA to verify whether the dev_data is properly set/unset by the IOMMU_SET/UNSET_DEV_DATA.

---
 tools/testing/selftests/iommu/iommufd.c       | 22 +++++++
 tools/testing/selftests/iommu/iommufd_utils.h | 59 +++++++++++++++++++
 2 files changed, 81 insertions(+)
tools/testing/selftests/iommu/iommufd.c +22 -0
index bf0082d88a38..b437b890e4f5 100644
@@ -123,6 +123,8 @@ TEST_F(iommufd, cmd_length)
 	TEST_LENGTH(iommu_ioas_copy, IOMMU_IOAS_COPY);
 	TEST_LENGTH(iommu_ioas_unmap, IOMMU_IOAS_UNMAP);
 	TEST_LENGTH(iommu_option, IOMMU_OPTION);
+	TEST_LENGTH(iommu_set_dev_data, IOMMU_SET_DEV_DATA);
+	TEST_LENGTH(iommu_unset_dev_data, IOMMU_UNSET_DEV_DATA);
 	TEST_LENGTH(iommu_vfio_ioas, IOMMU_VFIO_IOAS);
 #undef TEST_LENGTH
 }
@@ -1517,6 +1519,26 @@ TEST_F(iommufd_mock_domain, alloc_hwpt)
 	}
 }
 
+TEST_F(iommufd_mock_domain, set_dev_data)
+{
+	struct iommu_test_device_data dev_data = {
+		.val = IOMMU_DEVICE_DATA_SELFTEST,
+	};
+	int i;
+
+	for (i = 0; i != variant->mock_domains; i++) {
+		test_err_set_dev_data(ENOENT, 0, &dev_data);
+		test_err_set_dev_data(EINVAL, self->idev_ids[i], NULL);
+		test_cmd_set_dev_data(self->idev_ids[i], &dev_data);
+		test_err_set_dev_data(EEXIST, self->idev_ids[i], &dev_data);
+		test_cmd_dev_check_data(self->idev_ids[i], dev_data.val);
+		test_err_unset_dev_data(ENOENT, 0);
+		test_cmd_unset_dev_data(self->idev_ids[i]);
+		test_err_unset_dev_data(ENOENT, self->idev_ids[i]);
+		test_cmd_dev_check_data(self->idev_ids[i], 0);
+	}
+}
+
 /* VFIO compatibility IOCTLs */
 
 TEST_F(iommufd, simple_ioctls)
tools/testing/selftests/iommu/iommufd_utils.h +59 -0

62 lines not on this page · raw message

Signature · 1 line
2.41.0