Skip to contentSkip to conversation

iommu: Prepare to deliver page faults to user space

PATCH Superseded Lu Baolu 3 participants 70 messages Jul 27, 2023 – Aug 13, 2023

Subsystems (5)

v2 of 13 Superseded Latest is v13 →

lore
Share & tools
lore

Lu Baolu

author maintainer
#0 lore raw

v2 cover iommu: Prepare to deliver page faults to user space

When a user-managed page table is attached to an IOMMU, it is necessary to deliver IO page faults to user space so that they can be handled appropriately. One use case for this is nested translation, which is currently being discussed in the mailing list.

I have posted a RFC series [1] that describes the implementation of delivering page faults to user space through IOMMUFD. This series has received several comments on the IOMMU refactoring, which I have addressed in this series.

The major refactoring includes:

- [PATCH 01 ~ 03] Move include/uapi/linux/iommu.h to include/linux/iommu.h. Remove the unrecoverable fault data definition.
- [PATCH 04 ~ 07] Remove iommu_[un]register_device_fault_handler().
- [PATCH 08 ~ 12] Separate SVA and IOPF. Make IOPF a generic page fault handling framework.

This is also available at github [2]. I would appreciate your feedback on this series.

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://github.com/LuBaolu/intel-iommu/commits/preparatory-io-pgfault-delivery-v2

Change log:
v2:
- Remove unrecoverable fault data definition as suggested by Kevin.
- Drop the per-device fault cookie code considering that doesn't make much sense for SVA.
- Make the IOMMU page fault handling framework generic. So that it can avaible for use cases other than SVA.

v1: https://lore.kernel.org/linux-iommu/[email protected]/

Lu Baolu (12):
iommu: Move iommu fault data to linux/iommu.h iommu/arm-smmu-v3: Remove unrecoverable faults reporting
iommu: Remove unrecoverable fault data
iommu: Replace device fault handler with iommu_queue_iopf()
iommu: Change the return value of dev_iommu_get()
iommu: Make dev->fault_param static
iommu: Remove iommu_[un]register_device_fault_handler()
iommu: Prepare for separating SVA and IOPF
iommu: Move iopf_handler() to iommu-sva.c
iommu: Make iommu_queue_iopf() more generic
iommu: Separate SVA and IOPF in Makefile and Kconfig
iommu: Add helper to set iopf handler for domain

 include/linux/iommu.h                         | 196 +++++++++++++++---
 drivers/iommu/iommu-sva.h                     |  71 -------
 include/uapi/linux/iommu.h                    | 161 --------------
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  14 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |  47 ++---
 drivers/iommu/intel/iommu.c                   |  19 +-
 drivers/iommu/intel/svm.c                     |   1 -
 drivers/iommu/io-pgfault.c                    |  90 +++-----
 drivers/iommu/iommu-sva.c                     |  47 ++++-
 drivers/iommu/iommu.c                         | 153 ++++----------
 MAINTAINERS                                   |   1 -
 drivers/iommu/Kconfig                         |   4 +
 drivers/iommu/Makefile                        |   3 +-
 drivers/iommu/intel/Kconfig                   |   1 +
 14 files changed, 314 insertions(+), 494 deletions(-)
 delete mode 100644 drivers/iommu/iommu-sva.h
 delete mode 100644 include/uapi/linux/iommu.h
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v2 1/12 iommu: Move iommu fault data to linux/iommu.h

The iommu fault data is currently defined in uapi/linux/iommu.h, but is only used inside the iommu subsystem. Move it to linux/iommu.h, where it will be more accessible to kernel drivers.

With this done, uapi/linux/iommu.h becomes empty and can be removed from the tree.

---
 include/linux/iommu.h      | 152 +++++++++++++++++++++++++++++++++-
 include/uapi/linux/iommu.h | 161 -------------------------------------
 MAINTAINERS                |   1 -
 3 files changed, 151 insertions(+), 163 deletions(-)
 delete mode 100644 include/uapi/linux/iommu.h
include/linux/iommu.h +151 -1
index b1dcb1b9b170..b3537736f9cb 100644
@@ -13,7 +13,6 @@
 #include <linux/errno.h>
 #include <linux/err.h>
 #include <linux/of.h>
-#include <uapi/linux/iommu.h>
 
 #define IOMMU_READ	(1 << 0)
 #define IOMMU_WRITE	(1 << 1)
@@ -42,6 +41,157 @@ struct iommu_sva;
 struct iommu_fault_event;
 struct iommu_dma_cookie;
 
+#define IOMMU_FAULT_PERM_READ	(1 << 0) /* read */
+#define IOMMU_FAULT_PERM_WRITE	(1 << 1) /* write */
+#define IOMMU_FAULT_PERM_EXEC	(1 << 2) /* exec */
+#define IOMMU_FAULT_PERM_PRIV	(1 << 3) /* privileged */
+
+/* Generic fault types, can be expanded IRQ remapping fault */
+enum iommu_fault_type {
+	IOMMU_FAULT_DMA_UNRECOV = 1,	/* unrecoverable fault */
+	IOMMU_FAULT_PAGE_REQ,		/* page request fault */
+};
+
+enum iommu_fault_reason {
+	IOMMU_FAULT_REASON_UNKNOWN = 0,
+
+	/* Could not access the PASID table (fetch caused external abort) */
+	IOMMU_FAULT_REASON_PASID_FETCH,
+
+	/* PASID entry is invalid or has configuration errors */
+	IOMMU_FAULT_REASON_BAD_PASID_ENTRY,
+
+	/*
+	 * PASID is out of range (e.g. exceeds the maximum PASID
+	 * supported by the IOMMU) or disabled.
+	 */
+	IOMMU_FAULT_REASON_PASID_INVALID,
+
+	/*
+	 * An external abort occurred fetching (or updating) a translation
+	 * table descriptor
+	 */
+	IOMMU_FAULT_REASON_WALK_EABT,
+
+	/*
+	 * Could not access the page table entry (Bad address),
+	 * actual translation fault
+	 */
+	IOMMU_FAULT_REASON_PTE_FETCH,
+
+	/* Protection flag check failed */
+	IOMMU_FAULT_REASON_PERMISSION,
+
+	/* access flag check failed */
+	IOMMU_FAULT_REASON_ACCESS,
+
+	/* Output address of a translation stage caused Address Size fault */
+	IOMMU_FAULT_REASON_OOR_ADDRESS,
+};
+
+/**
+ * struct iommu_fault_unrecoverable - Unrecoverable fault data
+ * @reason: reason of the fault, from &enum iommu_fault_reason
+ * @flags: parameters of this fault (IOMMU_FAULT_UNRECOV_* values)
+ * @pasid: Process Address Space ID
+ * @perm: requested permission access using by the incoming transaction
+ *        (IOMMU_FAULT_PERM_* values)
+ * @addr: offending page address
+ * @fetch_addr: address that caused a fetch abort, if any
+ */
+struct iommu_fault_unrecoverable {
+	__u32	reason;
+#define IOMMU_FAULT_UNRECOV_PASID_VALID		(1 << 0)
+#define IOMMU_FAULT_UNRECOV_ADDR_VALID		(1 << 1)
+#define IOMMU_FAULT_UNRECOV_FETCH_ADDR_VALID	(1 << 2)
+	__u32	flags;
+	__u32	pasid;
+	__u32	perm;
+	__u64	addr;
+	__u64	fetch_addr;
+};
+
+/**
+ * struct iommu_fault_page_request - Page Request data
+ * @flags: encodes whether the corresponding fields are valid and whether this
+ *         is the last page in group (IOMMU_FAULT_PAGE_REQUEST_* values).
+ *         When IOMMU_FAULT_PAGE_RESPONSE_NEEDS_PASID is set, the page response
+ *         must have the same PASID value as the page request. When it is clear,
+ *         the page response should not have a PASID.
+ * @pasid: Process Address Space ID
+ * @grpid: Page Request Group Index
+ * @perm: requested page permissions (IOMMU_FAULT_PERM_* values)
+ * @addr: page address
+ * @private_data: device-specific private information
+ */
+struct iommu_fault_page_request {
+#define IOMMU_FAULT_PAGE_REQUEST_PASID_VALID	(1 << 0)
+#define IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE	(1 << 1)
+#define IOMMU_FAULT_PAGE_REQUEST_PRIV_DATA	(1 << 2)
+#define IOMMU_FAULT_PAGE_RESPONSE_NEEDS_PASID	(1 << 3)
+	__u32	flags;
+	__u32	pasid;
+	__u32	grpid;
+	__u32	perm;
+	__u64	addr;
+	__u64	private_data[2];
+};
+
+/**
+ * struct iommu_fault - Generic fault data
+ * @type: fault type from &enum iommu_fault_type
+ * @padding: reserved for future use (should be zero)
+ * @event: fault event, when @type is %IOMMU_FAULT_DMA_UNRECOV
+ * @prm: Page Request message, when @type is %IOMMU_FAULT_PAGE_REQ
+ * @padding2: sets the fault size to allow for future extensions
+ */
+struct iommu_fault {
+	__u32	type;
+	__u32	padding;
+	union {
+		struct iommu_fault_unrecoverable event;
+		struct iommu_fault_page_request prm;
+		__u8 padding2[56];
+	};
+};
+
+/**
+ * enum iommu_page_response_code - Return status of fault handlers
+ * @IOMMU_PAGE_RESP_SUCCESS: Fault has been handled and the page tables
+ *	populated, retry the access. This is "Success" in PCI PRI.
+ * @IOMMU_PAGE_RESP_FAILURE: General error. Drop all subsequent faults from
+ *	this device if possible. This is "Response Failure" in PCI PRI.
+ * @IOMMU_PAGE_RESP_INVALID: Could not handle this fault, don't retry the
+ *	access. This is "Invalid Request" in PCI PRI.
+ */
+enum iommu_page_response_code {
+	IOMMU_PAGE_RESP_SUCCESS = 0,
+	IOMMU_PAGE_RESP_INVALID,
+	IOMMU_PAGE_RESP_FAILURE,
+};
+
+/**
+ * struct iommu_page_response - Generic page response information
+ * @argsz: User filled size of this data
+ * @version: API version of this structure
+ * @flags: encodes whether the corresponding fields are valid
+ *         (IOMMU_FAULT_PAGE_RESPONSE_* values)
+ * @pasid: Process Address Space ID
+ * @grpid: Page Request Group Index
+ * @code: response code from &enum iommu_page_response_code
+ */
+struct iommu_page_response {
+	__u32	argsz;
+#define IOMMU_PAGE_RESP_VERSION_1	1
+	__u32	version;
+#define IOMMU_PAGE_RESP_PASID_VALID	(1 << 0)
+	__u32	flags;
+	__u32	pasid;
+	__u32	grpid;
+	__u32	code;
+};
+
+
 /* iommu fault flags */
 #define IOMMU_FAULT_READ	0x0
 #define IOMMU_FAULT_WRITE	0x1
include/uapi/linux/iommu.h deleted +0 -161

161 lines not on this page · raw message

MAINTAINERS +0 -1
index 3be1bdfe8ecc..8fded0298961 100644
@@ -10877,7 +10877,6 @@ F:	drivers/iommu/
 F:	include/linux/iommu.h
 F:	include/linux/iova.h
 F:	include/linux/of_iommu.h
-F:	include/uapi/linux/iommu.h
 
 IOMMUFD
 M:	Jason Gunthorpe <[email protected]>
Signature · 1 line
2.34.1
#2 lore raw

RE: [PATCH v2 01/12] iommu: Move iommu fault data to linux/iommu.h

Quoted 11 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

The iommu fault data is currently defined in uapi/linux/iommu.h, but is only used inside the iommu subsystem. Move it to linux/iommu.h, where it will be more accessible to kernel drivers.

With this done, uapi/linux/iommu.h becomes empty and can be removed from
the tree.

Quoted 8 lines
---
 include/linux/iommu.h      | 152 +++++++++++++++++++++++++++++++++-
 include/uapi/linux/iommu.h | 161 -------------------------------------
 MAINTAINERS                |   1 -
 3 files changed, 151 insertions(+), 163 deletions(-)
 delete mode 100644 include/uapi/linux/iommu.h

put this behind patch2/3 then there are less lines to be moved.

author maintainer
#3 lore raw

Re: [PATCH v2 01/12] iommu: Move iommu fault data to linux/iommu.h

Quoted 20 lines · Kevin
Quoted 11 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

The iommu fault data is currently defined in uapi/linux/iommu.h, but is only used inside the iommu subsystem. Move it to linux/iommu.h, where it will be more accessible to kernel drivers.

With this done, uapi/linux/iommu.h becomes empty and can be removed from
the tree.

Quoted 8 lines
---
  include/linux/iommu.h      | 152 +++++++++++++++++++++++++++++++++-
  include/uapi/linux/iommu.h | 161 -------------------------------------
  MAINTAINERS                |   1 -
  3 files changed, 151 insertions(+), 163 deletions(-)

delete mode 100644 include/uapi/linux/iommu.h

Quoted 1 line

put this behind patch2/3 then there are less lines to be moved.

My idea was that "include/uapi/linux.h" is not used anywhere and we no longer want to maintain it. Hence, we should remove it before changing any lines inside it.

Best regards,
baolu

Reviewed-by on v2 1/12
#4 lore raw

Re: [PATCH v2 01/12] iommu: Move iommu fault data to linux/iommu.h

In reply to Lu Baolu
Quoted 7 lines · Lu Baolu

The iommu fault data is currently defined in uapi/linux/iommu.h, but is only used inside the iommu subsystem. Move it to linux/iommu.h, where it will be more accessible to kernel drivers.

With this done, uapi/linux/iommu.h becomes empty and can be removed from the tree.

Quoted 7 lines
---
 include/linux/iommu.h      | 152 +++++++++++++++++++++++++++++++++-
 include/uapi/linux/iommu.h | 161 -------------------------------------
 MAINTAINERS                |   1 -
 3 files changed, 151 insertions(+), 163 deletions(-)
 delete mode 100644 include/uapi/linux/iommu.h

Jason

author maintainer
#5 lore raw

PATCH v2 2/12 iommu/arm-smmu-v3: Remove unrecoverable faults reporting

In reply to Lu Baolu

No device driver registers fault handler to handle the reported unrecoveraable faults. Remove it to avoid dead code.

---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 46 ++++++---------------
 1 file changed, 13 insertions(+), 33 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +13 -33
index 9b0dc3505601..1a43f904b6b8 100644
@@ -1469,7 +1469,6 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid)
 static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
 {
 	int ret;
-	u32 reason;
 	u32 perm = 0;
 	struct arm_smmu_master *master;
 	bool ssid_valid = evt[0] & EVTQ_0_SSV;
@@ -1479,16 +1478,9 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
 
 	switch (FIELD_GET(EVTQ_0_ID, evt[0])) {
 	case EVT_ID_TRANSLATION_FAULT:
-		reason = IOMMU_FAULT_REASON_PTE_FETCH;
-		break;
 	case EVT_ID_ADDR_SIZE_FAULT:
-		reason = IOMMU_FAULT_REASON_OOR_ADDRESS;
-		break;
 	case EVT_ID_ACCESS_FAULT:
-		reason = IOMMU_FAULT_REASON_ACCESS;
-		break;
 	case EVT_ID_PERMISSION_FAULT:
-		reason = IOMMU_FAULT_REASON_PERMISSION;
 		break;
 	default:
 		return -EOPNOTSUPP;
@@ -1498,6 +1490,9 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
 	if (evt[1] & EVTQ_1_S2)
 		return -EFAULT;
 
+	if (!(evt[1] & EVTQ_1_STALL))
+		return -EOPNOTSUPP;
+
 	if (evt[1] & EVTQ_1_RnW)
 		perm |= IOMMU_FAULT_PERM_READ;
 	else
@@ -1509,32 +1504,17 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
 	if (evt[1] & EVTQ_1_PnU)
 		perm |= IOMMU_FAULT_PERM_PRIV;
 
-	if (evt[1] & EVTQ_1_STALL) {
-		flt->type = IOMMU_FAULT_PAGE_REQ;
-		flt->prm = (struct iommu_fault_page_request) {
-			.flags = IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE,
-			.grpid = FIELD_GET(EVTQ_1_STAG, evt[1]),
-			.perm = perm,
-			.addr = FIELD_GET(EVTQ_2_ADDR, evt[2]),
-		};
+	flt->type = IOMMU_FAULT_PAGE_REQ;
+	flt->prm = (struct iommu_fault_page_request) {
+		.flags = IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE,
+		.grpid = FIELD_GET(EVTQ_1_STAG, evt[1]),
+		.perm = perm,
+		.addr = FIELD_GET(EVTQ_2_ADDR, evt[2]),
+	};
 
-		if (ssid_valid) {
-			flt->prm.flags |= IOMMU_FAULT_PAGE_REQUEST_PASID_VALID;
-			flt->prm.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]);
-		}
-	} else {
-		flt->type = IOMMU_FAULT_DMA_UNRECOV;
-		flt->event = (struct iommu_fault_unrecoverable) {
-			.reason = reason,
-			.flags = IOMMU_FAULT_UNRECOV_ADDR_VALID,
-			.perm = perm,
-			.addr = FIELD_GET(EVTQ_2_ADDR, evt[2]),
-		};
-
-		if (ssid_valid) {
-			flt->event.flags |= IOMMU_FAULT_UNRECOV_PASID_VALID;
-			flt->event.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]);
-		}
+	if (ssid_valid) {
+		flt->prm.flags |= IOMMU_FAULT_PAGE_REQUEST_PASID_VALID;
+		flt->prm.pasid = FIELD_GET(EVTQ_0_SSID, evt[0]);
 	}
 
 	mutex_lock(&smmu->streams_mutex);
Signature · 1 line
2.34.1
author maintainer
#6 lore raw

PATCH v2 3/12 iommu: Remove unrecoverable fault data

In reply to Lu Baolu

The unrecoverable fault data is not used anywhere. Remove it to avoid dead code.

---
 include/linux/iommu.h | 70 +------------------------------------------
 1 file changed, 1 insertion(+), 69 deletions(-)
include/linux/iommu.h +1 -69
index b3537736f9cb..a1e4390d05a8 100644
@@ -48,69 +48,9 @@ struct iommu_dma_cookie;
 
 /* Generic fault types, can be expanded IRQ remapping fault */
 enum iommu_fault_type {
-	IOMMU_FAULT_DMA_UNRECOV = 1,	/* unrecoverable fault */
 	IOMMU_FAULT_PAGE_REQ,		/* page request fault */
 };
 
-enum iommu_fault_reason {
-	IOMMU_FAULT_REASON_UNKNOWN = 0,
-
-	/* Could not access the PASID table (fetch caused external abort) */
-	IOMMU_FAULT_REASON_PASID_FETCH,
-
-	/* PASID entry is invalid or has configuration errors */
-	IOMMU_FAULT_REASON_BAD_PASID_ENTRY,
-
-	/*
-	 * PASID is out of range (e.g. exceeds the maximum PASID
-	 * supported by the IOMMU) or disabled.
-	 */
-	IOMMU_FAULT_REASON_PASID_INVALID,
-
-	/*
-	 * An external abort occurred fetching (or updating) a translation
-	 * table descriptor
-	 */
-	IOMMU_FAULT_REASON_WALK_EABT,
-
-	/*
-	 * Could not access the page table entry (Bad address),
-	 * actual translation fault
-	 */
-	IOMMU_FAULT_REASON_PTE_FETCH,
-
-	/* Protection flag check failed */
-	IOMMU_FAULT_REASON_PERMISSION,
-
-	/* access flag check failed */
-	IOMMU_FAULT_REASON_ACCESS,
-
-	/* Output address of a translation stage caused Address Size fault */
-	IOMMU_FAULT_REASON_OOR_ADDRESS,
-};
-
-/**
- * struct iommu_fault_unrecoverable - Unrecoverable fault data
- * @reason: reason of the fault, from &enum iommu_fault_reason
- * @flags: parameters of this fault (IOMMU_FAULT_UNRECOV_* values)
- * @pasid: Process Address Space ID
- * @perm: requested permission access using by the incoming transaction
- *        (IOMMU_FAULT_PERM_* values)
- * @addr: offending page address
- * @fetch_addr: address that caused a fetch abort, if any
- */
-struct iommu_fault_unrecoverable {
-	__u32	reason;
-#define IOMMU_FAULT_UNRECOV_PASID_VALID		(1 << 0)
-#define IOMMU_FAULT_UNRECOV_ADDR_VALID		(1 << 1)
-#define IOMMU_FAULT_UNRECOV_FETCH_ADDR_VALID	(1 << 2)
-	__u32	flags;
-	__u32	pasid;
-	__u32	perm;
-	__u64	addr;
-	__u64	fetch_addr;
-};
-
 /**
  * struct iommu_fault_page_request - Page Request data
  * @flags: encodes whether the corresponding fields are valid and whether this
@@ -140,19 +80,11 @@ struct iommu_fault_page_request {
 /**
  * struct iommu_fault - Generic fault data
  * @type: fault type from &enum iommu_fault_type
- * @padding: reserved for future use (should be zero)
- * @event: fault event, when @type is %IOMMU_FAULT_DMA_UNRECOV
  * @prm: Page Request message, when @type is %IOMMU_FAULT_PAGE_REQ
- * @padding2: sets the fault size to allow for future extensions
  */
 struct iommu_fault {
 	__u32	type;
-	__u32	padding;
-	union {
-		struct iommu_fault_unrecoverable event;
-		struct iommu_fault_page_request prm;
-		__u8 padding2[56];
-	};
+	struct iommu_fault_page_request prm;
 };
 
 /**
Signature · 1 line
2.34.1
#7 lore raw

RE: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 6 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

struct iommu_fault {

include/linux/iommu.h +0 -1
 	__u32	type;
-	__u32	padding;

this padding should be kept.

Quoted 7 lines
include/linux/iommu.h +1 -5
-	union {
-		struct iommu_fault_unrecoverable event;
-		struct iommu_fault_page_request prm;
-		__u8 padding2[56];
-	};
+	struct iommu_fault_page_request prm;
 };
author maintainer
#8 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 8 lines · Kevin

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 6 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

  struct iommu_fault {
  	__u32	type;
include/linux/iommu.h +0 -1
-	__u32	padding;

this padding should be kept.

To keep above 64-bit aligned, right?

I can't think of any other reason to keep it.

Best regards,
baolu

#9 lore raw

RE: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 7 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 10:59 AM

Quoted 3 lines · Kevin
Quoted 3 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

Quoted 8 lines
Quoted 5 lines

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 3 lines
  struct iommu_fault {
  	__u32	type;
include/linux/iommu.h +0 -1
-	__u32	padding;

this padding should be kept.

To keep above 64-bit aligned, right?

yes

author maintainer
#10 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 13 lines · Kevin
Quoted 13 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 10:59 AM

Quoted 8 lines · Kevin

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 6 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

   struct iommu_fault {
   	__u32	type;
include/linux/iommu.h +0 -1
-	__u32	padding;

this padding should be kept.

Quoted 4 lines
Quoted 2 lines

To keep above 64-bit aligned, right?

yes

Okay, thanks!

Best regards,
baolu

#11 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

In reply to Tian, Kevin
Quoted 13 lines · Kevin
Quoted 13 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 10:59 AM

Quoted 8 lines · Kevin

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 6 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

  struct iommu_fault {
  	__u32	type;
include/linux/iommu.h +0 -1
-	__u32	padding;

this padding should be kept.

Quoted 4 lines
Quoted 2 lines

To keep above 64-bit aligned, right?

yes

If it is not uapi we should not explicitly document padding (and __u32 should be u32). The compiler will add it if it is necessary.

If the compiler isn't right for some reason then something else has gone wrong.

Jason

#12 lore raw

RE: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 20 lines

From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, August 9, 2023 2:40 AM

Quoted 16 lines · Kevin
Quoted 15 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 10:59 AM

Quoted 8 lines · Kevin

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 6 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

  struct iommu_fault {
  	__u32	type;
include/linux/iommu.h +0 -1
-	__u32	padding;

this padding should be kept.

To keep above 64-bit aligned, right?

Quoted 8 lines
Quoted 1 line

yes

If it is not uapi we should not explicitly document padding (and __u32 should be u32). The compiler will add it if it is necessary.

If the compiler isn't right for some reason then something else has gone wrong.

I thought this will be used as uAPI later. I'm fine to leave it be and add the padding when the uAPI is introduced.

#13 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 25 lines · Kevin
Quoted 25 lines

From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, August 9, 2023 2:40 AM

Quoted 13 lines · Kevin
Quoted 13 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 10:59 AM

Quoted 8 lines · Kevin

include/linux/iommu.h struct iommu_fault_page_request { · line 150

Quoted 6 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

  struct iommu_fault {
  	__u32	type;
include/linux/iommu.h +0 -1
-	__u32	padding;

this padding should be kept.

Quoted 4 lines
Quoted 2 lines

To keep above 64-bit aligned, right?

yes

If it is not uapi we should not explicitly document padding (and __u32 should be u32). The compiler will add it if it is necessary.

Quoted 6 lines
Quoted 3 lines

If the compiler isn't right for some reason then something else has gone wrong.

I thought this will be used as uAPI later. I'm fine to leave it be and add the padding when the uAPI is introduced.

Yes

Jason

Reviewed-by on v2 3/12
#14 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

In reply to Lu Baolu
Quoted 8 lines · Lu Baolu

The unrecoverable fault data is not used anywhere. Remove it to avoid dead code.

---
 include/linux/iommu.h | 70 +------------------------------------------
 1 file changed, 1 insertion(+), 69 deletions(-)

Do we plan to bring this back in some form? A driver specific fault report via iommufd?

Jason

author maintainer
#15 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 11 lines · Jason Gunthorpe
Quoted 8 lines · Lu Baolu

The unrecoverable fault data is not used anywhere. Remove it to avoid dead code.

---
  include/linux/iommu.h | 70 +------------------------------------------
  1 file changed, 1 insertion(+), 69 deletions(-)

Do we plan to bring this back in some form? A driver specific fault report via iommufd?

I can hardly see the possibility.

The only necessary dma fault messages are the offending address and the permissions. With these, the user space device model software knows that "a DMA fault was generated when the IOMMU hardware tried to translate the offending address with the given permissions".

And then, the device model software will walk the page table and figure out what is missed before injecting the vendor-specific fault messages to the VM guest.

Best regards,
baolu

#16 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 15 lines · Baolu Lu
Quoted 11 lines · Jason Gunthorpe
Quoted 8 lines · Lu Baolu

The unrecoverable fault data is not used anywhere. Remove it to avoid dead code.

---
  include/linux/iommu.h | 70 +------------------------------------------
  1 file changed, 1 insertion(+), 69 deletions(-)

Do we plan to bring this back in some form? A driver specific fault report via iommufd?

I can hardly see the possibility.

Quoted 8 lines

The only necessary dma fault messages are the offending address and the permissions. With these, the user space device model software knows that "a DMA fault was generated when the IOMMU hardware tried to translate the offending address with the given permissions".

And then, the device model software will walk the page table and figure out what is missed before injecting the vendor-specific fault messages to the VM guest.

Avoiding walking the page table sounds like a pretty big win if we could manage it by forwarding more event data..

Jason

author maintainer
#17 lore raw

Re: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 20 lines · Jason Gunthorpe
Quoted 19 lines · Baolu Lu
Quoted 11 lines · Jason Gunthorpe
Quoted 8 lines · Lu Baolu

The unrecoverable fault data is not used anywhere. Remove it to avoid dead code.

---
   include/linux/iommu.h | 70 +------------------------------------------
   1 file changed, 1 insertion(+), 69 deletions(-)

Do we plan to bring this back in some form? A driver specific fault report via iommufd?

I can hardly see the possibility.

The only necessary dma fault messages are the offending address and the permissions. With these, the user space device model software knows that "a DMA fault was generated when the IOMMU hardware tried to translate the offending address with the given permissions".

Quoted 5 lines
Quoted 3 lines

And then, the device model software will walk the page table and figure out what is missed before injecting the vendor-specific fault messages to the VM guest.

Avoiding walking the page table sounds like a pretty big win if we could manage it by forwarding more event data..

Fair enough. We can discuss what kind of extra event data could be included later when we have real code for dma fault forwarding support in iommufd.

Best regards,
baolu

#18 lore raw

RE: [PATCH v2 03/12] iommu: Remove unrecoverable fault data

Quoted 28 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 11, 2023 9:16 AM

Quoted 11 lines · Jason Gunthorpe
Quoted 10 lines · Baolu Lu
Quoted 9 lines · Jason Gunthorpe
Quoted 8 lines · Lu Baolu

The unrecoverable fault data is not used anywhere. Remove it to avoid dead code.

---
   include/linux/iommu.h | 70 +------------------------------------------
   1 file changed, 1 insertion(+), 69 deletions(-)
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines

Do we plan to bring this back in some form? A driver specific fault report via iommufd?

I can hardly see the possibility.

The only necessary dma fault messages are the offending address and the permissions. With these, the user space device model software knows

that

Quoted 6 lines
Quoted 6 lines

"a DMA fault was generated when the IOMMU hardware tried to translate the offending address with the given permissions".

And then, the device model software will walk the page table and figure out what is missed before injecting the vendor-specific fault messages to the VM guest.

Quoted 7 lines
Quoted 2 lines

Avoiding walking the page table sounds like a pretty big win if we could manage it by forwarding more event data..

Fair enough. We can discuss what kind of extra event data could be included later when we have real code for dma fault forwarding support in iommufd.

I'm afraid there might be cases where VMM cannot rely on the hw event data. e.g. if there is a misconfiguation in virtual context entry or ste then vIOMMU may decide to unmap and leave IOAS empty to trigger unrecoverable fault when any DMA comes. Then upon notification vIOMMU needs to check virtual context entry/ste to decide the right error code when injecting unrecoverable fault into the guest. Here the hw event data will be about missing stage-2 mapping hence incorrect.

and this is not a performance critical path. 😊

author maintainer
#19 lore raw

PATCH v2 4/12 iommu: Replace device fault handler with iommu_queue_iopf()

In reply to Lu Baolu

The individual iommu drivers report iommu faults by calling iommu_report_device_fault(), where a pre-registered device fault handler is called to route the fault to another fault handler installed on the corresponding iommu domain.

The pre-registered device fault handler is static and won't be dynamic as the fault handler is eventually per iommu domain. Replace calling device fault handler with iommu_queue_iopf().

---
 drivers/iommu/iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/iommu.c +1 -1
index 4352a149a935..00309f66153b 100644
@@ -1381,7 +1381,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
 		mutex_unlock(&fparam->lock);
 	}
 
-	ret = fparam->handler(&evt->fault, fparam->data);
+	ret = iommu_queue_iopf(&evt->fault, dev);
 	if (ret && evt_pending) {
 		mutex_lock(&fparam->lock);
 		list_del(&evt_pending->list);
Signature · 1 line
2.34.1
Reviewed-by on v2 4/12
#20 lore raw

RE: [PATCH v2 04/12] iommu: Replace device fault handler with iommu_queue_iopf()

Quoted 8 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:48 PM

The individual iommu drivers report iommu faults by calling iommu_report_device_fault(), where a pre-registered device fault handler is called to route the fault to another fault handler installed on the corresponding iommu domain.

Quoted 5 lines

The pre-registered device fault handler is static and won't be dynamic as the fault handler is eventually per iommu domain. Replace calling device fault handler with iommu_queue_iopf().

#21 lore raw

Re: [PATCH v2 04/12] iommu: Replace device fault handler with iommu_queue_iopf()

In reply to Lu Baolu
Quoted 19 lines · Lu Baolu

The individual iommu drivers report iommu faults by calling iommu_report_device_fault(), where a pre-registered device fault handler is called to route the fault to another fault handler installed on the corresponding iommu domain.

The pre-registered device fault handler is static and won't be dynamic as the fault handler is eventually per iommu domain. Replace calling device fault handler with iommu_queue_iopf().

---
 drivers/iommu/iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/iommu.c +0 -0
index 4352a149a935..00309f66153b 100644

drivers/iommu/iommu.c int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) · line 1387

Quoted 8 lines
drivers/iommu/iommu.c +1 -1
@@ -1381,7 +1381,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
 		mutex_unlock(&fparam->lock);
 	}
 
-	ret = fparam->handler(&evt->fault, fparam->data);
+	ret = iommu_queue_iopf(&evt->fault, dev);
 	if (ret && evt_pending) {
 		mutex_lock(&fparam->lock);
 		list_del(&evt_pending->list);

I don't get it, why not remove fparam->handler/data entirely in this patch? There is no user once you do this change?

Jason

author maintainer
#22 lore raw

Re: [PATCH v2 04/12] iommu: Replace device fault handler with iommu_queue_iopf()

Quoted 23 lines · Jason Gunthorpe
Quoted 22 lines · Lu Baolu

The individual iommu drivers report iommu faults by calling iommu_report_device_fault(), where a pre-registered device fault handler is called to route the fault to another fault handler installed on the corresponding iommu domain.

The pre-registered device fault handler is static and won't be dynamic as the fault handler is eventually per iommu domain. Replace calling device fault handler with iommu_queue_iopf().

---
  drivers/iommu/iommu.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/iommu.c +0 -0
index 4352a149a935..00309f66153b 100644
@@ -1381,7 +1381,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
  		mutex_unlock(&fparam->lock);
  	}
 
Quoted 7 lines

drivers/iommu/iommu.c int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) · line 1387

Quoted 5 lines
drivers/iommu/iommu.c +1 -1
-	ret = fparam->handler(&evt->fault, fparam->data);
+	ret = iommu_queue_iopf(&evt->fault, dev);
  	if (ret && evt_pending) {
  		mutex_lock(&fparam->lock);
  		list_del(&evt_pending->list);

I don't get it, why not remove fparam->handler/data entirely in this patch? There is no user once you do this change?

It needs some cleanups elsewhere, so I put it in a separate patch.

Best regards,
baolu

#23 lore raw

Re: [PATCH v2 04/12] iommu: Replace device fault handler with iommu_queue_iopf()

In reply to Lu Baolu
Quoted 19 lines · Lu Baolu

The individual iommu drivers report iommu faults by calling iommu_report_device_fault(), where a pre-registered device fault handler is called to route the fault to another fault handler installed on the corresponding iommu domain.

The pre-registered device fault handler is static and won't be dynamic as the fault handler is eventually per iommu domain. Replace calling device fault handler with iommu_queue_iopf().

---
 drivers/iommu/iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/iommu.c +0 -0
index 4352a149a935..00309f66153b 100644

drivers/iommu/iommu.c int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) · line 1384

Quoted 5 lines
drivers/iommu/iommu.c +1 -1
@@ -1381,7 +1381,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
 		mutex_unlock(&fparam->lock);
 	}
 
-	ret = fparam->handler(&evt->fault, fparam->data);
+	ret = iommu_queue_iopf(&evt->fault, dev);

Also fix the function signature at this point:

int iommu_queue_iopf(struct iommu_fault *fault, void *cookie)

It should not be 'void *cookie' anymore, it is just 'struct device *dev'

Jason

author maintainer
#24 lore raw

Re: [PATCH v2 04/12] iommu: Replace device fault handler with iommu_queue_iopf()

Quoted 23 lines · Jason Gunthorpe
Quoted 22 lines · Lu Baolu

The individual iommu drivers report iommu faults by calling iommu_report_device_fault(), where a pre-registered device fault handler is called to route the fault to another fault handler installed on the corresponding iommu domain.

The pre-registered device fault handler is static and won't be dynamic as the fault handler is eventually per iommu domain. Replace calling device fault handler with iommu_queue_iopf().

---
  drivers/iommu/iommu.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/iommu.c +0 -0
index 4352a149a935..00309f66153b 100644
@@ -1381,7 +1381,7 @@ int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
  		mutex_unlock(&fparam->lock);
  	}
 
Quoted 7 lines

drivers/iommu/iommu.c int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) · line 1384

Quoted 2 lines
drivers/iommu/iommu.c +1 -1
-	ret = fparam->handler(&evt->fault, fparam->data);
+	ret = iommu_queue_iopf(&evt->fault, dev);

Also fix the function signature at this point:

int iommu_queue_iopf(struct iommu_fault *fault, void *cookie)

It should not be 'void *cookie' anymore, it is just 'struct device *dev'

I have included this change in the subsequent cleanup patch.

Best regards,
baolu

author maintainer
#25 lore raw

PATCH v2 5/12 iommu: Change the return value of dev_iommu_get()

In reply to Lu Baolu

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
 drivers/iommu/iommu.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +11 -8
index 00309f66153b..4ba3bb692993 100644
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct iommu_device *iommu)
 }
 EXPORT_SYMBOL_GPL(iommu_device_unregister);
 
-static struct dev_iommu *dev_iommu_get(struct device *dev)
+static int dev_iommu_get(struct device *dev)
 {
 	struct dev_iommu *param = dev->iommu;
 
 	if (param)
-		return param;
+		return 0;
 
 	param = kzalloc(sizeof(*param), GFP_KERNEL);
 	if (!param)
-		return NULL;
+		return -ENOMEM;
 
 	mutex_init(&param->lock);
 	dev->iommu = param;
-	return param;
+	return 0;
 }
 
 static void dev_iommu_free(struct device *dev)
@@ -346,8 +346,9 @@ static int iommu_init_device(struct device *dev, const struct iommu_ops *ops)
 	struct iommu_group *group;
 	int ret;
 
-	if (!dev_iommu_get(dev))
-		return -ENOMEM;
+	ret = dev_iommu_get(dev);
+	if (ret)
+		return ret;
 
 	if (!try_module_get(ops->owner)) {
 		ret = -EINVAL;
@@ -2743,12 +2744,14 @@ int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode,
 		      const struct iommu_ops *ops)
 {
 	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	int ret;
 
 	if (fwspec)
 		return ops == fwspec->ops ? 0 : -EINVAL;
 
-	if (!dev_iommu_get(dev))
-		return -ENOMEM;
+	ret = dev_iommu_get(dev);
+	if (ret)
+		return ret;
 
 	/* Preallocate for the overwhelmingly common case of 1 ID */
 	fwspec = kzalloc(struct_size(fwspec, ids, 1), GFP_KERNEL);
Signature · 1 line
2.34.1
#26 lore raw

RE: [PATCH v2 05/12] iommu: Change the return value of dev_iommu_get()

Quoted 31 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
 drivers/iommu/iommu.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +2 -2
index 00309f66153b..4ba3bb692993 100644
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct
 iommu_device *iommu)
 }
 EXPORT_SYMBOL_GPL(iommu_device_unregister);
 
-static struct dev_iommu *dev_iommu_get(struct device *dev)
+static int dev_iommu_get(struct device *dev)
 {
 	struct dev_iommu *param = dev->iommu;
 
 	if (param)
-		return param;
+		return 0;
 
 	param = kzalloc(sizeof(*param), GFP_KERNEL);

drivers/iommu/iommu.c void iommu_device_unregister(struct · line 309

Quoted 10 lines
drivers/iommu/iommu.c +2 -2
 	if (!param)
-		return NULL;
+		return -ENOMEM;
 
 	mutex_init(&param->lock);
 	dev->iommu = param;
-	return param;
+	return 0;
 }
 

Jason's series [1] has been queued. Time to refine according to the discussion in [2].

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/

author maintainer
#27 lore raw

Re: [PATCH v2 05/12] iommu: Change the return value of dev_iommu_get()

Quoted 45 lines · Kevin
Quoted 31 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
  drivers/iommu/iommu.c | 19 +++++++++++--------
  1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +2 -2
index 00309f66153b..4ba3bb692993 100644
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct
 iommu_device *iommu)
  }
  EXPORT_SYMBOL_GPL(iommu_device_unregister);
 
-static struct dev_iommu *dev_iommu_get(struct device *dev)
+static int dev_iommu_get(struct device *dev)
  {
  	struct dev_iommu *param = dev->iommu;
 
  	if (param)
-		return param;
+		return 0;
 
  	param = kzalloc(sizeof(*param), GFP_KERNEL);

drivers/iommu/iommu.c void iommu_device_unregister(struct · line 309

Quoted 10 lines
drivers/iommu/iommu.c +2 -2
  	if (!param)
-		return NULL;
+		return -ENOMEM;
 
  	mutex_init(&param->lock);
  	dev->iommu = param;
-	return param;
+	return 0;
  }
 

Jason's series [1] has been queued. Time to refine according to the discussion in [2].

Quoted 2 lines

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/

I'm not sure I understand your point here. This only changes the return value of dev_iommu_get() to make the code more concise.

Best regards,
baolu

#28 lore raw

RE: [PATCH v2 05/12] iommu: Change the return value of dev_iommu_get()

Quoted 49 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 11:10 AM

Quoted 45 lines · Kevin
Quoted 31 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
  drivers/iommu/iommu.c | 19 +++++++++++--------
  1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +2 -2
index 00309f66153b..4ba3bb692993 100644
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct
 iommu_device *iommu)
  }
  EXPORT_SYMBOL_GPL(iommu_device_unregister);
 
-static struct dev_iommu *dev_iommu_get(struct device *dev)
+static int dev_iommu_get(struct device *dev)
  {
  	struct dev_iommu *param = dev->iommu;
 
  	if (param)
-		return param;
+		return 0;
 
  	param = kzalloc(sizeof(*param), GFP_KERNEL);

drivers/iommu/iommu.c void iommu_device_unregister(struct · line 309

Quoted 10 lines
drivers/iommu/iommu.c +2 -2
  	if (!param)
-		return NULL;
+		return -ENOMEM;
 
  	mutex_init(&param->lock);
  	dev->iommu = param;
-	return param;
+	return 0;
  }
 

Jason's series [1] has been queued. Time to refine according to the discussion in [2].

Quoted 7 lines
Quoted 2 lines

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/c815fa2b-00df-91e1-8353-

[email protected]/

I'm not sure I understand your point here. This only changes the return value of dev_iommu_get() to make the code more concise.

I thought the purpose of this patch was to prepare for next patch which moves dev->fault_param initialization to dev_iommu_get().

with Jason's rework IMHO that initialization more fits in iommu_init_device().

that's my real point. If you still want to clean up dev_iommu_get() it's fine but then it may not belong to this series. 😊

author maintainer
#29 lore raw

Re: [PATCH v2 05/12] iommu: Change the return value of dev_iommu_get()

Quoted 50 lines · Kevin
Quoted 50 lines

From: Baolu Lu<[email protected]>
Sent: Friday, August 4, 2023 11:10 AM

Quoted 46 lines · Kevin
Quoted 31 lines

From: Lu Baolu<[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
   drivers/iommu/iommu.c | 19 +++++++++++--------
   1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +2 -2
index 00309f66153b..4ba3bb692993 100644
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct
 iommu_device *iommu)
   }
   EXPORT_SYMBOL_GPL(iommu_device_unregister);
 
-static struct dev_iommu *dev_iommu_get(struct device *dev)
+static int dev_iommu_get(struct device *dev)
   {
   	struct dev_iommu *param = dev->iommu;
 
   	if (param)
-		return param;
+		return 0;
 
   	param = kzalloc(sizeof(*param), GFP_KERNEL);

drivers/iommu/iommu.c void iommu_device_unregister(struct · line 309

Quoted 10 lines
drivers/iommu/iommu.c +2 -2
   	if (!param)
-		return NULL;
+		return -ENOMEM;
 
   	mutex_init(&param->lock);
   	dev->iommu = param;
-	return param;
+	return 0;
   }
 

Jason's series [1] has been queued. Time to refine according to the discussion in [2].

[1]https://lore.kernel.org/linux-iommu/[email protected]/ [2]https://lore.kernel.org/linux-iommu/c815fa2b-00df-91e1-8353-

Quoted 7 lines
Quoted 5 lines

[email protected]/

I'm not sure I understand your point here. This only changes the return value of dev_iommu_get() to make the code more concise.

I thought the purpose of this patch was to prepare for next patch which moves dev->fault_param initialization to dev_iommu_get().

Yes.

Quoted 5 lines

with Jason's rework IMHO that initialization more fits in iommu_init_device().

that's my real point. If you still want to clean up dev_iommu_get() it's fine but then it may not belong to this series. 😊

Ah, I see. Let me make a choice in the next version.

Best regards,
baolu

#30 lore raw

Re: [PATCH v2 05/12] iommu: Change the return value of dev_iommu_get()

In reply to Lu Baolu
Quoted 14 lines · Lu Baolu

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
 drivers/iommu/iommu.c | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +0 -0
index 00309f66153b..4ba3bb692993 100644

drivers/iommu/iommu.c void iommu_device_unregister(struct iommu_device *iommu) · line 291

Quoted 2 lines
drivers/iommu/iommu.c +0 -0
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct iommu_device *iommu)
 }
 EXPORT_SYMBOL_GPL(iommu_device_unregister);

It could probably use a nicer name too?

iommu_alloc_dev_iommu() ?

Also with Joerg's current tree we can add a device_lock_assert() to this function, from what I can tell.

Jason

author maintainer
#31 lore raw

Re: [PATCH v2 05/12] iommu: Change the return value of dev_iommu_get()

Quoted 19 lines · Jason Gunthorpe
Quoted 14 lines · Lu Baolu

Make dev_iommu_get() return 0 for success and error numbers for failure. This will make the code neat and readable. No functionality changes.

---
  drivers/iommu/iommu.c | 19 +++++++++++--------
  1 file changed, 11 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +0 -0
index 00309f66153b..4ba3bb692993 100644

drivers/iommu/iommu.c void iommu_device_unregister(struct iommu_device *iommu) · line 291

Quoted 2 lines
drivers/iommu/iommu.c +0 -0
@@ -290,20 +290,20 @@ void iommu_device_unregister(struct iommu_device *iommu)
  }
  EXPORT_SYMBOL_GPL(iommu_device_unregister);

It could probably use a nicer name too?

Quoted 4 lines

iommu_alloc_dev_iommu() ?

Also with Joerg's current tree we can add a device_lock_assert() to this function, from what I can tell.

Sure. Will update them in the next version.

Best regards,
baolu

author maintainer
#32 lore raw

PATCH v2 6/12 iommu: Make dev->fault_param static

In reply to Lu Baolu

The iommu faults, including recoverable faults (IO page faults) and unrecoverable faults (DMA faults), are generic to all devices. The iommu faults could possibly be triggered for every device.

The fault_param pointer under struct dev_iommu is the per-device fault data. Therefore, the fault_param pointer could be static and allocated during iommu device probe and freed when the device is released.

With this done, the individual iommu drivers that support iopf have no need to call iommu_[un]register_device_fault_handler() any more. This will make it easier for the iommu drivers to support iopf, and it will also make the fault_param allocation and free simpler.

---
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c    | 13 +------------
 drivers/iommu/intel/iommu.c                    | 18 ++++--------------
 drivers/iommu/iommu.c                          | 14 ++++++++++++++
 3 files changed, 19 insertions(+), 26 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +1 -12
index a5a63b1c947e..fa8ab9d413f8 100644
@@ -437,7 +437,6 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master)
 
 static int arm_smmu_master_sva_enable_iopf(struct arm_smmu_master *master)
 {
-	int ret;
 	struct device *dev = master->dev;
 
 	/*
@@ -450,16 +449,7 @@ static int arm_smmu_master_sva_enable_iopf(struct arm_smmu_master *master)
 	if (!master->iopf_enabled)
 		return -EINVAL;
 
-	ret = iopf_queue_add_device(master->smmu->evtq.iopf, dev);
-	if (ret)
-		return ret;
-
-	ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev);
-	if (ret) {
-		iopf_queue_remove_device(master->smmu->evtq.iopf, dev);
-		return ret;
-	}
-	return 0;
+	return iopf_queue_add_device(master->smmu->evtq.iopf, dev);
 }
 
 static void arm_smmu_master_sva_disable_iopf(struct arm_smmu_master *master)
@@ -469,7 +459,6 @@ static void arm_smmu_master_sva_disable_iopf(struct arm_smmu_master *master)
 	if (!master->iopf_enabled)
 		return;
 
-	iommu_unregister_device_fault_handler(dev);
 	iopf_queue_remove_device(master->smmu->evtq.iopf, dev);
 }
 
drivers/iommu/intel/iommu.c +4 -14
index d5ca2387e65c..e94e833638a7 100644
@@ -4587,23 +4587,14 @@ static int intel_iommu_enable_iopf(struct device *dev)
 	if (ret)
 		return ret;
 
-	ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev);
-	if (ret)
-		goto iopf_remove_device;
-
 	ret = pci_enable_pri(pdev, PRQ_DEPTH);
-	if (ret)
-		goto iopf_unregister_handler;
+	if (ret) {
+		iopf_queue_remove_device(iommu->iopf_queue, dev);
+		return ret;
+	}
 	info->pri_enabled = 1;
 
 	return 0;
-
-iopf_unregister_handler:
-	iommu_unregister_device_fault_handler(dev);
-iopf_remove_device:
-	iopf_queue_remove_device(iommu->iopf_queue, dev);
-
-	return ret;
 }
 
 static int intel_iommu_disable_iopf(struct device *dev)
@@ -4630,7 +4621,6 @@ static int intel_iommu_disable_iopf(struct device *dev)
 	 * fault handler and removing device from iopf queue should never
 	 * fail.
 	 */
-	WARN_ON(iommu_unregister_device_fault_handler(dev));
 	WARN_ON(iopf_queue_remove_device(iommu->iopf_queue, dev));
 
 	return 0;
drivers/iommu/iommu.c +14 -0
index 4ba3bb692993..3e4ff984aa85 100644
@@ -302,7 +302,15 @@ static int dev_iommu_get(struct device *dev)
 		return -ENOMEM;
 
 	mutex_init(&param->lock);
+	param->fault_param = kzalloc(sizeof(*param->fault_param), GFP_KERNEL);
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);
 	dev->iommu = param;
+
 	return 0;
 }
 
@@ -315,6 +323,12 @@ static void dev_iommu_free(struct device *dev)
 		fwnode_handle_put(param->fwspec->iommu_fwnode);
 		kfree(param->fwspec);
 	}
+	/*
+	 * All pending faults should have been drained before
+	 * device release.
+	 */
+	WARN_ON_ONCE(!list_empty(&param->fault_param->faults));
+	kfree(param->fault_param);
 	kfree(param);
 }
 
Signature · 1 line
2.34.1
#33 lore raw

RE: [PATCH v2 06/12] iommu: Make dev->fault_param static

drivers/iommu/intel/iommu.c static int intel_iommu_disable_iopf(struct device · line 4625

Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

drivers/iommu/intel/iommu.c +0 -1
@@ -4630,7 +4621,6 @@ static int intel_iommu_disable_iopf(struct device
 *dev)
 	 * fault handler and removing device from iopf queue should never
 	 * fail.
 	 */
-	WARN_ON(iommu_unregister_device_fault_handler(dev));
 	WARN_ON(iopf_queue_remove_device(iommu->iopf_queue, dev));

the comment should be updated too.

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 311

Quoted 10 lines
(quoted diff) +1 -0
 	mutex_init(&param->lock);
+	param->fault_param = kzalloc(sizeof(*param->fault_param),

GFP_KERNEL);

drivers/iommu/iommu.c +6 -0
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);

let's also move 'partial' from struct iopf_device_param into struct iommu_fault_param. That logic is not specific to sva.

meanwhile probably iopf_device_param can be renamed to iopf_sva_param since all the remaining fields are only used by the sva handler.

current naming (iommu_fault_param vs. iopf_device_param) is a bit confusing when reading related code.

author maintainer
#34 lore raw

Re: [PATCH v2 06/12] iommu: Make dev->fault_param static

Quoted 12 lines · Kevin

drivers/iommu/intel/iommu.c static int intel_iommu_disable_iopf(struct device · line 4625

Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

drivers/iommu/intel/iommu.c +0 -1
@@ -4630,7 +4621,6 @@ static int intel_iommu_disable_iopf(struct device
 *dev)
  	 * fault handler and removing device from iopf queue should never
  	 * fail.
  	 */
-	WARN_ON(iommu_unregister_device_fault_handler(dev));
  	WARN_ON(iopf_queue_remove_device(iommu->iopf_queue, dev));

the comment should be updated too.

Ack.

Quoted 15 lines

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 311

Quoted 11 lines
(quoted diff) +1 -0
  	mutex_init(&param->lock);
+	param->fault_param = kzalloc(sizeof(*param->fault_param),

GFP_KERNEL);

drivers/iommu/iommu.c +6 -0
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);

let's also move 'partial' from struct iopf_device_param into struct iommu_fault_param. That logic is not specific to sva.

Quoted 6 lines

meanwhile probably iopf_device_param can be renamed to iopf_sva_param since all the remaining fields are only used by the sva handler.

current naming (iommu_fault_param vs. iopf_device_param) is a bit confusing when reading related code.

My understanding is that iommu_fault_param is for all kinds of iommu faults. Currently they probably include recoverable IO page faults or unrecoverable DMA faults.

While, iopf_device_param is for the recoverable IO page faults. I agree that this naming is not specific and even confusing. Perhaps renaming it to something like iommu_iopf_param?

Best regards,
baolu

#35 lore raw

RE: [PATCH v2 06/12] iommu: Make dev->fault_param static

Quoted 28 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 11:17 AM

Quoted 17 lines · Kevin

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 311

Quoted 13 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

  	mutex_init(&param->lock);
(quoted diff) +1 -0
+	param->fault_param = kzalloc(sizeof(*param->fault_param),

GFP_KERNEL);

drivers/iommu/iommu.c +6 -0
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);

let's also move 'partial' from struct iopf_device_param into struct iommu_fault_param. That logic is not specific to sva.

Quoted 6 lines

meanwhile probably iopf_device_param can be renamed to iopf_sva_param since all the remaining fields are only used by the sva handler.

current naming (iommu_fault_param vs. iopf_device_param) is a bit confusing when reading related code.

Quoted 8 lines

My understanding is that iommu_fault_param is for all kinds of iommu faults. Currently they probably include recoverable IO page faults or unrecoverable DMA faults.

While, iopf_device_param is for the recoverable IO page faults. I agree that this naming is not specific and even confusing. Perhaps renaming it to something like iommu_iopf_param?

or just iopf_param...

author maintainer
#36 lore raw

Re: [PATCH v2 06/12] iommu: Make dev->fault_param static

Quoted 32 lines · Kevin
Quoted 32 lines

From: Baolu Lu <[email protected]>
Sent: Friday, August 4, 2023 11:17 AM

Quoted 17 lines · Kevin

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 311

Quoted 13 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

   	mutex_init(&param->lock);
(quoted diff) +1 -0
+	param->fault_param = kzalloc(sizeof(*param->fault_param),

GFP_KERNEL);

drivers/iommu/iommu.c +6 -0
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);

let's also move 'partial' from struct iopf_device_param into struct iommu_fault_param. That logic is not specific to sva.

Quoted 6 lines

meanwhile probably iopf_device_param can be renamed to iopf_sva_param since all the remaining fields are only used by the sva handler.

current naming (iommu_fault_param vs. iopf_device_param) is a bit confusing when reading related code.

My understanding is that iommu_fault_param is for all kinds of iommu faults. Currently they probably include recoverable IO page faults or unrecoverable DMA faults.

Quoted 6 lines
Quoted 4 lines

While, iopf_device_param is for the recoverable IO page faults. I agree that this naming is not specific and even confusing. Perhaps renaming it to something like iommu_iopf_param?

or just iopf_param.

Okay.

Best regards,
baolu

#37 lore raw

Re: [PATCH v2 06/12] iommu: Make dev->fault_param static

In reply to Lu Baolu
Quoted 6 lines · Lu Baolu
drivers/iommu/iommu.c +0 -0
index 4ba3bb692993..3e4ff984aa85 100644
@@ -302,7 +302,15 @@ static int dev_iommu_get(struct device *dev)
 		return -ENOMEM;

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 312

Quoted 10 lines
drivers/iommu/iommu.c +7 -0
 
 	mutex_init(&param->lock);
+	param->fault_param = kzalloc(sizeof(*param->fault_param), GFP_KERNEL);
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);
 	dev->iommu = param;

This allocation seems pointless?

If we always allocate the fault param then just don't make it a pointer in the first place.

The appeal of allocation would be to save a few bytes in the common case that the driver doesn't support faulting.

Which means the driver needs to make some call to enable faulting for a device. In this case I'd continue to lazy free on release like this patch does.

Jason

#38 lore raw

Re: [PATCH v2 06/12] iommu: Make dev->fault_param static

Quoted 23 lines · Jason Gunthorpe
Quoted 6 lines · Lu Baolu
drivers/iommu/iommu.c +0 -0
index 4ba3bb692993..3e4ff984aa85 100644
@@ -302,7 +302,15 @@ static int dev_iommu_get(struct device *dev)
 		return -ENOMEM;

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 312

Quoted 10 lines
drivers/iommu/iommu.c +7 -0
 
 	mutex_init(&param->lock);
+	param->fault_param = kzalloc(sizeof(*param->fault_param), GFP_KERNEL);
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);
 	dev->iommu = param;

This allocation seems pointless?

If we always allocate the fault param then just don't make it a pointer in the first place.

Quoted 6 lines

The appeal of allocation would be to save a few bytes in the common case that the driver doesn't support faulting.

Which means the driver needs to make some call to enable faulting for a device. In this case I'd continue to lazy free on release like this patch does.

For instance allocate the fault_param in iopf_queue_add_device() which is the only thing that needs it.

Actually probably just merge struct iopf_device_param and iommu_fault_param ?

When you call iopf_queue_add_device() it enables page faulting mode, does 1 additional allocation for all additional required per-device memory and thats it.

Jason

author maintainer
#39 lore raw

Re: [PATCH v2 06/12] iommu: Make dev->fault_param static

Quoted 32 lines · Jason Gunthorpe
Quoted 22 lines · Jason Gunthorpe
Quoted 6 lines · Lu Baolu
drivers/iommu/iommu.c +0 -0
index 4ba3bb692993..3e4ff984aa85 100644
@@ -302,7 +302,15 @@ static int dev_iommu_get(struct device *dev)
  		return -ENOMEM;

drivers/iommu/iommu.c static int dev_iommu_get(struct device *dev) · line 312

Quoted 10 lines
drivers/iommu/iommu.c +7 -0
 
  	mutex_init(&param->lock);
+	param->fault_param = kzalloc(sizeof(*param->fault_param), GFP_KERNEL);
+	if (!param->fault_param) {
+		kfree(param);
+		return -ENOMEM;
+	}
+	mutex_init(&param->fault_param->lock);
+	INIT_LIST_HEAD(&param->fault_param->faults);
  	dev->iommu = param;

This allocation seems pointless?

If we always allocate the fault param then just don't make it a pointer in the first place.

Quoted 6 lines

The appeal of allocation would be to save a few bytes in the common case that the driver doesn't support faulting.

Which means the driver needs to make some call to enable faulting for a device. In this case I'd continue to lazy free on release like this patch does.

For instance allocate the fault_param in iopf_queue_add_device() which is the only thing that needs it.

Quoted 6 lines

Actually probably just merge struct iopf_device_param and iommu_fault_param ?

When you call iopf_queue_add_device() it enables page faulting mode, does 1 additional allocation for all additional required per-device memory and thats it.

Agreed.

I originally kept the iommu_fault_param structure because I thought it could also be used to store temporary data for unrecoverable faults, just like the iopf_device_param structure is used for iopf. However, I am not sure whether we actually need any temporary data for unrecoverable fault forwarding, which doesn't require any response.

So, it's better to do like you suggested. It's cleaner and simpler.

Best regards,
baolu

author maintainer
#40 lore raw

PATCH v2 7/12 iommu: Remove iommu_[un]register_device_fault_handler()

In reply to Lu Baolu

The pair of interfaces are not used anywhere in the tree. Remove it to avoid dead code.

---
 include/linux/iommu.h      | 23 ---------
 drivers/iommu/iommu-sva.h  |  4 +-
 drivers/iommu/io-pgfault.c |  6 +--
 drivers/iommu/iommu.c      | 96 --------------------------------------
 4 files changed, 4 insertions(+), 125 deletions(-)
include/linux/iommu.h +0 -23
index a1e4390d05a8..cb12bab38365 100644
@@ -130,7 +130,6 @@ struct iommu_page_response {
 
 typedef int (*iommu_fault_handler_t)(struct iommu_domain *,
 			struct device *, unsigned long, int, void *);
-typedef int (*iommu_dev_fault_handler_t)(struct iommu_fault *, void *);
 
 struct iommu_domain_geometry {
 	dma_addr_t aperture_start; /* First address that can be mapped    */
@@ -469,14 +468,10 @@ struct iommu_fault_event {
 
 /**
  * struct iommu_fault_param - per-device IOMMU fault data
- * @handler: Callback function to handle IOMMU faults at device level
- * @data: handler private data
  * @faults: holds the pending faults which needs response
  * @lock: protect pending faults list
  */
 struct iommu_fault_param {
-	iommu_dev_fault_handler_t handler;
-	void *data;
 	struct list_head faults;
 	struct mutex lock;
 };
@@ -602,11 +597,6 @@ extern int iommu_group_for_each_dev(struct iommu_group *group, void *data,
 extern struct iommu_group *iommu_group_get(struct device *dev);
 extern struct iommu_group *iommu_group_ref_get(struct iommu_group *group);
 extern void iommu_group_put(struct iommu_group *group);
-extern int iommu_register_device_fault_handler(struct device *dev,
-					iommu_dev_fault_handler_t handler,
-					void *data);
-
-extern int iommu_unregister_device_fault_handler(struct device *dev);
 
 extern int iommu_report_device_fault(struct device *dev,
 				     struct iommu_fault_event *evt);
@@ -988,19 +978,6 @@ static inline void iommu_group_put(struct iommu_group *group)
 {
 }
 
-static inline
-int iommu_register_device_fault_handler(struct device *dev,
-					iommu_dev_fault_handler_t handler,
-					void *data)
-{
-	return -ENODEV;
-}
-
-static inline int iommu_unregister_device_fault_handler(struct device *dev)
-{
-	return 0;
-}
-
 static inline
 int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt)
 {
drivers/iommu/iommu-sva.h +2 -2
index 54946b5a7caf..de7819c796ce 100644
@@ -13,7 +13,7 @@ struct iommu_fault;
 struct iopf_queue;
 
 #ifdef CONFIG_IOMMU_SVA
-int iommu_queue_iopf(struct iommu_fault *fault, void *cookie);
+int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev);
 
 int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev);
 int iopf_queue_remove_device(struct iopf_queue *queue,
@@ -26,7 +26,7 @@ enum iommu_page_response_code
 iommu_sva_handle_iopf(struct iommu_fault *fault, void *data);
 
 #else /* CONFIG_IOMMU_SVA */
-static inline int iommu_queue_iopf(struct iommu_fault *fault, void *cookie)
+static inline int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 {
 	return -ENODEV;
 }
drivers/iommu/io-pgfault.c +2 -4
index e5b8b9110c13..7e6697083f9d 100644
@@ -103,7 +103,7 @@ static void iopf_handler(struct work_struct *work)
 /**
  * iommu_queue_iopf - IO Page Fault handler
  * @fault: fault event
- * @cookie: struct device, passed to iommu_register_device_fault_handler.
+ * @dev: struct device.
  *
  * Add a fault to the device workqueue, to be handled by mm.
  *
@@ -140,14 +140,12 @@ static void iopf_handler(struct work_struct *work)
  *
  * Return: 0 on success and <0 on error.
  */
-int iommu_queue_iopf(struct iommu_fault *fault, void *cookie)
+int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 {
 	int ret;
 	struct iopf_group *group;
 	struct iopf_fault *iopf, *next;
 	struct iopf_device_param *iopf_param;
-
-	struct device *dev = cookie;
 	struct dev_iommu *param = dev->iommu;
 
 	lockdep_assert_held(&param->lock);
drivers/iommu/iommu.c +0 -96

108 lines not on this page · raw message

Signature · 1 line
2.34.1
Reviewed-by on v2 7/12
#42 lore raw

Re: [PATCH v2 07/12] iommu: Remove iommu_[un]register_device_fault_handler()

In reply to Lu Baolu
Quoted 10 lines · Lu Baolu

The pair of interfaces are not used anywhere in the tree. Remove it to avoid dead code.

---
 include/linux/iommu.h      | 23 ---------
 drivers/iommu/iommu-sva.h  |  4 +-
 drivers/iommu/io-pgfault.c |  6 +--
 drivers/iommu/iommu.c      | 96 --------------------------------------
 4 files changed, 4 insertions(+), 125 deletions(-)

Some of these hunks should be moved earlier, but

Happy to see it go

Jason

author maintainer
#43 lore raw

PATCH v2 8/12 iommu: Prepare for separating SVA and IOPF

In reply to Lu Baolu

Move iopf_group data structure to iommu.h. This is being done to make it a minimal set of faults that a domain's page fault handler should handle.

Add two new helpers for the domain's page fault handler:
- iopf_free_group: free a fault group after all faults in the group are handled.
- iopf_queue_work: queue a given work item for a fault group.

---
 include/linux/iommu.h      | 12 ++++++++++
 drivers/iommu/io-pgfault.c | 48 ++++++++++++++++++++++----------------
 2 files changed, 40 insertions(+), 20 deletions(-)
include/linux/iommu.h +12 -0
index cb12bab38365..607740e548f2 100644
@@ -503,6 +503,18 @@ struct dev_iommu {
 	u32				pci_32bit_workaround:1;
 };
 
+struct iopf_fault {
+	struct iommu_fault		fault;
+	struct list_head		list;
+};
+
+struct iopf_group {
+	struct iopf_fault		last_fault;
+	struct list_head		faults;
+	struct work_struct		work;
+	struct device			*dev;
+};
+
 int iommu_device_register(struct iommu_device *iommu,
 			  const struct iommu_ops *ops,
 			  struct device *hwdev);
drivers/iommu/io-pgfault.c +28 -20

90 lines not on this page · raw message

Signature · 1 line
2.34.1
#44 lore raw

RE: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 17 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

drivers/iommu/io-pgfault.c +1 -1
@@ -82,7 +82,7 @@ static void iopf_handler(struct work_struct *work)
 	if (!domain || !domain->iopf_handler)
 		status = IOMMU_PAGE_RESP_INVALID;
 
-	list_for_each_entry_safe(iopf, next, &group->faults, list) {
+	list_for_each_entry(iopf, &group->faults, list) {
 		/*
 		 * For the moment, errors are sticky: don't handle

subsequent

drivers/iommu/io-pgfault.c +0 -0
 		 * faults in the group if there is an error.
@@ -90,14 +90,20 @@ static void iopf_handler(struct work_struct *work)
 		if (status == IOMMU_PAGE_RESP_SUCCESS)
 			status = domain->iopf_handler(&iopf->fault,
 						      domain->fault_data);

drivers/iommu/io-pgfault.c static void iopf_handler(struct work_struct *work) · line 97

Quoted 10 lines
drivers/iommu/io-pgfault.c +2 -5
-
-		if (!(iopf->fault.prm.flags &
-		      IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE))
-			kfree(iopf);
 	}
 
 	iopf_complete_group(group->dev, &group->last_fault, status);
-	kfree(group);
+	iopf_free_group(group);
+}

this is perf-critical path. It's not good to traverse the list twice.

drivers/iommu/io-pgfault.c static void iopf_handler(struct work_struct *work) · line 107

Quoted 11 lines
drivers/iommu/io-pgfault.c +9 -0
+
+static int iopf_queue_work(struct iopf_group *group, work_func_t func)
+{
+	struct iopf_device_param *iopf_param = group->dev->iommu-
 >iopf_param;
+
+	INIT_WORK(&group->work, func);
+	queue_work(iopf_param->queue->wq, &group->work);
+
+	return 0;
 }

Is there plan to introduce further error in the future? otherwise this should be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

drivers/iommu/io-pgfault.c int iommu_queue_iopf(struct iommu_fault *fault, · line 213

Quoted 13 lines
drivers/iommu/io-pgfault.c +5 -2
@@ -199,8 +204,11 @@ int iommu_queue_iopf(struct iommu_fault *fault,
 struct device *dev)
 			list_move(&iopf->list, &group->faults);
 	}
 
-	queue_work(iopf_param->queue->wq, &group->work);
-	return 0;
+	ret = iopf_queue_work(group, iopf_handler);
+	if (ret)
+		iopf_free_group(group);
+
+	return ret;
 

Here we can document that the iopf handler (in patch10) should free the group, allowing the optimization inside the handler.

author maintainer
#45 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 23 lines · Kevin
Quoted 23 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

drivers/iommu/io-pgfault.c +1 -1
@@ -82,7 +82,7 @@ static void iopf_handler(struct work_struct *work)
  	if (!domain || !domain->iopf_handler)
  		status = IOMMU_PAGE_RESP_INVALID;
 
-	list_for_each_entry_safe(iopf, next, &group->faults, list) {
+	list_for_each_entry(iopf, &group->faults, list) {
  		/*
  		 * For the moment, errors are sticky: don't handle

subsequent
* faults in the group if there is an error.

drivers/iommu/io-pgfault.c +0 -4
@@ -90,14 +90,20 @@ static void iopf_handler(struct work_struct *work)
  		if (status == IOMMU_PAGE_RESP_SUCCESS)
  			status = domain->iopf_handler(&iopf->fault,
  						      domain->fault_data);
-
-		if (!(iopf->fault.prm.flags &
-		      IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE))
-			kfree(iopf);
  	}
 
Quoted 6 lines

drivers/iommu/io-pgfault.c static void iopf_handler(struct work_struct *work) · line 97

Quoted 4 lines
drivers/iommu/io-pgfault.c +2 -1
  	iopf_complete_group(group->dev, &group->last_fault, status);
-	kfree(group);
+	iopf_free_group(group);
+}

this is perf-critical path. It's not good to traverse the list twice.

Freeing the fault group is not critical anymore, right?

Quoted 6 lines
Quoted 6 lines
drivers/iommu/io-pgfault.c +4 -0
+
+static int iopf_queue_work(struct iopf_group *group, work_func_t func)
+{
+	struct iopf_device_param *iopf_param = group->dev->iommu-
 > iopf_param;
Quoted 9 lines

drivers/iommu/io-pgfault.c static void iopf_handler(struct work_struct *work) · line 107

Quoted 6 lines
drivers/iommu/io-pgfault.c +5 -0
+
+	INIT_WORK(&group->work, func);
+	queue_work(iopf_param->queue->wq, &group->work);
+
+	return 0;
  }

Is there plan to introduce further error in the future? otherwise this should be void.

queue_work() return true or false. I should check and return the value.

Quoted 3 lines

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

The definition of struct iopf_device_param is in this file. So I added a helper to avoid making iopf_device_param visible globally.

Quoted 6 lines
Quoted 6 lines
drivers/iommu/io-pgfault.c +0 -0
@@ -199,8 +204,11 @@ int iommu_queue_iopf(struct iommu_fault *fault,
 struct device *dev)
  			list_move(&iopf->list, &group->faults);
  	}
 
Quoted 11 lines

drivers/iommu/io-pgfault.c int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev) · line 212

Quoted 8 lines
drivers/iommu/io-pgfault.c +5 -2
-	queue_work(iopf_param->queue->wq, &group->work);
-	return 0;
+	ret = iopf_queue_work(group, iopf_handler);
+	if (ret)
+		iopf_free_group(group);
+
+	return ret;
 

Here we can document that the iopf handler (in patch10) should free the group, allowing the optimization inside the handler.

Yeah!

Best regards,
baolu

#46 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

In reply to Tian, Kevin
Quoted 5 lines · Kevin

Is there plan to introduce further error in the future? otherwise this should be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

I think more than just SVA will need a work queue context to process their faults.

Jason

#47 lore raw

RE: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 5 lines

From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

On Thu, Aug 03, 2023 at 08:16:47AM +0000, Tian, Kevin wrote:

Quoted 9 lines
Quoted 5 lines

Is there plan to introduce further error in the future? otherwise this should be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

I think more than just SVA will need a work queue context to process their faults.

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

author maintainer
#48 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 11 lines · Kevin
Quoted 11 lines

From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

Quoted 5 lines · Kevin

Is there plan to introduce further error in the future? otherwise this should be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

Quoted 6 lines
Quoted 3 lines

I think more than just SVA will need a work queue context to process their faults.

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

Do you mind elaborate a bit here? workqueue is a basic infrastructure in the fault handling framework, but it lets the consumers choose to use it, or not to.

Best regards,
baolu

#49 lore raw

RE: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 20 lines

From: Baolu Lu <[email protected]>
Sent: Wednesday, August 9, 2023 6:41 PM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

Quoted 1 line · Kevin

Is there plan to introduce further error in the future? otherwise this

should

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

I think more than just SVA will need a work queue context to process their faults.

Quoted 7 lines
Quoted 2 lines

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

Do you mind elaborate a bit here? workqueue is a basic infrastructure in the fault handling framework, but it lets the consumers choose to use it, or not to.

My understanding of Jason's comment was to make the workqueue the default path instead of being opted by the consumer.. that is my 1st impression but might be wrong...

#50 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 23 lines · Kevin
Quoted 23 lines

From: Baolu Lu <[email protected]>
Sent: Wednesday, August 9, 2023 6:41 PM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jason Gunthorpe <[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

Quoted 1 line · Kevin

Is there plan to introduce further error in the future? otherwise this

should

Quoted 11 lines
Quoted 8 lines
Quoted 4 lines

be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

I think more than just SVA will need a work queue context to process their faults.

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

Quoted 8 lines
Quoted 4 lines

Do you mind elaborate a bit here? workqueue is a basic infrastructure in the fault handling framework, but it lets the consumers choose to use it, or not to.

My understanding of Jason's comment was to make the workqueue the default path instead of being opted by the consumer.. that is my 1st impression but might be wrong...

Yeah, that is one path. Do we have anyone that uses this that doesn't want the WQ? (actually who even uses this besides SVA?)

Jason

author maintainer
#51 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 25 lines · Jason Gunthorpe
Quoted 24 lines · Kevin
Quoted 24 lines

From: Baolu Lu<[email protected]>
Sent: Wednesday, August 9, 2023 6:41 PM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jason Gunthorpe<[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

Quoted 1 line · Kevin

Is there plan to introduce further error in the future? otherwise this

should

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

I think more than just SVA will need a work queue context to process their faults.

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

Do you mind elaborate a bit here? workqueue is a basic infrastructure in the fault handling framework, but it lets the consumers choose to use it, or not to.

Quoted 5 lines
Quoted 3 lines

My understanding of Jason's comment was to make the workqueue the default path instead of being opted by the consumer.. that is my 1st impression but might be wrong...

Yeah, that is one path. Do we have anyone that uses this that doesn't want the WQ? (actually who even uses this besides SVA?)

I am still confused. When we forward iopf's to user space through the iommufd, we don't need to schedule a WQ, right? Or I misunderstood here?

Best regards,
baolu

#52 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 29 lines · Baolu Lu
Quoted 28 lines · Jason Gunthorpe
Quoted 27 lines · Kevin
Quoted 15 lines

From: Baolu Lu<[email protected]>
Sent: Wednesday, August 9, 2023 6:41 PM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jason Gunthorpe<[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

Quoted 1 line · Kevin

Is there plan to introduce further error in the future? otherwise this

should

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

I think more than just SVA will need a work queue context to process their faults.

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

Do you mind elaborate a bit here? workqueue is a basic infrastructure in the fault handling framework, but it lets the consumers choose to use it, or not to.

My understanding of Jason's comment was to make the workqueue the default path instead of being opted by the consumer.. that is my 1st impression but might be wrong...

Quoted 6 lines
Quoted 2 lines

Yeah, that is one path. Do we have anyone that uses this that doesn't want the WQ? (actually who even uses this besides SVA?)

I am still confused. When we forward iopf's to user space through the iommufd, we don't need to schedule a WQ, right? Or I misunderstood here?

Yes, that could be true, iommufd could just queue it from the interrupt context and trigger a wakeup.

But other iommufd modes would want to invoke hmm_range_fault() which would need the work queue.

Jason

author maintainer
#53 lore raw

Re: [PATCH v2 08/12] iommu: Prepare for separating SVA and IOPF

Quoted 32 lines · Jason Gunthorpe
Quoted 31 lines · Baolu Lu
Quoted 30 lines · Jason Gunthorpe
Quoted 20 lines · Kevin
Quoted 20 lines

From: Baolu Lu<[email protected]>
Sent: Wednesday, August 9, 2023 6:41 PM

Quoted 6 lines · Kevin
Quoted 6 lines

From: Jason Gunthorpe<[email protected]>
Sent: Wednesday, August 9, 2023 2:43 AM

Quoted 1 line · Kevin

Is there plan to introduce further error in the future? otherwise this

should

Quoted 9 lines
Quoted 7 lines
Quoted 4 lines

be void.

btw the work queue is only for sva. If there is no other caller this can be just kept in iommu-sva.c. No need to create a helper.

I think more than just SVA will need a work queue context to process their faults.

then this series needs more work. Currently the abstraction doesn't include workqueue in the common fault reporting layer.

Quoted 7 lines
Quoted 4 lines

Do you mind elaborate a bit here? workqueue is a basic infrastructure in the fault handling framework, but it lets the consumers choose to use it, or not to.

My understanding of Jason's comment was to make the workqueue the default path instead of being opted by the consumer.. that is my 1st impression but might be wrong...

Yeah, that is one path. Do we have anyone that uses this that doesn't want the WQ? (actually who even uses this besides SVA?)

Quoted 8 lines
Quoted 3 lines

I am still confused. When we forward iopf's to user space through the iommufd, we don't need to schedule a WQ, right? Or I misunderstood here?

Yes, that could be true, iommufd could just queue it from the interrupt context and trigger a wakeup.

But other iommufd modes would want to invoke hmm_range_fault() which would need the work queue.

Yes. That's the reason why I added below helper

int iopf_queue_work(struct iopf_group *group, work_func_t func)

in the patch 09/12.

Best regards,
baolu

author maintainer
#54 lore raw

PATCH v2 9/12 iommu: Move iopf_handler() to iommu-sva.c

In reply to Lu Baolu

The iopf_handler() function handles a fault_group for a SVA domain. Move it to the right place.

---
 drivers/iommu/iommu-sva.h  | 17 +++++++++++++
 drivers/iommu/io-pgfault.c | 50 +++-----------------------------------
 drivers/iommu/iommu-sva.c  | 49 +++++++++++++++++++++++++++++++++++++
 3 files changed, 69 insertions(+), 47 deletions(-)
drivers/iommu/iommu-sva.h +17 -0
index de7819c796ce..510a7df23fba 100644
@@ -24,6 +24,9 @@ void iopf_queue_free(struct iopf_queue *queue);
 int iopf_queue_discard_partial(struct iopf_queue *queue);
 enum iommu_page_response_code
 iommu_sva_handle_iopf(struct iommu_fault *fault, void *data);
+void iopf_free_group(struct iopf_group *group);
+int iopf_queue_work(struct iopf_group *group, work_func_t func);
+int iommu_sva_handle_iopf_group(struct iopf_group *group);
 
 #else /* CONFIG_IOMMU_SVA */
 static inline int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
@@ -67,5 +70,19 @@ iommu_sva_handle_iopf(struct iommu_fault *fault, void *data)
 {
 	return IOMMU_PAGE_RESP_INVALID;
 }
+
+static inline void iopf_free_group(struct iopf_group *group)
+{
+}
+
+static inline int iopf_queue_work(struct iopf_group *group, work_func_t func)
+{
+	return -ENODEV;
+}
+
+static inline int iommu_sva_handle_iopf_group(struct iopf_group *group)
+{
+	return -ENODEV;
+}
 #endif /* CONFIG_IOMMU_SVA */
 #endif /* _IOMMU_SVA_H */
drivers/iommu/io-pgfault.c +3 -47

68 lines not on this page · raw message

drivers/iommu/iommu-sva.c +49 -0

52 lines not on this page · raw message

Signature · 1 line
2.34.1
#55 lore raw

RE: [PATCH v2 09/12] iommu: Move iopf_handler() to iommu-sva.c

drivers/iommu/iommu-sva.c void mm_pasid_drop(struct mm_struct *mm) · line 224

Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

drivers/iommu/iommu-sva.c +3 -0
@@ -219,3 +219,52 @@ void mm_pasid_drop(struct mm_struct *mm)
 
 	ida_free(&iommu_global_pasid_ida, mm->pasid);
 }
+
+static int iopf_complete_group(struct device *dev, struct iopf_fault *iopf,
+			       enum iommu_page_response_code status)

iommu_sva_complete_iopf()

drivers/iommu/iommu-sva.c void mm_pasid_drop(struct mm_struct *mm) · line 241

Quoted 3 lines
drivers/iommu/iommu-sva.c +3 -0
+
+static void iopf_handler(struct work_struct *work)
+{

iommu_sva_iopf_handler()

author maintainer
#56 lore raw

Re: [PATCH v2 09/12] iommu: Move iopf_handler() to iommu-sva.c

Quoted 13 lines · Kevin

drivers/iommu/iommu-sva.c void mm_pasid_drop(struct mm_struct *mm) · line 224

Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Thursday, July 27, 2023 1:49 PM

drivers/iommu/iommu-sva.c +3 -0
@@ -219,3 +219,52 @@ void mm_pasid_drop(struct mm_struct *mm)
 
  	ida_free(&iommu_global_pasid_ida, mm->pasid);
  }
+
+static int iopf_complete_group(struct device *dev, struct iopf_fault *iopf,
+			       enum iommu_page_response_code status)

iommu_sva_complete_iopf()

Quoted 6 lines

drivers/iommu/iommu-sva.c void mm_pasid_drop(struct mm_struct *mm) · line 241

Quoted 3 lines
drivers/iommu/iommu-sva.c +3 -0
+
+static void iopf_handler(struct work_struct *work)
+{

iommu_sva_iopf_handler()

Ack to both.

Best regards,
baolu

#57 lore raw

Re: [PATCH v2 09/12] iommu: Move iopf_handler() to iommu-sva.c

In reply to Lu Baolu
Quoted 31 lines · Lu Baolu

The iopf_handler() function handles a fault_group for a SVA domain. Move it to the right place.

---
 drivers/iommu/iommu-sva.h  | 17 +++++++++++++
 drivers/iommu/io-pgfault.c | 50 +++-----------------------------------
 drivers/iommu/iommu-sva.c  | 49 +++++++++++++++++++++++++++++++++++++
 3 files changed, 69 insertions(+), 47 deletions(-)
drivers/iommu/iommu-sva.c +16 -0
index 05c0fb2acbc4..ab42cfdd7636 100644
@@ -219,3 +219,52 @@ void mm_pasid_drop(struct mm_struct *mm)
 
+static void iopf_handler(struct work_struct *work)
+{
+	struct iopf_fault *iopf;
+	struct iopf_group *group;
+	struct iommu_domain *domain;
+	enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS;
+
+	group = container_of(work, struct iopf_group, work);
+	domain = iommu_get_domain_for_dev_pasid(group->dev,
+				group->last_fault.fault.prm.pasid, 0);
+	if (!domain || !domain->iopf_handler)
+		status = IOMMU_PAGE_RESP_INVALID;
+
+	list_for_each_entry(iopf, &group->faults, list) {
+		/*
+		 * For the moment, errors are sticky: don't handle subsequent

drivers/iommu/iommu-sva.c void mm_pasid_drop(struct mm_struct *mm) · line 245

Quoted 10 lines
drivers/iommu/iommu-sva.c +10 -0
+		 * faults in the group if there is an error.
+		 */
+		if (status == IOMMU_PAGE_RESP_SUCCESS)
+			status = domain->iopf_handler(&iopf->fault,
+						      domain->fault_data);
+	}
+
+	iopf_complete_group(group->dev, &group->last_fault, status);
+	iopf_free_group(group);
+}

Routing faults to domains is generic code, not SVA code.

SVA starts at domain->iopf_handler

Jason

author maintainer
#58 lore raw

Re: [PATCH v2 09/12] iommu: Move iopf_handler() to iommu-sva.c

Quoted 36 lines · Jason Gunthorpe
Quoted 35 lines · Lu Baolu

The iopf_handler() function handles a fault_group for a SVA domain. Move it to the right place.

---
  drivers/iommu/iommu-sva.h  | 17 +++++++++++++
  drivers/iommu/io-pgfault.c | 50 +++-----------------------------------
  drivers/iommu/iommu-sva.c  | 49 +++++++++++++++++++++++++++++++++++++
  3 files changed, 69 insertions(+), 47 deletions(-)
drivers/iommu/iommu-sva.c +20 -0
index 05c0fb2acbc4..ab42cfdd7636 100644
@@ -219,3 +219,52 @@ void mm_pasid_drop(struct mm_struct *mm)
 
+static void iopf_handler(struct work_struct *work)
+{
+	struct iopf_fault *iopf;
+	struct iopf_group *group;
+	struct iommu_domain *domain;
+	enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS;
+
+	group = container_of(work, struct iopf_group, work);
+	domain = iommu_get_domain_for_dev_pasid(group->dev,
+				group->last_fault.fault.prm.pasid, 0);
+	if (!domain || !domain->iopf_handler)
+		status = IOMMU_PAGE_RESP_INVALID;
+
+	list_for_each_entry(iopf, &group->faults, list) {
+		/*
+		 * For the moment, errors are sticky: don't handle subsequent
+		 * faults in the group if there is an error.
+		 */
+		if (status == IOMMU_PAGE_RESP_SUCCESS)
+			status = domain->iopf_handler(&iopf->fault,
Quoted 8 lines

drivers/iommu/iommu-sva.c void mm_pasid_drop(struct mm_struct *mm) · line 265

Quoted 6 lines
drivers/iommu/iommu-sva.c +6 -0
+						      domain->fault_data);
+	}
+
+	iopf_complete_group(group->dev, &group->last_fault, status);
+	iopf_free_group(group);
+}

Routing faults to domains is generic code, not SVA code.

You are right. This happens in the latter patch.

Quoted 4 lines

SVA starts at domain->iopf_handler

Jason

Best regards,
baolu

author maintainer
#59 lore raw

PATCH v2 10/12 iommu: Make iommu_queue_iopf() more generic

In reply to Lu Baolu

This completely separates the IO page fault handling framework from the SVA implementation. Previously, the SVA implementation was tightly coupled with the IO page fault handling framework. This makes SVA a "customer" of the IO page fault handling framework by converting domain's page fault handler to handle a group of faults and calling it directly from iommu_queue_iopf().

---
 include/linux/iommu.h      |  5 +++--
 drivers/iommu/iommu-sva.h  |  8 --------
 drivers/iommu/io-pgfault.c | 16 +++++++++++++---
 drivers/iommu/iommu-sva.c  | 11 ++---------
 drivers/iommu/iommu.c      |  2 +-
 5 files changed, 19 insertions(+), 23 deletions(-)
include/linux/iommu.h +3 -2
index 607740e548f2..1dafe031a56c 100644
@@ -40,6 +40,7 @@ struct notifier_block;
 struct iommu_sva;
 struct iommu_fault_event;
 struct iommu_dma_cookie;
+struct iopf_group;
 
 #define IOMMU_FAULT_PERM_READ	(1 << 0) /* read */
 #define IOMMU_FAULT_PERM_WRITE	(1 << 1) /* write */
@@ -179,8 +180,7 @@ struct iommu_domain {
 	unsigned long pgsize_bitmap;	/* Bitmap of page sizes in use */
 	struct iommu_domain_geometry geometry;
 	struct iommu_dma_cookie *iova_cookie;
-	enum iommu_page_response_code (*iopf_handler)(struct iommu_fault *fault,
-						      void *data);
+	int (*iopf_handler)(struct iopf_group *group);
 	void *fault_data;
 	union {
 		struct {
@@ -513,6 +513,7 @@ struct iopf_group {
 	struct list_head		faults;
 	struct work_struct		work;
 	struct device			*dev;
+	void				*data;
 };
 
 int iommu_device_register(struct iommu_device *iommu,
drivers/iommu/iommu-sva.h +0 -8
index 510a7df23fba..cf41e88fac17 100644
@@ -22,8 +22,6 @@ int iopf_queue_flush_dev(struct device *dev);
 struct iopf_queue *iopf_queue_alloc(const char *name);
 void iopf_queue_free(struct iopf_queue *queue);
 int iopf_queue_discard_partial(struct iopf_queue *queue);
-enum iommu_page_response_code
-iommu_sva_handle_iopf(struct iommu_fault *fault, void *data);
 void iopf_free_group(struct iopf_group *group);
 int iopf_queue_work(struct iopf_group *group, work_func_t func);
 int iommu_sva_handle_iopf_group(struct iopf_group *group);
@@ -65,12 +63,6 @@ static inline int iopf_queue_discard_partial(struct iopf_queue *queue)
 	return -ENODEV;
 }
 
-static inline enum iommu_page_response_code
-iommu_sva_handle_iopf(struct iommu_fault *fault, void *data)
-{
-	return IOMMU_PAGE_RESP_INVALID;
-}
-
 static inline void iopf_free_group(struct iopf_group *group)
 {
 }
drivers/iommu/io-pgfault.c +13 -3
index 3614a800638c..c5cfa3dfa77b 100644
@@ -11,8 +11,6 @@
 #include <linux/slab.h>
 #include <linux/workqueue.h>
 
-#include "iommu-sva.h"
-
 /**
  * struct iopf_queue - IO Page Fault queue
  * @wq: the fault workqueue
@@ -106,6 +104,7 @@ int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 {
 	int ret;
 	struct iopf_group *group;
+	struct iommu_domain *domain;
 	struct iopf_fault *iopf, *next;
 	struct iopf_device_param *iopf_param;
 	struct dev_iommu *param = dev->iommu;
@@ -137,6 +136,16 @@ int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 		return 0;
 	}
 
+	if (fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)
+		domain = iommu_get_domain_for_dev_pasid(dev, fault->prm.pasid, 0);
+	else
+		domain = iommu_get_domain_for_dev(dev);
+
+	if (!domain || !domain->iopf_handler) {
+		ret = -ENODEV;
+		goto cleanup_partial;
+	}
+
 	group = kzalloc(sizeof(*group), GFP_KERNEL);
 	if (!group) {
 		/*
@@ -150,6 +159,7 @@ int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 
 	group->dev = dev;
 	group->last_fault.fault = *fault;
+	group->data = domain->fault_data;
 	INIT_LIST_HEAD(&group->faults);
 	list_add(&group->last_fault.list, &group->faults);
 
@@ -160,7 +170,7 @@ int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 			list_move(&iopf->list, &group->faults);
 	}
 
-	ret = iommu_sva_handle_iopf_group(group);
+	ret = domain->iopf_handler(group);
 	if (ret)
 		iopf_free_group(group);
 
drivers/iommu/iommu-sva.c +2 -9
index ab42cfdd7636..668f4c2bcf65 100644
@@ -157,7 +157,7 @@ EXPORT_SYMBOL_GPL(iommu_sva_get_pasid);
 /*
  * I/O page fault handler for SVA
  */
-enum iommu_page_response_code
+static enum iommu_page_response_code
 iommu_sva_handle_iopf(struct iommu_fault *fault, void *data)
 {
 	vm_fault_t ret;
@@ -241,23 +241,16 @@ static void iopf_handler(struct work_struct *work)
 {
 	struct iopf_fault *iopf;
 	struct iopf_group *group;
-	struct iommu_domain *domain;
 	enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS;
 
 	group = container_of(work, struct iopf_group, work);
-	domain = iommu_get_domain_for_dev_pasid(group->dev,
-				group->last_fault.fault.prm.pasid, 0);
-	if (!domain || !domain->iopf_handler)
-		status = IOMMU_PAGE_RESP_INVALID;
-
 	list_for_each_entry(iopf, &group->faults, list) {
 		/*
 		 * For the moment, errors are sticky: don't handle subsequent
 		 * faults in the group if there is an error.
 		 */
 		if (status == IOMMU_PAGE_RESP_SUCCESS)
-			status = domain->iopf_handler(&iopf->fault,
-						      domain->fault_data);
+			status = iommu_sva_handle_iopf(&iopf->fault, group->data);
 	}
 
 	iopf_complete_group(group->dev, &group->last_fault, status);
drivers/iommu/iommu.c +1 -1
index 157a28a49473..535a36e3edc9 100644
@@ -3330,7 +3330,7 @@ struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
 	domain->type = IOMMU_DOMAIN_SVA;
 	mmgrab(mm);
 	domain->mm = mm;
-	domain->iopf_handler = iommu_sva_handle_iopf;
+	domain->iopf_handler = iommu_sva_handle_iopf_group;
 	domain->fault_data = mm;
 
 	return domain;
Signature · 1 line
2.34.1
#60 lore raw

Re: [PATCH v2 10/12] iommu: Make iommu_queue_iopf() more generic

drivers/iommu/io-pgfault.c int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev) · line 142

Quoted 8 lines · Lu Baolu
drivers/iommu/io-pgfault.c +4 -0
@@ -137,6 +136,16 @@ int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
 		return 0;
 	}
 
+	if (fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)
+		domain = iommu_get_domain_for_dev_pasid(dev, fault->prm.pasid, 0);
+	else
+		domain = iommu_get_domain_for_dev(dev);

How does the lifetime work for this? What prevents UAF on domain?

Quoted 43 lines
drivers/iommu/iommu-sva.c +2 -9
index ab42cfdd7636..668f4c2bcf65 100644
@@ -157,7 +157,7 @@ EXPORT_SYMBOL_GPL(iommu_sva_get_pasid);
 /*
  * I/O page fault handler for SVA
  */
-enum iommu_page_response_code
+static enum iommu_page_response_code
 iommu_sva_handle_iopf(struct iommu_fault *fault, void *data)
 {
 	vm_fault_t ret;
@@ -241,23 +241,16 @@ static void iopf_handler(struct work_struct *work)
 {
 	struct iopf_fault *iopf;
 	struct iopf_group *group;
-	struct iommu_domain *domain;
 	enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS;
 
 	group = container_of(work, struct iopf_group, work);
-	domain = iommu_get_domain_for_dev_pasid(group->dev,
-				group->last_fault.fault.prm.pasid, 0);
-	if (!domain || !domain->iopf_handler)
-		status = IOMMU_PAGE_RESP_INVALID;
-
 	list_for_each_entry(iopf, &group->faults, list) {
 		/*
 		 * For the moment, errors are sticky: don't handle subsequent
 		 * faults in the group if there is an error.
 		 */
 		if (status == IOMMU_PAGE_RESP_SUCCESS)
-			status = domain->iopf_handler(&iopf->fault,
-						      domain->fault_data);
+			status = iommu_sva_handle_iopf(&iopf->fault, group->data);
 	}
 
 	iopf_complete_group(group->dev, &group->last_fault, status);
drivers/iommu/iommu.c +0 -0
index 157a28a49473..535a36e3edc9 100644

drivers/iommu/iommu.c struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, · line 3334

Quoted 6 lines
drivers/iommu/iommu.c +1 -1
@@ -3330,7 +3330,7 @@ struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
 	domain->type = IOMMU_DOMAIN_SVA;
 	mmgrab(mm);
 	domain->mm = mm;
-	domain->iopf_handler = iommu_sva_handle_iopf;
+	domain->iopf_handler = iommu_sva_handle_iopf_group;
 	domain->fault_data = mm;

This also has lifetime problems on the mm.

The domain should flow into the iommu_sva_handle_iopf() instead of the void *data.

The SVA code can then just use domain->mm directly.

We need to document/figure out some how to ensure that the faults are all done processing before a fault enabled domain can be freed.

This patch would be better ordered before the prior patch.

Jason

author maintainer
#61 lore raw

Re: [PATCH v2 10/12] iommu: Make iommu_queue_iopf() more generic

Quoted 11 lines · Jason Gunthorpe

drivers/iommu/io-pgfault.c int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev) · line 142

Quoted 8 lines · Lu Baolu
drivers/iommu/io-pgfault.c +4 -0
@@ -137,6 +136,16 @@ int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
  		return 0;
  	}
 
+	if (fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID)
+		domain = iommu_get_domain_for_dev_pasid(dev, fault->prm.pasid, 0);
+	else
+		domain = iommu_get_domain_for_dev(dev);

How does the lifetime work for this? What prevents UAF on domain?

Replied below.

Quoted 53 lines
Quoted 44 lines
drivers/iommu/iommu-sva.c +2 -9
index ab42cfdd7636..668f4c2bcf65 100644
@@ -157,7 +157,7 @@ EXPORT_SYMBOL_GPL(iommu_sva_get_pasid);
  /*
   * I/O page fault handler for SVA
   */
-enum iommu_page_response_code
+static enum iommu_page_response_code
  iommu_sva_handle_iopf(struct iommu_fault *fault, void *data)
  {
  	vm_fault_t ret;
@@ -241,23 +241,16 @@ static void iopf_handler(struct work_struct *work)
  {
  	struct iopf_fault *iopf;
  	struct iopf_group *group;
-	struct iommu_domain *domain;
  	enum iommu_page_response_code status = IOMMU_PAGE_RESP_SUCCESS;
 
  	group = container_of(work, struct iopf_group, work);
-	domain = iommu_get_domain_for_dev_pasid(group->dev,
-				group->last_fault.fault.prm.pasid, 0);
-	if (!domain || !domain->iopf_handler)
-		status = IOMMU_PAGE_RESP_INVALID;
-
  	list_for_each_entry(iopf, &group->faults, list) {
  		/*
  		 * For the moment, errors are sticky: don't handle subsequent
  		 * faults in the group if there is an error.
  		 */
  		if (status == IOMMU_PAGE_RESP_SUCCESS)
-			status = domain->iopf_handler(&iopf->fault,
-						      domain->fault_data);
+			status = iommu_sva_handle_iopf(&iopf->fault, group->data);
  	}
 
  	iopf_complete_group(group->dev, &group->last_fault, status);
drivers/iommu/iommu.c +0 -0
index 157a28a49473..535a36e3edc9 100644

drivers/iommu/iommu.c struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, · line 3334

Quoted 6 lines
drivers/iommu/iommu.c +1 -1
@@ -3330,7 +3330,7 @@ struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
  	domain->type = IOMMU_DOMAIN_SVA;
  	mmgrab(mm);
  	domain->mm = mm;
-	domain->iopf_handler = iommu_sva_handle_iopf;
+	domain->iopf_handler = iommu_sva_handle_iopf_group;
  	domain->fault_data = mm;

This also has lifetime problems on the mm.

Quoted 2 lines

The domain should flow into the iommu_sva_handle_iopf() instead of the void *data.

Okay, but I still want to keep void *data as a private pointer of the iopf consumer. For SVA, it's probably NULL.

Quoted 2 lines

The SVA code can then just use domain->mm directly.

Yes.

Quoted 3 lines

We need to document/figure out some how to ensure that the faults are all done processing before a fault enabled domain can be freed.

This has been documented in drivers/iommu/io-pgfault.c:

[...]
* Any valid page fault will be eventually routed to an iommu domain
and the
* page fault handler installed there will get called. The users of this
* handling framework should guarantee that the iommu domain could only be
* freed after the device has stopped generating page faults (or the iommu
* hardware has been set to block the page faults) and the pending page
faults
* have been flushed.
*
* Return: 0 on success and <0 on error.
*/
int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) [...]

Quoted 1 line

This patch would be better ordered before the prior patch.

Let me try this in the next version.

Best regards,
baolu

#62 lore raw

Re: [PATCH v2 10/12] iommu: Make iommu_queue_iopf() more generic

Quoted 7 lines · Baolu Lu
Quoted 4 lines

This also has lifetime problems on the mm.

The domain should flow into the iommu_sva_handle_iopf() instead of the void *data.

Okay, but I still want to keep void *data as a private pointer of the iopf consumer. For SVA, it's probably NULL.

I'd rather give the iommu_domain some 'private' void * than pass around weird pointers all over the place... That might be broadly useful, eg iommufd could store the hwpt in there.

Quoted 19 lines
Quoted 2 lines

We need to document/figure out some how to ensure that the faults are all done processing before a fault enabled domain can be freed.

This has been documented in drivers/iommu/io-pgfault.c:

[...]
* Any valid page fault will be eventually routed to an iommu domain and the
* page fault handler installed there will get called. The users of this
* handling framework should guarantee that the iommu domain could only be
* freed after the device has stopped generating page faults (or the iommu
* hardware has been set to block the page faults) and the pending page
faults
* have been flushed.
*
* Return: 0 on success and <0 on error.
*/
int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) [...]

Quoted 3 lines
Quoted 1 line

This patch would be better ordered before the prior patch.

Let me try this in the next version.

Okay.. but can we have some debugging to enforce this maybe? Also add a comment when we obtain the domain on this path to see the above about the lifetime

Jason

author maintainer
#63 lore raw

Re: [PATCH v2 10/12] iommu: Make iommu_queue_iopf() more generic

Quoted 7 lines · Jason Gunthorpe
Quoted 5 lines · Baolu Lu
Quoted 4 lines

This also has lifetime problems on the mm.

The domain should flow into the iommu_sva_handle_iopf() instead of the void *data.

Quoted 6 lines
Quoted 2 lines

Okay, but I still want to keep void *data as a private pointer of the iopf consumer. For SVA, it's probably NULL.

I'd rather give the iommu_domain some 'private' void * than pass around weird pointers all over the place... That might be broadly useful, eg iommufd could store the hwpt in there.

Yes, you are right. With the private pointer stored in domain and domain is passed to the iopf handler, there will be no need for a @data parameter for iopf handler.

Quoted 20 lines
Quoted 20 lines
Quoted 3 lines

We need to document/figure out some how to ensure that the faults are all done processing before a fault enabled domain can be freed.

This has been documented in drivers/iommu/io-pgfault.c:

[...]
* Any valid page fault will be eventually routed to an iommu domain and the
* page fault handler installed there will get called. The users of this
* handling framework should guarantee that the iommu domain could only be
* freed after the device has stopped generating page faults (or the iommu
* hardware has been set to block the page faults) and the pending page
faults
* have been flushed.
*
* Return: 0 on success and <0 on error.
*/
int iommu_queue_iopf(struct iommu_fault *fault, void *cookie) [...]

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

This patch would be better ordered before the prior patch.

Let me try this in the next version.

Okay.. but can we have some debugging to enforce this maybe? Also add a comment when we obtain the domain on this path to see the above about the lifetime

Yes. Sure.

Probably I will add a dev_warn() when get_domain for device or pasid returns NULL...

In the paths of removing domain from device or pasid, I will add a check for pending faults. Will trigger a dev_warn() if there is any pending faults for the affected domain.

Best regards,
baolu

author maintainer
#64 lore raw

PATCH v2 11/12 iommu: Separate SVA and IOPF in Makefile and Kconfig

In reply to Lu Baolu

Add CONFIG_IOMMU_IOPF for page fault handling framework and select it from its real consumer. Move iopf function declaration from iommu-sva.h to iommu.h and remove iommu-sva.h as it's empty now.

---
 include/linux/iommu.h                         | 63 +++++++++++++++
 drivers/iommu/iommu-sva.h                     | 80 -------------------
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  1 -
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |  1 -
 drivers/iommu/intel/iommu.c                   |  1 -
 drivers/iommu/intel/svm.c                     |  1 -
 drivers/iommu/iommu-sva.c                     |  3 +-
 drivers/iommu/iommu.c                         |  2 -
 drivers/iommu/Kconfig                         |  4 +
 drivers/iommu/Makefile                        |  3 +-
 drivers/iommu/intel/Kconfig                   |  1 +
 11 files changed, 71 insertions(+), 89 deletions(-)
 delete mode 100644 drivers/iommu/iommu-sva.h
include/linux/iommu.h +63 -0
index 1dafe031a56c..d725500344a4 100644
@@ -41,6 +41,7 @@ struct iommu_sva;
 struct iommu_fault_event;
 struct iommu_dma_cookie;
 struct iopf_group;
+struct iopf_queue;
 
 #define IOMMU_FAULT_PERM_READ	(1 << 0) /* read */
 #define IOMMU_FAULT_PERM_WRITE	(1 << 1) /* write */
@@ -1259,6 +1260,7 @@ struct iommu_sva *iommu_sva_bind_device(struct device *dev,
 					struct mm_struct *mm);
 void iommu_sva_unbind_device(struct iommu_sva *handle);
 u32 iommu_sva_get_pasid(struct iommu_sva *handle);
+int iommu_sva_handle_iopf_group(struct iopf_group *group);
 #else
 static inline struct iommu_sva *
 iommu_sva_bind_device(struct device *dev, struct mm_struct *mm)
@@ -1277,6 +1279,67 @@ static inline u32 iommu_sva_get_pasid(struct iommu_sva *handle)
 static inline void mm_pasid_init(struct mm_struct *mm) {}
 static inline bool mm_valid_pasid(struct mm_struct *mm) { return false; }
 static inline void mm_pasid_drop(struct mm_struct *mm) {}
+static inline int iommu_sva_handle_iopf_group(struct iopf_group *group)
+{
+	return -ENODEV;
+}
 #endif /* CONFIG_IOMMU_SVA */
 
+#ifdef CONFIG_IOMMU_IOPF
+int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev);
+int iopf_queue_add_device(struct iopf_queue *queue, struct device *dev);
+int iopf_queue_remove_device(struct iopf_queue *queue,
+			     struct device *dev);
+int iopf_queue_flush_dev(struct device *dev);
+struct iopf_queue *iopf_queue_alloc(const char *name);
+void iopf_queue_free(struct iopf_queue *queue);
+int iopf_queue_discard_partial(struct iopf_queue *queue);
+void iopf_free_group(struct iopf_group *group);
+int iopf_queue_work(struct iopf_group *group, work_func_t func);
+#else /* CONFIG_IOMMU_IOPF */
+static inline int iommu_queue_iopf(struct iommu_fault *fault, struct device *dev)
+{
+	return -ENODEV;
+}
+
+static inline int iopf_queue_add_device(struct iopf_queue *queue,
+					struct device *dev)
+{
+	return -ENODEV;
+}
+
+static inline int iopf_queue_remove_device(struct iopf_queue *queue,
+					   struct device *dev)
+{
+	return -ENODEV;
+}
+
+static inline int iopf_queue_flush_dev(struct device *dev)
+{
+	return -ENODEV;
+}
+
+static inline struct iopf_queue *iopf_queue_alloc(const char *name)
+{
+	return NULL;
+}
+
+static inline void iopf_queue_free(struct iopf_queue *queue)
+{
+}
+
+static inline int iopf_queue_discard_partial(struct iopf_queue *queue)
+{
+	return -ENODEV;
+}
+
+static inline void iopf_free_group(struct iopf_group *group)
+{
+}
+
+static inline int iopf_queue_work(struct iopf_group *group, work_func_t func)
+{
+	return -ENODEV;
+}
+#endif /* CONFIG_IOMMU_IOPF */
 #endif /* __LINUX_IOMMU_H */
drivers/iommu/iommu-sva.h deleted +0 -80

80 lines not on this page · raw message

drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +0 -1
index fa8ab9d413f8..003426327e20 100644
@@ -10,7 +10,6 @@
 #include <linux/slab.h>
 
 #include "arm-smmu-v3.h"
-#include "../../iommu-sva.h"
 #include "../../io-pgtable-arm.h"
 
 struct arm_smmu_mmu_notifier {
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +0 -1
index 1a43f904b6b8..cc4223692537 100644
@@ -29,7 +29,6 @@
 
 #include "arm-smmu-v3.h"
 #include "../../dma-iommu.h"
-#include "../../iommu-sva.h"
 
 static bool disable_bypass = true;
 module_param(disable_bypass, bool, 0444);
drivers/iommu/intel/iommu.c +0 -1
index e94e833638a7..5a8949d01391 100644
@@ -26,7 +26,6 @@
 #include "iommu.h"
 #include "../dma-iommu.h"
 #include "../irq_remapping.h"
-#include "../iommu-sva.h"
 #include "pasid.h"
 #include "cap_audit.h"
 #include "perfmon.h"
drivers/iommu/intel/svm.c +0 -1
index e95b339e9cdc..8d409dcda166 100644
@@ -22,7 +22,6 @@
 #include "iommu.h"
 #include "pasid.h"
 #include "perf.h"
-#include "../iommu-sva.h"
 #include "trace.h"
 
 static irqreturn_t prq_event_thread(int irq, void *d);
drivers/iommu/iommu-sva.c +1 -2
index 668f4c2bcf65..3328f00963de 100644
@@ -6,8 +6,7 @@
 #include <linux/mutex.h>
 #include <linux/sched/mm.h>
 #include <linux/iommu.h>
-
-#include "iommu-sva.h"
+#include <linux/mm_types.h>
 
 static DEFINE_MUTEX(iommu_sva_lock);
 static DEFINE_IDA(iommu_global_pasid_ida);
drivers/iommu/iommu.c +0 -2
index 535a36e3edc9..9d964d16d0ad 100644
@@ -35,8 +35,6 @@
 
 #include "dma-iommu.h"
 
-#include "iommu-sva.h"
-
 static struct kset *iommu_group_kset;
 static DEFINE_IDA(iommu_group_ida);
 
drivers/iommu/Kconfig +4 -0
index 2b12b583ef4b..e7b87ec525a7 100644
@@ -158,6 +158,9 @@ config IOMMU_DMA
 config IOMMU_SVA
 	bool
 
+config IOMMU_IOPF
+	bool
+
 config FSL_PAMU
 	bool "Freescale IOMMU support"
 	depends on PCI
@@ -404,6 +407,7 @@ config ARM_SMMU_V3_SVA
 	bool "Shared Virtual Addressing support for the ARM SMMUv3"
 	depends on ARM_SMMU_V3
 	select IOMMU_SVA
+	select IOMMU_IOPF
 	select MMU_NOTIFIER
 	help
 	  Support for sharing process address spaces with devices using the
drivers/iommu/Makefile +2 -1
index 769e43d780ce..6d114fe7ae89 100644
@@ -27,6 +27,7 @@ obj-$(CONFIG_FSL_PAMU) += fsl_pamu.o fsl_pamu_domain.o
 obj-$(CONFIG_S390_IOMMU) += s390-iommu.o
 obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o
 obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o
-obj-$(CONFIG_IOMMU_SVA) += iommu-sva.o io-pgfault.o
+obj-$(CONFIG_IOMMU_SVA) += iommu-sva.o
+obj-$(CONFIG_IOMMU_IOPF) += io-pgfault.o
 obj-$(CONFIG_SPRD_IOMMU) += sprd-iommu.o
 obj-$(CONFIG_APPLE_DART) += apple-dart.o
drivers/iommu/intel/Kconfig +1 -0
index 2e56bd79f589..613f149510a7 100644
@@ -50,6 +50,7 @@ config INTEL_IOMMU_SVM
 	depends on X86_64
 	select MMU_NOTIFIER
 	select IOMMU_SVA
+	select IOMMU_IOPF
 	help
 	  Shared Virtual Memory (SVM) provides a facility for devices
 	  to access DMA resources through process address space by
Signature · 1 line
2.34.1
Reviewed-by on v2 11/12
#65 lore raw

Re: [PATCH v2 11/12] iommu: Separate SVA and IOPF in Makefile and Kconfig

Quoted 5 lines · Lu Baolu

Add CONFIG_IOMMU_IOPF for page fault handling framework and select it from its real consumer. Move iopf function declaration from iommu-sva.h to iommu.h and remove iommu-sva.h as it's empty now.

Quoted 14 lines
---
 include/linux/iommu.h                         | 63 +++++++++++++++
 drivers/iommu/iommu-sva.h                     | 80 -------------------
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  1 -
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |  1 -
 drivers/iommu/intel/iommu.c                   |  1 -
 drivers/iommu/intel/svm.c                     |  1 -
 drivers/iommu/iommu-sva.c                     |  3 +-
 drivers/iommu/iommu.c                         |  2 -
 drivers/iommu/Kconfig                         |  4 +
 drivers/iommu/Makefile                        |  3 +-
 drivers/iommu/intel/Kconfig                   |  1 +
 11 files changed, 71 insertions(+), 89 deletions(-)
 delete mode 100644 drivers/iommu/iommu-sva.h

At some point it would be a nice touch to split iommu.h into the consumer and iommu driver interfaces

Jason

author maintainer
#66 lore raw

PATCH v2 12/12 iommu: Add helper to set iopf handler for domain

In reply to Lu Baolu

To avoid open code everywhere.

---
 include/linux/iommu.h | 11 ++++++++++-
 drivers/iommu/iommu.c | 20 ++++++++++++++++++--
 2 files changed, 28 insertions(+), 3 deletions(-)
include/linux/iommu.h +10 -1
index d725500344a4..4e0b75befefc 100644
@@ -132,6 +132,7 @@ struct iommu_page_response {
 
 typedef int (*iommu_fault_handler_t)(struct iommu_domain *,
 			struct device *, unsigned long, int, void *);
+typedef int (*iommu_iopf_handler_t)(struct iopf_group *group);
 
 struct iommu_domain_geometry {
 	dma_addr_t aperture_start; /* First address that can be mapped    */
@@ -181,7 +182,7 @@ struct iommu_domain {
 	unsigned long pgsize_bitmap;	/* Bitmap of page sizes in use */
 	struct iommu_domain_geometry geometry;
 	struct iommu_dma_cookie *iova_cookie;
-	int (*iopf_handler)(struct iopf_group *group);
+	iommu_iopf_handler_t iopf_handler;
 	void *fault_data;
 	union {
 		struct {
@@ -581,6 +582,8 @@ extern ssize_t iommu_map_sg(struct iommu_domain *domain, unsigned long iova,
 extern phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova);
 extern void iommu_set_fault_handler(struct iommu_domain *domain,
 			iommu_fault_handler_t handler, void *token);
+void iommu_domain_set_iopf_handler(struct iommu_domain *domain,
+				   iommu_iopf_handler_t handler, void *data);
 
 extern void iommu_get_resv_regions(struct device *dev, struct list_head *list);
 extern void iommu_put_resv_regions(struct device *dev, struct list_head *list);
@@ -904,6 +907,12 @@ static inline void iommu_set_fault_handler(struct iommu_domain *domain,
 {
 }
 
+static inline void iommu_domain_set_iopf_handler(struct iommu_domain *domain,
+						 iommu_iopf_handler_t handler,
+						 void *data)
+{
+}
+
 static inline void iommu_get_resv_regions(struct device *dev,
 					struct list_head *list)
 {
drivers/iommu/iommu.c +18 -2
index 9d964d16d0ad..2254b0ef91fb 100644
@@ -1839,6 +1839,23 @@ void iommu_set_fault_handler(struct iommu_domain *domain,
 }
 EXPORT_SYMBOL_GPL(iommu_set_fault_handler);
 
+/**
+ * iommu_domain_set_iopf_handler() - set io page fault handler for a domain
+ * @domain: iommu domain
+ * @handler: fault handler
+ * @data: user data, will be passed back to the fault handler
+ *
+ * This function should be used by iommu domain users which want to be notified
+ * whenever an IOMMU I/O page fault happens.
+ */
+void iommu_domain_set_iopf_handler(struct iommu_domain *domain,
+				   iommu_iopf_handler_t handler, void *data)
+{
+	domain->iopf_handler = handler;
+	domain->fault_data = data;
+}
+EXPORT_SYMBOL_GPL(iommu_domain_set_iopf_handler);
+
 static struct iommu_domain *__iommu_domain_alloc(const struct bus_type *bus,
 						 unsigned type)
 {
@@ -3328,8 +3345,7 @@ struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
 	domain->type = IOMMU_DOMAIN_SVA;
 	mmgrab(mm);
 	domain->mm = mm;
-	domain->iopf_handler = iommu_sva_handle_iopf_group;
-	domain->fault_data = mm;
+	iommu_domain_set_iopf_handler(domain, iommu_sva_handle_iopf_group, mm);
 
 	return domain;
 }
Signature · 1 line
2.34.1
#67 lore raw

Re: [PATCH v2 12/12] iommu: Add helper to set iopf handler for domain

Quoted 7 lines · Lu Baolu

To avoid open code everywhere.

---
 include/linux/iommu.h | 11 ++++++++++-
 drivers/iommu/iommu.c | 20 ++++++++++++++++++--
 2 files changed, 28 insertions(+), 3 deletions(-)

Seems like overkill at this point..

Also, I think this is probably upside down.

We want to create the domains as fault enabled in the first place.

A fault enabled domain should never be attached to something that cannot support faults. It should also not support changing the fault handler while it exists.

Thus at the creation point would be the time to supply the fault handler as part of requesting faulting.

Taking an existing domain and making it faulting enabled is going to be really messy in all the corner cases.

My advice (and Robin will probably hate me), is to define a new op:

struct domain_alloc_paging_args {
       struct fault_handler *fault_handler;
       void *fault_data
};

struct iommu_domain *domain_alloc_paging2(struct device *dev, struct
domain_alloc_paging_args *args)

The point would be to leave the majority of drivers using the simplified, core assisted, domain_alloc_paging() interface and they just don't have to touch any of this stuff at all.

Obviously if handler is given then the domain will be initialized as faulting.

Jason

author maintainer
#68 lore raw

Re: [PATCH v2 12/12] iommu: Add helper to set iopf handler for domain

Quoted 19 lines · Jason Gunthorpe
Quoted 7 lines · Lu Baolu

To avoid open code everywhere.

---
  include/linux/iommu.h | 11 ++++++++++-
  drivers/iommu/iommu.c | 20 ++++++++++++++++++--
  2 files changed, 28 insertions(+), 3 deletions(-)

Seems like overkill at this point..

Also, I think this is probably upside down.

We want to create the domains as fault enabled in the first place.

A fault enabled domain should never be attached to something that cannot support faults. It should also not support changing the fault handler while it exists.

Quoted 5 lines

Thus at the creation point would be the time to supply the fault handler as part of requesting faulting.

Taking an existing domain and making it faulting enabled is going to be really messy in all the corner cases.

Yes. Agreed.

Quoted 11 lines

My advice (and Robin will probably hate me), is to define a new op:

struct domain_alloc_paging_args {
        struct fault_handler *fault_handler;
        void *fault_data
};

struct iommu_domain *domain_alloc_paging2(struct device *dev, struct
domain_alloc_paging_args *args)

Quoted 6 lines

The point would be to leave the majority of drivers using the simplified, core assisted, domain_alloc_paging() interface and they just don't have to touch any of this stuff at all.

Obviously if handler is given then the domain will be initialized as faulting.

Perhaps we also need an internal helper for iommu drivers to check the iopf capability of the domain.

Best regards,
baolu

#69 lore raw

Re: [PATCH v2 12/12] iommu: Add helper to set iopf handler for domain

Quoted 43 lines · Baolu Lu
Quoted 19 lines · Jason Gunthorpe
Quoted 7 lines · Lu Baolu

To avoid open code everywhere.

---
  include/linux/iommu.h | 11 ++++++++++-
  drivers/iommu/iommu.c | 20 ++++++++++++++++++--
  2 files changed, 28 insertions(+), 3 deletions(-)

Seems like overkill at this point..

Also, I think this is probably upside down.

We want to create the domains as fault enabled in the first place.

A fault enabled domain should never be attached to something that cannot support faults. It should also not support changing the fault handler while it exists.

Quoted 5 lines

Thus at the creation point would be the time to supply the fault handler as part of requesting faulting.

Taking an existing domain and making it faulting enabled is going to be really messy in all the corner cases.

Yes. Agreed.

Quoted 15 lines

My advice (and Robin will probably hate me), is to define a new op:

struct domain_alloc_paging_args {
        struct fault_handler *fault_handler;
        void *fault_data
};

struct iommu_domain *domain_alloc_paging2(struct device *dev, struct
domain_alloc_paging_args *args)

The point would be to leave the majority of drivers using the simplified, core assisted, domain_alloc_paging() interface and they just don't have to touch any of this stuff at all.

Quoted 5 lines
Quoted 2 lines

Obviously if handler is given then the domain will be initialized as faulting.

Perhaps we also need an internal helper for iommu drivers to check the iopf capability of the domain.

Yeah, maybe.

I've been mulling over this for a a bit here

Robin suggested to wrap everything in a arg to domain_alloc and build a giant super multiplexor

I don't really like that because it makes it quite complicated for the driver and multiplexor APIs are rarely good.

So for simple drivers I like the 'domain_alloc_paging' as the only op they implement and it is obviously simple and hard to implement wrong. Most drivers would do this.

We also need a:

struct iommu_domain *domain_alloc_sva(struct device *dev, struct mm_struct *mm)

So SVA can be fully setup at allocation time. SVA doesn't have any legal permutations so it can be kept simple.

Then we need something to bundle:
- Dirty tracking yes/no
- The iommufd user space blob
- Fault handling yes/no

For complex drivers.

So maybe we should just have a 3rd option

// I'm a complex driver and many people checked that I implemented // this right:

struct domain_alloc_args {
       struct device *dev;
       unsigned int flags; // For requesting dirty tracking

       // alloc_domain_user interface
       struct iommu_domain *parent;
       void *user_data;
       size_t user_len;

       // Faulting
       struct fault_handler *fault_handler;
       void *fault_data;	
};

struct iommu_domain *domain_alloc(struct domain_alloc_args *args);

?

IDK, multiplexor APIs are rarely good, but maybe this is the right direction?

Jason