Skip to contentSkip to conversation

iommu: Handle iommu faults for a bad iopf setup

PATCH Superseded Pranjal Shrivastava 3 participants 4 messages Aug 15, 2024

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 iommu: Handle iommu faults for a bad iopf setup

The iommu_report_device_fault function was updated to return void while assuming that drivers only need to call iommu_report_device_fault() for reporting an iopf. This implementation causes following problems:

1. The drivers rely on the core code to call it's page_reponse, however, when a fault is received and no fault capable domain is attached / iopf_param is NULL, the ops->page_response is NOT called causing the device to stall in case the fault type was PAGE_REQ.

2. The arm_smmu_v3 driver relies on the returned value to log errors returning void from iommu_report_device_fault causes these events to be missed while logging.

Modify the iommu_report_device_fault function to return -EINVAL for cases where no fault capable domain is attached or iopf_param was NULL and calls back to the driver (ops->page_response) in case the fault type was IOMMU_FAULT_PAGE_REQ. The returned value can be used by the drivers to log the fault/event as needed.

Fixes 3dfa64aecbaf ("iommu: Make iommu_report_device_fault() return void")
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |   2 +-
 drivers/iommu/io-pgfault.c                  | 121 ++++++++++++++------
 include/linux/iommu.h                       |   5 +-
 3 files changed, 87 insertions(+), 41 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +1 -1
index 9bc50bded5af..8a6cd0adfcf2 100644
@@ -1777,7 +1777,7 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt)
 		goto out_unlock;
 	}
 
-	iommu_report_device_fault(master->dev, &fault_evt);
+	ret = iommu_report_device_fault(master->dev, &fault_evt);
 out_unlock:
 	mutex_unlock(&smmu->streams_mutex);
 	return ret;
drivers/iommu/io-pgfault.c +83 -38

162 lines not on this page · raw message

include/linux/iommu.h +3 -2
index 04cbdae0052e..bd722f473635 100644
@@ -1563,7 +1563,7 @@ struct iopf_queue *iopf_queue_alloc(const char *name);
 void iopf_queue_free(struct iopf_queue *queue);
 int iopf_queue_discard_partial(struct iopf_queue *queue);
 void iopf_free_group(struct iopf_group *group);
-void iommu_report_device_fault(struct device *dev, struct iopf_fault *evt);
+int iommu_report_device_fault(struct device *dev, struct iopf_fault *evt);
 void iopf_group_response(struct iopf_group *group,
 			 enum iommu_page_response_code status);
 #else
@@ -1601,9 +1601,10 @@ static inline void iopf_free_group(struct iopf_group *group)
 {
 }
 
-static inline void
+static inline int
 iommu_report_device_fault(struct device *dev, struct iopf_fault *evt)
 {
+	return -ENODEV;
 }
 
 static inline void iopf_group_response(struct iopf_group *group,
Signature · 1 line
2.46.0.184.g6999bdac58-goog
Reviewed-by on v2 1/1
#1 lore raw
Quoted 19 lines · Pranjal Shrivastava

The iommu_report_device_fault function was updated to return void while assuming that drivers only need to call iommu_report_device_fault() for reporting an iopf. This implementation causes following problems:

1. The drivers rely on the core code to call it's page_reponse, however, when a fault is received and no fault capable domain is attached / iopf_param is NULL, the ops->page_response is NOT called causing the device to stall in case the fault type was PAGE_REQ.

2. The arm_smmu_v3 driver relies on the returned value to log errors returning void from iommu_report_device_fault causes these events to be missed while logging.

Modify the iommu_report_device_fault function to return -EINVAL for cases where no fault capable domain is attached or iopf_param was NULL and calls back to the driver (ops->page_response) in case the fault type was IOMMU_FAULT_PAGE_REQ. The returned value can be used by the drivers to log the fault/event as needed.

Quoted 5 lines
Fixes 3dfa64aecbaf ("iommu: Make iommu_report_device_fault() return void")

Pedantically these two Signed-off's should be swapped (I wouldn't resend for that)

Quoted 5 lines
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c |   2 +-
 drivers/iommu/io-pgfault.c                  | 121 ++++++++++++++------
 include/linux/iommu.h                       |   5 +-
 3 files changed, 87 insertions(+), 41 deletions(-)

Looks OK to me

Joerg, this would beed to be for-rc

Thanks,
Jason

reviewer
#2 lore raw
In reply to Pranjal Shrivastava

On 15/08/2024 1:46 pm, Pranjal Shrivastava wrote:
[...]> @@ -155,22 +208,37 @@ static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param,

Quoted 19 lines

* hardware has been set to block the page faults) and the pending page faults
* have been flushed.
*/

drivers/iommu/io-pgfault.c +9 -1
-void iommu_report_device_fault(struct device *dev, struct iopf_fault *evt)
+int iommu_report_device_fault(struct device *dev, struct iopf_fault *evt)
  {
+	struct iommu_attach_handle *attach_handle;
  	struct iommu_fault *fault = &evt->fault;
  	struct iommu_fault_param *iopf_param;
  	struct iopf_group abort_group = {};
  	struct iopf_group *group;
+	int ret = 0;
 
+	attach_handle = find_fault_handler(dev, evt);
+	if (!attach_handle) {
+		ret = -EINVAL;
+		goto err_bad_iopf;
+	}
+

drivers/iommu/io-pgfault.c static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · line 233

Quoted 10 lines
drivers/iommu/io-pgfault.c +7 -2
+	/*
+	 * Something has gone wrong if a fault capable domain is attached but no
+	 * iopf_param is setup
+	 */
  	iopf_param = iopf_get_dev_fault_param(dev);
-	if (WARN_ON(!iopf_param))
-		return;
+	if (WARN_ON(!iopf_param)) {
+		ret = -EINVAL;
+		goto err_bad_iopf;

Nit: the err_bad_iopf path is only ever returning -EINVAL, so you could simply hard-code that there...

drivers/iommu/io-pgfault.c static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · line 237

Quoted 5 lines
drivers/iommu/io-pgfault.c +2 -1
+	}
 
  	if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) {
-		report_partial_fault(iopf_param, fault);
+		ret = report_partial_fault(iopf_param, fault);

...and declare "ret" here in the one scope it's actually needed.

Thanks,
Robin.

Quoted 8 lines

iopf_put_dev_fault_param(iopf_param);
/* A request that is not the last does not need to be ack'd */

drivers/iommu/io-pgfault.c +2 -1
-		return;
+
+		return ret;
  	}
 
  	/*
#3 lore raw
Quoted 35 lines · Robin Murphy

On 15/08/2024 1:46 pm, Pranjal Shrivastava wrote:
[...]> @@ -155,22 +208,37 @@ static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param,

Quoted 18 lines

* hardware has been set to block the page faults) and the pending page faults
* have been flushed.
*/

drivers/iommu/io-pgfault.c +9 -1
-void iommu_report_device_fault(struct device *dev, struct iopf_fault *evt)
+int iommu_report_device_fault(struct device *dev, struct iopf_fault *evt)
  {
+	struct iommu_attach_handle *attach_handle;
  	struct iommu_fault *fault = &evt->fault;
  	struct iommu_fault_param *iopf_param;
  	struct iopf_group abort_group = {};
  	struct iopf_group *group;
+	int ret = 0;
+	attach_handle = find_fault_handler(dev, evt);
+	if (!attach_handle) {
+		ret = -EINVAL;
+		goto err_bad_iopf;
+	}
+

drivers/iommu/io-pgfault.c static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · line 233

Quoted 10 lines
drivers/iommu/io-pgfault.c +7 -2
+	/*
+	 * Something has gone wrong if a fault capable domain is attached but no
+	 * iopf_param is setup
+	 */
  	iopf_param = iopf_get_dev_fault_param(dev);
-	if (WARN_ON(!iopf_param))
-		return;
+	if (WARN_ON(!iopf_param)) {
+		ret = -EINVAL;
+		goto err_bad_iopf;

Nit: the err_bad_iopf path is only ever returning -EINVAL, so you could simply hard-code that there...

Quoted 6 lines

drivers/iommu/io-pgfault.c static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · line 237

Quoted 4 lines
drivers/iommu/io-pgfault.c +2 -1
+	}
  	if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) {
-		report_partial_fault(iopf_param, fault);
+		ret = report_partial_fault(iopf_param, fault);

...and declare "ret" here in the one scope it's actually needed.

Sure, will refactor this in the next version. Let me also swap the signed-off-bys and prefix with rc as suggested by Jason in v3.

Quoted 11 lines

Thanks,
Robin.

drivers/iommu/io-pgfault.c static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · line 244

Quoted 7 lines

iopf_put_dev_fault_param(iopf_param);
/* A request that is not the last does not need to be ack'd */

drivers/iommu/io-pgfault.c +2 -1
-		return;
+
+		return ret;
  	}
  	/*

Thanks,
Pranjal