Skip to contentSkip to conversation

iommufd/fault: Two bug fixes prior to vEVENTQ

PATCH Superseded Nicolin Chen 4 participants 10 messages Jan 14, 2025 – Jan 15, 2025

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#1 lore raw

PATCH 1/2 iommufd/fault: Destroy response and mutex in iommufd_fault_destroy()

In reply to Nicolin Chen

Both were missing in the initial patch.

Fixes 07838f7fd529 ("iommufd: Add iommufd fault object")
---
 drivers/iommu/iommufd/fault.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/iommu/iommufd/fault.c +8 -0
index 1fe804e28a86..685510224d05 100644
@@ -213,6 +213,7 @@ void iommufd_fault_destroy(struct iommufd_object *obj)
 {
 	struct iommufd_fault *fault = container_of(obj, struct iommufd_fault, obj);
 	struct iopf_group *group, *next;
+	unsigned long index;
 
 	/*
 	 * The iommufd object's reference count is zero at this point.
@@ -225,6 +226,13 @@ void iommufd_fault_destroy(struct iommufd_object *obj)
 		iopf_group_response(group, IOMMU_PAGE_RESP_INVALID);
 		iopf_free_group(group);
 	}
+	xa_for_each(&fault->response, index, group) {
+		xa_erase(&fault->response, index);
+		iopf_group_response(group, IOMMU_PAGE_RESP_INVALID);
+		iopf_free_group(group);
+	}
+	xa_destroy(&fault->response);
+	mutex_destroy(&fault->mutex);
 }
 
 static void iommufd_compose_fault_message(struct iommu_fault *fault,
Signature · 1 line
2.43.0
#0 lore raw

cover iommufd/fault: Two bug fixes prior to vEVENTQ

It is too late to get the vEVENTQ series merged in this cycle, while some bug fixes for the fault.c file can go in first. These fix two small bugs.

I will send vEVENTQ v6 later to get reviewed. And hopefully that can get merged in the early weeks of the next cycle.

Thanks
Nicolin

Nicolin Chen (2):
iommufd/fault: Destroy response and mutex in iommufd_fault_destroy() iommufd/fault: Use a separate spinlock to protect fault->deliver list

 drivers/iommu/iommufd/iommufd_private.h | 26 +++++++++++++
 drivers/iommu/iommufd/fault.c           | 51 ++++++++++++++++---------
 2 files changed, 58 insertions(+), 19 deletions(-)
Signature · 1 line
2.43.0
#5 lore raw

PATCH 2/2 iommufd/fault: Use a separate spinlock to protect fault->deliver list

The fault->mutex was to serialize the fault read()/write() fops and the iommufd_fault_auto_response_faults(). And it was also conveniently used to protect fault->deliver in poll() and iommufd_fault_iopf_handler().

However, copy_from/to_user() may sleep if pagefaults are enabled. Thus, they could take a long time to wait for user pages to swap in, blocking iommufd_fault_iopf_handler() and its caller that is typically a shared IRQ handler of an IOMMU driver, resulting in a potential global DOS.

Instead of resuing the mutex to protect the fault->deliver list, add a separate spinlock to do the job, so iommufd_fault_iopf_handler() would no longer be blocked by copy_from/to_user().

Provide two list manipulation helpers for fault->deliver:
- Extract the first iopf_group out of the fault->deliver list
- Restore an iopf_group back to the head of the fault->deliver list

Replace list_first_entry and list_for_each accordingly.

Fixes 07838f7fd529 ("iommufd: Add iommufd fault object")
---
 drivers/iommu/iommufd/iommufd_private.h | 26 +++++++++++++++
 drivers/iommu/iommufd/fault.c           | 43 ++++++++++++++-----------
 2 files changed, 50 insertions(+), 19 deletions(-)
drivers/iommu/iommufd/iommufd_private.h +26 -0
index b6d706cf2c66..d3097c857abf 100644
@@ -445,12 +445,38 @@ struct iommufd_fault {
 
 	/* The lists of outstanding faults protected by below mutex. */
 	struct mutex mutex;
+	spinlock_t lock; /* protects the deliver list */
 	struct list_head deliver;
 	struct xarray response;
 
 	struct wait_queue_head wait_queue;
 };
 
+/* Extract the first node out of the fault->deliver list */
+static inline struct iopf_group *
+iommufd_fault_deliver_extract(struct iommufd_fault *fault)
+{
+	struct list_head *list = &fault->deliver;
+	struct iopf_group *group = NULL;
+
+	spin_lock(&fault->lock);
+	if (!list_empty(list)) {
+		group = list_first_entry(list, struct iopf_group, node);
+		list_del(&group->node);
+	}
+	spin_unlock(&fault->lock);
+	return group;
+}
+
+/* Restore a node back to the head in fault->deliver */
+static inline void iommufd_fault_deliver_restore(struct iommufd_fault *fault,
+						 struct iopf_group *group)
+{
+	spin_lock(&fault->lock);
+	list_add(&fault->deliver, &group->node);
+	spin_unlock(&fault->lock);
+}
+
 struct iommufd_attach_handle {
 	struct iommu_attach_handle handle;
 	struct iommufd_device *idev;
drivers/iommu/iommufd/fault.c +24 -19

112 lines not on this page · raw message

Signature · 1 line
2.43.0
maintainer Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH rc 1/2] iommufd/fault: Destroy response and mutex in iommufd_fault_destroy()

In reply to Nicolin Chen
Quoted 8 lines · Nicolin Chen

Both were missing in the initial patch.

Fixes 07838f7fd529 ("iommufd: Add iommufd fault object")
---
 drivers/iommu/iommufd/fault.c | 8 ++++++++
 1 file changed, 8 insertions(+)

Jason

maintainer Reviewed-by on v1 2/2
#6 lore raw

RE: [PATCH rc 2/2] iommufd/fault: Use a separate spinlock to protect fault->deliver list

In reply to Nicolin Chen

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 450

Quoted 10 lines

From: Nicolin Chen <[email protected]>
Sent: Wednesday, January 15, 2025 7:29 AM

drivers/iommu/iommufd/iommufd_private.h +1 -0
@@ -445,12 +445,38 @@ struct iommufd_fault {
 
 	/* The lists of outstanding faults protected by below mutex. */
 	struct mutex mutex;
+	spinlock_t lock; /* protects the deliver list */
 	struct list_head deliver;
 	struct xarray response;

Move 'mutex' together with response then?

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 457

Quoted 4 lines
drivers/iommu/iommufd/iommufd_private.h +3 -0
+/* Extract the first node out of the fault->deliver list */
+static inline struct iopf_group *
+iommufd_fault_deliver_extract(struct iommufd_fault *fault)

Probably simpler be iommufd_fault_fetch()

Quoted 8 lines
drivers/iommu/iommufd/fault.c +1 -1
@@ -102,17 +102,19 @@ static void iommufd_auto_response_faults(struct
 iommufd_hw_pagetable *hwpt,
 					 struct iommufd_attach_handle
 *handle)
 {
 	struct iommufd_fault *fault = hwpt->fault;
-	struct iopf_group *group, *next;
+	struct iopf_group *group;

drivers/iommu/iommufd/fault.c static void iommufd_auto_response_faults(struct · line 115

Quoted 10 lines
drivers/iommu/iommufd/fault.c +2 -2
 	unsigned long index;
 
 	if (!fault)
 		return;
 
 	mutex_lock(&fault->mutex);
-	list_for_each_entry_safe(group, next, &fault->deliver, node) {
-		if (group->attach_handle != &handle->handle)
+	for (group = iommufd_fault_deliver_extract(fault); group;
+	     group = iommufd_fault_deliver_extract(fault)) {
	while (group = iommufd_fault_fetch(fault)) {
		...
	}
Quoted 15 lines
drivers/iommu/iommufd/fault.c +5 -5
@@ -266,17 +268,20 @@ static ssize_t iommufd_fault_fops_read(struct file
 *filep, char __user *buf,
 		return -ESPIPE;
 
 	mutex_lock(&fault->mutex);
-	while (!list_empty(&fault->deliver) && count > done) {
-		group = list_first_entry(&fault->deliver,
-					 struct iopf_group, node);
-
-		if (group->fault_count * fault_size > count - done)
+	for (group = iommufd_fault_deliver_extract(fault); group;
+	     group = iommufd_fault_deliver_extract(fault)) {
+		if (done >= count ||
+		    group->fault_count * fault_size > count - done) {
+			iommufd_fault_deliver_restore(fault, group);

drivers/iommu/iommufd/fault.c static ssize_t iommufd_fault_fops_read(struct file · line 285

Quoted 10 lines
drivers/iommu/iommufd/fault.c +4 -1
 			break;
+		}
 
 		rc = xa_alloc(&fault->response, &group->cookie, group,
 			      xa_limit_32b, GFP_KERNEL);
-		if (rc)
+		if (rc) {
+			iommufd_fault_deliver_restore(fault, group);
 			break;
+		}

The scope of mutex can be reduced to just protect the smaller trunk touching fault->response.

Otherwise looks good:

Reviewed-by on v1 2/2
#8 lore raw

Re: [PATCH rc 2/2] iommufd/fault: Use a separate spinlock to protect fault->deliver list

In reply to Nicolin Chen
Quoted 33 lines · Nicolin Chen

The fault->mutex was to serialize the fault read()/write() fops and the iommufd_fault_auto_response_faults(). And it was also conveniently used to protect fault->deliver in poll() and iommufd_fault_iopf_handler().

However, copy_from/to_user() may sleep if pagefaults are enabled. Thus, they could take a long time to wait for user pages to swap in, blocking iommufd_fault_iopf_handler() and its caller that is typically a shared IRQ handler of an IOMMU driver, resulting in a potential global DOS.

Instead of resuing the mutex to protect the fault->deliver list, add a separate spinlock to do the job, so iommufd_fault_iopf_handler() would no longer be blocked by copy_from/to_user().

Provide two list manipulation helpers for fault->deliver:
- Extract the first iopf_group out of the fault->deliver list
- Restore an iopf_group back to the head of the fault->deliver list

Replace list_first_entry and list_for_each accordingly.

Fixes 07838f7fd529 ("iommufd: Add iommufd fault object")
---
  drivers/iommu/iommufd/iommufd_private.h | 26 +++++++++++++++
  drivers/iommu/iommufd/fault.c           | 43 ++++++++++++++-----------
  2 files changed, 50 insertions(+), 19 deletions(-)
drivers/iommu/iommufd/iommufd_private.h +0 -0
index b6d706cf2c66..d3097c857abf 100644

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 446

Quoted 2 lines
drivers/iommu/iommufd/iommufd_private.h +0 -0
@@ -445,12 +445,38 @@ struct iommufd_fault {
 
  	/* The lists of outstanding faults protected by below mutex. */

It's better to update above comment as well.

Quoted 20 lines
drivers/iommu/iommufd/iommufd_private.h +13 -0
  	struct mutex mutex;
+	spinlock_t lock; /* protects the deliver list */
  	struct list_head deliver;
  	struct xarray response;
 
  	struct wait_queue_head wait_queue;
  };
 
+/* Extract the first node out of the fault->deliver list */
+static inline struct iopf_group *
+iommufd_fault_deliver_extract(struct iommufd_fault *fault)
+{
+	struct list_head *list = &fault->deliver;
+	struct iopf_group *group = NULL;
+
+	spin_lock(&fault->lock);
+	if (!list_empty(list)) {
+		group = list_first_entry(list, struct iopf_group, node);
+		list_del(&group->node);
+	}

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 476

Quoted 10 lines
drivers/iommu/iommufd/iommufd_private.h +10 -0
+	spin_unlock(&fault->lock);
+	return group;
+}
+
+/* Restore a node back to the head in fault->deliver */
+static inline void iommufd_fault_deliver_restore(struct iommufd_fault *fault,
+						 struct iopf_group *group)
+{
+	spin_lock(&fault->lock);
+	list_add(&fault->deliver, &group->node);

This is not right. It should be

	list_add(&group->node, &fault->deliver);

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 478

Quoted 2 lines
drivers/iommu/iommufd/iommufd_private.h +2 -0
+	spin_unlock(&fault->lock);
+}

Others look good to me. With above addressed,

#7 lore raw

Re: [PATCH rc 2/2] iommufd/fault: Use a separate spinlock to protect fault->deliver list

In reply to Tian, Kevin
Quoted 5 lines · Kevin
Quoted 5 lines

From: Nicolin Chen <[email protected]>
Sent: Wednesday, January 15, 2025 7:29 AM

(quoted diff) +0 -0
@@ -445,12 +445,38 @@ struct iommufd_fault {
 
Quoted 7 lines

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 450

Quoted 5 lines
drivers/iommu/iommufd/iommufd_private.h +1 -0
 	/* The lists of outstanding faults protected by below mutex. */
 	struct mutex mutex;
+	spinlock_t lock; /* protects the deliver list */
 	struct list_head deliver;
 	struct xarray response;

Move 'mutex' together with response then?

Ack.

Quoted 6 lines

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 457

Quoted 4 lines
drivers/iommu/iommufd/iommufd_private.h +3 -0
+/* Extract the first node out of the fault->deliver list */
+static inline struct iopf_group *
+iommufd_fault_deliver_extract(struct iommufd_fault *fault)

Probably simpler be iommufd_fault_fetch()

We have deliver and response two lists. So I think that "deliver" would be necessary. Yet, I can do "fetch" v.s. "extract".

Quoted 13 lines
Quoted 13 lines
drivers/iommu/iommufd/fault.c +1 -1
@@ -102,17 +102,19 @@ static void iommufd_auto_response_faults(struct
 iommufd_hw_pagetable *hwpt,
 					 struct iommufd_attach_handle
 *handle)
 {
 	struct iommufd_fault *fault = hwpt->fault;
-	struct iopf_group *group, *next;
+	struct iopf_group *group;
 	unsigned long index;
 
 	if (!fault)
 		return;
 
Quoted 9 lines

drivers/iommu/iommufd/fault.c static void iommufd_auto_response_faults(struct iommufd_hw_pagetable *hwpt, · line 113

Quoted 5 lines
drivers/iommu/iommufd/fault.c +2 -2
 	mutex_lock(&fault->mutex);
-	list_for_each_entry_safe(group, next, &fault->deliver, node) {
-		if (group->attach_handle != &handle->handle)
+	for (group = iommufd_fault_deliver_extract(fault); group;
+	     group = iommufd_fault_deliver_extract(fault)) {
	while (group = iommufd_fault_fetch(fault)) {
		...
	}

Ah, right...how didn't I see this lol.

Quoted 19 lines
Quoted 19 lines
drivers/iommu/iommufd/fault.c +6 -5
@@ -266,17 +268,20 @@ static ssize_t iommufd_fault_fops_read(struct file
 *filep, char __user *buf,
 		return -ESPIPE;
 
 	mutex_lock(&fault->mutex);
-	while (!list_empty(&fault->deliver) && count > done) {
-		group = list_first_entry(&fault->deliver,
-					 struct iopf_group, node);
-
-		if (group->fault_count * fault_size > count - done)
+	for (group = iommufd_fault_deliver_extract(fault); group;
+	     group = iommufd_fault_deliver_extract(fault)) {
+		if (done >= count ||
+		    group->fault_count * fault_size > count - done) {
+			iommufd_fault_deliver_restore(fault, group);
 			break;
+		}
 
Quoted 10 lines

drivers/iommu/iommufd/fault.c static ssize_t iommufd_fault_fops_read(struct file *filep, char __user *buf, · line 284

Quoted 7 lines
drivers/iommu/iommufd/fault.c +3 -1
 		rc = xa_alloc(&fault->response, &group->cookie, group,
 			      xa_limit_32b, GFP_KERNEL);
-		if (rc)
+		if (rc) {
+			iommufd_fault_deliver_restore(fault, group);
 			break;
+		}

The scope of mutex can be reduced to just protect the smaller trunk touching fault->response.

Ack.

Quoted 3 lines

Otherwise looks good:

Thanks!
Nicolin

#9 lore raw

Re: [PATCH rc 2/2] iommufd/fault: Use a separate spinlock to protect fault->deliver list

In reply to Baolu Lu
Quoted 10 lines · Baolu Lu

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 445

Quoted 2 lines · Nicolin Chen
drivers/iommu/iommufd/iommufd_private.h +0 -0
@@ -445,12 +445,38 @@ struct iommufd_fault {
  	/* The lists of outstanding faults protected by below mutex. */

It's better to update above comment as well.

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 450

Quoted 4 lines
drivers/iommu/iommufd/iommufd_private.h +1 -0
  	struct mutex mutex;
+	spinlock_t lock; /* protects the deliver list */
  	struct list_head deliver;
  	struct xarray response;

Ack. I'll do:
- /* The lists of outstanding faults protected by below mutex. */
- struct mutex mutex;
+ spinlock_t lock; /* protects the deliver list */
struct list_head deliver;
+ struct mutex mutex; /* serializes response flows */
struct xarray response;

Quoted 27 lines
Quoted 14 lines
drivers/iommu/iommufd/iommufd_private.h +12 -0
  	struct wait_queue_head wait_queue;
  };
+/* Extract the first node out of the fault->deliver list */
+static inline struct iopf_group *
+iommufd_fault_deliver_extract(struct iommufd_fault *fault)
+{
+	struct list_head *list = &fault->deliver;
+	struct iopf_group *group = NULL;
+
+	spin_lock(&fault->lock);
+	if (!list_empty(list)) {
+		group = list_first_entry(list, struct iopf_group, node);
+		list_del(&group->node);
+	}

drivers/iommu/iommufd/iommufd_private.h struct iommufd_fault { · line 476

Quoted 10 lines
drivers/iommu/iommufd/iommufd_private.h +10 -0
+	spin_unlock(&fault->lock);
+	return group;
+}
+
+/* Restore a node back to the head in fault->deliver */
+static inline void iommufd_fault_deliver_restore(struct iommufd_fault *fault,
+						 struct iopf_group *group)
+{
+	spin_lock(&fault->lock);
+	list_add(&fault->deliver, &group->node);

This is not right. It should be

Quoted 4 lines
	list_add(&group->node, &fault->deliver);

Review struct iommufd_fault { · line 475

Quoted 2 lines
(quoted diff) +2 -0
+	spin_unlock(&fault->lock);
+}

Oh, right!

Quoted 3 lines

Others look good to me. With above addressed,

Thanks!
Nicolin