Skip to contentSkip to conversation

Fix for IBI handling order

PATCH Applied Zbigniew Lukwinski 2 participants 3 messages Oct 15, 2023 – Nov 3, 2023 I3C

lore
Share & tools
lore
#0 lore raw

cover Fix for IBI handling order

I would like to introduce here patch to cover IBI handling order broken.

IBI shall be handled in order they appear on the bus. Otherwise could hit case when order of handling them in device driver will be different. It may lead to invalid assembling fragmented packets or events order broken.

There are two use cases when this problem can happen.

1. MCTP over I3C (aligned with DMTF specification)
Here problem can happen when target device (CPU) wants to send MCTP request fragment into a few MCTP packets. In such case flow will look like (having three MCTP packets):
a. CPU wants to send first MCTP packet
b. BMC gets IBI for the first MCTP packet and task is created to handle that IBI
d. created task handles IBI and runs read transaction to get MCTP packet and waits for read transaction completion
e. when read transaction happens, CPU wants to send second MCTP packet right away
f. BMC gets IBI for the second MCTP packet and task is created to handle that IBI
h. created task (second one) handles IBI and runs read transaction to get MCTP packet and waits for read transaction completion
e. when read transaction happens, CPU wants to send third MCTP packet right away
f. BMC gets IBI for the third MCTP packet and task is created to handle that IBI

At this point having three IBI task without explicitly configured order of execution which could result with the second one will go first and second MCTP packet will reach MCTP recipient first.

2. Debug for I3C (aligned with MIPI specification)
Here problem can happen when target device (CPU) sends multiple IBIs one after another. This could result with the wrong event order will reach recipient/consumer.

Added separate workqueue with option WQ_MEM_RECLAIM for each device driver. This ensures IBI handling order and improves IBI handling performance: IBI handlers for device B are not blocked by IBI handlers for device A.

Zbigniew Lukwinski (1):
i3c: master: handle IBIs in order they came

 drivers/i3c/master.c       | 14 +++++++++++++-
 include/linux/i3c/master.h |  4 +++-
 2 files changed, 16 insertions(+), 2 deletions(-)
Signature · 7 lines
2.34.1


-- 
linux-i3c mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-i3c
#1 lore raw

PATCH 1/1 i3c: master: handle IBIs in order they came

IBI shall be handled in order they appear on the bus. Otherwise could hit case when order of handling them in device driver will be different. It may lead to invalid assembling fragmented packets or events order broken.

Added separate workqueue with option WQ_MEM_RECLAIM for each device driver. This ensures IBI handling order and improves IBI handling performance: IBI handlers for device B are not blocked by IBI handlers for device A.

Original solution (single workqueue in main driver) was able to handle also general IBI (not related to specific device) like HJ or MR. So leaving this for such purposes.

---
 drivers/i3c/master.c       | 14 +++++++++++++-
 include/linux/i3c/master.h |  4 +++-
 2 files changed, 16 insertions(+), 2 deletions(-)
drivers/i3c/master.c +13 -1
index 87283e4a4607..7d3785d11336 100644
@@ -2372,7 +2372,7 @@ static void i3c_master_unregister_i3c_devs(struct i3c_master_controller *master)
 void i3c_master_queue_ibi(struct i3c_dev_desc *dev, struct i3c_ibi_slot *slot)
 {
 	atomic_inc(&dev->ibi->pending_ibis);
-	queue_work(dev->common.master->wq, &slot->work);
+	queue_work(dev->ibi->wq, &slot->work);
 }
 EXPORT_SYMBOL_GPL(i3c_master_queue_ibi);
 
@@ -2813,6 +2813,12 @@ int i3c_dev_request_ibi_locked(struct i3c_dev_desc *dev,
 	if (!ibi)
 		return -ENOMEM;
 
+	ibi->wq = alloc_ordered_workqueue(dev_name(i3cdev_to_dev(dev->dev)), WQ_MEM_RECLAIM);
+	if (!ibi->wq) {
+		kfree(ibi);
+		return -ENOMEM;
+	}
+
 	atomic_set(&ibi->pending_ibis, 0);
 	init_completion(&ibi->all_ibis_handled);
 	ibi->handler = req->handler;
@@ -2840,6 +2846,12 @@ void i3c_dev_free_ibi_locked(struct i3c_dev_desc *dev)
 		WARN_ON(i3c_dev_disable_ibi_locked(dev));
 
 	master->ops->free_ibi(dev);
+
+	if (dev->ibi->wq) {
+		destroy_workqueue(dev->ibi->wq);
+		dev->ibi->wq = NULL;
+	}
+
 	kfree(dev->ibi);
 	dev->ibi = NULL;
 }
include/linux/i3c/master.h +3 -1
index 0b52da4f2346..0a2a33a0f3af 100644
@@ -129,6 +129,7 @@ struct i3c_ibi_slot {
  *		     rejected by the master
  * @num_slots: number of IBI slots reserved for this device
  * @enabled: reflect the IBI status
+ * @wq: workqueue used to execute IBI handlers.
  * @handler: IBI handler specified at i3c_device_request_ibi() call time. This
  *	     handler will be called from the controller workqueue, and as such
  *	     is allowed to sleep (though it is recommended to process the IBI
@@ -151,6 +152,7 @@ struct i3c_device_ibi_info {
 	unsigned int max_payload_len;
 	unsigned int num_slots;
 	unsigned int enabled;
+	struct workqueue_struct *wq;
 	void (*handler)(struct i3c_device *dev,
 			const struct i3c_ibi_payload *payload);
 };
@@ -469,7 +471,7 @@ struct i3c_master_controller_ops {
  * @boardinfo.i2c: list of I2C boardinfo objects
  * @boardinfo: board-level information attached to devices connected on the bus
  * @bus: I3C bus exposed by this master
- * @wq: workqueue used to execute IBI handlers. Can also be used by master
+ * @wq: workqueue which can be used by master
  *	drivers if they need to postpone operations that need to take place
  *	in a thread context. Typical examples are Hot Join processing which
  *	requires taking the bus lock in maintenance, which in turn, can only
Signature · 7 lines
2.34.1


-- 
linux-i3c mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-i3c
maintainer Applied
#2 lore raw
In reply to Zbigniew Lukwinski
Quoted 9 lines · Zbigniew Lukwinski

I would like to introduce here patch to cover IBI handling order broken.

IBI shall be handled in order they appear on the bus. Otherwise could hit case when order of handling them in device driver will be different. It may lead to invalid assembling fragmented packets or events order broken.

There are two use cases when this problem can happen.

[...]

Applied, thanks!

[1/1] i3c: master: handle IBIs in order they came

      commit: 9fd00df05e81a2e1080ce6e9abc35533dca99d74

Best regards,

Signature · 8 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

-- 
linux-i3c mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-i3c