RDMA/rxe: Add workqueue support for tasks
Subsystems (2)
PATCH v6 1/1 RDMA/rxe: Add workqueue support for tasks
Replace tasklets by work queues for the three main rxe tasklets:rxe_requester, rxe_completer and rxe_responder.
This patch is all that is left from an earlier patch series that also converted tasklets to workqueues with the same subject.
With this patch the rxe driver does not exhibit the soft lockups reported by Daisuke Matsuda in the link below.
This patch depends on an earlier patch series titled "RDMA/rxe: Correct qp reference counting" which corrects the same errors for the current tasklet version.
---
v6:
Fixed left over references to tasklets in the comments. Added WQ_UNBOUND to the parameters for alloc_workqueue(). This shows a significant performance improvement.
v5:
Based on corrected task logic for tasklets and simplified to only convert from tasklets to workqueues and not provide a flexible interface.
drivers/infiniband/sw/rxe/rxe.c | 9 ++- drivers/infiniband/sw/rxe/rxe_task.c | 84 ++++++++++++++++++---------- drivers/infiniband/sw/rxe/rxe_task.h | 9 ++- 3 files changed, 69 insertions(+), 33 deletions(-)
drivers/infiniband/sw/rxe/rxe.c +8 -1
@@ -212,10 +212,16 @@ static int __init rxe_module_init(void) {int err; - err =rxe_net_init (); + err =rxe_alloc_wq ();if (err)return err; + err =rxe_net_init (); +if (err) { +rxe_destroy_wq (); +return err; + } +rdma_link_register (&rxe_link_ops);pr_info ("loaded\n" );return 0 ;
@@ -226,6 +232,7 @@ static void __exit rxe_module_exit(void)rdma_link_unregister (&rxe_link_ops);ib_unregister_driver (RDMA_DRIVER_RXE );rxe_net_exit (); +rxe_destroy_wq ();pr_info ("unloaded\n" ); }
drivers/infiniband/sw/rxe/rxe_task.c +54 -30
200 lines not on this page · raw message
drivers/infiniband/sw/rxe/rxe_task.h +7 -2
@@ -22,23 +22,28 @@ enum {* called again. */ struct rxe_task { -struct tasklet_struct tasklet; +struct work_struct work;int state;spinlock_t lock;struct rxe_qp *qp; +const char *name;int (*func)(struct rxe_qp *qp);int ret;long num_sched;long num_done; }; +int rxe_alloc_wq (void ); + +void rxe_destroy_wq (void ); +/* * init rxe_task structure * qp => parameter to pass to func * func => function to call until it returns != 0 */ int rxe_init_task (struct rxe_task *task,struct rxe_qp *qp, -int (*func)(struct rxe_qp *)); +int (*func)(struct rxe_qp *),const char *name);/* cleanup task */ void rxe_cleanup_task (struct rxe_task *task);
base-commit: 6a22f7fbde87336002886583d053bfa1cd8ff1d1
Signature · 1 line
2.37.2
Quoted 7 lines · Bob Pearson
Replace tasklets by work queues for the three main rxe tasklets:
rxe_requester,rxe_completerandrxe_responder.This patch is all that is left from an earlier patch series that also converted tasklets to workqueues with the same subject.
Quoted 6 lines
With this patch the rxe driver does not exhibit the soft lockups reported by Daisuke Matsuda in the link below.
This patch depends on an earlier patch series titled "RDMA/rxe: Correct qp reference counting" which corrects the same errors for the current tasklet version.
Isn't it better to add the link to the patch series for easier reference?
Quoted 21 lines
Link:
https://lore.kernel.org/linux-rdma/TYCPR01MB845522FD536170D75068DD41E5099@TYCPR01MB8455.jpnprd01.prod.
outlook.com/Signed-off-by Ian Ziemba <[email protected]>Signed-off-by Bob Pearson <[email protected]>---
v6:
Fixed left over references to tasklets in the comments. AddedWQ_UNBOUNDto the parameters foralloc_workqueue(). This shows a significant performance improvement.
v5:
Based on corrected task logic for tasklets and simplified to only convert from tasklets to workqueues and not provide a flexible interface.drivers/infiniband/sw/rxe/rxe.c | 9 ++- drivers/infiniband/sw/rxe/rxe_task.c | 84 ++++++++++++++++++---------- drivers/infiniband/sw/rxe/rxe_task.h | 9 ++- 3 files changed, 69 insertions(+), 33 deletions(-)
<...>
Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_task.h +0
-1@@ -22,23 +22,28 @@ enum {* called again. */ struct rxe_task { -struct tasklet_struct tasklet;
drivers/infiniband/sw/rxe/rxe_task.h enum { · line 34
Quoted 10 lines
drivers/infiniband/sw/rxe/rxe_task.h +2
-0+struct work_struct work;int state;spinlock_t lock;struct rxe_qp *qp; +const char *name;int (*func)(struct rxe_qp *qp);int ret;long num_sched;long num_done; };
The member '*name' is newly added here but not used elsewhere. It seems we should just remove it.
drivers/infiniband/sw/rxe/rxe_task.h enum { · line 46
Quoted 13 lines
drivers/infiniband/sw/rxe/rxe_task.h +5
-1+int rxe_alloc_wq (void ); + +void rxe_destroy_wq (void ); +/* * init rxe_task structure * qp => parameter to pass to func * func => function to call until it returns != 0 */ int rxe_init_task (struct rxe_task *task,struct rxe_qp *qp, -int (*func)(struct rxe_qp *)); +int (*func)(struct rxe_qp *),const char *name);
This does not matches with the function definition. Kernel build fails here unless the 4th argument is removed. Additionally, the argument "*name" is not used in rxe_init_task().
Thanks,
Daisuke
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_task.h +0
-0/* cleanup task */ void rxe_cleanup_task (struct rxe_task *task);base-commit:
6a22f7fbde87336002886583d053bfa1cd8ff1d1--
2.37.2