RDMA/rxe: Fix qp error handler
Subsystems (2)
PATCH v2 1/1 RDMA/rxe: Fix qp error handler
From: Zhu Yanjun <[email protected]>
About 7 spin locks in qp creation needs to be initialized. Now these spin locks are initialized in the function rxe_qp_init_misc. This will avoid the error "initialize spin locks before use".
--- drivers/infiniband/sw/rxe/rxe_qp.c | 12 ++++++++---- drivers/infiniband/sw/rxe/rxe_task.c | 1 - 2 files changed, 8 insertions(+), 5 deletions(-)
drivers/infiniband/sw/rxe/rxe_qp.c +8 -4
@@ -172,6 +172,14 @@ static void rxe_qp_init_misc(struct rxe_dev *rxe, struct rxe_qp *qp,spin_lock_init (&qp->state_lock); +spin_lock_init (&qp->req.task.state_lock); +spin_lock_init (&qp->resp.task.state_lock); +spin_lock_init (&qp->comp.task.state_lock); + +spin_lock_init (&qp->sq.sq_lock); +spin_lock_init (&qp->rq.producer_lock); +spin_lock_init (&qp->rq.consumer_lock); +atomic_set (&qp->ssn,0 );atomic_set (&qp->skb_out,0 ); }
@@ -231,7 +239,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, qp->req.opcode =-1 ; qp->comp.opcode =-1 ; -spin_lock_init (&qp->sq.sq_lock);skb_queue_head_init (&qp->req_pkts); rxe_init_task(rxe, &qp->req.task, qp,
@@ -282,9 +289,6 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, } } -spin_lock_init (&qp->rq.producer_lock); -spin_lock_init (&qp->rq.consumer_lock); -skb_queue_head_init (&qp->resp_pkts); rxe_init_task(rxe, &qp->resp.task, qp,
drivers/infiniband/sw/rxe/rxe_task.c +0 -1
@@ -98,7 +98,6 @@ int rxe_init_task(void *obj, struct rxe_task *task,tasklet_setup (&task->tasklet, rxe_do_task); task->state =TASK_STATE_START ; -spin_lock_init (&task->state_lock);return 0 ; }
Signature · 1 line
2.25.1
Quoted 55 lines · On 7/9/22 23:37, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
About 7 spin locks in qp creation needs to be initialized. Now these spin locks are initialized in the function
rxe_qp_init_misc. This will avoid the error "initialize spin locks before use".Reported-by [email protected]Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_qp.c | 12 ++++++++---- drivers/infiniband/sw/rxe/rxe_task.c | 1 - 2 files changed, 8 insertions(+), 5 deletions(-)drivers/infiniband/sw/rxe/rxe_qp.c +8
-4@@ -172,6 +172,14 @@ static void rxe_qp_init_misc(struct rxe_dev *rxe, struct rxe_qp *qp,spin_lock_init (&qp->state_lock); +spin_lock_init (&qp->req.task.state_lock); +spin_lock_init (&qp->resp.task.state_lock); +spin_lock_init (&qp->comp.task.state_lock); + +spin_lock_init (&qp->sq.sq_lock); +spin_lock_init (&qp->rq.producer_lock); +spin_lock_init (&qp->rq.consumer_lock); +atomic_set (&qp->ssn,0 );atomic_set (&qp->skb_out,0 ); }@@ -231,7 +239,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, qp->req.opcode =-1 ; qp->comp.opcode =-1 ; -spin_lock_init (&qp->sq.sq_lock);skb_queue_head_init (&qp->req_pkts); rxe_init_task(rxe, &qp->req.task, qp,@@ -282,9 +289,6 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, } } -spin_lock_init (&qp->rq.producer_lock); -spin_lock_init (&qp->rq.consumer_lock); -skb_queue_head_init (&qp->resp_pkts); rxe_init_task(rxe, &qp->resp.task, qp,drivers/infiniband/sw/rxe/rxe_task.c +0
-0
drivers/infiniband/sw/rxe/rxe_task.c int rxe_init_task(void *obj, struct rxe_task *task, · line 103
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_task.c +0
-1@@ -98,7 +98,6 @@ int rxe_init_task(void *obj, struct rxe_task *task,tasklet_setup (&task->tasklet, rxe_do_task); task->state =TASK_STATE_START ; -spin_lock_init (&task->state_lock);return 0 ; }
Zhu,
The task.state_lock spinlocks are an implementation detail of the tasklet code. Seems strange to move the spin_lock_init() calls up into the qp code for these. This breaks encapsulation. We (HPE) have a patch coming that extends the tasklet code to support tasklets and/or work queues which allow steering the work to specific cpus. This gives a significant performance boost for IO intensive work flows.
The only other issue with this patch is that for xrc QPs, which we don't support yet, the QPs only have one side implemented and there won't be a reason to do unneeded work. Not a big issue though.
Bob
Quoted 65 lines · Bob Pearson
Quoted 55 lines · On 7/9/22 23:37, [email protected] wrote:
From: Zhu Yanjun <[email protected]>
About 7 spin locks in qp creation needs to be initialized. Now these spin locks are initialized in the function
rxe_qp_init_misc. This will avoid the error "initialize spin locks before use".Reported-by [email protected]Signed-off-by Zhu Yanjun <[email protected]>--- drivers/infiniband/sw/rxe/rxe_qp.c | 12 ++++++++---- drivers/infiniband/sw/rxe/rxe_task.c | 1 - 2 files changed, 8 insertions(+), 5 deletions(-)drivers/infiniband/sw/rxe/rxe_qp.c +8
-4@@ -172,6 +172,14 @@ static void rxe_qp_init_misc(struct rxe_dev *rxe, struct rxe_qp *qp,spin_lock_init (&qp->state_lock); +spin_lock_init (&qp->req.task.state_lock); +spin_lock_init (&qp->resp.task.state_lock); +spin_lock_init (&qp->comp.task.state_lock); + +spin_lock_init (&qp->sq.sq_lock); +spin_lock_init (&qp->rq.producer_lock); +spin_lock_init (&qp->rq.consumer_lock); +atomic_set (&qp->ssn,0 );atomic_set (&qp->skb_out,0 ); }@@ -231,7 +239,6 @@ static int rxe_qp_init_req(struct rxe_dev *rxe, struct rxe_qp *qp, qp->req.opcode =-1 ; qp->comp.opcode =-1 ; -spin_lock_init (&qp->sq.sq_lock);skb_queue_head_init (&qp->req_pkts); rxe_init_task(rxe, &qp->req.task, qp,@@ -282,9 +289,6 @@ static int rxe_qp_init_resp(struct rxe_dev *rxe, struct rxe_qp *qp, } } -spin_lock_init (&qp->rq.producer_lock); -spin_lock_init (&qp->rq.consumer_lock); -skb_queue_head_init (&qp->resp_pkts); rxe_init_task(rxe, &qp->resp.task, qp,drivers/infiniband/sw/rxe/rxe_task.c +0
-0drivers/infiniband/sw/rxe/rxe_task.c int rxe_init_task(void *obj, struct rxe_task *task, · line 103
Quoted 7 lines
drivers/infiniband/sw/rxe/rxe_task.c +0
-1@@ -98,7 +98,6 @@ int rxe_init_task(void *obj, struct rxe_task *task,tasklet_setup (&task->tasklet, rxe_do_task); task->state =TASK_STATE_START ; -spin_lock_init (&task->state_lock);return 0 ; }Zhu,
Quoted 5 lines
The task.state_lock spinlocks are an implementation detail of the tasklet code. Seems strange to move the
spin_lock_init()calls up into the qp code for these. This breaks encapsulation. We (HPE) have a patch coming that extends the tasklet code to support tasklets and/or work queues which allow steering the work to specific cpus. This gives a significant performance boost for IO intensive work flows.
OK. The reason that I move spin_lock_init() into rxe_qp_init_misc is to avoid the error "initialize spin locks before use".
Thanks for sharing your features in HPE. If you want to backport these new features into linux upstream, I can
keep spin_lock_init in rxe_init_task for future use.
I will send the latest commit very soon.
And look forward to your feature that extends the tasklet code to support tasklets and/or work queues which allow steering the work to specific cpus in linux upstream.
I am curious about this feautre. And hope I can see it in linux upstream very soon ^_^
Zhu Yanjun
Quoted 5 lines
The only other issue with this patch is that for xrc QPs, which we don't support yet, the QPs only have one side implemented and there won't be a reason to do unneeded work. Not a big issue though.
Bob