RDMA/rxe: Fix deadlock in rxe_do_local_ops()
Subsystems (2)
PATCH 1/1 RDMA/rxe: Fix deadlock in rxe_do_local_ops()
When a local operation (invalidate mr, reg mr, bind mw) is finished there will be no ack packet coming from a responder to cause the wqe to be completed. This may happen anyway if a subsequent wqe performs IO. Currently if the wqe is signalled the completer tasklet is scheduled immediately but not otherwise.
This leads to a deadlock if the next wqe has the fence bit set in send flags and the operation is not signalled. This patch removes the condition that the wqe must be signalled in order to schedule the completer tasklet which is the simplest fix for this deadlock and is fairly low cost. This is the analog for local operations of always setting the ackreq bit in all last or only request packets even if the operation is not signalled.
--- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/infiniband/sw/rxe/rxe_req.c +5 -3
@@ -586,9 +586,11 @@ static int rxe_do_local_ops(struct rxe_qp *qp, struct rxe_send_wqe *wqe) wqe->status =IB_WC_SUCCESS ; qp->req.wqe_index =queue_next_index (qp->sq.queue, qp->req.wqe_index); -if ((wqe->wr.send_flags &IB_SEND_SIGNALED ) || - qp->sq_sig_type ==IB_SIGNAL_ALL_WR ) -rxe_run_task (&qp->comp.task,1 ); +/* There is no ack coming for local work requests +* which can lead to a deadlock. So go ahead and complete +* it now. +*/ +rxe_run_task (&qp->comp.task,1 );return 0 ; }
base-commit: c5eb0a61238dd6faf37f58c9ce61c9980aaffd7a
Signature · 1 line
2.34.1
Quoted 14 lines · Bob Pearson
When a local operation (invalidate mr, reg mr, bind mw) is finished there will be no ack packet coming from a responder to cause the wqe to be completed. This may happen anyway if a subsequent wqe performs IO. Currently if the wqe is signalled the completer tasklet is scheduled immediately but not otherwise.
This leads to a deadlock if the next wqe has the fence bit set in send flags and the operation is not signalled. This patch removes the condition that the wqe must be signalled in order to schedule the completer tasklet which is the simplest fix for this deadlock and is fairly low cost. This is the analog for local operations of always setting the ackreq bit in all last or only request packets even if the operation is not signalled.
Quoted 6 lines
Reported-by Jenny Hack <[email protected]>Fixes c1a411268a4b1 ("RDMA/rxe: Move local ops to subroutine")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
Applied to for-next, thanks
Jason
Pearson, Robert B
AppliedThanks Jason.
-----Original Message-----
From: Jason Gunthorpe <[email protected]>
Sent: Thursday, June 30, 2022 6:58 PM
To: Bob Pearson <[email protected]>
Cc: [email protected]; Hack, Jenny (Ft. Collins) <[email protected]>; Zago, Frank <[email protected]>; [email protected]
Subject: Re: [PATCH] RDMA/rxe: Fix deadlock in rxe_do_local_ops()
Quoted 14 lines · Bob Pearson
When a local operation (invalidate mr, reg mr, bind mw) is finished there will be no ack packet coming from a responder to cause the wqe to be completed. This may happen anyway if a subsequent wqe performs IO. Currently if the wqe is signalled the completer tasklet is scheduled immediately but not otherwise.
This leads to a deadlock if the next wqe has the fence bit set in send flags and the operation is not signalled. This patch removes the condition that the wqe must be signalled in order to schedule the completer tasklet which is the simplest fix for this deadlock and is fairly low cost. This is the analog for local operations of always setting the ackreq bit in all last or only request packets even if the operation is not signalled.
Quoted 6 lines
Reported-by Jenny Hack <[email protected]>Fixes c1a411268a4b1 ("RDMA/rxe: Move local ops to subroutine")Signed-off-by Bob Pearson <[email protected]>--- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
Applied to for-next, thanks
Jason