RDMA/iw_cxgb4: initiate CLOSE when entering TERM
Subsystems (2)
PATCH 1/1 RDMA/iw_cxgb4: initiate CLOSE when entering TERM
- As per draft-hilland-iwarp-verbs-v1.0, sec 6.2.3, always initiate a CLOSE when entering into TERM state.
- In c4iw_modify_qp(), disconnect operation should only be performed when the modify_qp call is invoked from ib_core. And all other internal modify_qp calls(invoked within iw_cxgb4) that needs 'disconnect' should call c4iw_ep_disconnect() explicitly after modify_qp. Otherwise, deadlocks like below can occur:
Call Trace: schedule+0x2f/0xa0 schedule_preempt_disabled+0xa/0x10 __mutex_lock.isra.5+0x2d0/0x4a0 c4iw_ep_disconnect+0x39/0x430 => tries to reacquire ep lock again c4iw_modify_qp+0x468/0x10d0 rx_data+0x218/0x570 => acquires ep lock process_work+0x5f/0x70 process_one_work+0x1a7/0x3b0 worker_thread+0x30/0x390 kthread+0x112/0x130 ret_from_fork+0x35/0x40
transitioning to TERMINATE state")
--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++++ drivers/infiniband/hw/cxgb4/qp.c | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-)
drivers/infiniband/hw/cxgb4/cm.c +4 -0
@@ -3036,6 +3036,10 @@ static int terminate(struct c4iw_dev *dev, struct sk_buff *skb) C4IW_QP_ATTR_NEXT_STATE, &attrs, 1); } +/* As per draft-hilland-iwarp-verbs-v1.0, sec 6.2.3, +* when entering the TERM state the RNIC MUST initiate a CLOSE. +*/ +c4iw_ep_disconnect (ep,1 ,GFP_KERNEL );c4iw_put_ep (&ep->com); }else pr_warn ("TERM received tid %u no ep/qp\n" , tid);
drivers/infiniband/hw/cxgb4/qp.c +2 -2
@@ -1948,10 +1948,10 @@ int c4iw_modify_qp(struct c4iw_dev *rhp, struct c4iw_qp *qhp, qhp->attr.layer_etype = attrs->layer_etype; qhp->attr.ecode = attrs->ecode; ep = qhp->ep; -c4iw_get_ep (&ep->com); - disconnect =1 ;if (!internal) { +c4iw_get_ep (&ep->com); terminate =1 ; + disconnect =1 ; }else { terminate = qhp->attr.send_term; ret =rdma_fini (rhp, qhp, ep);
Signature · 1 line
2.23.0.rc0
Jason Gunthorpe
AppliedQuoted 22 lines · Krishnamraju Eraparaju
- As per draft-hilland-iwarp-verbs-v1.0, sec 6.2.3, always initiate a CLOSE when entering into TERM state.
- In
c4iw_modify_qp(), disconnect operation should only be performed when themodify_qpcall is invoked fromib_core. And all other internalmodify_qpcalls(invoked within iw_cxgb4)that needs 'disconnect' should callc4iw_ep_disconnect()explicitly aftermodify_qp. Otherwise, deadlocks like below can occur:Call Trace: schedule+0x2f/0xa0 schedule_preempt_disabled+0xa/0x10 __mutex_lock.isra.5+0x2d0/0x4a0 c4iw_ep_disconnect+0x39/0x430 => tries to reacquire ep lock again c4iw_modify_qp+0x468/0x10d0 rx_data+0x218/0x570 => acquires ep lock process_work+0x5f/0x70 process_one_work+0x1a7/0x3b0 worker_thread+0x30/0x390 kthread+0x112/0x130 ret_from_fork+0x35/0x40
Quoted 7 lines
Fixes d2c33370ae73 ("RDMA/iw_cxgb4: Always disconnect when QP istransitioning to TERMINATE state")
Signed-off-by Krishnamraju Eraparaju <[email protected]>--- drivers/infiniband/hw/cxgb4/cm.c | 4 ++++ drivers/infiniband/hw/cxgb4/qp.c | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-)
Applied to for-rc, thanks
Jason