RDMA/siw: Fix broken RDMA Read Fence/Resume logic.
Subsystems (2)
PATCH 1/1 RDMA/siw: Fix broken RDMA Read Fence/Resume logic.
Code unconditionally resumed fenced SQ processing after next RDMA Read completion, even if other RDMA Read responses are still outstanding, or ORQ is full. Also adds comments for better readability of fence processing, and removes orq_get_tail() helper, which is not needed anymore.
Fixes 8b6a361b8c48 ("rdma/siw: receive path")
Fixes a531975279f3 ("rdma/siw: main include file")
Reported-by Jared Holzman <[email protected]>
Signed-off-by Bernard Metzler <[email protected]>
--- drivers/infiniband/sw/siw/siw.h | 7 +------ drivers/infiniband/sw/siw/siw_qp_rx.c | 20 +++++++++++--------- 2 files changed, 12 insertions(+), 15 deletions(-)
drivers/infiniband/sw/siw/siw.h +1 -6
@@ -644,14 +644,9 @@ static inline struct siw_sqe *orq_get_current(struct siw_qp *qp)return &qp->orq[qp->orq_get % qp->attrs.orq_size]; } -static inline struct siw_sqe *orq_get_tail (struct siw_qp *qp) -{ -return &qp->orq[qp->orq_put % qp->attrs.orq_size]; -} -static inline struct siw_sqe *orq_get_free (struct siw_qp *qp) { -struct siw_sqe *orq_e =orq_get_tail (qp); +struct siw_sqe *orq_e = &qp->orq[qp->orq_put % qp->attrs.orq_size];if (READ_ONCE (orq_e->flags) ==0 )return orq_e;
drivers/infiniband/sw/siw/siw_qp_rx.c +11 -9
@@ -1153,11 +1153,12 @@ static int siw_check_tx_fence(struct siw_qp *qp)spin_lock_irqsave (&qp->orq_lock, flags); - rreq =orq_get_current (qp); -/* free current orq entry */ + rreq =orq_get_current (qp);WRITE_ONCE (rreq->flags,0 ); + qp->orq_get++; +if (qp->tx_ctx.orq_fence) {if (unlikely (tx_waiting->wr_status !=SIW_WR_QUEUED )) { pr_warn("siw: [QP %u]: fence resume: bad status %d\n",
@@ -1165,10 +1166,12 @@ static int siw_check_tx_fence(struct siw_qp *qp) rv = -EPROTO ; goto out; } -/* resume SQ processing */ +/* resume SQ processing, if possible */ if (tx_waiting->sqe.opcode ==SIW_OP_READ || tx_waiting->sqe.opcode ==SIW_OP_READ_LOCAL_INV ) { - rreq =orq_get_tail (qp); + +/* SQ processing was stopped because of a full ORQ */ + rreq =orq_get_free (qp);if (unlikely (!rreq)) {pr_warn ("siw: [QP %u]: no ORQE\n" ,qp_id (qp)); rv = -EPROTO ;
@@ -1181,15 +1184,14 @@ static int siw_check_tx_fence(struct siw_qp *qp) resume_tx =1 ; }else if (siw_orq_empty (qp)) { +/* +* SQ processing was stopped by fenced work request. +* Resume since all previous Read's are now completed. +*/ qp->tx_ctx.orq_fence =0 ; resume_tx =1 ; - }else { -pr_warn ("siw: [QP %u]: fence resume: orq idx: %d:%d\n" , -qp_id (qp), qp->orq_get, qp->orq_put); - rv = -EPROTO ; } } - qp->orq_get++; out:spin_unlock_irqrestore (&qp->orq_lock, flags);
Signature · 1 line
2.34.1
Quoted 6 lines · Bernard Metzler
Code unconditionally resumed fenced SQ processing after next RDMA Read completion, even if other RDMA Read responses are still outstanding, or ORQ is full. Also adds comments for better readability of fence processing, and removes
orq_get_tail()helper, which is not needed anymore.
Quoted 8 lines
Fixes 8b6a361b8c48 ("rdma/siw: receive path")Fixes a531975279f3 ("rdma/siw: main include file")Reported-by Jared Holzman <[email protected]>Signed-off-by Bernard Metzler <[email protected]>--- drivers/infiniband/sw/siw/siw.h | 7 +------ drivers/infiniband/sw/siw/siw_qp_rx.c | 20 +++++++++++--------- 2 files changed, 12 insertions(+), 15 deletions(-)
Applied to for-rc, thanks
Jason