Skip to contentSkip to conversation

iw_cxgb4: only insert drain cqes if wq is flushed

PATCH Applied Steve Wise 2 participants 2 messages Nov 27, 2017 – Dec 11, 2017 1 message not in the archive

Subsystems (2)
InfiniBand CXGB4 iWARP RNIC driver (IW_CXGB4)
lore
Share & tools
lore

Steve Wise

author
#0 lore raw

PATCH 1/1 iw_cxgb4: only insert drain cqes if wq is flushed

Only insert our special drain CQEs to support ib_drain_sq/rq() after the wq is flushed. Otherwise, existing but not yet polled CQEs can be returned out of order to the user application. This can happen when the QP has exited RTS but not yet flushed the QP, which can happen during a normal close (vs abortive close).

In addition never count the drain CQEs when determining how many CQEs need to be synthesized during the flush operation. This latter issue should never happen if the QP is properly flushed before inserting the drain CQE, but I wanted to avoid corrupting the CQ state. So we handle it and log a warning once.

Fixes 4fe7c2962e11 ("iw_cxgb4: refactor sq/rq drain logic")
Signed-off-by Steve Wise <[email protected]>
Cc [email protected]
---
 drivers/infiniband/hw/cxgb4/cq.c |  5 +++++
 drivers/infiniband/hw/cxgb4/qp.c | 14 ++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)
drivers/infiniband/hw/cxgb4/cq.c +5 -0
index ea55e95cd2c5..b7bfc536e00f 100644
@@ -395,6 +395,11 @@ void c4iw_flush_hw_cq(struct c4iw_cq *chp)
 
 static int cqe_completes_wr(struct t4_cqe *cqe, struct t4_wq *wq)
 {
+	if (CQE_OPCODE(cqe) == C4IW_DRAIN_OPCODE) {
+		WARN_ONCE(1, "Unexpected DRAIN CQE qp id %u!\n", wq->sq.qid);
+		return 0;
+	}
+
 	if (CQE_OPCODE(cqe) == FW_RI_TERMINATE)
 		return 0;
drivers/infiniband/hw/cxgb4/qp.c +12 -2
index 355e288ec969..38bddd02a943 100644
@@ -868,7 +868,12 @@ int c4iw_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
 
 	qhp = to_c4iw_qp(ibqp);
 	spin_lock_irqsave(&qhp->lock, flag);
-	if (t4_wq_in_error(&qhp->wq)) {
+
+	/*
+	 * If the qp has been flushed, then just insert a special
+	 * drain cqe.
+	 */
+	if (qhp->wq.flushed) {
 		spin_unlock_irqrestore(&qhp->lock, flag);
 		complete_sq_drain_wr(qhp, wr);
 		return err;

@@ -1011,7 +1016,12 @@ int c4iw_post_receive(struct ib_qp *ibqp, struct ib_recv_wr *wr,
 
 	qhp = to_c4iw_qp(ibqp);
 	spin_lock_irqsave(&qhp->lock, flag);
-	if (t4_wq_in_error(&qhp->wq)) {
+
+	/*
+	 * If the qp has been flushed, then just insert a special
+	 * drain cqe.
+	 */
+	if (qhp->wq.flushed) {
 		spin_unlock_irqrestore(&qhp->lock, flag);
 		complete_rq_drain_wr(qhp, wr);
 		return err;
Signature · 6 lines
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

Applied
#2 lore raw
Quoted 12 lines · Steve Wise

Only insert our special drain CQEs to support ib_drain_sq/rq() after the wq is flushed. Otherwise, existing but not yet polled CQEs can be returned out of order to the user application. This can happen when the QP has exited RTS but not yet flushed the QP, which can happen during a normal close (vs abortive close).

In addition never count the drain CQEs when determining how many CQEs need to be synthesized during the flush operation. This latter issue should never happen if the QP is properly flushed before inserting the drain CQE, but I wanted to avoid corrupting the CQ state. So we handle it and log a warning once.

Quoted 6 lines
Fixes 4fe7c2962e11 ("iw_cxgb4: refactor sq/rq drain logic")
Signed-off-by Steve Wise <[email protected]>
Cc [email protected]
 drivers/infiniband/hw/cxgb4/cq.c |  5 +++++
 drivers/infiniband/hw/cxgb4/qp.c | 14 ++++++++++++--
 2 files changed, 17 insertions(+), 2 deletions(-)

Thanks applied to -rc

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html