Skip to contentSkip to conversation

IB/hfi1: Fixes for 5.1 rc cycle

PATCH Applied Dennis Dalessandro 2 participants 4 messages Mar 18, 2019 – Mar 27, 2019 1 message not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover IB/hfi1: Fixes for 5.1 rc cycle

Here is a set of small fixes that we would like to land in the rc cycle for 5.1. One of them is marked as stable.

The first patch fixes an unnecessary stack trace due to incorrect WQ flags. The next three fix issues uncovered due to TID RDMA which was brought in for 5.1. The last one is a bug in VNIC.

---

Kaike Wan (4):
IB/hfi1: Failed to drain send queue when QP is put into error state IB/hfi1: Clear the IOWAIT pending bits when QP is put into error state IB/hfi1: Eliminate opcode tests on mr deref
IB/hfi1: Fix the allocation of RSM table

Mike Marciniszyn (1):
IB/hfi1: Fix WQ_MEM_RECLAIM warning

 drivers/infiniband/hw/hfi1/chip.c |   26 +++++++++++++++++++-------
 drivers/infiniband/hw/hfi1/init.c |    3 ++-
 drivers/infiniband/hw/hfi1/qp.c   |    4 +++-
 drivers/infiniband/hw/hfi1/rc.c   |    4 ++--
 4 files changed, 26 insertions(+), 11 deletions(-)
#1 lore raw

PATCH 2/5 IB/hfi1: Failed to drain send queue when QP is put into error state

From: Kaike Wan <[email protected]>

When a QP is put into error state, all pending requests in the send work queue should be drained. The following sequence of events could lead to a failure, causing a request to hang:
(1) The QP builds a packet and tries to send through SDMA engine. However, PIO engine is still busy. Consequently, this packet is put on the QP's tx list and the QP is put on the PIO waiting list. The field qp->s_flags is set with HFI1_S_WAIT_PIO_DRAIN;
(2) The QP is put into error state by the user application and notify_error_qp() is called, which removes the QP from the PIO waiting list and the packet from the QP's tx list. In addition, qp->s_flags is cleared of RVT_S_ANY_WAIT_IO bits, which does not include HFI1_S_WAIT_PIO_DRAIN bit;
(3) The hfi1_schdule_send() function is called to drain the QP's send queue. Subsequently, hfi1_do_send() is called. Since the flag bit HFI1_S_WAIT_PIO_DRAIN is set in qp->s_flags, hfi1_send_ok() fails. As a result, hfi1_do_send() bails out without draining any request from the send queue;
(4) The PIO engine completes the sending and tries to wake up any QP on its waiting list. But the QP has been removed from the PIO waiting list and therefore is kept in sleep forever.

The fix is to clear qp->s_flags of HFI1_S_ANY_WAIT_IO bits in step (2). HFI1_S_ANY_WAIT_IO includes RVT_S_ANY_WAIT_IO and HFI1_S_WAIT_PIO_DRAIN.

Fixes 2e2ba09e48b7 ("IB/rdmavt, IB/hfi1: Create device dependent s_flags")
Cc <[email protected]> # 4.19.x+
---
 drivers/infiniband/hw/hfi1/qp.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
drivers/infiniband/hw/hfi1/qp.c +1 -1
index 9b643c2..64889ed 100644
@@ -898,7 +898,7 @@ void notify_error_qp(struct rvt_qp *qp)
 		if (!list_empty(&priv->s_iowait.list) &&
 		    !(qp->s_flags & RVT_S_BUSY) &&
 		    !(priv->s_flags & RVT_S_BUSY)) {
-			qp->s_flags &= ~RVT_S_ANY_WAIT_IO;
+			qp->s_flags &= ~HFI1_S_ANY_WAIT_IO;
 			list_del_init(&priv->s_iowait.list);
 			priv->s_iowait.lock = NULL;
 			rvt_put_qp(qp);
#3 lore raw

Re: [PATCH for-rc 2/5] IB/hfi1: Failed to drain send queue when QP is put into error state

In reply to a message not in the archive
Quoted 7 lines · Sasha Levin

Hi,

[This is an automated email]

This commit has been processed because it contains a "Fixes:" tag, fixing commit: 2e2ba09e48b7 IB/rdmavt, IB/hfi1: Create device dependent s_flags.

This patch hasn't even been accepted to any trees yet. Why are you looking at it?

Jason

#4 lore raw
In reply to Dennis Dalessandro
Quoted 8 lines · Dennis Dalessandro

Here is a set of small fixes that we would like to land in the rc cycle for 5.1. One of them is marked as stable.

The first patch fixes an unnecessary stack trace due to incorrect WQ flags. The next three fix issues uncovered due to TID RDMA which was brought in for 5.1. The last one is a bug in VNIC.

Quoted 5 lines

Kaike Wan (4):
IB/hfi1: Failed to drain send queue when QP is put into error state IB/hfi1: Clear the IOWAIT pending bits when QP is put into error state IB/hfi1: Eliminate opcode tests on mr deref
IB/hfi1: Fix the allocation of RSM table

Applied these ones to for-rc

Quoted 2 lines

Mike Marciniszyn (1):
IB/hfi1: Fix WQ_MEM_RECLAIM warning

Let us see how the discussion goes here

Thanks,
Jason