Skip to contentSkip to conversation

ksmbd: call ib_drain_qp when disconnected

PATCH In review Namjae Jeon 2 participants 4 messages Oct 2, 2022 – Oct 3, 2022

Subsystems (2)
lore
Share & tools
lore

Namjae Jeon

author maintainer
#0 lore raw

PATCH 1/1 ksmbd: call ib_drain_qp when disconnected

When disconnected, call ib_drain_qp to cancel all pending work requests and prevent ksmbd_conn_handler_loop from waiting for a long time for those work requests to compelete.

---
 fs/ksmbd/transport_rdma.c | 2 ++
 1 file changed, 2 insertions(+)
fs/ksmbd/transport_rdma.c +2 -0
index 0315bca3d53b..096eda9ef873 100644
@@ -1527,6 +1527,8 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
 	}
 	case RDMA_CM_EVENT_DEVICE_REMOVAL:
 	case RDMA_CM_EVENT_DISCONNECTED: {
+		ib_drain_qp(t->qp);
+
 		t->status = SMB_DIRECT_CS_DISCONNECTED;
 		wake_up_interruptible(&t->wait_status);
 		wake_up_interruptible(&t->wait_reassembly_queue);
Signature · 1 line
2.25.1
reviewer
#1 lore raw
Quoted 15 lines · Namjae Jeon

When disconnected, call ib_drain_qp to cancel all pending work requests and prevent ksmbd_conn_handler_loop from waiting for a long time for those work requests to compelete.

---
  fs/ksmbd/transport_rdma.c | 2 ++
  1 file changed, 2 insertions(+)
fs/ksmbd/transport_rdma.c +0 -0
index 0315bca3d53b..096eda9ef873 100644

fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1534

Quoted 8 lines
fs/ksmbd/transport_rdma.c +2 -0
@@ -1527,6 +1527,8 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
  	}
  	case RDMA_CM_EVENT_DEVICE_REMOVAL:
  	case RDMA_CM_EVENT_DISCONNECTED: {
+		ib_drain_qp(t->qp);
+
  		t->status = SMB_DIRECT_CS_DISCONNECTED;
  		wake_up_interruptible(&t->wait_status);
  		wake_up_interruptible(&t->wait_reassembly_queue);

Because we're now flushing the cancelled work requests, don't we need to also wake up &t->wait_send_pending, to abort any waiters on the send wq?

Tom.

author maintainer
#2 lore raw

2022-10-03 3:11 GMT+09:00, Tom Talpey <[email protected]>:

Quoted 21 lines · Namjae Jeon
Quoted 20 lines

When disconnected, call ib_drain_qp to cancel all pending work requests and prevent ksmbd_conn_handler_loop from waiting for a long time for those work requests to compelete.

---
  fs/ksmbd/transport_rdma.c | 2 ++
  1 file changed, 2 insertions(+)
fs/ksmbd/transport_rdma.c +1 -0
index 0315bca3d53b..096eda9ef873 100644
@@ -1527,6 +1527,8 @@ static int smb_direct_cm_handler(struct rdma_cm_id
 *cm_id,
  	}
  	case RDMA_CM_EVENT_DEVICE_REMOVAL:
  	case RDMA_CM_EVENT_DISCONNECTED: {
+		ib_drain_qp(t->qp);
Quoted 8 lines
Quoted 4 lines
fs/ksmbd/transport_rdma.c +1 -0
+
  		t->status = SMB_DIRECT_CS_DISCONNECTED;
  		wake_up_interruptible(&t->wait_status);
  		wake_up_interruptible(&t->wait_reassembly_queue);

Because we're now flushing the cancelled work requests, don't we need to also wake up &t->wait_send_pending, to abort any waiters on the send wq?

Could you please elaborate more how it could be a problem ? send_pending is decreased and wake_up &t->wait_send_pending if it is zero in send_done().

Quoted 3 lines

Tom.

reviewer Reviewed-by
#3 lore raw
Quoted 30 lines · Namjae Jeon

2022-10-03 3:11 GMT+09:00, Tom Talpey <[email protected]>:

Quoted 21 lines · Namjae Jeon
Quoted 20 lines

When disconnected, call ib_drain_qp to cancel all pending work requests and prevent ksmbd_conn_handler_loop from waiting for a long time for those work requests to compelete.

---
   fs/ksmbd/transport_rdma.c | 2 ++
   1 file changed, 2 insertions(+)
fs/ksmbd/transport_rdma.c +1 -0
index 0315bca3d53b..096eda9ef873 100644
@@ -1527,6 +1527,8 @@ static int smb_direct_cm_handler(struct rdma_cm_id
 *cm_id,
   	}
   	case RDMA_CM_EVENT_DEVICE_REMOVAL:
   	case RDMA_CM_EVENT_DISCONNECTED: {
+		ib_drain_qp(t->qp);
Quoted 8 lines
Quoted 4 lines
fs/ksmbd/transport_rdma.c +1 -0
+
   		t->status = SMB_DIRECT_CS_DISCONNECTED;
   		wake_up_interruptible(&t->wait_status);
   		wake_up_interruptible(&t->wait_reassembly_queue);

Because we're now flushing the cancelled work requests, don't we need to also wake up &t->wait_send_pending, to abort any waiters on the send wq?

Quoted 3 lines

Could you please elaborate more how it could be a problem ? send_pending is decreased and wake_up &t->wait_send_pending if it is zero in send_done().

I took another look at __ib_drain_sq() and yes it does put the qp into ERR, and only steals the single WR that it itself posted. As long as we're not using IB_POLL_DIRECT (which ksmbd isn't), we're good.