Skip to contentSkip to conversation

ksmbd: remove duplicate flag set in smb2_write

PATCH Superseded Hyunchul Lee 3 participants 9 messages Jun 13, 2022 – Jun 17, 2022

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 ksmbd: remove duplicate flag set in smb2_write

The writethrough flag is set again if
is_rdma_channel is false.

---
 fs/ksmbd/smb2pdu.c | 3 ---
 1 file changed, 3 deletions(-)
fs/ksmbd/smb2pdu.c +0 -3
index e6f4ccc12f49..553aad4ca6fa 100644
@@ -6506,9 +6506,6 @@ int smb2_write(struct ksmbd_work *work)
 				    le16_to_cpu(req->DataOffset));
 
 		ksmbd_debug(SMB, "flags %u\n", le32_to_cpu(req->Flags));
-		if (le32_to_cpu(req->Flags) & SMB2_WRITEFLAG_WRITE_THROUGH)
-			writethrough = true;
-
 		ksmbd_debug(SMB, "filename %pd, offset %lld, len %zu\n",
 			    fp->filp->f_path.dentry, offset, length);
 		err = ksmbd_vfs_write(work, fp, data_buf, length, &offset,
Signature · 1 line
2.25.1
#1 lore raw

PATCH 2/2 ksmbd: smbd: handle RDMA CM time wait event

After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,
RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.

---
 fs/ksmbd/transport_rdma.c | 1 +
 1 file changed, 1 insertion(+)
fs/ksmbd/transport_rdma.c +1 -0
index d035e060c2f0..4b1a471afcd0 100644
@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
 		wake_up_interruptible(&t->wait_status);
 		break;
 	}
+	case RDMA_CM_EVENT_TIMEWAIT_EXIT:
 	case RDMA_CM_EVENT_DEVICE_REMOVAL:
 	case RDMA_CM_EVENT_DISCONNECTED: {
 		t->status = SMB_DIRECT_CS_DISCONNECTED;
Signature · 1 line
2.25.1
maintainer
#7 lore raw
In reply to Hyunchul Lee

2022-06-14 8:01 GMT+09:00, Hyunchul Lee <[email protected]>:

Quoted 13 lines

The writethrough flag is set again if
is_rdma_channel is false.

---
 fs/ksmbd/smb2pdu.c | 3 ---
 1 file changed, 3 deletions(-)
fs/ksmbd/smb2pdu.c +0 -0
index e6f4ccc12f49..553aad4ca6fa 100644

fs/ksmbd/smb2pdu.c int smb2_write(struct ksmbd_work *work) · line 6508

Quoted 3 lines
fs/ksmbd/smb2pdu.c +0 -0
@@ -6506,9 +6506,6 @@ int smb2_write(struct ksmbd_work *work)
 				    le16_to_cpu(req->DataOffset));
 
 		ksmbd_debug(SMB, "flags %u\n", le32_to_cpu(req->Flags));

Could you move debug print to flags check above also ?

Quoted 10 lines
fs/ksmbd/smb2pdu.c +0 -4
-		if (le32_to_cpu(req->Flags) & SMB2_WRITEFLAG_WRITE_THROUGH)
-			writethrough = true;
-
 		ksmbd_debug(SMB, "filename %pd, offset %lld, len %zu\n",
 			    fp->filp->f_path.dentry, offset, length);
 		err = ksmbd_vfs_write(work, fp, data_buf, length, &offset,
--

2.25.1

maintainer Acked-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event

In reply to Hyunchul Lee

2022-06-14 8:01 GMT+09:00, Hyunchul Lee <[email protected]>:

Quoted 8 lines

After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,
RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.

Thanks!

#8 lore raw
In reply to Namjae Jeon

2022년 6월 14일 (화) 오전 8:08, Namjae Jeon <[email protected]>님이 작성:

Quoted 17 lines

2022-06-14 8:01 GMT+09:00, Hyunchul Lee <[email protected]>:

Quoted 15 lines

The writethrough flag is set again if
is_rdma_channel is false.

---
 fs/ksmbd/smb2pdu.c | 3 ---
 1 file changed, 3 deletions(-)
fs/ksmbd/smb2pdu.c +0 -0
index e6f4ccc12f49..553aad4ca6fa 100644
@@ -6506,9 +6506,6 @@ int smb2_write(struct ksmbd_work *work)
                                  le16_to_cpu(req->DataOffset));
 
Quoted 3 lines

fs/ksmbd/smb2pdu.c int smb2_write(struct ksmbd_work *work) · line 6508

Quoted 1 line
fs/ksmbd/smb2pdu.c +0 -0
              ksmbd_debug(SMB, "flags %u\n", le32_to_cpu(req->Flags));

Could you move debug print to flags check above also ?

Okay, I will.

Quoted 10 lines
Quoted 10 lines
fs/ksmbd/smb2pdu.c +0 -4
-             if (le32_to_cpu(req->Flags) & SMB2_WRITEFLAG_WRITE_THROUGH)
-                     writethrough = true;
-
              ksmbd_debug(SMB, "filename %pd, offset %lld, len %zu\n",
                          fp->filp->f_path.dentry, offset, length);
              err = ksmbd_vfs_write(work, fp, data_buf, length, &offset,
--

2.25.1

Signature · 2 lines
Thanks,
Hyunchul
reviewer
#3 lore raw

Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event

In reply to Hyunchul Lee
Quoted 17 lines · Hyunchul Lee

After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,
RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.

---
  fs/ksmbd/transport_rdma.c | 1 +
  1 file changed, 1 insertion(+)
fs/ksmbd/transport_rdma.c +0 -0
index d035e060c2f0..4b1a471afcd0 100644

fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541

Quoted 7 lines
fs/ksmbd/transport_rdma.c +1 -0
@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
  		wake_up_interruptible(&t->wait_status);
  		break;
  	}
+	case RDMA_CM_EVENT_TIMEWAIT_EXIT:
  	case RDMA_CM_EVENT_DEVICE_REMOVAL:
  	case RDMA_CM_EVENT_DISCONNECTED: {
  		t->status = SMB_DIRECT_CS_DISCONNECTED;

Is this issue seen on all RDMA providers? Because I would normally expect that an RDMA_CM_EVENT_DISCONNECTED will precede the TIMEWAIT event. What scenarios have you seen this not occur?

Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.

Tom.

#4 lore raw

Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event

2022년 6월 14일 (화) 오후 8:56, Tom Talpey <[email protected]>님이 작성:

Quoted 15 lines · Hyunchul Lee
Quoted 12 lines

After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,
RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.

---
  fs/ksmbd/transport_rdma.c | 1 +
  1 file changed, 1 insertion(+)
Quoted 17 lines

fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541

Quoted 12 lines
fs/ksmbd/transport_rdma.c +1 -0
index d035e060c2f0..4b1a471afcd0 100644
@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
              wake_up_interruptible(&t->wait_status);
              break;
      }
+     case RDMA_CM_EVENT_TIMEWAIT_EXIT:
      case RDMA_CM_EVENT_DEVICE_REMOVAL:
      case RDMA_CM_EVENT_DISCONNECTED: {
              t->status = SMB_DIRECT_CS_DISCONNECTED;

Is this issue seen on all RDMA providers? Because I would normally expect that an RDMA_CM_EVENT_DISCONNECTED will precede the TIMEWAIT event. What scenarios have you seen this not occur?

There was an issue that ksmbd got stuck after attempting to shutdown. We are trying to reproduce it, but we haven't reproduced it yet, but It seems to be related to the TIMEWAIT event.

And other drivers such as nvme have disconnected on the TIMEWAIT event.

Quoted 3 lines

Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.

Right, ksmbd doesn't reuse QP.

Quoted 2 lines

Tom.

Signature · 2 lines
Thanks,
Hyunchul
reviewer
#5 lore raw

Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event

Quoted 34 lines · Hyunchul Lee

2022년 6월 14일 (화) 오후 8:56, Tom Talpey <[email protected]>님이 작성:

Quoted 15 lines · Hyunchul Lee
Quoted 12 lines

After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,
RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.

---
   fs/ksmbd/transport_rdma.c | 1 +
   1 file changed, 1 insertion(+)
Quoted 17 lines

fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541

Quoted 12 lines
fs/ksmbd/transport_rdma.c +1 -0
index d035e060c2f0..4b1a471afcd0 100644
@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
               wake_up_interruptible(&t->wait_status);
               break;
       }
+     case RDMA_CM_EVENT_TIMEWAIT_EXIT:
       case RDMA_CM_EVENT_DEVICE_REMOVAL:
       case RDMA_CM_EVENT_DISCONNECTED: {
               t->status = SMB_DIRECT_CS_DISCONNECTED;

Is this issue seen on all RDMA providers? Because I would normally expect that an RDMA_CM_EVENT_DISCONNECTED will precede the TIMEWAIT event. What scenarios have you seen this not occur?

Quoted 3 lines

There was an issue that ksmbd got stuck after attempting to shutdown. We are trying to reproduce it, but we haven't reproduced it yet, but It seems to be related to the TIMEWAIT event.

I don't think it's appropriate to add this case to SMB. I think it's quite unlikely that it will address anything, because an RDMA provider must have indicated a CM_EVENT_DISCONNECTED prior to any TIMEWAIT. So, the QP (and connection) will already have been torn down by ksmbd at the earlier event. Perhaps ksmbd did not properly drain the QP at the initial disconnect.

Quoted 1 line

And other drivers such as nvme have disconnected on the TIMEWAIT event.

NVME is a completely different upper layer, and has different client/ server transport behavior. The SMB session insulates its peers from most transport errors, and should not be requesting timewait for its connections, and definitely not waiting for timewait to expire before initiating teardown (or recovery). The NFS/RDMA client and server ignore this event, btw.

Quoted 5 lines
Quoted 3 lines

Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.

Right, ksmbd doesn't reuse QP.

Then there appears to be no good justification for the change. Sorry, but it's a NAK from me.

Tom.

#6 lore raw

Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event

2022년 6월 16일 (목) 오전 3:53, Tom Talpey <[email protected]>님이 작성:

Quoted 48 lines · Hyunchul Lee
Quoted 34 lines

2022년 6월 14일 (화) 오후 8:56, Tom Talpey <[email protected]>님이 작성:

Quoted 15 lines · Hyunchul Lee
Quoted 12 lines

After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,
RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.

---
   fs/ksmbd/transport_rdma.c | 1 +
   1 file changed, 1 insertion(+)
Quoted 17 lines

fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541

Quoted 12 lines
fs/ksmbd/transport_rdma.c +1 -0
index d035e060c2f0..4b1a471afcd0 100644
@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,
               wake_up_interruptible(&t->wait_status);
               break;
       }
+     case RDMA_CM_EVENT_TIMEWAIT_EXIT:
       case RDMA_CM_EVENT_DEVICE_REMOVAL:
       case RDMA_CM_EVENT_DISCONNECTED: {
               t->status = SMB_DIRECT_CS_DISCONNECTED;

Is this issue seen on all RDMA providers? Because I would normally expect that an RDMA_CM_EVENT_DISCONNECTED will precede the TIMEWAIT event. What scenarios have you seen this not occur?

Quoted 3 lines

There was an issue that ksmbd got stuck after attempting to shutdown. We are trying to reproduce it, but we haven't reproduced it yet, but It seems to be related to the TIMEWAIT event.

I don't think it's appropriate to add this case to SMB. I think it's quite unlikely that it will address anything, because an RDMA provider must have indicated a CM_EVENT_DISCONNECTED prior to any TIMEWAIT. So, the QP (and connection) will already have been torn down by ksmbd at the earlier event. Perhaps ksmbd did not properly drain the QP at the initial disconnect.

Quoted 9 lines
Quoted 1 line

And other drivers such as nvme have disconnected on the TIMEWAIT event.

NVME is a completely different upper layer, and has different client/ server transport behavior. The SMB session insulates its peers from most transport errors, and should not be requesting timewait for its connections, and definitely not waiting for timewait to expire before initiating teardown (or recovery). The NFS/RDMA client and server ignore this event, btw.

Okay, I got it.
I am looking for the cause and have found some clues.

Quoted 9 lines
Quoted 5 lines
Quoted 3 lines

Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.

Right, ksmbd doesn't reuse QP.

Then there appears to be no good justification for the change. Sorry, but it's a NAK from me.

Really thank you for the detailed explanation.

Quoted 1 line

Tom.

Signature · 2 lines
Thanks,
Hyunchul