ksmbd: remove duplicate flag set in smb2_write
Subsystems (2)
PATCH 1/2 ksmbd: remove duplicate flag set in smb2_write
The writethrough flag is set again ifis_rdma_channel is false.
--- fs/ksmbd/smb2pdu.c | 3 --- 1 file changed, 3 deletions(-)
fs/ksmbd/smb2pdu.c +0 -3
@@ -6506,9 +6506,6 @@ int smb2_write(struct ksmbd_work *work) le16_to_cpu(req->DataOffset));ksmbd_debug (SMB ,"flags %u\n" ,le32_to_cpu (req->Flags)); -if (le32_to_cpu (req->Flags) &SMB2_WRITEFLAG_WRITE_THROUGH ) - writethrough = true; -ksmbd_debug (SMB ,"filename %pd, offset %lld, len %zu\n" , fp->filp->f_path.dentry, offset, length); err = ksmbd_vfs_write(work, fp, data_buf, length, &offset,
Signature · 1 line
2.25.1
PATCH 2/2 ksmbd: smbd: handle RDMA CM time wait event
After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,RDMA_CM_EVENT_TIMEWAIT_EXIT is reported.
Disconnect on RDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.
--- fs/ksmbd/transport_rdma.c | 1 + 1 file changed, 1 insertion(+)
fs/ksmbd/transport_rdma.c +1 -0
@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,wake_up_interruptible (&t->wait_status);break ; } +case RDMA_CM_EVENT_TIMEWAIT_EXIT :case RDMA_CM_EVENT_DEVICE_REMOVAL :case RDMA_CM_EVENT_DISCONNECTED : { t->status =SMB_DIRECT_CS_DISCONNECTED ;
Signature · 1 line
2.25.1
Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event
2022-06-14 8:01 GMT+09:00, Hyunchul Lee <[email protected]>:
Quoted 8 lines
After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,RDMA_CM_EVENT_TIMEWAIT_EXITis reported.
Disconnect onRDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.Signed-off-by Hyunchul Lee <[email protected]>
Thanks!
Tom Talpey
reviewerRe: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event
In reply to Hyunchul LeeQuoted 17 lines · Hyunchul Lee
After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,RDMA_CM_EVENT_TIMEWAIT_EXITis reported.
Disconnect onRDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.Signed-off-by Hyunchul Lee <[email protected]>--- fs/ksmbd/transport_rdma.c | 1 + 1 file changed, 1 insertion(+)fs/ksmbd/transport_rdma.c +0
-0
fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541
Quoted 7 lines
fs/ksmbd/transport_rdma.c +1
-0@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,wake_up_interruptible (&t->wait_status);break ; } +case RDMA_CM_EVENT_TIMEWAIT_EXIT :case RDMA_CM_EVENT_DEVICE_REMOVAL :case RDMA_CM_EVENT_DISCONNECTED : { t->status =SMB_DIRECT_CS_DISCONNECTED ;
Is this issue seen on all RDMA providers? Because I would normally expect that an RDMA_CM_EVENT_DISCONNECTED will precede the TIMEWAIT event. What scenarios have you seen this not occur?
Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.
Tom.
Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event
2022년 6월 14일 (화) 오후 8:56, Tom Talpey <[email protected]>님이 작성:
Quoted 15 lines · Hyunchul Lee
Quoted 12 lines
After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,RDMA_CM_EVENT_TIMEWAIT_EXITis reported.
Disconnect onRDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.Signed-off-by Hyunchul Lee <[email protected]>--- fs/ksmbd/transport_rdma.c | 1 + 1 file changed, 1 insertion(+)
Quoted 17 lines
fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541
Quoted 12 lines
fs/ksmbd/transport_rdma.c +1
-0@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,wake_up_interruptible (&t->wait_status);break ; } +case RDMA_CM_EVENT_TIMEWAIT_EXIT :case RDMA_CM_EVENT_DEVICE_REMOVAL :case RDMA_CM_EVENT_DISCONNECTED : { t->status =SMB_DIRECT_CS_DISCONNECTED ;Is this issue seen on all RDMA providers? Because I would normally expect that an
RDMA_CM_EVENT_DISCONNECTEDwill precede the TIMEWAIT event. What scenarios have you seen this not occur?
There was an issue that ksmbd got stuck after attempting to shutdown. We are trying to reproduce it, but we haven't reproduced it yet, but It seems to be related to the TIMEWAIT event.
And other drivers such as nvme have disconnected on the TIMEWAIT event.
Quoted 3 lines
Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.
Right, ksmbd doesn't reuse QP.
Quoted 2 lines
Tom.
Signature · 2 lines
Thanks, Hyunchul
Tom Talpey
reviewerRe: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event
Quoted 34 lines · Hyunchul Lee
2022년 6월 14일 (화) 오후 8:56, Tom Talpey <[email protected]>님이 작성:
Quoted 15 lines · Hyunchul Lee
Quoted 12 lines
After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,RDMA_CM_EVENT_TIMEWAIT_EXITis reported.
Disconnect onRDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.Signed-off-by Hyunchul Lee <[email protected]>--- fs/ksmbd/transport_rdma.c | 1 + 1 file changed, 1 insertion(+)Quoted 17 lines
fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541
Quoted 12 lines
fs/ksmbd/transport_rdma.c +1
-0@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,wake_up_interruptible (&t->wait_status);break ; } +case RDMA_CM_EVENT_TIMEWAIT_EXIT :case RDMA_CM_EVENT_DEVICE_REMOVAL :case RDMA_CM_EVENT_DISCONNECTED : { t->status =SMB_DIRECT_CS_DISCONNECTED ;Is this issue seen on all RDMA providers? Because I would normally expect that an
RDMA_CM_EVENT_DISCONNECTEDwill precede the TIMEWAIT event. What scenarios have you seen this not occur?
Quoted 3 lines
There was an issue that ksmbd got stuck after attempting to shutdown. We are trying to reproduce it, but we haven't reproduced it yet, but It seems to be related to the TIMEWAIT event.
I don't think it's appropriate to add this case to SMB. I think it's quite unlikely that it will address anything, because an RDMA provider must have indicated a CM_EVENT_DISCONNECTED prior to any TIMEWAIT. So, the QP (and connection) will already have been torn down by ksmbd at the earlier event. Perhaps ksmbd did not properly drain the QP at the initial disconnect.
Quoted 1 line
And other drivers such as nvme have disconnected on the TIMEWAIT event.
NVME is a completely different upper layer, and has different client/ server transport behavior. The SMB session insulates its peers from most transport errors, and should not be requesting timewait for its connections, and definitely not waiting for timewait to expire before initiating teardown (or recovery). The NFS/RDMA client and server ignore this event, btw.
Quoted 5 lines
Quoted 3 lines
Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.
Right, ksmbd doesn't reuse QP.
Then there appears to be no good justification for the change. Sorry, but it's a NAK from me.
Tom.
Re: [PATCH 2/2] ksmbd: smbd: handle RDMA CM time wait event
2022년 6월 16일 (목) 오전 3:53, Tom Talpey <[email protected]>님이 작성:
Quoted 48 lines · Hyunchul Lee
Quoted 34 lines
2022년 6월 14일 (화) 오후 8:56, Tom Talpey <[email protected]>님이 작성:
Quoted 15 lines · Hyunchul Lee
Quoted 12 lines
After a QP has been disconnected, it stays
in a timewait state for in flight packets.
After the state has completed,RDMA_CM_EVENT_TIMEWAIT_EXITis reported.
Disconnect onRDMA_CM_EVENT_TIMEWAIT_EXIT
so that ksmbd can restart.Signed-off-by Hyunchul Lee <[email protected]>--- fs/ksmbd/transport_rdma.c | 1 + 1 file changed, 1 insertion(+)Quoted 17 lines
fs/ksmbd/transport_rdma.c static int smb_direct_cm_handler(struct rdma_cm_id *cm_id, · line 1541
Quoted 12 lines
fs/ksmbd/transport_rdma.c +1
-0@@ -1535,6 +1535,7 @@ static int smb_direct_cm_handler(struct rdma_cm_id *cm_id,wake_up_interruptible (&t->wait_status);break ; } +case RDMA_CM_EVENT_TIMEWAIT_EXIT :case RDMA_CM_EVENT_DEVICE_REMOVAL :case RDMA_CM_EVENT_DISCONNECTED : { t->status =SMB_DIRECT_CS_DISCONNECTED ;Is this issue seen on all RDMA providers? Because I would normally expect that an
RDMA_CM_EVENT_DISCONNECTEDwill precede the TIMEWAIT event. What scenarios have you seen this not occur?Quoted 3 lines
There was an issue that ksmbd got stuck after attempting to shutdown. We are trying to reproduce it, but we haven't reproduced it yet, but It seems to be related to the TIMEWAIT event.
I don't think it's appropriate to add this case to SMB. I think it's quite unlikely that it will address anything, because an RDMA provider must have indicated a
CM_EVENT_DISCONNECTEDprior to any TIMEWAIT. So, the QP (and connection) will already have been torn down by ksmbd at the earlier event. Perhaps ksmbd did not properly drain the QP at the initial disconnect.
Quoted 9 lines
Quoted 1 line
And other drivers such as nvme have disconnected on the TIMEWAIT event.
NVME is a completely different upper layer, and has different client/ server transport behavior. The SMB session insulates its peers from most transport errors, and should not be requesting timewait for its connections, and definitely not waiting for timewait to expire before initiating teardown (or recovery). The NFS/RDMA client and server ignore this event, btw.
Okay, I got it.
I am looking for the cause and have found some clues.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
Unless ksmbd wishes to reuse its QP's, which is not currently the case (right?), there's pretty much no reason to manage QP state and hang around for TIMEWAIT.
Right, ksmbd doesn't reuse QP.
Then there appears to be no good justification for the change. Sorry, but it's a NAK from me.
Really thank you for the detailed explanation.
Quoted 1 line
Tom.
Signature · 2 lines
Thanks, Hyunchul
Namjae Jeon
maintainer2022-06-14 8:01 GMT+09:00, Hyunchul Lee <[email protected]>:
Quoted 13 lines
The writethrough flag is set again if
is_rdma_channelis false.Signed-off-by Hyunchul Lee <[email protected]>--- fs/ksmbd/smb2pdu.c | 3 --- 1 file changed, 3 deletions(-)fs/ksmbd/smb2pdu.c +0
-0
fs/ksmbd/smb2pdu.c int smb2_write(struct ksmbd_work *work) · line 6508
Quoted 3 lines
fs/ksmbd/smb2pdu.c +0
-0@@ -6506,9 +6506,6 @@ int smb2_write(struct ksmbd_work *work) le16_to_cpu(req->DataOffset));ksmbd_debug (SMB ,"flags %u\n" ,le32_to_cpu (req->Flags));
Could you move debug print to flags check above also ?
Quoted 10 lines
fs/ksmbd/smb2pdu.c +0
-4-if (le32_to_cpu (req->Flags) &SMB2_WRITEFLAG_WRITE_THROUGH ) - writethrough = true; -ksmbd_debug (SMB ,"filename %pd, offset %lld, len %zu\n" , fp->filp->f_path.dentry, offset, length); err = ksmbd_vfs_write(work, fp, data_buf, length, &offset, --2.25.1
2022년 6월 14일 (화) 오전 8:08, Namjae Jeon <[email protected]>님이 작성:
Quoted 17 lines
2022-06-14 8:01 GMT+09:00, Hyunchul Lee <[email protected]>:
Quoted 15 lines
The writethrough flag is set again if
is_rdma_channelis false.Signed-off-by Hyunchul Lee <[email protected]>--- fs/ksmbd/smb2pdu.c | 3 --- 1 file changed, 3 deletions(-)fs/ksmbd/smb2pdu.c +0
-0@@ -6506,9 +6506,6 @@ int smb2_write(struct ksmbd_work *work) le16_to_cpu(req->DataOffset));
Quoted 3 lines
fs/ksmbd/smb2pdu.c int smb2_write(struct ksmbd_work *work) · line 6508
Quoted 1 line
fs/ksmbd/smb2pdu.c +0
-0ksmbd_debug (SMB ,"flags %u\n" ,le32_to_cpu (req->Flags));Could you move debug print to flags check above also ?
Okay, I will.
Quoted 10 lines
Quoted 10 lines
fs/ksmbd/smb2pdu.c +0
-4-if (le32_to_cpu (req->Flags) &SMB2_WRITEFLAG_WRITE_THROUGH ) - writethrough = true; -ksmbd_debug (SMB ,"filename %pd, offset %lld, len %zu\n" , fp->filp->f_path.dentry, offset, length); err = ksmbd_vfs_write(work, fp, data_buf, length, &offset, --2.25.1
Signature · 2 lines
Thanks, Hyunchul