smb: client/smbdirect: connect bug fixes/cleanups and smbdirect_socket.status_wait
Subsystems (3)
Stefan Metzmacher
reviewercover smb: client/smbdirect: connect bug fixes/cleanups and smbdirect_socket.status_wait
Hi,
this is the next step towards a common smbdirect layer between cifs.ko and ksmbd.ko, with the aim to provide a socket layer for userspace usage at the end of the road.
This patchset focuses on the client side today.
The first one is a fix for very long timeouts against unreachable servers.
The others prepare the use of a single wait_queue for state changes. This removes a lot of special handling during the connect and negotiate phases.
The last two move the state_wait queue into the common smbdirect_socket.status_wait.
For the server I have only a single patch that also uses smbdirect_socket.status_wait, but I'm skipping the server patches today.
I plan a lot more progress on the server side tomorrow and hopefully finish the moving everything from struct smb_direct_transport into struct smbdirect_socket.
I used the following xfstests as regression tests:
cifs/001 generic/001 generic/002 generic/005 generic/006 generic/007 generic/010 generic/011
Between cifs.ko against ksmbd.ko via siw.ko.
This is on top of the patches for the client I posted yesterday...
Stefan Metzmacher (9):
smb: client: return an error if rdma_connect does not return within 5
seconds
smb: client: improve logging in smbd_conn_upcall()
smb: client: don't call init_waitqueue_head(&info->conn_wait) twice in
_smbd_get_connection
smb: client: only use a single wait_queue to monitor smbdirect
connection status
smb: client/smbdirect: replace SMBDIRECT_SOCKET_CONNECTING with more
detailed states
smb: client: use status_wait and SMBDIRECT_SOCKET_NEGOTIATE_RUNNING
for completion
smb: client: use status_wait and
SMBDIRECT_SOCKET_RESOLVE_{ADDR,ROUTE}_RUNNING for completion
smb: smbdirect: introduce smbdirect_socket.status_wait
smb: client: make use of smbdirect_socket.status_wait
fs/smb/client/smbdirect.c | 137 ++++++++++++++------- fs/smb/client/smbdirect.h | 8 -- fs/smb/common/smbdirect/smbdirect_socket.h | 15 ++- 3 files changed, 105 insertions(+), 55 deletions(-)
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 1/9 smb: client: return an error if rdma_connect does not return within 5 seconds
This matches the timeout for tcp connections.
--- fs/smb/client/smbdirect.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
fs/smb/client/smbdirect.c +4 -2
@@ -1653,8 +1653,10 @@ static struct smbd_connection *_smbd_get_connection( goto rdma_connect_failed; } -wait_event_interruptible ( - info->conn_wait, sc->status !=SMBDIRECT_SOCKET_CONNECTING ); +wait_event_interruptible_timeout ( + info->conn_wait, + sc->status !=SMBDIRECT_SOCKET_CONNECTING , +msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) {log_rdma_event (ERR ,"rdma_connect failed port=%d\n" , port);
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 2/9 smb: client: improve logging in smbd_conn_upcall()
In reply to Stefan Metzmacher--- fs/smb/client/smbdirect.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
fs/smb/client/smbdirect.c +10 -4
@@ -178,9 +178,10 @@ static int smbd_conn_upcall( {struct smbd_connection *info = id->context;struct smbdirect_socket *sc = &info->socket; +const char *event_name =rdma_event_msg (event->event); -log_rdma_event (INFO ,"event=%d status=%d\n" , - event->event, event->status); +log_rdma_event (INFO ,"event=%s status=%d\n" , + event_name, event->status);switch (event->event) {case RDMA_CM_EVENT_ADDR_RESOLVED :
@@ -190,17 +191,19 @@ static int smbd_conn_upcall(break ;case RDMA_CM_EVENT_ADDR_ERROR : +log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); info->ri_rc = -EHOSTUNREACH ;complete (&info->ri_done);break ;case RDMA_CM_EVENT_ROUTE_ERROR : +log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); info->ri_rc = -ENETUNREACH ;complete (&info->ri_done);break ;case RDMA_CM_EVENT_ESTABLISHED : -log_rdma_event (INFO ,"connected event=%d\n" , event->event); +log_rdma_event (INFO ,"connected event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_CONNECTED ;wake_up_interruptible (&info->conn_wait);break ;
@@ -208,7 +211,7 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_CONNECT_ERROR :case RDMA_CM_EVENT_UNREACHABLE :case RDMA_CM_EVENT_REJECTED : -log_rdma_event (INFO ,"connecting failed event=%d\n" , event->event); +log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_DISCONNECTED ;wake_up_interruptible (&info->conn_wait);break ;
@@ -217,6 +220,7 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_DISCONNECTED :/* This happens when we fail the negotiation */ if (sc->status ==SMBDIRECT_SOCKET_NEGOTIATE_FAILED ) { +log_rdma_event (ERR ,"event=%s during negotiation\n" , event_name); sc->status =SMBDIRECT_SOCKET_DISCONNECTED ;wake_up (&info->conn_wait);break ;
@@ -229,6 +233,8 @@ static int smbd_conn_upcall(break ;default : +log_rdma_event (ERR ,"unexpected event=%s status=%d\n" , + event_name, event->status);break ; }
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 3/9 smb: client: don't call init_waitqueue_head(&info->conn_wait) twice in _smbd_get_connection
In reply to Stefan MetzmacherIt is already called long before we may hit this cleanup code path.
--- fs/smb/client/smbdirect.c | 1 - 1 file changed, 1 deletion(-)
fs/smb/client/smbdirect.c +0 -1
@@ -1716,7 +1716,6 @@ static struct smbd_connection *_smbd_get_connection(cancel_delayed_work_sync (&info->idle_timer_work);destroy_caches_and_workqueue (info); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; -init_waitqueue_head (&info->conn_wait);rdma_disconnect (sc->rdma.cm_id);wait_event (info->conn_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED );
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 4/9 smb: client: only use a single wait_queue to monitor smbdirect connection status
In reply to Stefan MetzmacherThere's no need for separate conn_wait and disconn_wait queues.
This will simplify the move to common code, the server code already a single wait_queue for this.
--- fs/smb/client/smbdirect.c | 17 ++++++++--------- fs/smb/client/smbdirect.h | 3 +-- 2 files changed, 9 insertions(+), 11 deletions(-)
fs/smb/client/smbdirect.c +8 -9
@@ -205,7 +205,7 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_ESTABLISHED :log_rdma_event (INFO ,"connected event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_CONNECTED ; -wake_up_interruptible (&info->conn_wait); +wake_up_interruptible (&info->status_wait);break ;case RDMA_CM_EVENT_CONNECT_ERROR :
@@ -213,7 +213,7 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_REJECTED :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up_interruptible (&info->conn_wait); +wake_up_interruptible (&info->status_wait);break ;case RDMA_CM_EVENT_DEVICE_REMOVAL :
@@ -222,12 +222,12 @@ static int smbd_conn_upcall(if (sc->status ==SMBDIRECT_SOCKET_NEGOTIATE_FAILED ) {log_rdma_event (ERR ,"event=%s during negotiation\n" , event_name); sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up (&info->conn_wait); +wake_up (&info->status_wait);break ; } sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up_interruptible (&info->disconn_wait); +wake_up_interruptible (&info->status_wait);wake_up_interruptible (&sc->recv_io.reassembly.wait_queue);wake_up_interruptible_all (&info->wait_send_queue);break ;
@@ -1325,7 +1325,7 @@ void smbd_destroy(struct TCP_Server_Info *server)rdma_disconnect (sc->rdma.cm_id);log_rdma_event (INFO ,"wait for transport being disconnected\n" );wait_event_interruptible ( - info->disconn_wait, + info->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); }
@@ -1650,8 +1650,7 @@ static struct smbd_connection *_smbd_get_connection(log_rdma_event (INFO ,"connecting to IP %pI4 port %d\n" , &addr_in->sin_addr, port); -init_waitqueue_head (&info->conn_wait); -init_waitqueue_head (&info->disconn_wait); +init_waitqueue_head (&info->status_wait);init_waitqueue_head (&sc->recv_io.reassembly.wait_queue); rc =rdma_connect (sc->rdma.cm_id, &conn_param);if (rc) {
@@ -1660,7 +1659,7 @@ static struct smbd_connection *_smbd_get_connection( }wait_event_interruptible_timeout ( - info->conn_wait, + info->status_wait, sc->status !=SMBDIRECT_SOCKET_CONNECTING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));
@@ -1717,7 +1716,7 @@ static struct smbd_connection *_smbd_get_connection(destroy_caches_and_workqueue (info); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ;rdma_disconnect (sc->rdma.cm_id); -wait_event (info->conn_wait, +wait_event (info->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); allocate_cache_failed:
fs/smb/client/smbdirect.h +1 -2
@@ -47,8 +47,7 @@ struct smbd_connection {int ri_rc;struct completion ri_done; -wait_queue_head_t conn_wait; -wait_queue_head_t disconn_wait; +wait_queue_head_t status_wait;struct completion negotiate_completion;bool negotiate_done;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 5/9 smb: client/smbdirect: replace SMBDIRECT_SOCKET_CONNECTING with more detailed states
In reply to Stefan MetzmacherThe process of reaching a functional connection regresented by SMBDIRECT_SOCKET_CONNECTED, is more complex than using a single SMBDIRECT_SOCKET_CONNECTING state.
This will allow us to remove a lot of special variables and completions in the following commits.
--- fs/smb/client/smbdirect.c | 44 +++++++++++++++++++--- fs/smb/common/smbdirect/smbdirect_socket.h | 14 ++++++- 2 files changed, 51 insertions(+), 7 deletions(-)
fs/smb/client/smbdirect.c +39 -5
@@ -185,26 +185,39 @@ static int smbd_conn_upcall(switch (event->event) {case RDMA_CM_EVENT_ADDR_RESOLVED : +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); + sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ; + info->ri_rc =0 ; +complete (&info->ri_done); +break ; +case RDMA_CM_EVENT_ROUTE_RESOLVED : +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); + sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ; info->ri_rc =0 ;complete (&info->ri_done);break ;case RDMA_CM_EVENT_ADDR_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); + sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; info->ri_rc = -EHOSTUNREACH ;complete (&info->ri_done);break ;case RDMA_CM_EVENT_ROUTE_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); + sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; info->ri_rc = -ENETUNREACH ;complete (&info->ri_done);break ;case RDMA_CM_EVENT_ESTABLISHED :log_rdma_event (INFO ,"connected event=%s\n" , event_name); - sc->status =SMBDIRECT_SOCKET_CONNECTED ; +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ); + sc->status =SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ;wake_up_interruptible (&info->status_wait);break ;
@@ -212,7 +225,8 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_UNREACHABLE :case RDMA_CM_EVENT_REJECTED :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name); - sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ); + sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED ;wake_up_interruptible (&info->status_wait);break ;
@@ -481,6 +495,12 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) info->negotiate_done =process_negotiation_response (response, wc->byte_len);put_receive_buffer (info, response); +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ); +if (!info->negotiate_done) { + sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; + }else { + sc->status =SMBDIRECT_SOCKET_CONNECTED ; + }complete (&info->negotiate_completion);return ;
@@ -556,6 +576,7 @@ static struct rdma_cm_id *smbd_create_id(struct smbd_connection *info, struct sockaddr *dstaddr, int port) { + struct smbdirect_socket *sc = &info->socket;struct rdma_cm_id *id;int rc;__be16 *sport;
@@ -578,6 +599,8 @@ static struct rdma_cm_id *smbd_create_id(init_completion (&info->ri_done); info->ri_rc = -ETIMEDOUT ; +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED ); + sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ; rc =rdma_resolve_addr (id,NULL , (struct sockaddr *)dstaddr,RDMA_RESOLVE_TIMEOUT );if (rc) {
@@ -598,6 +621,8 @@ static struct rdma_cm_id *smbd_create_id( } info->ri_rc = -ETIMEDOUT ; +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ); + sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ; rc =rdma_resolve_route (id,RDMA_RESOLVE_TIMEOUT );if (rc) {log_rdma_event (ERR ,"rdma_resolve_route() failed %i\n" , rc);
@@ -644,6 +669,9 @@ static int smbd_ia_open(struct smbdirect_socket *sc = &info->socket;int rc; +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_CREATED ); + sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED ; + sc->rdma.cm_id =smbd_create_id (info, dstaddr, port);if (IS_ERR (sc->rdma.cm_id)) { rc =PTR_ERR (sc->rdma.cm_id);
@@ -1085,6 +1113,9 @@ static int smbd_negotiate(struct smbd_connection *info)int rc;struct smbdirect_recv_io *response =get_receive_buffer (info); +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ); + sc->status =SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ; + sc->recv_io.expected =SMBDIRECT_EXPECT_NEGOTIATE_REP ; rc =smbd_post_recv (info, response); log_rdma_event(INFO, "smbd_post_recv rc=%d iov.addr=0x%llx iov.length=%u iov.lkey=0x%x\n",
@@ -1540,7 +1571,7 @@ static struct smbd_connection *_smbd_get_connection( sc = &info->socket; sp = &sc->parameters; - sc->status =SMBDIRECT_SOCKET_CONNECTING ; + sc->status =SMBDIRECT_SOCKET_CREATED ; rc =smbd_ia_open (info, dstaddr, port);if (rc) {log_rdma_event (INFO ,"smbd_ia_open rc=%d\n" , rc);
@@ -1652,6 +1683,9 @@ static struct smbd_connection *_smbd_get_connection(init_waitqueue_head (&info->status_wait);init_waitqueue_head (&sc->recv_io.reassembly.wait_queue); + +WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ); + sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ; rc =rdma_connect (sc->rdma.cm_id, &conn_param);if (rc) {log_rdma_event (ERR ,"rdma_connect() failed with %i\n" , rc);
@@ -1660,10 +1694,10 @@ static struct smbd_connection *_smbd_get_connection(wait_event_interruptible_timeout ( info->status_wait, - sc->status !=SMBDIRECT_SOCKET_CONNECTING , + sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT )); -if (sc->status !=SMBDIRECT_SOCKET_CONNECTED ) { +if (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ) {log_rdma_event (ERR ,"rdma_connect failed port=%d\n" , port); goto rdma_connect_failed; }
fs/smb/common/smbdirect/smbdirect_socket.h +12 -2
@@ -8,9 +8,19 @@enum smbdirect_socket_status {SMBDIRECT_SOCKET_CREATED , -SMBDIRECT_SOCKET_CONNECTING , -SMBDIRECT_SOCKET_CONNECTED , +SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED , +SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING , +SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED , +SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED , +SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING , +SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED , +SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED , +SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING , +SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED , +SMBDIRECT_SOCKET_NEGOTIATE_NEEDED , +SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ,SMBDIRECT_SOCKET_NEGOTIATE_FAILED , +SMBDIRECT_SOCKET_CONNECTED ,SMBDIRECT_SOCKET_DISCONNECTING ,SMBDIRECT_SOCKET_DISCONNECTED ,SMBDIRECT_SOCKET_DESTROYED
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 6/9 smb: client: use status_wait and SMBDIRECT_SOCKET_NEGOTIATE_RUNNING for completion
In reply to Stefan MetzmacherWe can use the state change from SMBDIRECT_SOCKET_NEGOTIATE_RUNNING to SMBDIRECT_SOCKET_CONNECTED or SMBDIRECT_SOCKET_NEGOTIATE_FAILED in order to notify the caller if the negotiation is over.
--- fs/smb/client/smbdirect.c | 19 ++++++++++--------- fs/smb/client/smbdirect.h | 3 --- 2 files changed, 10 insertions(+), 12 deletions(-)
fs/smb/client/smbdirect.c +10 -9
@@ -470,6 +470,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)struct smbd_connection *info = container_of(sc, struct smbd_connection, socket);int data_length =0 ; +bool negotiate_done = false; log_rdma_recv(INFO, "response=0x%p type=%d wc status=%d wc opcode %d byte_len=%d pkey_index=%u\n", response, sc->recv_io.expected, wc->status, wc->opcode,
@@ -492,16 +493,16 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc)case SMBDIRECT_EXPECT_NEGOTIATE_REP :dump_smbdirect_negotiate_resp (smbdirect_recv_io_payload (response)); sc->recv_io.reassembly.full_packet_received = true; - info->negotiate_done = + negotiate_done =process_negotiation_response (response, wc->byte_len);put_receive_buffer (info, response);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ); -if (!info->negotiate_done) { +if (!negotiate_done) { sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ; }else { sc->status =SMBDIRECT_SOCKET_CONNECTED ; } -complete (&info->negotiate_completion); +wake_up_interruptible (&info->status_wait);return ;/* SMBD data transfer packet */
@@ -1124,17 +1125,17 @@ static int smbd_negotiate(struct smbd_connection *info)if (rc)return rc; -init_completion (&info->negotiate_completion); - info->negotiate_done = false; rc =smbd_post_send_negotiate_req (info);if (rc)return rc; - rc =wait_for_completion_interruptible_timeout ( - &info->negotiate_completion,SMBD_NEGOTIATE_TIMEOUT *HZ ); -log_rdma_event (INFO ,"wait_for_completion_timeout rc=%d\n" , rc); + rc =wait_event_interruptible_timeout ( + info->status_wait, + sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING , +secs_to_jiffies (SMBD_NEGOTIATE_TIMEOUT )); +log_rdma_event (INFO ,"wait_event_interruptible_timeout rc=%d\n" , rc); -if (info->negotiate_done) +if (sc->status ==SMBDIRECT_SOCKET_CONNECTED )return 0 ;if (rc ==0 )
fs/smb/client/smbdirect.h +0 -3
@@ -49,9 +49,6 @@ struct smbd_connection {struct completion ri_done;wait_queue_head_t status_wait; -struct completion negotiate_completion; -bool negotiate_done; -struct work_struct disconnect_work;struct work_struct post_send_credits_work;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 7/9 smb: client: use status_wait and SMBDIRECT_SOCKET_RESOLVE_{ADDR,ROUTE}_RUNNING for completion
In reply to Stefan MetzmacherWe can use the state change from SMBDIRECT_SOCKET_RESOLVE_{ADDR,ROUTE}_RUNNING to the next state in order to wake the caller to do the next step.
--- fs/smb/client/smbdirect.c | 46 ++++++++++++++++++++++----------------- fs/smb/client/smbdirect.h | 2 -- 2 files changed, 26 insertions(+), 22 deletions(-)
fs/smb/client/smbdirect.c +26 -20
@@ -187,31 +187,27 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_ADDR_RESOLVED :WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ; - info->ri_rc =0 ; -complete (&info->ri_done); +wake_up_interruptible (&info->status_wait);break ;case RDMA_CM_EVENT_ROUTE_RESOLVED :WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ; - info->ri_rc =0 ; -complete (&info->ri_done); +wake_up_interruptible (&info->status_wait);break ;case RDMA_CM_EVENT_ADDR_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; - info->ri_rc = -EHOSTUNREACH ; -complete (&info->ri_done); +wake_up_interruptible (&info->status_wait);break ;case RDMA_CM_EVENT_ROUTE_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; - info->ri_rc = -ENETUNREACH ; -complete (&info->ri_done); +wake_up_interruptible (&info->status_wait);break ;case RDMA_CM_EVENT_ESTABLISHED :
@@ -597,9 +593,6 @@ static struct rdma_cm_id *smbd_create_id( *sport =htons (port); -init_completion (&info->ri_done); - info->ri_rc = -ETIMEDOUT ; -WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_NEEDED ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ; rc = rdma_resolve_addr(id, NULL, (struct sockaddr *)dstaddr,
@@ -608,20 +601,26 @@ static struct rdma_cm_id *smbd_create_id(log_rdma_event (ERR ,"rdma_resolve_addr() failed %i\n" , rc); goto out; } - rc =wait_for_completion_interruptible_timeout ( - &info->ri_done,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT )); + rc =wait_event_interruptible_timeout ( + info->status_wait, + sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING , +msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));/* e.g. if interrupted returns -ERESTARTSYS */ if (rc <0 ) {log_rdma_event (ERR ,"rdma_resolve_addr timeout rc: %i\n" , rc); goto out; } - rc = info->ri_rc; -if (rc) { +if (sc->status ==SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ) { + rc = -ETIMEDOUT ; +log_rdma_event (ERR ,"rdma_resolve_addr() completed %i\n" , rc); + goto out; + } +if (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ) { + rc = -EHOSTUNREACH ;log_rdma_event (ERR ,"rdma_resolve_addr() completed %i\n" , rc); goto out; } - info->ri_rc = -ETIMEDOUT ;WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ; rc =rdma_resolve_route (id,RDMA_RESOLVE_TIMEOUT );
@@ -629,15 +628,22 @@ static struct rdma_cm_id *smbd_create_id(log_rdma_event (ERR ,"rdma_resolve_route() failed %i\n" , rc); goto out; } - rc =wait_for_completion_interruptible_timeout ( - &info->ri_done,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT )); + rc =wait_event_interruptible_timeout ( + info->status_wait, + sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING , +msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));/* e.g. if interrupted returns -ERESTARTSYS */ if (rc <0 ) {log_rdma_event (ERR ,"rdma_resolve_addr timeout rc: %i\n" , rc); goto out; } - rc = info->ri_rc; -if (rc) { +if (sc->status ==SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ) { + rc = -ETIMEDOUT ; +log_rdma_event (ERR ,"rdma_resolve_route() completed %i\n" , rc); + goto out; + } +if (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ) { + rc = -ENETUNREACH ;log_rdma_event (ERR ,"rdma_resolve_route() completed %i\n" , rc); goto out; }
fs/smb/client/smbdirect.h +0 -2
@@ -45,8 +45,6 @@ enum keep_alive_status {struct smbd_connection {struct smbdirect_socket socket; -int ri_rc; -struct completion ri_done;wait_queue_head_t status_wait;struct work_struct disconnect_work;
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 8/9 smb: smbdirect: introduce smbdirect_socket.status_wait
In reply to Stefan MetzmacherThis will be used by server and client soon.
--- fs/smb/common/smbdirect/smbdirect_socket.h | 1 + 1 file changed, 1 insertion(+)
fs/smb/common/smbdirect/smbdirect_socket.h +1 -0
@@ -28,6 +28,7 @@ enum smbdirect_socket_status {struct smbdirect_socket {enum smbdirect_socket_status status; +wait_queue_head_t status_wait;/* RDMA related */ struct {
Signature · 1 line
2.43.0
Stefan Metzmacher
reviewerPATCH 9/9 smb: client: make use of smbdirect_socket.status_wait
In reply to Stefan MetzmacherThis will allow us to have common helper functions soon.
--- fs/smb/client/smbdirect.c | 32 ++++++++++++++++---------------- fs/smb/client/smbdirect.h | 2 -- 2 files changed, 16 insertions(+), 18 deletions(-)
fs/smb/client/smbdirect.c +16 -16
@@ -187,34 +187,34 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_ADDR_RESOLVED :WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ROUTE_RESOLVED :WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ADDR_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ROUTE_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ESTABLISHED :log_rdma_event (INFO ,"connected event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_CONNECT_ERROR :
@@ -223,7 +223,7 @@ static int smbd_conn_upcall(log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ); sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_DEVICE_REMOVAL :
@@ -232,12 +232,12 @@ static int smbd_conn_upcall(if (sc->status ==SMBDIRECT_SOCKET_NEGOTIATE_FAILED ) {log_rdma_event (ERR ,"event=%s during negotiation\n" , event_name); sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up (&info->status_wait); +wake_up (&sc->status_wait);break ; } sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);wake_up_interruptible (&sc->recv_io.reassembly.wait_queue);wake_up_interruptible_all (&info->wait_send_queue);break ;
@@ -498,7 +498,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) }else { sc->status =SMBDIRECT_SOCKET_CONNECTED ; } -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);return ;/* SMBD data transfer packet */
@@ -602,7 +602,7 @@ static struct rdma_cm_id *smbd_create_id( goto out; } rc =wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));/* e.g. if interrupted returns -ERESTARTSYS */
@@ -629,7 +629,7 @@ static struct rdma_cm_id *smbd_create_id( goto out; } rc =wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));/* e.g. if interrupted returns -ERESTARTSYS */
@@ -1136,7 +1136,7 @@ static int smbd_negotiate(struct smbd_connection *info)return rc; rc =wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ,secs_to_jiffies (SMBD_NEGOTIATE_TIMEOUT ));log_rdma_event (INFO ,"wait_event_interruptible_timeout rc=%d\n" , rc);
@@ -1363,7 +1363,7 @@ void smbd_destroy(struct TCP_Server_Info *server)rdma_disconnect (sc->rdma.cm_id);log_rdma_event (INFO ,"wait for transport being disconnected\n" );wait_event_interruptible ( - info->status_wait, + sc->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); }
@@ -1688,7 +1688,7 @@ static struct smbd_connection *_smbd_get_connection(log_rdma_event (INFO ,"connecting to IP %pI4 port %d\n" , &addr_in->sin_addr, port); -init_waitqueue_head (&info->status_wait); +init_waitqueue_head (&sc->status_wait);init_waitqueue_head (&sc->recv_io.reassembly.wait_queue);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED );
@@ -1700,7 +1700,7 @@ static struct smbd_connection *_smbd_get_connection( }wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));
@@ -1757,7 +1757,7 @@ static struct smbd_connection *_smbd_get_connection(destroy_caches_and_workqueue (info); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ;rdma_disconnect (sc->rdma.cm_id); -wait_event (info->status_wait, +wait_event (sc->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); allocate_cache_failed:
fs/smb/client/smbdirect.h +0 -2
@@ -45,8 +45,6 @@ enum keep_alive_status {struct smbd_connection {struct smbdirect_socket socket; -wait_queue_head_t status_wait; -struct work_struct disconnect_work;struct work_struct post_send_credits_work;
Signature · 1 line
2.43.0
Tentatively merged into cifs-2.6.git for-next pending additional review/testing
I did fix up minor checkpatch warnings in patches 5 and 6 (unneeded { and } in if ... else)
Quoted 59 lines · Stefan Metzmacher
Hi,
this is the next step towards a common smbdirect layer between cifs.ko and ksmbd.ko, with the aim to provide a socket layer for userspace usage at the end of the road.
This patchset focuses on the client side today.
The first one is a fix for very long timeouts against unreachable servers.
The others prepare the use of a single
wait_queuefor state changes. This removes a lot of special handling during the connect and negotiate phases.The last two move the
state_waitqueue into the common smbdirect_socket.status_wait.For the server I have only a single patch that also uses smbdirect_socket.status_wait, but I'm skipping the server patches today.
I plan a lot more progress on the server side tomorrow and hopefully finish the moving everything from struct
smb_direct_transportinto structsmbdirect_socket.I used the following xfstests as regression tests:
cifs/001 generic/001 generic/002 generic/005 generic/006 generic/007 generic/010 generic/011Between cifs.ko against ksmbd.ko via siw.ko.
This is on top of the patches for the client I posted yesterday...
Stefan Metzmacher (9):
smb: client: return an error ifrdma_connectdoes not return within 5
seconds
smb: client: improve logging insmbd_conn_upcall()
smb: client: don't callinit_waitqueue_head(&info->conn_wait)twice in
_smbd_get_connection
smb: client: only use a singlewait_queueto monitor smbdirect
connection status
smb: client/smbdirect: replaceSMBDIRECT_SOCKET_CONNECTINGwith more
detailed states
smb: client: usestatus_waitandSMBDIRECT_SOCKET_NEGOTIATE_RUNNING
for completion
smb: client: usestatus_waitand
SMBDIRECT_SOCKET_RESOLVE_{ADDR,ROUTE}_RUNNING for completion
smb: smbdirect: introduce smbdirect_socket.status_wait
smb: client: make use of smbdirect_socket.status_waitfs/smb/client/smbdirect.c | 137 ++++++++++++++------- fs/smb/client/smbdirect.h | 8 -- fs/smb/common/smbdirect/smbdirect_socket.h | 15 ++- 3 files changed, 105 insertions(+), 55 deletions(-)--
2.43.0
Signature · 3 lines
Thanks, Steve
Re: [PATCH 9/9] smb: client: make use of smbdirect_socket.status_wait
In reply to Stefan MetzmacherHi Metze,
&sc -> status_wait is initialized in _smbd_get_connection() in line 1691 but it is being used by smbd_ia_open() -> smbd_create_id() before that. This is giving an OOPS on RDMA mount (attached). Could you please check?
Thanks
Meetakshi
Quoted 168 lines · Stefan Metzmacher
This will allow us to have common helper functions soon.
Cc Steve French <[email protected]>Cc Tom Talpey <[email protected]>Cc Long Li <[email protected]>Signed-off-by Stefan Metzmacher <[email protected]>--- fs/smb/client/smbdirect.c | 32 ++++++++++++++++---------------- fs/smb/client/smbdirect.h | 2 -- 2 files changed, 16 insertions(+), 18 deletions(-)fs/smb/client/smbdirect.c +16
-16@@ -187,34 +187,34 @@ static int smbd_conn_upcall(case RDMA_CM_EVENT_ADDR_RESOLVED :WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_NEEDED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ROUTE_RESOLVED :WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ADDR_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ADDR_FAILED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ROUTE_ERROR :log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ); sc->status =SMBDIRECT_SOCKET_RESOLVE_ROUTE_FAILED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_ESTABLISHED :log_rdma_event (INFO ,"connected event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_NEEDED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_CONNECT_ERROR :@@ -223,7 +223,7 @@ static int smbd_conn_upcall(log_rdma_event (ERR ,"connecting failed event=%s\n" , event_name);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ); sc->status =SMBDIRECT_SOCKET_RDMA_CONNECT_FAILED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);break ;case RDMA_CM_EVENT_DEVICE_REMOVAL :@@ -232,12 +232,12 @@ static int smbd_conn_upcall(if (sc->status ==SMBDIRECT_SOCKET_NEGOTIATE_FAILED ) {log_rdma_event (ERR ,"event=%s during negotiation\n" , event_name); sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up (&info->status_wait); +wake_up (&sc->status_wait);break ; } sc->status =SMBDIRECT_SOCKET_DISCONNECTED ; -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);wake_up_interruptible (&sc->recv_io.reassembly.wait_queue);wake_up_interruptible_all (&info->wait_send_queue);break ;@@ -498,7 +498,7 @@ static void recv_done(struct ib_cq *cq, struct ib_wc *wc) }else { sc->status =SMBDIRECT_SOCKET_CONNECTED ; } -wake_up_interruptible (&info->status_wait); +wake_up_interruptible (&sc->status_wait);return ;/* SMBD data transfer packet */ @@ -602,7 +602,7 @@ static struct rdma_cm_id *smbd_create_id( goto out; } rc =wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_RESOLVE_ADDR_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));/* e.g. if interrupted returns -ERESTARTSYS */ @@ -629,7 +629,7 @@ static struct rdma_cm_id *smbd_create_id( goto out; } rc =wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_RESOLVE_ROUTE_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));/* e.g. if interrupted returns -ERESTARTSYS */ @@ -1136,7 +1136,7 @@ static int smbd_negotiate(struct smbd_connection *info)return rc; rc =wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_NEGOTIATE_RUNNING ,secs_to_jiffies (SMBD_NEGOTIATE_TIMEOUT ));log_rdma_event (INFO ,"wait_event_interruptible_timeout rc=%d\n" , rc);@@ -1363,7 +1363,7 @@ void smbd_destroy(struct TCP_Server_Info *server)rdma_disconnect (sc->rdma.cm_id);log_rdma_event (INFO ,"wait for transport being disconnected\n" );wait_event_interruptible ( - info->status_wait, + sc->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); }@@ -1688,7 +1688,7 @@ static struct smbd_connection *_smbd_get_connection(log_rdma_event (INFO ,"connecting to IP %pI4 port %d\n" , &addr_in->sin_addr, port); -init_waitqueue_head (&info->status_wait); +init_waitqueue_head (&sc->status_wait);init_waitqueue_head (&sc->recv_io.reassembly.wait_queue);WARN_ON_ONCE (sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_NEEDED );@@ -1700,7 +1700,7 @@ static struct smbd_connection *_smbd_get_connection( }wait_event_interruptible_timeout ( - info->status_wait, + sc->status_wait, sc->status !=SMBDIRECT_SOCKET_RDMA_CONNECT_RUNNING ,msecs_to_jiffies (RDMA_RESOLVE_TIMEOUT ));@@ -1757,7 +1757,7 @@ static struct smbd_connection *_smbd_get_connection(destroy_caches_and_workqueue (info); sc->status =SMBDIRECT_SOCKET_NEGOTIATE_FAILED ;rdma_disconnect (sc->rdma.cm_id); -wait_event (info->status_wait, +wait_event (sc->status_wait, sc->status ==SMBDIRECT_SOCKET_DISCONNECTED ); allocate_cache_failed:fs/smb/client/smbdirect.h +0
-3@@ -45,8 +45,6 @@ enum keep_alive_status {struct smbd_connection {struct smbdirect_socket socket; -wait_queue_head_t status_wait; -struct work_struct disconnect_work;struct work_struct post_send_credits_work; --
Quoted 3 lines
2.43.0
[ 2972.050661] CIFS: Attempting to mount //10.5.0.13/MyShare [ 2972.050704] BUG: unable to handle page fault for address: ffffffffffffffe8 [ 2972.054229] #PF: supervisor read access in kernel mode [ 2972.056242] #PF: error_code(0x0000) - not-present page [ 2972.058357] PGD a3163f067 P4D a3163f067 PUD a31641067 PMD 0 [ 2972.060597] Oops: Oops: 0000 [#1] SMP NOPTI [ 2972.062256] CPU: 12 UID: 0 PID: 208040 Comm: mount.cifs Tainted: G W OE 6.16.06.17+ #1 VOLUNTARY [ 2972.066177] Tainted: [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE [ 2972.068729] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 03/08/2024 [ 2972.073001] RIP: 0010:prepare_to_wait_event+0x10e/0x130 [ 2972.075024] Code: 08 4d 8d 44 24 08 48 8d 57 e8 48 89 fe 4c 39 c7 74 1d 4c 89 c7 eb 13 48 8b 42 18 48 89 f7 48 8d 50 e8 49 39 c0 74 08 48 89 c6 <f6> 02 10 75 e8 48 8b 07 48 89 48 08 49 89 47 18 49 89 7f 20 48 89 [ 2972.082356] RSP: 0018:ffffd1da0e71b850 EFLAGS: 00010007 [ 2972.084435] RAX: ffffd1da0e71b8f0 RBX: 0000000000000001 RCX: ffffd1da0e71b8f0 [ 2972.087239] RDX: ffffffffffffffe8 RSI: 0000000000000000 RDI: ffff8f1ec3a92810 [ 2972.090116] RBP: ffffd1da0e71b878 R08: ffff8f1ec3a92810 R09: 0000000000000000 [ 2972.092970] R10: ffffd1da0e71b888 R11: 0000000000000002 R12: ffff8f1ec3a92808 [ 2972.095815] R13: ffff8f1ed864a8c0 R14: 0000000000000246 R15: ffffd1da0e71b8d8 [ 2972.098768] FS: 000070cd1d31b780(0000) GS:ffff8f271bce3000(0000) knlGS:0000000000000000 [ 2972.101955] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2972.104323] CR2: ffffffffffffffe8 CR3: 000000014197d000 CR4: 0000000000350ef0 [ 2972.107135] Call Trace: [ 2972.108265] <TASK> [ 2972.109145] _smbd_get_connection+0xa43/0x1970 [cifs] [ 2972.111310] ? srso_return_thunk+0x5/0x5f [ 2972.112940] ? __pfx_autoremove_wake_function+0x10/0x10 [ 2972.115013] smbd_get_connection+0x26/0x50 [cifs] [ 2972.116957] cifs_get_tcp_session+0x6a8/0x970 [cifs] [ 2972.119152] cifs_mount_get_session+0x48/0x1c0 [cifs] [ 2972.121358] dfs_mount_share+0x7b/0xb10 [cifs] [ 2972.123161] ? srso_return_thunk+0x5/0x5f [ 2972.124767] ? __wake_up_klogd+0x56/0x70 [ 2972.126407] ? srso_return_thunk+0x5/0x5f [ 2972.128179] ? vprintk_emit+0x15d/0x370 [ 2972.129846] ? srso_return_thunk+0x5/0x5f [ 2972.131461] ? __kmalloc_node_track_caller_noprof+0x172/0x520 [ 2972.133813] cifs_mount+0x5b/0x2e0 [cifs] [ 2972.135549] cifs_smb3_do_mount+0x180/0x7e0 [cifs] [ 2972.137626] ? vprintk+0x18/0x40 [ 2972.138997] ? srso_return_thunk+0x5/0x5f [ 2972.140692] ? _printk+0x5f/0x80 [ 2972.142163] smb3_get_tree+0x16d/0x360 [cifs] [ 2972.144015] vfs_get_tree+0x2a/0xf0 [ 2972.145411] path_mount+0x500/0xb00 [ 2972.146839] ? srso_return_thunk+0x5/0x5f [ 2972.148479] __x64_sys_mount+0x116/0x150 [ 2972.150493] x64_sys_call+0x2082/0x20d0 [ 2972.152667] do_syscall_64+0x7b/0x4f0 [ 2972.154649] ? srso_return_thunk+0x5/0x5f [ 2972.156815] ? __handle_mm_fault+0xa9c/0x1090 [ 2972.159232] ? __put_user_8+0xd/0x20 [ 2972.161060] ? srso_return_thunk+0x5/0x5f [ 2972.163109] ? count_memcg_events+0xba/0x1a0 [ 2972.165247] ? srso_return_thunk+0x5/0x5f [ 2972.167192] ? handle_mm_fault+0xbc/0x300 [ 2972.169118] ? srso_return_thunk+0x5/0x5f [ 2972.171012] ? do_user_addr_fault+0x1b9/0x860 [ 2972.173152] ? srso_return_thunk+0x5/0x5f [ 2972.175090] ? irqentry_exit_to_user_mode+0x2e/0x240 [ 2972.177519] ? srso_return_thunk+0x5/0x5f [ 2972.179467] ? irqentry_exit+0x1d/0x30 [ 2972.181473] ? srso_return_thunk+0x5/0x5f [ 2972.183554] ? exc_page_fault+0x84/0x150 [ 2972.185402] entry_SYSCALL_64_after_hwframe+0x76/0x7e [ 2972.187708] RIP: 0033:0x70cd1d12af0e [ 2972.189706] Code: 48 8b 0d 0d 7f 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d da 7e 0d 00 f7 d8 64 89 01 48 [ 2972.197683] RSP: 002b:00007ffecb09f028 EFLAGS: 00000246 ORIG_RAX: 00000000000000a5 [ 2972.201254] RAX: ffffffffffffffda RBX: 000000000000000a RCX: 000070cd1d12af0e [ 2972.204410] RDX: 000060acc558a47e RSI: 000060acc558a4e4 RDI: 00007ffecb0a0608 [ 2972.207687] RBP: 00007ffecb09f0e0 R08: 000060acea83eeb0 R09: 0000000000000000 [ 2972.210865] R10: 0000000000000000 R11: 0000000000000246 R12: 000060acc558a047 [ 2972.214110] R13: 000060acea83ff20 R14: 00007ffecb0a0608 R15: 000070cd1d32a000 [ 2972.217378] </TASK> [ 2972.218656] Modules linked in: rpcrdma sunrpc rdma_ucm ib_iser libiscsi scsi_transport_iscsi siw ib_uverbs cifs(OE) nls_utf8 cifs_md4 netfs tls qrtr cfg80211 8021q garp mrp stp llc xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xt_owner xt_tcpudp nft_compat nf_tables x_tables binfmt_misc nls_iso8859_1 intel_rapl_msr intel_rapl_common joydev mac_hid hyperv_drm polyval_clmulni serio_raw hid_generic ghash_clmulni_intel hid_hyperv aesni_intel hv_netvsc hyperv_fb hid hyperv_keyboard vmgenid ksmbd sch_fq_codel cifs_arc4 nls_ucs2_utils rdma_cm iw_cm ib_cm ib_core dm_multipath msr nvme_fabrics efi_pstore nfnetlink autofs4 [last unloaded: cifs] [ 2972.244156] CR2: ffffffffffffffe8 [ 2972.246119] ---[ end trace 0000000000000000 ]--- [ 2972.345267] RIP: 0010:prepare_to_wait_event+0x10e/0x130 [ 2972.347983] Code: 08 4d 8d 44 24 08 48 8d 57 e8 48 89 fe 4c 39 c7 74 1d 4c 89 c7 eb 13 48 8b 42 18 48 89 f7 48 8d 50 e8 49 39 c0 74 08 48 89 c6 <f6> 02 10 75 e8 48 8b 07 48 89 48 08 49 89 47 18 49 89 7f 20 48 89 [ 2972.356513] RSP: 0018:ffffd1da0e71b850 EFLAGS: 00010007 [ 2972.359150] RAX: ffffd1da0e71b8f0 RBX: 0000000000000001 RCX: ffffd1da0e71b8f0 [ 2972.362453] RDX: ffffffffffffffe8 RSI: 0000000000000000 RDI: ffff8f1ec3a92810 [ 2972.365782] RBP: ffffd1da0e71b878 R08: ffff8f1ec3a92810 R09: 0000000000000000 [ 2972.369295] R10: ffffd1da0e71b888 R11: 0000000000000002 R12: ffff8f1ec3a92808 [ 2972.372907] R13: ffff8f1ed864a8c0 R14: 0000000000000246 R15: ffffd1da0e71b8d8 [ 2972.376455] FS: 000070cd1d31b780(0000) GS:ffff8f271bce3000(0000) knlGS:0000000000000000 [ 2972.380271] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2972.383161] CR2: ffffffffffffffe8 CR3: 000000014197d000 CR4: 0000000000350ef0 [ 2972.386551] note: mount.cifs[208040] exited with irqs disabled
Stefan Metzmacher
reviewerRe: [PATCH 9/9] smb: client: make use of smbdirect_socket.status_wait
Hi Meetakshi,
Quoted 4 lines
&sc ->
status_waitis initialized in_smbd_get_connection()in line 1691 but it is being used bysmbd_ia_open()->smbd_create_id()before that. This is giving an OOPS on RDMA mount (attached). Could you please check?
The problem comes with this patch:
[PATCH 7/9] smb: client: use status_wait and SMBDIRECT_SOCKET_RESOLVE_{ADDR, ROUTE}_RUNNING for completion
And yesterday I was thinking the whole day about in which patch
I move init_waitqueue_head(&info->status_wait); to the top and then forgot it.
The strange thing is that all tests I made yesterday evening worked just fine something between 3-5 times.
And today I got the crash immediately and was about to debug it.
Thanks that you found it! And sorry for introducing the stupid problem.
I'll post v2 soon.
Thanks!
metze