net/smc: fixes 2019-01-30
cover net/smc: fixes 2019-01-30
Dave,
here are some fixes in different areas of the smc code for the net tree.
Thanks, Ursula
Hans Wippel (1):
net/smc: allow 16 byte pnetids in netlink policy
Karsten Graul (7):
net/smc: prevent races between smc_lgr_terminate() and smc_conn_free() net/smc: don't wait for send buffer space when data was already sent net/smc: recvmsg and splice_read should return 0 after shutdown net/smc: do not wait under send_lock
net/smc: call smc_cdc_msg_send() under send_lock
net/smc: use device link provided in qp_context
net/smc: fix use of variable in cleared area
Ursula Braun (1):
net/smc: fix another sizeof to int comparison
net/smc/af_smc.c | 11 ++++++++++- net/smc/smc_cdc.c | 5 ++++- net/smc/smc_clc.c | 2 +- net/smc/smc_core.c | 4 ++++ net/smc/smc_ib.c | 6 +++--- net/smc/smc_pnet.c | 2 +- net/smc/smc_tx.c | 17 +++++++---------- net/smc/smc_wr.c | 8 ++++---- 8 files changed, 34 insertions(+), 21 deletions(-)
Signature · 1 line
2.16.4
PATCH 1/9 net/smc: fix another sizeof to int comparison
Comparing an int to a size, which is unsigned, causes the int to become unsigned, giving the wrong result. kernel_sendmsg can return a negative error code.
--- net/smc/smc_clc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/smc/smc_clc.c +1 -1
@@ -378,7 +378,7 @@ int smc_clc_send_decline(struct smc_sock *smc, u32 peer_diag_info) vec.iov_len =sizeof (struct smc_clc_msg_decline ); len =kernel_sendmsg (smc->clcsock, &msg, &vec,1 ,sizeof (struct smc_clc_msg_decline )); -if (len <sizeof (struct smc_clc_msg_decline )) +if (len <0 || len <sizeof (struct smc_clc_msg_decline )) len = -EPROTO ;return len >0 ?0 : len; }
Signature · 1 line
2.16.4
PATCH 2/9 net/smc: allow 16 byte pnetids in netlink policy
In reply to Ursula BraunFrom: Hans Wippel <[email protected]>
Currently, users can only send pnetids with a maximum length of 15 bytes over the SMC netlink interface although the maximum pnetid length is 16 bytes. This patch changes the SMC netlink policy to accept 16 byte pnetids.
--- net/smc/smc_pnet.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/smc/smc_pnet.c +1 -1
@@ -27,7 +27,7 @@static struct nla_policy smc_pnet_policy[SMC_PNETID_MAX +1 ] = { [SMC_PNETID_NAME ] = { .type =NLA_NUL_STRING , - .len =SMC_MAX_PNETID_LEN -1 + .len =SMC_MAX_PNETID_LEN }, [SMC_PNETID_ETHNAME ] = { .type =NLA_NUL_STRING ,
Signature · 1 line
2.16.4
PATCH 3/9 net/smc: prevent races between smc_lgr_terminate() and smc_conn_free()
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
To prevent races between smc_lgr_terminate() and smc_conn_free() add an extra check of the lgr field before accessing it, and cancel a delayed free_work when a new smc connection is created.
This fixes the problem that free_work cleared the lgr variable but smc_lgr_terminate() or smc_conn_free() still access it in parallel.
--- net/smc/smc_core.c | 4 ++++ 1 file changed, 4 insertions(+)
net/smc/smc_core.c +4 -0
@@ -128,6 +128,8 @@ static void smc_lgr_unregister_conn(struct smc_connection *conn) {struct smc_link_group *lgr = conn->lgr; +if (!lgr) +return ;write_lock_bh (&lgr->conns_lock);if (conn->alert_token_local) {__smc_lgr_unregister_conn (conn);
@@ -628,6 +630,8 @@ int smc_conn_create(struct smc_sock *smc, bool is_smcd, int srv_first_contact, local_contact =SMC_REUSE_CONTACT ; conn->lgr = lgr;smc_lgr_register_conn (conn);/* add smc conn to lgr */ +if (delayed_work_pending (&lgr->free_work)) +cancel_delayed_work (&lgr->free_work);write_unlock_bh (&lgr->conns_lock);break ; }
Signature · 1 line
2.16.4
PATCH 4/9 net/smc: don't wait for send buffer space when data was already sent
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
When there is no more send buffer space and at least 1 byte was already sent then return to user space. The wait is only done when no data was sent by the sendmsg() call.
This fixes smc_tx_sendmsg() which tried to always send all user data and started to wait for free send buffer space when needed. During this wait the user space program was blocked in the sendmsg() call and hence not able to receive incoming data. When both sides were in such a situation then the connection stalled forever.
--- net/smc/smc_tx.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
net/smc/smc_tx.c +3 -4
@@ -165,12 +165,11 @@ int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len) conn->local_tx_ctrl.prod_flags.urg_data_pending =1 ;if (!atomic_read (&conn->sndbuf_space) || conn->urg_tx_pend) { +if (send_done) +return send_done; rc =smc_tx_wait (smc, msg->msg_flags); -if (rc) { -if (send_done) -return send_done; +if (rc) goto out_err; - }continue ; }
Signature · 1 line
2.16.4
PATCH 5/9 net/smc: recvmsg and splice_read should return 0 after shutdown
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
When a socket was connected and is now shut down for read, return 0 to indicate end of data in recvmsg and splice_read (like TCP) and do not return ENOTCONN. This behavior is required by the socket api.
--- net/smc/af_smc.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
net/smc/af_smc.c +10 -1
@@ -1505,6 +1505,11 @@ static int smc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len, smc =smc_sk (sk);lock_sock (sk); +if (sk->sk_state ==SMC_CLOSED && (sk->sk_shutdown &RCV_SHUTDOWN )) { +/* socket was connected before, no more data to read */ + rc =0 ; + goto out; + }if ((sk->sk_state ==SMC_INIT ) || (sk->sk_state ==SMC_LISTEN ) || (sk->sk_state ==SMC_CLOSED ))
@@ -1840,7 +1845,11 @@ static ssize_t smc_splice_read(struct socket *sock, loff_t *ppos, smc =smc_sk (sk);lock_sock (sk); - +if (sk->sk_state ==SMC_CLOSED && (sk->sk_shutdown &RCV_SHUTDOWN )) { +/* socket was connected before, no more data to read */ + rc =0 ; + goto out; + }if (sk->sk_state ==SMC_INIT || sk->sk_state ==SMC_LISTEN || sk->sk_state ==SMC_CLOSED )
Signature · 1 line
2.16.4
PATCH 6/9 net/smc: do not wait under send_lock
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
smc_cdc_get_free_slot() might wait for free transfer buffers when using SMC-R. This wait should not be done under the send_lock, which is a spin_lock. This fixes a cpu loop in parallel threads waiting for the send_lock.
--- net/smc/smc_tx.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-)
net/smc/smc_tx.c +4 -6
@@ -488,25 +488,23 @@ static int smcr_tx_sndbuf_nonempty(struct smc_connection *conn)struct smc_wr_buf *wr_buf;int rc; -spin_lock_bh (&conn->send_lock); rc =smc_cdc_get_free_slot (conn, &wr_buf, &pend);if (rc <0 ) {if (rc == -EBUSY ) {struct smc_sock *smc = container_of(conn, struct smc_sock, conn); -if (smc->sk.sk_err ==ECONNABORTED ) { - rc =sock_error (&smc->sk); - goto out_unlock; - } +if (smc->sk.sk_err ==ECONNABORTED ) +return sock_error (&smc->sk); rc =0 ;if (conn->alert_token_local)/* connection healthy */ mod_delayed_work (system_wq, &conn->tx_work,SMC_TX_WORK_DELAY ); } - goto out_unlock; +return rc; } +spin_lock_bh (&conn->send_lock);if (!conn->local_tx_ctrl.prod_flags.urg_data_present) { rc =smc_tx_rdma_writes (conn);if (rc) {
Signature · 1 line
2.16.4
PATCH 7/9 net/smc: call smc_cdc_msg_send() under send_lock
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
Call smc_cdc_msg_send() under the connection send_lock to make sure all send operations for one connection are serialized.
--- net/smc/smc_cdc.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
net/smc/smc_cdc.c +4 -1
@@ -125,7 +125,10 @@ static int smcr_cdc_get_slot_and_msg_send(struct smc_connection *conn)if (rc)return rc; -return smc_cdc_msg_send (conn, wr_buf, pend); +spin_lock_bh (&conn->send_lock); + rc =smc_cdc_msg_send (conn, wr_buf, pend); +spin_unlock_bh (&conn->send_lock); +return rc; }int smc_cdc_get_slot_and_msg_send (struct smc_connection *conn)
Signature · 1 line
2.16.4
PATCH 8/9 net/smc: use device link provided in qp_context
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
The device field of the IB event structure does not always point to the SMC IB device. Load the pointer from the qp_context which is always provided to smc_ib_qp_event_handler() in the priv field. And for qp events the affected port is given in the qp structure of the ibevent, derive it from there.
--- net/smc/smc_ib.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
net/smc/smc_ib.c +3 -3
@@ -289,8 +289,8 @@ int smc_ib_create_protection_domain(struct smc_link *lnk)static void smc_ib_qp_event_handler (struct ib_event *ibevent,void *priv) { -struct smc_ib_device *smcibdev = - (struct smc_ib_device *)ibevent->device; +struct smc_link *lnk = (struct smc_link *)priv; +struct smc_ib_device *smcibdev = lnk->smcibdev;u8 port_idx;switch (ibevent->event) {
@@ -298,7 +298,7 @@ static void smc_ib_qp_event_handler(struct ib_event *ibevent, void *priv)case IB_EVENT_GID_CHANGE :case IB_EVENT_PORT_ERR :case IB_EVENT_QP_ACCESS_ERR : - port_idx = ibevent->element.port_num -1 ; + port_idx = ibevent->element.qp->port -1 ;set_bit (port_idx, &smcibdev->port_event_mask);schedule_work (&smcibdev->port_event_work);break ;
Signature · 1 line
2.16.4
PATCH 9/9 net/smc: fix use of variable in cleared area
In reply to Ursula BraunFrom: Karsten Graul <[email protected]>
Do not use pend->idx as index for the arrays because its value is located in the cleared area. Use the existing local variable instead. Without this fix the wrong area might be cleared.
--- net/smc/smc_wr.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +4 -4
@@ -218,10 +218,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,u32 idx = pend->idx;/* clear the full struct smc_wr_tx_pend including .priv */ -memset (&link->wr_tx_pends[pend->idx],0 , -sizeof (link->wr_tx_pends[pend->idx])); -memset (&link->wr_tx_bufs[pend->idx],0 , -sizeof (link->wr_tx_bufs[pend->idx])); +memset (&link->wr_tx_pends[idx],0 , +sizeof (link->wr_tx_pends[idx])); +memset (&link->wr_tx_bufs[idx],0 , +sizeof (link->wr_tx_bufs[idx]));test_and_clear_bit (idx, link->wr_tx_mask);return 1 ; }
Signature · 1 line
2.16.4
David Miller
maintainerFrom: Ursula Braun <[email protected]>
Date: Wed, 30 Jan 2019 18:50:59 +0100
Quoted 2 lines
here are some fixes in different areas of the smc code for the net tree.
Series applied, thanks.