Skip to contentSkip to conversation

net/smc: fixes 2019-01-30

PATCH In review Ursula Braun 2 participants 11 messages Jan 30, 2019 – Feb 1, 2019

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover net/smc: fixes 2019-01-30

Dave,

here are some fixes in different areas of the smc code for the net tree.

Thanks, Ursula

Hans Wippel (1):
net/smc: allow 16 byte pnetids in netlink policy

Karsten Graul (7):
net/smc: prevent races between smc_lgr_terminate() and smc_conn_free() net/smc: don't wait for send buffer space when data was already sent net/smc: recvmsg and splice_read should return 0 after shutdown net/smc: do not wait under send_lock
net/smc: call smc_cdc_msg_send() under send_lock
net/smc: use device link provided in qp_context
net/smc: fix use of variable in cleared area

Ursula Braun (1):
net/smc: fix another sizeof to int comparison

 net/smc/af_smc.c   | 11 ++++++++++-
 net/smc/smc_cdc.c  |  5 ++++-
 net/smc/smc_clc.c  |  2 +-
 net/smc/smc_core.c |  4 ++++
 net/smc/smc_ib.c   |  6 +++---
 net/smc/smc_pnet.c |  2 +-
 net/smc/smc_tx.c   | 17 +++++++----------
 net/smc/smc_wr.c   |  8 ++++----
 8 files changed, 34 insertions(+), 21 deletions(-)
Signature · 1 line
2.16.4
#1 lore raw

PATCH 1/9 net/smc: fix another sizeof to int comparison

Comparing an int to a size, which is unsigned, causes the int to become unsigned, giving the wrong result. kernel_sendmsg can return a negative error code.

---
 net/smc/smc_clc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/smc/smc_clc.c +1 -1
index 776e9dfc915d..d53fd588d1f5 100644
@@ -378,7 +378,7 @@ int smc_clc_send_decline(struct smc_sock *smc, u32 peer_diag_info)
 	vec.iov_len = sizeof(struct smc_clc_msg_decline);
 	len = kernel_sendmsg(smc->clcsock, &msg, &vec, 1,
 			     sizeof(struct smc_clc_msg_decline));
-	if (len < sizeof(struct smc_clc_msg_decline))
+	if (len < 0 || len < sizeof(struct smc_clc_msg_decline))
 		len = -EPROTO;
 	return len > 0 ? 0 : len;
 }
Signature · 1 line
2.16.4
#2 lore raw

PATCH 2/9 net/smc: allow 16 byte pnetids in netlink policy

In reply to Ursula Braun

From: Hans Wippel <[email protected]>

Currently, users can only send pnetids with a maximum length of 15 bytes over the SMC netlink interface although the maximum pnetid length is 16 bytes. This patch changes the SMC netlink policy to accept 16 byte pnetids.

---
 net/smc/smc_pnet.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
net/smc/smc_pnet.c +1 -1
index 7cb3e4f07c10..632c3109dee5 100644
@@ -27,7 +27,7 @@
 static struct nla_policy smc_pnet_policy[SMC_PNETID_MAX + 1] = {
 	[SMC_PNETID_NAME] = {
 		.type = NLA_NUL_STRING,
-		.len = SMC_MAX_PNETID_LEN - 1
+		.len = SMC_MAX_PNETID_LEN
 	},
 	[SMC_PNETID_ETHNAME] = {
 		.type = NLA_NUL_STRING,
Signature · 1 line
2.16.4
#3 lore raw

PATCH 3/9 net/smc: prevent races between smc_lgr_terminate() and smc_conn_free()

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

To prevent races between smc_lgr_terminate() and smc_conn_free() add an extra check of the lgr field before accessing it, and cancel a delayed free_work when a new smc connection is created.
This fixes the problem that free_work cleared the lgr variable but smc_lgr_terminate() or smc_conn_free() still access it in parallel.

---
 net/smc/smc_core.c | 4 ++++
 1 file changed, 4 insertions(+)
net/smc/smc_core.c +4 -0
index 35c1cdc93e1c..097c798983ca 100644
@@ -128,6 +128,8 @@ static void smc_lgr_unregister_conn(struct smc_connection *conn)
 {
 	struct smc_link_group *lgr = conn->lgr;
 
+	if (!lgr)
+		return;
 	write_lock_bh(&lgr->conns_lock);
 	if (conn->alert_token_local) {
 		__smc_lgr_unregister_conn(conn);
@@ -628,6 +630,8 @@ int smc_conn_create(struct smc_sock *smc, bool is_smcd, int srv_first_contact,
 			local_contact = SMC_REUSE_CONTACT;
 			conn->lgr = lgr;
 			smc_lgr_register_conn(conn); /* add smc conn to lgr */
+			if (delayed_work_pending(&lgr->free_work))
+				cancel_delayed_work(&lgr->free_work);
 			write_unlock_bh(&lgr->conns_lock);
 			break;
 		}
Signature · 1 line
2.16.4
#4 lore raw

PATCH 4/9 net/smc: don't wait for send buffer space when data was already sent

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

When there is no more send buffer space and at least 1 byte was already sent then return to user space. The wait is only done when no data was sent by the sendmsg() call.
This fixes smc_tx_sendmsg() which tried to always send all user data and started to wait for free send buffer space when needed. During this wait the user space program was blocked in the sendmsg() call and hence not able to receive incoming data. When both sides were in such a situation then the connection stalled forever.

---
 net/smc/smc_tx.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
net/smc/smc_tx.c +3 -4
index d8366ed51757..f99951f3f7fd 100644
@@ -165,12 +165,11 @@ int smc_tx_sendmsg(struct smc_sock *smc, struct msghdr *msg, size_t len)
 			conn->local_tx_ctrl.prod_flags.urg_data_pending = 1;
 
 		if (!atomic_read(&conn->sndbuf_space) || conn->urg_tx_pend) {
+			if (send_done)
+				return send_done;
 			rc = smc_tx_wait(smc, msg->msg_flags);
-			if (rc) {
-				if (send_done)
-					return send_done;
+			if (rc)
 				goto out_err;
-			}
 			continue;
 		}
 
Signature · 1 line
2.16.4
#5 lore raw

PATCH 5/9 net/smc: recvmsg and splice_read should return 0 after shutdown

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

When a socket was connected and is now shut down for read, return 0 to indicate end of data in recvmsg and splice_read (like TCP) and do not return ENOTCONN. This behavior is required by the socket api.

---
 net/smc/af_smc.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
net/smc/af_smc.c +10 -1
index c4e56602e0c6..b04a813fc865 100644
@@ -1505,6 +1505,11 @@ static int smc_recvmsg(struct socket *sock, struct msghdr *msg, size_t len,
 
 	smc = smc_sk(sk);
 	lock_sock(sk);
+	if (sk->sk_state == SMC_CLOSED && (sk->sk_shutdown & RCV_SHUTDOWN)) {
+		/* socket was connected before, no more data to read */
+		rc = 0;
+		goto out;
+	}
 	if ((sk->sk_state == SMC_INIT) ||
 	    (sk->sk_state == SMC_LISTEN) ||
 	    (sk->sk_state == SMC_CLOSED))
@@ -1840,7 +1845,11 @@ static ssize_t smc_splice_read(struct socket *sock, loff_t *ppos,
 
 	smc = smc_sk(sk);
 	lock_sock(sk);
-
+	if (sk->sk_state == SMC_CLOSED && (sk->sk_shutdown & RCV_SHUTDOWN)) {
+		/* socket was connected before, no more data to read */
+		rc = 0;
+		goto out;
+	}
 	if (sk->sk_state == SMC_INIT ||
 	    sk->sk_state == SMC_LISTEN ||
 	    sk->sk_state == SMC_CLOSED)
Signature · 1 line
2.16.4
#6 lore raw

PATCH 6/9 net/smc: do not wait under send_lock

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

smc_cdc_get_free_slot() might wait for free transfer buffers when using SMC-R. This wait should not be done under the send_lock, which is a spin_lock. This fixes a cpu loop in parallel threads waiting for the send_lock.

---
 net/smc/smc_tx.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
net/smc/smc_tx.c +4 -6
index f99951f3f7fd..36af3de731b9 100644
@@ -488,25 +488,23 @@ static int smcr_tx_sndbuf_nonempty(struct smc_connection *conn)
 	struct smc_wr_buf *wr_buf;
 	int rc;
 
-	spin_lock_bh(&conn->send_lock);
 	rc = smc_cdc_get_free_slot(conn, &wr_buf, &pend);
 	if (rc < 0) {
 		if (rc == -EBUSY) {
 			struct smc_sock *smc =
 				container_of(conn, struct smc_sock, conn);
 
-			if (smc->sk.sk_err == ECONNABORTED) {
-				rc = sock_error(&smc->sk);
-				goto out_unlock;
-			}
+			if (smc->sk.sk_err == ECONNABORTED)
+				return sock_error(&smc->sk);
 			rc = 0;
 			if (conn->alert_token_local) /* connection healthy */
 				mod_delayed_work(system_wq, &conn->tx_work,
 						 SMC_TX_WORK_DELAY);
 		}
-		goto out_unlock;
+		return rc;
 	}
 
+	spin_lock_bh(&conn->send_lock);
 	if (!conn->local_tx_ctrl.prod_flags.urg_data_present) {
 		rc = smc_tx_rdma_writes(conn);
 		if (rc) {
Signature · 1 line
2.16.4
#7 lore raw

PATCH 7/9 net/smc: call smc_cdc_msg_send() under send_lock

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

Call smc_cdc_msg_send() under the connection send_lock to make sure all send operations for one connection are serialized.

---
 net/smc/smc_cdc.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
net/smc/smc_cdc.c +4 -1
index db83332ac1c8..1c5333d494e9 100644
@@ -125,7 +125,10 @@ static int smcr_cdc_get_slot_and_msg_send(struct smc_connection *conn)
 	if (rc)
 		return rc;
 
-	return smc_cdc_msg_send(conn, wr_buf, pend);
+	spin_lock_bh(&conn->send_lock);
+	rc = smc_cdc_msg_send(conn, wr_buf, pend);
+	spin_unlock_bh(&conn->send_lock);
+	return rc;
 }
 
 int smc_cdc_get_slot_and_msg_send(struct smc_connection *conn)
Signature · 1 line
2.16.4
#8 lore raw

PATCH 8/9 net/smc: use device link provided in qp_context

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

The device field of the IB event structure does not always point to the SMC IB device. Load the pointer from the qp_context which is always provided to smc_ib_qp_event_handler() in the priv field. And for qp events the affected port is given in the qp structure of the ibevent, derive it from there.

---
 net/smc/smc_ib.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
net/smc/smc_ib.c +3 -3
index e519ef29c0ff..76487a16934e 100644
@@ -289,8 +289,8 @@ int smc_ib_create_protection_domain(struct smc_link *lnk)
 
 static void smc_ib_qp_event_handler(struct ib_event *ibevent, void *priv)
 {
-	struct smc_ib_device *smcibdev =
-		(struct smc_ib_device *)ibevent->device;
+	struct smc_link *lnk = (struct smc_link *)priv;
+	struct smc_ib_device *smcibdev = lnk->smcibdev;
 	u8 port_idx;
 
 	switch (ibevent->event) {
@@ -298,7 +298,7 @@ static void smc_ib_qp_event_handler(struct ib_event *ibevent, void *priv)
 	case IB_EVENT_GID_CHANGE:
 	case IB_EVENT_PORT_ERR:
 	case IB_EVENT_QP_ACCESS_ERR:
-		port_idx = ibevent->element.port_num - 1;
+		port_idx = ibevent->element.qp->port - 1;
 		set_bit(port_idx, &smcibdev->port_event_mask);
 		schedule_work(&smcibdev->port_event_work);
 		break;
Signature · 1 line
2.16.4
#9 lore raw

PATCH 9/9 net/smc: fix use of variable in cleared area

In reply to Ursula Braun

From: Karsten Graul <[email protected]>

Do not use pend->idx as index for the arrays because its value is located in the cleared area. Use the existing local variable instead. Without this fix the wrong area might be cleared.

---
 net/smc/smc_wr.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
net/smc/smc_wr.c +4 -4
index c2694750a6a8..1dc88c32d6bb 100644
@@ -218,10 +218,10 @@ int smc_wr_tx_put_slot(struct smc_link *link,
 		u32 idx = pend->idx;
 
 		/* clear the full struct smc_wr_tx_pend including .priv */
-		memset(&link->wr_tx_pends[pend->idx], 0,
-		       sizeof(link->wr_tx_pends[pend->idx]));
-		memset(&link->wr_tx_bufs[pend->idx], 0,
-		       sizeof(link->wr_tx_bufs[pend->idx]));
+		memset(&link->wr_tx_pends[idx], 0,
+		       sizeof(link->wr_tx_pends[idx]));
+		memset(&link->wr_tx_bufs[idx], 0,
+		       sizeof(link->wr_tx_bufs[idx]));
 		test_and_clear_bit(idx, link->wr_tx_mask);
 		return 1;
 	}
Signature · 1 line
2.16.4